2024-04-26T01:27:11.622 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-26T01:27:11.622 DEBUG:teuthology.run:Teuthology command: teuthology --name leonidus-2024-04-26_01:02:28-fs-wip-lusov-quiescer-distro-default-smithi --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsx}} --archive /home/teuthworker/archive/leonidus-2024-04-26_01:02:28-fs-wip-lusov-quiescer-distro-default-smithi/7674253 --verbose --owner scheduled_leonidus@teuthology -- /home/teuthworker/archive/leonidus-2024-04-26_01:02:28-fs-wip-lusov-quiescer-distro-default-smithi/7674253/orig.config.yaml 2024-04-26T01:27:11.684 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T01:27:11.779 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-04-26_01:02:28-fs-wip-lusov-quiescer-distro-default-smithi/7674253 branch: wip-lusov-quiescer description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsx}} email: null first_in_suite: false job_id: '7674253' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: leonidus-2024-04-26_01:02:28-fs-wip-lusov-quiescer-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-quiescer ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f subvols: create: 2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - nowsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-quiescer sha1: afa1933b0fdbb6c99c947d1eda34d661d23cd327 owner: scheduled_leonidus@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 3448 sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f sleep_before_teardown: 0 subset: 200/111111 suite: fs suite_branch: wip-lusov-quiescer suite_path: /home/teuthworker/src/github.com_ceph_ceph_afa1933b0fdbb6c99c947d1eda34d661d23cd327/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph suite_sha1: afa1933b0fdbb6c99c947d1eda34d661d23cd327 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi151.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH5eI+zLAwOyqKHO3yUFJXuJ//5tmdxDhJa0QRXVQZSqT5yojRFAyvovJgjyjdEoAw8FRy3/FjxjBzCWNkC6yi8= smithi165.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI1356vmhCuFyl689556Sf+yRF6TBoJxUlnmGPUgAYptvGVpxJ3FrtwNOHTKpLnfecfYe5KWS3KL2sF24th1FkM= smithi172.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN/7lYb1YhHr7/JM9QMIueTDR6wNMrnclJ6RX6LxpEp0q7VgRWV9JlIFZn1WQ0AclB8ZpSwI32ggaUv/FZ1B048= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/fsx.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-26_01:02:28 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-26T01:27:11.779 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph_afa1933b0fdbb6c99c947d1eda34d661d23cd327/qa; will attempt to use it 2024-04-26T01:27:11.780 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph_afa1933b0fdbb6c99c947d1eda34d661d23cd327/qa/tasks 2024-04-26T01:27:11.780 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-26T01:27:11.782 INFO:teuthology.task.internal:Checking packages... 2024-04-26T01:27:11.804 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd69f13d827952bb07bb9b7973bca14f97d764d6f' 2024-04-26T01:27:11.805 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-26T01:27:11.805 INFO:teuthology.packaging:ref: None 2024-04-26T01:27:11.805 INFO:teuthology.packaging:tag: None 2024-04-26T01:27:11.805 INFO:teuthology.packaging:branch: wip-lusov-quiescer 2024-04-26T01:27:11.805 INFO:teuthology.packaging:sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-26T01:27:11.805 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-quiescer 2024-04-26T01:27:11.975 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3156.gd69f13d8 2024-04-26T01:27:11.977 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-26T01:27:11.993 INFO:teuthology.task.internal:no buildpackages task found 2024-04-26T01:27:11.994 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-26T01:27:12.077 INFO:teuthology.task.internal:Saving configuration 2024-04-26T01:27:12.103 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-26T01:27:12.110 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-26T01:27:12.132 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi151.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-26_01:02:28-fs-wip-lusov-quiescer-distro-default-smithi/7674253', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 01:21:54.034037', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH5eI+zLAwOyqKHO3yUFJXuJ//5tmdxDhJa0QRXVQZSqT5yojRFAyvovJgjyjdEoAw8FRy3/FjxjBzCWNkC6yi8='} 2024-04-26T01:27:12.155 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi165.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-26_01:02:28-fs-wip-lusov-quiescer-distro-default-smithi/7674253', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 01:21:54.030852', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI1356vmhCuFyl689556Sf+yRF6TBoJxUlnmGPUgAYptvGVpxJ3FrtwNOHTKpLnfecfYe5KWS3KL2sF24th1FkM='} 2024-04-26T01:27:12.172 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi172.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-26_01:02:28-fs-wip-lusov-quiescer-distro-default-smithi/7674253', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 01:21:54.032539', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN/7lYb1YhHr7/JM9QMIueTDR6wNMrnclJ6RX6LxpEp0q7VgRWV9JlIFZn1WQ0AclB8ZpSwI32ggaUv/FZ1B048='} 2024-04-26T01:27:12.172 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-26T01:27:12.184 INFO:teuthology.task.internal:roles: ubuntu@smithi151.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-26T01:27:12.185 INFO:teuthology.task.internal:roles: ubuntu@smithi165.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-26T01:27:12.185 INFO:teuthology.task.internal:roles: ubuntu@smithi172.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-26T01:27:12.185 INFO:teuthology.run_tasks:Running task console_log... 2024-04-26T01:27:12.305 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f3d124e60d0>, signals=[15]) 2024-04-26T01:27:12.305 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-26T01:27:12.313 INFO:teuthology.task.internal:Opening connections... 2024-04-26T01:27:12.313 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi151.front.sepia.ceph.com 2024-04-26T01:27:12.319 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T01:27:12.395 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi165.front.sepia.ceph.com 2024-04-26T01:27:12.396 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T01:27:12.471 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi172.front.sepia.ceph.com 2024-04-26T01:27:12.472 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T01:27:12.547 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-26T01:27:12.555 DEBUG:teuthology.orchestra.run.smithi151:> uname -m 2024-04-26T01:27:12.573 INFO:teuthology.orchestra.run.smithi151.stdout:x86_64 2024-04-26T01:27:12.573 DEBUG:teuthology.orchestra.run.smithi151:> cat /etc/os-release 2024-04-26T01:27:12.630 INFO:teuthology.orchestra.run.smithi151.stdout:NAME="CentOS Stream" 2024-04-26T01:27:12.630 INFO:teuthology.orchestra.run.smithi151.stdout:VERSION="9" 2024-04-26T01:27:12.630 INFO:teuthology.orchestra.run.smithi151.stdout:ID="centos" 2024-04-26T01:27:12.630 INFO:teuthology.orchestra.run.smithi151.stdout:ID_LIKE="rhel fedora" 2024-04-26T01:27:12.630 INFO:teuthology.orchestra.run.smithi151.stdout:VERSION_ID="9" 2024-04-26T01:27:12.631 INFO:teuthology.orchestra.run.smithi151.stdout:PLATFORM_ID="platform:el9" 2024-04-26T01:27:12.631 INFO:teuthology.orchestra.run.smithi151.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T01:27:12.631 INFO:teuthology.orchestra.run.smithi151.stdout:ANSI_COLOR="0;31" 2024-04-26T01:27:12.631 INFO:teuthology.orchestra.run.smithi151.stdout:LOGO="fedora-logo-icon" 2024-04-26T01:27:12.631 INFO:teuthology.orchestra.run.smithi151.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T01:27:12.631 INFO:teuthology.orchestra.run.smithi151.stdout:HOME_URL="https://centos.org/" 2024-04-26T01:27:12.631 INFO:teuthology.orchestra.run.smithi151.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T01:27:12.631 INFO:teuthology.orchestra.run.smithi151.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T01:27:12.631 INFO:teuthology.orchestra.run.smithi151.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T01:27:12.632 INFO:teuthology.lock.ops:Updating smithi151.front.sepia.ceph.com on lock server 2024-04-26T01:27:12.880 DEBUG:teuthology.orchestra.run.smithi165:> uname -m 2024-04-26T01:27:12.897 INFO:teuthology.orchestra.run.smithi165.stdout:x86_64 2024-04-26T01:27:12.898 DEBUG:teuthology.orchestra.run.smithi165:> cat /etc/os-release 2024-04-26T01:27:12.955 INFO:teuthology.orchestra.run.smithi165.stdout:NAME="CentOS Stream" 2024-04-26T01:27:12.956 INFO:teuthology.orchestra.run.smithi165.stdout:VERSION="9" 2024-04-26T01:27:12.956 INFO:teuthology.orchestra.run.smithi165.stdout:ID="centos" 2024-04-26T01:27:12.956 INFO:teuthology.orchestra.run.smithi165.stdout:ID_LIKE="rhel fedora" 2024-04-26T01:27:12.956 INFO:teuthology.orchestra.run.smithi165.stdout:VERSION_ID="9" 2024-04-26T01:27:12.956 INFO:teuthology.orchestra.run.smithi165.stdout:PLATFORM_ID="platform:el9" 2024-04-26T01:27:12.956 INFO:teuthology.orchestra.run.smithi165.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T01:27:12.956 INFO:teuthology.orchestra.run.smithi165.stdout:ANSI_COLOR="0;31" 2024-04-26T01:27:12.956 INFO:teuthology.orchestra.run.smithi165.stdout:LOGO="fedora-logo-icon" 2024-04-26T01:27:12.956 INFO:teuthology.orchestra.run.smithi165.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T01:27:12.956 INFO:teuthology.orchestra.run.smithi165.stdout:HOME_URL="https://centos.org/" 2024-04-26T01:27:12.956 INFO:teuthology.orchestra.run.smithi165.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T01:27:12.957 INFO:teuthology.orchestra.run.smithi165.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T01:27:12.957 INFO:teuthology.orchestra.run.smithi165.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T01:27:12.957 INFO:teuthology.lock.ops:Updating smithi165.front.sepia.ceph.com on lock server 2024-04-26T01:27:12.978 DEBUG:teuthology.orchestra.run.smithi172:> uname -m 2024-04-26T01:27:12.996 INFO:teuthology.orchestra.run.smithi172.stdout:x86_64 2024-04-26T01:27:12.996 DEBUG:teuthology.orchestra.run.smithi172:> cat /etc/os-release 2024-04-26T01:27:13.053 INFO:teuthology.orchestra.run.smithi172.stdout:NAME="CentOS Stream" 2024-04-26T01:27:13.054 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION="9" 2024-04-26T01:27:13.054 INFO:teuthology.orchestra.run.smithi172.stdout:ID="centos" 2024-04-26T01:27:13.054 INFO:teuthology.orchestra.run.smithi172.stdout:ID_LIKE="rhel fedora" 2024-04-26T01:27:13.054 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION_ID="9" 2024-04-26T01:27:13.054 INFO:teuthology.orchestra.run.smithi172.stdout:PLATFORM_ID="platform:el9" 2024-04-26T01:27:13.054 INFO:teuthology.orchestra.run.smithi172.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T01:27:13.054 INFO:teuthology.orchestra.run.smithi172.stdout:ANSI_COLOR="0;31" 2024-04-26T01:27:13.054 INFO:teuthology.orchestra.run.smithi172.stdout:LOGO="fedora-logo-icon" 2024-04-26T01:27:13.054 INFO:teuthology.orchestra.run.smithi172.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T01:27:13.055 INFO:teuthology.orchestra.run.smithi172.stdout:HOME_URL="https://centos.org/" 2024-04-26T01:27:13.055 INFO:teuthology.orchestra.run.smithi172.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T01:27:13.055 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T01:27:13.055 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T01:27:13.055 INFO:teuthology.lock.ops:Updating smithi172.front.sepia.ceph.com on lock server 2024-04-26T01:27:13.086 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-26T01:27:13.098 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-26T01:27:13.123 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-26T01:27:13.123 DEBUG:teuthology.orchestra.run.smithi151:> test '!' -e /home/ubuntu/cephtest 2024-04-26T01:27:13.126 DEBUG:teuthology.orchestra.run.smithi165:> test '!' -e /home/ubuntu/cephtest 2024-04-26T01:27:13.128 DEBUG:teuthology.orchestra.run.smithi172:> test '!' -e /home/ubuntu/cephtest 2024-04-26T01:27:13.144 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-26T01:27:13.205 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-26T01:27:13.205 DEBUG:teuthology.orchestra.run.smithi151:> test -z $(ls -A /var/lib/ceph) 2024-04-26T01:27:13.208 DEBUG:teuthology.orchestra.run.smithi165:> test -z $(ls -A /var/lib/ceph) 2024-04-26T01:27:13.210 DEBUG:teuthology.orchestra.run.smithi172:> test -z $(ls -A /var/lib/ceph) 2024-04-26T01:27:13.313 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-26T01:27:13.380 INFO:teuthology.run_tasks:Running task kernel... 2024-04-26T01:27:13.396 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2024-04-26T01:27:13.396 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-04-26T01:27:13.396 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-04-26T01:27:13.396 DEBUG:teuthology.orchestra.run.smithi151:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T01:27:13.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T01:27:13.413 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-04-26T01:27:13.665 DEBUG:teuthology.task.kernel:sha1 for client.0 is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T01:27:13.666 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-26T01:27:13.666 DEBUG:teuthology.orchestra.run.smithi151:> uname -r 2024-04-26T01:27:13.683 INFO:teuthology.orchestra.run.smithi151.stdout:5.14.0-437.el9.x86_64 2024-04-26T01:27:13.683 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T01:27:13.683 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-26T01:27:13.684 DEBUG:teuthology.orchestra.run.smithi151:> sudo yum upgrade -y linux-firmware 2024-04-26T01:27:16.066 INFO:teuthology.orchestra.run.smithi151.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.0 MB 00:00 2024-04-26T01:27:18.151 INFO:teuthology.orchestra.run.smithi151.stdout:CentOS Stream 9 - AppStream 19 MB/s | 19 MB 00:01 2024-04-26T01:27:22.427 INFO:teuthology.orchestra.run.smithi151.stdout:CentOS Stream 9 - CRB 9.1 MB/s | 6.2 MB 00:00 2024-04-26T01:27:23.991 INFO:teuthology.orchestra.run.smithi151.stdout:CentOS Stream 9 - Extras packages 79 kB/s | 16 kB 00:00 2024-04-26T01:27:29.217 INFO:teuthology.orchestra.run.smithi151.stdout:Extra Packages for Enterprise Linux 4.1 MB/s | 21 MB 00:05 2024-04-26T01:27:34.477 INFO:teuthology.orchestra.run.smithi151.stdout:lab-extras 5.4 kB/s | 1.7 kB 00:00 2024-04-26T01:27:36.534 INFO:teuthology.orchestra.run.smithi151.stdout:Dependencies resolved. 2024-04-26T01:27:36.535 INFO:teuthology.orchestra.run.smithi151.stdout:Nothing to do. 2024-04-26T01:27:36.535 INFO:teuthology.orchestra.run.smithi151.stdout:Complete! 2024-04-26T01:27:36.649 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-26T01:27:36.649 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T01:27:36.800 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-26T01:27:36.800 DEBUG:teuthology.orchestra.run.smithi151:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-26T01:27:36.802 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi151.front.sepia.ceph.com to complete... 2024-04-26T01:27:55.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26 01:27:55 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-26T01:27:55.331 INFO:teuthology.orchestra.run.smithi151.stderr:FINISHED --2024-04-26 01:27:55-- 2024-04-26T01:27:55.331 INFO:teuthology.orchestra.run.smithi151.stderr:Total wall clock time: 18s 2024-04-26T01:27:55.331 INFO:teuthology.orchestra.run.smithi151.stderr:Downloaded: 1 files, 341M in 18s (19.0 MB/s) 2024-04-26T01:27:55.334 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-26T01:27:55.334 DEBUG:teuthology.orchestra.run.smithi151:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-26T01:27:57.464 INFO:teuthology.orchestra.run.smithi151.stdout:Verifying... ######################################## 2024-04-26T01:27:57.602 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing... ######################################## 2024-04-26T01:27:57.770 INFO:teuthology.orchestra.run.smithi151.stdout:Updating / installing... 2024-04-26T01:28:05.817 INFO:teuthology.orchestra.run.smithi151.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-26T01:28:45.639 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi151.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-26T01:28:45.640 DEBUG:teuthology.orchestra.run.smithi151:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-26T01:28:45.728 INFO:teuthology.orchestra.run.smithi151.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-26T01:28:45.728 INFO:teuthology.orchestra.run.smithi151.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-26T01:28:45.728 INFO:teuthology.orchestra.run.smithi151.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-26T01:28:45.728 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-26T01:28:45.728 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-26T01:28:45.728 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-26T01:28:45.728 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-26T01:28:45.729 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-26T01:28:45.729 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-26T01:28:45.729 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-26T01:28:45.729 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-26T01:28:45.729 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-26T01:28:45.729 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-26T01:28:45.729 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-26T01:28:45.729 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-26T01:28:45.729 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-26T01:28:45.729 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-26T01:28:45.729 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-26T01:28:45.729 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-26T01:28:45.729 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-26T01:28:45.729 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-26T01:28:45.730 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-26T01:28:45.730 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-26T01:28:45.730 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-26T01:28:45.730 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-26T01:28:45.730 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-26T01:28:45.730 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-26T01:28:45.730 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-26T01:28:45.730 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-26T01:28:45.730 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-26T01:28:45.730 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-26T01:28:45.730 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-26T01:28:45.730 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-26T01:28:45.731 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-26T01:28:45.731 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-26T01:28:45.731 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-26T01:28:45.731 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-26T01:28:45.731 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-26T01:28:45.731 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-26T01:28:45.731 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-26T01:28:45.731 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-26T01:28:45.731 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-26T01:28:45.731 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-26T01:28:45.731 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-26T01:28:45.731 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-26T01:28:45.731 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-26T01:28:45.732 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-26T01:28:45.732 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-26T01:28:45.732 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-26T01:28:45.732 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-26T01:28:45.732 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-26T01:28:45.732 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-26T01:28:45.732 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-26T01:28:45.732 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-26T01:28:45.732 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-26T01:28:45.732 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-26T01:28:45.732 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-26T01:28:45.732 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-26T01:28:45.732 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-26T01:28:45.733 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-26T01:28:45.733 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-26T01:28:45.733 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-26T01:28:45.733 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-26T01:28:45.733 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-26T01:28:45.733 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-26T01:28:45.733 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-26T01:28:45.733 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-26T01:28:45.733 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-26T01:28:45.733 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-26T01:28:45.733 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-26T01:28:45.733 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-26T01:28:45.733 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-26T01:28:45.734 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-26T01:28:45.734 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-26T01:28:45.734 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-26T01:28:45.734 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-26T01:28:45.734 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-26T01:28:45.734 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-26T01:28:45.734 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-26T01:28:45.734 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-26T01:28:45.734 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-26T01:28:45.734 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-26T01:28:45.734 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-26T01:28:45.734 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-26T01:28:45.734 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-26T01:28:45.735 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-26T01:28:45.735 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-26T01:28:45.735 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-26T01:28:45.735 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-26T01:28:45.735 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-26T01:28:45.735 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-26T01:28:45.735 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-26T01:28:45.735 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-26T01:28:45.735 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-26T01:28:45.735 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-26T01:28:45.735 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-26T01:28:45.735 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-26T01:28:45.735 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-26T01:28:45.736 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-26T01:28:45.736 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-26T01:28:45.736 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-26T01:28:45.736 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-26T01:28:45.736 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-26T01:28:45.736 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-26T01:28:45.736 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-26T01:28:45.736 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-26T01:28:45.736 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-26T01:28:45.736 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-26T01:28:45.736 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-26T01:28:45.736 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-26T01:28:45.736 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-26T01:28:45.737 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-26T01:28:45.737 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-26T01:28:45.737 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-26T01:28:45.737 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-26T01:28:45.737 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-26T01:28:45.737 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-26T01:28:45.737 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-26T01:28:45.737 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-26T01:28:45.737 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-26T01:28:45.737 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-26T01:28:45.737 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-26T01:28:45.737 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-26T01:28:45.737 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-26T01:28:45.738 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-26T01:28:45.738 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-26T01:28:45.738 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-26T01:28:45.738 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-26T01:28:45.738 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-26T01:28:45.738 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-26T01:28:45.738 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-26T01:28:45.738 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-26T01:28:45.738 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-26T01:28:45.738 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-26T01:28:45.738 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-26T01:28:45.738 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-26T01:28:45.739 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-26T01:28:45.739 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-26T01:28:45.739 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-26T01:28:45.739 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-26T01:28:45.739 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-26T01:28:45.739 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-26T01:28:45.739 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-26T01:28:45.739 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-26T01:28:45.739 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-26T01:28:45.739 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-26T01:28:45.739 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-26T01:28:45.739 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-26T01:28:45.739 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-26T01:28:45.739 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-26T01:28:45.740 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-26T01:28:45.740 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-26T01:28:45.740 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-26T01:28:45.740 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-26T01:28:45.740 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-26T01:28:45.740 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-26T01:28:45.740 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-26T01:28:45.740 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-26T01:28:45.740 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-26T01:28:45.740 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-26T01:28:45.740 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-26T01:28:45.740 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-26T01:28:45.740 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-26T01:28:45.741 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-26T01:28:45.741 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-26T01:28:45.741 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-26T01:28:45.741 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-26T01:28:45.741 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-26T01:28:45.741 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-26T01:28:45.741 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-26T01:28:45.741 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-26T01:28:45.741 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-26T01:28:45.741 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-26T01:28:45.741 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-26T01:28:45.741 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-26T01:28:45.741 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-26T01:28:45.742 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-26T01:28:45.742 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-26T01:28:45.742 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-26T01:28:45.742 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-26T01:28:45.742 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-26T01:28:45.742 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-26T01:28:45.742 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-26T01:28:45.742 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-26T01:28:45.742 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-26T01:28:45.742 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-26T01:28:45.742 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-26T01:28:45.742 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-26T01:28:45.743 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-26T01:28:45.743 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-26T01:28:45.743 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-26T01:28:45.743 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-26T01:28:45.743 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-26T01:28:45.743 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-26T01:28:45.743 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-26T01:28:45.743 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-26T01:28:45.743 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-26T01:28:45.743 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-26T01:28:45.743 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-26T01:28:45.743 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-26T01:28:45.743 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-26T01:28:45.743 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-26T01:28:45.744 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-26T01:28:45.744 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-26T01:28:45.744 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-26T01:28:45.744 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-26T01:28:45.744 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-26T01:28:45.744 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-26T01:28:45.744 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-26T01:28:45.744 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-26T01:28:45.744 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-26T01:28:45.744 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-26T01:28:45.744 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-26T01:28:45.744 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-26T01:28:45.744 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-26T01:28:45.744 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-26T01:28:45.745 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-26T01:28:45.745 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-26T01:28:45.745 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-26T01:28:45.745 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-26T01:28:45.745 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-26T01:28:45.745 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-26T01:28:45.745 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-26T01:28:45.745 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-26T01:28:45.745 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-26T01:28:45.745 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-26T01:28:45.745 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-26T01:28:45.745 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-26T01:28:45.745 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-26T01:28:45.745 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-26T01:28:45.746 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-26T01:28:45.746 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-26T01:28:45.746 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-26T01:28:45.746 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-26T01:28:45.746 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-26T01:28:45.746 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-26T01:28:45.746 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-26T01:28:45.746 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-26T01:28:45.746 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-26T01:28:45.746 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-26T01:28:45.746 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-26T01:28:45.746 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-26T01:28:45.747 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-26T01:28:45.747 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-26T01:28:45.747 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-26T01:28:45.747 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-26T01:28:45.747 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-26T01:28:45.747 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-26T01:28:45.747 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-26T01:28:45.747 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-26T01:28:45.747 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-26T01:28:45.747 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-26T01:28:45.747 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-26T01:28:45.747 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-26T01:28:45.747 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-26T01:28:45.748 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-26T01:28:45.748 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-26T01:28:45.748 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-26T01:28:45.748 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-26T01:28:45.748 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-26T01:28:45.748 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-26T01:28:45.748 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-26T01:28:45.748 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-26T01:28:45.748 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-26T01:28:45.748 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-26T01:28:45.748 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-26T01:28:45.748 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-26T01:28:45.748 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-26T01:28:45.749 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-26T01:28:45.749 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-26T01:28:45.749 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-26T01:28:45.749 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-26T01:28:45.749 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-26T01:28:45.749 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-26T01:28:45.749 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-26T01:28:45.749 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-26T01:28:45.749 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-26T01:28:45.749 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-26T01:28:45.749 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-26T01:28:45.749 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-26T01:28:45.749 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-26T01:28:45.749 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-26T01:28:45.750 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-26T01:28:45.750 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-26T01:28:45.750 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-26T01:28:45.750 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-26T01:28:45.750 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-26T01:28:45.750 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-26T01:28:45.750 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-26T01:28:45.750 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-26T01:28:45.750 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-26T01:28:45.750 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-26T01:28:45.750 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-26T01:28:45.750 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-26T01:28:45.750 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-26T01:28:45.751 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-26T01:28:45.751 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-26T01:28:45.751 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-26T01:28:45.751 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-26T01:28:45.751 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-26T01:28:45.751 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-26T01:28:45.751 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-26T01:28:45.751 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-26T01:28:45.751 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-26T01:28:45.751 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-26T01:28:45.751 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-26T01:28:45.751 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-26T01:28:45.751 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-26T01:28:45.751 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-26T01:28:45.751 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-26T01:28:45.752 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-26T01:28:45.752 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-26T01:28:45.752 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-26T01:28:45.752 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-26T01:28:45.752 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-26T01:28:45.752 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-26T01:28:45.752 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-26T01:28:45.752 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-26T01:28:45.752 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-26T01:28:45.752 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-26T01:28:45.752 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-26T01:28:45.752 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-26T01:28:45.752 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-26T01:28:45.753 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-26T01:28:45.753 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-26T01:28:45.753 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-26T01:28:45.753 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-26T01:28:45.753 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-26T01:28:45.753 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-26T01:28:45.753 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-26T01:28:45.753 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-26T01:28:45.753 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-26T01:28:45.753 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-26T01:28:45.753 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-26T01:28:45.753 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-26T01:28:45.753 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-26T01:28:45.753 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-26T01:28:45.753 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-26T01:28:45.754 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-26T01:28:45.754 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-26T01:28:45.754 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-26T01:28:45.754 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-26T01:28:45.754 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-26T01:28:45.754 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-26T01:28:45.754 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-26T01:28:45.754 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-26T01:28:45.754 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-26T01:28:45.754 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-26T01:28:45.754 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-26T01:28:45.754 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-26T01:28:45.754 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-26T01:28:45.755 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-26T01:28:45.755 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-26T01:28:45.755 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-26T01:28:45.755 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-26T01:28:45.755 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-26T01:28:45.755 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-26T01:28:45.755 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-26T01:28:45.755 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-26T01:28:45.755 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-26T01:28:45.755 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-26T01:28:45.755 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-26T01:28:45.755 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-26T01:28:45.755 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-26T01:28:45.756 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-26T01:28:45.756 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-26T01:28:45.756 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-26T01:28:45.756 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-26T01:28:45.756 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-26T01:28:45.756 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-26T01:28:45.756 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-26T01:28:45.756 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-26T01:28:45.756 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-26T01:28:45.756 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-26T01:28:45.756 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-26T01:28:45.756 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-26T01:28:45.756 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-26T01:28:45.757 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-26T01:28:45.757 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-26T01:28:45.757 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-26T01:28:45.757 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-26T01:28:45.757 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-26T01:28:45.757 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-26T01:28:45.757 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-26T01:28:45.757 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-26T01:28:45.757 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-26T01:28:45.757 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-26T01:28:45.757 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-26T01:28:45.757 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-26T01:28:45.757 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-26T01:28:45.757 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-26T01:28:45.758 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-26T01:28:45.758 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-26T01:28:45.758 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-26T01:28:45.758 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-26T01:28:45.758 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-26T01:28:45.758 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-26T01:28:45.758 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-26T01:28:45.758 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-26T01:28:45.758 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-26T01:28:45.758 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-26T01:28:45.758 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-26T01:28:45.758 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-26T01:28:45.758 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-26T01:28:45.759 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-26T01:28:45.759 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-26T01:28:45.759 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-26T01:28:45.759 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-26T01:28:45.759 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-26T01:28:45.759 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-26T01:28:45.759 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-26T01:28:45.759 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-26T01:28:45.759 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-26T01:28:45.759 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-26T01:28:45.759 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-26T01:28:45.759 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-26T01:28:45.759 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-26T01:28:45.759 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-26T01:28:45.760 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-26T01:28:45.760 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-26T01:28:45.760 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-26T01:28:45.760 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-26T01:28:45.760 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-26T01:28:45.760 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-26T01:28:45.760 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-26T01:28:45.760 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-26T01:28:45.760 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-26T01:28:45.760 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-26T01:28:45.760 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-26T01:28:45.760 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-26T01:28:45.761 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-26T01:28:45.761 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-26T01:28:45.761 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-26T01:28:45.761 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-26T01:28:45.761 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-26T01:28:45.761 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-26T01:28:45.761 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-26T01:28:45.761 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-26T01:28:45.761 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-26T01:28:45.761 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-26T01:28:45.761 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-26T01:28:45.761 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-26T01:28:45.761 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-26T01:28:45.761 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-26T01:28:45.762 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-26T01:28:45.762 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-26T01:28:45.762 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-26T01:28:45.762 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-26T01:28:45.762 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-26T01:28:45.762 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-26T01:28:45.762 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-26T01:28:45.762 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-26T01:28:45.762 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-26T01:28:45.762 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-26T01:28:45.762 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-26T01:28:45.762 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-26T01:28:45.762 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-26T01:28:45.762 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-26T01:28:45.763 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-26T01:28:45.763 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-26T01:28:45.763 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-26T01:28:45.763 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-26T01:28:45.763 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-26T01:28:45.763 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-26T01:28:45.763 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-26T01:28:45.763 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-26T01:28:45.763 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-26T01:28:45.763 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-26T01:28:45.763 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-26T01:28:45.763 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-26T01:28:45.763 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-26T01:28:45.764 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-26T01:28:45.764 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-26T01:28:45.764 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-26T01:28:45.764 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-26T01:28:45.764 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-26T01:28:45.764 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-26T01:28:45.764 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-26T01:28:45.764 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-26T01:28:45.764 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-26T01:28:45.764 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-26T01:28:45.764 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-26T01:28:45.764 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-26T01:28:45.764 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-26T01:28:45.765 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-26T01:28:45.765 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-26T01:28:45.765 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-26T01:28:45.765 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-26T01:28:45.765 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-26T01:28:45.765 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-26T01:28:45.765 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-26T01:28:45.765 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-26T01:28:45.765 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-26T01:28:45.765 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-26T01:28:45.765 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-26T01:28:45.765 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-26T01:28:45.765 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-26T01:28:45.765 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-26T01:28:45.766 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-26T01:28:45.766 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-26T01:28:45.766 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-26T01:28:45.766 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-26T01:28:45.766 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-26T01:28:45.766 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-26T01:28:45.766 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-26T01:28:45.766 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-26T01:28:45.766 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-26T01:28:45.766 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-26T01:28:45.766 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-26T01:28:45.766 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-26T01:28:45.766 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-26T01:28:45.767 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-26T01:28:45.767 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-26T01:28:45.767 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-26T01:28:45.767 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-26T01:28:45.767 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-26T01:28:45.767 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-26T01:28:45.767 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-26T01:28:45.767 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-26T01:28:45.767 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-26T01:28:45.767 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-26T01:28:45.767 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-26T01:28:45.767 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-26T01:28:45.767 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-26T01:28:45.768 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-26T01:28:45.768 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-26T01:28:45.768 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-26T01:28:45.768 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-26T01:28:45.768 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-26T01:28:45.768 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-26T01:28:45.768 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-26T01:28:45.768 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-26T01:28:45.768 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-26T01:28:45.768 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-26T01:28:45.768 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-26T01:28:45.768 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-26T01:28:45.768 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-26T01:28:45.769 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-26T01:28:45.769 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-26T01:28:45.769 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-26T01:28:45.769 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-26T01:28:45.769 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-26T01:28:45.769 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-26T01:28:45.769 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-26T01:28:45.769 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-26T01:28:45.769 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-26T01:28:45.769 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-26T01:28:45.769 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-26T01:28:45.769 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-26T01:28:45.769 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-26T01:28:45.769 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-26T01:28:45.770 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-26T01:28:45.770 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-26T01:28:45.770 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-26T01:28:45.770 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-26T01:28:45.770 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-26T01:28:45.770 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-26T01:28:45.770 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-26T01:28:45.770 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-26T01:28:45.770 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-26T01:28:45.770 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-26T01:28:45.770 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-26T01:28:45.770 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-26T01:28:45.770 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-26T01:28:45.771 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-26T01:28:45.771 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-26T01:28:45.771 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-26T01:28:45.771 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-26T01:28:45.771 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-26T01:28:45.771 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-26T01:28:45.771 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-26T01:28:45.771 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-26T01:28:45.771 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-26T01:28:45.771 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-26T01:28:45.771 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-26T01:28:45.771 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-26T01:28:45.771 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-26T01:28:45.772 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-26T01:28:45.772 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-26T01:28:45.772 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-26T01:28:45.772 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-26T01:28:45.772 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-26T01:28:45.772 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-26T01:28:45.772 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-26T01:28:45.772 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-26T01:28:45.772 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-26T01:28:45.772 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-26T01:28:45.772 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-26T01:28:45.772 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-26T01:28:45.772 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-26T01:28:45.773 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-26T01:28:45.773 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-26T01:28:45.773 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-26T01:28:45.773 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-26T01:28:45.773 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-26T01:28:45.773 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-26T01:28:45.773 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-26T01:28:45.773 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-26T01:28:45.773 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-26T01:28:45.773 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-26T01:28:45.773 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-26T01:28:45.773 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-26T01:28:45.773 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-26T01:28:45.774 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-26T01:28:45.774 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-26T01:28:45.774 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-26T01:28:45.774 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-26T01:28:45.774 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-26T01:28:45.774 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-26T01:28:45.774 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-26T01:28:45.774 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-26T01:28:45.774 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-26T01:28:45.774 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-26T01:28:45.774 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-26T01:28:45.774 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-26T01:28:45.774 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-26T01:28:45.775 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-26T01:28:45.775 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-26T01:28:45.775 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-26T01:28:45.775 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-26T01:28:45.775 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-26T01:28:45.775 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-26T01:28:45.775 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-26T01:28:45.775 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-26T01:28:45.775 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-26T01:28:45.775 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-26T01:28:45.775 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-26T01:28:45.775 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-26T01:28:45.775 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-26T01:28:45.775 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-26T01:28:45.776 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-26T01:28:45.776 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-26T01:28:45.776 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-26T01:28:45.776 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-26T01:28:45.776 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-26T01:28:45.776 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-26T01:28:45.776 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-26T01:28:45.776 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-26T01:28:45.776 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-26T01:28:45.776 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-26T01:28:45.776 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-26T01:28:45.776 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-26T01:28:45.777 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-26T01:28:45.777 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-26T01:28:45.777 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-26T01:28:45.777 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-26T01:28:45.777 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-26T01:28:45.777 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-26T01:28:45.777 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-26T01:28:45.777 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-26T01:28:45.777 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-26T01:28:45.777 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-26T01:28:45.777 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-26T01:28:45.777 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-26T01:28:45.777 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-26T01:28:45.778 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-26T01:28:45.778 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-26T01:28:45.778 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-26T01:28:45.778 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-26T01:28:45.778 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-26T01:28:45.778 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-26T01:28:45.778 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-26T01:28:45.778 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-26T01:28:45.778 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-26T01:28:45.778 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-26T01:28:45.778 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-26T01:28:45.778 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-26T01:28:45.778 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-26T01:28:45.779 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-26T01:28:45.779 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-26T01:28:45.779 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-26T01:28:45.779 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-26T01:28:45.779 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-26T01:28:45.779 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-26T01:28:45.779 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-26T01:28:45.779 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-26T01:28:45.779 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-26T01:28:45.779 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-26T01:28:45.779 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-26T01:28:45.780 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-26T01:28:45.780 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-26T01:28:45.780 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-26T01:28:45.780 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-26T01:28:45.780 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-26T01:28:45.780 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-26T01:28:45.780 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-26T01:28:45.780 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-26T01:28:45.780 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-26T01:28:45.780 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-26T01:28:45.780 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-26T01:28:45.780 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-26T01:28:45.780 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-26T01:28:45.781 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-26T01:28:45.781 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-26T01:28:45.781 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-26T01:28:45.781 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-26T01:28:45.781 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-26T01:28:45.781 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-26T01:28:45.781 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-26T01:28:45.781 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-26T01:28:45.781 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-26T01:28:45.781 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-26T01:28:45.781 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-26T01:28:45.781 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-26T01:28:45.781 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-26T01:28:45.782 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-26T01:28:45.782 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-26T01:28:45.782 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-26T01:28:45.782 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-26T01:28:45.782 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-26T01:28:45.782 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-26T01:28:45.782 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-26T01:28:45.782 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-26T01:28:45.782 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-26T01:28:45.782 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-26T01:28:45.782 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-26T01:28:45.782 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-26T01:28:45.782 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-26T01:28:45.783 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-26T01:28:45.783 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-26T01:28:45.783 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-26T01:28:45.783 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-26T01:28:45.783 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-26T01:28:45.783 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-26T01:28:45.783 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-26T01:28:45.783 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-26T01:28:45.783 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-26T01:28:45.783 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-26T01:28:45.783 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-26T01:28:45.783 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-26T01:28:45.783 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-26T01:28:45.784 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-26T01:28:45.784 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-26T01:28:45.784 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-26T01:28:45.784 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-26T01:28:45.784 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-26T01:28:45.784 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-26T01:28:45.784 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-26T01:28:45.784 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-26T01:28:45.784 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-26T01:28:45.784 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-26T01:28:45.784 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-26T01:28:45.784 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-26T01:28:45.784 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-26T01:28:45.785 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-26T01:28:45.785 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-26T01:28:45.785 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-26T01:28:45.785 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-26T01:28:45.785 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-26T01:28:45.785 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-26T01:28:45.785 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-26T01:28:45.785 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-26T01:28:45.785 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-26T01:28:45.785 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-26T01:28:45.785 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-26T01:28:45.785 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-26T01:28:45.785 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-26T01:28:45.786 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-26T01:28:45.786 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-26T01:28:45.786 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-26T01:28:45.786 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-26T01:28:45.786 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-26T01:28:45.786 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-26T01:28:45.786 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-26T01:28:45.786 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-26T01:28:45.786 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-26T01:28:45.786 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-26T01:28:45.786 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-26T01:28:45.786 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-26T01:28:45.786 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-26T01:28:45.787 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-26T01:28:45.787 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-26T01:28:45.787 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-26T01:28:45.787 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-26T01:28:45.787 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-26T01:28:45.787 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-26T01:28:45.787 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-26T01:28:45.787 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-26T01:28:45.787 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-26T01:28:45.787 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-26T01:28:45.787 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-26T01:28:45.787 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-26T01:28:45.788 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-26T01:28:45.788 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-26T01:28:45.788 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-26T01:28:45.788 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-26T01:28:45.788 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-26T01:28:45.788 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-26T01:28:45.788 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-26T01:28:45.788 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-26T01:28:45.788 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-26T01:28:45.788 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-26T01:28:45.788 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-26T01:28:45.788 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-26T01:28:45.788 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-26T01:28:45.789 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-26T01:28:45.789 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-26T01:28:45.789 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-26T01:28:45.789 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-26T01:28:45.789 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-26T01:28:45.789 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-26T01:28:45.789 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-26T01:28:45.789 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-26T01:28:45.789 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-26T01:28:45.789 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-26T01:28:45.789 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-26T01:28:45.789 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-26T01:28:45.789 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-26T01:28:45.789 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-26T01:28:45.790 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-26T01:28:45.790 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-26T01:28:45.790 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-26T01:28:45.790 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-26T01:28:45.790 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-26T01:28:45.790 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-26T01:28:45.790 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-26T01:28:45.790 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-26T01:28:45.790 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-26T01:28:45.790 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-26T01:28:45.790 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-26T01:28:45.791 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-26T01:28:45.791 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-26T01:28:45.791 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-26T01:28:45.791 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-26T01:28:45.791 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-26T01:28:45.791 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-26T01:28:45.791 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-26T01:28:45.791 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-26T01:28:45.791 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-26T01:28:45.791 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-26T01:28:45.791 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-26T01:28:45.791 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-26T01:28:45.791 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-26T01:28:45.791 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-26T01:28:45.792 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-26T01:28:45.792 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-26T01:28:45.792 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-26T01:28:45.792 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-26T01:28:45.792 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-26T01:28:45.792 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-26T01:28:45.792 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-26T01:28:45.792 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-26T01:28:45.792 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-26T01:28:45.792 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-26T01:28:45.792 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-26T01:28:45.792 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-26T01:28:45.792 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-26T01:28:45.792 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-26T01:28:45.793 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-26T01:28:45.793 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-26T01:28:45.793 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-26T01:28:45.793 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-26T01:28:45.793 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-26T01:28:45.793 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-26T01:28:45.793 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-26T01:28:45.793 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-26T01:28:45.793 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-26T01:28:45.793 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-26T01:28:45.793 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-26T01:28:45.793 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-26T01:28:45.793 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-26T01:28:45.793 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-26T01:28:45.794 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-26T01:28:45.794 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-26T01:28:45.794 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-26T01:28:45.794 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-26T01:28:45.794 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-26T01:28:45.794 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-26T01:28:45.794 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-26T01:28:45.794 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-26T01:28:45.794 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-26T01:28:45.794 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-26T01:28:45.794 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-26T01:28:45.794 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-26T01:28:45.794 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-26T01:28:45.795 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-26T01:28:45.795 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-26T01:28:45.795 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-26T01:28:45.795 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-26T01:28:45.795 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-26T01:28:45.795 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-26T01:28:45.795 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-26T01:28:45.795 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-26T01:28:45.795 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-26T01:28:45.795 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-26T01:28:45.795 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-26T01:28:45.795 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-26T01:28:45.795 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-26T01:28:45.796 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-26T01:28:45.796 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-26T01:28:45.796 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-26T01:28:45.796 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-26T01:28:45.796 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-26T01:28:45.796 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-26T01:28:45.796 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-26T01:28:45.796 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-26T01:28:45.796 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-26T01:28:45.796 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-26T01:28:45.796 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-26T01:28:45.796 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-26T01:28:45.796 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-26T01:28:45.796 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-26T01:28:45.797 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-26T01:28:45.797 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-26T01:28:45.797 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-26T01:28:45.797 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-26T01:28:45.797 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-26T01:28:45.797 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-26T01:28:45.797 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-26T01:28:45.797 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-26T01:28:45.797 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-26T01:28:45.797 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-26T01:28:45.797 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-26T01:28:45.797 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-26T01:28:45.797 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-26T01:28:45.798 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-26T01:28:45.798 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-26T01:28:45.798 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-26T01:28:45.798 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-26T01:28:45.798 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-26T01:28:45.798 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-26T01:28:45.798 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-26T01:28:45.798 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-26T01:28:45.798 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-26T01:28:45.798 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-26T01:28:45.798 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-26T01:28:45.798 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-26T01:28:45.798 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-26T01:28:45.798 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-26T01:28:45.799 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-26T01:28:45.799 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-26T01:28:45.799 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-26T01:28:45.799 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-26T01:28:45.799 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-26T01:28:45.799 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-26T01:28:45.799 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-26T01:28:45.799 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-26T01:28:45.799 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-26T01:28:45.799 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-26T01:28:45.799 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-26T01:28:45.799 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-26T01:28:45.799 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-26T01:28:45.799 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-26T01:28:45.800 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-26T01:28:45.800 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-26T01:28:45.800 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-26T01:28:45.800 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-26T01:28:45.800 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-26T01:28:45.800 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-26T01:28:45.800 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-26T01:28:45.800 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-26T01:28:45.800 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-26T01:28:45.800 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-26T01:28:45.800 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-26T01:28:45.800 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-26T01:28:45.800 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-26T01:28:45.800 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-26T01:28:45.801 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-26T01:28:45.801 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-26T01:28:45.801 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-26T01:28:45.801 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-26T01:28:45.801 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-26T01:28:45.801 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-26T01:28:45.801 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-26T01:28:45.801 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-26T01:28:45.801 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-26T01:28:45.801 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-26T01:28:45.801 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-26T01:28:45.801 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-26T01:28:45.801 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-26T01:28:45.801 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-26T01:28:45.802 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-26T01:28:45.802 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-26T01:28:45.802 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-26T01:28:45.802 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-26T01:28:45.802 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-26T01:28:45.802 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-26T01:28:45.802 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-26T01:28:45.802 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-26T01:28:45.802 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-26T01:28:45.802 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-26T01:28:45.802 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-26T01:28:45.802 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-26T01:28:45.802 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-26T01:28:45.803 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-26T01:28:45.803 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-26T01:28:45.803 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-26T01:28:45.803 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-26T01:28:45.803 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-26T01:28:45.803 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-26T01:28:45.803 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-26T01:28:45.803 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-26T01:28:45.803 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-26T01:28:45.803 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-26T01:28:45.803 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-26T01:28:45.803 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-26T01:28:45.803 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-26T01:28:45.803 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-26T01:28:45.804 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-26T01:28:45.804 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-26T01:28:45.804 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-26T01:28:45.804 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-26T01:28:45.804 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-26T01:28:45.804 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-26T01:28:45.804 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-26T01:28:45.804 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-26T01:28:45.804 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-26T01:28:45.804 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-26T01:28:45.804 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-26T01:28:45.804 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-26T01:28:45.804 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-26T01:28:45.804 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-26T01:28:45.805 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-26T01:28:45.805 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-26T01:28:45.805 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-26T01:28:45.805 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-26T01:28:45.805 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-26T01:28:45.805 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-26T01:28:45.805 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-26T01:28:45.805 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-26T01:28:45.805 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-26T01:28:45.805 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-26T01:28:45.805 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-26T01:28:45.805 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-26T01:28:45.805 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-26T01:28:45.806 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-26T01:28:45.806 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-26T01:28:45.806 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-26T01:28:45.806 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-26T01:28:45.806 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-26T01:28:45.806 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-26T01:28:45.806 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-26T01:28:45.806 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-26T01:28:45.806 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-26T01:28:45.806 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-26T01:28:45.806 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-26T01:28:45.806 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-26T01:28:45.807 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-26T01:28:45.807 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-26T01:28:45.807 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-26T01:28:45.807 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-26T01:28:45.807 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-26T01:28:45.807 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-26T01:28:45.807 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-26T01:28:45.807 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-26T01:28:45.807 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-26T01:28:45.807 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-26T01:28:45.807 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-26T01:28:45.807 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-26T01:28:45.807 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-26T01:28:45.808 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-26T01:28:45.808 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-26T01:28:45.808 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-26T01:28:45.808 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-26T01:28:45.808 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-26T01:28:45.808 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-26T01:28:45.808 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-26T01:28:45.808 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-26T01:28:45.808 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-26T01:28:45.808 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-26T01:28:45.808 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-26T01:28:45.808 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-26T01:28:45.808 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-26T01:28:45.808 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-26T01:28:45.809 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-26T01:28:45.809 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-26T01:28:45.809 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-26T01:28:45.809 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-26T01:28:45.809 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-26T01:28:45.809 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-26T01:28:45.809 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-26T01:28:45.809 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-26T01:28:45.809 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-26T01:28:45.809 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-26T01:28:45.809 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-26T01:28:45.809 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-26T01:28:45.809 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-26T01:28:45.809 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-26T01:28:45.810 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-26T01:28:45.810 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-26T01:28:45.810 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-26T01:28:45.810 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-26T01:28:45.810 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-26T01:28:45.810 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-26T01:28:45.810 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-26T01:28:45.810 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-26T01:28:45.810 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-26T01:28:45.810 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-26T01:28:45.810 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-26T01:28:45.810 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-26T01:28:45.810 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-26T01:28:45.811 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-26T01:28:45.811 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-26T01:28:45.811 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-26T01:28:45.811 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-26T01:28:45.811 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-26T01:28:45.811 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-26T01:28:45.811 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-26T01:28:45.811 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-26T01:28:45.811 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-26T01:28:45.811 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-26T01:28:45.811 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-26T01:28:45.811 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-26T01:28:45.811 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-26T01:28:45.812 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-26T01:28:45.812 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-26T01:28:45.812 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-26T01:28:45.812 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-26T01:28:45.812 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-26T01:28:45.812 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-26T01:28:45.812 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-26T01:28:45.812 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-26T01:28:45.812 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-26T01:28:45.812 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-26T01:28:45.812 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-26T01:28:45.812 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-26T01:28:45.812 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-26T01:28:45.812 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-26T01:28:45.813 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-26T01:28:45.813 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-26T01:28:45.813 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-26T01:28:45.813 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-26T01:28:45.813 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-26T01:28:45.813 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-26T01:28:45.813 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-26T01:28:45.813 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-26T01:28:45.813 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-26T01:28:45.813 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-26T01:28:45.813 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-26T01:28:45.813 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-26T01:28:45.813 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-26T01:28:45.814 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-26T01:28:45.814 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-26T01:28:45.814 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-26T01:28:45.814 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-26T01:28:45.814 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-26T01:28:45.814 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-26T01:28:45.814 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-26T01:28:45.814 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-26T01:28:45.814 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-26T01:28:45.814 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-26T01:28:45.815 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-26T01:28:45.815 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-26T01:28:45.815 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-26T01:28:45.815 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-26T01:28:45.815 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-26T01:28:45.815 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-26T01:28:45.815 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-26T01:28:45.815 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-26T01:28:45.815 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-26T01:28:45.815 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-26T01:28:45.815 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-26T01:28:45.815 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-26T01:28:45.816 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-26T01:28:45.816 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-26T01:28:45.816 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-26T01:28:45.816 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-26T01:28:45.816 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-26T01:28:45.816 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-26T01:28:45.816 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-26T01:28:45.816 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-26T01:28:45.816 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-26T01:28:45.816 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-26T01:28:45.816 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-26T01:28:45.816 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-26T01:28:45.816 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-26T01:28:45.816 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-26T01:28:45.817 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-26T01:28:45.817 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-26T01:28:45.817 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-26T01:28:45.817 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-26T01:28:45.817 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-26T01:28:45.817 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-26T01:28:45.817 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-26T01:28:45.817 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-26T01:28:45.817 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-26T01:28:45.817 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-26T01:28:45.817 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-26T01:28:45.817 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-26T01:28:45.817 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-26T01:28:45.817 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-26T01:28:45.818 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-26T01:28:45.818 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-26T01:28:45.818 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-26T01:28:45.818 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-26T01:28:45.818 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-26T01:28:45.818 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-26T01:28:45.818 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-26T01:28:45.818 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-26T01:28:45.818 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-26T01:28:45.818 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-26T01:28:45.818 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-26T01:28:45.818 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-26T01:28:45.818 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-26T01:28:45.819 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-26T01:28:45.819 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-26T01:28:45.819 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-26T01:28:45.819 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-26T01:28:45.819 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-26T01:28:45.819 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-26T01:28:45.819 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-26T01:28:45.819 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-26T01:28:45.819 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-26T01:28:45.819 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-26T01:28:45.819 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-26T01:28:45.819 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-26T01:28:45.819 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-26T01:28:45.819 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-26T01:28:45.819 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-26T01:28:45.820 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-26T01:28:45.820 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-26T01:28:45.820 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-26T01:28:45.820 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-26T01:28:45.820 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-26T01:28:45.820 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-26T01:28:45.820 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-26T01:28:45.820 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-26T01:28:45.820 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-26T01:28:45.820 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-26T01:28:45.820 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-26T01:28:45.820 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-26T01:28:45.820 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-26T01:28:45.821 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-26T01:28:45.821 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-26T01:28:45.821 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-26T01:28:45.821 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-26T01:28:45.821 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-26T01:28:45.821 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-26T01:28:45.821 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-26T01:28:45.821 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-26T01:28:45.821 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-26T01:28:45.821 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-26T01:28:45.821 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-26T01:28:45.821 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-26T01:28:45.821 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-26T01:28:45.821 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-26T01:28:45.822 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-26T01:28:45.822 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-26T01:28:45.822 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-26T01:28:45.822 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-26T01:28:45.822 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-26T01:28:45.822 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-26T01:28:45.822 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-26T01:28:45.822 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-26T01:28:45.822 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-26T01:28:45.822 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-26T01:28:45.822 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-26T01:28:45.822 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-26T01:28:45.823 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-26T01:28:45.823 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-26T01:28:45.823 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-26T01:28:45.823 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-26T01:28:45.823 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-26T01:28:45.823 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-26T01:28:45.823 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-26T01:28:45.823 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-26T01:28:45.823 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-26T01:28:45.823 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-26T01:28:45.823 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-26T01:28:45.824 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-26T01:28:45.824 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-26T01:28:45.824 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-26T01:28:45.824 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-26T01:28:45.824 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-26T01:28:45.824 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-26T01:28:45.824 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-26T01:28:45.824 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-26T01:28:45.824 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-26T01:28:45.824 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-26T01:28:45.824 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-26T01:28:45.824 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-26T01:28:45.824 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-26T01:28:45.825 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-26T01:28:45.825 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-26T01:28:45.825 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-26T01:28:45.825 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-26T01:28:45.825 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-26T01:28:45.825 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-26T01:28:45.825 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-26T01:28:45.825 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-26T01:28:45.825 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-26T01:28:45.825 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-26T01:28:45.825 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-26T01:28:45.825 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-26T01:28:45.825 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-26T01:28:45.826 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-26T01:28:45.826 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-26T01:28:45.826 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-26T01:28:45.826 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-26T01:28:45.826 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-26T01:28:45.826 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-26T01:28:45.826 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-26T01:28:45.826 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-26T01:28:45.826 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-26T01:28:45.826 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-26T01:28:45.826 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-26T01:28:45.826 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-26T01:28:45.826 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-26T01:28:45.826 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-26T01:28:45.827 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-26T01:28:45.827 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-26T01:28:45.827 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-26T01:28:45.827 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-26T01:28:45.827 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-26T01:28:45.827 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-26T01:28:45.827 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-26T01:28:45.827 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-26T01:28:45.827 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-26T01:28:45.827 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-26T01:28:45.827 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-26T01:28:45.827 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-26T01:28:45.827 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-26T01:28:45.828 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-26T01:28:45.828 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-26T01:28:45.828 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-26T01:28:45.828 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-26T01:28:45.828 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-26T01:28:45.828 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-26T01:28:45.828 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-26T01:28:45.828 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-26T01:28:45.828 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-26T01:28:45.828 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-26T01:28:45.828 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-26T01:28:45.828 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-26T01:28:45.828 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-26T01:28:45.828 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-26T01:28:45.829 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-26T01:28:45.829 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-26T01:28:45.829 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-26T01:28:45.829 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-26T01:28:45.829 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-26T01:28:45.829 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-26T01:28:45.829 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-26T01:28:45.829 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-26T01:28:45.829 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-26T01:28:45.829 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-26T01:28:45.829 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-26T01:28:45.829 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-26T01:28:45.829 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-26T01:28:45.830 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-26T01:28:45.830 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-26T01:28:45.830 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-26T01:28:45.830 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-26T01:28:45.830 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-26T01:28:45.830 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-26T01:28:45.830 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-26T01:28:45.830 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-26T01:28:45.830 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-26T01:28:45.830 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-26T01:28:45.830 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-26T01:28:45.830 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-26T01:28:45.830 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-26T01:28:45.831 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-26T01:28:45.831 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-26T01:28:45.831 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-26T01:28:45.831 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-26T01:28:45.831 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-26T01:28:45.831 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-26T01:28:45.831 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-26T01:28:45.831 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-26T01:28:45.831 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-26T01:28:45.831 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-26T01:28:45.831 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-26T01:28:45.831 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-26T01:28:45.831 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-26T01:28:45.832 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-26T01:28:45.832 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-26T01:28:45.832 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-26T01:28:45.832 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-26T01:28:45.832 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-26T01:28:45.832 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-26T01:28:45.832 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-26T01:28:45.832 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-26T01:28:45.832 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-26T01:28:45.832 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-26T01:28:45.832 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-26T01:28:45.832 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-26T01:28:45.832 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-26T01:28:45.833 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-26T01:28:45.833 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-26T01:28:45.833 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-26T01:28:45.833 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-26T01:28:45.833 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-26T01:28:45.833 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-26T01:28:45.833 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-26T01:28:45.833 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-26T01:28:45.833 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-26T01:28:45.833 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-26T01:28:45.833 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-26T01:28:45.833 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-26T01:28:45.833 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-26T01:28:45.833 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-26T01:28:45.834 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-26T01:28:45.834 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-26T01:28:45.834 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-26T01:28:45.834 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-26T01:28:45.834 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-26T01:28:45.834 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-26T01:28:45.834 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-26T01:28:45.834 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-26T01:28:45.834 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-26T01:28:45.834 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-26T01:28:45.834 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-26T01:28:45.834 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-26T01:28:45.834 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-26T01:28:45.835 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-26T01:28:45.835 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-26T01:28:45.835 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-26T01:28:45.835 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-26T01:28:45.835 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-26T01:28:45.835 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-26T01:28:45.835 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-26T01:28:45.835 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-26T01:28:45.835 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-26T01:28:45.835 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-26T01:28:45.835 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-26T01:28:45.835 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-26T01:28:45.835 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-26T01:28:45.836 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-26T01:28:45.836 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-26T01:28:45.836 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-26T01:28:45.836 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-26T01:28:45.836 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-26T01:28:45.836 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-26T01:28:45.836 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-26T01:28:45.836 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-26T01:28:45.836 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-26T01:28:45.836 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-26T01:28:45.836 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-26T01:28:45.836 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-26T01:28:45.836 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-26T01:28:45.836 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-26T01:28:45.837 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-26T01:28:45.837 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-26T01:28:45.837 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-26T01:28:45.837 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-26T01:28:45.837 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-26T01:28:45.837 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-26T01:28:45.837 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-26T01:28:45.837 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-26T01:28:45.837 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-26T01:28:45.837 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-26T01:28:45.837 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-26T01:28:45.837 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-26T01:28:45.837 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-26T01:28:45.837 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-26T01:28:45.838 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-26T01:28:45.838 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-26T01:28:45.838 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-26T01:28:45.838 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-26T01:28:45.838 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-26T01:28:45.838 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-26T01:28:45.838 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-26T01:28:45.838 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-26T01:28:45.838 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-26T01:28:45.838 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-26T01:28:45.838 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-26T01:28:45.838 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-26T01:28:45.838 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-26T01:28:45.839 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-26T01:28:45.839 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-26T01:28:45.839 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-26T01:28:45.839 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-26T01:28:45.839 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-26T01:28:45.839 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-26T01:28:45.839 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-26T01:28:45.839 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-26T01:28:45.839 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-26T01:28:45.839 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-26T01:28:45.839 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-26T01:28:45.839 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-26T01:28:45.839 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-26T01:28:45.840 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-26T01:28:45.840 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-26T01:28:45.840 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-26T01:28:45.840 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-26T01:28:45.840 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-26T01:28:45.840 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-26T01:28:45.840 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-26T01:28:45.840 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-26T01:28:45.840 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-26T01:28:45.840 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-26T01:28:45.840 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-26T01:28:45.840 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-26T01:28:45.840 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-26T01:28:45.840 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-26T01:28:45.841 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-26T01:28:45.841 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-26T01:28:45.841 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-26T01:28:45.841 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-26T01:28:45.841 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-26T01:28:45.841 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-26T01:28:45.841 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-26T01:28:45.841 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-26T01:28:45.841 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-26T01:28:45.841 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-26T01:28:45.841 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-26T01:28:45.841 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-26T01:28:45.841 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-26T01:28:45.841 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-26T01:28:45.842 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-26T01:28:45.842 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-26T01:28:45.842 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-26T01:28:45.842 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-26T01:28:45.842 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-26T01:28:45.842 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-26T01:28:45.842 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-26T01:28:45.842 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-26T01:28:45.842 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-26T01:28:45.842 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-26T01:28:45.842 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-26T01:28:45.842 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-26T01:28:45.842 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-26T01:28:45.843 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-26T01:28:45.843 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-26T01:28:45.843 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-26T01:28:45.843 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-26T01:28:45.843 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-26T01:28:45.843 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-26T01:28:45.843 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-26T01:28:45.843 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-26T01:28:45.843 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-26T01:28:45.843 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-26T01:28:45.843 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-26T01:28:45.843 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-26T01:28:45.843 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-26T01:28:45.843 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-26T01:28:45.844 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-26T01:28:45.844 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-26T01:28:45.844 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-26T01:28:45.844 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-26T01:28:45.844 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-26T01:28:45.844 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-26T01:28:45.844 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-26T01:28:45.844 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-26T01:28:45.844 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-26T01:28:45.844 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-26T01:28:45.844 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-26T01:28:45.844 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-26T01:28:45.844 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-26T01:28:45.844 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-26T01:28:45.845 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-26T01:28:45.845 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-26T01:28:45.845 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-26T01:28:45.845 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-26T01:28:45.845 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-26T01:28:45.845 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-26T01:28:45.845 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-26T01:28:45.845 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-26T01:28:45.845 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-26T01:28:45.845 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-26T01:28:45.845 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-26T01:28:45.845 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-26T01:28:45.845 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-26T01:28:45.846 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-26T01:28:45.846 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-26T01:28:45.846 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-26T01:28:45.846 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-26T01:28:45.846 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-26T01:28:45.846 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-26T01:28:45.846 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-26T01:28:45.846 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-26T01:28:45.846 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-26T01:28:45.846 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-26T01:28:45.846 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-26T01:28:45.846 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-26T01:28:45.846 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-26T01:28:45.847 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-26T01:28:45.847 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-26T01:28:45.847 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-26T01:28:45.847 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-26T01:28:45.847 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-26T01:28:45.847 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-26T01:28:45.847 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-26T01:28:45.847 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-26T01:28:45.847 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-26T01:28:45.847 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-26T01:28:45.847 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-26T01:28:45.847 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-26T01:28:45.847 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-26T01:28:45.847 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-26T01:28:45.848 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-26T01:28:45.848 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-26T01:28:45.848 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-26T01:28:45.848 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-26T01:28:45.848 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-26T01:28:45.848 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-26T01:28:45.848 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-26T01:28:45.848 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-26T01:28:45.848 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-26T01:28:45.848 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-26T01:28:45.848 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-26T01:28:45.848 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-26T01:28:45.848 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-26T01:28:45.848 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-26T01:28:45.849 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-26T01:28:45.849 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-26T01:28:45.849 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-26T01:28:45.849 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-26T01:28:45.849 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-26T01:28:45.849 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-26T01:28:45.849 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-26T01:28:45.849 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-26T01:28:45.849 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-26T01:28:45.849 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-26T01:28:45.849 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-26T01:28:45.849 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-26T01:28:45.849 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-26T01:28:45.850 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-26T01:28:45.850 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-26T01:28:45.850 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-26T01:28:45.850 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-26T01:28:45.850 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-26T01:28:45.850 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-26T01:28:45.850 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-26T01:28:45.850 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-26T01:28:45.850 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-26T01:28:45.850 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-26T01:28:45.850 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-26T01:28:45.850 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-26T01:28:45.850 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-26T01:28:45.851 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-26T01:28:45.851 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-26T01:28:45.851 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-26T01:28:45.851 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-26T01:28:45.851 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-26T01:28:45.851 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-26T01:28:45.851 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-26T01:28:45.851 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-26T01:28:45.851 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-26T01:28:45.851 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-26T01:28:45.851 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-26T01:28:45.851 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-26T01:28:45.851 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-26T01:28:45.852 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-26T01:28:45.852 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-26T01:28:45.852 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-26T01:28:45.852 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-26T01:28:45.852 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-26T01:28:45.852 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-26T01:28:45.852 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-26T01:28:45.852 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-26T01:28:45.852 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-26T01:28:45.852 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-26T01:28:45.852 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-26T01:28:45.852 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-26T01:28:45.852 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-26T01:28:45.852 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-26T01:28:45.853 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-26T01:28:45.853 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-26T01:28:45.853 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-26T01:28:45.853 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-26T01:28:45.853 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-26T01:28:45.853 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-26T01:28:45.853 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-26T01:28:45.853 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-26T01:28:45.853 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-26T01:28:45.853 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-26T01:28:45.853 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-26T01:28:45.853 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-26T01:28:45.853 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-26T01:28:45.853 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-26T01:28:45.854 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-26T01:28:45.854 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-26T01:28:45.854 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-26T01:28:45.854 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-26T01:28:45.854 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-26T01:28:45.854 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-26T01:28:45.854 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-26T01:28:45.854 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-26T01:28:45.854 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-26T01:28:45.854 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-26T01:28:45.854 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-26T01:28:45.854 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-26T01:28:45.854 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-26T01:28:45.854 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-26T01:28:45.855 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-26T01:28:45.855 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-26T01:28:45.855 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-26T01:28:45.855 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-26T01:28:45.855 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-26T01:28:45.855 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-26T01:28:45.855 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-26T01:28:45.855 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-26T01:28:45.855 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-26T01:28:45.855 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-26T01:28:45.855 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-26T01:28:45.855 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-26T01:28:45.855 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-26T01:28:45.855 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-26T01:28:45.856 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-26T01:28:45.856 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-26T01:28:45.856 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-26T01:28:45.856 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-26T01:28:45.856 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-26T01:28:45.856 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-26T01:28:45.856 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-26T01:28:45.856 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-26T01:28:45.856 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-26T01:28:45.856 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-26T01:28:45.856 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-26T01:28:45.856 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-26T01:28:45.856 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-26T01:28:45.857 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-26T01:28:45.857 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-26T01:28:45.857 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-26T01:28:45.857 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-26T01:28:45.857 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-26T01:28:45.857 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-26T01:28:45.857 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-26T01:28:45.857 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-26T01:28:45.857 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-26T01:28:45.857 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-26T01:28:45.857 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-26T01:28:45.857 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-26T01:28:45.857 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-26T01:28:45.857 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-26T01:28:45.858 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-26T01:28:45.858 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-26T01:28:45.858 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-26T01:28:45.858 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-26T01:28:45.858 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-26T01:28:45.858 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-26T01:28:45.858 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-26T01:28:45.858 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-26T01:28:45.858 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-26T01:28:45.858 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-26T01:28:45.858 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-26T01:28:45.858 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-26T01:28:45.858 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-26T01:28:45.859 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-26T01:28:45.859 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-26T01:28:45.859 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-26T01:28:45.859 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-26T01:28:45.859 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-26T01:28:45.859 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-26T01:28:45.859 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-26T01:28:45.859 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-26T01:28:45.859 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-26T01:28:45.859 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-26T01:28:45.859 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-26T01:28:45.859 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-26T01:28:45.859 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-26T01:28:45.859 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-26T01:28:45.860 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-26T01:28:45.860 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-26T01:28:45.860 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-26T01:28:45.860 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-26T01:28:45.860 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-26T01:28:45.860 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-26T01:28:45.860 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-26T01:28:45.860 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-26T01:28:45.860 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-26T01:28:45.860 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-26T01:28:45.860 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-26T01:28:45.860 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-26T01:28:45.860 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-26T01:28:45.860 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-26T01:28:45.861 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-26T01:28:45.861 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-26T01:28:45.861 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-26T01:28:45.861 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-26T01:28:45.861 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-26T01:28:45.861 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-26T01:28:45.861 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-26T01:28:45.861 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-26T01:28:45.861 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-26T01:28:45.861 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-26T01:28:45.861 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-26T01:28:45.861 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-26T01:28:45.861 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-26T01:28:45.862 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-26T01:28:45.862 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-26T01:28:45.862 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-26T01:28:45.862 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-26T01:28:45.862 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-26T01:28:45.862 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-26T01:28:45.862 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-26T01:28:45.862 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-26T01:28:45.862 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-26T01:28:45.862 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-26T01:28:45.862 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-26T01:28:45.862 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-26T01:28:45.862 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-26T01:28:45.863 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-26T01:28:45.863 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-26T01:28:45.863 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-26T01:28:45.863 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-26T01:28:45.863 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-26T01:28:45.863 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-26T01:28:45.863 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-26T01:28:45.863 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-26T01:28:45.863 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-26T01:28:45.863 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-26T01:28:45.863 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-26T01:28:45.863 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-26T01:28:45.863 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-26T01:28:45.863 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-26T01:28:45.863 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-26T01:28:45.864 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-26T01:28:45.864 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-26T01:28:45.864 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-26T01:28:45.864 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-26T01:28:45.864 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-26T01:28:45.864 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-26T01:28:45.864 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-26T01:28:45.864 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-26T01:28:45.864 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-26T01:28:45.864 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-26T01:28:45.864 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-26T01:28:45.864 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-26T01:28:45.865 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-26T01:28:45.865 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-26T01:28:45.865 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-26T01:28:45.865 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-26T01:28:45.865 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-26T01:28:45.865 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-26T01:28:45.865 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-26T01:28:45.865 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-26T01:28:45.865 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-26T01:28:45.865 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-26T01:28:45.865 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-26T01:28:45.865 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-26T01:28:45.865 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-26T01:28:45.865 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-26T01:28:45.866 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-26T01:28:45.866 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-26T01:28:45.866 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-26T01:28:45.866 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-26T01:28:45.866 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-26T01:28:45.866 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-26T01:28:45.866 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-26T01:28:45.866 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-26T01:28:45.866 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-26T01:28:45.866 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-26T01:28:45.866 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-26T01:28:45.866 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-26T01:28:45.866 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-26T01:28:45.867 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-26T01:28:45.867 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-26T01:28:45.867 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-26T01:28:45.867 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-26T01:28:45.867 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-26T01:28:45.867 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-26T01:28:45.867 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-26T01:28:45.867 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-26T01:28:45.867 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-26T01:28:45.867 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-26T01:28:45.867 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-26T01:28:45.867 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-26T01:28:45.867 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-26T01:28:45.867 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-26T01:28:45.868 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-26T01:28:45.868 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-26T01:28:45.868 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-26T01:28:45.868 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-26T01:28:45.868 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-26T01:28:45.868 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-26T01:28:45.868 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-26T01:28:45.868 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-26T01:28:45.868 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-26T01:28:45.868 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-26T01:28:45.868 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-26T01:28:45.868 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-26T01:28:45.868 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-26T01:28:45.868 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-26T01:28:45.869 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-26T01:28:45.869 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-26T01:28:45.869 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-26T01:28:45.869 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-26T01:28:45.869 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-26T01:28:45.869 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-26T01:28:45.869 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-26T01:28:45.869 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-26T01:28:45.869 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-26T01:28:45.869 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-26T01:28:45.869 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-26T01:28:45.869 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-26T01:28:45.869 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-26T01:28:45.870 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-26T01:28:45.870 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-26T01:28:45.870 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-26T01:28:45.870 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-26T01:28:45.870 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-26T01:28:45.870 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-26T01:28:45.870 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-26T01:28:45.870 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-26T01:28:45.870 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-26T01:28:45.870 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-26T01:28:45.870 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-26T01:28:45.870 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-26T01:28:45.870 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-26T01:28:45.870 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-26T01:28:45.871 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-26T01:28:45.871 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-26T01:28:45.871 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-26T01:28:45.871 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-26T01:28:45.871 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-26T01:28:45.871 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-26T01:28:45.871 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-26T01:28:45.871 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-26T01:28:45.871 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-26T01:28:45.871 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-26T01:28:45.871 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-26T01:28:45.871 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-26T01:28:45.871 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-26T01:28:45.871 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-26T01:28:45.872 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-26T01:28:45.872 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-26T01:28:45.872 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-26T01:28:45.872 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-26T01:28:45.872 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-26T01:28:45.872 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-26T01:28:45.872 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-26T01:28:45.872 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-26T01:28:45.872 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-26T01:28:45.872 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-26T01:28:45.872 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-26T01:28:45.872 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-26T01:28:45.872 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-26T01:28:45.872 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-26T01:28:45.872 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-26T01:28:45.873 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-26T01:28:45.873 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-26T01:28:45.873 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-26T01:28:45.873 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-26T01:28:45.873 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-26T01:28:45.873 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-26T01:28:45.873 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-26T01:28:45.873 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-26T01:28:45.873 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-26T01:28:45.873 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-26T01:28:45.873 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-26T01:28:45.873 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-26T01:28:45.873 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-26T01:28:45.873 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-26T01:28:45.874 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-26T01:28:45.874 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-26T01:28:45.874 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-26T01:28:45.874 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-26T01:28:45.874 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-26T01:28:45.874 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-26T01:28:45.874 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-26T01:28:45.874 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-26T01:28:45.874 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-26T01:28:45.874 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-26T01:28:45.874 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-26T01:28:45.878 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-26T01:28:45.879 DEBUG:teuthology.orchestra.run.smithi151:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-26T01:28:45.902 INFO:teuthology.orchestra.run.smithi151.stdout:postinstall scriptlet (using /bin/sh): 2024-04-26T01:28:45.902 INFO:teuthology.orchestra.run.smithi151.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-26T01:28:45.902 INFO:teuthology.orchestra.run.smithi151.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-26T01:28:45.902 INFO:teuthology.orchestra.run.smithi151.stdout:fi 2024-04-26T01:28:45.902 INFO:teuthology.orchestra.run.smithi151.stdout:for file in vmlinuz System.map config; do 2024-04-26T01:28:45.903 INFO:teuthology.orchestra.run.smithi151.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-26T01:28:45.903 INFO:teuthology.orchestra.run.smithi151.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-26T01:28:45.903 INFO:teuthology.orchestra.run.smithi151.stdout: fi 2024-04-26T01:28:45.903 INFO:teuthology.orchestra.run.smithi151.stdout:done 2024-04-26T01:28:45.903 INFO:teuthology.orchestra.run.smithi151.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-26T01:28:45.903 INFO:teuthology.orchestra.run.smithi151.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-26T01:28:45.903 INFO:teuthology.orchestra.run.smithi151.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-26T01:28:45.903 INFO:teuthology.orchestra.run.smithi151.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-26T01:28:45.903 INFO:teuthology.orchestra.run.smithi151.stdout:fi 2024-04-26T01:28:45.903 INFO:teuthology.orchestra.run.smithi151.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-26T01:28:45.903 INFO:teuthology.orchestra.run.smithi151.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-26T01:28:45.903 INFO:teuthology.orchestra.run.smithi151.stdout:fi 2024-04-26T01:28:45.904 INFO:teuthology.orchestra.run.smithi151.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-26T01:28:45.904 INFO:teuthology.orchestra.run.smithi151.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-26T01:28:45.904 INFO:teuthology.orchestra.run.smithi151.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-26T01:28:45.904 INFO:teuthology.orchestra.run.smithi151.stdout:fi 2024-04-26T01:28:45.905 DEBUG:teuthology.orchestra.run.smithi151:> sudo rpm -qi grub2-tools 2024-04-26T01:28:45.977 INFO:teuthology.orchestra.run.smithi151.stdout:Name : grub2-tools 2024-04-26T01:28:45.977 INFO:teuthology.orchestra.run.smithi151.stdout:Epoch : 1 2024-04-26T01:28:45.977 INFO:teuthology.orchestra.run.smithi151.stdout:Version : 2.06 2024-04-26T01:28:45.977 INFO:teuthology.orchestra.run.smithi151.stdout:Release : 68.el9 2024-04-26T01:28:45.977 INFO:teuthology.orchestra.run.smithi151.stdout:Architecture: x86_64 2024-04-26T01:28:45.977 INFO:teuthology.orchestra.run.smithi151.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-26T01:28:45.977 INFO:teuthology.orchestra.run.smithi151.stdout:Group : Unspecified 2024-04-26T01:28:45.977 INFO:teuthology.orchestra.run.smithi151.stdout:Size : 8264254 2024-04-26T01:28:45.978 INFO:teuthology.orchestra.run.smithi151.stdout:License : GPLv3+ 2024-04-26T01:28:45.978 INFO:teuthology.orchestra.run.smithi151.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-26T01:28:45.978 INFO:teuthology.orchestra.run.smithi151.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-26T01:28:45.978 INFO:teuthology.orchestra.run.smithi151.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-26T01:28:45.978 INFO:teuthology.orchestra.run.smithi151.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-26T01:28:45.978 INFO:teuthology.orchestra.run.smithi151.stdout:Packager : builder@centos.org 2024-04-26T01:28:45.978 INFO:teuthology.orchestra.run.smithi151.stdout:Vendor : CentOS 2024-04-26T01:28:45.978 INFO:teuthology.orchestra.run.smithi151.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-26T01:28:45.978 INFO:teuthology.orchestra.run.smithi151.stdout:Summary : Support tools for GRUB. 2024-04-26T01:28:45.978 INFO:teuthology.orchestra.run.smithi151.stdout:Description : 2024-04-26T01:28:45.978 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:28:45.978 INFO:teuthology.orchestra.run.smithi151.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-26T01:28:45.978 INFO:teuthology.orchestra.run.smithi151.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-26T01:28:45.979 INFO:teuthology.orchestra.run.smithi151.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-26T01:28:45.979 INFO:teuthology.orchestra.run.smithi151.stdout:hardware devices. 2024-04-26T01:28:45.979 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:28:45.979 INFO:teuthology.orchestra.run.smithi151.stdout:This subpackage provides tools for support of all platforms. 2024-04-26T01:28:45.980 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-26T01:28:45.980 INFO:teuthology.task.kernel:Updating grub on smithi151 to boot 6.9.0-rc3-g806ec36b2112 2024-04-26T01:28:45.980 DEBUG:teuthology.orchestra.run.smithi151:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-26T01:28:46.503 INFO:teuthology.orchestra.run.smithi151.stderr:Generating grub configuration file ... 2024-04-26T01:28:47.914 INFO:teuthology.orchestra.run.smithi151.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-26T01:28:47.939 INFO:teuthology.orchestra.run.smithi151.stderr:done 2024-04-26T01:28:47.941 DEBUG:teuthology.orchestra.run.smithi151:> mktemp 2024-04-26T01:28:47.956 INFO:teuthology.orchestra.run.smithi151.stdout:/tmp/tmp.iQpoJrNHP2 2024-04-26T01:28:47.956 DEBUG:teuthology.orchestra.run.smithi151:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.iQpoJrNHP2 2024-04-26T01:28:48.022 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod 0666 /tmp/tmp.iQpoJrNHP2 2024-04-26T01:28:48.161 DEBUG:teuthology.orchestra.remote:smithi151:/tmp/tmp.iQpoJrNHP2 is 6KB 2024-04-26T01:28:48.210 DEBUG:teuthology.orchestra.run.smithi151:> rm -fr /tmp/tmp.iQpoJrNHP2 2024-04-26T01:28:48.224 DEBUG:teuthology.orchestra.run.smithi151:> sudo /bin/ls /boot/loader/entries || true 2024-04-26T01:28:48.286 INFO:teuthology.orchestra.run.smithi151.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-26T01:28:48.286 INFO:teuthology.orchestra.run.smithi151.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-26T01:28:48.286 INFO:teuthology.orchestra.run.smithi151.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-26T01:28:48.286 INFO:teuthology.orchestra.run.smithi151.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-26T01:28:48.287 DEBUG:teuthology.orchestra.run.smithi151:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-26T01:28:48.439 DEBUG:teuthology.orchestra.run.smithi151:> sudo shutdown -r now 2024-04-26T01:29:18.442 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-26T01:29:18.443 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-04-26T01:29:18.444 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T01:29:52.926 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.151 2024-04-26T01:30:01.928 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-04-26T01:30:01.929 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T01:31:01.932 DEBUG:teuthology.orchestra.remote:timed out 2024-04-26T01:31:13.933 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-04-26T01:31:13.934 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T01:31:14.317 DEBUG:teuthology.orchestra.run.smithi151:> true 2024-04-26T01:31:14.966 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-04-26T01:31:14.967 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-g806ec36b2112"... 2024-04-26T01:31:14.967 DEBUG:teuthology.orchestra.run.smithi151:> uname -r 2024-04-26T01:31:15.020 INFO:teuthology.orchestra.run.smithi151.stdout:6.9.0-rc3-g806ec36b2112 2024-04-26T01:31:15.020 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-26T01:31:15.020 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-26T01:31:15.021 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-26T01:31:16.022 DEBUG:teuthology.parallel:result is None 2024-04-26T01:31:16.022 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-26T01:31:16.031 INFO:teuthology.task.internal:Creating test directory... 2024-04-26T01:31:16.031 DEBUG:teuthology.orchestra.run.smithi151:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T01:31:16.038 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T01:31:16.041 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T01:31:16.088 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-26T01:31:16.094 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-26T01:31:16.134 INFO:teuthology.task.internal:Creating archive directory... 2024-04-26T01:31:16.134 DEBUG:teuthology.orchestra.run.smithi151:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T01:31:16.140 DEBUG:teuthology.orchestra.run.smithi165:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T01:31:16.143 DEBUG:teuthology.orchestra.run.smithi172:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T01:31:16.239 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-26T01:31:16.248 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-26T01:31:16.249 DEBUG:teuthology.orchestra.run.smithi165:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T01:31:16.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T01:31:16.265 DEBUG:teuthology.orchestra.run.smithi172:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T01:31:16.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T01:31:16.282 DEBUG:teuthology.orchestra.run.smithi151:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T01:31:16.289 DEBUG:teuthology.orchestra.run.smithi165:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T01:31:16.308 DEBUG:teuthology.orchestra.run.smithi172:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T01:31:16.365 INFO:teuthology.orchestra.run.smithi165.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T01:31:16.375 INFO:teuthology.orchestra.run.smithi172.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T01:31:16.381 INFO:teuthology.orchestra.run.smithi165.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T01:31:16.395 INFO:teuthology.orchestra.run.smithi172.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T01:31:16.399 INFO:teuthology.orchestra.run.smithi151.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T01:31:16.448 INFO:teuthology.orchestra.run.smithi151.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T01:31:16.453 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-26T01:31:16.461 INFO:teuthology.task.internal:Configuring sudo... 2024-04-26T01:31:16.461 DEBUG:teuthology.orchestra.run.smithi151:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T01:31:16.466 DEBUG:teuthology.orchestra.run.smithi165:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T01:31:16.468 DEBUG:teuthology.orchestra.run.smithi172:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T01:31:16.547 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-26T01:31:16.556 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-26T01:31:16.556 DEBUG:teuthology.orchestra.run.smithi151:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T01:31:16.563 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T01:31:16.566 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T01:31:16.609 DEBUG:teuthology.orchestra.run.smithi151:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T01:31:16.693 DEBUG:teuthology.orchestra.run.smithi151:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T01:31:16.785 DEBUG:teuthology.orchestra.run.smithi151:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T01:31:16.866 DEBUG:teuthology.orchestra.run.smithi151:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T01:31:16.953 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-26T01:31:16.954 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T01:31:17.037 DEBUG:teuthology.orchestra.run.smithi165:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T01:31:17.082 DEBUG:teuthology.orchestra.run.smithi165:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T01:31:17.135 DEBUG:teuthology.orchestra.run.smithi165:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T01:31:17.217 DEBUG:teuthology.orchestra.run.smithi165:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T01:31:17.260 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-26T01:31:17.260 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T01:31:17.326 DEBUG:teuthology.orchestra.run.smithi172:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T01:31:17.368 DEBUG:teuthology.orchestra.run.smithi172:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T01:31:17.429 DEBUG:teuthology.orchestra.run.smithi172:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T01:31:17.509 DEBUG:teuthology.orchestra.run.smithi172:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T01:31:17.554 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-26T01:31:17.554 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T01:31:17.619 DEBUG:teuthology.orchestra.run.smithi151:> sudo service rsyslog restart 2024-04-26T01:31:17.626 DEBUG:teuthology.orchestra.run.smithi165:> sudo service rsyslog restart 2024-04-26T01:31:17.628 DEBUG:teuthology.orchestra.run.smithi172:> sudo service rsyslog restart 2024-04-26T01:31:17.669 INFO:teuthology.orchestra.run.smithi165.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T01:31:17.701 INFO:teuthology.orchestra.run.smithi172.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T01:31:17.736 INFO:teuthology.orchestra.run.smithi151.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T01:31:18.201 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-26T01:31:18.210 INFO:teuthology.task.internal:Starting timer... 2024-04-26T01:31:18.210 INFO:teuthology.run_tasks:Running task pcp... 2024-04-26T01:31:18.231 INFO:teuthology.run_tasks:Running task selinux... 2024-04-26T01:31:18.245 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-26T01:31:18.245 DEBUG:teuthology.orchestra.run.smithi151:> sudo service auditd rotate 2024-04-26T01:31:18.347 INFO:teuthology.orchestra.run.smithi151.stdout:Rotating logs: 2024-04-26T01:31:18.352 DEBUG:teuthology.orchestra.run.smithi165:> sudo service auditd rotate 2024-04-26T01:31:18.422 INFO:teuthology.orchestra.run.smithi165.stdout:Rotating logs: 2024-04-26T01:31:18.425 DEBUG:teuthology.orchestra.run.smithi172:> sudo service auditd rotate 2024-04-26T01:31:18.485 INFO:teuthology.orchestra.run.smithi172.stdout:Rotating logs: 2024-04-26T01:31:18.486 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-26T01:31:18.486 DEBUG:teuthology.orchestra.run.smithi151:> /usr/sbin/getenforce 2024-04-26T01:31:18.550 INFO:teuthology.orchestra.run.smithi151.stdout:Permissive 2024-04-26T01:31:18.550 DEBUG:teuthology.orchestra.run.smithi165:> /usr/sbin/getenforce 2024-04-26T01:31:18.573 INFO:teuthology.orchestra.run.smithi165.stdout:Permissive 2024-04-26T01:31:18.573 DEBUG:teuthology.orchestra.run.smithi172:> /usr/sbin/getenforce 2024-04-26T01:31:18.594 INFO:teuthology.orchestra.run.smithi172.stdout:Permissive 2024-04-26T01:31:18.594 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi151.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi165.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi172.front.sepia.ceph.com': 'permissive'} 2024-04-26T01:31:18.594 DEBUG:teuthology.orchestra.run.smithi151:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T01:31:18.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T01:31:18.680 DEBUG:teuthology.orchestra.run.smithi165:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T01:31:18.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T01:31:18.707 DEBUG:teuthology.orchestra.run.smithi172:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T01:31:18.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T01:31:18.734 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-26T01:31:18.734 DEBUG:teuthology.orchestra.run.smithi151:> sudo /usr/sbin/setenforce permissive 2024-04-26T01:31:18.806 DEBUG:teuthology.orchestra.run.smithi165:> sudo /usr/sbin/setenforce permissive 2024-04-26T01:31:18.832 DEBUG:teuthology.orchestra.run.smithi172:> sudo /usr/sbin/setenforce permissive 2024-04-26T01:31:18.858 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-26T01:31:18.868 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-26T01:31:18.875 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-26T01:31:19.023 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-26T01:31:19.043 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-26T01:31:19.044 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi151.front.sepia.ceph.com,smithi165.front.sepia.ceph.com,smithi172.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-26T01:36:22.262 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi151.front.sepia.ceph.com'), Remote(name='ubuntu@smithi165.front.sepia.ceph.com'), Remote(name='ubuntu@smithi172.front.sepia.ceph.com')] 2024-04-26T01:36:22.264 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-04-26T01:36:22.265 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T01:36:22.377 DEBUG:teuthology.orchestra.run.smithi151:> true 2024-04-26T01:36:22.526 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-04-26T01:36:22.527 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi165.front.sepia.ceph.com' 2024-04-26T01:36:22.527 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T01:36:22.602 DEBUG:teuthology.orchestra.run.smithi165:> true 2024-04-26T01:36:22.678 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi165.front.sepia.ceph.com' 2024-04-26T01:36:22.678 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi172.front.sepia.ceph.com' 2024-04-26T01:36:22.679 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T01:36:22.796 DEBUG:teuthology.orchestra.run.smithi172:> true 2024-04-26T01:36:22.875 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi172.front.sepia.ceph.com' 2024-04-26T01:36:22.875 INFO:teuthology.run_tasks:Running task clock... 2024-04-26T01:36:22.888 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-26T01:36:22.889 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T01:36:22.890 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T01:36:22.901 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T01:36:22.901 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T01:36:22.904 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T01:36:22.904 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T01:36:22.931 INFO:teuthology.orchestra.run.smithi165.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T01:36:22.945 INFO:teuthology.orchestra.run.smithi172.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T01:36:22.945 INFO:teuthology.orchestra.run.smithi165.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T01:36:22.959 INFO:teuthology.orchestra.run.smithi172.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T01:36:22.969 INFO:teuthology.orchestra.run.smithi165.stderr:sudo: ntpd: command not found 2024-04-26T01:36:22.980 INFO:teuthology.orchestra.run.smithi165.stdout:506 Cannot talk to daemon 2024-04-26T01:36:22.981 INFO:teuthology.orchestra.run.smithi151.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T01:36:22.983 INFO:teuthology.orchestra.run.smithi172.stderr:sudo: ntpd: command not found 2024-04-26T01:36:22.994 INFO:teuthology.orchestra.run.smithi172.stdout:506 Cannot talk to daemon 2024-04-26T01:36:22.995 INFO:teuthology.orchestra.run.smithi165.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T01:36:23.008 INFO:teuthology.orchestra.run.smithi165.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T01:36:23.008 INFO:teuthology.orchestra.run.smithi172.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T01:36:23.022 INFO:teuthology.orchestra.run.smithi172.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T01:36:23.022 INFO:teuthology.orchestra.run.smithi151.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T01:36:23.061 INFO:teuthology.orchestra.run.smithi165.stderr:bash: line 1: ntpq: command not found 2024-04-26T01:36:23.064 INFO:teuthology.orchestra.run.smithi165.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T01:36:23.064 INFO:teuthology.orchestra.run.smithi165.stdout:=============================================================================== 2024-04-26T01:36:23.064 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T01:36:23.064 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T01:36:23.065 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T01:36:23.065 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T01:36:23.074 INFO:teuthology.orchestra.run.smithi172.stderr:bash: line 1: ntpq: command not found 2024-04-26T01:36:23.077 INFO:teuthology.orchestra.run.smithi172.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T01:36:23.077 INFO:teuthology.orchestra.run.smithi172.stdout:=============================================================================== 2024-04-26T01:36:23.077 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T01:36:23.077 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T01:36:23.077 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T01:36:23.077 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T01:36:23.093 INFO:teuthology.orchestra.run.smithi151.stderr:sudo: ntpd: command not found 2024-04-26T01:36:23.125 INFO:teuthology.orchestra.run.smithi151.stdout:506 Cannot talk to daemon 2024-04-26T01:36:23.163 INFO:teuthology.orchestra.run.smithi151.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T01:36:23.201 INFO:teuthology.orchestra.run.smithi151.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T01:36:23.329 INFO:teuthology.orchestra.run.smithi151.stderr:bash: line 1: ntpq: command not found 2024-04-26T01:36:23.337 INFO:teuthology.orchestra.run.smithi151.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T01:36:23.337 INFO:teuthology.orchestra.run.smithi151.stdout:=============================================================================== 2024-04-26T01:36:23.337 INFO:teuthology.orchestra.run.smithi151.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T01:36:23.338 INFO:teuthology.orchestra.run.smithi151.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T01:36:23.338 INFO:teuthology.orchestra.run.smithi151.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T01:36:23.338 INFO:teuthology.orchestra.run.smithi151.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T01:36:23.339 INFO:teuthology.run_tasks:Running task install... 2024-04-26T01:36:23.348 DEBUG:teuthology.task.install:project ceph 2024-04-26T01:36:23.348 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f'}} 2024-04-26T01:36:23.348 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f'} 2024-04-26T01:36:23.348 INFO:teuthology.task.install:Using flavor: default 2024-04-26T01:36:23.354 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-26T01:36:23.354 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-26T01:36:23.354 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-26T01:36:23.355 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-26T01:36:23.357 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-26T01:36:23.357 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-26T01:36:23.358 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-26T01:36:23.358 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-26T01:36:23.518 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-26T01:36:23.518 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-26T01:36:23.521 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-26T01:36:23.521 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-26T01:36:23.582 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-26T01:36:23.582 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-26T01:36:23.630 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T01:36:23.630 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-26T01:36:23.631 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T01:36:23.658 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T01:36:23.659 DEBUG:teuthology.orchestra.run.smithi172:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T01:36:23.691 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T01:36:23.692 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-26T01:36:23.692 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T01:36:23.694 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T01:36:23.694 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-26T01:36:23.694 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T01:36:23.721 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T01:36:23.722 DEBUG:teuthology.orchestra.run.smithi165:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T01:36:23.725 DEBUG:teuthology.orchestra.run.smithi172:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T01:36:23.784 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T01:36:23.784 DEBUG:teuthology.orchestra.run.smithi151:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T01:36:23.788 DEBUG:teuthology.orchestra.run.smithi165:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T01:36:23.800 DEBUG:teuthology.orchestra.run.smithi172:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T01:36:23.863 DEBUG:teuthology.orchestra.run.smithi165:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T01:36:23.865 INFO:teuthology.orchestra.run.smithi172.stdout:check_obsoletes = 1 2024-04-26T01:36:23.866 DEBUG:teuthology.orchestra.run.smithi172:> sudo yum clean all 2024-04-26T01:36:23.910 DEBUG:teuthology.orchestra.run.smithi151:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T01:36:23.927 INFO:teuthology.orchestra.run.smithi165.stdout:check_obsoletes = 1 2024-04-26T01:36:23.929 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum clean all 2024-04-26T01:36:24.015 DEBUG:teuthology.orchestra.run.smithi151:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T01:36:24.093 INFO:teuthology.orchestra.run.smithi172.stdout:48 files removed 2024-04-26T01:36:24.097 INFO:teuthology.orchestra.run.smithi151.stdout:check_obsoletes = 1 2024-04-26T01:36:24.102 DEBUG:teuthology.orchestra.run.smithi151:> sudo yum clean all 2024-04-26T01:36:24.121 DEBUG:teuthology.orchestra.run.smithi172:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T01:36:24.155 INFO:teuthology.orchestra.run.smithi165.stdout:48 files removed 2024-04-26T01:36:24.183 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T01:36:24.428 INFO:teuthology.orchestra.run.smithi151.stdout:48 files removed 2024-04-26T01:36:24.467 DEBUG:teuthology.orchestra.run.smithi151:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T01:36:24.771 INFO:teuthology.orchestra.run.smithi172.stdout:ceph packages for x86_64 214 kB/s | 83 kB 00:00 2024-04-26T01:36:24.877 INFO:teuthology.orchestra.run.smithi165.stdout:ceph packages for x86_64 193 kB/s | 83 kB 00:00 2024-04-26T01:36:25.130 INFO:teuthology.orchestra.run.smithi172.stdout:ceph noarch packages 37 kB/s | 12 kB 00:00 2024-04-26T01:36:25.294 INFO:teuthology.orchestra.run.smithi165.stdout:ceph noarch packages 31 kB/s | 12 kB 00:00 2024-04-26T01:36:25.391 INFO:teuthology.orchestra.run.smithi151.stdout:ceph packages for x86_64 169 kB/s | 83 kB 00:00 2024-04-26T01:36:25.539 INFO:teuthology.orchestra.run.smithi172.stdout:ceph source packages 5.1 kB/s | 1.9 kB 00:00 2024-04-26T01:36:25.645 INFO:teuthology.orchestra.run.smithi165.stdout:ceph source packages 6.0 kB/s | 1.9 kB 00:00 2024-04-26T01:36:25.991 INFO:teuthology.orchestra.run.smithi151.stdout:ceph noarch packages 23 kB/s | 12 kB 00:00 2024-04-26T01:36:26.216 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-04-26T01:36:26.305 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.0 MB 00:00 2024-04-26T01:36:26.467 INFO:teuthology.orchestra.run.smithi151.stdout:ceph source packages 4.9 kB/s | 1.9 kB 00:00 2024-04-26T01:36:27.503 INFO:teuthology.orchestra.run.smithi151.stdout:CentOS Stream 9 - BaseOS 8.4 MB/s | 8.0 MB 00:00 2024-04-26T01:36:28.210 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 9 - AppStream 20 MB/s | 19 MB 00:00 2024-04-26T01:36:28.789 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - AppStream 13 MB/s | 19 MB 00:01 2024-04-26T01:36:29.921 INFO:teuthology.orchestra.run.smithi151.stdout:CentOS Stream 9 - AppStream 16 MB/s | 19 MB 00:01 2024-04-26T01:36:32.877 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 9 - CRB 5.7 MB/s | 6.2 MB 00:01 2024-04-26T01:36:33.065 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - CRB 8.2 MB/s | 6.2 MB 00:00 2024-04-26T01:36:34.283 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 9 - Extras packages 62 kB/s | 16 kB 00:00 2024-04-26T01:36:34.515 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - Extras packages 50 kB/s | 16 kB 00:00 2024-04-26T01:36:34.565 INFO:teuthology.orchestra.run.smithi151.stdout:CentOS Stream 9 - CRB 8.4 MB/s | 6.2 MB 00:00 2024-04-26T01:36:35.502 INFO:teuthology.orchestra.run.smithi165.stdout:Extra Packages for Enterprise Linux 24 MB/s | 21 MB 00:00 2024-04-26T01:36:36.695 INFO:teuthology.orchestra.run.smithi151.stdout:CentOS Stream 9 - Extras packages 21 kB/s | 16 kB 00:00 2024-04-26T01:36:37.685 INFO:teuthology.orchestra.run.smithi172.stdout:Extra Packages for Enterprise Linux 6.4 MB/s | 21 MB 00:03 2024-04-26T01:36:40.516 INFO:teuthology.orchestra.run.smithi165.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-04-26T01:36:42.734 INFO:teuthology.orchestra.run.smithi172.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-04-26T01:36:42.876 INFO:teuthology.orchestra.run.smithi165.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T01:36:42.877 INFO:teuthology.orchestra.run.smithi165.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T01:36:42.888 INFO:teuthology.orchestra.run.smithi165.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T01:36:42.889 INFO:teuthology.orchestra.run.smithi165.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T01:36:42.889 INFO:teuthology.orchestra.run.smithi165.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T01:36:42.890 INFO:teuthology.orchestra.run.smithi165.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T01:36:42.892 INFO:teuthology.orchestra.run.smithi165.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T01:36:42.894 INFO:teuthology.orchestra.run.smithi165.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T01:36:42.895 INFO:teuthology.orchestra.run.smithi165.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T01:36:42.956 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-04-26T01:36:42.974 INFO:teuthology.orchestra.run.smithi165.stdout:======================================================================================= 2024-04-26T01:36:42.974 INFO:teuthology.orchestra.run.smithi165.stdout: Package Arch Version Repository Size 2024-04-26T01:36:42.974 INFO:teuthology.orchestra.run.smithi165.stdout:======================================================================================= 2024-04-26T01:36:42.974 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2024-04-26T01:36:42.974 INFO:teuthology.orchestra.run.smithi165.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T01:36:42.974 INFO:teuthology.orchestra.run.smithi165.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-26T01:36:42.974 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-26T01:36:42.975 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-26T01:36:42.975 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-26T01:36:42.975 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-26T01:36:42.975 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-26T01:36:42.975 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-26T01:36:42.975 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-26T01:36:42.975 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-26T01:36:42.975 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-26T01:36:42.975 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-26T01:36:42.975 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-26T01:36:42.975 INFO:teuthology.orchestra.run.smithi165.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-26T01:36:42.975 INFO:teuthology.orchestra.run.smithi165.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-26T01:36:42.975 INFO:teuthology.orchestra.run.smithi165.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-26T01:36:42.976 INFO:teuthology.orchestra.run.smithi165.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-26T01:36:42.976 INFO:teuthology.orchestra.run.smithi165.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T01:36:42.976 INFO:teuthology.orchestra.run.smithi165.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T01:36:42.976 INFO:teuthology.orchestra.run.smithi165.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T01:36:42.976 INFO:teuthology.orchestra.run.smithi165.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-26T01:36:42.976 INFO:teuthology.orchestra.run.smithi165.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-26T01:36:42.976 INFO:teuthology.orchestra.run.smithi165.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-26T01:36:42.976 INFO:teuthology.orchestra.run.smithi165.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T01:36:42.976 INFO:teuthology.orchestra.run.smithi165.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T01:36:42.976 INFO:teuthology.orchestra.run.smithi165.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T01:36:42.976 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T01:36:42.976 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T01:36:42.977 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T01:36:42.977 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-26T01:36:42.977 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-26T01:36:42.977 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-26T01:36:42.977 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-26T01:36:42.977 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-26T01:36:42.977 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-26T01:36:42.977 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-26T01:36:42.977 INFO:teuthology.orchestra.run.smithi165.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T01:36:42.977 INFO:teuthology.orchestra.run.smithi165.stdout:Upgrading: 2024-04-26T01:36:42.977 INFO:teuthology.orchestra.run.smithi165.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-26T01:36:42.977 INFO:teuthology.orchestra.run.smithi165.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-26T01:36:42.977 INFO:teuthology.orchestra.run.smithi165.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T01:36:42.978 INFO:teuthology.orchestra.run.smithi165.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T01:36:42.978 INFO:teuthology.orchestra.run.smithi165.stdout:Installing dependencies: 2024-04-26T01:36:42.978 INFO:teuthology.orchestra.run.smithi165.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T01:36:42.978 INFO:teuthology.orchestra.run.smithi165.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T01:36:42.978 INFO:teuthology.orchestra.run.smithi165.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T01:36:42.978 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-26T01:36:42.978 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-26T01:36:42.978 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-26T01:36:42.978 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-26T01:36:42.978 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-26T01:36:42.978 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-26T01:36:42.978 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-26T01:36:42.979 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-26T01:36:42.979 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T01:36:42.979 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T01:36:42.979 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T01:36:42.979 INFO:teuthology.orchestra.run.smithi165.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T01:36:42.979 INFO:teuthology.orchestra.run.smithi165.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T01:36:42.979 INFO:teuthology.orchestra.run.smithi165.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T01:36:42.979 INFO:teuthology.orchestra.run.smithi165.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-26T01:36:42.979 INFO:teuthology.orchestra.run.smithi165.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-26T01:36:42.979 INFO:teuthology.orchestra.run.smithi165.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-26T01:36:42.979 INFO:teuthology.orchestra.run.smithi165.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T01:36:42.979 INFO:teuthology.orchestra.run.smithi165.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T01:36:42.979 INFO:teuthology.orchestra.run.smithi165.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T01:36:42.980 INFO:teuthology.orchestra.run.smithi165.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T01:36:42.980 INFO:teuthology.orchestra.run.smithi165.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T01:36:42.980 INFO:teuthology.orchestra.run.smithi165.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-26T01:36:42.980 INFO:teuthology.orchestra.run.smithi165.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T01:36:42.980 INFO:teuthology.orchestra.run.smithi165.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-26T01:36:42.980 INFO:teuthology.orchestra.run.smithi165.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T01:36:42.980 INFO:teuthology.orchestra.run.smithi165.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T01:36:42.980 INFO:teuthology.orchestra.run.smithi165.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T01:36:42.980 INFO:teuthology.orchestra.run.smithi165.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T01:36:42.980 INFO:teuthology.orchestra.run.smithi165.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T01:36:42.980 INFO:teuthology.orchestra.run.smithi165.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T01:36:42.980 INFO:teuthology.orchestra.run.smithi165.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T01:36:42.980 INFO:teuthology.orchestra.run.smithi165.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T01:36:42.981 INFO:teuthology.orchestra.run.smithi165.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-26T01:36:42.981 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T01:36:42.981 INFO:teuthology.orchestra.run.smithi165.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T01:36:42.981 INFO:teuthology.orchestra.run.smithi165.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T01:36:42.981 INFO:teuthology.orchestra.run.smithi165.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T01:36:42.981 INFO:teuthology.orchestra.run.smithi165.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T01:36:42.981 INFO:teuthology.orchestra.run.smithi165.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T01:36:42.981 INFO:teuthology.orchestra.run.smithi165.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T01:36:42.981 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T01:36:42.981 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-26T01:36:42.981 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-26T01:36:42.981 INFO:teuthology.orchestra.run.smithi165.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T01:36:42.981 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T01:36:42.982 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T01:36:42.982 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T01:36:42.982 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T01:36:42.982 INFO:teuthology.orchestra.run.smithi165.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T01:36:42.982 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T01:36:42.982 INFO:teuthology.orchestra.run.smithi165.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T01:36:42.982 INFO:teuthology.orchestra.run.smithi165.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T01:36:42.982 INFO:teuthology.orchestra.run.smithi165.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T01:36:42.982 INFO:teuthology.orchestra.run.smithi165.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T01:36:42.982 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T01:36:42.982 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T01:36:42.982 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T01:36:42.983 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T01:36:42.983 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T01:36:42.983 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T01:36:42.983 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T01:36:42.983 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T01:36:42.983 INFO:teuthology.orchestra.run.smithi165.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T01:36:42.983 INFO:teuthology.orchestra.run.smithi165.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T01:36:42.983 INFO:teuthology.orchestra.run.smithi165.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T01:36:42.983 INFO:teuthology.orchestra.run.smithi165.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T01:36:42.983 INFO:teuthology.orchestra.run.smithi165.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T01:36:42.983 INFO:teuthology.orchestra.run.smithi165.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T01:36:42.983 INFO:teuthology.orchestra.run.smithi165.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T01:36:42.983 INFO:teuthology.orchestra.run.smithi165.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T01:36:42.984 INFO:teuthology.orchestra.run.smithi165.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T01:36:42.984 INFO:teuthology.orchestra.run.smithi165.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T01:36:42.984 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T01:36:42.984 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T01:36:42.984 INFO:teuthology.orchestra.run.smithi165.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T01:36:42.984 INFO:teuthology.orchestra.run.smithi165.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T01:36:42.984 INFO:teuthology.orchestra.run.smithi165.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T01:36:42.984 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T01:36:42.984 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T01:36:42.984 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T01:36:42.985 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T01:36:42.985 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T01:36:42.985 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T01:36:42.985 INFO:teuthology.orchestra.run.smithi165.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T01:36:42.985 INFO:teuthology.orchestra.run.smithi165.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T01:36:42.985 INFO:teuthology.orchestra.run.smithi165.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T01:36:42.985 INFO:teuthology.orchestra.run.smithi165.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T01:36:42.985 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T01:36:42.985 INFO:teuthology.orchestra.run.smithi165.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T01:36:42.985 INFO:teuthology.orchestra.run.smithi165.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T01:36:42.985 INFO:teuthology.orchestra.run.smithi165.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T01:36:42.985 INFO:teuthology.orchestra.run.smithi165.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T01:36:42.985 INFO:teuthology.orchestra.run.smithi165.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T01:36:42.986 INFO:teuthology.orchestra.run.smithi165.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T01:36:42.986 INFO:teuthology.orchestra.run.smithi165.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T01:36:42.986 INFO:teuthology.orchestra.run.smithi165.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T01:36:42.986 INFO:teuthology.orchestra.run.smithi165.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T01:36:42.986 INFO:teuthology.orchestra.run.smithi165.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T01:36:42.986 INFO:teuthology.orchestra.run.smithi165.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T01:36:42.986 INFO:teuthology.orchestra.run.smithi165.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T01:36:42.986 INFO:teuthology.orchestra.run.smithi165.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T01:36:42.986 INFO:teuthology.orchestra.run.smithi165.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T01:36:42.986 INFO:teuthology.orchestra.run.smithi165.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T01:36:42.986 INFO:teuthology.orchestra.run.smithi165.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T01:36:42.986 INFO:teuthology.orchestra.run.smithi165.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T01:36:42.986 INFO:teuthology.orchestra.run.smithi165.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T01:36:42.987 INFO:teuthology.orchestra.run.smithi165.stdout:Installing weak dependencies: 2024-04-26T01:36:42.987 INFO:teuthology.orchestra.run.smithi165.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T01:36:42.987 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-26T01:36:42.987 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2024-04-26T01:36:42.987 INFO:teuthology.orchestra.run.smithi165.stdout:======================================================================================= 2024-04-26T01:36:42.987 INFO:teuthology.orchestra.run.smithi165.stdout:Install 148 Packages 2024-04-26T01:36:42.987 INFO:teuthology.orchestra.run.smithi165.stdout:Upgrade 4 Packages 2024-04-26T01:36:42.987 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-26T01:36:42.988 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 222 M 2024-04-26T01:36:42.988 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2024-04-26T01:36:43.893 INFO:teuthology.orchestra.run.smithi165.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 38 kB/s | 6.6 kB 00:00 2024-04-26T01:36:44.092 INFO:teuthology.orchestra.run.smithi165.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 4.4 MB/s | 890 kB 00:00 2024-04-26T01:36:44.168 INFO:teuthology.orchestra.run.smithi165.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 1.9 MB/s | 145 kB 00:00 2024-04-26T01:36:44.310 INFO:teuthology.orchestra.run.smithi165.stdout:(4/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 17 MB/s | 2.4 MB 00:00 2024-04-26T01:36:44.410 INFO:teuthology.orchestra.run.smithi165.stdout:(5/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 15 MB/s | 1.5 MB 00:00 2024-04-26T01:36:44.669 INFO:teuthology.orchestra.run.smithi165.stdout:(6/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 18 MB/s | 4.7 MB 00:00 2024-04-26T01:36:44.852 INFO:teuthology.orchestra.run.smithi172.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T01:36:44.852 INFO:teuthology.orchestra.run.smithi172.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T01:36:44.863 INFO:teuthology.orchestra.run.smithi172.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T01:36:44.864 INFO:teuthology.orchestra.run.smithi172.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T01:36:44.864 INFO:teuthology.orchestra.run.smithi172.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T01:36:44.865 INFO:teuthology.orchestra.run.smithi172.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T01:36:44.867 INFO:teuthology.orchestra.run.smithi172.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T01:36:44.869 INFO:teuthology.orchestra.run.smithi172.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T01:36:44.870 INFO:teuthology.orchestra.run.smithi172.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T01:36:44.930 INFO:teuthology.orchestra.run.smithi172.stdout:Dependencies resolved. 2024-04-26T01:36:44.944 INFO:teuthology.orchestra.run.smithi172.stdout:======================================================================================= 2024-04-26T01:36:44.944 INFO:teuthology.orchestra.run.smithi172.stdout: Package Arch Version Repository Size 2024-04-26T01:36:44.944 INFO:teuthology.orchestra.run.smithi172.stdout:======================================================================================= 2024-04-26T01:36:44.944 INFO:teuthology.orchestra.run.smithi172.stdout:Installing: 2024-04-26T01:36:44.945 INFO:teuthology.orchestra.run.smithi172.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T01:36:44.945 INFO:teuthology.orchestra.run.smithi172.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-26T01:36:44.945 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-26T01:36:44.945 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-26T01:36:44.945 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-26T01:36:44.945 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-26T01:36:44.945 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-26T01:36:44.945 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-26T01:36:44.945 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-26T01:36:44.945 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-26T01:36:44.945 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-26T01:36:44.945 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-26T01:36:44.945 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-26T01:36:44.946 INFO:teuthology.orchestra.run.smithi172.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-26T01:36:44.946 INFO:teuthology.orchestra.run.smithi172.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-26T01:36:44.946 INFO:teuthology.orchestra.run.smithi172.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-26T01:36:44.946 INFO:teuthology.orchestra.run.smithi172.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-26T01:36:44.946 INFO:teuthology.orchestra.run.smithi172.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T01:36:44.946 INFO:teuthology.orchestra.run.smithi172.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T01:36:44.946 INFO:teuthology.orchestra.run.smithi172.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T01:36:44.946 INFO:teuthology.orchestra.run.smithi172.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-26T01:36:44.946 INFO:teuthology.orchestra.run.smithi172.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-26T01:36:44.947 INFO:teuthology.orchestra.run.smithi172.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-26T01:36:44.947 INFO:teuthology.orchestra.run.smithi172.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T01:36:44.947 INFO:teuthology.orchestra.run.smithi172.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T01:36:44.947 INFO:teuthology.orchestra.run.smithi172.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T01:36:44.947 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T01:36:44.947 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T01:36:44.947 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T01:36:44.947 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-26T01:36:44.947 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-26T01:36:44.947 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-26T01:36:44.947 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-26T01:36:44.947 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-26T01:36:44.948 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-26T01:36:44.948 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-26T01:36:44.948 INFO:teuthology.orchestra.run.smithi172.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T01:36:44.948 INFO:teuthology.orchestra.run.smithi172.stdout:Upgrading: 2024-04-26T01:36:44.948 INFO:teuthology.orchestra.run.smithi172.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-26T01:36:44.948 INFO:teuthology.orchestra.run.smithi172.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-26T01:36:44.948 INFO:teuthology.orchestra.run.smithi172.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T01:36:44.948 INFO:teuthology.orchestra.run.smithi172.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T01:36:44.948 INFO:teuthology.orchestra.run.smithi172.stdout:Installing dependencies: 2024-04-26T01:36:44.948 INFO:teuthology.orchestra.run.smithi172.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T01:36:44.949 INFO:teuthology.orchestra.run.smithi172.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T01:36:44.949 INFO:teuthology.orchestra.run.smithi172.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T01:36:44.949 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-26T01:36:44.949 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-26T01:36:44.949 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-26T01:36:44.949 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-26T01:36:44.950 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-26T01:36:44.950 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-26T01:36:44.950 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-26T01:36:44.950 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-26T01:36:44.950 INFO:teuthology.orchestra.run.smithi172.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T01:36:44.950 INFO:teuthology.orchestra.run.smithi172.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T01:36:44.950 INFO:teuthology.orchestra.run.smithi172.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T01:36:44.950 INFO:teuthology.orchestra.run.smithi172.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T01:36:44.950 INFO:teuthology.orchestra.run.smithi172.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T01:36:44.950 INFO:teuthology.orchestra.run.smithi172.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T01:36:44.951 INFO:teuthology.orchestra.run.smithi172.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-26T01:36:44.951 INFO:teuthology.orchestra.run.smithi172.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-26T01:36:44.951 INFO:teuthology.orchestra.run.smithi172.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-26T01:36:44.951 INFO:teuthology.orchestra.run.smithi172.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T01:36:44.951 INFO:teuthology.orchestra.run.smithi172.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T01:36:44.951 INFO:teuthology.orchestra.run.smithi172.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T01:36:44.951 INFO:teuthology.orchestra.run.smithi172.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T01:36:44.951 INFO:teuthology.orchestra.run.smithi172.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T01:36:44.951 INFO:teuthology.orchestra.run.smithi172.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-26T01:36:44.951 INFO:teuthology.orchestra.run.smithi172.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T01:36:44.951 INFO:teuthology.orchestra.run.smithi172.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-26T01:36:44.951 INFO:teuthology.orchestra.run.smithi172.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T01:36:44.951 INFO:teuthology.orchestra.run.smithi172.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T01:36:44.951 INFO:teuthology.orchestra.run.smithi172.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T01:36:44.952 INFO:teuthology.orchestra.run.smithi172.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T01:36:44.952 INFO:teuthology.orchestra.run.smithi172.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T01:36:44.952 INFO:teuthology.orchestra.run.smithi172.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T01:36:44.952 INFO:teuthology.orchestra.run.smithi172.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T01:36:44.952 INFO:teuthology.orchestra.run.smithi172.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T01:36:44.952 INFO:teuthology.orchestra.run.smithi172.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-26T01:36:44.952 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T01:36:44.952 INFO:teuthology.orchestra.run.smithi172.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T01:36:44.952 INFO:teuthology.orchestra.run.smithi172.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T01:36:44.952 INFO:teuthology.orchestra.run.smithi172.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T01:36:44.953 INFO:teuthology.orchestra.run.smithi172.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T01:36:44.953 INFO:teuthology.orchestra.run.smithi172.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T01:36:44.953 INFO:teuthology.orchestra.run.smithi172.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T01:36:44.953 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T01:36:44.953 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-26T01:36:44.953 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-26T01:36:44.953 INFO:teuthology.orchestra.run.smithi172.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T01:36:44.953 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T01:36:44.953 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T01:36:44.953 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T01:36:44.953 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T01:36:44.953 INFO:teuthology.orchestra.run.smithi172.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T01:36:44.953 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T01:36:44.954 INFO:teuthology.orchestra.run.smithi172.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T01:36:44.954 INFO:teuthology.orchestra.run.smithi172.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T01:36:44.954 INFO:teuthology.orchestra.run.smithi172.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T01:36:44.954 INFO:teuthology.orchestra.run.smithi172.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T01:36:44.954 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T01:36:44.954 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T01:36:44.954 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T01:36:44.954 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T01:36:44.954 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T01:36:44.954 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T01:36:44.955 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T01:36:44.955 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T01:36:44.955 INFO:teuthology.orchestra.run.smithi172.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T01:36:44.955 INFO:teuthology.orchestra.run.smithi172.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T01:36:44.955 INFO:teuthology.orchestra.run.smithi172.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T01:36:44.955 INFO:teuthology.orchestra.run.smithi172.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T01:36:44.955 INFO:teuthology.orchestra.run.smithi172.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T01:36:44.955 INFO:teuthology.orchestra.run.smithi172.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T01:36:44.955 INFO:teuthology.orchestra.run.smithi172.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T01:36:44.955 INFO:teuthology.orchestra.run.smithi172.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T01:36:44.955 INFO:teuthology.orchestra.run.smithi172.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T01:36:44.956 INFO:teuthology.orchestra.run.smithi172.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T01:36:44.956 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T01:36:44.956 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T01:36:44.956 INFO:teuthology.orchestra.run.smithi172.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T01:36:44.956 INFO:teuthology.orchestra.run.smithi172.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T01:36:44.956 INFO:teuthology.orchestra.run.smithi172.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T01:36:44.956 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T01:36:44.956 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T01:36:44.956 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T01:36:44.956 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T01:36:44.956 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T01:36:44.956 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T01:36:44.956 INFO:teuthology.orchestra.run.smithi172.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T01:36:44.956 INFO:teuthology.orchestra.run.smithi172.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T01:36:44.957 INFO:teuthology.orchestra.run.smithi172.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T01:36:44.957 INFO:teuthology.orchestra.run.smithi172.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T01:36:44.957 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T01:36:44.957 INFO:teuthology.orchestra.run.smithi172.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T01:36:44.957 INFO:teuthology.orchestra.run.smithi172.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T01:36:44.957 INFO:teuthology.orchestra.run.smithi172.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T01:36:44.957 INFO:teuthology.orchestra.run.smithi172.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T01:36:44.957 INFO:teuthology.orchestra.run.smithi172.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T01:36:44.957 INFO:teuthology.orchestra.run.smithi172.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T01:36:44.958 INFO:teuthology.orchestra.run.smithi172.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T01:36:44.958 INFO:teuthology.orchestra.run.smithi172.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T01:36:44.958 INFO:teuthology.orchestra.run.smithi172.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T01:36:44.958 INFO:teuthology.orchestra.run.smithi172.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T01:36:44.958 INFO:teuthology.orchestra.run.smithi172.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T01:36:44.958 INFO:teuthology.orchestra.run.smithi172.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T01:36:44.958 INFO:teuthology.orchestra.run.smithi172.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T01:36:44.958 INFO:teuthology.orchestra.run.smithi172.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T01:36:44.958 INFO:teuthology.orchestra.run.smithi172.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T01:36:44.958 INFO:teuthology.orchestra.run.smithi172.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T01:36:44.958 INFO:teuthology.orchestra.run.smithi172.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T01:36:44.958 INFO:teuthology.orchestra.run.smithi172.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T01:36:44.958 INFO:teuthology.orchestra.run.smithi172.stdout:Installing weak dependencies: 2024-04-26T01:36:44.959 INFO:teuthology.orchestra.run.smithi172.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T01:36:44.959 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-26T01:36:44.959 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction Summary 2024-04-26T01:36:44.959 INFO:teuthology.orchestra.run.smithi172.stdout:======================================================================================= 2024-04-26T01:36:44.959 INFO:teuthology.orchestra.run.smithi172.stdout:Install 148 Packages 2024-04-26T01:36:44.959 INFO:teuthology.orchestra.run.smithi172.stdout:Upgrade 4 Packages 2024-04-26T01:36:44.959 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-26T01:36:44.959 INFO:teuthology.orchestra.run.smithi172.stdout:Total download size: 222 M 2024-04-26T01:36:44.960 INFO:teuthology.orchestra.run.smithi172.stdout:Downloading Packages: 2024-04-26T01:36:45.586 INFO:teuthology.orchestra.run.smithi165.stdout:(7/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x86 18 MB/s | 17 MB 00:00 2024-04-26T01:36:45.669 INFO:teuthology.orchestra.run.smithi165.stdout:(8/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 2.8 MB/s | 5.4 MB 00:01 2024-04-26T01:36:45.752 INFO:teuthology.orchestra.run.smithi165.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 302 kB/s | 25 kB 00:00 2024-04-26T01:36:45.761 INFO:teuthology.orchestra.run.smithi172.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 25 kB/s | 6.6 kB 00:00 2024-04-26T01:36:46.227 INFO:teuthology.orchestra.run.smithi165.stdout:(10/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el 17 MB/s | 11 MB 00:00 2024-04-26T01:36:46.294 INFO:teuthology.orchestra.run.smithi165.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 3.3 MB/s | 225 kB 00:00 2024-04-26T01:36:46.311 INFO:teuthology.orchestra.run.smithi172.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.6 MB/s | 890 kB 00:00 2024-04-26T01:36:46.353 INFO:teuthology.orchestra.run.smithi165.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 464 kB/s | 27 kB 00:00 2024-04-26T01:36:46.386 INFO:teuthology.orchestra.run.smithi172.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 1.9 MB/s | 145 kB 00:00 2024-04-26T01:36:46.420 INFO:teuthology.orchestra.run.smithi165.stdout:(13/152): libcephfs-devel-19.0.0-3156.gd69f13d8 475 kB/s | 32 kB 00:00 2024-04-26T01:36:46.503 INFO:teuthology.orchestra.run.smithi165.stdout:(14/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 8.8 MB/s | 747 kB 00:00 2024-04-26T01:36:46.571 INFO:teuthology.orchestra.run.smithi165.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 2.4 MB/s | 163 kB 00:00 2024-04-26T01:36:46.638 INFO:teuthology.orchestra.run.smithi165.stdout:(16/152): librados-devel-19.0.0-3156.gd69f13d8. 1.8 MB/s | 126 kB 00:00 2024-04-26T01:36:46.730 INFO:teuthology.orchestra.run.smithi165.stdout:(17/152): libradosstriper1-19.0.0-3156.gd69f13d 5.3 MB/s | 500 kB 00:00 2024-04-26T01:36:47.222 INFO:teuthology.orchestra.run.smithi165.stdout:(18/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 11 MB/s | 5.4 MB 00:00 2024-04-26T01:36:47.280 INFO:teuthology.orchestra.run.smithi165.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 777 kB/s | 45 kB 00:00 2024-04-26T01:36:47.311 INFO:teuthology.orchestra.run.smithi172.stdout:(4/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 2.6 MB/s | 2.4 MB 00:00 2024-04-26T01:36:47.339 INFO:teuthology.orchestra.run.smithi165.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 2.3 MB/s | 135 kB 00:00 2024-04-26T01:36:47.431 INFO:teuthology.orchestra.run.smithi165.stdout:(21/152): python3-cephfs-19.0.0-3156.gd69f13d8. 1.7 MB/s | 161 kB 00:00 2024-04-26T01:36:47.514 INFO:teuthology.orchestra.run.smithi165.stdout:(22/152): python3-rados-19.0.0-3156.gd69f13d8.e 3.8 MB/s | 320 kB 00:00 2024-04-26T01:36:47.598 INFO:teuthology.orchestra.run.smithi165.stdout:(23/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 3.5 MB/s | 298 kB 00:00 2024-04-26T01:36:47.673 INFO:teuthology.orchestra.run.smithi165.stdout:(24/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 1.3 MB/s | 99 kB 00:00 2024-04-26T01:36:47.703 INFO:teuthology.orchestra.run.smithi172.stdout:(5/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 2.4 MB/s | 5.4 MB 00:02 2024-04-26T01:36:47.732 INFO:teuthology.orchestra.run.smithi165.stdout:(25/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.4 MB/s | 85 kB 00:00 2024-04-26T01:36:47.937 INFO:teuthology.orchestra.run.smithi172.stdout:(6/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 2.4 MB/s | 1.5 MB 00:00 2024-04-26T01:36:48.124 INFO:teuthology.orchestra.run.smithi165.stdout:(26/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 7.9 MB/s | 3.1 MB 00:00 2024-04-26T01:36:48.216 INFO:teuthology.orchestra.run.smithi165.stdout:(27/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 1.8 MB/s | 170 kB 00:00 2024-04-26T01:36:48.291 INFO:teuthology.orchestra.run.smithi165.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3156.g 456 kB/s | 34 kB 00:00 2024-04-26T01:36:48.374 INFO:teuthology.orchestra.run.smithi165.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 1.7 MB/s | 142 kB 00:00 2024-04-26T01:36:48.841 INFO:teuthology.orchestra.run.smithi165.stdout:(30/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 7.6 MB/s | 3.5 MB 00:00 2024-04-26T01:36:49.070 INFO:teuthology.orchestra.run.smithi172.stdout:(7/152): ceph-common-19.0.0-3156.gd69f13d8.el7. 5.9 MB/s | 21 MB 00:03 2024-04-26T01:36:49.575 INFO:teuthology.orchestra.run.smithi165.stdout:(31/152): ceph-mgr-diskprediction-local-19.0.0- 10 MB/s | 7.4 MB 00:00 2024-04-26T01:36:49.650 INFO:teuthology.orchestra.run.smithi165.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 3.3 MB/s | 249 kB 00:00 2024-04-26T01:36:49.717 INFO:teuthology.orchestra.run.smithi165.stdout:(33/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 739 kB/s | 49 kB 00:00 2024-04-26T01:36:49.792 INFO:teuthology.orchestra.run.smithi165.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3156.gd 219 kB/s | 16 kB 00:00 2024-04-26T01:36:49.876 INFO:teuthology.orchestra.run.smithi165.stdout:(35/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 3.2 MB/s | 273 kB 00:00 2024-04-26T01:36:49.937 INFO:teuthology.orchestra.run.smithi172.stdout:(8/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 2.1 MB/s | 4.7 MB 00:02 2024-04-26T01:36:49.993 INFO:teuthology.orchestra.run.smithi165.stdout:(36/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 6.4 MB/s | 762 kB 00:00 2024-04-26T01:36:50.004 INFO:teuthology.orchestra.run.smithi172.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 378 kB/s | 25 kB 00:00 2024-04-26T01:36:50.043 INFO:teuthology.orchestra.run.smithi165.stdout:(37/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 365 kB/s | 18 kB 00:00 2024-04-26T01:36:50.112 INFO:teuthology.orchestra.run.smithi172.stdout:(10/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el 10 MB/s | 11 MB 00:01 2024-04-26T01:36:50.160 INFO:teuthology.orchestra.run.smithi165.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 688 kB/s | 80 kB 00:00 2024-04-26T01:36:50.171 INFO:teuthology.orchestra.run.smithi172.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 3.8 MB/s | 225 kB 00:00 2024-04-26T01:36:50.221 INFO:teuthology.orchestra.run.smithi172.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 543 kB/s | 27 kB 00:00 2024-04-26T01:36:50.263 INFO:teuthology.orchestra.run.smithi172.stdout:(13/152): libcephfs-devel-19.0.0-3156.gd69f13d8 767 kB/s | 32 kB 00:00 2024-04-26T01:36:50.268 INFO:teuthology.orchestra.run.smithi165.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 7.3 MB/s | 803 kB 00:00 2024-04-26T01:36:50.311 INFO:teuthology.orchestra.run.smithi165.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 4.6 MB/s | 195 kB 00:00 2024-04-26T01:36:50.352 INFO:teuthology.orchestra.run.smithi165.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 6.0 MB/s | 253 kB 00:00 2024-04-26T01:36:50.363 INFO:teuthology.orchestra.run.smithi172.stdout:(14/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 7.3 MB/s | 747 kB 00:00 2024-04-26T01:36:50.411 INFO:teuthology.orchestra.run.smithi165.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 21 MB/s | 1.2 MB 00:00 2024-04-26T01:36:50.422 INFO:teuthology.orchestra.run.smithi172.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 2.7 MB/s | 163 kB 00:00 2024-04-26T01:36:50.444 INFO:teuthology.orchestra.run.smithi165.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 3.1 MB/s | 106 kB 00:00 2024-04-26T01:36:50.478 INFO:teuthology.orchestra.run.smithi165.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 4.0 MB/s | 135 kB 00:00 2024-04-26T01:36:50.481 INFO:teuthology.orchestra.run.smithi172.stdout:(16/152): librados-devel-19.0.0-3156.gd69f13d8. 2.1 MB/s | 126 kB 00:00 2024-04-26T01:36:50.511 INFO:teuthology.orchestra.run.smithi165.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 3.7 MB/s | 125 kB 00:00 2024-04-26T01:36:50.545 INFO:teuthology.orchestra.run.smithi165.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 6.2 MB/s | 215 kB 00:00 2024-04-26T01:36:50.556 INFO:teuthology.orchestra.run.smithi172.stdout:(17/152): libradosstriper1-19.0.0-3156.gd69f13d 6.5 MB/s | 500 kB 00:00 2024-04-26T01:36:50.596 INFO:teuthology.orchestra.run.smithi165.stdout:(47/152): automake-1.16.2-8.el9.noarch.rpm 13 MB/s | 689 kB 00:00 2024-04-26T01:36:50.683 INFO:teuthology.orchestra.run.smithi165.stdout:(48/152): bison-3.7.4-5.el9.x86_64.rpm 11 MB/s | 943 kB 00:00 2024-04-26T01:36:50.721 INFO:teuthology.orchestra.run.smithi165.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 2.7 MB/s | 107 kB 00:00 2024-04-26T01:36:50.755 INFO:teuthology.orchestra.run.smithi165.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 646 kB/s | 21 kB 00:00 2024-04-26T01:36:50.797 INFO:teuthology.orchestra.run.smithi165.stdout:(51/152): flex-2.6.4-9.el9.x86_64.rpm 7.4 MB/s | 313 kB 00:00 2024-04-26T01:36:50.830 INFO:teuthology.orchestra.run.smithi165.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 980 kB/s | 33 kB 00:00 2024-04-26T01:36:50.931 INFO:teuthology.orchestra.run.smithi165.stdout:(53/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 30 MB/s | 3.0 MB 00:00 2024-04-26T01:36:50.956 INFO:teuthology.orchestra.run.smithi165.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 731 kB/s | 18 kB 00:00 2024-04-26T01:36:50.981 INFO:teuthology.orchestra.run.smithi165.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 447 kB/s | 11 kB 00:00 2024-04-26T01:36:51.015 INFO:teuthology.orchestra.run.smithi165.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 4.7 MB/s | 160 kB 00:00 2024-04-26T01:36:51.040 INFO:teuthology.orchestra.run.smithi172.stdout:(18/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 11 MB/s | 5.4 MB 00:00 2024-04-26T01:36:51.048 INFO:teuthology.orchestra.run.smithi165.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 45 kB 00:00 2024-04-26T01:36:51.084 INFO:teuthology.orchestra.run.smithi172.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 1.0 MB/s | 45 kB 00:00 2024-04-26T01:36:51.102 INFO:teuthology.orchestra.run.smithi165.stdout:(58/152): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 662 kB 00:00 2024-04-26T01:36:51.134 INFO:teuthology.orchestra.run.smithi172.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 2.6 MB/s | 135 kB 00:00 2024-04-26T01:36:51.149 INFO:teuthology.orchestra.run.smithi165.stdout:(59/152): libtool-2.4.6-45.el9.x86_64.rpm 11 MB/s | 582 kB 00:00 2024-04-26T01:36:51.185 INFO:teuthology.orchestra.run.smithi172.stdout:(21/152): python3-cephfs-19.0.0-3156.gd69f13d8. 3.1 MB/s | 161 kB 00:00 2024-04-26T01:36:51.191 INFO:teuthology.orchestra.run.smithi165.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 610 kB/s | 25 kB 00:00 2024-04-26T01:36:51.233 INFO:teuthology.orchestra.run.smithi165.stdout:(61/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-04-26T01:36:51.243 INFO:teuthology.orchestra.run.smithi172.stdout:(22/152): python3-rados-19.0.0-3156.gd69f13d8.e 5.4 MB/s | 320 kB 00:00 2024-04-26T01:36:51.275 INFO:teuthology.orchestra.run.smithi165.stdout:(62/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.9 MB/s | 292 kB 00:00 2024-04-26T01:36:51.310 INFO:teuthology.orchestra.run.smithi172.stdout:(23/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 4.4 MB/s | 298 kB 00:00 2024-04-26T01:36:51.317 INFO:teuthology.orchestra.run.smithi165.stdout:(63/152): lua-5.4.4-4.el9.x86_64.rpm 4.4 MB/s | 188 kB 00:00 2024-04-26T01:36:51.359 INFO:teuthology.orchestra.run.smithi165.stdout:(64/152): openblas-0.3.21-2.el9.x86_64.rpm 841 kB/s | 35 kB 00:00 2024-04-26T01:36:51.360 INFO:teuthology.orchestra.run.smithi172.stdout:(24/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 1.9 MB/s | 99 kB 00:00 2024-04-26T01:36:51.402 INFO:teuthology.orchestra.run.smithi172.stdout:(25/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 2.0 MB/s | 85 kB 00:00 2024-04-26T01:36:51.584 INFO:teuthology.orchestra.run.smithi165.stdout:(65/152): openblas-openmp-0.3.21-2.el9.x86_64.r 21 MB/s | 4.7 MB 00:00 2024-04-26T01:36:51.661 INFO:teuthology.orchestra.run.smithi172.stdout:(26/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 12 MB/s | 3.1 MB 00:00 2024-04-26T01:36:51.714 INFO:teuthology.orchestra.run.smithi172.stdout:(27/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 3.3 MB/s | 170 kB 00:00 2024-04-26T01:36:51.761 INFO:teuthology.orchestra.run.smithi172.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3156.g 685 kB/s | 34 kB 00:00 2024-04-26T01:36:51.792 INFO:teuthology.orchestra.run.smithi165.stdout:(66/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 21 MB/s | 4.4 MB 00:00 2024-04-26T01:36:51.812 INFO:teuthology.orchestra.run.smithi172.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 2.8 MB/s | 142 kB 00:00 2024-04-26T01:36:51.868 INFO:teuthology.orchestra.run.smithi165.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-04-26T01:36:51.943 INFO:teuthology.orchestra.run.smithi165.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 911 kB 00:00 2024-04-26T01:36:52.010 INFO:teuthology.orchestra.run.smithi165.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 133 kB 00:00 2024-04-26T01:36:52.111 INFO:teuthology.orchestra.run.smithi172.stdout:(30/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 12 MB/s | 3.5 MB 00:00 2024-04-26T01:36:52.218 INFO:teuthology.orchestra.run.smithi165.stdout:(70/152): postgresql-server-13.14-1.el9.x86_64. 28 MB/s | 5.8 MB 00:00 2024-04-26T01:36:52.271 INFO:teuthology.orchestra.run.smithi172.stdout:(31/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x8 3.9 MB/s | 17 MB 00:04 2024-04-26T01:36:52.285 INFO:teuthology.orchestra.run.smithi165.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-04-26T01:36:52.319 INFO:teuthology.orchestra.run.smithi165.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 2.6 MB/s | 87 kB 00:00 2024-04-26T01:36:52.396 INFO:teuthology.orchestra.run.smithi172.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 1.9 MB/s | 249 kB 00:00 2024-04-26T01:36:52.463 INFO:teuthology.orchestra.run.smithi172.stdout:(33/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 744 kB/s | 49 kB 00:00 2024-04-26T01:36:52.519 INFO:teuthology.orchestra.run.smithi165.stdout:(73/152): python3-babel-2.9.1-2.el9.noarch.rpm 30 MB/s | 6.0 MB 00:00 2024-04-26T01:36:52.530 INFO:teuthology.orchestra.run.smithi172.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3156.gd 247 kB/s | 16 kB 00:00 2024-04-26T01:36:52.561 INFO:teuthology.orchestra.run.smithi165.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 5.8 MB/s | 244 kB 00:00 2024-04-26T01:36:52.603 INFO:teuthology.orchestra.run.smithi165.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 5.8 MB/s | 248 kB 00:00 2024-04-26T01:36:52.636 INFO:teuthology.orchestra.run.smithi165.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 766 kB/s | 26 kB 00:00 2024-04-26T01:36:52.670 INFO:teuthology.orchestra.run.smithi165.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 571 kB/s | 19 kB 00:00 2024-04-26T01:36:52.672 INFO:teuthology.orchestra.run.smithi172.stdout:(35/152): ceph-mgr-diskprediction-local-19.0.0- 13 MB/s | 7.4 MB 00:00 2024-04-26T01:36:52.697 INFO:teuthology.orchestra.run.smithi172.stdout:(36/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 1.6 MB/s | 273 kB 00:00 2024-04-26T01:36:52.704 INFO:teuthology.orchestra.run.smithi165.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 5.0 MB/s | 172 kB 00:00 2024-04-26T01:36:52.737 INFO:teuthology.orchestra.run.smithi165.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 1.0 MB/s | 35 kB 00:00 2024-04-26T01:36:52.747 INFO:teuthology.orchestra.run.smithi172.stdout:(37/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 9.9 MB/s | 762 kB 00:00 2024-04-26T01:36:52.764 INFO:teuthology.orchestra.run.smithi172.stdout:(38/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 273 kB/s | 18 kB 00:00 2024-04-26T01:36:52.881 INFO:teuthology.orchestra.run.smithi172.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 602 kB/s | 80 kB 00:00 2024-04-26T01:36:52.896 INFO:teuthology.orchestra.run.smithi165.stdout:(80/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 32 MB/s | 5.1 MB 00:00 2024-04-26T01:36:52.938 INFO:teuthology.orchestra.run.smithi165.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.9 MB/s | 250 kB 00:00 2024-04-26T01:36:52.940 INFO:teuthology.orchestra.run.smithi172.stdout:(40/152): libgfortran-11.4.1-3.el9.x86_64.rpm 4.5 MB/s | 803 kB 00:00 2024-04-26T01:36:52.965 INFO:teuthology.orchestra.run.smithi172.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 195 kB 00:00 2024-04-26T01:36:52.972 INFO:teuthology.orchestra.run.smithi165.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 6.5 MB/s | 222 kB 00:00 2024-04-26T01:36:52.999 INFO:teuthology.orchestra.run.smithi172.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.2 MB/s | 253 kB 00:00 2024-04-26T01:36:53.014 INFO:teuthology.orchestra.run.smithi165.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 1.8 MB/s | 77 kB 00:00 2024-04-26T01:36:53.049 INFO:teuthology.orchestra.run.smithi172.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 15 MB/s | 1.2 MB 00:00 2024-04-26T01:36:53.055 INFO:teuthology.orchestra.run.smithi165.stdout:(84/152): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-04-26T01:36:53.074 INFO:teuthology.orchestra.run.smithi172.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-04-26T01:36:53.100 INFO:teuthology.orchestra.run.smithi172.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-04-26T01:36:53.106 INFO:teuthology.orchestra.run.smithi165.stdout:(85/152): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-04-26T01:36:53.125 INFO:teuthology.orchestra.run.smithi172.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-26T01:36:53.142 INFO:teuthology.orchestra.run.smithi165.stdout:(86/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 4.7 MB/s | 159 kB 00:00 2024-04-26T01:36:53.150 INFO:teuthology.orchestra.run.smithi172.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-04-26T01:36:53.181 INFO:teuthology.orchestra.run.smithi165.stdout:(87/152): python3-pyasn1-modules-0.4.8-6.el9.no 6.5 MB/s | 279 kB 00:00 2024-04-26T01:36:53.184 INFO:teuthology.orchestra.run.smithi172.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 12 MB/s | 689 kB 00:00 2024-04-26T01:36:53.206 INFO:teuthology.orchestra.run.smithi165.stdout:(88/152): python3-pytz-2021.1-5.el9.noarch.rpm 2.0 MB/s | 51 kB 00:00 2024-04-26T01:36:53.218 INFO:teuthology.orchestra.run.smithi172.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 14 MB/s | 943 kB 00:00 2024-04-26T01:36:53.232 INFO:teuthology.orchestra.run.smithi165.stdout:(89/152): python3-requests-oauthlib-1.3.0-12.el 2.1 MB/s | 54 kB 00:00 2024-04-26T01:36:53.243 INFO:teuthology.orchestra.run.smithi172.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-04-26T01:36:53.268 INFO:teuthology.orchestra.run.smithi172.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 429 kB/s | 21 kB 00:00 2024-04-26T01:36:53.294 INFO:teuthology.orchestra.run.smithi172.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-04-26T01:36:53.319 INFO:teuthology.orchestra.run.smithi172.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 661 kB/s | 33 kB 00:00 2024-04-26T01:36:53.419 INFO:teuthology.orchestra.run.smithi172.stdout:(54/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-04-26T01:36:53.444 INFO:teuthology.orchestra.run.smithi172.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 147 kB/s | 18 kB 00:00 2024-04-26T01:36:53.478 INFO:teuthology.orchestra.run.smithi172.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 337 kB/s | 11 kB 00:00 2024-04-26T01:36:53.503 INFO:teuthology.orchestra.run.smithi172.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-04-26T01:36:53.529 INFO:teuthology.orchestra.run.smithi172.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 907 kB/s | 45 kB 00:00 2024-04-26T01:36:53.562 INFO:teuthology.orchestra.run.smithi172.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-04-26T01:36:53.587 INFO:teuthology.orchestra.run.smithi172.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 9.9 MB/s | 582 kB 00:00 2024-04-26T01:36:53.613 INFO:teuthology.orchestra.run.smithi172.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 516 kB/s | 25 kB 00:00 2024-04-26T01:36:53.638 INFO:teuthology.orchestra.run.smithi172.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 943 kB/s | 47 kB 00:00 2024-04-26T01:36:53.663 INFO:teuthology.orchestra.run.smithi172.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-04-26T01:36:53.689 INFO:teuthology.orchestra.run.smithi172.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-04-26T01:36:53.690 INFO:teuthology.orchestra.run.smithi165.stdout:(90/152): ceph-test-19.0.0-3156.gd69f13d8.el7.x 6.2 MB/s | 49 MB 00:07 2024-04-26T01:36:53.714 INFO:teuthology.orchestra.run.smithi172.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 709 kB/s | 35 kB 00:00 2024-04-26T01:36:53.873 INFO:teuthology.orchestra.run.smithi172.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 26 MB/s | 4.7 MB 00:00 2024-04-26T01:36:53.965 INFO:teuthology.orchestra.run.smithi172.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-04-26T01:36:54.007 INFO:teuthology.orchestra.run.smithi172.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-04-26T01:36:54.048 INFO:teuthology.orchestra.run.smithi172.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 11 MB/s | 911 kB 00:00 2024-04-26T01:36:54.074 INFO:teuthology.orchestra.run.smithi172.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 133 kB 00:00 2024-04-26T01:36:54.107 INFO:teuthology.orchestra.run.smithi165.stdout:(91/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 19 MB/s | 16 MB 00:00 2024-04-26T01:36:54.133 INFO:teuthology.orchestra.run.smithi165.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 95 kB/s | 42 kB 00:00 2024-04-26T01:36:54.166 INFO:teuthology.orchestra.run.smithi165.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 760 kB/s | 44 kB 00:00 2024-04-26T01:36:54.191 INFO:teuthology.orchestra.run.smithi165.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-04-26T01:36:54.217 INFO:teuthology.orchestra.run.smithi165.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 720 kB/s | 36 kB 00:00 2024-04-26T01:36:54.224 INFO:teuthology.orchestra.run.smithi172.stdout:(71/152): postgresql-server-13.14-1.el9.x86_64. 33 MB/s | 5.8 MB 00:00 2024-04-26T01:36:54.242 INFO:teuthology.orchestra.run.smithi165.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-04-26T01:36:54.257 INFO:teuthology.orchestra.run.smithi172.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 5.5 MB/s | 1.0 MB 00:00 2024-04-26T01:36:54.267 INFO:teuthology.orchestra.run.smithi165.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-04-26T01:36:54.283 INFO:teuthology.orchestra.run.smithi172.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-26T01:36:54.292 INFO:teuthology.orchestra.run.smithi165.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-04-26T01:36:54.318 INFO:teuthology.orchestra.run.smithi165.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-04-26T01:36:54.343 INFO:teuthology.orchestra.run.smithi165.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-26T01:36:54.368 INFO:teuthology.orchestra.run.smithi165.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 898 kB/s | 45 kB 00:00 2024-04-26T01:36:54.393 INFO:teuthology.orchestra.run.smithi165.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 445 kB/s | 22 kB 00:00 2024-04-26T01:36:54.425 INFO:teuthology.orchestra.run.smithi172.stdout:(74/152): python3-babel-2.9.1-2.el9.noarch.rpm 36 MB/s | 6.0 MB 00:00 2024-04-26T01:36:54.435 INFO:teuthology.orchestra.run.smithi165.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 13 MB/s | 863 kB 00:00 2024-04-26T01:36:54.450 INFO:teuthology.orchestra.run.smithi172.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.4 MB/s | 244 kB 00:00 2024-04-26T01:36:54.476 INFO:teuthology.orchestra.run.smithi172.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-04-26T01:36:54.501 INFO:teuthology.orchestra.run.smithi172.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 513 kB/s | 26 kB 00:00 2024-04-26T01:36:54.526 INFO:teuthology.orchestra.run.smithi172.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 380 kB/s | 19 kB 00:00 2024-04-26T01:36:54.551 INFO:teuthology.orchestra.run.smithi172.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-04-26T01:36:54.577 INFO:teuthology.orchestra.run.smithi172.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 699 kB/s | 35 kB 00:00 2024-04-26T01:36:54.644 INFO:teuthology.orchestra.run.smithi165.stdout:(104/152): ceph-common-19.0.0-3156.gd69f13d8.el 1.9 MB/s | 21 MB 00:10 2024-04-26T01:36:54.685 INFO:teuthology.orchestra.run.smithi172.stdout:(81/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 38 MB/s | 5.1 MB 00:00 2024-04-26T01:36:54.716 INFO:teuthology.orchestra.run.smithi172.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.8 MB/s | 250 kB 00:00 2024-04-26T01:36:54.744 INFO:teuthology.orchestra.run.smithi172.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-04-26T01:36:54.770 INFO:teuthology.orchestra.run.smithi172.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.4 MB/s | 77 kB 00:00 2024-04-26T01:36:54.795 INFO:teuthology.orchestra.run.smithi172.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 844 kB/s | 42 kB 00:00 2024-04-26T01:36:54.820 INFO:teuthology.orchestra.run.smithi172.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-04-26T01:36:54.845 INFO:teuthology.orchestra.run.smithi172.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-04-26T01:36:54.871 INFO:teuthology.orchestra.run.smithi172.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 5.5 MB/s | 279 kB 00:00 2024-04-26T01:36:54.896 INFO:teuthology.orchestra.run.smithi172.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-26T01:36:54.911 INFO:teuthology.orchestra.run.smithi165.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 73 kB/s | 19 kB 00:00 2024-04-26T01:36:54.921 INFO:teuthology.orchestra.run.smithi172.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-26T01:36:55.078 INFO:teuthology.orchestra.run.smithi165.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 481 kB/s | 309 kB 00:00 2024-04-26T01:36:55.078 INFO:teuthology.orchestra.run.smithi165.stdout:[MIRROR] libarrow-9.0.0-10.el9.x86_64.rpm: Status code: 404 for http://mirrors.sonic.net/epel/9/Everything/x86_64/Packages/l/libarrow-9.0.0-10.el9.x86_64.rpm (IP: 157.131.224.201) 2024-04-26T01:36:55.097 INFO:teuthology.orchestra.run.smithi172.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 241 kB/s | 42 kB 00:00 2024-04-26T01:36:55.161 INFO:teuthology.orchestra.run.smithi165.stdout:(107/152): abseil-cpp-20211102.0-3.el9.x86_64.r 715 kB/s | 548 kB 00:00 2024-04-26T01:36:55.242 INFO:teuthology.orchestra.run.smithi165.stdout:[MIRROR] libarrow-doc-9.0.0-10.el9.noarch.rpm: Status code: 404 for http://mirrors.sonic.net/epel/9/Everything/x86_64/Packages/l/libarrow-doc-9.0.0-10.el9.noarch.rpm (IP: 157.131.224.201) 2024-04-26T01:36:55.255 INFO:teuthology.orchestra.run.smithi172.stdout:(92/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 45 MB/s | 16 MB 00:00 2024-04-26T01:36:55.281 INFO:teuthology.orchestra.run.smithi172.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 244 kB/s | 44 kB 00:00 2024-04-26T01:36:55.306 INFO:teuthology.orchestra.run.smithi172.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-04-26T01:36:55.331 INFO:teuthology.orchestra.run.smithi172.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 719 kB/s | 36 kB 00:00 2024-04-26T01:36:55.382 INFO:teuthology.orchestra.run.smithi172.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 631 kB/s | 47 kB 00:00 2024-04-26T01:36:55.407 INFO:teuthology.orchestra.run.smithi172.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 4.0 MB/s | 305 kB 00:00 2024-04-26T01:36:55.432 INFO:teuthology.orchestra.run.smithi172.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-04-26T01:36:55.433 INFO:teuthology.orchestra.run.smithi165.stdout:[MIRROR] libarrow-9.0.0-10.el9.x86_64.rpm: Status code: 404 for https://mirrors.sonic.net/epel/9/Everything/x86_64/Packages/l/libarrow-9.0.0-10.el9.x86_64.rpm (IP: 157.131.224.201) 2024-04-26T01:36:55.457 INFO:teuthology.orchestra.run.smithi172.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-04-26T01:36:55.483 INFO:teuthology.orchestra.run.smithi172.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-26T01:36:55.508 INFO:teuthology.orchestra.run.smithi172.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 896 kB/s | 45 kB 00:00 2024-04-26T01:36:55.528 INFO:teuthology.orchestra.run.smithi165.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 143 kB/s | 49 kB 00:00 2024-04-26T01:36:55.533 INFO:teuthology.orchestra.run.smithi172.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 443 kB/s | 22 kB 00:00 2024-04-26T01:36:55.567 INFO:teuthology.orchestra.run.smithi172.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 14 MB/s | 863 kB 00:00 2024-04-26T01:36:55.609 INFO:teuthology.orchestra.run.smithi165.stdout:[MIRROR] libarrow-doc-9.0.0-10.el9.noarch.rpm: Status code: 404 for https://mirrors.sonic.net/epel/9/Everything/x86_64/Packages/l/libarrow-doc-9.0.0-10.el9.noarch.rpm (IP: 157.131.224.201) 2024-04-26T01:36:55.726 INFO:teuthology.orchestra.run.smithi172.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.9 MB/s | 309 kB 00:00 2024-04-26T01:36:55.768 INFO:teuthology.orchestra.run.smithi172.stdout:(105/152): abseil-cpp-20211102.0-3.el9.x86_64.r 2.3 MB/s | 548 kB 00:00 2024-04-26T01:36:55.784 INFO:teuthology.orchestra.run.smithi172.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 332 kB/s | 19 kB 00:00 2024-04-26T01:36:55.820 INFO:teuthology.orchestra.run.smithi165.stdout:(109/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 35 kB/s | 26 kB 00:00 2024-04-26T01:36:55.868 INFO:teuthology.orchestra.run.smithi172.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 311 kB/s | 26 kB 00:00 2024-04-26T01:36:55.870 INFO:teuthology.orchestra.run.smithi165.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 197 kB/s | 67 kB 00:00 2024-04-26T01:36:55.910 INFO:teuthology.orchestra.run.smithi172.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 1.1 MB/s | 49 kB 00:00 2024-04-26T01:36:55.977 INFO:teuthology.orchestra.run.smithi172.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-04-26T01:36:56.030 INFO:teuthology.orchestra.run.smithi165.stdout:[MIRROR] parquet-libs-9.0.0-10.el9.x86_64.rpm: Status code: 404 for http://mirrors.sonic.net/epel/9/Everything/x86_64/Packages/p/parquet-libs-9.0.0-10.el9.x86_64.rpm (IP: 157.131.224.201) 2024-04-26T01:36:56.044 INFO:teuthology.orchestra.run.smithi172.stdout:(110/152): luarocks-3.9.2-1.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-04-26T01:36:56.138 INFO:teuthology.orchestra.run.smithi165.stdout:(111/152): libarrow-9.0.0-10.el9.x86_64.rpm 3.6 MB/s | 4.4 MB 00:01 2024-04-26T01:36:56.246 INFO:teuthology.orchestra.run.smithi165.stdout:(112/152): luarocks-3.9.2-1.el9.noarch.rpm 355 kB/s | 151 kB 00:00 2024-04-26T01:36:56.252 INFO:teuthology.orchestra.run.smithi172.stdout:(111/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.9 MB/s | 838 kB 00:00 2024-04-26T01:36:56.328 INFO:teuthology.orchestra.run.smithi172.stdout:(112/152): libarrow-9.0.0-10.el9.x86_64.rpm 7.9 MB/s | 4.4 MB 00:00 2024-04-26T01:36:56.386 INFO:teuthology.orchestra.run.smithi172.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 745 kB/s | 43 kB 00:00 2024-04-26T01:36:56.420 INFO:teuthology.orchestra.run.smithi172.stdout:(114/152): python3-asyncssh-2.13.2-5.el9.noarch 3.2 MB/s | 548 kB 00:00 2024-04-26T01:36:56.434 INFO:teuthology.orchestra.run.smithi165.stdout:[MIRROR] parquet-libs-9.0.0-10.el9.x86_64.rpm: Status code: 404 for https://mirrors.sonic.net/epel/9/Everything/x86_64/Packages/p/parquet-libs-9.0.0-10.el9.x86_64.rpm (IP: 157.131.224.201) 2024-04-26T01:36:56.445 INFO:teuthology.orchestra.run.smithi172.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 549 kB/s | 32 kB 00:00 2024-04-26T01:36:56.470 INFO:teuthology.orchestra.run.smithi172.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 299 kB/s | 15 kB 00:00 2024-04-26T01:36:56.496 INFO:teuthology.orchestra.run.smithi172.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-04-26T01:36:56.521 INFO:teuthology.orchestra.run.smithi165.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.3 MB/s | 838 kB 00:00 2024-04-26T01:36:56.546 INFO:teuthology.orchestra.run.smithi172.stdout:(118/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-04-26T01:36:56.571 INFO:teuthology.orchestra.run.smithi172.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-04-26T01:36:56.597 INFO:teuthology.orchestra.run.smithi172.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 703 kB/s | 35 kB 00:00 2024-04-26T01:36:56.621 INFO:teuthology.orchestra.run.smithi165.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 124 kB/s | 43 kB 00:00 2024-04-26T01:36:56.639 INFO:teuthology.orchestra.run.smithi172.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 3.0 MB/s | 206 kB 00:00 2024-04-26T01:36:56.739 INFO:teuthology.orchestra.run.smithi172.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 1.4 MB/s | 144 kB 00:00 2024-04-26T01:36:56.805 INFO:teuthology.orchestra.run.smithi165.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 114 kB/s | 32 kB 00:00 2024-04-26T01:36:56.806 INFO:teuthology.orchestra.run.smithi172.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 160 kB/s | 11 kB 00:00 2024-04-26T01:36:56.881 INFO:teuthology.orchestra.run.smithi172.stdout:(124/152): python3-jaraco-classes-3.2.1-5.el9.n 236 kB/s | 18 kB 00:00 2024-04-26T01:36:56.888 INFO:teuthology.orchestra.run.smithi165.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 56 kB/s | 15 kB 00:00 2024-04-26T01:36:56.931 INFO:teuthology.orchestra.run.smithi172.stdout:(125/152): python3-grpcio-1.46.7-10.el9.x86_64. 6.1 MB/s | 2.0 MB 00:00 2024-04-26T01:36:56.957 INFO:teuthology.orchestra.run.smithi172.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 307 kB/s | 23 kB 00:00 2024-04-26T01:36:57.007 INFO:teuthology.orchestra.run.smithi172.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 259 kB/s | 19 kB 00:00 2024-04-26T01:36:57.022 INFO:teuthology.orchestra.run.smithi165.stdout:(117/152): python3-asyncssh-2.13.2-5.el9.noarch 640 kB/s | 548 kB 00:00 2024-04-26T01:36:57.032 INFO:teuthology.orchestra.run.smithi172.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 262 kB/s | 20 kB 00:00 2024-04-26T01:36:57.099 INFO:teuthology.orchestra.run.smithi172.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 695 kB/s | 46 kB 00:00 2024-04-26T01:36:57.174 INFO:teuthology.orchestra.run.smithi172.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 6.1 MB/s | 1.0 MB 00:00 2024-04-26T01:36:57.200 INFO:teuthology.orchestra.run.smithi172.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 789 kB/s | 79 kB 00:00 2024-04-26T01:36:57.221 INFO:teuthology.orchestra.run.smithi151.stdout:Extra Packages for Enterprise Linux 1.0 MB/s | 21 MB 00:20 2024-04-26T01:36:57.230 INFO:teuthology.orchestra.run.smithi165.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 404 kB/s | 172 kB 00:00 2024-04-26T01:36:57.242 INFO:teuthology.orchestra.run.smithi172.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 863 kB/s | 58 kB 00:00 2024-04-26T01:36:57.309 INFO:teuthology.orchestra.run.smithi172.stdout:(133/152): python3-portend-3.1.0-2.el9.noarch.r 246 kB/s | 16 kB 00:00 2024-04-26T01:36:57.351 INFO:teuthology.orchestra.run.smithi172.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-04-26T01:36:57.376 INFO:teuthology.orchestra.run.smithi172.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-04-26T01:36:57.406 INFO:teuthology.orchestra.run.smithi165.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 693 kB/s | 358 kB 00:00 2024-04-26T01:36:57.418 INFO:teuthology.orchestra.run.smithi172.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 422 kB/s | 28 kB 00:00 2024-04-26T01:36:57.443 INFO:teuthology.orchestra.run.smithi172.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 460 kB/s | 31 kB 00:00 2024-04-26T01:36:57.498 INFO:teuthology.orchestra.run.smithi165.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 132 kB/s | 35 kB 00:00 2024-04-26T01:36:57.510 INFO:teuthology.orchestra.run.smithi172.stdout:(138/152): python3-rsa-4.9-2.el9.noarch.rpm 888 kB/s | 59 kB 00:00 2024-04-26T01:36:57.531 INFO:teuthology.orchestra.run.smithi165.stdout:(121/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 596 kB/s | 303 kB 00:00 2024-04-26T01:36:57.535 INFO:teuthology.orchestra.run.smithi172.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.6 MB/s | 188 kB 00:00 2024-04-26T01:36:57.561 INFO:teuthology.orchestra.run.smithi172.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-04-26T01:36:57.586 INFO:teuthology.orchestra.run.smithi172.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-04-26T01:36:57.611 INFO:teuthology.orchestra.run.smithi172.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 4.5 MB/s | 230 kB 00:00 2024-04-26T01:36:57.636 INFO:teuthology.orchestra.run.smithi172.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-04-26T01:36:57.678 INFO:teuthology.orchestra.run.smithi172.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 6.3 MB/s | 427 kB 00:00 2024-04-26T01:36:57.695 INFO:teuthology.orchestra.run.smithi172.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 341 kB/s | 20 kB 00:00 2024-04-26T01:36:57.721 INFO:teuthology.orchestra.run.smithi172.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 4.5 MB/s | 192 kB 00:00 2024-04-26T01:36:57.931 INFO:teuthology.orchestra.run.smithi165.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 391 kB/s | 206 kB 00:00 2024-04-26T01:36:57.957 INFO:teuthology.orchestra.run.smithi165.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 340 kB/s | 144 kB 00:00 2024-04-26T01:36:57.988 INFO:teuthology.orchestra.run.smithi172.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 5.4 MB/s | 1.6 MB 00:00 2024-04-26T01:36:58.124 INFO:teuthology.orchestra.run.smithi165.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 56 kB/s | 11 kB 00:00 2024-04-26T01:36:58.224 INFO:teuthology.orchestra.run.smithi165.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 66 kB/s | 18 kB 00:00 2024-04-26T01:36:58.274 INFO:teuthology.orchestra.run.smithi165.stdout:(126/152): python3-grpcio-1.46.7-10.el9.x86_64. 2.6 MB/s | 2.0 MB 00:00 2024-04-26T01:36:58.408 INFO:teuthology.orchestra.run.smithi165.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 82 kB/s | 23 kB 00:00 2024-04-26T01:36:58.491 INFO:teuthology.orchestra.run.smithi165.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 73 kB/s | 19 kB 00:00 2024-04-26T01:36:58.542 INFO:teuthology.orchestra.run.smithi165.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 74 kB/s | 20 kB 00:00 2024-04-26T01:36:58.655 INFO:teuthology.orchestra.run.smithi172.stdout:(148/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 4.7 MB/s | 3.1 MB 00:00 2024-04-26T01:36:58.722 INFO:teuthology.orchestra.run.smithi172.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 20 MB/s | 1.3 MB 00:00 2024-04-26T01:36:58.830 INFO:teuthology.orchestra.run.smithi172.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 22 MB/s | 2.4 MB 00:00 2024-04-26T01:36:58.834 INFO:teuthology.orchestra.run.smithi165.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 135 kB/s | 46 kB 00:00 2024-04-26T01:36:58.884 INFO:teuthology.orchestra.run.smithi165.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 230 kB/s | 79 kB 00:00 2024-04-26T01:36:59.076 INFO:teuthology.orchestra.run.smithi165.stdout:(132/152): python3-kubernetes-26.1.0-2.el9.noar 1.5 MB/s | 1.0 MB 00:00 2024-04-26T01:36:59.168 INFO:teuthology.orchestra.run.smithi165.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 173 kB/s | 58 kB 00:00 2024-04-26T01:36:59.343 INFO:teuthology.orchestra.run.smithi165.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 61 kB/s | 16 kB 00:00 2024-04-26T01:36:59.393 INFO:teuthology.orchestra.run.smithi165.stdout:(135/152): python3-pecan-1.4.2-3.el9.noarch.rpm 535 kB/s | 272 kB 00:00 2024-04-26T01:36:59.510 INFO:teuthology.orchestra.run.smithi165.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 263 kB/s | 90 kB 00:00 2024-04-26T01:36:59.602 INFO:teuthology.orchestra.run.smithi165.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 109 kB/s | 28 kB 00:00 2024-04-26T01:36:59.652 INFO:teuthology.orchestra.run.smithi165.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 119 kB/s | 31 kB 00:00 2024-04-26T01:36:59.911 INFO:teuthology.orchestra.run.smithi165.stdout:(139/152): python3-tempora-5.0.0-2.el9.noarch.r 138 kB/s | 36 kB 00:00 2024-04-26T01:37:00.014 INFO:teuthology.orchestra.run.smithi172.stdout:(151/152): librados2-19.0.0-3156.gd69f13d8.el7. 1.5 MB/s | 3.4 MB 00:02 2024-04-26T01:37:00.027 INFO:teuthology.orchestra.run.smithi165.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 139 kB/s | 59 kB 00:00 2024-04-26T01:37:00.053 INFO:teuthology.orchestra.run.smithi165.stdout:(141/152): python3-routes-2.5.1-5.el9.noarch.rp 347 kB/s | 188 kB 00:00 2024-04-26T01:37:00.287 INFO:teuthology.orchestra.run.smithi165.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 136 kB/s | 51 kB 00:00 2024-04-26T01:37:00.479 INFO:teuthology.orchestra.run.smithi165.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 211 kB/s | 90 kB 00:00 2024-04-26T01:37:00.537 INFO:teuthology.orchestra.run.smithi165.stdout:(144/152): python3-webob-1.8.7-6.el9.noarch.rpm 451 kB/s | 230 kB 00:00 2024-04-26T01:37:00.746 INFO:teuthology.orchestra.run.smithi165.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 75 kB/s | 20 kB 00:00 2024-04-26T01:37:00.756 INFO:teuthology.orchestra.run.smithi172.stdout:(152/152): ceph-test-19.0.0-3156.gd69f13d8.el7. 4.6 MB/s | 49 MB 00:10 2024-04-26T01:37:00.762 INFO:teuthology.orchestra.run.smithi172.stdout:-------------------------------------------------------------------------------- 2024-04-26T01:37:00.762 INFO:teuthology.orchestra.run.smithi172.stdout:Total 14 MB/s | 222 MB 00:15 2024-04-26T01:37:00.871 INFO:teuthology.orchestra.run.smithi165.stdout:(146/152): python3-werkzeug-2.0.3-3.el9.1.noarc 731 kB/s | 427 kB 00:00 2024-04-26T01:37:01.046 INFO:teuthology.orchestra.run.smithi165.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 377 kB/s | 192 kB 00:00 2024-04-26T01:37:01.263 INFO:teuthology.orchestra.run.smithi165.stdout:(148/152): librados2-19.0.0-3156.gd69f13d8.el7. 8.7 MB/s | 3.4 MB 00:00 2024-04-26T01:37:01.413 INFO:teuthology.orchestra.run.smithi165.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 9.0 MB/s | 1.3 MB 00:00 2024-04-26T01:37:01.505 INFO:teuthology.orchestra.run.smithi165.stdout:(150/152): thrift-0.15.0-2.el9.x86_64.rpm 2.1 MB/s | 1.6 MB 00:00 2024-04-26T01:37:01.597 INFO:teuthology.orchestra.run.smithi165.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 13 MB/s | 2.4 MB 00:00 2024-04-26T01:37:01.866 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction check 2024-04-26T01:37:01.880 INFO:teuthology.orchestra.run.smithi165.stdout:(152/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 3.8 MB/s | 3.1 MB 00:00 2024-04-26T01:37:01.886 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2024-04-26T01:37:01.886 INFO:teuthology.orchestra.run.smithi165.stdout:Total 12 MB/s | 222 MB 00:18 2024-04-26T01:37:02.012 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction check succeeded. 2024-04-26T01:37:02.012 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction test 2024-04-26T01:37:03.009 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2024-04-26T01:37:03.017 INFO:teuthology.orchestra.run.smithi151.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-04-26T01:37:03.155 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2024-04-26T01:37:03.155 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2024-04-26T01:37:03.778 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction test succeeded. 2024-04-26T01:37:03.779 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction 2024-04-26T01:37:04.922 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2024-04-26T01:37:04.922 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2024-04-26T01:37:05.468 INFO:teuthology.orchestra.run.smithi151.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T01:37:05.470 INFO:teuthology.orchestra.run.smithi151.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T01:37:05.490 INFO:teuthology.orchestra.run.smithi151.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T01:37:05.491 INFO:teuthology.orchestra.run.smithi151.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T01:37:05.492 INFO:teuthology.orchestra.run.smithi151.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T01:37:05.493 INFO:teuthology.orchestra.run.smithi151.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T01:37:05.496 INFO:teuthology.orchestra.run.smithi151.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T01:37:05.500 INFO:teuthology.orchestra.run.smithi151.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T01:37:05.500 INFO:teuthology.orchestra.run.smithi151.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T01:37:05.571 INFO:teuthology.orchestra.run.smithi151.stdout:Dependencies resolved. 2024-04-26T01:37:05.592 INFO:teuthology.orchestra.run.smithi151.stdout:======================================================================================= 2024-04-26T01:37:05.592 INFO:teuthology.orchestra.run.smithi151.stdout: Package Arch Version Repository Size 2024-04-26T01:37:05.592 INFO:teuthology.orchestra.run.smithi151.stdout:======================================================================================= 2024-04-26T01:37:05.592 INFO:teuthology.orchestra.run.smithi151.stdout:Installing: 2024-04-26T01:37:05.592 INFO:teuthology.orchestra.run.smithi151.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T01:37:05.592 INFO:teuthology.orchestra.run.smithi151.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-26T01:37:05.593 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-26T01:37:05.593 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-26T01:37:05.593 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-26T01:37:05.593 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-26T01:37:05.593 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-26T01:37:05.593 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-26T01:37:05.593 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-26T01:37:05.593 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-26T01:37:05.593 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-26T01:37:05.593 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-26T01:37:05.593 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-26T01:37:05.593 INFO:teuthology.orchestra.run.smithi151.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-26T01:37:05.593 INFO:teuthology.orchestra.run.smithi151.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-26T01:37:05.593 INFO:teuthology.orchestra.run.smithi151.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-26T01:37:05.594 INFO:teuthology.orchestra.run.smithi151.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-26T01:37:05.594 INFO:teuthology.orchestra.run.smithi151.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T01:37:05.594 INFO:teuthology.orchestra.run.smithi151.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T01:37:05.594 INFO:teuthology.orchestra.run.smithi151.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T01:37:05.594 INFO:teuthology.orchestra.run.smithi151.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-26T01:37:05.594 INFO:teuthology.orchestra.run.smithi151.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-26T01:37:05.594 INFO:teuthology.orchestra.run.smithi151.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-26T01:37:05.594 INFO:teuthology.orchestra.run.smithi151.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T01:37:05.594 INFO:teuthology.orchestra.run.smithi151.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T01:37:05.594 INFO:teuthology.orchestra.run.smithi151.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T01:37:05.594 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T01:37:05.594 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T01:37:05.595 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T01:37:05.595 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-26T01:37:05.595 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-26T01:37:05.595 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-26T01:37:05.595 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-26T01:37:05.595 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-26T01:37:05.595 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-26T01:37:05.595 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-26T01:37:05.595 INFO:teuthology.orchestra.run.smithi151.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T01:37:05.595 INFO:teuthology.orchestra.run.smithi151.stdout:Upgrading: 2024-04-26T01:37:05.595 INFO:teuthology.orchestra.run.smithi151.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-26T01:37:05.595 INFO:teuthology.orchestra.run.smithi151.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-26T01:37:05.595 INFO:teuthology.orchestra.run.smithi151.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T01:37:05.596 INFO:teuthology.orchestra.run.smithi151.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T01:37:05.596 INFO:teuthology.orchestra.run.smithi151.stdout:Installing dependencies: 2024-04-26T01:37:05.596 INFO:teuthology.orchestra.run.smithi151.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T01:37:05.596 INFO:teuthology.orchestra.run.smithi151.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T01:37:05.596 INFO:teuthology.orchestra.run.smithi151.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T01:37:05.596 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-26T01:37:05.596 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-26T01:37:05.596 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-26T01:37:05.596 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-26T01:37:05.596 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-26T01:37:05.596 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-26T01:37:05.597 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-26T01:37:05.597 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-26T01:37:05.597 INFO:teuthology.orchestra.run.smithi151.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T01:37:05.597 INFO:teuthology.orchestra.run.smithi151.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T01:37:05.597 INFO:teuthology.orchestra.run.smithi151.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T01:37:05.597 INFO:teuthology.orchestra.run.smithi151.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T01:37:05.597 INFO:teuthology.orchestra.run.smithi151.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T01:37:05.597 INFO:teuthology.orchestra.run.smithi151.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T01:37:05.597 INFO:teuthology.orchestra.run.smithi151.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-26T01:37:05.597 INFO:teuthology.orchestra.run.smithi151.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-26T01:37:05.597 INFO:teuthology.orchestra.run.smithi151.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-26T01:37:05.597 INFO:teuthology.orchestra.run.smithi151.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T01:37:05.597 INFO:teuthology.orchestra.run.smithi151.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T01:37:05.597 INFO:teuthology.orchestra.run.smithi151.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T01:37:05.598 INFO:teuthology.orchestra.run.smithi151.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T01:37:05.598 INFO:teuthology.orchestra.run.smithi151.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T01:37:05.598 INFO:teuthology.orchestra.run.smithi151.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-26T01:37:05.598 INFO:teuthology.orchestra.run.smithi151.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T01:37:05.598 INFO:teuthology.orchestra.run.smithi151.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-26T01:37:05.598 INFO:teuthology.orchestra.run.smithi151.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T01:37:05.598 INFO:teuthology.orchestra.run.smithi151.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T01:37:05.598 INFO:teuthology.orchestra.run.smithi151.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T01:37:05.598 INFO:teuthology.orchestra.run.smithi151.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T01:37:05.598 INFO:teuthology.orchestra.run.smithi151.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T01:37:05.598 INFO:teuthology.orchestra.run.smithi151.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T01:37:05.598 INFO:teuthology.orchestra.run.smithi151.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T01:37:05.598 INFO:teuthology.orchestra.run.smithi151.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T01:37:05.598 INFO:teuthology.orchestra.run.smithi151.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-26T01:37:05.599 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T01:37:05.599 INFO:teuthology.orchestra.run.smithi151.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T01:37:05.599 INFO:teuthology.orchestra.run.smithi151.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T01:37:05.599 INFO:teuthology.orchestra.run.smithi151.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T01:37:05.599 INFO:teuthology.orchestra.run.smithi151.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T01:37:05.599 INFO:teuthology.orchestra.run.smithi151.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T01:37:05.599 INFO:teuthology.orchestra.run.smithi151.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T01:37:05.599 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T01:37:05.599 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-26T01:37:05.599 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-26T01:37:05.599 INFO:teuthology.orchestra.run.smithi151.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T01:37:05.599 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T01:37:05.600 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T01:37:05.600 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T01:37:05.600 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T01:37:05.600 INFO:teuthology.orchestra.run.smithi151.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T01:37:05.600 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T01:37:05.600 INFO:teuthology.orchestra.run.smithi151.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T01:37:05.600 INFO:teuthology.orchestra.run.smithi151.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T01:37:05.600 INFO:teuthology.orchestra.run.smithi151.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T01:37:05.600 INFO:teuthology.orchestra.run.smithi151.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T01:37:05.600 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T01:37:05.600 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T01:37:05.600 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T01:37:05.600 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T01:37:05.600 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T01:37:05.601 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T01:37:05.601 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T01:37:05.601 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T01:37:05.601 INFO:teuthology.orchestra.run.smithi151.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T01:37:05.601 INFO:teuthology.orchestra.run.smithi151.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T01:37:05.601 INFO:teuthology.orchestra.run.smithi151.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T01:37:05.601 INFO:teuthology.orchestra.run.smithi151.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T01:37:05.601 INFO:teuthology.orchestra.run.smithi151.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T01:37:05.601 INFO:teuthology.orchestra.run.smithi151.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T01:37:05.601 INFO:teuthology.orchestra.run.smithi151.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T01:37:05.601 INFO:teuthology.orchestra.run.smithi151.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T01:37:05.601 INFO:teuthology.orchestra.run.smithi151.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T01:37:05.601 INFO:teuthology.orchestra.run.smithi151.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T01:37:05.602 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T01:37:05.602 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T01:37:05.602 INFO:teuthology.orchestra.run.smithi151.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T01:37:05.602 INFO:teuthology.orchestra.run.smithi151.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T01:37:05.602 INFO:teuthology.orchestra.run.smithi151.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T01:37:05.602 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T01:37:05.603 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T01:37:05.603 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T01:37:05.603 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T01:37:05.603 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T01:37:05.603 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T01:37:05.603 INFO:teuthology.orchestra.run.smithi151.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T01:37:05.603 INFO:teuthology.orchestra.run.smithi151.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T01:37:05.603 INFO:teuthology.orchestra.run.smithi151.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T01:37:05.603 INFO:teuthology.orchestra.run.smithi151.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T01:37:05.603 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T01:37:05.603 INFO:teuthology.orchestra.run.smithi151.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T01:37:05.604 INFO:teuthology.orchestra.run.smithi151.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T01:37:05.604 INFO:teuthology.orchestra.run.smithi151.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T01:37:05.604 INFO:teuthology.orchestra.run.smithi151.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T01:37:05.604 INFO:teuthology.orchestra.run.smithi151.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T01:37:05.604 INFO:teuthology.orchestra.run.smithi151.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T01:37:05.604 INFO:teuthology.orchestra.run.smithi151.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T01:37:05.604 INFO:teuthology.orchestra.run.smithi151.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T01:37:05.604 INFO:teuthology.orchestra.run.smithi151.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T01:37:05.604 INFO:teuthology.orchestra.run.smithi151.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T01:37:05.604 INFO:teuthology.orchestra.run.smithi151.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T01:37:05.604 INFO:teuthology.orchestra.run.smithi151.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T01:37:05.604 INFO:teuthology.orchestra.run.smithi151.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T01:37:05.604 INFO:teuthology.orchestra.run.smithi151.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T01:37:05.604 INFO:teuthology.orchestra.run.smithi151.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T01:37:05.605 INFO:teuthology.orchestra.run.smithi151.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T01:37:05.605 INFO:teuthology.orchestra.run.smithi151.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T01:37:05.605 INFO:teuthology.orchestra.run.smithi151.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T01:37:05.605 INFO:teuthology.orchestra.run.smithi151.stdout:Installing weak dependencies: 2024-04-26T01:37:05.605 INFO:teuthology.orchestra.run.smithi151.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T01:37:05.605 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:37:05.605 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction Summary 2024-04-26T01:37:05.605 INFO:teuthology.orchestra.run.smithi151.stdout:======================================================================================= 2024-04-26T01:37:05.605 INFO:teuthology.orchestra.run.smithi151.stdout:Install 148 Packages 2024-04-26T01:37:05.605 INFO:teuthology.orchestra.run.smithi151.stdout:Upgrade 4 Packages 2024-04-26T01:37:05.605 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:37:05.606 INFO:teuthology.orchestra.run.smithi151.stdout:Total download size: 222 M 2024-04-26T01:37:05.606 INFO:teuthology.orchestra.run.smithi151.stdout:Downloading Packages: 2024-04-26T01:37:06.969 INFO:teuthology.orchestra.run.smithi151.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 12 kB/s | 6.6 kB 00:00 2024-04-26T01:37:07.627 INFO:teuthology.orchestra.run.smithi151.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.3 MB/s | 890 kB 00:00 2024-04-26T01:37:07.802 INFO:teuthology.orchestra.run.smithi151.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 831 kB/s | 145 kB 00:00 2024-04-26T01:37:09.044 INFO:teuthology.orchestra.run.smithi151.stdout:(4/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 1.9 MB/s | 2.4 MB 00:01 2024-04-26T01:37:09.261 INFO:teuthology.orchestra.run.smithi151.stdout:(5/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 1.9 MB/s | 5.4 MB 00:02 2024-04-26T01:37:09.734 INFO:teuthology.orchestra.run.smithi172.stdout: Preparing : 1/1 2024-04-26T01:37:09.834 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T01:37:09.837 INFO:teuthology.orchestra.run.smithi151.stdout:(6/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 1.9 MB/s | 1.5 MB 00:00 2024-04-26T01:37:09.872 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T01:37:09.934 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T01:37:09.968 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T01:37:09.994 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T01:37:10.036 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T01:37:10.413 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T01:37:10.470 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-26T01:37:10.589 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-26T01:37:10.629 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-26T01:37:10.720 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-26T01:37:10.750 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-26T01:37:10.897 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T01:37:10.947 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T01:37:10.975 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-26T01:37:11.034 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2024-04-26T01:37:11.078 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-26T01:37:11.101 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-26T01:37:11.109 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T01:37:11.146 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T01:37:11.163 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-26T01:37:11.185 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T01:37:11.200 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T01:37:11.211 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T01:37:11.234 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T01:37:11.253 INFO:teuthology.orchestra.run.smithi151.stdout:(7/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 2.4 MB/s | 4.7 MB 00:01 2024-04-26T01:37:11.259 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T01:37:11.301 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T01:37:11.385 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T01:37:11.418 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T01:37:11.525 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T01:37:11.614 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-26T01:37:11.638 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T01:37:11.651 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T01:37:11.711 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T01:37:11.736 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-26T01:37:11.785 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T01:37:11.827 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T01:37:11.857 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-26T01:37:11.871 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T01:37:11.886 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-26T01:37:11.903 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T01:37:11.931 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T01:37:11.978 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-26T01:37:12.007 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-26T01:37:12.014 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T01:37:12.062 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T01:37:12.100 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T01:37:12.144 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T01:37:12.155 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T01:37:12.204 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T01:37:12.221 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T01:37:12.223 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-26T01:37:12.254 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-26T01:37:12.298 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-26T01:37:12.326 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-26T01:37:12.327 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-26T01:37:12.349 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-26T01:37:12.397 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T01:37:12.412 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-26T01:37:12.433 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T01:37:12.441 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T01:37:12.459 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T01:37:12.516 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T01:37:12.659 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T01:37:12.659 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T01:37:12.691 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T01:37:12.719 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T01:37:12.791 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T01:37:12.888 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-26T01:37:12.924 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T01:37:12.959 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T01:37:13.033 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T01:37:13.075 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T01:37:13.119 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T01:37:13.150 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T01:37:13.178 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T01:37:13.248 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T01:37:13.249 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T01:37:13.300 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T01:37:13.331 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T01:37:13.339 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T01:37:13.368 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T01:37:13.374 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T01:37:13.410 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T01:37:13.449 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-26T01:37:13.450 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T01:37:13.485 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-26T01:37:13.499 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T01:37:13.528 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-26T01:37:13.557 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-26T01:37:13.627 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T01:37:13.662 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T01:37:13.680 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T01:37:13.719 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T01:37:13.773 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T01:37:13.781 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T01:37:13.846 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T01:37:13.877 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T01:37:13.913 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T01:37:13.961 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T01:37:13.997 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T01:37:14.040 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T01:37:14.082 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T01:37:14.126 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T01:37:14.182 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T01:37:14.220 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T01:37:14.271 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T01:37:14.301 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T01:37:14.342 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T01:37:14.436 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T01:37:14.476 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T01:37:14.516 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T01:37:14.541 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T01:37:14.566 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-26T01:37:14.641 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T01:37:14.704 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T01:37:14.767 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T01:37:14.786 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T01:37:14.799 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T01:37:14.815 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T01:37:14.818 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T01:37:14.852 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T01:37:14.857 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T01:37:14.891 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T01:37:14.895 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-26T01:37:14.923 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T01:37:14.938 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T01:37:14.978 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T01:37:15.013 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T01:37:15.046 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T01:37:15.087 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T01:37:15.101 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T01:37:15.122 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T01:37:15.148 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T01:37:15.203 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T01:37:15.209 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T01:37:15.276 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T01:37:15.306 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T01:37:15.343 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T01:37:15.389 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T01:37:15.420 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T01:37:15.426 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T01:37:15.451 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T01:37:15.469 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T01:37:15.498 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T01:37:15.511 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T01:37:15.597 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T01:37:15.648 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T01:37:15.651 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T01:37:15.691 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T01:37:15.721 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T01:37:15.763 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T01:37:15.856 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T01:37:15.887 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T01:37:15.927 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T01:37:15.953 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T01:37:15.977 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-26T01:37:16.053 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T01:37:16.180 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T01:37:16.218 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T01:37:16.246 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T01:37:16.280 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T01:37:16.319 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T01:37:16.359 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T01:37:16.377 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T01:37:16.389 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T01:37:16.423 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T01:37:16.424 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T01:37:16.456 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T01:37:16.457 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T01:37:16.498 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T01:37:16.533 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T01:37:16.556 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T01:37:16.607 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T01:37:16.619 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T01:37:16.671 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T01:37:16.694 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T01:37:16.702 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T01:37:16.749 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T01:37:16.756 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T01:37:16.848 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T01:37:16.895 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T01:37:16.929 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T01:37:16.966 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T01:37:17.043 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T01:37:17.178 INFO:teuthology.orchestra.run.smithi151.stdout:(8/152): ceph-common-19.0.0-3156.gd69f13d8.el7. 2.0 MB/s | 21 MB 00:10 2024-04-26T01:37:17.278 INFO:teuthology.orchestra.run.smithi151.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 254 kB/s | 25 kB 00:00 2024-04-26T01:37:17.628 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T01:37:17.675 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T01:37:17.689 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T01:37:17.709 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T01:37:17.809 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T01:37:17.810 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T01:37:17.858 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T01:37:17.948 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T01:37:17.977 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T01:37:18.155 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T01:37:18.183 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T01:37:18.261 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T01:37:18.311 INFO:teuthology.orchestra.run.smithi151.stdout:(10/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el 1.5 MB/s | 11 MB 00:07 2024-04-26T01:37:18.570 INFO:teuthology.orchestra.run.smithi151.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 872 kB/s | 225 kB 00:00 2024-04-26T01:37:18.687 INFO:teuthology.orchestra.run.smithi151.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 233 kB/s | 27 kB 00:00 2024-04-26T01:37:18.787 INFO:teuthology.orchestra.run.smithi151.stdout:(13/152): libcephfs-devel-19.0.0-3156.gd69f13d8 320 kB/s | 32 kB 00:00 2024-04-26T01:37:18.910 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T01:37:19.031 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T01:37:19.038 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T01:37:19.121 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T01:37:19.148 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T01:37:19.181 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T01:37:19.338 INFO:teuthology.orchestra.run.smithi151.stdout:(14/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 1.3 MB/s | 747 kB 00:00 2024-04-26T01:37:19.496 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T01:37:19.513 INFO:teuthology.orchestra.run.smithi151.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 937 kB/s | 163 kB 00:00 2024-04-26T01:37:19.544 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-26T01:37:19.635 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-26T01:37:19.666 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-26T01:37:19.680 INFO:teuthology.orchestra.run.smithi151.stdout:(16/152): librados-devel-19.0.0-3156.gd69f13d8. 765 kB/s | 126 kB 00:00 2024-04-26T01:37:19.702 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T01:37:20.055 INFO:teuthology.orchestra.run.smithi151.stdout:(17/152): libradosstriper1-19.0.0-3156.gd69f13d 1.3 MB/s | 500 kB 00:00 2024-04-26T01:37:20.181 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-26T01:37:20.214 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-26T01:37:20.264 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T01:37:20.285 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-26T01:37:20.307 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-26T01:37:20.344 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T01:37:20.372 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T01:37:20.405 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T01:37:20.716 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T01:37:20.759 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-26T01:37:20.849 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-26T01:37:20.881 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-26T01:37:20.917 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T01:37:21.394 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-26T01:37:21.420 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-26T01:37:21.491 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-26T01:37:21.513 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-26T01:37:22.207 INFO:teuthology.orchestra.run.smithi151.stdout:(18/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x8 1.4 MB/s | 17 MB 00:12 2024-04-26T01:37:22.431 INFO:teuthology.orchestra.run.smithi151.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 203 kB/s | 45 kB 00:00 2024-04-26T01:37:22.628 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-26T01:37:22.662 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-26T01:37:22.739 INFO:teuthology.orchestra.run.smithi151.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 438 kB/s | 135 kB 00:00 2024-04-26T01:37:23.114 INFO:teuthology.orchestra.run.smithi151.stdout:(21/152): python3-cephfs-19.0.0-3156.gd69f13d8. 432 kB/s | 161 kB 00:00 2024-04-26T01:37:23.207 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-26T01:37:23.249 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-26T01:37:23.333 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-26T01:37:23.333 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T01:37:23.333 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T01:37:23.333 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-26T01:37:23.367 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-26T01:37:23.807 INFO:teuthology.orchestra.run.smithi151.stdout:(22/152): python3-rados-19.0.0-3156.gd69f13d8.e 464 kB/s | 320 kB 00:00 2024-04-26T01:37:23.824 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-26T01:37:23.859 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-26T01:37:24.068 INFO:teuthology.orchestra.run.smithi151.stdout:(23/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 1.3 MB/s | 5.4 MB 00:04 2024-04-26T01:37:24.166 INFO:teuthology.orchestra.run.smithi151.stdout:(24/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 1.0 MB/s | 99 kB 00:00 2024-04-26T01:37:24.249 INFO:teuthology.orchestra.run.smithi151.stdout:(25/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.0 MB/s | 85 kB 00:00 2024-04-26T01:37:24.341 INFO:teuthology.orchestra.run.smithi151.stdout:(26/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 558 kB/s | 298 kB 00:00 2024-04-26T01:37:24.409 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-26T01:37:24.454 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-26T01:37:24.546 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-26T01:37:24.547 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T01:37:24.547 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T01:37:24.547 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-26T01:37:24.572 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-26T01:37:24.641 INFO:teuthology.orchestra.run.smithi151.stdout:(27/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 568 kB/s | 170 kB 00:00 2024-04-26T01:37:24.758 INFO:teuthology.orchestra.run.smithi151.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3156.g 294 kB/s | 34 kB 00:00 2024-04-26T01:37:25.050 INFO:teuthology.orchestra.run.smithi151.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 489 kB/s | 142 kB 00:00 2024-04-26T01:37:26.475 INFO:teuthology.orchestra.run.smithi151.stdout:(30/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 1.4 MB/s | 3.1 MB 00:02 2024-04-26T01:37:29.667 INFO:teuthology.orchestra.run.smithi151.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 785 kB/s | 3.5 MB 00:04 2024-04-26T01:37:30.376 INFO:teuthology.orchestra.run.smithi151.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 352 kB/s | 249 kB 00:00 2024-04-26T01:37:30.493 INFO:teuthology.orchestra.run.smithi151.stdout:(33/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 426 kB/s | 49 kB 00:00 2024-04-26T01:37:30.610 INFO:teuthology.orchestra.run.smithi151.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3156.gd 142 kB/s | 16 kB 00:00 2024-04-26T01:37:31.002 INFO:teuthology.orchestra.run.smithi151.stdout:(35/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 699 kB/s | 273 kB 00:00 2024-04-26T01:37:31.869 INFO:teuthology.orchestra.run.smithi151.stdout:(36/152): ceph-mgr-diskprediction-local-19.0.0- 1.4 MB/s | 7.4 MB 00:05 2024-04-26T01:37:31.960 INFO:teuthology.orchestra.run.smithi151.stdout:(37/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 796 kB/s | 762 kB 00:00 2024-04-26T01:37:32.002 INFO:teuthology.orchestra.run.smithi151.stdout:(38/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 137 kB/s | 18 kB 00:00 2024-04-26T01:37:32.253 INFO:teuthology.orchestra.run.smithi151.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 276 kB/s | 80 kB 00:00 2024-04-26T01:37:32.353 INFO:teuthology.orchestra.run.smithi151.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.9 MB/s | 195 kB 00:00 2024-04-26T01:37:32.420 INFO:teuthology.orchestra.run.smithi151.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.9 MB/s | 253 kB 00:00 2024-04-26T01:37:32.537 INFO:teuthology.orchestra.run.smithi151.stdout:(42/152): libgfortran-11.4.1-3.el9.x86_64.rpm 1.5 MB/s | 803 kB 00:00 2024-04-26T01:37:32.595 INFO:teuthology.orchestra.run.smithi151.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 7.1 MB/s | 1.2 MB 00:00 2024-04-26T01:37:32.621 INFO:teuthology.orchestra.run.smithi151.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.3 MB/s | 106 kB 00:00 2024-04-26T01:37:32.646 INFO:teuthology.orchestra.run.smithi151.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.7 MB/s | 135 kB 00:00 2024-04-26T01:37:32.671 INFO:teuthology.orchestra.run.smithi151.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.5 MB/s | 125 kB 00:00 2024-04-26T01:37:32.722 INFO:teuthology.orchestra.run.smithi151.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.9 MB/s | 215 kB 00:00 2024-04-26T01:37:32.772 INFO:teuthology.orchestra.run.smithi151.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 6.9 MB/s | 689 kB 00:00 2024-04-26T01:37:32.822 INFO:teuthology.orchestra.run.smithi151.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 9.9 MB/s | 943 kB 00:00 2024-04-26T01:37:32.847 INFO:teuthology.orchestra.run.smithi151.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-04-26T01:37:32.881 INFO:teuthology.orchestra.run.smithi151.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 378 kB/s | 21 kB 00:00 2024-04-26T01:37:32.940 INFO:teuthology.orchestra.run.smithi151.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 3.4 MB/s | 313 kB 00:00 2024-04-26T01:37:32.956 INFO:teuthology.orchestra.run.smithi151.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 437 kB/s | 33 kB 00:00 2024-04-26T01:37:33.080 INFO:teuthology.orchestra.run.smithi151.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 148 kB/s | 18 kB 00:00 2024-04-26T01:37:33.174 INFO:teuthology.orchestra.run.smithi151.stdout:(55/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 13 MB/s | 3.0 MB 00:00 2024-04-26T01:37:33.199 INFO:teuthology.orchestra.run.smithi151.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 100 kB/s | 11 kB 00:00 2024-04-26T01:37:33.241 INFO:teuthology.orchestra.run.smithi151.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.4 MB/s | 160 kB 00:00 2024-04-26T01:37:33.275 INFO:teuthology.orchestra.run.smithi151.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 605 kB/s | 45 kB 00:00 2024-04-26T01:37:33.334 INFO:teuthology.orchestra.run.smithi151.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 7.1 MB/s | 662 kB 00:00 2024-04-26T01:37:33.376 INFO:teuthology.orchestra.run.smithi151.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 6.1 MB/s | 582 kB 00:00 2024-04-26T01:37:33.401 INFO:teuthology.orchestra.run.smithi151.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 417 kB/s | 25 kB 00:00 2024-04-26T01:37:33.451 INFO:teuthology.orchestra.run.smithi151.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 640 kB/s | 47 kB 00:00 2024-04-26T01:37:33.485 INFO:teuthology.orchestra.run.smithi151.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.5 MB/s | 292 kB 00:00 2024-04-26T01:37:33.527 INFO:teuthology.orchestra.run.smithi151.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 2.6 MB/s | 188 kB 00:00 2024-04-26T01:37:33.552 INFO:teuthology.orchestra.run.smithi151.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 532 kB/s | 35 kB 00:00 2024-04-26T01:37:33.761 INFO:teuthology.orchestra.run.smithi151.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 20 MB/s | 4.7 MB 00:00 2024-04-26T01:37:33.894 INFO:teuthology.orchestra.run.smithi151.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-04-26T01:37:33.961 INFO:teuthology.orchestra.run.smithi151.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 8.6 MB/s | 1.6 MB 00:00 2024-04-26T01:37:34.003 INFO:teuthology.orchestra.run.smithi151.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 9.5 MB/s | 911 kB 00:00 2024-04-26T01:37:34.028 INFO:teuthology.orchestra.run.smithi151.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-04-26T01:37:34.196 INFO:teuthology.orchestra.run.smithi151.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 6.2 MB/s | 1.0 MB 00:00 2024-04-26T01:37:34.454 INFO:teuthology.orchestra.run.smithi151.stdout:(72/152): postgresql-server-13.14-1.el9.x86_64. 13 MB/s | 5.8 MB 00:00 2024-04-26T01:37:34.604 INFO:teuthology.orchestra.run.smithi151.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 222 kB/s | 87 kB 00:00 2024-04-26T01:37:34.771 INFO:teuthology.orchestra.run.smithi151.stdout:(74/152): python3-babel-2.9.1-2.el9.noarch.rpm 19 MB/s | 6.0 MB 00:00 2024-04-26T01:37:34.805 INFO:teuthology.orchestra.run.smithi151.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.3 MB/s | 244 kB 00:00 2024-04-26T01:37:34.839 INFO:teuthology.orchestra.run.smithi151.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 3.8 MB/s | 248 kB 00:00 2024-04-26T01:37:34.864 INFO:teuthology.orchestra.run.smithi151.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 462 kB/s | 26 kB 00:00 2024-04-26T01:37:34.920 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-26T01:37:34.920 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /sys 2024-04-26T01:37:34.921 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /proc 2024-04-26T01:37:34.921 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /mnt 2024-04-26T01:37:34.921 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /var/tmp 2024-04-26T01:37:34.921 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /home 2024-04-26T01:37:34.921 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /root 2024-04-26T01:37:34.921 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /tmp 2024-04-26T01:37:34.921 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-26T01:37:34.922 INFO:teuthology.orchestra.run.smithi151.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 232 kB/s | 19 kB 00:00 2024-04-26T01:37:34.956 INFO:teuthology.orchestra.run.smithi151.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 1.8 MB/s | 172 kB 00:00 2024-04-26T01:37:34.982 INFO:teuthology.orchestra.run.smithi151.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 620 kB/s | 35 kB 00:00 2024-04-26T01:37:35.087 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-26T01:37:35.102 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-26T01:37:35.102 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:37:35.102 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T01:37:35.103 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-26T01:37:35.260 INFO:teuthology.orchestra.run.smithi151.stdout:(81/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 17 MB/s | 5.1 MB 00:00 2024-04-26T01:37:35.290 INFO:teuthology.orchestra.run.smithi151.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 818 kB/s | 250 kB 00:00 2024-04-26T01:37:35.324 INFO:teuthology.orchestra.run.smithi151.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.6 MB/s | 222 kB 00:00 2024-04-26T01:37:35.341 INFO:teuthology.orchestra.run.smithi151.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.6 MB/s | 77 kB 00:00 2024-04-26T01:37:35.366 INFO:teuthology.orchestra.run.smithi151.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-04-26T01:37:35.408 INFO:teuthology.orchestra.run.smithi151.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 4.0 MB/s | 269 kB 00:00 2024-04-26T01:37:35.442 INFO:teuthology.orchestra.run.smithi151.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-04-26T01:37:35.475 INFO:teuthology.orchestra.run.smithi151.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 4.2 MB/s | 279 kB 00:00 2024-04-26T01:37:35.509 INFO:teuthology.orchestra.run.smithi151.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 800 kB/s | 51 kB 00:00 2024-04-26T01:37:35.535 INFO:teuthology.orchestra.run.smithi151.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 933 kB/s | 54 kB 00:00 2024-04-26T01:37:35.643 INFO:teuthology.orchestra.run.smithi151.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 388 kB/s | 42 kB 00:00 2024-04-26T01:37:35.918 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-26T01:37:35.918 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /sys 2024-04-26T01:37:35.918 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /proc 2024-04-26T01:37:35.918 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /mnt 2024-04-26T01:37:35.918 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /var/tmp 2024-04-26T01:37:35.918 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /home 2024-04-26T01:37:35.918 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /root 2024-04-26T01:37:35.918 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /tmp 2024-04-26T01:37:35.918 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-26T01:37:36.044 INFO:teuthology.orchestra.run.smithi151.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 112 kB/s | 44 kB 00:00 2024-04-26T01:37:36.115 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-26T01:37:36.131 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-26T01:37:36.131 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:37:36.131 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T01:37:36.131 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-26T01:37:36.486 INFO:teuthology.orchestra.run.smithi151.stdout:(93/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 17 MB/s | 16 MB 00:00 2024-04-26T01:37:36.519 INFO:teuthology.orchestra.run.smithi151.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 479 kB/s | 221 kB 00:00 2024-04-26T01:37:36.544 INFO:teuthology.orchestra.run.smithi151.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 643 kB/s | 36 kB 00:00 2024-04-26T01:37:36.570 INFO:teuthology.orchestra.run.smithi151.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 963 kB/s | 47 kB 00:00 2024-04-26T01:37:36.590 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-26T01:37:36.603 INFO:teuthology.orchestra.run.smithi151.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.2 MB/s | 305 kB 00:00 2024-04-26T01:37:36.617 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-26T01:37:36.617 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:37:36.617 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T01:37:36.617 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T01:37:36.617 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T01:37:36.617 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-26T01:37:36.628 INFO:teuthology.orchestra.run.smithi151.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.0 MB/s | 58 kB 00:00 2024-04-26T01:37:36.654 INFO:teuthology.orchestra.run.smithi151.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.9 MB/s | 94 kB 00:00 2024-04-26T01:37:36.679 INFO:teuthology.orchestra.run.smithi151.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.3 MB/s | 64 kB 00:00 2024-04-26T01:37:36.729 INFO:teuthology.orchestra.run.smithi151.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 607 kB/s | 45 kB 00:00 2024-04-26T01:37:36.755 INFO:teuthology.orchestra.run.smithi151.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 296 kB/s | 22 kB 00:00 2024-04-26T01:37:36.851 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-26T01:37:36.873 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-26T01:37:36.873 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:37:36.873 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T01:37:36.874 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T01:37:36.874 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T01:37:36.874 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-26T01:37:36.947 INFO:teuthology.orchestra.run.smithi151.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 3.9 MB/s | 863 kB 00:00 2024-04-26T01:37:37.074 INFO:teuthology.orchestra.run.smithi151.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.7 MB/s | 548 kB 00:00 2024-04-26T01:37:37.123 INFO:teuthology.orchestra.run.smithi151.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 407 kB/s | 19 kB 00:00 2024-04-26T01:37:37.154 INFO:teuthology.orchestra.run.smithi151.stdout:[MIRROR] libarrow-9.0.0-10.el9.x86_64.rpm: Status code: 404 for http://ohioix.mm.fcix.net/epel/9/Everything/x86_64/Packages/l/libarrow-9.0.0-10.el9.x86_64.rpm (IP: 134.195.207.11) 2024-04-26T01:37:37.190 INFO:teuthology.orchestra.run.smithi151.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.3 MB/s | 309 kB 00:00 2024-04-26T01:37:37.290 INFO:teuthology.orchestra.run.smithi151.stdout:[MIRROR] libarrow-doc-9.0.0-10.el9.noarch.rpm: Status code: 404 for http://ohioix.mm.fcix.net/epel/9/Everything/x86_64/Packages/l/libarrow-doc-9.0.0-10.el9.noarch.rpm (IP: 134.195.207.11) 2024-04-26T01:37:37.318 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-26T01:37:37.319 INFO:teuthology.orchestra.run.smithi151.stdout:[MIRROR] libarrow-9.0.0-10.el9.x86_64.rpm: Status code: 404 for https://ohioix.mm.fcix.net/epel/9/Everything/x86_64/Packages/l/libarrow-9.0.0-10.el9.x86_64.rpm (IP: 134.195.207.11) 2024-04-26T01:37:37.340 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-26T01:37:37.340 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:37:37.340 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T01:37:37.341 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T01:37:37.341 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T01:37:37.341 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-26T01:37:37.381 INFO:teuthology.orchestra.run.smithi151.stdout:[MIRROR] libarrow-9.0.0-10.el9.x86_64.rpm: Status code: 404 for http://forksystems.mm.fcix.net/epel/9/Everything/x86_64/Packages/l/libarrow-9.0.0-10.el9.x86_64.rpm (IP: 23.147.64.7) 2024-04-26T01:37:37.450 INFO:teuthology.orchestra.run.smithi151.stdout:[MIRROR] libarrow-doc-9.0.0-10.el9.noarch.rpm: Status code: 404 for https://ohioix.mm.fcix.net/epel/9/Everything/x86_64/Packages/l/libarrow-doc-9.0.0-10.el9.noarch.rpm (IP: 134.195.207.11) 2024-04-26T01:37:37.463 INFO:teuthology.orchestra.run.smithi151.stdout:[MIRROR] libarrow-doc-9.0.0-10.el9.noarch.rpm: Status code: 404 for http://forksystems.mm.fcix.net/epel/9/Everything/x86_64/Packages/l/libarrow-doc-9.0.0-10.el9.noarch.rpm (IP: 23.147.64.7) 2024-04-26T01:37:37.464 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T01:37:37.482 INFO:teuthology.orchestra.run.smithi151.stdout:[MIRROR] libarrow-9.0.0-10.el9.x86_64.rpm: Status code: 404 for https://forksystems.mm.fcix.net/epel/9/Everything/x86_64/Packages/l/libarrow-9.0.0-10.el9.x86_64.rpm (IP: 23.147.64.7) 2024-04-26T01:37:37.513 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T01:37:37.568 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T01:37:37.592 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-26T01:37:37.615 INFO:teuthology.orchestra.run.smithi151.stdout:[MIRROR] libarrow-doc-9.0.0-10.el9.noarch.rpm: Status code: 404 for https://forksystems.mm.fcix.net/epel/9/Everything/x86_64/Packages/l/libarrow-doc-9.0.0-10.el9.noarch.rpm (IP: 23.147.64.7) 2024-04-26T01:37:37.618 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-26T01:37:37.618 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:37:37.618 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T01:37:37.618 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T01:37:37.618 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T01:37:37.618 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-26T01:37:37.743 INFO:teuthology.orchestra.run.smithi151.stdout:[MIRROR] libarrow-9.0.0-10.el9.x86_64.rpm: Status code: 404 for https://pubmirror2.math.uh.edu/fedora-buffet/epel/9/Everything/x86_64/Packages/l/libarrow-9.0.0-10.el9.x86_64.rpm (IP: 129.7.128.190) 2024-04-26T01:37:37.755 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T01:37:37.827 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T01:37:37.834 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-26T01:37:37.841 INFO:teuthology.orchestra.run.smithi151.stdout:[MIRROR] libarrow-doc-9.0.0-10.el9.noarch.rpm: Status code: 404 for https://pubmirror2.math.uh.edu/fedora-buffet/epel/9/Everything/x86_64/Packages/l/libarrow-doc-9.0.0-10.el9.noarch.rpm (IP: 129.7.128.190) 2024-04-26T01:37:37.842 INFO:teuthology.orchestra.run.smithi151.stdout:[MIRROR] libarrow-9.0.0-10.el9.x86_64.rpm: Status code: 404 for http://pubmirror2.math.uh.edu/fedora-buffet/epel/9/Everything/x86_64/Packages/l/libarrow-9.0.0-10.el9.x86_64.rpm (IP: 129.7.128.190) 2024-04-26T01:37:37.857 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-26T01:37:37.857 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:37:37.858 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T01:37:37.858 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T01:37:37.858 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T01:37:37.858 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-26T01:37:37.886 INFO:teuthology.orchestra.run.smithi151.stdout:[MIRROR] libarrow-doc-9.0.0-10.el9.noarch.rpm: Status code: 404 for http://pubmirror2.math.uh.edu/fedora-buffet/epel/9/Everything/x86_64/Packages/l/libarrow-doc-9.0.0-10.el9.noarch.rpm (IP: 129.7.128.190) 2024-04-26T01:37:37.905 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T01:37:37.993 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T01:37:38.073 INFO:teuthology.orchestra.run.smithi151.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 32 kB/s | 26 kB 00:00 2024-04-26T01:37:38.124 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T01:37:38.169 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T01:37:38.182 INFO:teuthology.orchestra.run.smithi151.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 451 kB/s | 49 kB 00:00 2024-04-26T01:37:38.201 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T01:37:38.229 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T01:37:38.265 INFO:teuthology.orchestra.run.smithi151.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 817 kB/s | 67 kB 00:00 2024-04-26T01:37:38.293 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-26T01:37:38.316 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-26T01:37:38.316 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:37:38.316 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T01:37:38.316 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T01:37:38.316 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T01:37:38.316 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-26T01:37:38.365 INFO:teuthology.orchestra.run.smithi151.stdout:(110/152): luarocks-3.9.2-1.el9.noarch.rpm 1.5 MB/s | 151 kB 00:00 2024-04-26T01:37:38.397 INFO:teuthology.orchestra.run.smithi151.stdout:[MIRROR] parquet-libs-9.0.0-10.el9.x86_64.rpm: Status code: 404 for https://ohioix.mm.fcix.net/epel/9/Everything/x86_64/Packages/p/parquet-libs-9.0.0-10.el9.x86_64.rpm (IP: 134.195.207.11) 2024-04-26T01:37:38.423 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T01:37:38.428 INFO:teuthology.orchestra.run.smithi151.stdout:[MIRROR] parquet-libs-9.0.0-10.el9.x86_64.rpm: Status code: 404 for http://ohioix.mm.fcix.net/epel/9/Everything/x86_64/Packages/p/parquet-libs-9.0.0-10.el9.x86_64.rpm (IP: 134.195.207.11) 2024-04-26T01:37:38.439 INFO:teuthology.orchestra.run.smithi151.stdout:[MIRROR] parquet-libs-9.0.0-10.el9.x86_64.rpm: Status code: 404 for http://forksystems.mm.fcix.net/epel/9/Everything/x86_64/Packages/p/parquet-libs-9.0.0-10.el9.x86_64.rpm (IP: 23.147.64.7) 2024-04-26T01:37:38.453 INFO:teuthology.orchestra.run.smithi151.stdout:[MIRROR] parquet-libs-9.0.0-10.el9.x86_64.rpm: Status code: 404 for https://forksystems.mm.fcix.net/epel/9/Everything/x86_64/Packages/p/parquet-libs-9.0.0-10.el9.x86_64.rpm (IP: 23.147.64.7) 2024-04-26T01:37:38.471 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T01:37:38.493 INFO:teuthology.orchestra.run.smithi151.stdout:[MIRROR] parquet-libs-9.0.0-10.el9.x86_64.rpm: Status code: 404 for https://pubmirror2.math.uh.edu/fedora-buffet/epel/9/Everything/x86_64/Packages/p/parquet-libs-9.0.0-10.el9.x86_64.rpm (IP: 129.7.128.190) 2024-04-26T01:37:38.527 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T01:37:38.532 INFO:teuthology.orchestra.run.smithi151.stdout:[MIRROR] parquet-libs-9.0.0-10.el9.x86_64.rpm: Status code: 404 for http://pubmirror2.math.uh.edu/fedora-buffet/epel/9/Everything/x86_64/Packages/p/parquet-libs-9.0.0-10.el9.x86_64.rpm (IP: 129.7.128.190) 2024-04-26T01:37:38.664 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-26T01:37:38.694 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-26T01:37:38.708 INFO:teuthology.orchestra.run.smithi151.stdout:(111/152): libarrow-9.0.0-10.el9.x86_64.rpm 2.8 MB/s | 4.4 MB 00:01 2024-04-26T01:37:38.714 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T01:37:38.718 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-26T01:37:38.761 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T01:37:38.788 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-26T01:37:38.817 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-26T01:37:38.825 INFO:teuthology.orchestra.run.smithi151.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 4.6 MB/s | 548 kB 00:00 2024-04-26T01:37:38.837 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T01:37:38.900 INFO:teuthology.orchestra.run.smithi151.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 583 kB/s | 43 kB 00:00 2024-04-26T01:37:38.928 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T01:37:38.959 INFO:teuthology.orchestra.run.smithi151.stdout:(114/152): python3-cachetools-4.2.4-1.el9.noarc 559 kB/s | 32 kB 00:00 2024-04-26T01:37:39.017 INFO:teuthology.orchestra.run.smithi151.stdout:(115/152): python3-certifi-2021.10.8-2.el9.noar 259 kB/s | 15 kB 00:00 2024-04-26T01:37:39.067 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T01:37:39.078 INFO:teuthology.orchestra.run.smithi151.stdout:(116/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-04-26T01:37:39.110 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T01:37:39.142 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T01:37:39.143 INFO:teuthology.orchestra.run.smithi151.stdout:(117/152): python3-cherrypy-18.6.1-2.el9.noarch 5.3 MB/s | 358 kB 00:00 2024-04-26T01:37:39.170 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T01:37:39.241 INFO:teuthology.orchestra.run.smithi151.stdout:(118/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.0 MB/s | 303 kB 00:00 2024-04-26T01:37:39.302 INFO:teuthology.orchestra.run.smithi151.stdout:(119/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 896 kB/s | 838 kB 00:00 2024-04-26T01:37:39.319 INFO:teuthology.orchestra.run.smithi151.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 460 kB/s | 35 kB 00:00 2024-04-26T01:37:39.480 INFO:teuthology.orchestra.run.smithi151.stdout:(121/152): python3-grpcio-1.46.7-10.el9.x86_64. 13 MB/s | 2.0 MB 00:00 2024-04-26T01:37:39.536 INFO:teuthology.orchestra.run.smithi151.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 2.6 MB/s | 144 kB 00:00 2024-04-26T01:37:39.612 INFO:teuthology.orchestra.run.smithi151.stdout:(123/152): python3-google-auth-2.29.0-1.el9.noa 667 kB/s | 206 kB 00:00 2024-04-26T01:37:39.621 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-26T01:37:39.637 INFO:teuthology.orchestra.run.smithi151.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 107 kB/s | 11 kB 00:00 2024-04-26T01:37:39.652 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-26T01:37:39.670 INFO:teuthology.orchestra.run.smithi151.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 306 kB/s | 18 kB 00:00 2024-04-26T01:37:39.684 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-26T01:37:39.704 INFO:teuthology.orchestra.run.smithi151.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 350 kB/s | 23 kB 00:00 2024-04-26T01:37:39.729 INFO:teuthology.orchestra.run.smithi151.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 336 kB/s | 19 kB 00:00 2024-04-26T01:37:39.744 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-26T01:37:39.771 INFO:teuthology.orchestra.run.smithi151.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 298 kB/s | 20 kB 00:00 2024-04-26T01:37:39.783 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-26T01:37:39.830 INFO:teuthology.orchestra.run.smithi151.stdout:(129/152): python3-kubernetes-26.1.0-2.el9.noar 10 MB/s | 1.0 MB 00:00 2024-04-26T01:37:39.854 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-26T01:37:39.855 INFO:teuthology.orchestra.run.smithi151.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 559 kB/s | 46 kB 00:00 2024-04-26T01:37:39.881 INFO:teuthology.orchestra.run.smithi151.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.6 MB/s | 79 kB 00:00 2024-04-26T01:37:39.885 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-26T01:37:39.981 INFO:teuthology.orchestra.run.smithi151.stdout:(132/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.7 MB/s | 272 kB 00:00 2024-04-26T01:37:40.014 INFO:teuthology.orchestra.run.smithi151.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 365 kB/s | 58 kB 00:00 2024-04-26T01:37:40.040 INFO:teuthology.orchestra.run.smithi151.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 283 kB/s | 16 kB 00:00 2024-04-26T01:37:40.101 INFO:teuthology.orchestra.run.smithi151.stdout:(135/152): python3-pyperclip-1.8.2-2.el9.noarch 489 kB/s | 28 kB 00:00 2024-04-26T01:37:40.123 INFO:teuthology.orchestra.run.smithi151.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 832 kB/s | 90 kB 00:00 2024-04-26T01:37:40.157 INFO:teuthology.orchestra.run.smithi151.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 534 kB/s | 31 kB 00:00 2024-04-26T01:37:40.216 INFO:teuthology.orchestra.run.smithi151.stdout:(138/152): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-04-26T01:37:40.275 INFO:teuthology.orchestra.run.smithi151.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-04-26T01:37:40.302 INFO:teuthology.orchestra.run.smithi151.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 431 kB/s | 36 kB 00:00 2024-04-26T01:37:40.359 INFO:teuthology.orchestra.run.smithi151.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 617 kB/s | 51 kB 00:00 2024-04-26T01:37:40.392 INFO:teuthology.orchestra.run.smithi151.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.5 MB/s | 230 kB 00:00 2024-04-26T01:37:40.459 INFO:teuthology.orchestra.run.smithi151.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 902 kB/s | 90 kB 00:00 2024-04-26T01:37:40.492 INFO:teuthology.orchestra.run.smithi151.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 4.2 MB/s | 427 kB 00:00 2024-04-26T01:37:40.526 INFO:teuthology.orchestra.run.smithi151.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 302 kB/s | 20 kB 00:00 2024-04-26T01:37:40.560 INFO:teuthology.orchestra.run.smithi151.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-04-26T01:37:40.830 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-26T01:37:40.847 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-26T01:37:40.868 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-26T01:37:40.878 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-26T01:37:40.979 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-26T01:37:41.074 INFO:teuthology.orchestra.run.smithi151.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 2.9 MB/s | 1.6 MB 00:00 2024-04-26T01:37:41.149 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-26T01:37:41.180 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-26T01:37:41.202 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-26T01:37:41.203 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:37:41.203 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T01:37:41.203 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T01:37:41.203 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T01:37:41.203 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-26T01:37:41.247 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-26T01:37:41.258 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-26T01:37:41.356 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-26T01:37:41.404 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-26T01:37:41.838 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-26T01:37:41.868 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-26T01:37:41.973 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-26T01:37:42.132 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-26T01:37:42.170 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-26T01:37:42.193 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-26T01:37:42.194 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:37:42.194 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T01:37:42.194 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T01:37:42.194 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T01:37:42.194 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-26T01:37:42.254 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-26T01:37:42.266 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-26T01:37:42.277 INFO:teuthology.orchestra.run.smithi151.stdout:(148/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 2.6 MB/s | 3.1 MB 00:01 2024-04-26T01:37:42.334 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T01:37:42.365 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-26T01:37:42.366 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-26T01:37:42.389 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-26T01:37:42.390 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:37:42.390 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T01:37:42.390 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T01:37:42.390 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T01:37:42.390 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-26T01:37:42.420 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-26T01:37:42.434 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-26T01:37:42.456 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-26T01:37:42.456 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:37:42.456 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T01:37:42.456 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-26T01:37:42.501 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-26T01:37:42.524 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-26T01:37:42.524 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:37:42.524 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T01:37:42.524 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T01:37:42.524 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T01:37:42.524 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-26T01:37:42.552 INFO:teuthology.orchestra.run.smithi151.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 4.9 MB/s | 1.3 MB 00:00 2024-04-26T01:37:42.686 INFO:teuthology.orchestra.run.smithi151.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 18 MB/s | 2.4 MB 00:00 2024-04-26T01:37:42.827 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-26T01:37:42.849 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-26T01:37:42.849 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:37:42.850 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T01:37:42.850 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T01:37:42.850 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T01:37:42.850 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-26T01:37:42.937 INFO:teuthology.orchestra.run.smithi151.stdout:(151/152): librados2-19.0.0-3156.gd69f13d8.el7. 1.4 MB/s | 3.4 MB 00:02 2024-04-26T01:37:43.342 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T01:37:43.373 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-26T01:37:43.396 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-26T01:37:43.397 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:37:43.397 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T01:37:43.397 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T01:37:43.397 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T01:37:43.397 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-26T01:37:43.441 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-26T01:37:43.463 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-26T01:37:43.463 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:37:43.463 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T01:37:43.463 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-26T01:37:43.508 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-26T01:37:43.531 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-26T01:37:43.531 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:37:43.531 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T01:37:43.531 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T01:37:43.532 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T01:37:43.532 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-26T01:37:43.736 INFO:teuthology.orchestra.run.smithi151.stdout:(152/152): ceph-test-19.0.0-3156.gd69f13d8.el7. 1.9 MB/s | 49 MB 00:26 2024-04-26T01:37:43.743 INFO:teuthology.orchestra.run.smithi151.stdout:-------------------------------------------------------------------------------- 2024-04-26T01:37:43.744 INFO:teuthology.orchestra.run.smithi151.stdout:Total 5.8 MB/s | 222 MB 00:38 2024-04-26T01:37:43.842 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-26T01:37:43.865 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-26T01:37:43.865 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:37:43.865 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T01:37:43.865 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T01:37:43.865 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T01:37:43.865 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-26T01:37:46.608 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction check 2024-04-26T01:37:46.860 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction check succeeded. 2024-04-26T01:37:46.860 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction test 2024-04-26T01:37:47.438 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-26T01:37:47.480 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-26T01:37:47.521 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-26T01:37:47.553 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T01:37:47.604 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T01:37:47.780 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-26T01:37:47.826 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T01:37:47.856 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-26T01:37:48.447 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T01:37:48.478 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-26T01:37:48.487 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T01:37:48.507 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T01:37:48.519 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-26T01:37:48.569 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-26T01:37:48.601 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T01:37:48.628 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T01:37:48.812 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-26T01:37:48.857 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T01:37:48.895 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-26T01:37:49.348 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-26T01:37:49.434 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T01:37:49.471 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T01:37:49.475 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T01:37:49.509 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T01:37:49.529 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T01:37:49.556 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T01:37:49.588 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T01:37:49.588 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T01:37:49.600 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction test succeeded. 2024-04-26T01:37:49.601 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction 2024-04-26T01:37:49.664 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T01:37:49.689 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T01:37:49.689 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T01:37:49.764 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T01:37:49.829 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T01:37:50.388 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-26T01:37:50.471 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T01:37:50.512 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T01:37:50.603 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T01:37:50.642 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T01:37:50.642 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T01:37:50.709 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T01:37:50.734 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T01:37:50.734 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T01:37:50.827 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T01:37:50.893 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T01:37:55.628 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T01:37:55.628 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-26T01:37:55.628 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-26T01:37:55.628 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-26T01:37:55.628 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-26T01:37:55.629 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-26T01:37:55.629 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-26T01:37:55.629 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-26T01:37:55.629 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-26T01:37:55.629 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-26T01:37:55.629 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-26T01:37:55.629 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-26T01:37:55.629 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-26T01:37:55.629 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-26T01:37:55.629 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-26T01:37:55.629 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-26T01:37:55.629 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-26T01:37:55.630 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-26T01:37:55.630 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-26T01:37:55.630 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-26T01:37:55.630 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-26T01:37:55.630 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-26T01:37:55.630 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-26T01:37:55.630 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-26T01:37:55.630 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-26T01:37:55.630 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-26T01:37:55.630 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-26T01:37:55.630 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-26T01:37:55.642 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-26T01:37:55.643 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-26T01:37:55.643 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-26T01:37:55.643 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-26T01:37:55.643 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-26T01:37:55.643 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-26T01:37:55.643 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-26T01:37:55.643 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-26T01:37:55.643 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-26T01:37:55.643 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-26T01:37:55.643 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-26T01:37:55.643 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-26T01:37:55.643 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T01:37:55.643 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T01:37:55.643 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T01:37:55.644 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T01:37:55.644 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T01:37:55.644 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T01:37:55.644 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T01:37:55.644 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T01:37:55.644 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T01:37:55.644 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T01:37:55.644 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T01:37:55.644 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T01:37:55.644 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T01:37:55.644 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T01:37:55.644 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T01:37:55.644 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T01:37:55.644 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T01:37:55.645 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T01:37:55.645 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T01:37:55.645 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T01:37:55.645 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T01:37:55.645 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T01:37:55.645 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T01:37:55.645 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T01:37:55.645 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T01:37:55.645 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T01:37:55.645 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T01:37:55.645 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T01:37:55.645 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T01:37:55.645 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T01:37:55.646 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T01:37:55.646 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T01:37:55.646 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T01:37:55.646 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T01:37:55.646 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T01:37:55.646 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T01:37:55.646 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T01:37:55.646 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T01:37:55.646 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T01:37:55.646 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T01:37:55.646 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T01:37:55.646 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T01:37:55.646 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T01:37:55.647 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T01:37:55.647 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T01:37:55.647 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T01:37:55.647 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T01:37:55.647 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T01:37:55.647 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T01:37:55.647 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T01:37:55.647 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T01:37:55.647 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T01:37:55.647 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T01:37:55.647 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T01:37:55.647 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T01:37:55.647 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T01:37:55.647 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T01:37:55.648 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T01:37:55.648 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T01:37:55.648 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T01:37:55.648 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T01:37:55.648 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T01:37:55.648 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T01:37:55.648 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T01:37:55.648 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T01:37:55.648 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T01:37:55.648 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T01:37:55.648 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T01:37:55.648 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-26T01:37:55.649 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-26T01:37:55.649 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T01:37:55.649 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T01:37:55.649 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T01:37:55.649 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-26T01:37:55.649 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T01:37:55.649 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T01:37:55.649 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T01:37:55.649 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T01:37:55.649 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T01:37:55.649 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T01:37:55.649 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T01:37:55.649 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T01:37:55.650 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T01:37:55.650 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T01:37:55.650 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T01:37:55.650 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T01:37:55.650 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T01:37:55.650 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T01:37:55.650 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T01:37:55.650 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T01:37:55.650 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T01:37:55.650 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T01:37:55.650 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T01:37:55.650 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T01:37:55.650 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T01:37:55.651 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T01:37:55.651 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T01:37:55.651 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T01:37:55.651 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T01:37:55.651 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T01:37:55.651 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T01:37:55.651 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T01:37:55.651 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T01:37:55.651 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T01:37:55.651 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T01:37:55.651 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T01:37:55.651 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T01:37:55.651 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T01:37:55.651 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T01:37:55.652 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-26T01:37:55.652 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T01:37:55.652 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-26T01:37:55.652 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T01:37:55.652 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T01:37:55.652 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T01:37:55.652 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T01:37:56.181 INFO:teuthology.orchestra.run.smithi151.stdout: Preparing : 1/1 2024-04-26T01:37:56.276 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T01:37:56.374 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T01:37:56.495 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T01:37:56.495 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-26T01:37:56.495 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-26T01:37:56.495 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-26T01:37:56.495 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-26T01:37:56.495 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-26T01:37:56.495 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-26T01:37:56.496 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-26T01:37:56.496 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-26T01:37:56.496 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-26T01:37:56.496 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-26T01:37:56.496 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-26T01:37:56.496 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-26T01:37:56.496 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-26T01:37:56.496 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-26T01:37:56.496 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-26T01:37:56.496 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-26T01:37:56.496 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-26T01:37:56.496 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-26T01:37:56.496 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-26T01:37:56.497 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-26T01:37:56.497 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-26T01:37:56.497 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-26T01:37:56.497 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-26T01:37:56.497 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-26T01:37:56.497 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-26T01:37:56.497 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-26T01:37:56.497 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-26T01:37:56.497 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-26T01:37:56.497 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-26T01:37:56.497 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-26T01:37:56.505 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T01:37:56.509 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-26T01:37:56.509 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-26T01:37:56.509 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-26T01:37:56.509 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-26T01:37:56.509 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-26T01:37:56.509 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-26T01:37:56.509 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-26T01:37:56.509 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-26T01:37:56.510 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-26T01:37:56.510 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T01:37:56.510 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T01:37:56.510 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T01:37:56.510 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T01:37:56.510 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T01:37:56.510 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T01:37:56.510 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T01:37:56.510 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T01:37:56.510 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T01:37:56.510 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T01:37:56.510 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T01:37:56.510 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T01:37:56.511 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T01:37:56.511 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T01:37:56.511 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T01:37:56.511 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T01:37:56.511 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T01:37:56.511 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T01:37:56.511 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T01:37:56.511 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T01:37:56.511 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T01:37:56.511 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T01:37:56.511 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T01:37:56.511 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T01:37:56.511 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T01:37:56.512 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T01:37:56.512 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T01:37:56.512 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T01:37:56.512 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T01:37:56.512 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T01:37:56.512 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T01:37:56.512 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T01:37:56.512 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T01:37:56.512 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T01:37:56.512 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T01:37:56.512 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T01:37:56.512 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T01:37:56.512 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T01:37:56.513 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T01:37:56.513 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T01:37:56.513 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T01:37:56.513 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T01:37:56.513 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T01:37:56.513 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T01:37:56.513 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T01:37:56.513 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T01:37:56.513 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T01:37:56.513 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T01:37:56.513 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T01:37:56.513 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T01:37:56.513 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T01:37:56.514 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T01:37:56.514 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T01:37:56.514 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T01:37:56.514 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T01:37:56.514 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T01:37:56.514 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T01:37:56.514 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T01:37:56.514 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T01:37:56.514 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T01:37:56.514 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T01:37:56.514 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T01:37:56.514 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T01:37:56.514 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T01:37:56.514 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T01:37:56.515 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T01:37:56.515 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T01:37:56.515 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T01:37:56.515 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-26T01:37:56.515 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-26T01:37:56.515 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T01:37:56.515 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T01:37:56.515 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T01:37:56.515 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-26T01:37:56.515 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T01:37:56.515 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T01:37:56.515 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T01:37:56.515 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T01:37:56.516 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T01:37:56.516 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T01:37:56.516 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T01:37:56.516 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T01:37:56.516 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T01:37:56.516 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T01:37:56.516 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T01:37:56.516 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T01:37:56.516 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T01:37:56.516 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T01:37:56.516 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T01:37:56.516 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T01:37:56.516 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T01:37:56.516 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T01:37:56.517 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T01:37:56.517 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T01:37:56.517 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T01:37:56.517 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T01:37:56.517 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T01:37:56.517 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T01:37:56.517 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T01:37:56.517 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T01:37:56.517 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T01:37:56.517 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T01:37:56.517 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T01:37:56.517 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T01:37:56.517 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T01:37:56.518 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T01:37:56.518 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T01:37:56.518 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T01:37:56.518 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T01:37:56.518 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-26T01:37:56.518 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T01:37:56.518 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-26T01:37:56.518 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T01:37:56.518 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T01:37:56.518 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T01:37:56.518 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T01:37:56.556 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T01:37:56.594 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T01:37:56.727 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T01:37:57.201 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T01:37:57.327 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-26T01:37:57.512 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-26T01:37:57.543 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-26T01:37:57.681 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-26T01:37:57.728 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-26T01:37:57.769 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T01:37:57.770 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-26T01:37:57.770 INFO:teuthology.orchestra.run.smithi172.stdout:Upgraded: 2024-04-26T01:37:57.770 INFO:teuthology.orchestra.run.smithi172.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:37:57.770 INFO:teuthology.orchestra.run.smithi172.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:37:57.770 INFO:teuthology.orchestra.run.smithi172.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T01:37:57.770 INFO:teuthology.orchestra.run.smithi172.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T01:37:57.770 INFO:teuthology.orchestra.run.smithi172.stdout:Installed: 2024-04-26T01:37:57.770 INFO:teuthology.orchestra.run.smithi172.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T01:37:57.770 INFO:teuthology.orchestra.run.smithi172.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T01:37:57.770 INFO:teuthology.orchestra.run.smithi172.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T01:37:57.770 INFO:teuthology.orchestra.run.smithi172.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T01:37:57.771 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:37:57.771 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:37:57.771 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:37:57.771 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:37:57.771 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:37:57.771 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:37:57.771 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:37:57.771 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:37:57.771 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:37:57.771 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:37:57.771 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:37:57.771 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:37:57.772 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:37:57.772 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:37:57.772 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:37:57.772 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:37:57.772 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:37:57.772 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:37:57.772 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:37:57.772 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:37:57.772 INFO:teuthology.orchestra.run.smithi172.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:37:57.772 INFO:teuthology.orchestra.run.smithi172.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:37:57.772 INFO:teuthology.orchestra.run.smithi172.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:37:57.772 INFO:teuthology.orchestra.run.smithi172.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:37:57.772 INFO:teuthology.orchestra.run.smithi172.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T01:37:57.773 INFO:teuthology.orchestra.run.smithi172.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T01:37:57.773 INFO:teuthology.orchestra.run.smithi172.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T01:37:57.773 INFO:teuthology.orchestra.run.smithi172.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T01:37:57.773 INFO:teuthology.orchestra.run.smithi172.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T01:37:57.773 INFO:teuthology.orchestra.run.smithi172.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T01:37:57.773 INFO:teuthology.orchestra.run.smithi172.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T01:37:57.773 INFO:teuthology.orchestra.run.smithi172.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T01:37:57.773 INFO:teuthology.orchestra.run.smithi172.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T01:37:57.773 INFO:teuthology.orchestra.run.smithi172.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-26T01:37:57.773 INFO:teuthology.orchestra.run.smithi172.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-26T01:37:57.773 INFO:teuthology.orchestra.run.smithi172.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:37:57.773 INFO:teuthology.orchestra.run.smithi172.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:37:57.773 INFO:teuthology.orchestra.run.smithi172.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:37:57.774 INFO:teuthology.orchestra.run.smithi172.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T01:37:57.774 INFO:teuthology.orchestra.run.smithi172.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T01:37:57.774 INFO:teuthology.orchestra.run.smithi172.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T01:37:57.774 INFO:teuthology.orchestra.run.smithi172.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T01:37:57.774 INFO:teuthology.orchestra.run.smithi172.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T01:37:57.774 INFO:teuthology.orchestra.run.smithi172.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:37:57.774 INFO:teuthology.orchestra.run.smithi172.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:37:57.774 INFO:teuthology.orchestra.run.smithi172.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T01:37:57.774 INFO:teuthology.orchestra.run.smithi172.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:37:57.774 INFO:teuthology.orchestra.run.smithi172.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T01:37:57.774 INFO:teuthology.orchestra.run.smithi172.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T01:37:57.774 INFO:teuthology.orchestra.run.smithi172.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T01:37:57.774 INFO:teuthology.orchestra.run.smithi172.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T01:37:57.775 INFO:teuthology.orchestra.run.smithi172.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T01:37:57.775 INFO:teuthology.orchestra.run.smithi172.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T01:37:57.775 INFO:teuthology.orchestra.run.smithi172.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T01:37:57.775 INFO:teuthology.orchestra.run.smithi172.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T01:37:57.775 INFO:teuthology.orchestra.run.smithi172.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T01:37:57.775 INFO:teuthology.orchestra.run.smithi172.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T01:37:57.775 INFO:teuthology.orchestra.run.smithi172.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T01:37:57.775 INFO:teuthology.orchestra.run.smithi172.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-26T01:37:57.775 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T01:37:57.775 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T01:37:57.775 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T01:37:57.775 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T01:37:57.775 INFO:teuthology.orchestra.run.smithi172.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T01:37:57.775 INFO:teuthology.orchestra.run.smithi172.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T01:37:57.776 INFO:teuthology.orchestra.run.smithi172.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T01:37:57.776 INFO:teuthology.orchestra.run.smithi172.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T01:37:57.776 INFO:teuthology.orchestra.run.smithi172.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T01:37:57.776 INFO:teuthology.orchestra.run.smithi172.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T01:37:57.776 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T01:37:57.776 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:37:57.776 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:37:57.776 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:37:57.776 INFO:teuthology.orchestra.run.smithi172.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T01:37:57.776 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T01:37:57.776 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T01:37:57.776 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T01:37:57.776 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T01:37:57.776 INFO:teuthology.orchestra.run.smithi172.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T01:37:57.777 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T01:37:57.777 INFO:teuthology.orchestra.run.smithi172.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T01:37:57.777 INFO:teuthology.orchestra.run.smithi172.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T01:37:57.777 INFO:teuthology.orchestra.run.smithi172.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T01:37:57.777 INFO:teuthology.orchestra.run.smithi172.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T01:37:57.777 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T01:37:57.777 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T01:37:57.777 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T01:37:57.777 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T01:37:57.777 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T01:37:57.777 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T01:37:57.777 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T01:37:57.777 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T01:37:57.778 INFO:teuthology.orchestra.run.smithi172.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T01:37:57.778 INFO:teuthology.orchestra.run.smithi172.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T01:37:57.778 INFO:teuthology.orchestra.run.smithi172.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T01:37:57.778 INFO:teuthology.orchestra.run.smithi172.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T01:37:57.778 INFO:teuthology.orchestra.run.smithi172.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T01:37:57.778 INFO:teuthology.orchestra.run.smithi172.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T01:37:57.778 INFO:teuthology.orchestra.run.smithi172.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T01:37:57.778 INFO:teuthology.orchestra.run.smithi172.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T01:37:57.778 INFO:teuthology.orchestra.run.smithi172.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T01:37:57.778 INFO:teuthology.orchestra.run.smithi172.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T01:37:57.778 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T01:37:57.778 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T01:37:57.779 INFO:teuthology.orchestra.run.smithi172.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T01:37:57.779 INFO:teuthology.orchestra.run.smithi172.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T01:37:57.779 INFO:teuthology.orchestra.run.smithi172.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T01:37:57.779 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T01:37:57.779 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T01:37:57.779 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T01:37:57.779 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T01:37:57.779 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T01:37:57.779 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T01:37:57.779 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:37:57.779 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:37:57.779 INFO:teuthology.orchestra.run.smithi172.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T01:37:57.779 INFO:teuthology.orchestra.run.smithi172.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T01:37:57.779 INFO:teuthology.orchestra.run.smithi172.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T01:37:57.780 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:37:57.780 INFO:teuthology.orchestra.run.smithi172.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T01:37:57.780 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T01:37:57.780 INFO:teuthology.orchestra.run.smithi172.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T01:37:57.780 INFO:teuthology.orchestra.run.smithi172.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T01:37:57.780 INFO:teuthology.orchestra.run.smithi172.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T01:37:57.780 INFO:teuthology.orchestra.run.smithi172.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T01:37:57.780 INFO:teuthology.orchestra.run.smithi172.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T01:37:57.780 INFO:teuthology.orchestra.run.smithi172.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T01:37:57.780 INFO:teuthology.orchestra.run.smithi172.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T01:37:57.780 INFO:teuthology.orchestra.run.smithi172.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T01:37:57.780 INFO:teuthology.orchestra.run.smithi172.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T01:37:57.780 INFO:teuthology.orchestra.run.smithi172.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T01:37:57.781 INFO:teuthology.orchestra.run.smithi172.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T01:37:57.781 INFO:teuthology.orchestra.run.smithi172.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T01:37:57.781 INFO:teuthology.orchestra.run.smithi172.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T01:37:57.781 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:37:57.781 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:37:57.781 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:37:57.781 INFO:teuthology.orchestra.run.smithi172.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T01:37:57.781 INFO:teuthology.orchestra.run.smithi172.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T01:37:57.781 INFO:teuthology.orchestra.run.smithi172.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T01:37:57.781 INFO:teuthology.orchestra.run.smithi172.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T01:37:57.781 INFO:teuthology.orchestra.run.smithi172.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T01:37:57.781 INFO:teuthology.orchestra.run.smithi172.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T01:37:57.782 INFO:teuthology.orchestra.run.smithi172.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T01:37:57.782 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-26T01:37:57.782 INFO:teuthology.orchestra.run.smithi172.stdout:Complete! 2024-04-26T01:37:58.091 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T01:37:58.202 DEBUG:teuthology.parallel:result is None 2024-04-26T01:37:58.203 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T01:37:58.231 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-26T01:37:58.395 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-26T01:37:58.424 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-26T01:37:58.498 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-26T01:37:58.549 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T01:37:58.627 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T01:37:58.711 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T01:37:58.711 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-26T01:37:58.711 INFO:teuthology.orchestra.run.smithi165.stdout:Upgraded: 2024-04-26T01:37:58.711 INFO:teuthology.orchestra.run.smithi165.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:37:58.711 INFO:teuthology.orchestra.run.smithi165.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:37:58.711 INFO:teuthology.orchestra.run.smithi165.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T01:37:58.712 INFO:teuthology.orchestra.run.smithi165.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T01:37:58.712 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2024-04-26T01:37:58.712 INFO:teuthology.orchestra.run.smithi165.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T01:37:58.712 INFO:teuthology.orchestra.run.smithi165.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T01:37:58.712 INFO:teuthology.orchestra.run.smithi165.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T01:37:58.712 INFO:teuthology.orchestra.run.smithi165.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T01:37:58.712 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:37:58.712 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:37:58.712 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:37:58.712 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:37:58.712 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:37:58.712 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:37:58.712 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:37:58.713 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:37:58.713 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:37:58.713 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:37:58.713 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:37:58.713 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:37:58.713 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:37:58.713 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:37:58.713 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:37:58.713 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:37:58.713 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:37:58.713 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:37:58.713 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:37:58.713 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:37:58.714 INFO:teuthology.orchestra.run.smithi165.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:37:58.714 INFO:teuthology.orchestra.run.smithi165.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:37:58.714 INFO:teuthology.orchestra.run.smithi165.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:37:58.714 INFO:teuthology.orchestra.run.smithi165.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:37:58.714 INFO:teuthology.orchestra.run.smithi165.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T01:37:58.714 INFO:teuthology.orchestra.run.smithi165.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T01:37:58.714 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T01:37:58.714 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T01:37:58.714 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T01:37:58.714 INFO:teuthology.orchestra.run.smithi165.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T01:37:58.714 INFO:teuthology.orchestra.run.smithi165.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T01:37:58.714 INFO:teuthology.orchestra.run.smithi165.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T01:37:58.715 INFO:teuthology.orchestra.run.smithi165.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T01:37:58.715 INFO:teuthology.orchestra.run.smithi165.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-26T01:37:58.715 INFO:teuthology.orchestra.run.smithi165.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-26T01:37:58.715 INFO:teuthology.orchestra.run.smithi165.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:37:58.715 INFO:teuthology.orchestra.run.smithi165.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:37:58.715 INFO:teuthology.orchestra.run.smithi165.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:37:58.715 INFO:teuthology.orchestra.run.smithi165.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T01:37:58.715 INFO:teuthology.orchestra.run.smithi165.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T01:37:58.715 INFO:teuthology.orchestra.run.smithi165.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T01:37:58.715 INFO:teuthology.orchestra.run.smithi165.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T01:37:58.715 INFO:teuthology.orchestra.run.smithi165.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T01:37:58.715 INFO:teuthology.orchestra.run.smithi165.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:37:58.715 INFO:teuthology.orchestra.run.smithi165.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:37:58.716 INFO:teuthology.orchestra.run.smithi165.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T01:37:58.716 INFO:teuthology.orchestra.run.smithi165.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:37:58.716 INFO:teuthology.orchestra.run.smithi165.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T01:37:58.716 INFO:teuthology.orchestra.run.smithi165.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T01:37:58.716 INFO:teuthology.orchestra.run.smithi165.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T01:37:58.716 INFO:teuthology.orchestra.run.smithi165.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T01:37:58.716 INFO:teuthology.orchestra.run.smithi165.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T01:37:58.716 INFO:teuthology.orchestra.run.smithi165.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T01:37:58.716 INFO:teuthology.orchestra.run.smithi165.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T01:37:58.716 INFO:teuthology.orchestra.run.smithi165.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T01:37:58.716 INFO:teuthology.orchestra.run.smithi165.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T01:37:58.717 INFO:teuthology.orchestra.run.smithi165.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T01:37:58.717 INFO:teuthology.orchestra.run.smithi165.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T01:37:58.717 INFO:teuthology.orchestra.run.smithi165.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-26T01:37:58.717 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T01:37:58.717 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T01:37:58.717 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T01:37:58.717 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T01:37:58.717 INFO:teuthology.orchestra.run.smithi165.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T01:37:58.717 INFO:teuthology.orchestra.run.smithi165.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T01:37:58.717 INFO:teuthology.orchestra.run.smithi165.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T01:37:58.717 INFO:teuthology.orchestra.run.smithi165.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T01:37:58.717 INFO:teuthology.orchestra.run.smithi165.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T01:37:58.717 INFO:teuthology.orchestra.run.smithi165.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T01:37:58.718 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T01:37:58.718 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:37:58.718 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:37:58.718 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:37:58.718 INFO:teuthology.orchestra.run.smithi165.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T01:37:58.718 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T01:37:58.718 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T01:37:58.718 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T01:37:58.718 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T01:37:58.718 INFO:teuthology.orchestra.run.smithi165.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T01:37:58.718 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T01:37:58.718 INFO:teuthology.orchestra.run.smithi165.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T01:37:58.718 INFO:teuthology.orchestra.run.smithi165.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T01:37:58.718 INFO:teuthology.orchestra.run.smithi165.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T01:37:58.719 INFO:teuthology.orchestra.run.smithi165.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T01:37:58.719 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T01:37:58.719 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T01:37:58.719 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T01:37:58.719 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T01:37:58.719 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T01:37:58.719 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T01:37:58.719 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T01:37:58.719 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T01:37:58.719 INFO:teuthology.orchestra.run.smithi165.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T01:37:58.719 INFO:teuthology.orchestra.run.smithi165.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T01:37:58.719 INFO:teuthology.orchestra.run.smithi165.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T01:37:58.719 INFO:teuthology.orchestra.run.smithi165.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T01:37:58.720 INFO:teuthology.orchestra.run.smithi165.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T01:37:58.720 INFO:teuthology.orchestra.run.smithi165.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T01:37:58.720 INFO:teuthology.orchestra.run.smithi165.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T01:37:58.720 INFO:teuthology.orchestra.run.smithi165.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T01:37:58.720 INFO:teuthology.orchestra.run.smithi165.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T01:37:58.720 INFO:teuthology.orchestra.run.smithi165.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T01:37:58.720 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T01:37:58.720 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T01:37:58.720 INFO:teuthology.orchestra.run.smithi165.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T01:37:58.720 INFO:teuthology.orchestra.run.smithi165.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T01:37:58.720 INFO:teuthology.orchestra.run.smithi165.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T01:37:58.720 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T01:37:58.720 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T01:37:58.720 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T01:37:58.721 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T01:37:58.721 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T01:37:58.721 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T01:37:58.721 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:37:58.721 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:37:58.721 INFO:teuthology.orchestra.run.smithi165.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T01:37:58.721 INFO:teuthology.orchestra.run.smithi165.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T01:37:58.721 INFO:teuthology.orchestra.run.smithi165.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T01:37:58.721 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:37:58.721 INFO:teuthology.orchestra.run.smithi165.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T01:37:58.721 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T01:37:58.721 INFO:teuthology.orchestra.run.smithi165.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T01:37:58.721 INFO:teuthology.orchestra.run.smithi165.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T01:37:58.721 INFO:teuthology.orchestra.run.smithi165.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T01:37:58.722 INFO:teuthology.orchestra.run.smithi165.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T01:37:58.722 INFO:teuthology.orchestra.run.smithi165.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T01:37:58.722 INFO:teuthology.orchestra.run.smithi165.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T01:37:58.722 INFO:teuthology.orchestra.run.smithi165.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T01:37:58.722 INFO:teuthology.orchestra.run.smithi165.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T01:37:58.722 INFO:teuthology.orchestra.run.smithi165.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T01:37:58.722 INFO:teuthology.orchestra.run.smithi165.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T01:37:58.722 INFO:teuthology.orchestra.run.smithi165.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T01:37:58.722 INFO:teuthology.orchestra.run.smithi165.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T01:37:58.722 INFO:teuthology.orchestra.run.smithi165.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T01:37:58.722 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:37:58.722 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:37:58.722 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:37:58.722 INFO:teuthology.orchestra.run.smithi165.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T01:37:58.723 INFO:teuthology.orchestra.run.smithi165.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T01:37:58.723 INFO:teuthology.orchestra.run.smithi165.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T01:37:58.723 INFO:teuthology.orchestra.run.smithi165.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T01:37:58.723 INFO:teuthology.orchestra.run.smithi165.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T01:37:58.723 INFO:teuthology.orchestra.run.smithi165.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T01:37:58.723 INFO:teuthology.orchestra.run.smithi165.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T01:37:58.723 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-26T01:37:58.723 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-04-26T01:37:58.934 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T01:37:59.009 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T01:37:59.111 DEBUG:teuthology.parallel:result is None 2024-04-26T01:37:59.211 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T01:37:59.366 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-26T01:37:59.408 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T01:37:59.457 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T01:37:59.562 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T01:37:59.635 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T01:37:59.701 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T01:37:59.752 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T01:37:59.791 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T01:37:59.909 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T01:37:59.989 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T01:38:00.034 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T01:38:00.095 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T01:38:00.210 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T01:38:00.244 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-26T01:38:00.297 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-26T01:38:00.381 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-26T01:38:00.510 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T01:38:00.575 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T01:38:00.684 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T01:38:00.906 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T01:38:00.975 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T01:38:01.692 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T01:38:01.806 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T01:38:01.849 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T01:38:01.894 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T01:38:01.934 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-26T01:38:02.010 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T01:38:02.321 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T01:38:02.387 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T01:38:02.466 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T01:38:02.558 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T01:38:02.602 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T01:38:02.641 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T01:38:02.709 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T01:38:02.749 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T01:38:02.809 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T01:38:02.868 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T01:38:03.081 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T01:38:03.147 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T01:38:03.183 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T01:38:03.231 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T01:38:03.387 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T01:38:03.435 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T01:38:03.484 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T01:38:03.524 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T01:38:03.557 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-26T01:38:03.783 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T01:38:03.973 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T01:38:04.031 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T01:38:04.073 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T01:38:04.121 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T01:38:04.194 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T01:38:04.252 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T01:38:04.302 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T01:38:04.373 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T01:38:04.425 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T01:38:04.500 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T01:38:04.599 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T01:38:04.970 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T01:38:05.032 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T01:38:05.111 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T01:38:05.184 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T01:38:05.339 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T01:38:05.415 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T01:38:06.385 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T01:38:06.439 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T01:38:06.472 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T01:38:06.624 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T01:38:06.692 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T01:38:06.798 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T01:38:06.841 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T01:38:07.100 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T01:38:07.137 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T01:38:07.253 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T01:38:08.105 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T01:38:08.260 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T01:38:09.913 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T01:38:10.019 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T01:38:10.060 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T01:38:10.128 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T01:38:10.516 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T01:38:10.556 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-26T01:38:10.677 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-26T01:38:10.712 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-26T01:38:10.775 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T01:38:11.316 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-26T01:38:11.350 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-26T01:38:11.442 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-26T01:38:11.469 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-26T01:38:14.235 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-26T01:38:14.274 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-26T01:38:14.930 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-26T01:38:14.985 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-26T01:38:15.141 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-26T01:38:15.141 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T01:38:15.141 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T01:38:15.141 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:38:15.195 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-26T01:38:28.521 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-26T01:38:28.522 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /sys 2024-04-26T01:38:28.522 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /proc 2024-04-26T01:38:28.522 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /mnt 2024-04-26T01:38:28.522 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /var/tmp 2024-04-26T01:38:28.522 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /home 2024-04-26T01:38:28.522 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /root 2024-04-26T01:38:28.522 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /tmp 2024-04-26T01:38:28.522 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:38:28.760 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-26T01:38:28.806 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-26T01:38:28.806 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:38:28.806 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T01:38:28.807 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:38:30.427 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-26T01:38:30.508 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-26T01:38:30.509 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:38:30.509 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T01:38:30.509 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T01:38:30.509 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T01:38:30.509 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:38:30.770 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-26T01:38:30.839 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-26T01:38:30.839 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:38:30.839 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T01:38:30.839 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T01:38:30.839 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T01:38:30.839 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:38:31.338 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-26T01:38:31.404 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-26T01:38:31.404 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:38:31.404 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T01:38:31.404 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T01:38:31.404 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T01:38:31.405 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:38:31.546 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T01:38:31.611 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T01:38:31.684 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T01:38:31.919 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T01:38:31.983 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T01:38:32.070 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T01:38:32.178 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T01:38:32.372 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T01:38:32.435 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T01:38:32.473 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T01:38:32.506 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T01:38:33.202 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-26T01:38:33.243 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-26T01:38:33.329 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-26T01:38:33.450 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-26T01:38:33.487 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-26T01:38:34.819 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-26T01:38:34.872 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-26T01:38:35.979 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-26T01:38:36.022 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-26T01:38:36.168 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-26T01:38:36.346 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-26T01:38:36.374 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-26T01:38:36.445 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-26T01:38:36.445 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:38:36.445 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T01:38:36.446 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T01:38:36.446 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T01:38:36.446 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:38:36.500 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-26T01:38:36.537 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-26T01:38:36.677 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-26T01:38:36.753 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-26T01:38:37.764 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T01:38:37.794 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-26T01:38:37.862 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-26T01:38:37.863 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:38:37.863 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T01:38:37.863 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T01:38:37.863 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T01:38:37.863 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:38:37.920 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-26T01:38:37.988 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-26T01:38:37.988 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:38:37.988 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T01:38:37.988 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:38:38.071 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-26T01:38:38.145 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-26T01:38:38.145 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:38:38.146 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T01:38:38.146 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T01:38:38.146 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T01:38:38.146 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:38:38.504 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-26T01:38:38.572 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-26T01:38:38.572 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:38:38.572 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T01:38:38.572 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T01:38:38.572 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T01:38:38.572 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:38:43.871 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-26T01:38:43.942 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-26T01:38:44.042 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-26T01:38:44.095 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T01:38:44.165 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T01:38:44.520 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-26T01:38:44.591 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T01:38:44.619 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-26T01:38:45.525 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T01:38:45.593 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T01:38:45.650 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T01:38:47.033 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-26T01:38:47.141 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T01:38:47.225 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T01:38:47.405 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T01:38:47.456 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T01:38:47.456 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T01:38:47.580 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T01:38:47.621 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T01:38:47.622 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T01:38:47.787 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T01:38:47.927 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T01:38:48.222 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T01:38:48.223 INFO:teuthology.orchestra.run.smithi151.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-26T01:38:48.223 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:38:54.220 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-26T01:38:54.220 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-26T01:38:54.220 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-26T01:38:54.221 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-26T01:38:54.221 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-26T01:38:54.221 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-26T01:38:54.221 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-26T01:38:54.221 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-26T01:38:54.221 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-26T01:38:54.221 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-26T01:38:54.221 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-26T01:38:54.221 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-26T01:38:54.221 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-26T01:38:54.221 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-26T01:38:54.221 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-26T01:38:54.222 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-26T01:38:54.222 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-26T01:38:54.222 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-26T01:38:54.222 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-26T01:38:54.222 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-26T01:38:54.222 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-26T01:38:54.222 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-26T01:38:54.222 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-26T01:38:54.222 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-26T01:38:54.222 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-26T01:38:54.222 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-26T01:38:54.222 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-26T01:38:54.222 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-26T01:38:54.223 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-26T01:38:54.235 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-26T01:38:54.235 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-26T01:38:54.235 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-26T01:38:54.235 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-26T01:38:54.235 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-26T01:38:54.235 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-26T01:38:54.236 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-26T01:38:54.236 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-26T01:38:54.236 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-26T01:38:54.236 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-26T01:38:54.236 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T01:38:54.236 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T01:38:54.236 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T01:38:54.236 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T01:38:54.236 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T01:38:54.236 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T01:38:54.236 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T01:38:54.236 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T01:38:54.237 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T01:38:54.237 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T01:38:54.237 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T01:38:54.237 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T01:38:54.237 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T01:38:54.237 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T01:38:54.237 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T01:38:54.237 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T01:38:54.237 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T01:38:54.237 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T01:38:54.237 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T01:38:54.237 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T01:38:54.238 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T01:38:54.238 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T01:38:54.238 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T01:38:54.238 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T01:38:54.238 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T01:38:54.238 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T01:38:54.238 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T01:38:54.238 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T01:38:54.238 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T01:38:54.238 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T01:38:54.238 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T01:38:54.238 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T01:38:54.239 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T01:38:54.239 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T01:38:54.239 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T01:38:54.239 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T01:38:54.239 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T01:38:54.239 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T01:38:54.239 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T01:38:54.239 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T01:38:54.239 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T01:38:54.239 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T01:38:54.239 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T01:38:54.239 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T01:38:54.240 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T01:38:54.240 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T01:38:54.240 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T01:38:54.240 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T01:38:54.240 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T01:38:54.240 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T01:38:54.240 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T01:38:54.240 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T01:38:54.240 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T01:38:54.240 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T01:38:54.240 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T01:38:54.240 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T01:38:54.241 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T01:38:54.241 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T01:38:54.241 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T01:38:54.241 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T01:38:54.241 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T01:38:54.241 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T01:38:54.241 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T01:38:54.241 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T01:38:54.241 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T01:38:54.241 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T01:38:54.241 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T01:38:54.241 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T01:38:54.241 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-26T01:38:54.242 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-26T01:38:54.242 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T01:38:54.242 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T01:38:54.242 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T01:38:54.242 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-26T01:38:54.242 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T01:38:54.242 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T01:38:54.242 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T01:38:54.242 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T01:38:54.242 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T01:38:54.242 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T01:38:54.242 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T01:38:54.243 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T01:38:54.243 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T01:38:54.243 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T01:38:54.243 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T01:38:54.243 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T01:38:54.243 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T01:38:54.243 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T01:38:54.243 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T01:38:54.243 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T01:38:54.243 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T01:38:54.243 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T01:38:54.243 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T01:38:54.244 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T01:38:54.244 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T01:38:54.244 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T01:38:54.244 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T01:38:54.244 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T01:38:54.244 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T01:38:54.244 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T01:38:54.244 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T01:38:54.244 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T01:38:54.244 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T01:38:54.244 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T01:38:54.244 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T01:38:54.244 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T01:38:54.244 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T01:38:54.245 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T01:38:54.245 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T01:38:54.245 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-26T01:38:54.245 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T01:38:54.245 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-26T01:38:54.245 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T01:38:54.245 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T01:38:54.245 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T01:38:54.245 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T01:38:56.773 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T01:38:56.773 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:38:56.773 INFO:teuthology.orchestra.run.smithi151.stdout:Upgraded: 2024-04-26T01:38:56.773 INFO:teuthology.orchestra.run.smithi151.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:38:56.773 INFO:teuthology.orchestra.run.smithi151.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:38:56.773 INFO:teuthology.orchestra.run.smithi151.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T01:38:56.773 INFO:teuthology.orchestra.run.smithi151.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T01:38:56.773 INFO:teuthology.orchestra.run.smithi151.stdout:Installed: 2024-04-26T01:38:56.773 INFO:teuthology.orchestra.run.smithi151.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T01:38:56.773 INFO:teuthology.orchestra.run.smithi151.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T01:38:56.774 INFO:teuthology.orchestra.run.smithi151.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T01:38:56.774 INFO:teuthology.orchestra.run.smithi151.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T01:38:56.774 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:38:56.774 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:38:56.774 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:38:56.774 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:38:56.774 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:38:56.774 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:38:56.774 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:38:56.774 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:38:56.774 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:38:56.774 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:38:56.774 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:38:56.775 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:38:56.775 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:38:56.775 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:38:56.775 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:38:56.775 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:38:56.775 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:38:56.775 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:38:56.775 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:38:56.775 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:38:56.775 INFO:teuthology.orchestra.run.smithi151.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:38:56.775 INFO:teuthology.orchestra.run.smithi151.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:38:56.775 INFO:teuthology.orchestra.run.smithi151.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:38:56.775 INFO:teuthology.orchestra.run.smithi151.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:38:56.776 INFO:teuthology.orchestra.run.smithi151.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T01:38:56.776 INFO:teuthology.orchestra.run.smithi151.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T01:38:56.776 INFO:teuthology.orchestra.run.smithi151.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T01:38:56.776 INFO:teuthology.orchestra.run.smithi151.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T01:38:56.776 INFO:teuthology.orchestra.run.smithi151.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T01:38:56.776 INFO:teuthology.orchestra.run.smithi151.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T01:38:56.776 INFO:teuthology.orchestra.run.smithi151.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T01:38:56.776 INFO:teuthology.orchestra.run.smithi151.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T01:38:56.776 INFO:teuthology.orchestra.run.smithi151.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T01:38:56.776 INFO:teuthology.orchestra.run.smithi151.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-26T01:38:56.776 INFO:teuthology.orchestra.run.smithi151.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-26T01:38:56.776 INFO:teuthology.orchestra.run.smithi151.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:38:56.777 INFO:teuthology.orchestra.run.smithi151.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:38:56.777 INFO:teuthology.orchestra.run.smithi151.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:38:56.777 INFO:teuthology.orchestra.run.smithi151.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T01:38:56.777 INFO:teuthology.orchestra.run.smithi151.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T01:38:56.777 INFO:teuthology.orchestra.run.smithi151.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T01:38:56.777 INFO:teuthology.orchestra.run.smithi151.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T01:38:56.777 INFO:teuthology.orchestra.run.smithi151.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T01:38:56.777 INFO:teuthology.orchestra.run.smithi151.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:38:56.777 INFO:teuthology.orchestra.run.smithi151.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:38:56.777 INFO:teuthology.orchestra.run.smithi151.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T01:38:56.777 INFO:teuthology.orchestra.run.smithi151.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:38:56.777 INFO:teuthology.orchestra.run.smithi151.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T01:38:56.777 INFO:teuthology.orchestra.run.smithi151.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T01:38:56.778 INFO:teuthology.orchestra.run.smithi151.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T01:38:56.778 INFO:teuthology.orchestra.run.smithi151.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T01:38:56.778 INFO:teuthology.orchestra.run.smithi151.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T01:38:56.778 INFO:teuthology.orchestra.run.smithi151.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T01:38:56.778 INFO:teuthology.orchestra.run.smithi151.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T01:38:56.778 INFO:teuthology.orchestra.run.smithi151.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T01:38:56.778 INFO:teuthology.orchestra.run.smithi151.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T01:38:56.778 INFO:teuthology.orchestra.run.smithi151.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T01:38:56.778 INFO:teuthology.orchestra.run.smithi151.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T01:38:56.778 INFO:teuthology.orchestra.run.smithi151.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-26T01:38:56.778 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T01:38:56.778 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T01:38:56.779 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T01:38:56.779 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T01:38:56.779 INFO:teuthology.orchestra.run.smithi151.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T01:38:56.779 INFO:teuthology.orchestra.run.smithi151.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T01:38:56.779 INFO:teuthology.orchestra.run.smithi151.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T01:38:56.779 INFO:teuthology.orchestra.run.smithi151.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T01:38:56.779 INFO:teuthology.orchestra.run.smithi151.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T01:38:56.779 INFO:teuthology.orchestra.run.smithi151.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T01:38:56.779 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T01:38:56.779 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:38:56.779 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:38:56.779 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:38:56.779 INFO:teuthology.orchestra.run.smithi151.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T01:38:56.780 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T01:38:56.780 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T01:38:56.780 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T01:38:56.780 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T01:38:56.780 INFO:teuthology.orchestra.run.smithi151.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T01:38:56.780 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T01:38:56.780 INFO:teuthology.orchestra.run.smithi151.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T01:38:56.780 INFO:teuthology.orchestra.run.smithi151.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T01:38:56.780 INFO:teuthology.orchestra.run.smithi151.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T01:38:56.780 INFO:teuthology.orchestra.run.smithi151.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T01:38:56.780 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T01:38:56.780 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T01:38:56.780 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T01:38:56.781 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T01:38:56.781 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T01:38:56.781 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T01:38:56.781 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T01:38:56.781 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T01:38:56.781 INFO:teuthology.orchestra.run.smithi151.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T01:38:56.781 INFO:teuthology.orchestra.run.smithi151.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T01:38:56.781 INFO:teuthology.orchestra.run.smithi151.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T01:38:56.781 INFO:teuthology.orchestra.run.smithi151.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T01:38:56.781 INFO:teuthology.orchestra.run.smithi151.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T01:38:56.781 INFO:teuthology.orchestra.run.smithi151.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T01:38:56.781 INFO:teuthology.orchestra.run.smithi151.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T01:38:56.781 INFO:teuthology.orchestra.run.smithi151.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T01:38:56.782 INFO:teuthology.orchestra.run.smithi151.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T01:38:56.782 INFO:teuthology.orchestra.run.smithi151.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T01:38:56.782 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T01:38:56.782 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T01:38:56.782 INFO:teuthology.orchestra.run.smithi151.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T01:38:56.782 INFO:teuthology.orchestra.run.smithi151.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T01:38:56.782 INFO:teuthology.orchestra.run.smithi151.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T01:38:56.782 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T01:38:56.782 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T01:38:56.782 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T01:38:56.782 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T01:38:56.782 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T01:38:56.783 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T01:38:56.783 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:38:56.783 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:38:56.783 INFO:teuthology.orchestra.run.smithi151.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T01:38:56.783 INFO:teuthology.orchestra.run.smithi151.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T01:38:56.783 INFO:teuthology.orchestra.run.smithi151.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T01:38:56.783 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:38:56.783 INFO:teuthology.orchestra.run.smithi151.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T01:38:56.783 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T01:38:56.783 INFO:teuthology.orchestra.run.smithi151.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T01:38:56.783 INFO:teuthology.orchestra.run.smithi151.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T01:38:56.783 INFO:teuthology.orchestra.run.smithi151.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T01:38:56.783 INFO:teuthology.orchestra.run.smithi151.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T01:38:56.783 INFO:teuthology.orchestra.run.smithi151.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T01:38:56.784 INFO:teuthology.orchestra.run.smithi151.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T01:38:56.784 INFO:teuthology.orchestra.run.smithi151.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T01:38:56.784 INFO:teuthology.orchestra.run.smithi151.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T01:38:56.784 INFO:teuthology.orchestra.run.smithi151.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T01:38:56.784 INFO:teuthology.orchestra.run.smithi151.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T01:38:56.784 INFO:teuthology.orchestra.run.smithi151.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T01:38:56.784 INFO:teuthology.orchestra.run.smithi151.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T01:38:56.784 INFO:teuthology.orchestra.run.smithi151.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T01:38:56.784 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:38:56.784 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:38:56.784 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:38:56.784 INFO:teuthology.orchestra.run.smithi151.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T01:38:56.784 INFO:teuthology.orchestra.run.smithi151.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T01:38:56.785 INFO:teuthology.orchestra.run.smithi151.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T01:38:56.785 INFO:teuthology.orchestra.run.smithi151.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T01:38:56.785 INFO:teuthology.orchestra.run.smithi151.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T01:38:56.785 INFO:teuthology.orchestra.run.smithi151.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T01:38:56.785 INFO:teuthology.orchestra.run.smithi151.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T01:38:56.785 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:38:56.785 INFO:teuthology.orchestra.run.smithi151.stdout:Complete! 2024-04-26T01:38:57.381 DEBUG:teuthology.parallel:result is None 2024-04-26T01:38:57.382 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-26T01:38:57.604 DEBUG:teuthology.orchestra.run.smithi151:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T01:38:57.658 INFO:teuthology.orchestra.run.smithi151.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-26T01:38:57.659 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-26T01:38:57.659 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-26T01:38:57.661 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-26T01:38:57.802 DEBUG:teuthology.orchestra.run.smithi165:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T01:38:57.828 INFO:teuthology.orchestra.run.smithi165.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-26T01:38:57.828 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-26T01:38:57.828 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-26T01:38:57.830 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-26T01:38:58.059 DEBUG:teuthology.orchestra.run.smithi172:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T01:38:58.086 INFO:teuthology.orchestra.run.smithi172.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-26T01:38:58.086 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-26T01:38:58.086 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-26T01:38:58.088 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-26T01:38:58.088 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-26T01:38:58.088 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T01:38:58.177 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-26T01:38:58.177 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T01:38:58.209 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-26T01:38:58.209 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T01:38:58.239 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-26T01:38:58.240 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-26T01:38:58.240 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T01:38:58.311 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T01:38:58.417 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-26T01:38:58.417 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T01:38:58.442 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T01:38:58.507 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-26T01:38:58.507 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T01:38:58.535 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T01:38:58.602 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-26T01:38:58.602 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-26T01:38:58.602 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T01:38:58.671 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T01:38:58.776 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-26T01:38:58.776 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T01:38:58.801 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T01:38:58.865 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-26T01:38:58.865 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T01:38:58.890 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T01:38:58.954 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-26T01:38:58.955 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-26T01:38:58.955 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T01:38:59.023 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T01:38:59.130 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-26T01:38:59.130 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T01:38:59.155 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T01:38:59.218 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-26T01:38:59.219 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T01:38:59.244 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T01:38:59.309 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-26T01:38:59.429 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'subvols': {'create': 2}} 2024-04-26T01:38:59.429 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-26T01:38:59.430 INFO:tasks.cephadm:Cluster fsid is c0e49e64-036d-11ef-bc93-c7b262605968 2024-04-26T01:38:59.430 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-26T01:38:59.430 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.151', 'mon.b': '172.21.15.165', 'mon.c': '172.21.15.172'} 2024-04-26T01:38:59.430 INFO:tasks.cephadm:First mon is mon.a on smithi151 2024-04-26T01:38:59.430 INFO:tasks.cephadm:First mgr is x 2024-04-26T01:38:59.430 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-26T01:38:59.431 DEBUG:teuthology.orchestra.run.smithi151:> sudo hostname $(hostname -s) 2024-04-26T01:38:59.497 DEBUG:teuthology.orchestra.run.smithi165:> sudo hostname $(hostname -s) 2024-04-26T01:38:59.523 DEBUG:teuthology.orchestra.run.smithi172:> sudo hostname $(hostname -s) 2024-04-26T01:38:59.547 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-26T01:38:59.548 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-26T01:38:59.765 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78795/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3156-gd69f13d8', 'node_name': '172.21.5.39+adami09', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3156.gd69f13d8'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-22 12:00:06.590653', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-lusov-quiescer', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-26T01:39:00.013 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-lusov-quiescer, sha1 d69f13d827952bb07bb9b7973bca14f97d764d6f from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-26T01:39:00.015 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm 2024-04-26T01:39:00.015 DEBUG:teuthology.orchestra.run.smithi151:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-26T01:39:00.635 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 26 01:39 /home/ubuntu/cephtest/cephadm 2024-04-26T01:39:00.636 DEBUG:teuthology.orchestra.run.smithi165:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-26T01:39:01.218 INFO:teuthology.orchestra.run.smithi165.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 26 01:39 /home/ubuntu/cephtest/cephadm 2024-04-26T01:39:01.218 DEBUG:teuthology.orchestra.run.smithi172:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-26T01:39:01.704 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 26 01:39 /home/ubuntu/cephtest/cephadm 2024-04-26T01:39:01.705 DEBUG:teuthology.orchestra.run.smithi151:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-26T01:39:01.747 DEBUG:teuthology.orchestra.run.smithi165:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-26T01:39:01.766 DEBUG:teuthology.orchestra.run.smithi172:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-26T01:39:01.802 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f on all hosts... 2024-04-26T01:39:01.802 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-26T01:39:01.807 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-26T01:39:01.809 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-26T01:39:03.393 INFO:teuthology.orchestra.run.smithi165.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-26T01:39:03.401 INFO:teuthology.orchestra.run.smithi172.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-26T01:39:04.238 INFO:teuthology.orchestra.run.smithi151.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-26T01:39:42.003 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-26T01:39:42.004 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-26T01:39:42.004 INFO:teuthology.orchestra.run.smithi165.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-26T01:39:42.004 INFO:teuthology.orchestra.run.smithi165.stdout: "repo_digests": [ 2024-04-26T01:39:42.004 INFO:teuthology.orchestra.run.smithi165.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-26T01:39:42.004 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-26T01:39:42.004 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-26T01:39:49.562 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-26T01:39:49.562 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-26T01:39:49.562 INFO:teuthology.orchestra.run.smithi172.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-26T01:39:49.562 INFO:teuthology.orchestra.run.smithi172.stdout: "repo_digests": [ 2024-04-26T01:39:49.562 INFO:teuthology.orchestra.run.smithi172.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-26T01:39:49.562 INFO:teuthology.orchestra.run.smithi172.stdout: ] 2024-04-26T01:39:49.562 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-26T01:39:54.725 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-26T01:39:54.725 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-26T01:39:54.725 INFO:teuthology.orchestra.run.smithi151.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-26T01:39:54.725 INFO:teuthology.orchestra.run.smithi151.stdout: "repo_digests": [ 2024-04-26T01:39:54.725 INFO:teuthology.orchestra.run.smithi151.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-26T01:39:54.725 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-26T01:39:54.726 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-26T01:39:54.773 DEBUG:teuthology.orchestra.run.smithi151:> sudo mkdir -p /etc/ceph 2024-04-26T01:39:54.850 DEBUG:teuthology.orchestra.run.smithi165:> sudo mkdir -p /etc/ceph 2024-04-26T01:39:54.882 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /etc/ceph 2024-04-26T01:39:54.913 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod 777 /etc/ceph 2024-04-26T01:39:55.000 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod 777 /etc/ceph 2024-04-26T01:39:55.029 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 777 /etc/ceph 2024-04-26T01:39:55.058 INFO:tasks.cephadm:Writing seed config... 2024-04-26T01:39:55.059 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-26T01:39:55.059 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-26T01:39:55.059 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-26T01:39:55.059 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-26T01:39:55.059 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-26T01:39:55.059 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-26T01:39:55.060 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-26T01:39:55.060 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-26T01:39:55.060 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-26T01:39:55.060 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-26T01:39:55.060 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-26T01:39:55.060 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-26T01:39:55.060 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-26T01:39:55.060 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-26T01:39:55.060 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-26T01:39:55.060 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-26T01:39:55.060 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-26T01:39:55.060 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-04-26T01:39:55.060 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-04-26T01:39:55.060 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-26T01:39:55.060 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-26T01:39:55.061 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-26T01:39:55.061 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-26T01:39:55.061 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-26T01:39:55.061 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-26T01:39:55.061 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-26T01:39:55.061 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-26T01:39:55.061 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-26T01:39:55.061 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-26T01:39:55.061 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-26T01:39:55.061 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-26T01:39:55.061 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-26T01:39:55.061 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-04-26T01:39:55.061 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-04-26T01:39:55.061 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-04-26T01:39:55.061 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-26T01:39:55.062 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-26T01:39:55.062 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-04-26T01:39:55.062 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-04-26T01:39:55.062 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-26T01:39:55.062 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-26T01:39:55.062 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-04-26T01:39:55.062 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-26T01:39:55.062 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-26T01:39:55.062 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-26T01:39:55.062 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-26T01:39:55.062 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-26T01:39:55.062 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-26T01:39:55.062 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-26T01:39:55.062 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-26T01:39:55.063 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-26T01:39:55.063 DEBUG:teuthology.orchestra.run.smithi151:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-26T01:39:55.119 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = c0e49e64-036d-11ef-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-26T01:39:55.119 DEBUG:teuthology.orchestra.run.smithi151:mon.a> sudo journalctl -f -n 0 -u ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mon.a.service 2024-04-26T01:39:55.166 DEBUG:teuthology.orchestra.run.smithi151:mgr.x> sudo journalctl -f -n 0 -u ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mgr.x.service 2024-04-26T01:39:55.212 INFO:tasks.cephadm:Bootstrapping... 2024-04-26T01:39:55.213 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f -v bootstrap --fsid c0e49e64-036d-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.151 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-26T01:39:55.465 INFO:teuthology.orchestra.run.smithi151.stdout:-------------------------------------------------------------------------------- 2024-04-26T01:39:55.465 INFO:teuthology.orchestra.run.smithi151.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f', '-v', 'bootstrap', '--fsid', 'c0e49e64-036d-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.151', '--skip-admin-label'] 2024-04-26T01:39:55.466 INFO:teuthology.orchestra.run.smithi151.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-26T01:39:55.466 INFO:teuthology.orchestra.run.smithi151.stdout:Verifying podman|docker is present... 2024-04-26T01:39:55.534 INFO:teuthology.orchestra.run.smithi151.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-26T01:39:55.534 INFO:teuthology.orchestra.run.smithi151.stdout:Verifying lvm2 is present... 2024-04-26T01:39:55.534 INFO:teuthology.orchestra.run.smithi151.stdout:Verifying time synchronization is in place... 2024-04-26T01:39:55.549 INFO:teuthology.orchestra.run.smithi151.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-26T01:39:55.549 INFO:teuthology.orchestra.run.smithi151.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-26T01:39:55.563 INFO:teuthology.orchestra.run.smithi151.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-26T01:39:55.563 INFO:teuthology.orchestra.run.smithi151.stdout:systemctl: stdout inactive 2024-04-26T01:39:55.579 INFO:teuthology.orchestra.run.smithi151.stdout:systemctl: stdout enabled 2024-04-26T01:39:55.593 INFO:teuthology.orchestra.run.smithi151.stdout:systemctl: stdout active 2024-04-26T01:39:55.593 INFO:teuthology.orchestra.run.smithi151.stdout:Unit chronyd.service is enabled and running 2024-04-26T01:39:55.593 INFO:teuthology.orchestra.run.smithi151.stdout:Repeating the final host check... 2024-04-26T01:39:55.655 INFO:teuthology.orchestra.run.smithi151.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-26T01:39:55.655 INFO:teuthology.orchestra.run.smithi151.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-26T01:39:55.656 INFO:teuthology.orchestra.run.smithi151.stdout:systemctl is present 2024-04-26T01:39:55.656 INFO:teuthology.orchestra.run.smithi151.stdout:lvcreate is present 2024-04-26T01:39:55.671 INFO:teuthology.orchestra.run.smithi151.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-26T01:39:55.671 INFO:teuthology.orchestra.run.smithi151.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-26T01:39:55.688 INFO:teuthology.orchestra.run.smithi151.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-26T01:39:55.689 INFO:teuthology.orchestra.run.smithi151.stdout:systemctl: stdout inactive 2024-04-26T01:39:55.711 INFO:teuthology.orchestra.run.smithi151.stdout:systemctl: stdout enabled 2024-04-26T01:39:55.730 INFO:teuthology.orchestra.run.smithi151.stdout:systemctl: stdout active 2024-04-26T01:39:55.730 INFO:teuthology.orchestra.run.smithi151.stdout:Unit chronyd.service is enabled and running 2024-04-26T01:39:55.731 INFO:teuthology.orchestra.run.smithi151.stdout:Host looks OK 2024-04-26T01:39:55.731 INFO:teuthology.orchestra.run.smithi151.stdout:Cluster fsid: c0e49e64-036d-11ef-bc93-c7b262605968 2024-04-26T01:39:55.731 INFO:teuthology.orchestra.run.smithi151.stdout:Acquiring lock 140522519551136 on /run/cephadm/c0e49e64-036d-11ef-bc93-c7b262605968.lock 2024-04-26T01:39:55.731 INFO:teuthology.orchestra.run.smithi151.stdout:Lock 140522519551136 acquired on /run/cephadm/c0e49e64-036d-11ef-bc93-c7b262605968.lock 2024-04-26T01:39:55.732 INFO:teuthology.orchestra.run.smithi151.stdout:Verifying IP 172.21.15.151 port 3300 ... 2024-04-26T01:39:55.733 INFO:teuthology.orchestra.run.smithi151.stdout:Verifying IP 172.21.15.151 port 6789 ... 2024-04-26T01:39:55.733 INFO:teuthology.orchestra.run.smithi151.stdout:Base mon IP(s) is [172.21.15.151:3300, 172.21.15.151:6789], mon addrv is [v2:172.21.15.151:3300,v1:172.21.15.151:6789] 2024-04-26T01:39:55.744 INFO:teuthology.orchestra.run.smithi151.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.151 metric 100 2024-04-26T01:39:55.744 INFO:teuthology.orchestra.run.smithi151.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.151 metric 100 2024-04-26T01:39:55.754 INFO:teuthology.orchestra.run.smithi151.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-26T01:39:55.754 INFO:teuthology.orchestra.run.smithi151.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-04-26T01:39:55.754 INFO:teuthology.orchestra.run.smithi151.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1776sec hoplimit 64 pref medium 2024-04-26T01:39:55.763 INFO:teuthology.orchestra.run.smithi151.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-26T01:39:55.763 INFO:teuthology.orchestra.run.smithi151.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-26T01:39:55.763 INFO:teuthology.orchestra.run.smithi151.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-26T01:39:55.763 INFO:teuthology.orchestra.run.smithi151.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-04-26T01:39:55.763 INFO:teuthology.orchestra.run.smithi151.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe88:6f63/64 scope link 2024-04-26T01:39:55.763 INFO:teuthology.orchestra.run.smithi151.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-26T01:39:55.765 INFO:teuthology.orchestra.run.smithi151.stdout:Mon IP `172.21.15.151` is in CIDR network `172.21.0.0/20` 2024-04-26T01:39:55.765 INFO:teuthology.orchestra.run.smithi151.stdout:Mon IP `172.21.15.151` is in CIDR network `172.21.0.0/20` 2024-04-26T01:39:55.765 INFO:teuthology.orchestra.run.smithi151.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-26T01:39:55.767 INFO:teuthology.orchestra.run.smithi151.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-26T01:39:55.769 INFO:teuthology.orchestra.run.smithi151.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-26T01:39:56.447 INFO:teuthology.orchestra.run.smithi151.stdout:/bin/podman: stdout da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 2024-04-26T01:39:56.447 INFO:teuthology.orchestra.run.smithi151.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-26T01:39:56.447 INFO:teuthology.orchestra.run.smithi151.stdout:/bin/podman: stderr Getting image source signatures 2024-04-26T01:39:56.447 INFO:teuthology.orchestra.run.smithi151.stdout:/bin/podman: stderr Copying blob sha256:7f03586bcf9c6a2b4d690c118c4aa5b771f9c6f252d75a20fb283dd07093a738 2024-04-26T01:39:56.448 INFO:teuthology.orchestra.run.smithi151.stdout:/bin/podman: stderr Copying blob sha256:426fede9fffa2c7dfe40a4f9322a34794b72405170134d54d2c30d9cc6c6a5be 2024-04-26T01:39:56.448 INFO:teuthology.orchestra.run.smithi151.stdout:/bin/podman: stderr Copying config sha256:da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 2024-04-26T01:39:56.448 INFO:teuthology.orchestra.run.smithi151.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-26T01:39:57.685 INFO:teuthology.orchestra.run.smithi151.stdout:ceph: stdout ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-26T01:39:57.685 INFO:teuthology.orchestra.run.smithi151.stdout:Ceph version: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-26T01:39:57.686 INFO:teuthology.orchestra.run.smithi151.stdout:Extracting ceph user uid/gid from container image... 2024-04-26T01:39:58.968 INFO:teuthology.orchestra.run.smithi151.stdout:stat: stdout 167 167 2024-04-26T01:39:58.969 INFO:teuthology.orchestra.run.smithi151.stdout:Creating initial keys... 2024-04-26T01:40:00.167 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph-authtool: stdout AQDvBStmQKXwGBAAPWmkizd5Zo5oDeGGc5Vj0w== 2024-04-26T01:40:01.495 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph-authtool: stdout AQDwBStmvBesMhAAqzWUIl91BKoiw/cv7oIdxg== 2024-04-26T01:40:02.670 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph-authtool: stdout AQDyBStm3MoLABAAQxJELf3YdpahlSXvmrEuFg== 2024-04-26T01:40:02.674 INFO:teuthology.orchestra.run.smithi151.stdout:Creating initial monmap... 2024-04-26T01:40:03.840 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-26T01:40:03.840 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-26T01:40:03.840 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to c0e49e64-036d-11ef-bc93-c7b262605968 2024-04-26T01:40:03.841 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-26T01:40:03.841 INFO:teuthology.orchestra.run.smithi151.stdout:monmaptool for a [v2:172.21.15.151:3300,v1:172.21.15.151:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-26T01:40:03.841 INFO:teuthology.orchestra.run.smithi151.stdout:setting min_mon_release = quincy 2024-04-26T01:40:03.841 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/monmaptool: set fsid to c0e49e64-036d-11ef-bc93-c7b262605968 2024-04-26T01:40:03.841 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-26T01:40:03.841 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:40:03.841 INFO:teuthology.orchestra.run.smithi151.stdout:Creating mon... 2024-04-26T01:40:05.137 INFO:teuthology.orchestra.run.smithi151.stdout:create mon.a on 2024-04-26T01:40:05.475 INFO:teuthology.orchestra.run.smithi151.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-26T01:40:05.717 INFO:teuthology.orchestra.run.smithi151.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-26T01:40:05.998 INFO:teuthology.orchestra.run.smithi151.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-c0e49e64-036d-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-c0e49e64-036d-11ef-bc93-c7b262605968.target. 2024-04-26T01:40:05.998 INFO:teuthology.orchestra.run.smithi151.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-c0e49e64-036d-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-c0e49e64-036d-11ef-bc93-c7b262605968.target. 2024-04-26T01:40:06.374 INFO:teuthology.orchestra.run.smithi151.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mon.a 2024-04-26T01:40:06.374 INFO:teuthology.orchestra.run.smithi151.stdout:systemctl: stderr Failed to reset failed state of unit ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-26T01:40:06.693 INFO:teuthology.orchestra.run.smithi151.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-c0e49e64-036d-11ef-bc93-c7b262605968.target.wants/ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-c0e49e64-036d-11ef-bc93-c7b262605968@.service. 2024-04-26T01:40:07.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:07 smithi151 podman[27668]: 2024-04-26 01:40:07.115447476 +0000 UTC m=+0.164129571 container create 8bddb80a33aefe5556cf82aa58fff186f0e9534d169eeadbe7fcfa05e9632e05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T01:40:07.561 INFO:teuthology.orchestra.run.smithi151.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T01:40:07.561 INFO:teuthology.orchestra.run.smithi151.stdout:systemctl: stdout disabled 2024-04-26T01:40:07.579 INFO:teuthology.orchestra.run.smithi151.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T01:40:07.579 INFO:teuthology.orchestra.run.smithi151.stdout:systemctl: stdout inactive 2024-04-26T01:40:07.579 INFO:teuthology.orchestra.run.smithi151.stdout:firewalld.service is not enabled 2024-04-26T01:40:07.579 INFO:teuthology.orchestra.run.smithi151.stdout:Not possible to enable service . firewalld.service is not available 2024-04-26T01:40:07.580 INFO:teuthology.orchestra.run.smithi151.stdout:Waiting for mon to start... 2024-04-26T01:40:07.581 INFO:teuthology.orchestra.run.smithi151.stdout:Waiting for mon... 2024-04-26T01:40:07.733 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:07 smithi151 podman[27668]: 2024-04-26 01:40:07.394262617 +0000 UTC m=+0.442944735 container init 8bddb80a33aefe5556cf82aa58fff186f0e9534d169eeadbe7fcfa05e9632e05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-26T01:40:07.733 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:07 smithi151 podman[27668]: 2024-04-26 01:40:07.404706132 +0000 UTC m=+0.453388234 container start 8bddb80a33aefe5556cf82aa58fff186f0e9534d169eeadbe7fcfa05e9632e05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-26T01:40:07.733 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:07 smithi151 ceph-mon[27686]: mkfs c0e49e64-036d-11ef-bc93-c7b262605968 2024-04-26T01:40:07.733 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:07 smithi151 ceph-mon[27686]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-26T01:40:07.733 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:07 smithi151 bash[27668]: 8bddb80a33aefe5556cf82aa58fff186f0e9534d169eeadbe7fcfa05e9632e05 2024-04-26T01:40:07.733 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:07 smithi151 systemd[1]: Started Ceph mon.a for c0e49e64-036d-11ef-bc93-c7b262605968. 2024-04-26T01:40:08.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:08 smithi151 ceph-mon[27686]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-26T01:40:08.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:08 smithi151 ceph-mon[27686]: monmap epoch 1 2024-04-26T01:40:08.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:08 smithi151 ceph-mon[27686]: fsid c0e49e64-036d-11ef-bc93-c7b262605968 2024-04-26T01:40:08.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:08 smithi151 ceph-mon[27686]: last_changed 2024-04-26T01:40:03.179682+0000 2024-04-26T01:40:08.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:08 smithi151 ceph-mon[27686]: created 2024-04-26T01:40:03.179682+0000 2024-04-26T01:40:08.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:08 smithi151 ceph-mon[27686]: min_mon_release 19 (squid) 2024-04-26T01:40:08.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:08 smithi151 ceph-mon[27686]: election_strategy: 1 2024-04-26T01:40:08.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:08 smithi151 ceph-mon[27686]: 0: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.a 2024-04-26T01:40:08.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:08 smithi151 ceph-mon[27686]: fsmap 2024-04-26T01:40:08.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:08 smithi151 ceph-mon[27686]: osdmap e1: 0 total, 0 up, 0 in 2024-04-26T01:40:08.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:08 smithi151 ceph-mon[27686]: mgrmap e1: no daemons active 2024-04-26T01:40:08.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:08 smithi151 ceph-mon[27686]: from='client.? 172.21.15.151:0/4105289246' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-26T01:40:09.010 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout cluster: 2024-04-26T01:40:09.010 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout id: c0e49e64-036d-11ef-bc93-c7b262605968 2024-04-26T01:40:09.010 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-26T01:40:09.010 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout 2024-04-26T01:40:09.010 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout services: 2024-04-26T01:40:09.010 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.821802s) 2024-04-26T01:40:09.010 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-26T01:40:09.010 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-26T01:40:09.011 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout 2024-04-26T01:40:09.011 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout data: 2024-04-26T01:40:09.011 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-26T01:40:09.011 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-26T01:40:09.011 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-26T01:40:09.011 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout pgs: 2024-04-26T01:40:09.011 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout 2024-04-26T01:40:09.011 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:08.212+0000 7f2023e00700 1 Processor -- start 2024-04-26T01:40:09.011 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:08.212+0000 7f2023e00700 1 -- start start 2024-04-26T01:40:09.011 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:08.212+0000 7f2023e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f201c082720 0x7f201c082b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:09.011 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:08.212+0000 7f2023e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f201c083110 con 0x7f201c082720 2024-04-26T01:40:09.011 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:08.213+0000 7f2021400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f201c082720 0x7f201c082b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:09.011 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:08.213+0000 7f2021400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f201c082720 0x7f201c082b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36384/0 (socket says 172.21.15.151:36384) 2024-04-26T01:40:09.011 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:08.213+0000 7f2021400700 1 -- 172.21.15.151:0/666659145 learned_addr learned my addr 172.21.15.151:0/666659145 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:40:09.012 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:08.214+0000 7f2021400700 1 -- 172.21.15.151:0/666659145 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f201c10fef0 con 0x7f201c082720 2024-04-26T01:40:09.012 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:08.214+0000 7f2021400700 1 --2- 172.21.15.151:0/666659145 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f201c082720 0x7f201c082b40 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f20100098a0 tx=0x7f2010009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd1566fe9be711db server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:40:09.012 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:08.214+0000 7f201be00700 1 -- 172.21.15.151:0/666659145 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2010004160 con 0x7f201c082720 2024-04-26T01:40:09.012 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:08.214+0000 7f201be00700 1 -- 172.21.15.151:0/666659145 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2010004300 con 0x7f201c082720 2024-04-26T01:40:09.012 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:08.214+0000 7f201be00700 1 -- 172.21.15.151:0/666659145 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f20100036a0 con 0x7f201c082720 2024-04-26T01:40:09.012 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:08.215+0000 7f2023e00700 1 -- 172.21.15.151:0/666659145 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f201c082720 msgr2=0x7f201c082b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:09.012 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:08.215+0000 7f2023e00700 1 --2- 172.21.15.151:0/666659145 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f201c082720 0x7f201c082b40 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f20100098a0 tx=0x7f2010009bb0 comp rx=0 tx=0).stop 2024-04-26T01:40:09.012 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:08.215+0000 7f2023e00700 1 -- 172.21.15.151:0/666659145 shutdown_connections 2024-04-26T01:40:09.012 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:08.215+0000 7f2023e00700 1 --2- 172.21.15.151:0/666659145 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f201c082720 0x7f201c082b40 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:09.012 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:08.215+0000 7f2023e00700 1 -- 172.21.15.151:0/666659145 >> 172.21.15.151:0/666659145 conn(0x7f201c07dce0 msgr2=0x7f201c080100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:40:09.012 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:08.215+0000 7f2023e00700 1 -- 172.21.15.151:0/666659145 shutdown_connections 2024-04-26T01:40:09.012 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:08.215+0000 7f2023e00700 1 -- 172.21.15.151:0/666659145 wait complete. 2024-04-26T01:40:09.012 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:08.216+0000 7f2023e00700 1 Processor -- start 2024-04-26T01:40:09.013 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:08.216+0000 7f2023e00700 1 -- start start 2024-04-26T01:40:09.013 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:08.216+0000 7f2023e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f201c082720 0x7f201c1a2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:09.013 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:08.216+0000 7f2023e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f201c110420 con 0x7f201c082720 2024-04-26T01:40:09.013 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:08.217+0000 7f2021400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f201c082720 0x7f201c1a2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:09.013 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:08.217+0000 7f2021400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f201c082720 0x7f201c1a2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36394/0 (socket says 172.21.15.151:36394) 2024-04-26T01:40:09.013 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:08.217+0000 7f2021400700 1 -- 172.21.15.151:0/4105289246 learned_addr learned my addr 172.21.15.151:0/4105289246 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:40:09.015 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:08.217+0000 7f2021400700 1 -- 172.21.15.151:0/4105289246 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f201c1a2840 con 0x7f201c082720 2024-04-26T01:40:09.016 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:08.217+0000 7f2021400700 1 --2- 172.21.15.151:0/4105289246 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f201c082720 0x7f201c1a2300 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2010000c00 tx=0x7f2010004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:40:09.016 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:08.218+0000 7f201a000700 1 -- 172.21.15.151:0/4105289246 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2010003db0 con 0x7f201c082720 2024-04-26T01:40:09.016 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:08.218+0000 7f201a000700 1 -- 172.21.15.151:0/4105289246 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2010011630 con 0x7f201c082720 2024-04-26T01:40:09.016 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:08.218+0000 7f2023e00700 1 -- 172.21.15.151:0/4105289246 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f201c1a2ad0 con 0x7f201c082720 2024-04-26T01:40:09.016 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:08.218+0000 7f201a000700 1 -- 172.21.15.151:0/4105289246 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2010011950 con 0x7f201c082720 2024-04-26T01:40:09.016 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:08.218+0000 7f2023e00700 1 -- 172.21.15.151:0/4105289246 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f201c1a57c0 con 0x7f201c082720 2024-04-26T01:40:09.016 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:08.218+0000 7f201a000700 1 -- 172.21.15.151:0/4105289246 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f201001a460 con 0x7f201c082720 2024-04-26T01:40:09.016 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:08.218+0000 7f201a000700 1 -- 172.21.15.151:0/4105289246 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f201001baf0 con 0x7f201c082720 2024-04-26T01:40:09.016 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:08.219+0000 7f2023e00700 1 -- 172.21.15.151:0/4105289246 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f201c0640a0 con 0x7f201c082720 2024-04-26T01:40:09.016 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:08.222+0000 7f201a000700 1 -- 172.21.15.151:0/4105289246 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f2010032080 con 0x7f201c082720 2024-04-26T01:40:09.016 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:08.302+0000 7f2023e00700 1 -- 172.21.15.151:0/4105289246 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f201c02e1d0 con 0x7f201c082720 2024-04-26T01:40:09.016 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:08.302+0000 7f201a000700 1 -- 172.21.15.151:0/4105289246 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f2010004260 con 0x7f201c082720 2024-04-26T01:40:09.017 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:08.304+0000 7f2023e00700 1 -- 172.21.15.151:0/4105289246 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f201c082720 msgr2=0x7f201c1a2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:09.017 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:08.304+0000 7f2023e00700 1 --2- 172.21.15.151:0/4105289246 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f201c082720 0x7f201c1a2300 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2010000c00 tx=0x7f2010004750 comp rx=0 tx=0).stop 2024-04-26T01:40:09.017 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:08.304+0000 7f2023e00700 1 -- 172.21.15.151:0/4105289246 shutdown_connections 2024-04-26T01:40:09.017 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:08.304+0000 7f2023e00700 1 --2- 172.21.15.151:0/4105289246 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f201c082720 0x7f201c1a2300 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:09.017 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:08.304+0000 7f2023e00700 1 -- 172.21.15.151:0/4105289246 >> 172.21.15.151:0/4105289246 conn(0x7f201c07dce0 msgr2=0x7f201c199000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:40:09.017 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:08.304+0000 7f2023e00700 1 -- 172.21.15.151:0/4105289246 shutdown_connections 2024-04-26T01:40:09.017 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:08.304+0000 7f2023e00700 1 -- 172.21.15.151:0/4105289246 wait complete. 2024-04-26T01:40:09.017 INFO:teuthology.orchestra.run.smithi151.stdout:mon is available 2024-04-26T01:40:09.017 INFO:teuthology.orchestra.run.smithi151.stdout:Assimilating anything we can from ceph.conf... 2024-04-26T01:40:10.092 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:09 smithi151 ceph-mon[27686]: from='client.? 172.21.15.151:0/2657015346' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-26T01:40:10.092 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:09 smithi151 ceph-mon[27686]: from='client.? 172.21.15.151:0/2657015346' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-26T01:40:10.443 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout 2024-04-26T01:40:10.443 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout [global] 2024-04-26T01:40:10.444 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout fsid = c0e49e64-036d-11ef-bc93-c7b262605968 2024-04-26T01:40:10.444 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-26T01:40:10.444 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.151:3300,v1:172.21.15.151:6789] 2024-04-26T01:40:10.444 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-26T01:40:10.444 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-26T01:40:10.444 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-26T01:40:10.444 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-26T01:40:10.444 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout 2024-04-26T01:40:10.444 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-26T01:40:10.444 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-26T01:40:10.444 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout 2024-04-26T01:40:10.444 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout [osd] 2024-04-26T01:40:10.444 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-26T01:40:10.445 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-26T01:40:10.445 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-26T01:40:10.445 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-26T01:40:10.445 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-26T01:40:10.445 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-26T01:40:10.445 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:09.651+0000 7f7a6ac00700 1 Processor -- start 2024-04-26T01:40:10.445 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:09.651+0000 7f7a6ac00700 1 -- start start 2024-04-26T01:40:10.445 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:09.652+0000 7f7a6ac00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7a64108700 0x7f7a64108b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:10.445 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:09.652+0000 7f7a6ac00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7a641090f0 con 0x7f7a64108700 2024-04-26T01:40:10.445 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:09.652+0000 7f7a63e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7a64108700 0x7f7a64108b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:10.445 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:09.652+0000 7f7a63e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7a64108700 0x7f7a64108b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36404/0 (socket says 172.21.15.151:36404) 2024-04-26T01:40:10.446 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:09.652+0000 7f7a63e00700 1 -- 172.21.15.151:0/3815106659 learned_addr learned my addr 172.21.15.151:0/3815106659 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:40:10.446 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:09.653+0000 7f7a63e00700 1 -- 172.21.15.151:0/3815106659 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a64109900 con 0x7f7a64108700 2024-04-26T01:40:10.446 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:09.653+0000 7f7a63e00700 1 --2- 172.21.15.151:0/3815106659 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7a64108700 0x7f7a64108b20 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f7a540098a0 tx=0x7f7a54009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c098813e363b6bf7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:40:10.446 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:09.653+0000 7f7a62a00700 1 -- 172.21.15.151:0/3815106659 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a54004160 con 0x7f7a64108700 2024-04-26T01:40:10.446 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:09.654+0000 7f7a62a00700 1 -- 172.21.15.151:0/3815106659 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a54004300 con 0x7f7a64108700 2024-04-26T01:40:10.446 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:09.654+0000 7f7a62a00700 1 -- 172.21.15.151:0/3815106659 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a540036a0 con 0x7f7a64108700 2024-04-26T01:40:10.446 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:09.654+0000 7f7a6ac00700 1 -- 172.21.15.151:0/3815106659 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7a64108700 msgr2=0x7f7a64108b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:10.446 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:09.654+0000 7f7a6ac00700 1 --2- 172.21.15.151:0/3815106659 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7a64108700 0x7f7a64108b20 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f7a540098a0 tx=0x7f7a54009bb0 comp rx=0 tx=0).stop 2024-04-26T01:40:10.446 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:09.654+0000 7f7a6ac00700 1 -- 172.21.15.151:0/3815106659 shutdown_connections 2024-04-26T01:40:10.446 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:09.654+0000 7f7a6ac00700 1 --2- 172.21.15.151:0/3815106659 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7a64108700 0x7f7a64108b20 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:10.446 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:09.654+0000 7f7a6ac00700 1 -- 172.21.15.151:0/3815106659 >> 172.21.15.151:0/3815106659 conn(0x7f7a64103ce0 msgr2=0x7f7a64106100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:40:10.446 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:09.654+0000 7f7a6ac00700 1 -- 172.21.15.151:0/3815106659 shutdown_connections 2024-04-26T01:40:10.447 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:09.655+0000 7f7a6ac00700 1 -- 172.21.15.151:0/3815106659 wait complete. 2024-04-26T01:40:10.447 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:09.656+0000 7f7a6ac00700 1 Processor -- start 2024-04-26T01:40:10.447 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:09.656+0000 7f7a6ac00700 1 -- start start 2024-04-26T01:40:10.447 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:09.656+0000 7f7a6ac00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7a64108700 0x7f7a6419bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:10.447 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:09.656+0000 7f7a6ac00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7a64109e30 con 0x7f7a64108700 2024-04-26T01:40:10.447 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:09.656+0000 7f7a63e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7a64108700 0x7f7a6419bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:10.447 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:09.656+0000 7f7a63e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7a64108700 0x7f7a6419bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36412/0 (socket says 172.21.15.151:36412) 2024-04-26T01:40:10.447 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:09.656+0000 7f7a63e00700 1 -- 172.21.15.151:0/2657015346 learned_addr learned my addr 172.21.15.151:0/2657015346 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:40:10.447 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:09.657+0000 7f7a63e00700 1 -- 172.21.15.151:0/2657015346 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a6419c1c0 con 0x7f7a64108700 2024-04-26T01:40:10.447 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:09.657+0000 7f7a63e00700 1 --2- 172.21.15.151:0/2657015346 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7a64108700 0x7f7a6419bc80 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f7a54000c00 tx=0x7f7a54004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:40:10.447 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:09.657+0000 7f7a60c00700 1 -- 172.21.15.151:0/2657015346 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a54003db0 con 0x7f7a64108700 2024-04-26T01:40:10.447 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:09.657+0000 7f7a60c00700 1 -- 172.21.15.151:0/2657015346 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a54011630 con 0x7f7a64108700 2024-04-26T01:40:10.448 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:09.657+0000 7f7a60c00700 1 -- 172.21.15.151:0/2657015346 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a54011950 con 0x7f7a64108700 2024-04-26T01:40:10.448 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:09.657+0000 7f7a6ac00700 1 -- 172.21.15.151:0/2657015346 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a6419c450 con 0x7f7a64108700 2024-04-26T01:40:10.448 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:09.658+0000 7f7a6ac00700 1 -- 172.21.15.151:0/2657015346 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a6419f140 con 0x7f7a64108700 2024-04-26T01:40:10.448 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:09.658+0000 7f7a60c00700 1 -- 172.21.15.151:0/2657015346 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7a54011af0 con 0x7f7a64108700 2024-04-26T01:40:10.448 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:09.658+0000 7f7a60c00700 1 -- 172.21.15.151:0/2657015346 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7a5401c450 con 0x7f7a64108700 2024-04-26T01:40:10.448 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:09.659+0000 7f7a6ac00700 1 -- 172.21.15.151:0/2657015346 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a640640a0 con 0x7f7a64108700 2024-04-26T01:40:10.448 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:09.661+0000 7f7a60c00700 1 -- 172.21.15.151:0/2657015346 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f7a54032020 con 0x7f7a64108700 2024-04-26T01:40:10.448 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:09.739+0000 7f7a6ac00700 1 -- 172.21.15.151:0/2657015346 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f7a6402e1c0 con 0x7f7a64108700 2024-04-26T01:40:10.448 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:09.746+0000 7f7a60c00700 1 -- 172.21.15.151:0/2657015346 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+494 (secure 0 0 0) 0x7f7a54004260 con 0x7f7a64108700 2024-04-26T01:40:10.448 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:09.747+0000 7f7a6ac00700 1 -- 172.21.15.151:0/2657015346 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7a64108700 msgr2=0x7f7a6419bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:10.448 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:09.747+0000 7f7a6ac00700 1 --2- 172.21.15.151:0/2657015346 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7a64108700 0x7f7a6419bc80 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f7a54000c00 tx=0x7f7a54004750 comp rx=0 tx=0).stop 2024-04-26T01:40:10.448 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:09.747+0000 7f7a6ac00700 1 -- 172.21.15.151:0/2657015346 shutdown_connections 2024-04-26T01:40:10.448 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:09.747+0000 7f7a6ac00700 1 --2- 172.21.15.151:0/2657015346 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7a64108700 0x7f7a6419bc80 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:10.449 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:09.747+0000 7f7a6ac00700 1 -- 172.21.15.151:0/2657015346 >> 172.21.15.151:0/2657015346 conn(0x7f7a64103ce0 msgr2=0x7f7a64192910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:40:10.449 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:09.747+0000 7f7a6ac00700 1 -- 172.21.15.151:0/2657015346 shutdown_connections 2024-04-26T01:40:10.449 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:09.748+0000 7f7a6ac00700 1 -- 172.21.15.151:0/2657015346 wait complete. 2024-04-26T01:40:10.449 INFO:teuthology.orchestra.run.smithi151.stdout:Generating new minimal ceph.conf... 2024-04-26T01:40:11.370 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:11 smithi151 ceph-mon[27686]: from='client.? 172.21.15.151:0/4123662895' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:40:11.910 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:11.018+0000 7f6680000700 1 Processor -- start 2024-04-26T01:40:11.910 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:11.018+0000 7f6680000700 1 -- start start 2024-04-26T01:40:11.910 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:11.019+0000 7f6680000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6678108710 0x7f6678108b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:11.910 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:11.019+0000 7f6680000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6678109100 con 0x7f6678108710 2024-04-26T01:40:11.910 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:11.019+0000 7f667d600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6678108710 0x7f6678108b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:11.910 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:11.019+0000 7f667d600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6678108710 0x7f6678108b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49810/0 (socket says 172.21.15.151:49810) 2024-04-26T01:40:11.910 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:11.019+0000 7f667d600700 1 -- 172.21.15.151:0/768395335 learned_addr learned my addr 172.21.15.151:0/768395335 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:40:11.911 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:11.020+0000 7f667d600700 1 -- 172.21.15.151:0/768395335 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6678109910 con 0x7f6678108710 2024-04-26T01:40:11.911 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:11.020+0000 7f667d600700 1 --2- 172.21.15.151:0/768395335 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6678108710 0x7f6678108b30 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f666c009e80 tx=0x7f666c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=292d2a0b8392b75c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:40:11.911 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:11.020+0000 7f6677e00700 1 -- 172.21.15.151:0/768395335 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f666c017070 con 0x7f6678108710 2024-04-26T01:40:11.911 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:11.020+0000 7f6677e00700 1 -- 172.21.15.151:0/768395335 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f666c00b8c0 con 0x7f6678108710 2024-04-26T01:40:11.911 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:11.020+0000 7f6677e00700 1 -- 172.21.15.151:0/768395335 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f666c004490 con 0x7f6678108710 2024-04-26T01:40:11.911 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:11.021+0000 7f6680000700 1 -- 172.21.15.151:0/768395335 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6678108710 msgr2=0x7f6678108b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:11.911 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:11.021+0000 7f6680000700 1 --2- 172.21.15.151:0/768395335 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6678108710 0x7f6678108b30 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f666c009e80 tx=0x7f666c00b330 comp rx=0 tx=0).stop 2024-04-26T01:40:11.911 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:11.021+0000 7f6680000700 1 -- 172.21.15.151:0/768395335 shutdown_connections 2024-04-26T01:40:11.911 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:11.021+0000 7f6680000700 1 --2- 172.21.15.151:0/768395335 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6678108710 0x7f6678108b30 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:11.911 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:11.021+0000 7f6680000700 1 -- 172.21.15.151:0/768395335 >> 172.21.15.151:0/768395335 conn(0x7f6678103cf0 msgr2=0x7f6678106110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:40:11.914 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:11.021+0000 7f6680000700 1 -- 172.21.15.151:0/768395335 shutdown_connections 2024-04-26T01:40:11.914 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:11.021+0000 7f6680000700 1 -- 172.21.15.151:0/768395335 wait complete. 2024-04-26T01:40:11.915 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:11.022+0000 7f6680000700 1 Processor -- start 2024-04-26T01:40:11.915 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:11.022+0000 7f6680000700 1 -- start start 2024-04-26T01:40:11.915 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:11.023+0000 7f6680000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6678108710 0x7f667819c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:11.915 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:11.023+0000 7f6680000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f667810cb40 con 0x7f6678108710 2024-04-26T01:40:11.915 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:11.023+0000 7f667d600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6678108710 0x7f667819c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:11.915 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:11.023+0000 7f667d600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6678108710 0x7f667819c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49826/0 (socket says 172.21.15.151:49826) 2024-04-26T01:40:11.915 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:11.023+0000 7f667d600700 1 -- 172.21.15.151:0/4123662895 learned_addr learned my addr 172.21.15.151:0/4123662895 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:40:11.915 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:11.023+0000 7f667d600700 1 -- 172.21.15.151:0/4123662895 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f667819ca80 con 0x7f6678108710 2024-04-26T01:40:11.915 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:11.023+0000 7f667d600700 1 --2- 172.21.15.151:0/4123662895 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6678108710 0x7f667819c540 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f666c004a60 tx=0x7f666c00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:40:11.915 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:11.024+0000 7f6676000700 1 -- 172.21.15.151:0/4123662895 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f666c020020 con 0x7f6678108710 2024-04-26T01:40:11.915 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:11.024+0000 7f6676000700 1 -- 172.21.15.151:0/4123662895 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f666c012430 con 0x7f6678108710 2024-04-26T01:40:11.915 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:11.024+0000 7f6676000700 1 -- 172.21.15.151:0/4123662895 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f666c017040 con 0x7f6678108710 2024-04-26T01:40:11.915 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:11.024+0000 7f6680000700 1 -- 172.21.15.151:0/4123662895 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f667819cd10 con 0x7f6678108710 2024-04-26T01:40:11.915 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:11.024+0000 7f6680000700 1 -- 172.21.15.151:0/4123662895 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f667819fa00 con 0x7f6678108710 2024-04-26T01:40:11.916 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:11.025+0000 7f6676000700 1 -- 172.21.15.151:0/4123662895 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f666c023020 con 0x7f6678108710 2024-04-26T01:40:11.916 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:11.025+0000 7f6676000700 1 -- 172.21.15.151:0/4123662895 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f666c00f050 con 0x7f6678108710 2024-04-26T01:40:11.916 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:11.025+0000 7f6680000700 1 -- 172.21.15.151:0/4123662895 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66780640a0 con 0x7f6678108710 2024-04-26T01:40:11.916 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:11.028+0000 7f6676000700 1 -- 172.21.15.151:0/4123662895 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f666c02a020 con 0x7f6678108710 2024-04-26T01:40:11.916 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:11.105+0000 7f6680000700 1 -- 172.21.15.151:0/4123662895 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f667819fbd0 con 0x7f6678108710 2024-04-26T01:40:11.916 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:11.106+0000 7f6676000700 1 -- 172.21.15.151:0/4123662895 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+177 (secure 0 0 0) 0x7f666c0125e0 con 0x7f6678108710 2024-04-26T01:40:11.916 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:11.107+0000 7f6680000700 1 -- 172.21.15.151:0/4123662895 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6678108710 msgr2=0x7f667819c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:11.916 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:11.107+0000 7f6680000700 1 --2- 172.21.15.151:0/4123662895 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6678108710 0x7f667819c540 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f666c004a60 tx=0x7f666c00bd50 comp rx=0 tx=0).stop 2024-04-26T01:40:11.916 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:11.107+0000 7f6680000700 1 -- 172.21.15.151:0/4123662895 shutdown_connections 2024-04-26T01:40:11.916 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:11.107+0000 7f6680000700 1 --2- 172.21.15.151:0/4123662895 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6678108710 0x7f667819c540 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:11.916 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:11.107+0000 7f6680000700 1 -- 172.21.15.151:0/4123662895 >> 172.21.15.151:0/4123662895 conn(0x7f6678103cf0 msgr2=0x7f6678104860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:40:11.916 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:11.107+0000 7f6680000700 1 -- 172.21.15.151:0/4123662895 shutdown_connections 2024-04-26T01:40:11.916 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:11.108+0000 7f6680000700 1 -- 172.21.15.151:0/4123662895 wait complete. 2024-04-26T01:40:11.916 INFO:teuthology.orchestra.run.smithi151.stdout:Restarting the monitor... 2024-04-26T01:40:12.360 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:11 smithi151 systemd[1]: Stopping Ceph mon.a for c0e49e64-036d-11ef-bc93-c7b262605968... 2024-04-26T01:40:12.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:12 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mon-a[27682]: 2024-04-26T01:40:12.361+0000 7f2945c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T01:40:12.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:12 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mon-a[27682]: 2024-04-26T01:40:12.361+0000 7f2945c00700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-26T01:40:12.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:12 smithi151 podman[27927]: 2024-04-26 01:40:12.568530216 +0000 UTC m=+0.335790822 container died 8bddb80a33aefe5556cf82aa58fff186f0e9534d169eeadbe7fcfa05e9632e05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T01:40:22.467 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:22 smithi151 podman[27927]: 2024-04-26 01:40:22.235461236 +0000 UTC m=+10.002721830 container cleanup 8bddb80a33aefe5556cf82aa58fff186f0e9534d169eeadbe7fcfa05e9632e05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2) 2024-04-26T01:40:22.467 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:22 smithi151 bash[27927]: ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mon-a 2024-04-26T01:40:22.870 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:22 smithi151 podman[27941]: 2024-04-26 01:40:22.469792927 +0000 UTC m=+9.875182351 container remove 8bddb80a33aefe5556cf82aa58fff186f0e9534d169eeadbe7fcfa05e9632e05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_CLEAN=True) 2024-04-26T01:40:23.207 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:22 smithi151 systemd[1]: ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-26T01:40:23.207 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:22 smithi151 systemd[1]: Stopped Ceph mon.a for c0e49e64-036d-11ef-bc93-c7b262605968. 2024-04-26T01:40:23.207 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:22 smithi151 systemd[1]: ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mon.a.service: Consumed 1.840s CPU time. 2024-04-26T01:40:23.207 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:22 smithi151 systemd[1]: Starting Ceph mon.a for c0e49e64-036d-11ef-bc93-c7b262605968... 2024-04-26T01:40:23.532 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 podman[28037]: 2024-04-26 01:40:23.345826958 +0000 UTC m=+0.119266373 container create a43466754df1da1ddbf7029de95a586187e5aa19159900abd356e4f333a7083d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-26T01:40:23.532 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 podman[28037]: 2024-04-26 01:40:23.259099766 +0000 UTC m=+0.032539144 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-26T01:40:23.730 INFO:teuthology.orchestra.run.smithi151.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-26T01:40:23.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 podman[28037]: 2024-04-26 01:40:23.633629407 +0000 UTC m=+0.407068846 container init a43466754df1da1ddbf7029de95a586187e5aa19159900abd356e4f333a7083d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mon-a, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T01:40:23.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 podman[28037]: 2024-04-26 01:40:23.64479219 +0000 UTC m=+0.418231578 container start a43466754df1da1ddbf7029de95a586187e5aa19159900abd356e4f333a7083d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mon-a, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-26T01:40:23.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: set uid:gid to 167:167 (ceph:ceph) 2024-04-26T01:40:23.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev), process ceph-mon, pid 2 2024-04-26T01:40:23.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: pidfile_write: ignore empty --pid-file 2024-04-26T01:40:23.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: load: jerasure load: lrc 2024-04-26T01:40:23.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: RocksDB version: 7.9.2 2024-04-26T01:40:23.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Git sha 0 2024-04-26T01:40:23.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Compile date 2024-04-22 10:57:25 2024-04-26T01:40:23.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: DB SUMMARY 2024-04-26T01:40:23.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: DB Session ID: XJOKEQXAF0XUU5T14WB4 2024-04-26T01:40:23.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: CURRENT file: CURRENT 2024-04-26T01:40:23.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: IDENTITY file: IDENTITY 2024-04-26T01:40:23.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-26T01:40:23.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-26T01:40:23.873 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 106066 ; 2024-04-26T01:40:23.873 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.error_if_exists: 0 2024-04-26T01:40:23.873 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.create_if_missing: 0 2024-04-26T01:40:23.873 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.paranoid_checks: 1 2024-04-26T01:40:23.873 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-26T01:40:23.873 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-26T01:40:23.873 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-26T01:40:23.873 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.env: 0x560203e6ccc0 2024-04-26T01:40:23.873 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.fs: PosixFileSystem 2024-04-26T01:40:23.873 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.info_log: 0x5602105a5c40 2024-04-26T01:40:23.873 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.max_file_opening_threads: 16 2024-04-26T01:40:23.873 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.statistics: (nil) 2024-04-26T01:40:23.874 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.use_fsync: 0 2024-04-26T01:40:23.874 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.max_log_file_size: 0 2024-04-26T01:40:23.874 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-26T01:40:23.874 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-26T01:40:23.874 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.keep_log_file_num: 1000 2024-04-26T01:40:23.874 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.recycle_log_file_num: 0 2024-04-26T01:40:23.875 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.allow_fallocate: 1 2024-04-26T01:40:23.875 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.allow_mmap_reads: 0 2024-04-26T01:40:23.875 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.allow_mmap_writes: 0 2024-04-26T01:40:23.875 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.use_direct_reads: 0 2024-04-26T01:40:23.875 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-26T01:40:23.875 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.create_missing_column_families: 0 2024-04-26T01:40:23.875 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.db_log_dir: 2024-04-26T01:40:23.875 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.wal_dir: 2024-04-26T01:40:23.875 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-26T01:40:23.875 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-26T01:40:23.875 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-26T01:40:23.875 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-26T01:40:23.876 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-26T01:40:23.876 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-26T01:40:23.876 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.advise_random_on_open: 1 2024-04-26T01:40:23.876 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.db_write_buffer_size: 0 2024-04-26T01:40:23.876 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.write_buffer_manager: 0x5602105390e0 2024-04-26T01:40:23.876 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-26T01:40:23.876 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-26T01:40:23.876 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-26T01:40:23.876 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.rate_limiter: (nil) 2024-04-26T01:40:23.876 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-26T01:40:23.876 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.wal_recovery_mode: 2 2024-04-26T01:40:23.876 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.enable_thread_tracking: 0 2024-04-26T01:40:23.876 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.enable_pipelined_write: 0 2024-04-26T01:40:23.877 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.unordered_write: 0 2024-04-26T01:40:23.877 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-26T01:40:23.877 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-26T01:40:23.877 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-26T01:40:23.877 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-26T01:40:23.877 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.row_cache: None 2024-04-26T01:40:23.877 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.wal_filter: None 2024-04-26T01:40:23.877 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-26T01:40:23.877 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.allow_ingest_behind: 0 2024-04-26T01:40:23.877 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.two_write_queues: 0 2024-04-26T01:40:23.877 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.manual_wal_flush: 0 2024-04-26T01:40:23.877 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.wal_compression: 0 2024-04-26T01:40:23.877 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.atomic_flush: 0 2024-04-26T01:40:23.878 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-26T01:40:23.878 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-26T01:40:23.878 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-26T01:40:23.878 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.log_readahead_size: 0 2024-04-26T01:40:23.878 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-26T01:40:23.878 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.best_efforts_recovery: 0 2024-04-26T01:40:23.878 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-26T01:40:23.878 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-26T01:40:23.878 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.allow_data_in_errors: 0 2024-04-26T01:40:23.878 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.db_host_id: __hostname__ 2024-04-26T01:40:23.878 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-26T01:40:23.878 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.max_background_jobs: 2 2024-04-26T01:40:23.878 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.max_background_compactions: -1 2024-04-26T01:40:23.879 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.max_subcompactions: 1 2024-04-26T01:40:23.879 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-26T01:40:23.879 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-26T01:40:23.879 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-26T01:40:23.879 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.max_total_wal_size: 0 2024-04-26T01:40:23.879 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-26T01:40:23.879 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-26T01:40:23.879 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-26T01:40:23.879 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-26T01:40:23.879 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.max_open_files: -1 2024-04-26T01:40:23.879 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.bytes_per_sync: 0 2024-04-26T01:40:23.879 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-26T01:40:23.879 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-26T01:40:23.880 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.compaction_readahead_size: 0 2024-04-26T01:40:23.880 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.max_background_flushes: -1 2024-04-26T01:40:23.880 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Compression algorithms supported: 2024-04-26T01:40:23.880 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-26T01:40:23.880 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: kZSTD supported: 0 2024-04-26T01:40:23.880 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: kXpressCompression supported: 0 2024-04-26T01:40:23.880 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: kLZ4HCCompression supported: 1 2024-04-26T01:40:23.880 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: kZlibCompression supported: 1 2024-04-26T01:40:23.880 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: kSnappyCompression supported: 1 2024-04-26T01:40:23.880 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: kLZ4Compression supported: 1 2024-04-26T01:40:23.880 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: kBZip2Compression supported: 0 2024-04-26T01:40:23.880 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-26T01:40:23.880 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-26T01:40:23.881 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-26T01:40:23.881 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-26T01:40:23.881 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-26T01:40:23.881 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.merge_operator: 2024-04-26T01:40:23.881 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.compaction_filter: None 2024-04-26T01:40:23.881 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.compaction_filter_factory: None 2024-04-26T01:40:23.881 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.sst_partitioner_factory: None 2024-04-26T01:40:23.881 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-26T01:40:23.881 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-26T01:40:23.881 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5602105a5e00) 2024-04-26T01:40:23.881 INFO:journalctl@ceph.mon.a.smithi151.stdout: cache_index_and_filter_blocks: 1 2024-04-26T01:40:23.881 INFO:journalctl@ceph.mon.a.smithi151.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-26T01:40:23.881 INFO:journalctl@ceph.mon.a.smithi151.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-26T01:40:23.881 INFO:journalctl@ceph.mon.a.smithi151.stdout: pin_top_level_index_and_filter: 1 2024-04-26T01:40:23.882 INFO:journalctl@ceph.mon.a.smithi151.stdout: index_type: 0 2024-04-26T01:40:23.882 INFO:journalctl@ceph.mon.a.smithi151.stdout: data_block_index_type: 0 2024-04-26T01:40:23.882 INFO:journalctl@ceph.mon.a.smithi151.stdout: index_shortening: 1 2024-04-26T01:40:23.882 INFO:journalctl@ceph.mon.a.smithi151.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-26T01:40:23.882 INFO:journalctl@ceph.mon.a.smithi151.stdout: checksum: 4 2024-04-26T01:40:23.882 INFO:journalctl@ceph.mon.a.smithi151.stdout: no_block_cache: 0 2024-04-26T01:40:23.882 INFO:journalctl@ceph.mon.a.smithi151.stdout: block_cache: 0x56021056b1f0 2024-04-26T01:40:23.882 INFO:journalctl@ceph.mon.a.smithi151.stdout: block_cache_name: BinnedLRUCache 2024-04-26T01:40:23.882 INFO:journalctl@ceph.mon.a.smithi151.stdout: block_cache_options: 2024-04-26T01:40:23.882 INFO:journalctl@ceph.mon.a.smithi151.stdout: capacity : 536870912 2024-04-26T01:40:23.882 INFO:journalctl@ceph.mon.a.smithi151.stdout: num_shard_bits : 4 2024-04-26T01:40:23.882 INFO:journalctl@ceph.mon.a.smithi151.stdout: strict_capacity_limit : 0 2024-04-26T01:40:23.882 INFO:journalctl@ceph.mon.a.smithi151.stdout: high_pri_pool_ratio: 0.000 2024-04-26T01:40:23.883 INFO:journalctl@ceph.mon.a.smithi151.stdout: block_cache_compressed: (nil) 2024-04-26T01:40:23.883 INFO:journalctl@ceph.mon.a.smithi151.stdout: persistent_cache: (nil) 2024-04-26T01:40:23.883 INFO:journalctl@ceph.mon.a.smithi151.stdout: block_size: 4096 2024-04-26T01:40:23.883 INFO:journalctl@ceph.mon.a.smithi151.stdout: block_size_deviation: 10 2024-04-26T01:40:23.883 INFO:journalctl@ceph.mon.a.smithi151.stdout: block_restart_interval: 16 2024-04-26T01:40:23.883 INFO:journalctl@ceph.mon.a.smithi151.stdout: index_block_restart_interval: 1 2024-04-26T01:40:23.883 INFO:journalctl@ceph.mon.a.smithi151.stdout: metadata_block_size: 4096 2024-04-26T01:40:23.883 INFO:journalctl@ceph.mon.a.smithi151.stdout: partition_filters: 0 2024-04-26T01:40:23.883 INFO:journalctl@ceph.mon.a.smithi151.stdout: use_delta_encoding: 1 2024-04-26T01:40:23.883 INFO:journalctl@ceph.mon.a.smithi151.stdout: filter_policy: bloomfilter 2024-04-26T01:40:23.883 INFO:journalctl@ceph.mon.a.smithi151.stdout: whole_key_filtering: 1 2024-04-26T01:40:23.883 INFO:journalctl@ceph.mon.a.smithi151.stdout: verify_compression: 0 2024-04-26T01:40:23.883 INFO:journalctl@ceph.mon.a.smithi151.stdout: read_amp_bytes_per_bit: 0 2024-04-26T01:40:23.883 INFO:journalctl@ceph.mon.a.smithi151.stdout: format_version: 5 2024-04-26T01:40:23.884 INFO:journalctl@ceph.mon.a.smithi151.stdout: enable_index_compression: 1 2024-04-26T01:40:23.884 INFO:journalctl@ceph.mon.a.smithi151.stdout: block_align: 0 2024-04-26T01:40:23.884 INFO:journalctl@ceph.mon.a.smithi151.stdout: max_auto_readahead_size: 262144 2024-04-26T01:40:23.884 INFO:journalctl@ceph.mon.a.smithi151.stdout: prepopulate_block_cache: 0 2024-04-26T01:40:23.884 INFO:journalctl@ceph.mon.a.smithi151.stdout: initial_auto_readahead_size: 8192 2024-04-26T01:40:23.884 INFO:journalctl@ceph.mon.a.smithi151.stdout: num_file_reads_for_auto_readahead: 2 2024-04-26T01:40:23.885 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.write_buffer_size: 33554432 2024-04-26T01:40:23.885 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.max_write_buffer_number: 2 2024-04-26T01:40:23.885 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.compression: NoCompression 2024-04-26T01:40:23.885 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.bottommost_compression: Disabled 2024-04-26T01:40:23.885 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.prefix_extractor: nullptr 2024-04-26T01:40:23.885 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-26T01:40:23.885 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.num_levels: 7 2024-04-26T01:40:23.885 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-26T01:40:23.886 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-26T01:40:23.886 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-26T01:40:23.886 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-26T01:40:23.886 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-26T01:40:23.886 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-26T01:40:23.886 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-26T01:40:23.886 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-26T01:40:23.886 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-26T01:40:23.886 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-26T01:40:23.886 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-26T01:40:23.886 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-26T01:40:23.886 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-26T01:40:23.886 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.compression_opts.level: 32767 2024-04-26T01:40:23.886 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.compression_opts.strategy: 0 2024-04-26T01:40:23.887 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-26T01:40:23.887 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-26T01:40:23.887 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-26T01:40:23.887 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-26T01:40:23.887 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.compression_opts.enabled: false 2024-04-26T01:40:23.887 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-26T01:40:23.887 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-26T01:40:23.887 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-26T01:40:23.887 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-26T01:40:23.887 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.target_file_size_base: 67108864 2024-04-26T01:40:23.887 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-26T01:40:23.887 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-26T01:40:23.887 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-26T01:40:23.888 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-26T01:40:23.888 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-26T01:40:23.888 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-26T01:40:23.888 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-26T01:40:23.888 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-26T01:40:23.888 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-26T01:40:23.888 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-26T01:40:23.888 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-26T01:40:23.888 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-26T01:40:23.888 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-26T01:40:23.888 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-26T01:40:23.888 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.arena_block_size: 1048576 2024-04-26T01:40:23.888 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-26T01:40:23.888 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-26T01:40:23.889 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.disable_auto_compactions: 0 2024-04-26T01:40:23.889 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-26T01:40:23.889 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-26T01:40:23.889 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-26T01:40:23.889 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-26T01:40:23.889 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-26T01:40:23.889 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-26T01:40:23.889 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-26T01:40:23.889 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-26T01:40:23.889 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-26T01:40:23.889 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-26T01:40:23.889 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-26T01:40:23.889 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.inplace_update_support: 0 2024-04-26T01:40:23.889 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-26T01:40:23.890 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-26T01:40:23.890 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-26T01:40:23.890 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-26T01:40:23.890 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.bloom_locality: 0 2024-04-26T01:40:23.890 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.max_successive_merges: 0 2024-04-26T01:40:23.890 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-26T01:40:23.890 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.paranoid_file_checks: 0 2024-04-26T01:40:23.890 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.force_consistency_checks: 1 2024-04-26T01:40:23.890 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.report_bg_io_stats: 0 2024-04-26T01:40:23.890 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.ttl: 2592000 2024-04-26T01:40:23.890 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-26T01:40:23.890 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-26T01:40:23.890 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-26T01:40:23.890 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.enable_blob_files: false 2024-04-26T01:40:23.891 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.min_blob_size: 0 2024-04-26T01:40:23.891 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.blob_file_size: 268435456 2024-04-26T01:40:23.891 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-26T01:40:23.891 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-26T01:40:23.891 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-26T01:40:23.891 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-26T01:40:23.891 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-26T01:40:23.891 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.blob_file_starting_level: 0 2024-04-26T01:40:23.891 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-26T01:40:23.891 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-26T01:40:23.891 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-26T01:40:23.891 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: c14a9745-8dcb-4d41-97c8-11ce92de9f02 2024-04-26T01:40:23.891 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714095623698980, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-26T01:40:23.891 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-26T01:40:23.892 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714095623701067, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 101118, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 330, "table_properties": {"data_size": 99007, "index_size": 303, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14405, "raw_average_key_size": 49, "raw_value_size": 91186, "raw_average_value_size": 315, "num_data_blocks": 12, "num_entries": 289, "num_filter_entries": 289, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714095623, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "c14a9745-8dcb-4d41-97c8-11ce92de9f02", "db_session_id": "XJOKEQXAF0XUU5T14WB4", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-26T01:40:23.892 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714095623701271, "job": 1, "event": "recovery_finished"} 2024-04-26T01:40:23.892 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-26T01:40:23.892 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-26T01:40:23.892 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x560210680000 2024-04-26T01:40:23.892 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: DB pointer 0x56021066a000 2024-04-26T01:40:23.892 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-26T01:40:23.892 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-26T01:40:23.892 INFO:journalctl@ceph.mon.a.smithi151.stdout: ** DB Stats ** 2024-04-26T01:40:23.892 INFO:journalctl@ceph.mon.a.smithi151.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T01:40:23.892 INFO:journalctl@ceph.mon.a.smithi151.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-26T01:40:23.892 INFO:journalctl@ceph.mon.a.smithi151.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T01:40:23.892 INFO:journalctl@ceph.mon.a.smithi151.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T01:40:23.892 INFO:journalctl@ceph.mon.a.smithi151.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-26T01:40:23.893 INFO:journalctl@ceph.mon.a.smithi151.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T01:40:23.893 INFO:journalctl@ceph.mon.a.smithi151.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T01:40:23.893 INFO:journalctl@ceph.mon.a.smithi151.stdout: 2024-04-26T01:40:23.893 INFO:journalctl@ceph.mon.a.smithi151.stdout: ** Compaction Stats [default] ** 2024-04-26T01:40:23.893 INFO:journalctl@ceph.mon.a.smithi151.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T01:40:23.893 INFO:journalctl@ceph.mon.a.smithi151.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-26T01:40:23.893 INFO:journalctl@ceph.mon.a.smithi151.stdout: L0 2/0 100.61 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 73.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T01:40:23.893 INFO:journalctl@ceph.mon.a.smithi151.stdout: Sum 2/0 100.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 73.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T01:40:23.893 INFO:journalctl@ceph.mon.a.smithi151.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 73.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T01:40:23.893 INFO:journalctl@ceph.mon.a.smithi151.stdout: 2024-04-26T01:40:23.893 INFO:journalctl@ceph.mon.a.smithi151.stdout: ** Compaction Stats [default] ** 2024-04-26T01:40:23.893 INFO:journalctl@ceph.mon.a.smithi151.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T01:40:23.894 INFO:journalctl@ceph.mon.a.smithi151.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-26T01:40:23.894 INFO:journalctl@ceph.mon.a.smithi151.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 73.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T01:40:23.894 INFO:journalctl@ceph.mon.a.smithi151.stdout: 2024-04-26T01:40:23.894 INFO:journalctl@ceph.mon.a.smithi151.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-26T01:40:23.894 INFO:journalctl@ceph.mon.a.smithi151.stdout: 2024-04-26T01:40:23.894 INFO:journalctl@ceph.mon.a.smithi151.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T01:40:23.894 INFO:journalctl@ceph.mon.a.smithi151.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-26T01:40:23.894 INFO:journalctl@ceph.mon.a.smithi151.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-26T01:40:23.894 INFO:journalctl@ceph.mon.a.smithi151.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-26T01:40:23.894 INFO:journalctl@ceph.mon.a.smithi151.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-26T01:40:23.894 INFO:journalctl@ceph.mon.a.smithi151.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-26T01:40:23.894 INFO:journalctl@ceph.mon.a.smithi151.stdout: Cumulative compaction: 0.00 GB write, 14.89 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T01:40:23.895 INFO:journalctl@ceph.mon.a.smithi151.stdout: Interval compaction: 0.00 GB write, 14.89 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T01:40:23.895 INFO:journalctl@ceph.mon.a.smithi151.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-26T01:40:23.895 INFO:journalctl@ceph.mon.a.smithi151.stdout: Block cache BinnedLRUCache@0x56021056b1f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-04-26T01:40:23.895 INFO:journalctl@ceph.mon.a.smithi151.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-26T01:40:23.895 INFO:journalctl@ceph.mon.a.smithi151.stdout: 2024-04-26T01:40:23.895 INFO:journalctl@ceph.mon.a.smithi151.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-26T01:40:23.895 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: starting mon.a rank 0 at public addrs [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] at bind addrs [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid c0e49e64-036d-11ef-bc93-c7b262605968 2024-04-26T01:40:23.895 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: mon.a@-1(???) e1 preinit fsid c0e49e64-036d-11ef-bc93-c7b262605968 2024-04-26T01:40:23.895 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: mon.a@-1(???).mds e1 new map 2024-04-26T01:40:23.895 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: mon.a@-1(???).mds e1 print_map 2024-04-26T01:40:23.895 INFO:journalctl@ceph.mon.a.smithi151.stdout: e1 2024-04-26T01:40:23.895 INFO:journalctl@ceph.mon.a.smithi151.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T01:40:23.895 INFO:journalctl@ceph.mon.a.smithi151.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T01:40:23.895 INFO:journalctl@ceph.mon.a.smithi151.stdout: legacy client fscid: -1 2024-04-26T01:40:23.896 INFO:journalctl@ceph.mon.a.smithi151.stdout: 2024-04-26T01:40:23.896 INFO:journalctl@ceph.mon.a.smithi151.stdout: No filesystems configured 2024-04-26T01:40:23.896 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-26T01:40:23.896 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T01:40:23.896 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T01:40:23.896 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T01:40:23.896 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-26T01:40:23.896 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 bash[28037]: a43466754df1da1ddbf7029de95a586187e5aa19159900abd356e4f333a7083d 2024-04-26T01:40:23.896 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 systemd[1]: Started Ceph mon.a for c0e49e64-036d-11ef-bc93-c7b262605968. 2024-04-26T01:40:23.896 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-26T01:40:23.896 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: monmap epoch 1 2024-04-26T01:40:23.896 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: fsid c0e49e64-036d-11ef-bc93-c7b262605968 2024-04-26T01:40:23.896 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: last_changed 2024-04-26T01:40:03.179682+0000 2024-04-26T01:40:23.896 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: created 2024-04-26T01:40:03.179682+0000 2024-04-26T01:40:23.897 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: min_mon_release 19 (squid) 2024-04-26T01:40:23.897 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: election_strategy: 1 2024-04-26T01:40:23.897 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: 0: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.a 2024-04-26T01:40:23.897 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: fsmap 2024-04-26T01:40:23.897 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: osdmap e1: 0 total, 0 up, 0 in 2024-04-26T01:40:23.897 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:23 smithi151 ceph-mon[28057]: mgrmap e1: no daemons active 2024-04-26T01:40:25.161 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:24.376+0000 7f1863e00700 1 Processor -- start 2024-04-26T01:40:25.166 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:24.377+0000 7f1863e00700 1 -- start start 2024-04-26T01:40:25.166 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:24.377+0000 7f1863e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f185c07dda0 0x7f185c07c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:25.166 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:24.377+0000 7f1863e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f185c07c7d0 con 0x7f185c07dda0 2024-04-26T01:40:25.166 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:24.378+0000 7f1861400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f185c07dda0 0x7f185c07c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:25.166 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:24.378+0000 7f1861400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f185c07dda0 0x7f185c07c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53072/0 (socket says 172.21.15.151:53072) 2024-04-26T01:40:25.166 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:24.378+0000 7f1861400700 1 -- 172.21.15.151:0/233312770 learned_addr learned my addr 172.21.15.151:0/233312770 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:40:25.166 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:24.378+0000 7f1861400700 1 -- 172.21.15.151:0/233312770 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f185c07c950 con 0x7f185c07dda0 2024-04-26T01:40:25.166 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:24.378+0000 7f1861400700 1 --2- 172.21.15.151:0/233312770 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f185c07dda0 0x7f185c07c200 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f18500098a0 tx=0x7f1850009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c21baebf466f97ce server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:40:25.167 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:24.379+0000 7f185be00700 1 -- 172.21.15.151:0/233312770 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1850004160 con 0x7f185c07dda0 2024-04-26T01:40:25.167 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:24.379+0000 7f185be00700 1 -- 172.21.15.151:0/233312770 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1850004300 con 0x7f185c07dda0 2024-04-26T01:40:25.167 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:24.379+0000 7f185be00700 1 -- 172.21.15.151:0/233312770 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1850011af0 con 0x7f185c07dda0 2024-04-26T01:40:25.167 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:24.379+0000 7f1863e00700 1 -- 172.21.15.151:0/233312770 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f185c07dda0 msgr2=0x7f185c07c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:25.167 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:24.379+0000 7f1863e00700 1 --2- 172.21.15.151:0/233312770 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f185c07dda0 0x7f185c07c200 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f18500098a0 tx=0x7f1850009bb0 comp rx=0 tx=0).stop 2024-04-26T01:40:25.167 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:24.380+0000 7f1863e00700 1 -- 172.21.15.151:0/233312770 shutdown_connections 2024-04-26T01:40:25.167 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:24.380+0000 7f1863e00700 1 --2- 172.21.15.151:0/233312770 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f185c07dda0 0x7f185c07c200 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:25.167 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:24.380+0000 7f1863e00700 1 -- 172.21.15.151:0/233312770 >> 172.21.15.151:0/233312770 conn(0x7f185c103c40 msgr2=0x7f185c1060a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:40:25.167 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:24.380+0000 7f1863e00700 1 -- 172.21.15.151:0/233312770 shutdown_connections 2024-04-26T01:40:25.168 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:24.380+0000 7f1863e00700 1 -- 172.21.15.151:0/233312770 wait complete. 2024-04-26T01:40:25.168 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:24.381+0000 7f1863e00700 1 Processor -- start 2024-04-26T01:40:25.168 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:24.381+0000 7f1863e00700 1 -- start start 2024-04-26T01:40:25.168 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:24.381+0000 7f1863e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f185c07dda0 0x7f185c19e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:25.168 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:24.381+0000 7f1863e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f185c10eba0 con 0x7f185c07dda0 2024-04-26T01:40:25.168 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:24.381+0000 7f1861400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f185c07dda0 0x7f185c19e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:25.168 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:24.382+0000 7f1861400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f185c07dda0 0x7f185c19e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53080/0 (socket says 172.21.15.151:53080) 2024-04-26T01:40:25.168 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:24.382+0000 7f1861400700 1 -- 172.21.15.151:0/2511677927 learned_addr learned my addr 172.21.15.151:0/2511677927 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:40:25.168 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:24.382+0000 7f1861400700 1 -- 172.21.15.151:0/2511677927 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f185c19ec70 con 0x7f185c07dda0 2024-04-26T01:40:25.168 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:24.382+0000 7f1861400700 1 --2- 172.21.15.151:0/2511677927 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f185c07dda0 0x7f185c19e730 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1850000c00 tx=0x7f1850011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:40:25.168 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:24.382+0000 7f185a000700 1 -- 172.21.15.151:0/2511677927 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1850003b20 con 0x7f185c07dda0 2024-04-26T01:40:25.168 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:24.383+0000 7f1863e00700 1 -- 172.21.15.151:0/2511677927 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f185c19ef00 con 0x7f185c07dda0 2024-04-26T01:40:25.168 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:24.383+0000 7f185a000700 1 -- 172.21.15.151:0/2511677927 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f185001a8c0 con 0x7f185c07dda0 2024-04-26T01:40:25.169 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:24.383+0000 7f185a000700 1 -- 172.21.15.151:0/2511677927 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1850024940 con 0x7f185c07dda0 2024-04-26T01:40:25.169 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:24.383+0000 7f1863e00700 1 -- 172.21.15.151:0/2511677927 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f185c1a1bf0 con 0x7f185c07dda0 2024-04-26T01:40:25.169 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:24.383+0000 7f185a000700 1 -- 172.21.15.151:0/2511677927 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1850022020 con 0x7f185c07dda0 2024-04-26T01:40:25.169 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:24.383+0000 7f185a000700 1 -- 172.21.15.151:0/2511677927 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f185002dde0 con 0x7f185c07dda0 2024-04-26T01:40:25.169 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:24.384+0000 7f1863e00700 1 -- 172.21.15.151:0/2511677927 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f185c0640a0 con 0x7f185c07dda0 2024-04-26T01:40:25.169 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:24.387+0000 7f185a000700 1 -- 172.21.15.151:0/2511677927 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f18500042c0 con 0x7f185c07dda0 2024-04-26T01:40:25.169 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:24.463+0000 7f1863e00700 1 -- 172.21.15.151:0/2511677927 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f185c1a1dc0 con 0x7f185c07dda0 2024-04-26T01:40:25.169 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:24.468+0000 7f185a000700 1 -- 172.21.15.151:0/2511677927 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f185001aa70 con 0x7f185c07dda0 2024-04-26T01:40:25.169 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:24.470+0000 7f1863e00700 1 -- 172.21.15.151:0/2511677927 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f185c07dda0 msgr2=0x7f185c19e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:25.169 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:24.470+0000 7f1863e00700 1 --2- 172.21.15.151:0/2511677927 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f185c07dda0 0x7f185c19e730 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1850000c00 tx=0x7f1850011b70 comp rx=0 tx=0).stop 2024-04-26T01:40:25.169 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:24.470+0000 7f1863e00700 1 -- 172.21.15.151:0/2511677927 shutdown_connections 2024-04-26T01:40:25.169 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:24.470+0000 7f1863e00700 1 --2- 172.21.15.151:0/2511677927 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f185c07dda0 0x7f185c19e730 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:25.169 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:24.470+0000 7f1863e00700 1 -- 172.21.15.151:0/2511677927 >> 172.21.15.151:0/2511677927 conn(0x7f185c103c40 msgr2=0x7f185c1047e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:40:25.169 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:24.471+0000 7f1863e00700 1 -- 172.21.15.151:0/2511677927 shutdown_connections 2024-04-26T01:40:25.170 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:24.471+0000 7f1863e00700 1 -- 172.21.15.151:0/2511677927 wait complete. 2024-04-26T01:40:25.170 INFO:teuthology.orchestra.run.smithi151.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-26T01:40:25.187 INFO:teuthology.orchestra.run.smithi151.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-26T01:40:25.187 INFO:teuthology.orchestra.run.smithi151.stdout:Creating mgr... 2024-04-26T01:40:25.188 INFO:teuthology.orchestra.run.smithi151.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-26T01:40:25.188 INFO:teuthology.orchestra.run.smithi151.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-26T01:40:25.627 INFO:teuthology.orchestra.run.smithi151.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mgr.x 2024-04-26T01:40:25.628 INFO:teuthology.orchestra.run.smithi151.stdout:systemctl: stderr Failed to reset failed state of unit ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-26T01:40:25.870 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:25 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/2511677927' entity='client.admin' 2024-04-26T01:40:25.884 INFO:teuthology.orchestra.run.smithi151.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-c0e49e64-036d-11ef-bc93-c7b262605968.target.wants/ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-c0e49e64-036d-11ef-bc93-c7b262605968@.service. 2024-04-26T01:40:26.268 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:25 smithi151 systemd[1]: Starting Ceph mgr.x for c0e49e64-036d-11ef-bc93-c7b262605968... 2024-04-26T01:40:26.523 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:26 smithi151 podman[28281]: 2024-04-26 01:40:26.175567356 +0000 UTC m=+0.030314255 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-26T01:40:26.523 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:26 smithi151 podman[28281]: 2024-04-26 01:40:26.305959825 +0000 UTC m=+0.160706749 container create 463671658fa6ff519b25f1dba200036f07730f97c4647dadfa73cf303ba45bb5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-26T01:40:26.523 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:26 smithi151 podman[28281]: 2024-04-26 01:40:26.525257378 +0000 UTC m=+0.380004309 container init 463671658fa6ff519b25f1dba200036f07730f97c4647dadfa73cf303ba45bb5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-26T01:40:26.654 INFO:teuthology.orchestra.run.smithi151.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T01:40:26.655 INFO:teuthology.orchestra.run.smithi151.stdout:systemctl: stdout disabled 2024-04-26T01:40:26.676 INFO:teuthology.orchestra.run.smithi151.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T01:40:26.676 INFO:teuthology.orchestra.run.smithi151.stdout:systemctl: stdout inactive 2024-04-26T01:40:26.676 INFO:teuthology.orchestra.run.smithi151.stdout:firewalld.service is not enabled 2024-04-26T01:40:26.676 INFO:teuthology.orchestra.run.smithi151.stdout:Not possible to enable service . firewalld.service is not available 2024-04-26T01:40:26.705 INFO:teuthology.orchestra.run.smithi151.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T01:40:26.705 INFO:teuthology.orchestra.run.smithi151.stdout:systemctl: stdout disabled 2024-04-26T01:40:26.724 INFO:teuthology.orchestra.run.smithi151.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T01:40:26.725 INFO:teuthology.orchestra.run.smithi151.stdout:systemctl: stdout inactive 2024-04-26T01:40:26.725 INFO:teuthology.orchestra.run.smithi151.stdout:firewalld.service is not enabled 2024-04-26T01:40:26.725 INFO:teuthology.orchestra.run.smithi151.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-26T01:40:26.725 INFO:teuthology.orchestra.run.smithi151.stdout:Waiting for mgr to start... 2024-04-26T01:40:26.725 INFO:teuthology.orchestra.run.smithi151.stdout:Waiting for mgr... 2024-04-26T01:40:26.783 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:26 smithi151 podman[28281]: 2024-04-26 01:40:26.534850732 +0000 UTC m=+0.389597663 container start 463671658fa6ff519b25f1dba200036f07730f97c4647dadfa73cf303ba45bb5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-26T01:40:26.783 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:26 smithi151 bash[28281]: 463671658fa6ff519b25f1dba200036f07730f97c4647dadfa73cf303ba45bb5 2024-04-26T01:40:26.783 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:26 smithi151 systemd[1]: Started Ceph mgr.x for c0e49e64-036d-11ef-bc93-c7b262605968. 2024-04-26T01:40:27.036 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:26 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:40:26.785+0000 7f05a8bfd200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T01:40:27.036 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:27 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:40:27.038+0000 7f05a8bfd200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T01:40:27.292 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:27 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:40:27.177+0000 7f05a8bfd200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T01:40:27.596 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:27 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/2620315415' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T01:40:27.870 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:27 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:40:27.599+0000 7f05a8bfd200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T01:40:27.870 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:27 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:40:27.708+0000 7f05a8bfd200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T01:40:28.714 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout 2024-04-26T01:40:28.714 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout { 2024-04-26T01:40:28.714 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "fsid": "c0e49e64-036d-11ef-bc93-c7b262605968", 2024-04-26T01:40:28.714 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T01:40:28.714 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T01:40:28.714 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T01:40:28.714 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T01:40:28.714 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:40:28.714 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T01:40:28.714 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T01:40:28.715 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout 0 2024-04-26T01:40:28.715 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout ], 2024-04-26T01:40:28.715 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T01:40:28.715 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "a" 2024-04-26T01:40:28.715 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout ], 2024-04-26T01:40:28.715 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-04-26T01:40:28.715 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T01:40:28.715 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T01:40:28.715 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-26T01:40:28.715 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T01:40:28.715 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:40:28.715 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T01:40:28.715 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T01:40:28.715 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T01:40:28.715 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T01:40:28.716 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T01:40:28.724 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T01:40:28.724 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T01:40:28.724 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T01:40:28.724 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:40:28.724 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T01:40:28.724 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T01:40:28.724 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T01:40:28.724 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T01:40:28.725 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T01:40:28.725 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T01:40:28.725 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T01:40:28.725 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T01:40:28.725 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T01:40:28.725 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:40:28.725 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T01:40:28.725 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T01:40:28.725 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T01:40:28.725 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T01:40:28.725 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:40:28.725 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T01:40:28.725 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-26T01:40:28.725 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T01:40:28.726 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T01:40:28.726 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T01:40:28.726 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T01:40:28.726 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T01:40:28.726 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout ], 2024-04-26T01:40:28.726 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T01:40:28.726 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:40:28.726 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T01:40:28.726 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T01:40:28.726 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T01:40:07.483797+0000", 2024-04-26T01:40:28.726 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T01:40:28.726 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:40:28.726 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T01:40:28.727 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout } 2024-04-26T01:40:28.727 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:27.294+0000 7f961fc00700 1 Processor -- start 2024-04-26T01:40:28.727 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:27.294+0000 7f961fc00700 1 -- start start 2024-04-26T01:40:28.727 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:27.294+0000 7f961fc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9618109170 0x7f961810b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:28.727 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:27.294+0000 7f961fc00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9618063d00 con 0x7f9618109170 2024-04-26T01:40:28.727 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:27.295+0000 7f961d200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9618109170 0x7f961810b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:28.727 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:27.296+0000 7f961d200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9618109170 0x7f961810b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53110/0 (socket says 172.21.15.151:53110) 2024-04-26T01:40:28.727 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:27.296+0000 7f961d200700 1 -- 172.21.15.151:0/2573277103 learned_addr learned my addr 172.21.15.151:0/2573277103 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:40:28.727 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:27.296+0000 7f961d200700 1 -- 172.21.15.151:0/2573277103 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f961810baa0 con 0x7f9618109170 2024-04-26T01:40:28.727 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:27.296+0000 7f961d200700 1 --2- 172.21.15.151:0/2573277103 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9618109170 0x7f961810b560 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f960c0098a0 tx=0x7f960c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ba30bf98c42d2ae server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:40:28.727 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:27.296+0000 7f9617400700 1 -- 172.21.15.151:0/2573277103 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f960c004160 con 0x7f9618109170 2024-04-26T01:40:28.727 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:27.297+0000 7f9617400700 1 -- 172.21.15.151:0/2573277103 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f960c004300 con 0x7f9618109170 2024-04-26T01:40:28.727 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:27.297+0000 7f9617400700 1 -- 172.21.15.151:0/2573277103 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f960c011af0 con 0x7f9618109170 2024-04-26T01:40:28.728 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:27.297+0000 7f961fc00700 1 -- 172.21.15.151:0/2573277103 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9618109170 msgr2=0x7f961810b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:28.728 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:27.297+0000 7f961fc00700 1 --2- 172.21.15.151:0/2573277103 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9618109170 0x7f961810b560 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f960c0098a0 tx=0x7f960c009bb0 comp rx=0 tx=0).stop 2024-04-26T01:40:28.728 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:27.297+0000 7f961fc00700 1 -- 172.21.15.151:0/2573277103 shutdown_connections 2024-04-26T01:40:28.728 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:27.297+0000 7f961fc00700 1 --2- 172.21.15.151:0/2573277103 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9618109170 0x7f961810b560 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:28.728 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:27.297+0000 7f961fc00700 1 -- 172.21.15.151:0/2573277103 >> 172.21.15.151:0/2573277103 conn(0x7f9618102ba0 msgr2=0x7f9618104fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:40:28.728 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:27.297+0000 7f961fc00700 1 -- 172.21.15.151:0/2573277103 shutdown_connections 2024-04-26T01:40:28.728 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:27.298+0000 7f961fc00700 1 -- 172.21.15.151:0/2573277103 wait complete. 2024-04-26T01:40:28.728 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:27.299+0000 7f961fc00700 1 Processor -- start 2024-04-26T01:40:28.728 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:27.299+0000 7f961fc00700 1 -- start start 2024-04-26T01:40:28.728 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:27.299+0000 7f961fc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9618109170 0x7f961819c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:28.728 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:27.299+0000 7f961fc00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f961810c930 con 0x7f9618109170 2024-04-26T01:40:28.728 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:27.299+0000 7f961d200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9618109170 0x7f961819c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:28.728 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:27.299+0000 7f961d200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9618109170 0x7f961819c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53112/0 (socket says 172.21.15.151:53112) 2024-04-26T01:40:28.728 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:27.299+0000 7f961d200700 1 -- 172.21.15.151:0/2620315415 learned_addr learned my addr 172.21.15.151:0/2620315415 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:40:28.729 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:27.300+0000 7f961d200700 1 -- 172.21.15.151:0/2620315415 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f961819ca00 con 0x7f9618109170 2024-04-26T01:40:28.729 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:27.300+0000 7f961d200700 1 --2- 172.21.15.151:0/2620315415 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9618109170 0x7f961819c4c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f960c011ac0 tx=0x7f960c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:40:28.729 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:27.300+0000 7f9615600700 1 -- 172.21.15.151:0/2620315415 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f960c003b20 con 0x7f9618109170 2024-04-26T01:40:28.729 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:27.300+0000 7f9615600700 1 -- 172.21.15.151:0/2620315415 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f960c01a8c0 con 0x7f9618109170 2024-04-26T01:40:28.729 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:27.300+0000 7f9615600700 1 -- 172.21.15.151:0/2620315415 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f960c0248d0 con 0x7f9618109170 2024-04-26T01:40:28.729 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:27.300+0000 7f961fc00700 1 -- 172.21.15.151:0/2620315415 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f961819cc90 con 0x7f9618109170 2024-04-26T01:40:28.729 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:27.300+0000 7f961fc00700 1 -- 172.21.15.151:0/2620315415 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f961819f980 con 0x7f9618109170 2024-04-26T01:40:28.729 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:27.301+0000 7f9615600700 1 -- 172.21.15.151:0/2620315415 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f960c022020 con 0x7f9618109170 2024-04-26T01:40:28.729 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:27.301+0000 7f9615600700 1 -- 172.21.15.151:0/2620315415 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f960c0193c0 con 0x7f9618109170 2024-04-26T01:40:28.729 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:27.301+0000 7f961fc00700 1 -- 172.21.15.151:0/2620315415 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96180640a0 con 0x7f9618109170 2024-04-26T01:40:28.729 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:27.304+0000 7f9615600700 1 -- 172.21.15.151:0/2620315415 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f960c042e50 con 0x7f9618109170 2024-04-26T01:40:28.729 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:27.383+0000 7f961fc00700 1 -- 172.21.15.151:0/2620315415 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f961819fed0 con 0x7f9618109170 2024-04-26T01:40:28.729 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:27.384+0000 7f9615600700 1 -- 172.21.15.151:0/2620315415 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f960c031020 con 0x7f9618109170 2024-04-26T01:40:28.730 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:27.386+0000 7f961fc00700 1 -- 172.21.15.151:0/2620315415 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9618109170 msgr2=0x7f961819c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:28.730 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:27.386+0000 7f961fc00700 1 --2- 172.21.15.151:0/2620315415 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9618109170 0x7f961819c4c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f960c011ac0 tx=0x7f960c011ba0 comp rx=0 tx=0).stop 2024-04-26T01:40:28.730 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:27.386+0000 7f961fc00700 1 -- 172.21.15.151:0/2620315415 shutdown_connections 2024-04-26T01:40:28.730 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:27.386+0000 7f961fc00700 1 --2- 172.21.15.151:0/2620315415 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9618109170 0x7f961819c4c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:28.730 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:27.386+0000 7f961fc00700 1 -- 172.21.15.151:0/2620315415 >> 172.21.15.151:0/2620315415 conn(0x7f9618102ba0 msgr2=0x7f96181036d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:40:28.730 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:27.386+0000 7f961fc00700 1 -- 172.21.15.151:0/2620315415 shutdown_connections 2024-04-26T01:40:28.730 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:27.386+0000 7f961fc00700 1 -- 172.21.15.151:0/2620315415 wait complete. 2024-04-26T01:40:28.730 INFO:teuthology.orchestra.run.smithi151.stdout:mgr not available, waiting (1/15)... 2024-04-26T01:40:29.620 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:29 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:40:29.329+0000 7f05a8bfd200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T01:40:29.620 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:29 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:40:29.552+0000 7f05a8bfd200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T01:40:30.120 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:29 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:40:29.763+0000 7f05a8bfd200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T01:40:30.506 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:30 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:40:30.138+0000 7f05a8bfd200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T01:40:30.507 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:30 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:40:30.258+0000 7f05a8bfd200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T01:40:30.761 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:30 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:40:30.509+0000 7f05a8bfd200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T01:40:30.761 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:30 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:40:30.764+0000 7f05a8bfd200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T01:40:31.592 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:31 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/2085650128' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T01:40:31.855 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:31 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:40:31.595+0000 7f05a8bfd200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T01:40:32.069 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout 2024-04-26T01:40:32.069 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout { 2024-04-26T01:40:32.069 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "fsid": "c0e49e64-036d-11ef-bc93-c7b262605968", 2024-04-26T01:40:32.069 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T01:40:32.069 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T01:40:32.070 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T01:40:32.070 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T01:40:32.070 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:40:32.070 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T01:40:32.070 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T01:40:32.070 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout 0 2024-04-26T01:40:32.070 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout ], 2024-04-26T01:40:32.070 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T01:40:32.070 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "a" 2024-04-26T01:40:32.070 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout ], 2024-04-26T01:40:32.070 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-04-26T01:40:32.070 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T01:40:32.071 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T01:40:32.071 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-26T01:40:32.071 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T01:40:32.071 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:40:32.071 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T01:40:32.071 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T01:40:32.071 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T01:40:32.071 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T01:40:32.071 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T01:40:32.071 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T01:40:32.071 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T01:40:32.071 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T01:40:32.071 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:40:32.072 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T01:40:32.072 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T01:40:32.072 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T01:40:32.072 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T01:40:32.072 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T01:40:32.072 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T01:40:32.072 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T01:40:32.072 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T01:40:32.072 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T01:40:32.072 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:40:32.072 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T01:40:32.072 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T01:40:32.072 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T01:40:32.072 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T01:40:32.073 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:40:32.073 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T01:40:32.073 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-26T01:40:32.073 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T01:40:32.073 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T01:40:32.073 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T01:40:32.073 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T01:40:32.073 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T01:40:32.073 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout ], 2024-04-26T01:40:32.073 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T01:40:32.073 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:40:32.073 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T01:40:32.073 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T01:40:32.073 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T01:40:07.483797+0000", 2024-04-26T01:40:32.074 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T01:40:32.074 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:40:32.074 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T01:40:32.074 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout } 2024-04-26T01:40:32.074 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:31.288+0000 7f8e39000700 1 Processor -- start 2024-04-26T01:40:32.074 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:31.288+0000 7f8e39000700 1 -- start start 2024-04-26T01:40:32.074 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:31.288+0000 7f8e39000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e3407dd80 0x7f8e3407c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:32.074 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:31.288+0000 7f8e39000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3407c7b0 con 0x7f8e3407dd80 2024-04-26T01:40:32.074 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:31.289+0000 7f8e31e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e3407dd80 0x7f8e3407c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:32.074 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:31.289+0000 7f8e31e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e3407dd80 0x7f8e3407c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36188/0 (socket says 172.21.15.151:36188) 2024-04-26T01:40:32.074 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:31.289+0000 7f8e31e00700 1 -- 172.21.15.151:0/3328610758 learned_addr learned my addr 172.21.15.151:0/3328610758 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:40:32.074 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:31.289+0000 7f8e31e00700 1 -- 172.21.15.151:0/3328610758 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e3407c930 con 0x7f8e3407dd80 2024-04-26T01:40:32.074 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:31.290+0000 7f8e31e00700 1 --2- 172.21.15.151:0/3328610758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e3407dd80 0x7f8e3407c1e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8e240098a0 tx=0x7f8e24009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d839329945709b27 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:40:32.075 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:31.290+0000 7f8e30a00700 1 -- 172.21.15.151:0/3328610758 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e24004160 con 0x7f8e3407dd80 2024-04-26T01:40:32.075 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:31.290+0000 7f8e30a00700 1 -- 172.21.15.151:0/3328610758 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8e24004300 con 0x7f8e3407dd80 2024-04-26T01:40:32.075 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:31.290+0000 7f8e30a00700 1 -- 172.21.15.151:0/3328610758 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e24011af0 con 0x7f8e3407dd80 2024-04-26T01:40:32.075 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:31.291+0000 7f8e39000700 1 -- 172.21.15.151:0/3328610758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e3407dd80 msgr2=0x7f8e3407c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:32.075 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:31.291+0000 7f8e39000700 1 --2- 172.21.15.151:0/3328610758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e3407dd80 0x7f8e3407c1e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8e240098a0 tx=0x7f8e24009bb0 comp rx=0 tx=0).stop 2024-04-26T01:40:32.075 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:31.291+0000 7f8e39000700 1 -- 172.21.15.151:0/3328610758 shutdown_connections 2024-04-26T01:40:32.075 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:31.291+0000 7f8e39000700 1 --2- 172.21.15.151:0/3328610758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e3407dd80 0x7f8e3407c1e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:32.075 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:31.291+0000 7f8e39000700 1 -- 172.21.15.151:0/3328610758 >> 172.21.15.151:0/3328610758 conn(0x7f8e34103c40 msgr2=0x7f8e341060a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:40:32.075 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:31.291+0000 7f8e39000700 1 -- 172.21.15.151:0/3328610758 shutdown_connections 2024-04-26T01:40:32.075 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:31.291+0000 7f8e39000700 1 -- 172.21.15.151:0/3328610758 wait complete. 2024-04-26T01:40:32.075 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:31.292+0000 7f8e39000700 1 Processor -- start 2024-04-26T01:40:32.075 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:31.293+0000 7f8e39000700 1 -- start start 2024-04-26T01:40:32.075 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:31.293+0000 7f8e39000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e3407dd80 0x7f8e3419e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:32.075 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:31.293+0000 7f8e39000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3410ebc0 con 0x7f8e3407dd80 2024-04-26T01:40:32.076 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:31.293+0000 7f8e31e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e3407dd80 0x7f8e3419e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:32.076 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:31.293+0000 7f8e31e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e3407dd80 0x7f8e3419e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36204/0 (socket says 172.21.15.151:36204) 2024-04-26T01:40:32.076 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:31.293+0000 7f8e31e00700 1 -- 172.21.15.151:0/2085650128 learned_addr learned my addr 172.21.15.151:0/2085650128 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:40:32.076 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:31.294+0000 7f8e31e00700 1 -- 172.21.15.151:0/2085650128 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e3419ec10 con 0x7f8e3407dd80 2024-04-26T01:40:32.076 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:31.294+0000 7f8e31e00700 1 --2- 172.21.15.151:0/2085650128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e3407dd80 0x7f8e3419e6d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8e24011ac0 tx=0x7f8e24011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:40:32.076 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:31.294+0000 7f8e2aa00700 1 -- 172.21.15.151:0/2085650128 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e24003b20 con 0x7f8e3407dd80 2024-04-26T01:40:32.076 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:31.294+0000 7f8e2aa00700 1 -- 172.21.15.151:0/2085650128 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8e2401a8c0 con 0x7f8e3407dd80 2024-04-26T01:40:32.076 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:31.294+0000 7f8e39000700 1 -- 172.21.15.151:0/2085650128 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e3419eea0 con 0x7f8e3407dd80 2024-04-26T01:40:32.076 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:31.294+0000 7f8e2aa00700 1 -- 172.21.15.151:0/2085650128 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e240248d0 con 0x7f8e3407dd80 2024-04-26T01:40:32.076 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:31.294+0000 7f8e39000700 1 -- 172.21.15.151:0/2085650128 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e341a1b90 con 0x7f8e3407dd80 2024-04-26T01:40:32.076 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:31.295+0000 7f8e2aa00700 1 -- 172.21.15.151:0/2085650128 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8e24022020 con 0x7f8e3407dd80 2024-04-26T01:40:32.076 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:31.295+0000 7f8e2aa00700 1 -- 172.21.15.151:0/2085650128 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8e240193c0 con 0x7f8e3407dd80 2024-04-26T01:40:32.077 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:31.296+0000 7f8e39000700 1 -- 172.21.15.151:0/2085650128 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e340640a0 con 0x7f8e3407dd80 2024-04-26T01:40:32.077 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:31.298+0000 7f8e2aa00700 1 -- 172.21.15.151:0/2085650128 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f8e2402ddf0 con 0x7f8e3407dd80 2024-04-26T01:40:32.077 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:31.378+0000 7f8e39000700 1 -- 172.21.15.151:0/2085650128 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f8e341a20e0 con 0x7f8e3407dd80 2024-04-26T01:40:32.077 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:31.379+0000 7f8e2aa00700 1 -- 172.21.15.151:0/2085650128 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f8e2401aa70 con 0x7f8e3407dd80 2024-04-26T01:40:32.077 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:31.380+0000 7f8e39000700 1 -- 172.21.15.151:0/2085650128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e3407dd80 msgr2=0x7f8e3419e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:32.077 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:31.380+0000 7f8e39000700 1 --2- 172.21.15.151:0/2085650128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e3407dd80 0x7f8e3419e6d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8e24011ac0 tx=0x7f8e24011ba0 comp rx=0 tx=0).stop 2024-04-26T01:40:32.077 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:31.380+0000 7f8e39000700 1 -- 172.21.15.151:0/2085650128 shutdown_connections 2024-04-26T01:40:32.077 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:31.380+0000 7f8e39000700 1 --2- 172.21.15.151:0/2085650128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e3407dd80 0x7f8e3419e6d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:32.077 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:31.380+0000 7f8e39000700 1 -- 172.21.15.151:0/2085650128 >> 172.21.15.151:0/2085650128 conn(0x7f8e34103c40 msgr2=0x7f8e341047a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:40:32.077 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:31.380+0000 7f8e39000700 1 -- 172.21.15.151:0/2085650128 shutdown_connections 2024-04-26T01:40:32.077 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:31.380+0000 7f8e39000700 1 -- 172.21.15.151:0/2085650128 wait complete. 2024-04-26T01:40:32.077 INFO:teuthology.orchestra.run.smithi151.stdout:mgr not available, waiting (2/15)... 2024-04-26T01:40:32.547 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:32 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:40:32.167+0000 7f05a8bfd200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T01:40:32.547 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:32 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:40:32.290+0000 7f05a8bfd200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T01:40:32.547 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:32 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:40:32.550+0000 7f05a8bfd200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T01:40:33.620 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:33 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:40:33.195+0000 7f05a8bfd200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T01:40:33.620 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:33 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:40:33.317+0000 7f05a8bfd200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T01:40:33.934 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:33 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:40:33.634+0000 7f05a8bfd200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T01:40:33.935 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:33 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:40:33.811+0000 7f05a8bfd200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T01:40:33.935 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:33 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:40:33.936+0000 7f05a8bfd200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T01:40:34.219 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:34 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:40:34.089+0000 7f05a8bfd200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T01:40:34.762 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:34 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:40:34.473+0000 7f05a8bfd200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T01:40:35.086 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:34 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/3691003555' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T01:40:35.370 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:35 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:40:35.263+0000 7f05a8bfd200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T01:40:35.427 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout 2024-04-26T01:40:35.428 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout { 2024-04-26T01:40:35.428 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "fsid": "c0e49e64-036d-11ef-bc93-c7b262605968", 2024-04-26T01:40:35.429 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T01:40:35.429 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T01:40:35.429 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T01:40:35.429 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T01:40:35.429 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:40:35.429 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T01:40:35.429 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T01:40:35.429 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout 0 2024-04-26T01:40:35.429 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout ], 2024-04-26T01:40:35.431 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T01:40:35.431 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "a" 2024-04-26T01:40:35.431 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout ], 2024-04-26T01:40:35.431 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-04-26T01:40:35.431 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T01:40:35.431 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T01:40:35.431 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-26T01:40:35.431 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T01:40:35.431 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:40:35.432 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T01:40:35.432 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T01:40:35.432 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T01:40:35.432 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T01:40:35.432 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T01:40:35.435 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T01:40:35.435 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T01:40:35.435 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T01:40:35.435 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:40:35.435 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T01:40:35.435 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T01:40:35.435 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T01:40:35.435 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T01:40:35.435 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T01:40:35.435 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T01:40:35.435 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T01:40:35.435 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T01:40:35.436 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T01:40:35.436 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:40:35.436 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T01:40:35.436 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T01:40:35.436 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T01:40:35.436 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T01:40:35.436 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:40:35.436 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T01:40:35.436 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-26T01:40:35.436 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T01:40:35.436 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T01:40:35.436 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T01:40:35.436 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T01:40:35.436 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T01:40:35.436 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout ], 2024-04-26T01:40:35.437 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T01:40:35.437 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:40:35.437 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T01:40:35.441 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T01:40:35.442 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T01:40:07.483797+0000", 2024-04-26T01:40:35.442 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T01:40:35.442 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:40:35.442 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T01:40:35.442 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout } 2024-04-26T01:40:35.442 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:34.674+0000 7ff035000700 1 Processor -- start 2024-04-26T01:40:35.442 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:34.674+0000 7ff035000700 1 -- start start 2024-04-26T01:40:35.442 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:34.675+0000 7ff035000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff03010a8c0 0x7ff03010ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:35.442 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:34.675+0000 7ff035000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff03010b2b0 con 0x7ff03010a8c0 2024-04-26T01:40:35.442 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:34.675+0000 7ff02de00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff03010a8c0 0x7ff03010ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:35.442 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:34.675+0000 7ff02de00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff03010a8c0 0x7ff03010ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36208/0 (socket says 172.21.15.151:36208) 2024-04-26T01:40:35.442 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:34.675+0000 7ff02de00700 1 -- 172.21.15.151:0/1831428712 learned_addr learned my addr 172.21.15.151:0/1831428712 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:40:35.443 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:34.676+0000 7ff02de00700 1 -- 172.21.15.151:0/1831428712 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff03010bac0 con 0x7ff03010a8c0 2024-04-26T01:40:35.443 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:34.676+0000 7ff02de00700 1 --2- 172.21.15.151:0/1831428712 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff03010a8c0 0x7ff03010ace0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff0200098a0 tx=0x7ff020009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83b40c483456b773 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:40:35.443 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:34.676+0000 7ff02ca00700 1 -- 172.21.15.151:0/1831428712 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff020004160 con 0x7ff03010a8c0 2024-04-26T01:40:35.443 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:34.676+0000 7ff02ca00700 1 -- 172.21.15.151:0/1831428712 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff020004300 con 0x7ff03010a8c0 2024-04-26T01:40:35.443 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:34.676+0000 7ff02ca00700 1 -- 172.21.15.151:0/1831428712 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff020011af0 con 0x7ff03010a8c0 2024-04-26T01:40:35.443 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:34.677+0000 7ff035000700 1 -- 172.21.15.151:0/1831428712 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff03010a8c0 msgr2=0x7ff03010ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:35.443 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:34.677+0000 7ff035000700 1 --2- 172.21.15.151:0/1831428712 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff03010a8c0 0x7ff03010ace0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff0200098a0 tx=0x7ff020009bb0 comp rx=0 tx=0).stop 2024-04-26T01:40:35.443 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:34.677+0000 7ff035000700 1 -- 172.21.15.151:0/1831428712 shutdown_connections 2024-04-26T01:40:35.443 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:34.677+0000 7ff035000700 1 --2- 172.21.15.151:0/1831428712 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff03010a8c0 0x7ff03010ace0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:35.443 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:34.677+0000 7ff035000700 1 -- 172.21.15.151:0/1831428712 >> 172.21.15.151:0/1831428712 conn(0x7ff03007dc70 msgr2=0x7ff0301083d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:40:35.443 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:34.677+0000 7ff035000700 1 -- 172.21.15.151:0/1831428712 shutdown_connections 2024-04-26T01:40:35.443 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:34.677+0000 7ff035000700 1 -- 172.21.15.151:0/1831428712 wait complete. 2024-04-26T01:40:35.443 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:34.678+0000 7ff035000700 1 Processor -- start 2024-04-26T01:40:35.443 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:34.679+0000 7ff035000700 1 -- start start 2024-04-26T01:40:35.444 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:34.679+0000 7ff035000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff03010a8c0 0x7ff03019e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:35.444 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:34.679+0000 7ff035000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff03010ecf0 con 0x7ff03010a8c0 2024-04-26T01:40:35.444 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:34.679+0000 7ff02de00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff03010a8c0 0x7ff03019e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:35.444 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:34.679+0000 7ff02de00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff03010a8c0 0x7ff03019e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36222/0 (socket says 172.21.15.151:36222) 2024-04-26T01:40:35.444 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:34.679+0000 7ff02de00700 1 -- 172.21.15.151:0/3691003555 learned_addr learned my addr 172.21.15.151:0/3691003555 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:40:35.444 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:34.680+0000 7ff02de00700 1 -- 172.21.15.151:0/3691003555 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff03019ec10 con 0x7ff03010a8c0 2024-04-26T01:40:35.444 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:34.680+0000 7ff02de00700 1 --2- 172.21.15.151:0/3691003555 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff03010a8c0 0x7ff03019e6d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff020000c00 tx=0x7ff020011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:40:35.444 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:34.680+0000 7ff026a00700 1 -- 172.21.15.151:0/3691003555 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff020003b20 con 0x7ff03010a8c0 2024-04-26T01:40:35.444 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:34.680+0000 7ff026a00700 1 -- 172.21.15.151:0/3691003555 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff02001a8c0 con 0x7ff03010a8c0 2024-04-26T01:40:35.444 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:34.680+0000 7ff026a00700 1 -- 172.21.15.151:0/3691003555 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff020024940 con 0x7ff03010a8c0 2024-04-26T01:40:35.444 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:34.680+0000 7ff035000700 1 -- 172.21.15.151:0/3691003555 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff03019eea0 con 0x7ff03010a8c0 2024-04-26T01:40:35.444 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:34.680+0000 7ff035000700 1 -- 172.21.15.151:0/3691003555 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0301a1b90 con 0x7ff03010a8c0 2024-04-26T01:40:35.444 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:34.681+0000 7ff026a00700 1 -- 172.21.15.151:0/3691003555 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff020022020 con 0x7ff03010a8c0 2024-04-26T01:40:35.444 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:34.681+0000 7ff026a00700 1 -- 172.21.15.151:0/3691003555 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff02002ddd0 con 0x7ff03010a8c0 2024-04-26T01:40:35.445 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:34.681+0000 7ff035000700 1 -- 172.21.15.151:0/3691003555 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0300640a0 con 0x7ff03010a8c0 2024-04-26T01:40:35.445 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:34.686+0000 7ff026a00700 1 -- 172.21.15.151:0/3691003555 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7ff020042e50 con 0x7ff03010a8c0 2024-04-26T01:40:35.445 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:34.764+0000 7ff035000700 1 -- 172.21.15.151:0/3691003555 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7ff0301a20d0 con 0x7ff03010a8c0 2024-04-26T01:40:35.445 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:34.765+0000 7ff026a00700 1 -- 172.21.15.151:0/3691003555 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7ff020031020 con 0x7ff03010a8c0 2024-04-26T01:40:35.445 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:34.767+0000 7ff035000700 1 -- 172.21.15.151:0/3691003555 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff03010a8c0 msgr2=0x7ff03019e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:35.445 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:34.767+0000 7ff035000700 1 --2- 172.21.15.151:0/3691003555 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff03010a8c0 0x7ff03019e6d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff020000c00 tx=0x7ff020011b70 comp rx=0 tx=0).stop 2024-04-26T01:40:35.445 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:34.767+0000 7ff035000700 1 -- 172.21.15.151:0/3691003555 shutdown_connections 2024-04-26T01:40:35.445 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:34.767+0000 7ff035000700 1 --2- 172.21.15.151:0/3691003555 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff03010a8c0 0x7ff03019e6d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:35.446 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:34.767+0000 7ff035000700 1 -- 172.21.15.151:0/3691003555 >> 172.21.15.151:0/3691003555 conn(0x7ff03007dc70 msgr2=0x7ff030107b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:40:35.446 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:34.767+0000 7ff035000700 1 -- 172.21.15.151:0/3691003555 shutdown_connections 2024-04-26T01:40:35.446 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:34.767+0000 7ff035000700 1 -- 172.21.15.151:0/3691003555 wait complete. 2024-04-26T01:40:35.446 INFO:teuthology.orchestra.run.smithi151.stdout:mgr not available, waiting (3/15)... 2024-04-26T01:40:35.816 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:35 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:40:35.546+0000 7f05a8bfd200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T01:40:36.120 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:35 smithi151 ceph-mon[28057]: Activating manager daemon x 2024-04-26T01:40:36.120 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:35 smithi151 ceph-mon[28057]: mgrmap e2: x(active, starting, since 0.0108256s) 2024-04-26T01:40:36.120 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:35 smithi151 ceph-mon[28057]: from='mgr.14100 172.21.15.151:0/4130992748' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T01:40:36.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:35 smithi151 ceph-mon[28057]: from='mgr.14100 172.21.15.151:0/4130992748' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T01:40:36.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:35 smithi151 ceph-mon[28057]: from='mgr.14100 172.21.15.151:0/4130992748' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T01:40:36.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:35 smithi151 ceph-mon[28057]: from='mgr.14100 172.21.15.151:0/4130992748' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T01:40:36.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:35 smithi151 ceph-mon[28057]: from='mgr.14100 172.21.15.151:0/4130992748' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T01:40:36.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:35 smithi151 ceph-mon[28057]: from='mgr.14100 172.21.15.151:0/4130992748' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T01:40:36.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:35 smithi151 ceph-mon[28057]: from='mgr.14100 172.21.15.151:0/4130992748' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T01:40:36.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:35 smithi151 ceph-mon[28057]: from='mgr.14100 172.21.15.151:0/4130992748' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T01:40:36.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:35 smithi151 ceph-mon[28057]: Manager daemon x is now available 2024-04-26T01:40:36.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:35 smithi151 ceph-mon[28057]: from='mgr.14100 172.21.15.151:0/4130992748' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T01:40:36.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:35 smithi151 ceph-mon[28057]: from='mgr.14100 172.21.15.151:0/4130992748' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T01:40:36.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:35 smithi151 ceph-mon[28057]: from='mgr.14100 172.21.15.151:0/4130992748' entity='mgr.x' 2024-04-26T01:40:36.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:35 smithi151 ceph-mon[28057]: from='mgr.14100 172.21.15.151:0/4130992748' entity='mgr.x' 2024-04-26T01:40:36.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:35 smithi151 ceph-mon[28057]: from='mgr.14100 172.21.15.151:0/4130992748' entity='mgr.x' 2024-04-26T01:40:37.870 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:37 smithi151 ceph-mon[28057]: mgrmap e3: x(active, since 1.01909s) 2024-04-26T01:40:38.713 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:38 smithi151 ceph-mon[28057]: mgrmap e4: x(active, since 2s) 2024-04-26T01:40:38.714 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:38 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/1923876973' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T01:40:39.056 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout 2024-04-26T01:40:39.059 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout { 2024-04-26T01:40:39.059 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "fsid": "c0e49e64-036d-11ef-bc93-c7b262605968", 2024-04-26T01:40:39.059 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T01:40:39.059 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T01:40:39.060 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T01:40:39.060 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T01:40:39.060 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:40:39.060 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T01:40:39.060 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T01:40:39.060 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout 0 2024-04-26T01:40:39.060 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout ], 2024-04-26T01:40:39.060 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T01:40:39.060 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "a" 2024-04-26T01:40:39.060 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout ], 2024-04-26T01:40:39.060 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-04-26T01:40:39.061 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T01:40:39.061 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T01:40:39.061 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-26T01:40:39.061 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T01:40:39.061 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:40:39.061 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T01:40:39.061 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T01:40:39.061 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T01:40:39.061 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T01:40:39.061 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T01:40:39.070 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T01:40:39.070 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T01:40:39.070 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T01:40:39.070 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:40:39.071 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T01:40:39.071 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T01:40:39.071 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T01:40:39.071 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T01:40:39.071 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T01:40:39.071 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T01:40:39.071 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T01:40:39.071 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T01:40:39.071 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T01:40:39.071 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:40:39.071 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T01:40:39.072 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T01:40:39.072 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T01:40:39.072 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T01:40:39.072 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:40:39.072 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T01:40:39.072 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-26T01:40:39.072 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T01:40:39.072 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T01:40:39.072 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T01:40:39.072 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T01:40:39.072 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T01:40:39.072 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout ], 2024-04-26T01:40:39.072 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T01:40:39.072 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:40:39.073 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T01:40:39.073 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T01:40:39.073 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T01:40:07.483797+0000", 2024-04-26T01:40:39.073 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T01:40:39.073 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:40:39.073 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T01:40:39.073 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout } 2024-04-26T01:40:39.073 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:38.050+0000 7f96ede00700 1 Processor -- start 2024-04-26T01:40:39.073 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:38.050+0000 7f96ede00700 1 -- start start 2024-04-26T01:40:39.073 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:38.051+0000 7f96ede00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f96e8106f50 0x7f96e8109380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:39.073 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:38.051+0000 7f96ede00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f96e807cec0 con 0x7f96e8106f50 2024-04-26T01:40:39.073 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:38.051+0000 7f96e6a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f96e8106f50 0x7f96e8109380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:39.073 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:38.051+0000 7f96e6a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f96e8106f50 0x7f96e8109380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36294/0 (socket says 172.21.15.151:36294) 2024-04-26T01:40:39.073 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:38.051+0000 7f96e6a00700 1 -- 172.21.15.151:0/3033093562 learned_addr learned my addr 172.21.15.151:0/3033093562 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:40:39.074 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:38.052+0000 7f96e6a00700 1 -- 172.21.15.151:0/3033093562 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96e81098c0 con 0x7f96e8106f50 2024-04-26T01:40:39.074 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:38.052+0000 7f96e6a00700 1 --2- 172.21.15.151:0/3033093562 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f96e8106f50 0x7f96e8109380 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f96d80098a0 tx=0x7f96d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7e03c60a5ba84f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:40:39.074 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:38.052+0000 7f96e5600700 1 -- 172.21.15.151:0/3033093562 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96d8004160 con 0x7f96e8106f50 2024-04-26T01:40:39.074 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:38.052+0000 7f96e5600700 1 -- 172.21.15.151:0/3033093562 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f96d8004300 con 0x7f96e8106f50 2024-04-26T01:40:39.074 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:38.052+0000 7f96e5600700 1 -- 172.21.15.151:0/3033093562 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96d8011af0 con 0x7f96e8106f50 2024-04-26T01:40:39.074 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:38.053+0000 7f96ede00700 1 -- 172.21.15.151:0/3033093562 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f96e8106f50 msgr2=0x7f96e8109380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:39.074 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:38.053+0000 7f96ede00700 1 --2- 172.21.15.151:0/3033093562 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f96e8106f50 0x7f96e8109380 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f96d80098a0 tx=0x7f96d8009bb0 comp rx=0 tx=0).stop 2024-04-26T01:40:39.074 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:38.053+0000 7f96ede00700 1 -- 172.21.15.151:0/3033093562 shutdown_connections 2024-04-26T01:40:39.074 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:38.053+0000 7f96ede00700 1 --2- 172.21.15.151:0/3033093562 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f96e8106f50 0x7f96e8109380 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:39.074 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:38.053+0000 7f96ede00700 1 -- 172.21.15.151:0/3033093562 >> 172.21.15.151:0/3033093562 conn(0x7f96e8102c30 msgr2=0x7f96e8105050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:40:39.074 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:38.053+0000 7f96ede00700 1 -- 172.21.15.151:0/3033093562 shutdown_connections 2024-04-26T01:40:39.075 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:38.054+0000 7f96ede00700 1 -- 172.21.15.151:0/3033093562 wait complete. 2024-04-26T01:40:39.075 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:38.055+0000 7f96ede00700 1 Processor -- start 2024-04-26T01:40:39.075 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:38.055+0000 7f96ede00700 1 -- start start 2024-04-26T01:40:39.075 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:38.055+0000 7f96ede00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f96e8106f50 0x7f96e81a2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:39.075 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:38.055+0000 7f96ede00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f96e810a810 con 0x7f96e8106f50 2024-04-26T01:40:39.075 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:38.056+0000 7f96e6a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f96e8106f50 0x7f96e81a2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:39.075 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:38.056+0000 7f96e6a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f96e8106f50 0x7f96e81a2ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36298/0 (socket says 172.21.15.151:36298) 2024-04-26T01:40:39.075 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:38.056+0000 7f96e6a00700 1 -- 172.21.15.151:0/1923876973 learned_addr learned my addr 172.21.15.151:0/1923876973 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:40:39.075 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:38.056+0000 7f96e6a00700 1 -- 172.21.15.151:0/1923876973 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96e81a3020 con 0x7f96e8106f50 2024-04-26T01:40:39.075 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:38.056+0000 7f96e6a00700 1 --2- 172.21.15.151:0/1923876973 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f96e8106f50 0x7f96e81a2ae0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f96d8009fb0 tx=0x7f96d8011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:40:39.075 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:38.057+0000 7f96eca00700 1 -- 172.21.15.151:0/1923876973 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96d80036a0 con 0x7f96e8106f50 2024-04-26T01:40:39.075 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:38.057+0000 7f96eca00700 1 -- 172.21.15.151:0/1923876973 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f96d801a430 con 0x7f96e8106f50 2024-04-26T01:40:39.075 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:38.057+0000 7f96ede00700 1 -- 172.21.15.151:0/1923876973 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96e81a32b0 con 0x7f96e8106f50 2024-04-26T01:40:39.076 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:38.057+0000 7f96eca00700 1 -- 172.21.15.151:0/1923876973 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96d8019550 con 0x7f96e8106f50 2024-04-26T01:40:39.076 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:38.057+0000 7f96ede00700 1 -- 172.21.15.151:0/1923876973 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96e81a5fa0 con 0x7f96e8106f50 2024-04-26T01:40:39.076 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:38.058+0000 7f96eca00700 1 -- 172.21.15.151:0/1923876973 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f96d8019850 con 0x7f96e8106f50 2024-04-26T01:40:39.076 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:38.058+0000 7f96ede00700 1 -- 172.21.15.151:0/1923876973 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96e80640a0 con 0x7f96e8106f50 2024-04-26T01:40:39.076 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:38.059+0000 7f96eca00700 1 --2- 172.21.15.151:0/1923876973 >> [v2:172.21.15.151:6800/244725060,v1:172.21.15.151:6801/244725060] conn(0x7f96cc039510 0x7f96cc03b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:39.076 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:38.059+0000 7f96eca00700 1 -- 172.21.15.151:0/1923876973 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f96d804da20 con 0x7f96e8106f50 2024-04-26T01:40:39.076 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:38.059+0000 7f96e6000700 1 --2- 172.21.15.151:0/1923876973 >> [v2:172.21.15.151:6800/244725060,v1:172.21.15.151:6801/244725060] conn(0x7f96cc039510 0x7f96cc03b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:39.076 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:38.060+0000 7f96e6000700 1 --2- 172.21.15.151:0/1923876973 >> [v2:172.21.15.151:6800/244725060,v1:172.21.15.151:6801/244725060] conn(0x7f96cc039510 0x7f96cc03b9d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f96d4006fd0 tx=0x7f96d4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:40:39.076 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:38.065+0000 7f96eca00700 1 -- 172.21.15.151:0/1923876973 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f96d804d440 con 0x7f96e8106f50 2024-04-26T01:40:39.076 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:38.372+0000 7f96ede00700 1 -- 172.21.15.151:0/1923876973 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f96e80008d0 con 0x7f96e8106f50 2024-04-26T01:40:39.076 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:38.373+0000 7f96eca00700 1 -- 172.21.15.151:0/1923876973 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f96d801a5e0 con 0x7f96e8106f50 2024-04-26T01:40:39.076 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:38.377+0000 7f96ede00700 1 -- 172.21.15.151:0/1923876973 >> [v2:172.21.15.151:6800/244725060,v1:172.21.15.151:6801/244725060] conn(0x7f96cc039510 msgr2=0x7f96cc03b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:39.076 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:38.377+0000 7f96ede00700 1 --2- 172.21.15.151:0/1923876973 >> [v2:172.21.15.151:6800/244725060,v1:172.21.15.151:6801/244725060] conn(0x7f96cc039510 0x7f96cc03b9d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f96d4006fd0 tx=0x7f96d4006e40 comp rx=0 tx=0).stop 2024-04-26T01:40:39.076 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:38.377+0000 7f96ede00700 1 -- 172.21.15.151:0/1923876973 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f96e8106f50 msgr2=0x7f96e81a2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:39.077 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:38.377+0000 7f96ede00700 1 --2- 172.21.15.151:0/1923876973 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f96e8106f50 0x7f96e81a2ae0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f96d8009fb0 tx=0x7f96d8011940 comp rx=0 tx=0).stop 2024-04-26T01:40:39.077 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:38.377+0000 7f96ede00700 1 -- 172.21.15.151:0/1923876973 shutdown_connections 2024-04-26T01:40:39.077 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:38.377+0000 7f96ede00700 1 --2- 172.21.15.151:0/1923876973 >> [v2:172.21.15.151:6800/244725060,v1:172.21.15.151:6801/244725060] conn(0x7f96cc039510 0x7f96cc03b9d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:39.077 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:38.377+0000 7f96ede00700 1 --2- 172.21.15.151:0/1923876973 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f96e8106f50 0x7f96e81a2ae0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:39.077 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:38.377+0000 7f96ede00700 1 -- 172.21.15.151:0/1923876973 >> 172.21.15.151:0/1923876973 conn(0x7f96e8102c30 msgr2=0x7f96e810cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:40:39.077 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:38.377+0000 7f96ede00700 1 -- 172.21.15.151:0/1923876973 shutdown_connections 2024-04-26T01:40:39.077 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:38.377+0000 7f96ede00700 1 -- 172.21.15.151:0/1923876973 wait complete. 2024-04-26T01:40:39.077 INFO:teuthology.orchestra.run.smithi151.stdout:mgr is available 2024-04-26T01:40:40.265 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:39 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/138828816' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-26T01:40:40.607 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout 2024-04-26T01:40:40.608 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout [global] 2024-04-26T01:40:40.608 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout fsid = c0e49e64-036d-11ef-bc93-c7b262605968 2024-04-26T01:40:40.608 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-26T01:40:40.608 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-26T01:40:40.608 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-26T01:40:40.608 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-26T01:40:40.609 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-26T01:40:40.609 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout 2024-04-26T01:40:40.611 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-26T01:40:40.611 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-26T01:40:40.611 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout 2024-04-26T01:40:40.611 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout [osd] 2024-04-26T01:40:40.611 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-26T01:40:40.611 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-26T01:40:40.611 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-26T01:40:40.611 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-26T01:40:40.611 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-26T01:40:40.612 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-26T01:40:40.612 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:39.633+0000 7f2300c00700 1 Processor -- start 2024-04-26T01:40:40.612 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:39.633+0000 7f2300c00700 1 -- start start 2024-04-26T01:40:40.612 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:39.634+0000 7f2300c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f22fc082720 0x7f22fc082b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:40.612 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:39.634+0000 7f2300c00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f22fc083110 con 0x7f22fc082720 2024-04-26T01:40:40.612 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:39.634+0000 7f22f9e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f22fc082720 0x7f22fc082b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:40.612 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:39.634+0000 7f22f9e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f22fc082720 0x7f22fc082b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36302/0 (socket says 172.21.15.151:36302) 2024-04-26T01:40:40.612 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:39.634+0000 7f22f9e00700 1 -- 172.21.15.151:0/962478720 learned_addr learned my addr 172.21.15.151:0/962478720 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:40:40.612 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:39.634+0000 7f22f9e00700 1 -- 172.21.15.151:0/962478720 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22fc10fef0 con 0x7f22fc082720 2024-04-26T01:40:40.612 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:39.635+0000 7f22f9e00700 1 --2- 172.21.15.151:0/962478720 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f22fc082720 0x7f22fc082b40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f22ec0098a0 tx=0x7f22ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d875e82fe7e91c66 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:40:40.612 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:39.635+0000 7f22f8a00700 1 -- 172.21.15.151:0/962478720 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f22ec004160 con 0x7f22fc082720 2024-04-26T01:40:40.616 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:39.635+0000 7f22f8a00700 1 -- 172.21.15.151:0/962478720 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f22ec004300 con 0x7f22fc082720 2024-04-26T01:40:40.617 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:39.635+0000 7f22f8a00700 1 -- 172.21.15.151:0/962478720 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f22ec011af0 con 0x7f22fc082720 2024-04-26T01:40:40.617 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:39.636+0000 7f2300c00700 1 -- 172.21.15.151:0/962478720 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f22fc082720 msgr2=0x7f22fc082b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:40.617 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:39.636+0000 7f2300c00700 1 --2- 172.21.15.151:0/962478720 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f22fc082720 0x7f22fc082b40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f22ec0098a0 tx=0x7f22ec009bb0 comp rx=0 tx=0).stop 2024-04-26T01:40:40.617 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:39.636+0000 7f2300c00700 1 -- 172.21.15.151:0/962478720 shutdown_connections 2024-04-26T01:40:40.617 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:39.636+0000 7f2300c00700 1 --2- 172.21.15.151:0/962478720 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f22fc082720 0x7f22fc082b40 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:40.617 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:39.636+0000 7f2300c00700 1 -- 172.21.15.151:0/962478720 >> 172.21.15.151:0/962478720 conn(0x7f22fc07dce0 msgr2=0x7f22fc080100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:40:40.617 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:39.636+0000 7f2300c00700 1 -- 172.21.15.151:0/962478720 shutdown_connections 2024-04-26T01:40:40.617 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:39.636+0000 7f2300c00700 1 -- 172.21.15.151:0/962478720 wait complete. 2024-04-26T01:40:40.617 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:39.637+0000 7f2300c00700 1 Processor -- start 2024-04-26T01:40:40.617 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:39.637+0000 7f2300c00700 1 -- start start 2024-04-26T01:40:40.617 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:39.637+0000 7f2300c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f22fc082720 0x7f22fc1a2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:40.617 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:39.637+0000 7f2300c00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f22fc113120 con 0x7f22fc082720 2024-04-26T01:40:40.618 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:39.638+0000 7f22f9e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f22fc082720 0x7f22fc1a2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:40.618 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:39.638+0000 7f22f9e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f22fc082720 0x7f22fc1a2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36304/0 (socket says 172.21.15.151:36304) 2024-04-26T01:40:40.618 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:39.638+0000 7f22f9e00700 1 -- 172.21.15.151:0/138828816 learned_addr learned my addr 172.21.15.151:0/138828816 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:40:40.618 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:39.638+0000 7f22f9e00700 1 -- 172.21.15.151:0/138828816 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22fc1a30a0 con 0x7f22fc082720 2024-04-26T01:40:40.618 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:39.638+0000 7f22f9e00700 1 --2- 172.21.15.151:0/138828816 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f22fc082720 0x7f22fc1a2b60 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f22ec009fb0 tx=0x7f22ec011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:40:40.618 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:39.638+0000 7f22f2a00700 1 -- 172.21.15.151:0/138828816 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f22ec0036a0 con 0x7f22fc082720 2024-04-26T01:40:40.618 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:39.638+0000 7f22f2a00700 1 -- 172.21.15.151:0/138828816 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f22ec01a430 con 0x7f22fc082720 2024-04-26T01:40:40.618 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:39.638+0000 7f2300c00700 1 -- 172.21.15.151:0/138828816 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22fc1a3330 con 0x7f22fc082720 2024-04-26T01:40:40.618 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:39.639+0000 7f22f2a00700 1 -- 172.21.15.151:0/138828816 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f22ec019550 con 0x7f22fc082720 2024-04-26T01:40:40.618 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:39.639+0000 7f2300c00700 1 -- 172.21.15.151:0/138828816 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22fc1a6020 con 0x7f22fc082720 2024-04-26T01:40:40.618 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:39.640+0000 7f22f2a00700 1 -- 172.21.15.151:0/138828816 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f22ec003840 con 0x7f22fc082720 2024-04-26T01:40:40.618 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:39.640+0000 7f2300c00700 1 -- 172.21.15.151:0/138828816 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22fc0640a0 con 0x7f22fc082720 2024-04-26T01:40:40.618 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:39.640+0000 7f22f2a00700 1 --2- 172.21.15.151:0/138828816 >> [v2:172.21.15.151:6800/244725060,v1:172.21.15.151:6801/244725060] conn(0x7f22e0039560 0x7f22e003ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:40.619 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:39.640+0000 7f22f2a00700 1 -- 172.21.15.151:0/138828816 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f22ec04d850 con 0x7f22fc082720 2024-04-26T01:40:40.619 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:39.640+0000 7f22f9400700 1 --2- 172.21.15.151:0/138828816 >> [v2:172.21.15.151:6800/244725060,v1:172.21.15.151:6801/244725060] conn(0x7f22e0039560 0x7f22e003ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:40.619 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:39.641+0000 7f22f9400700 1 --2- 172.21.15.151:0/138828816 >> [v2:172.21.15.151:6800/244725060,v1:172.21.15.151:6801/244725060] conn(0x7f22e0039560 0x7f22e003ba20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f22e8006fd0 tx=0x7f22e8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:40:40.619 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:39.646+0000 7f22f2a00700 1 -- 172.21.15.151:0/138828816 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f22ec04d270 con 0x7f22fc082720 2024-04-26T01:40:40.619 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:39.854+0000 7f2300c00700 1 -- 172.21.15.151:0/138828816 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f22fc1a61f0 con 0x7f22fc082720 2024-04-26T01:40:40.619 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:39.856+0000 7f22f2a00700 1 -- 172.21.15.151:0/138828816 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+436 (secure 0 0 0) 0x7f22fc1a61f0 con 0x7f22fc082720 2024-04-26T01:40:40.619 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:39.858+0000 7f2300c00700 1 -- 172.21.15.151:0/138828816 >> [v2:172.21.15.151:6800/244725060,v1:172.21.15.151:6801/244725060] conn(0x7f22e0039560 msgr2=0x7f22e003ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:40.619 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:39.858+0000 7f2300c00700 1 --2- 172.21.15.151:0/138828816 >> [v2:172.21.15.151:6800/244725060,v1:172.21.15.151:6801/244725060] conn(0x7f22e0039560 0x7f22e003ba20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f22e8006fd0 tx=0x7f22e8006e40 comp rx=0 tx=0).stop 2024-04-26T01:40:40.619 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:39.858+0000 7f2300c00700 1 -- 172.21.15.151:0/138828816 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f22fc082720 msgr2=0x7f22fc1a2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:40.619 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:39.858+0000 7f2300c00700 1 --2- 172.21.15.151:0/138828816 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f22fc082720 0x7f22fc1a2b60 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f22ec009fb0 tx=0x7f22ec011940 comp rx=0 tx=0).stop 2024-04-26T01:40:40.619 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:39.859+0000 7f2300c00700 1 -- 172.21.15.151:0/138828816 shutdown_connections 2024-04-26T01:40:40.619 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:39.859+0000 7f2300c00700 1 --2- 172.21.15.151:0/138828816 >> [v2:172.21.15.151:6800/244725060,v1:172.21.15.151:6801/244725060] conn(0x7f22e0039560 0x7f22e003ba20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:40.619 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:39.859+0000 7f2300c00700 1 --2- 172.21.15.151:0/138828816 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f22fc082720 0x7f22fc1a2b60 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:40.619 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:39.859+0000 7f2300c00700 1 -- 172.21.15.151:0/138828816 >> 172.21.15.151:0/138828816 conn(0x7f22fc07dce0 msgr2=0x7f22fc07e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:40:40.620 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:39.859+0000 7f2300c00700 1 -- 172.21.15.151:0/138828816 shutdown_connections 2024-04-26T01:40:40.620 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:39.859+0000 7f2300c00700 1 -- 172.21.15.151:0/138828816 wait complete. 2024-04-26T01:40:40.620 INFO:teuthology.orchestra.run.smithi151.stdout:Enabling cephadm module... 2024-04-26T01:40:41.870 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:41 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/268558137' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-26T01:40:41.870 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:41 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: ignoring --setuser ceph since I am not root 2024-04-26T01:40:41.870 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:41 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: ignoring --setgroup ceph since I am not root 2024-04-26T01:40:42.250 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:41 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:40:41.878+0000 7fd1aaf58200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T01:40:42.250 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:42 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:40:42.120+0000 7fd1aaf58200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T01:40:42.250 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:42 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:40:42.252+0000 7fd1aaf58200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T01:40:42.361 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:41.233+0000 7f070aa00700 1 Processor -- start 2024-04-26T01:40:42.361 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:41.233+0000 7f070aa00700 1 -- start start 2024-04-26T01:40:42.361 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:41.234+0000 7f070aa00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f070407dde0 0x7f070407c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:42.362 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:41.234+0000 7f070aa00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f070407c810 con 0x7f070407dde0 2024-04-26T01:40:42.362 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:41.234+0000 7f0703e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f070407dde0 0x7f070407c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:42.362 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:41.234+0000 7f0703e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f070407dde0 0x7f070407c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49890/0 (socket says 172.21.15.151:49890) 2024-04-26T01:40:42.362 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:41.234+0000 7f0703e00700 1 -- 172.21.15.151:0/3668650201 learned_addr learned my addr 172.21.15.151:0/3668650201 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:40:42.362 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:41.235+0000 7f0703e00700 1 -- 172.21.15.151:0/3668650201 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f070407c990 con 0x7f070407dde0 2024-04-26T01:40:42.362 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:41.235+0000 7f0703e00700 1 --2- 172.21.15.151:0/3668650201 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f070407dde0 0x7f070407c240 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f06f40098a0 tx=0x7f06f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7ba30a7a2bb7656 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:40:42.362 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:41.235+0000 7f0703400700 1 -- 172.21.15.151:0/3668650201 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06f4004160 con 0x7f070407dde0 2024-04-26T01:40:42.362 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:41.235+0000 7f0703400700 1 -- 172.21.15.151:0/3668650201 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f06f4004300 con 0x7f070407dde0 2024-04-26T01:40:42.362 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:41.235+0000 7f0703400700 1 -- 172.21.15.151:0/3668650201 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06f4011af0 con 0x7f070407dde0 2024-04-26T01:40:42.362 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:41.236+0000 7f070aa00700 1 -- 172.21.15.151:0/3668650201 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f070407dde0 msgr2=0x7f070407c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:42.362 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:41.236+0000 7f070aa00700 1 --2- 172.21.15.151:0/3668650201 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f070407dde0 0x7f070407c240 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f06f40098a0 tx=0x7f06f4009bb0 comp rx=0 tx=0).stop 2024-04-26T01:40:42.363 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:41.236+0000 7f070aa00700 1 -- 172.21.15.151:0/3668650201 shutdown_connections 2024-04-26T01:40:42.363 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:41.236+0000 7f070aa00700 1 --2- 172.21.15.151:0/3668650201 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f070407dde0 0x7f070407c240 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:42.363 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:41.236+0000 7f070aa00700 1 -- 172.21.15.151:0/3668650201 >> 172.21.15.151:0/3668650201 conn(0x7f0704103d40 msgr2=0x7f0704106160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:40:42.363 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:41.236+0000 7f070aa00700 1 -- 172.21.15.151:0/3668650201 shutdown_connections 2024-04-26T01:40:42.363 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:41.236+0000 7f070aa00700 1 -- 172.21.15.151:0/3668650201 wait complete. 2024-04-26T01:40:42.363 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:41.237+0000 7f070aa00700 1 Processor -- start 2024-04-26T01:40:42.363 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:41.238+0000 7f070aa00700 1 -- start start 2024-04-26T01:40:42.363 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:41.238+0000 7f070aa00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f070407dde0 0x7f07041a2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:42.363 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:41.238+0000 7f070aa00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f070410a860 con 0x7f070407dde0 2024-04-26T01:40:42.363 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:41.238+0000 7f0703e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f070407dde0 0x7f07041a2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:42.363 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:41.238+0000 7f0703e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f070407dde0 0x7f07041a2ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49906/0 (socket says 172.21.15.151:49906) 2024-04-26T01:40:42.364 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:41.238+0000 7f0703e00700 1 -- 172.21.15.151:0/268558137 learned_addr learned my addr 172.21.15.151:0/268558137 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:40:42.364 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:41.238+0000 7f0703e00700 1 -- 172.21.15.151:0/268558137 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07041a30e0 con 0x7f070407dde0 2024-04-26T01:40:42.364 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:41.239+0000 7f0703e00700 1 --2- 172.21.15.151:0/268558137 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f070407dde0 0x7f07041a2ba0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f06f4009fb0 tx=0x7f06f4011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:40:42.365 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:41.239+0000 7f0701600700 1 -- 172.21.15.151:0/268558137 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06f40036a0 con 0x7f070407dde0 2024-04-26T01:40:42.365 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:41.239+0000 7f0701600700 1 -- 172.21.15.151:0/268558137 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f06f4003cd0 con 0x7f070407dde0 2024-04-26T01:40:42.365 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:41.239+0000 7f070aa00700 1 -- 172.21.15.151:0/268558137 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07041a3370 con 0x7f070407dde0 2024-04-26T01:40:42.365 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:41.239+0000 7f0701600700 1 -- 172.21.15.151:0/268558137 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06f40193c0 con 0x7f070407dde0 2024-04-26T01:40:42.365 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:41.239+0000 7f070aa00700 1 -- 172.21.15.151:0/268558137 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07041a6060 con 0x7f070407dde0 2024-04-26T01:40:42.365 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:41.241+0000 7f070aa00700 1 -- 172.21.15.151:0/268558137 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07040640a0 con 0x7f070407dde0 2024-04-26T01:40:42.365 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:41.241+0000 7f0701600700 1 -- 172.21.15.151:0/268558137 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f06f4003840 con 0x7f070407dde0 2024-04-26T01:40:42.366 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:41.241+0000 7f0701600700 1 --2- 172.21.15.151:0/268558137 >> [v2:172.21.15.151:6800/244725060,v1:172.21.15.151:6801/244725060] conn(0x7f06e8039560 0x7f06e803ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:42.366 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:41.241+0000 7f0701600700 1 -- 172.21.15.151:0/268558137 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f06f404d750 con 0x7f070407dde0 2024-04-26T01:40:42.366 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:41.242+0000 7f06fbe00700 1 --2- 172.21.15.151:0/268558137 >> [v2:172.21.15.151:6800/244725060,v1:172.21.15.151:6801/244725060] conn(0x7f06e8039560 0x7f06e803ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:42.366 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:41.242+0000 7f06fbe00700 1 --2- 172.21.15.151:0/268558137 >> [v2:172.21.15.151:6800/244725060,v1:172.21.15.151:6801/244725060] conn(0x7f06e8039560 0x7f06e803ba20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f06ec006fd0 tx=0x7f06ec006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:40:42.366 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:41.247+0000 7f0701600700 1 -- 172.21.15.151:0/268558137 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f06f4016df0 con 0x7f070407dde0 2024-04-26T01:40:42.366 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:41.533+0000 7f070aa00700 1 -- 172.21.15.151:0/268558137 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f07040008d0 con 0x7f070407dde0 2024-04-26T01:40:42.367 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:41.587+0000 7f0701600700 1 -- 172.21.15.151:0/268558137 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f06f404d170 con 0x7f070407dde0 2024-04-26T01:40:42.367 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:41.590+0000 7f0701600700 1 -- 172.21.15.151:0/268558137 <== mon.0 v2:172.21.15.151:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f06f401a6c0 con 0x7f070407dde0 2024-04-26T01:40:42.367 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:41.591+0000 7f070aa00700 1 -- 172.21.15.151:0/268558137 >> [v2:172.21.15.151:6800/244725060,v1:172.21.15.151:6801/244725060] conn(0x7f06e8039560 msgr2=0x7f06e803ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:42.368 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:41.591+0000 7f070aa00700 1 --2- 172.21.15.151:0/268558137 >> [v2:172.21.15.151:6800/244725060,v1:172.21.15.151:6801/244725060] conn(0x7f06e8039560 0x7f06e803ba20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f06ec006fd0 tx=0x7f06ec006e40 comp rx=0 tx=0).stop 2024-04-26T01:40:42.368 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:41.591+0000 7f070aa00700 1 -- 172.21.15.151:0/268558137 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f070407dde0 msgr2=0x7f07041a2ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:42.368 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:41.591+0000 7f070aa00700 1 --2- 172.21.15.151:0/268558137 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f070407dde0 0x7f07041a2ba0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f06f4009fb0 tx=0x7f06f4011940 comp rx=0 tx=0).stop 2024-04-26T01:40:42.368 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:41.591+0000 7f070aa00700 1 -- 172.21.15.151:0/268558137 shutdown_connections 2024-04-26T01:40:42.368 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:41.591+0000 7f070aa00700 1 --2- 172.21.15.151:0/268558137 >> [v2:172.21.15.151:6800/244725060,v1:172.21.15.151:6801/244725060] conn(0x7f06e8039560 0x7f06e803ba20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:42.368 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:41.591+0000 7f070aa00700 1 --2- 172.21.15.151:0/268558137 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f070407dde0 0x7f07041a2ba0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:42.368 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:41.591+0000 7f070aa00700 1 -- 172.21.15.151:0/268558137 >> 172.21.15.151:0/268558137 conn(0x7f0704103d40 msgr2=0x7f0704199a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:40:42.368 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:41.592+0000 7f070aa00700 1 -- 172.21.15.151:0/268558137 shutdown_connections 2024-04-26T01:40:42.368 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:41.592+0000 7f070aa00700 1 -- 172.21.15.151:0/268558137 wait complete. 2024-04-26T01:40:42.766 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:42 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/268558137' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-26T01:40:42.766 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:42 smithi151 ceph-mon[28057]: mgrmap e5: x(active, since 6s) 2024-04-26T01:40:42.766 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:42 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:40:42.664+0000 7fd1aaf58200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T01:40:43.117 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:42 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:40:42.777+0000 7fd1aaf58200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T01:40:43.870 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:43 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/3772200883' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-26T01:40:43.980 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout { 2024-04-26T01:40:43.980 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-26T01:40:43.980 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-26T01:40:43.980 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-26T01:40:43.980 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-26T01:40:43.980 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout } 2024-04-26T01:40:43.980 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:42.911+0000 7f2900c00700 1 Processor -- start 2024-04-26T01:40:43.980 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:42.912+0000 7f2900c00700 1 -- start start 2024-04-26T01:40:43.980 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:42.912+0000 7f2900c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28fc1086c0 0x7f28fc108ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:43.981 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:42.912+0000 7f2900c00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f28fc1090b0 con 0x7f28fc1086c0 2024-04-26T01:40:43.981 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:42.912+0000 7f28f9e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28fc1086c0 0x7f28fc108ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:43.986 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:42.913+0000 7f28f9e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28fc1086c0 0x7f28fc108ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49932/0 (socket says 172.21.15.151:49932) 2024-04-26T01:40:43.986 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:42.913+0000 7f28f9e00700 1 -- 172.21.15.151:0/875539241 learned_addr learned my addr 172.21.15.151:0/875539241 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:40:43.986 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:42.913+0000 7f28f9e00700 1 -- 172.21.15.151:0/875539241 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28fc1098c0 con 0x7f28fc1086c0 2024-04-26T01:40:43.986 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:42.913+0000 7f28f9e00700 1 --2- 172.21.15.151:0/875539241 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28fc1086c0 0x7f28fc108ae0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f28ec0098a0 tx=0x7f28ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=660d9b0676486b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:40:43.986 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:42.914+0000 7f28f8a00700 1 -- 172.21.15.151:0/875539241 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f28ec004160 con 0x7f28fc1086c0 2024-04-26T01:40:43.986 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:42.914+0000 7f28f8a00700 1 -- 172.21.15.151:0/875539241 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f28ec004300 con 0x7f28fc1086c0 2024-04-26T01:40:43.986 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:42.914+0000 7f28f8a00700 1 -- 172.21.15.151:0/875539241 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f28ec011af0 con 0x7f28fc1086c0 2024-04-26T01:40:43.986 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:42.914+0000 7f2900c00700 1 -- 172.21.15.151:0/875539241 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28fc1086c0 msgr2=0x7f28fc108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:43.986 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:42.914+0000 7f2900c00700 1 --2- 172.21.15.151:0/875539241 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28fc1086c0 0x7f28fc108ae0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f28ec0098a0 tx=0x7f28ec009bb0 comp rx=0 tx=0).stop 2024-04-26T01:40:43.986 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:42.914+0000 7f2900c00700 1 -- 172.21.15.151:0/875539241 shutdown_connections 2024-04-26T01:40:43.986 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:42.914+0000 7f2900c00700 1 --2- 172.21.15.151:0/875539241 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28fc1086c0 0x7f28fc108ae0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:43.987 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:42.914+0000 7f2900c00700 1 -- 172.21.15.151:0/875539241 >> 172.21.15.151:0/875539241 conn(0x7f28fc103c40 msgr2=0x7f28fc1060a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:40:43.987 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:42.914+0000 7f2900c00700 1 -- 172.21.15.151:0/875539241 shutdown_connections 2024-04-26T01:40:43.987 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:42.914+0000 7f2900c00700 1 -- 172.21.15.151:0/875539241 wait complete. 2024-04-26T01:40:43.987 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:42.915+0000 7f2900c00700 1 Processor -- start 2024-04-26T01:40:43.987 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:42.916+0000 7f2900c00700 1 -- start start 2024-04-26T01:40:43.987 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:42.916+0000 7f2900c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28fc1086c0 0x7f28fc19e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:43.987 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:42.916+0000 7f2900c00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f28fc10caf0 con 0x7f28fc1086c0 2024-04-26T01:40:43.987 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:42.916+0000 7f28f9e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28fc1086c0 0x7f28fc19e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:43.987 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:42.916+0000 7f28f9e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28fc1086c0 0x7f28fc19e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49936/0 (socket says 172.21.15.151:49936) 2024-04-26T01:40:43.987 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:42.916+0000 7f28f9e00700 1 -- 172.21.15.151:0/3772200883 learned_addr learned my addr 172.21.15.151:0/3772200883 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:40:43.987 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:42.917+0000 7f28f9e00700 1 -- 172.21.15.151:0/3772200883 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28fc19ec70 con 0x7f28fc1086c0 2024-04-26T01:40:43.987 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:42.917+0000 7f28f9e00700 1 --2- 172.21.15.151:0/3772200883 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28fc1086c0 0x7f28fc19e730 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f28ec003f40 tx=0x7f28ec011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:40:43.987 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:42.917+0000 7f28f2a00700 1 -- 172.21.15.151:0/3772200883 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f28ec0038b0 con 0x7f28fc1086c0 2024-04-26T01:40:43.988 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:42.917+0000 7f2900c00700 1 -- 172.21.15.151:0/3772200883 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28fc19ef00 con 0x7f28fc1086c0 2024-04-26T01:40:43.988 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:42.917+0000 7f28f2a00700 1 -- 172.21.15.151:0/3772200883 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f28ec003a50 con 0x7f28fc1086c0 2024-04-26T01:40:43.988 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:42.917+0000 7f28f2a00700 1 -- 172.21.15.151:0/3772200883 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f28ec01a5c0 con 0x7f28fc1086c0 2024-04-26T01:40:43.988 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:42.917+0000 7f2900c00700 1 -- 172.21.15.151:0/3772200883 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28fc1a1bf0 con 0x7f28fc1086c0 2024-04-26T01:40:43.988 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:42.919+0000 7f28f2a00700 1 -- 172.21.15.151:0/3772200883 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f28ec024430 con 0x7f28fc1086c0 2024-04-26T01:40:43.988 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:42.919+0000 7f2900c00700 1 -- 172.21.15.151:0/3772200883 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28fc0640a0 con 0x7f28fc1086c0 2024-04-26T01:40:43.988 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:42.919+0000 7f28f2a00700 1 --2- 172.21.15.151:0/3772200883 >> [v2:172.21.15.151:6800/244725060,v1:172.21.15.151:6801/244725060] conn(0x7f28e0039560 0x7f28e003ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:43.988 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:42.919+0000 7f28f2a00700 1 -- 172.21.15.151:0/3772200883 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f28ec04dac0 con 0x7f28fc1086c0 2024-04-26T01:40:43.988 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:42.919+0000 7f28f9400700 1 -- 172.21.15.151:0/3772200883 >> [v2:172.21.15.151:6800/244725060,v1:172.21.15.151:6801/244725060] conn(0x7f28e0039560 msgr2=0x7f28e003ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.151:6800/244725060 2024-04-26T01:40:43.988 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:42.919+0000 7f28f9400700 1 --2- 172.21.15.151:0/3772200883 >> [v2:172.21.15.151:6800/244725060,v1:172.21.15.151:6801/244725060] conn(0x7f28e0039560 0x7f28e003ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T01:40:43.988 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:42.929+0000 7f28f2a00700 1 -- 172.21.15.151:0/3772200883 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f28ec04d4e0 con 0x7f28fc1086c0 2024-04-26T01:40:43.988 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:43.120+0000 7f28f9400700 1 -- 172.21.15.151:0/3772200883 >> [v2:172.21.15.151:6800/244725060,v1:172.21.15.151:6801/244725060] conn(0x7f28e0039560 msgr2=0x7f28e003ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.151:6800/244725060 2024-04-26T01:40:43.988 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:43.120+0000 7f28f9400700 1 --2- 172.21.15.151:0/3772200883 >> [v2:172.21.15.151:6800/244725060,v1:172.21.15.151:6801/244725060] conn(0x7f28e0039560 0x7f28e003ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T01:40:43.988 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:43.213+0000 7f2900c00700 1 -- 172.21.15.151:0/3772200883 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f28fc1a1dc0 con 0x7f28fc1086c0 2024-04-26T01:40:43.989 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:43.214+0000 7f28f2a00700 1 -- 172.21.15.151:0/3772200883 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f28fc1a1dc0 con 0x7f28fc1086c0 2024-04-26T01:40:43.989 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:43.217+0000 7f2900c00700 1 -- 172.21.15.151:0/3772200883 >> [v2:172.21.15.151:6800/244725060,v1:172.21.15.151:6801/244725060] conn(0x7f28e0039560 msgr2=0x7f28e003ba20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T01:40:43.989 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:43.217+0000 7f2900c00700 1 --2- 172.21.15.151:0/3772200883 >> [v2:172.21.15.151:6800/244725060,v1:172.21.15.151:6801/244725060] conn(0x7f28e0039560 0x7f28e003ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:43.989 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:43.217+0000 7f2900c00700 1 -- 172.21.15.151:0/3772200883 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28fc1086c0 msgr2=0x7f28fc19e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:43.989 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:43.217+0000 7f2900c00700 1 --2- 172.21.15.151:0/3772200883 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28fc1086c0 0x7f28fc19e730 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f28ec003f40 tx=0x7f28ec011890 comp rx=0 tx=0).stop 2024-04-26T01:40:43.989 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:43.217+0000 7f2900c00700 1 -- 172.21.15.151:0/3772200883 shutdown_connections 2024-04-26T01:40:43.989 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:43.217+0000 7f2900c00700 1 --2- 172.21.15.151:0/3772200883 >> [v2:172.21.15.151:6800/244725060,v1:172.21.15.151:6801/244725060] conn(0x7f28e0039560 0x7f28e003ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:43.989 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:43.217+0000 7f2900c00700 1 --2- 172.21.15.151:0/3772200883 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28fc1086c0 0x7f28fc19e730 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:43.989 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:43.217+0000 7f2900c00700 1 -- 172.21.15.151:0/3772200883 >> 172.21.15.151:0/3772200883 conn(0x7f28fc103c40 msgr2=0x7f28fc104740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:40:43.989 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:43.217+0000 7f2900c00700 1 -- 172.21.15.151:0/3772200883 shutdown_connections 2024-04-26T01:40:43.990 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:43.217+0000 7f2900c00700 1 -- 172.21.15.151:0/3772200883 wait complete. 2024-04-26T01:40:43.990 INFO:teuthology.orchestra.run.smithi151.stdout:Waiting for the mgr to restart... 2024-04-26T01:40:43.990 INFO:teuthology.orchestra.run.smithi151.stdout:Waiting for mgr epoch 5... 2024-04-26T01:40:44.589 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:44 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:40:44.359+0000 7fd1aaf58200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T01:40:44.589 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:44 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:40:44.591+0000 7fd1aaf58200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T01:40:44.845 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:44 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:40:44.808+0000 7fd1aaf58200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T01:40:45.562 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:45 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:40:45.191+0000 7fd1aaf58200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T01:40:45.563 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:45 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:40:45.313+0000 7fd1aaf58200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T01:40:45.814 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:45 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:40:45.565+0000 7fd1aaf58200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T01:40:45.815 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:45 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:40:45.817+0000 7fd1aaf58200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T01:40:46.870 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:46 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:40:46.590+0000 7fd1aaf58200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T01:40:47.530 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:47 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:40:47.138+0000 7fd1aaf58200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T01:40:47.530 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:47 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:40:47.266+0000 7fd1aaf58200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T01:40:47.530 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:47 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:40:47.532+0000 7fd1aaf58200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T01:40:48.620 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:48 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:40:48.186+0000 7fd1aaf58200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T01:40:48.620 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:48 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:40:48.309+0000 7fd1aaf58200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T01:40:48.927 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:48 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:40:48.626+0000 7fd1aaf58200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T01:40:48.927 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:48 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:40:48.803+0000 7fd1aaf58200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T01:40:48.927 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:48 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:40:48.929+0000 7fd1aaf58200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T01:40:49.370 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:49 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:40:49.081+0000 7fd1aaf58200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T01:40:49.870 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:49 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:40:49.456+0000 7fd1aaf58200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T01:40:50.485 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:50 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:40:50.205+0000 7fd1aaf58200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T01:40:50.870 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:50 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:40:50.488+0000 7fd1aaf58200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T01:40:50.870 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:50 smithi151 ceph-mon[28057]: Active manager daemon x restarted 2024-04-26T01:40:50.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:50 smithi151 ceph-mon[28057]: Activating manager daemon x 2024-04-26T01:40:50.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:50 smithi151 ceph-mon[28057]: osdmap e2: 0 total, 0 up, 0 in 2024-04-26T01:40:50.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:50 smithi151 ceph-mon[28057]: mgrmap e6: x(active, starting, since 0.010398s) 2024-04-26T01:40:50.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:50 smithi151 ceph-mon[28057]: from='mgr.14120 172.21.15.151:0/3675138561' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T01:40:50.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:50 smithi151 ceph-mon[28057]: from='mgr.14120 172.21.15.151:0/3675138561' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T01:40:50.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:50 smithi151 ceph-mon[28057]: from='mgr.14120 172.21.15.151:0/3675138561' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T01:40:50.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:50 smithi151 ceph-mon[28057]: from='mgr.14120 172.21.15.151:0/3675138561' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T01:40:50.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:50 smithi151 ceph-mon[28057]: from='mgr.14120 172.21.15.151:0/3675138561' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T01:40:50.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:50 smithi151 ceph-mon[28057]: Manager daemon x is now available 2024-04-26T01:40:51.870 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:51 smithi151 ceph-mon[28057]: Found migration_current of "None". Setting to last migration. 2024-04-26T01:40:51.870 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:51 smithi151 ceph-mon[28057]: from='mgr.14120 172.21.15.151:0/3675138561' entity='mgr.x' 2024-04-26T01:40:51.870 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:51 smithi151 ceph-mon[28057]: from='mgr.14120 172.21.15.151:0/3675138561' entity='mgr.x' 2024-04-26T01:40:51.870 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:51 smithi151 ceph-mon[28057]: from='mgr.14120 172.21.15.151:0/3675138561' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:40:51.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:51 smithi151 ceph-mon[28057]: from='mgr.14120 172.21.15.151:0/3675138561' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:40:51.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:51 smithi151 ceph-mon[28057]: from='mgr.14120 172.21.15.151:0/3675138561' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T01:40:51.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:51 smithi151 ceph-mon[28057]: from='mgr.14120 172.21.15.151:0/3675138561' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T01:40:51.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:51 smithi151 ceph-mon[28057]: from='mgr.14120 172.21.15.151:0/3675138561' entity='mgr.x' 2024-04-26T01:40:51.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:51 smithi151 ceph-mon[28057]: from='mgr.14120 172.21.15.151:0/3675138561' entity='mgr.x' 2024-04-26T01:40:51.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:51 smithi151 ceph-mon[28057]: mgrmap e7: x(active, since 1.01794s) 2024-04-26T01:40:52.202 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout { 2024-04-26T01:40:52.202 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-26T01:40:52.202 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-26T01:40:52.202 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout } 2024-04-26T01:40:52.202 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:44.638+0000 7f922da00700 1 Processor -- start 2024-04-26T01:40:52.202 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:44.638+0000 7f922da00700 1 -- start start 2024-04-26T01:40:52.202 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:44.639+0000 7f922da00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9228108690 0x7f9228108ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:52.203 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:44.639+0000 7f922da00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9228109080 con 0x7f9228108690 2024-04-26T01:40:52.203 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:44.639+0000 7f9226a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9228108690 0x7f9228108ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:52.203 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:44.639+0000 7f9226a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9228108690 0x7f9228108ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49940/0 (socket says 172.21.15.151:49940) 2024-04-26T01:40:52.203 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:44.639+0000 7f9226a00700 1 -- 172.21.15.151:0/388381652 learned_addr learned my addr 172.21.15.151:0/388381652 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:40:52.203 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:44.640+0000 7f9226a00700 1 -- 172.21.15.151:0/388381652 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9228109890 con 0x7f9228108690 2024-04-26T01:40:52.203 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:44.640+0000 7f9226a00700 1 --2- 172.21.15.151:0/388381652 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9228108690 0x7f9228108ab0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f92180098a0 tx=0x7f9218009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=491ee8f67426cc5b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:40:52.203 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:44.640+0000 7f9225600700 1 -- 172.21.15.151:0/388381652 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9218004160 con 0x7f9228108690 2024-04-26T01:40:52.203 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:44.640+0000 7f9225600700 1 -- 172.21.15.151:0/388381652 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9218004300 con 0x7f9228108690 2024-04-26T01:40:52.203 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:44.640+0000 7f9225600700 1 -- 172.21.15.151:0/388381652 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9218011af0 con 0x7f9228108690 2024-04-26T01:40:52.203 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:44.641+0000 7f922da00700 1 -- 172.21.15.151:0/388381652 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9228108690 msgr2=0x7f9228108ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:52.203 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:44.641+0000 7f922da00700 1 --2- 172.21.15.151:0/388381652 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9228108690 0x7f9228108ab0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f92180098a0 tx=0x7f9218009bb0 comp rx=0 tx=0).stop 2024-04-26T01:40:52.204 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:44.641+0000 7f922da00700 1 -- 172.21.15.151:0/388381652 shutdown_connections 2024-04-26T01:40:52.204 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:44.641+0000 7f922da00700 1 --2- 172.21.15.151:0/388381652 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9228108690 0x7f9228108ab0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:52.204 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:44.641+0000 7f922da00700 1 -- 172.21.15.151:0/388381652 >> 172.21.15.151:0/388381652 conn(0x7f9228103c50 msgr2=0x7f9228106070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:40:52.204 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:44.641+0000 7f922da00700 1 -- 172.21.15.151:0/388381652 shutdown_connections 2024-04-26T01:40:52.204 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:44.642+0000 7f922da00700 1 -- 172.21.15.151:0/388381652 wait complete. 2024-04-26T01:40:52.204 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:44.643+0000 7f922da00700 1 Processor -- start 2024-04-26T01:40:52.204 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:44.643+0000 7f922da00700 1 -- start start 2024-04-26T01:40:52.204 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:44.643+0000 7f922da00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9228108690 0x7f922819e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:52.204 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:44.643+0000 7f922da00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f922810cac0 con 0x7f9228108690 2024-04-26T01:40:52.204 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:44.643+0000 7f9226a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9228108690 0x7f922819e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:52.204 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:44.643+0000 7f9226a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9228108690 0x7f922819e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49956/0 (socket says 172.21.15.151:49956) 2024-04-26T01:40:52.204 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:44.643+0000 7f9226a00700 1 -- 172.21.15.151:0/2875208229 learned_addr learned my addr 172.21.15.151:0/2875208229 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:40:52.205 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:44.644+0000 7f9226a00700 1 -- 172.21.15.151:0/2875208229 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f922819ecd0 con 0x7f9228108690 2024-04-26T01:40:52.205 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:44.644+0000 7f9226a00700 1 --2- 172.21.15.151:0/2875208229 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9228108690 0x7f922819e790 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f92180117b0 tx=0x7f9218011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:40:52.205 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:44.644+0000 7f921f400700 1 -- 172.21.15.151:0/2875208229 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9218003af0 con 0x7f9228108690 2024-04-26T01:40:52.205 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:44.644+0000 7f921f400700 1 -- 172.21.15.151:0/2875208229 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9218020070 con 0x7f9228108690 2024-04-26T01:40:52.205 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:44.644+0000 7f922da00700 1 -- 172.21.15.151:0/2875208229 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f922819ef60 con 0x7f9228108690 2024-04-26T01:40:52.205 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:44.644+0000 7f921f400700 1 -- 172.21.15.151:0/2875208229 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9218019630 con 0x7f9228108690 2024-04-26T01:40:52.205 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:44.644+0000 7f922da00700 1 -- 172.21.15.151:0/2875208229 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92281a1c50 con 0x7f9228108690 2024-04-26T01:40:52.205 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:44.645+0000 7f921f400700 1 -- 172.21.15.151:0/2875208229 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f92180198f0 con 0x7f9228108690 2024-04-26T01:40:52.205 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:44.646+0000 7f921f400700 1 --2- 172.21.15.151:0/2875208229 >> [v2:172.21.15.151:6800/244725060,v1:172.21.15.151:6801/244725060] conn(0x7f920c0395b0 0x7f920c03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:52.205 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:44.646+0000 7f922da00700 1 -- 172.21.15.151:0/2875208229 --> [v2:172.21.15.151:6800/244725060,v1:172.21.15.151:6801/244725060] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f92280640a0 con 0x7f920c0395b0 2024-04-26T01:40:52.205 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:44.646+0000 7f921f400700 1 -- 172.21.15.151:0/2875208229 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f921804dba0 con 0x7f9228108690 2024-04-26T01:40:52.205 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:44.646+0000 7f9226000700 1 -- 172.21.15.151:0/2875208229 >> [v2:172.21.15.151:6800/244725060,v1:172.21.15.151:6801/244725060] conn(0x7f920c0395b0 msgr2=0x7f920c03ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.151:6800/244725060 2024-04-26T01:40:52.206 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:44.646+0000 7f9226000700 1 --2- 172.21.15.151:0/2875208229 >> [v2:172.21.15.151:6800/244725060,v1:172.21.15.151:6801/244725060] conn(0x7f920c0395b0 0x7f920c03ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T01:40:52.206 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:44.847+0000 7f9226000700 1 -- 172.21.15.151:0/2875208229 >> [v2:172.21.15.151:6800/244725060,v1:172.21.15.151:6801/244725060] conn(0x7f920c0395b0 msgr2=0x7f920c03ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.151:6800/244725060 2024-04-26T01:40:52.206 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:44.847+0000 7f9226000700 1 --2- 172.21.15.151:0/2875208229 >> [v2:172.21.15.151:6800/244725060,v1:172.21.15.151:6801/244725060] conn(0x7f920c0395b0 0x7f920c03ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T01:40:52.206 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:45.248+0000 7f9226000700 1 -- 172.21.15.151:0/2875208229 >> [v2:172.21.15.151:6800/244725060,v1:172.21.15.151:6801/244725060] conn(0x7f920c0395b0 msgr2=0x7f920c03ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.151:6800/244725060 2024-04-26T01:40:52.206 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:45.248+0000 7f9226000700 1 --2- 172.21.15.151:0/2875208229 >> [v2:172.21.15.151:6800/244725060,v1:172.21.15.151:6801/244725060] conn(0x7f920c0395b0 0x7f920c03ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-26T01:40:52.206 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:46.049+0000 7f9226000700 1 -- 172.21.15.151:0/2875208229 >> [v2:172.21.15.151:6800/244725060,v1:172.21.15.151:6801/244725060] conn(0x7f920c0395b0 msgr2=0x7f920c03ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.151:6800/244725060 2024-04-26T01:40:52.206 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:46.049+0000 7f9226000700 1 --2- 172.21.15.151:0/2875208229 >> [v2:172.21.15.151:6800/244725060,v1:172.21.15.151:6801/244725060] conn(0x7f920c0395b0 0x7f920c03ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-26T01:40:52.206 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:47.651+0000 7f9226000700 1 -- 172.21.15.151:0/2875208229 >> [v2:172.21.15.151:6800/244725060,v1:172.21.15.151:6801/244725060] conn(0x7f920c0395b0 msgr2=0x7f920c03ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.151:6800/244725060 2024-04-26T01:40:52.206 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:47.651+0000 7f9226000700 1 --2- 172.21.15.151:0/2875208229 >> [v2:172.21.15.151:6800/244725060,v1:172.21.15.151:6801/244725060] conn(0x7f920c0395b0 0x7f920c03ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-26T01:40:52.206 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:50.499+0000 7f921f400700 1 -- 172.21.15.151:0/2875208229 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7f921802bd00 con 0x7f9228108690 2024-04-26T01:40:52.207 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:50.499+0000 7f921f400700 1 -- 172.21.15.151:0/2875208229 >> [v2:172.21.15.151:6800/244725060,v1:172.21.15.151:6801/244725060] conn(0x7f920c0395b0 msgr2=0x7f920c03ba70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T01:40:52.207 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:50.499+0000 7f921f400700 1 --2- 172.21.15.151:0/2875208229 >> [v2:172.21.15.151:6800/244725060,v1:172.21.15.151:6801/244725060] conn(0x7f920c0395b0 0x7f920c03ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:52.207 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:51.503+0000 7f921f400700 1 -- 172.21.15.151:0/2875208229 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f921804edc0 con 0x7f9228108690 2024-04-26T01:40:52.207 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:51.504+0000 7f921f400700 1 --2- 172.21.15.151:0/2875208229 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7f920c03ccb0 0x7f920c03f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:52.207 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:51.504+0000 7f921f400700 1 -- 172.21.15.151:0/2875208229 --> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f921800ce90 con 0x7f920c03ccb0 2024-04-26T01:40:52.207 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:51.506+0000 7f9226000700 1 --2- 172.21.15.151:0/2875208229 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7f920c03ccb0 0x7f920c03f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:52.207 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:51.509+0000 7f9226000700 1 --2- 172.21.15.151:0/2875208229 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7f920c03ccb0 0x7f920c03f0a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9214003a10 tx=0x7f9214009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:40:52.207 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:51.511+0000 7f921f400700 1 -- 172.21.15.151:0/2875208229 <== mgr.14120 v2:172.21.15.151:6800/796314699 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f921800ce90 con 0x7f920c03ccb0 2024-04-26T01:40:52.207 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:51.519+0000 7f922da00700 1 -- 172.21.15.151:0/2875208229 --> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f9228104e90 con 0x7f920c03ccb0 2024-04-26T01:40:52.207 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:51.519+0000 7f921f400700 1 -- 172.21.15.151:0/2875208229 <== mgr.14120 v2:172.21.15.151:6800/796314699 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f9228104e90 con 0x7f920c03ccb0 2024-04-26T01:40:52.207 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:51.520+0000 7f922da00700 1 -- 172.21.15.151:0/2875208229 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7f920c03ccb0 msgr2=0x7f920c03f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:52.208 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:51.520+0000 7f922da00700 1 --2- 172.21.15.151:0/2875208229 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7f920c03ccb0 0x7f920c03f0a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9214003a10 tx=0x7f9214009290 comp rx=0 tx=0).stop 2024-04-26T01:40:52.208 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:51.520+0000 7f922da00700 1 -- 172.21.15.151:0/2875208229 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9228108690 msgr2=0x7f922819e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:52.208 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:51.520+0000 7f922da00700 1 --2- 172.21.15.151:0/2875208229 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9228108690 0x7f922819e790 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f92180117b0 tx=0x7f9218011b70 comp rx=0 tx=0).stop 2024-04-26T01:40:52.208 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:51.520+0000 7f922da00700 1 -- 172.21.15.151:0/2875208229 shutdown_connections 2024-04-26T01:40:52.208 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:51.520+0000 7f922da00700 1 --2- 172.21.15.151:0/2875208229 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7f920c03ccb0 0x7f920c03f0a0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:52.208 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:51.520+0000 7f922da00700 1 --2- 172.21.15.151:0/2875208229 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9228108690 0x7f922819e790 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:52.208 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:51.520+0000 7f922da00700 1 --2- 172.21.15.151:0/2875208229 >> [v2:172.21.15.151:6800/244725060,v1:172.21.15.151:6801/244725060] conn(0x7f920c0395b0 0x7f920c03ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:52.208 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:51.520+0000 7f922da00700 1 -- 172.21.15.151:0/2875208229 >> 172.21.15.151:0/2875208229 conn(0x7f9228103c50 msgr2=0x7f9228104780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:40:52.208 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:51.520+0000 7f922da00700 1 -- 172.21.15.151:0/2875208229 shutdown_connections 2024-04-26T01:40:52.208 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:51.521+0000 7f922da00700 1 -- 172.21.15.151:0/2875208229 wait complete. 2024-04-26T01:40:52.208 INFO:teuthology.orchestra.run.smithi151.stdout:mgr epoch 5 is available 2024-04-26T01:40:52.209 INFO:teuthology.orchestra.run.smithi151.stdout:Setting orchestrator backend to cephadm... 2024-04-26T01:40:52.821 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:52 smithi151 ceph-mon[28057]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-26T01:40:52.821 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:52 smithi151 ceph-mon[28057]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-26T01:40:52.821 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:52 smithi151 ceph-mon[28057]: from='mgr.14120 172.21.15.151:0/3675138561' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:40:53.749 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:53 smithi151 ceph-mon[28057]: [26/Apr/2024:01:40:51] ENGINE Bus STARTING 2024-04-26T01:40:53.749 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:53 smithi151 ceph-mon[28057]: [26/Apr/2024:01:40:52] ENGINE Serving on https://172.21.15.151:7150 2024-04-26T01:40:53.749 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:53 smithi151 ceph-mon[28057]: [26/Apr/2024:01:40:52] ENGINE Serving on http://172.21.15.151:8765 2024-04-26T01:40:53.749 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:53 smithi151 ceph-mon[28057]: [26/Apr/2024:01:40:52] ENGINE Bus STARTED 2024-04-26T01:40:53.749 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:53 smithi151 ceph-mon[28057]: mgrmap e8: x(active, since 2s) 2024-04-26T01:40:53.750 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:53 smithi151 ceph-mon[28057]: from='mgr.14120 172.21.15.151:0/3675138561' entity='mgr.x' 2024-04-26T01:40:53.750 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:53 smithi151 ceph-mon[28057]: from='mgr.14120 172.21.15.151:0/3675138561' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:40:53.818 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:52.824+0000 7f9a70000700 1 Processor -- start 2024-04-26T01:40:53.818 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:52.824+0000 7f9a70000700 1 -- start start 2024-04-26T01:40:53.818 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:52.824+0000 7f9a70000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9a6807eeb0 0x7f9a6807d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:53.819 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:52.824+0000 7f9a70000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9a6807d8e0 con 0x7f9a6807eeb0 2024-04-26T01:40:53.819 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:52.825+0000 7f9a6d600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9a6807eeb0 0x7f9a6807d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:53.819 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:52.825+0000 7f9a6d600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9a6807eeb0 0x7f9a6807d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37238/0 (socket says 172.21.15.151:37238) 2024-04-26T01:40:53.819 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:52.825+0000 7f9a6d600700 1 -- 172.21.15.151:0/2051561285 learned_addr learned my addr 172.21.15.151:0/2051561285 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:40:53.819 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:52.825+0000 7f9a6d600700 1 -- 172.21.15.151:0/2051561285 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a6807da60 con 0x7f9a6807eeb0 2024-04-26T01:40:53.819 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:52.826+0000 7f9a6d600700 1 --2- 172.21.15.151:0/2051561285 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9a6807eeb0 0x7f9a6807d310 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c0098a0 tx=0x7f9a5c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39ea9bcccb8baef7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:40:53.819 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:52.826+0000 7f9a67e00700 1 -- 172.21.15.151:0/2051561285 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a5c004160 con 0x7f9a6807eeb0 2024-04-26T01:40:53.819 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:52.826+0000 7f9a67e00700 1 -- 172.21.15.151:0/2051561285 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9a5c004300 con 0x7f9a6807eeb0 2024-04-26T01:40:53.819 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:52.826+0000 7f9a67e00700 1 -- 172.21.15.151:0/2051561285 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a5c011af0 con 0x7f9a6807eeb0 2024-04-26T01:40:53.819 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:52.827+0000 7f9a70000700 1 -- 172.21.15.151:0/2051561285 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9a6807eeb0 msgr2=0x7f9a6807d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:53.819 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:52.827+0000 7f9a70000700 1 --2- 172.21.15.151:0/2051561285 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9a6807eeb0 0x7f9a6807d310 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c0098a0 tx=0x7f9a5c009bb0 comp rx=0 tx=0).stop 2024-04-26T01:40:53.820 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:52.827+0000 7f9a70000700 1 -- 172.21.15.151:0/2051561285 shutdown_connections 2024-04-26T01:40:53.820 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:52.827+0000 7f9a70000700 1 --2- 172.21.15.151:0/2051561285 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9a6807eeb0 0x7f9a6807d310 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:53.820 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:52.827+0000 7f9a70000700 1 -- 172.21.15.151:0/2051561285 >> 172.21.15.151:0/2051561285 conn(0x7f9a68103cf0 msgr2=0x7f9a68106110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:40:53.820 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:52.827+0000 7f9a70000700 1 -- 172.21.15.151:0/2051561285 shutdown_connections 2024-04-26T01:40:53.820 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:52.827+0000 7f9a70000700 1 -- 172.21.15.151:0/2051561285 wait complete. 2024-04-26T01:40:53.820 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:52.828+0000 7f9a70000700 1 Processor -- start 2024-04-26T01:40:53.820 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:52.828+0000 7f9a70000700 1 -- start start 2024-04-26T01:40:53.820 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:52.829+0000 7f9a70000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9a6807eeb0 0x7f9a6819e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:53.820 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:52.829+0000 7f9a70000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9a6810ebd0 con 0x7f9a6807eeb0 2024-04-26T01:40:53.820 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:52.829+0000 7f9a6d600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9a6807eeb0 0x7f9a6819e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:53.820 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:52.829+0000 7f9a6d600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9a6807eeb0 0x7f9a6819e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37248/0 (socket says 172.21.15.151:37248) 2024-04-26T01:40:53.820 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:52.829+0000 7f9a6d600700 1 -- 172.21.15.151:0/904523462 learned_addr learned my addr 172.21.15.151:0/904523462 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:40:53.820 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:52.829+0000 7f9a6d600700 1 -- 172.21.15.151:0/904523462 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a6819ecb0 con 0x7f9a6807eeb0 2024-04-26T01:40:53.821 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:52.829+0000 7f9a6d600700 1 --2- 172.21.15.151:0/904523462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9a6807eeb0 0x7f9a6819e770 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c0117b0 tx=0x7f9a5c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:40:53.821 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:52.830+0000 7f9a66000700 1 -- 172.21.15.151:0/904523462 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a5c003af0 con 0x7f9a6807eeb0 2024-04-26T01:40:53.821 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:52.830+0000 7f9a66000700 1 -- 172.21.15.151:0/904523462 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9a5c020070 con 0x7f9a6807eeb0 2024-04-26T01:40:53.821 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:52.830+0000 7f9a66000700 1 -- 172.21.15.151:0/904523462 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a5c019630 con 0x7f9a6807eeb0 2024-04-26T01:40:53.821 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:52.830+0000 7f9a70000700 1 -- 172.21.15.151:0/904523462 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a6819ef40 con 0x7f9a6807eeb0 2024-04-26T01:40:53.821 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:52.830+0000 7f9a70000700 1 -- 172.21.15.151:0/904523462 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a681a1c30 con 0x7f9a6807eeb0 2024-04-26T01:40:53.821 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:52.831+0000 7f9a70000700 1 -- 172.21.15.151:0/904523462 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a680640a0 con 0x7f9a6807eeb0 2024-04-26T01:40:53.821 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:52.831+0000 7f9a66000700 1 -- 172.21.15.151:0/904523462 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f9a5c01a9c0 con 0x7f9a6807eeb0 2024-04-26T01:40:53.821 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:52.832+0000 7f9a66000700 1 --2- 172.21.15.151:0/904523462 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7f9a500395b0 0x7f9a5003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:53.821 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:52.832+0000 7f9a66000700 1 -- 172.21.15.151:0/904523462 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9a5c04dc00 con 0x7f9a6807eeb0 2024-04-26T01:40:53.821 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:52.832+0000 7f9a6cc00700 1 --2- 172.21.15.151:0/904523462 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7f9a500395b0 0x7f9a5003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:53.821 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:52.833+0000 7f9a6cc00700 1 --2- 172.21.15.151:0/904523462 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7f9a500395b0 0x7f9a5003ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9a54006fd0 tx=0x7f9a54006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:40:53.821 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:52.837+0000 7f9a66000700 1 -- 172.21.15.151:0/904523462 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9a5c00e260 con 0x7f9a6807eeb0 2024-04-26T01:40:53.821 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:53.081+0000 7f9a70000700 1 -- 172.21.15.151:0/904523462 --> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f9a68104f90 con 0x7f9a500395b0 2024-04-26T01:40:53.822 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:53.093+0000 7f9a66000700 1 -- 172.21.15.151:0/904523462 <== mgr.14120 v2:172.21.15.151:6800/796314699 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9a68104f90 con 0x7f9a500395b0 2024-04-26T01:40:53.822 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:53.096+0000 7f9a70000700 1 -- 172.21.15.151:0/904523462 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7f9a500395b0 msgr2=0x7f9a5003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:53.822 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:53.096+0000 7f9a70000700 1 --2- 172.21.15.151:0/904523462 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7f9a500395b0 0x7f9a5003ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9a54006fd0 tx=0x7f9a54006e40 comp rx=0 tx=0).stop 2024-04-26T01:40:53.822 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:53.096+0000 7f9a70000700 1 -- 172.21.15.151:0/904523462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9a6807eeb0 msgr2=0x7f9a6819e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:53.822 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:53.096+0000 7f9a70000700 1 --2- 172.21.15.151:0/904523462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9a6807eeb0 0x7f9a6819e770 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c0117b0 tx=0x7f9a5c011b70 comp rx=0 tx=0).stop 2024-04-26T01:40:53.822 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:53.097+0000 7f9a70000700 1 -- 172.21.15.151:0/904523462 shutdown_connections 2024-04-26T01:40:53.822 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:53.097+0000 7f9a70000700 1 --2- 172.21.15.151:0/904523462 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7f9a500395b0 0x7f9a5003ba70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:53.822 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:53.097+0000 7f9a70000700 1 --2- 172.21.15.151:0/904523462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9a6807eeb0 0x7f9a6819e770 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:53.822 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:53.097+0000 7f9a70000700 1 -- 172.21.15.151:0/904523462 >> 172.21.15.151:0/904523462 conn(0x7f9a68103cf0 msgr2=0x7f9a68104880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:40:53.822 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:53.097+0000 7f9a70000700 1 -- 172.21.15.151:0/904523462 shutdown_connections 2024-04-26T01:40:53.822 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:53.097+0000 7f9a70000700 1 -- 172.21.15.151:0/904523462 wait complete. 2024-04-26T01:40:54.870 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:54 smithi151 ceph-mon[28057]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:40:55.418 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-26T01:40:55.418 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:54.418+0000 7fb27ac00700 1 Processor -- start 2024-04-26T01:40:55.418 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:54.418+0000 7fb27ac00700 1 -- start start 2024-04-26T01:40:55.418 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:54.419+0000 7fb27ac00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb27407dda0 0x7fb27407c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:55.418 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:54.419+0000 7fb27ac00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb27407c7d0 con 0x7fb27407dda0 2024-04-26T01:40:55.418 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:54.419+0000 7fb273e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb27407dda0 0x7fb27407c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:55.419 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:54.420+0000 7fb273e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb27407dda0 0x7fb27407c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37260/0 (socket says 172.21.15.151:37260) 2024-04-26T01:40:55.419 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:54.420+0000 7fb273e00700 1 -- 172.21.15.151:0/479451524 learned_addr learned my addr 172.21.15.151:0/479451524 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:40:55.419 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:54.420+0000 7fb273e00700 1 -- 172.21.15.151:0/479451524 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb27407c950 con 0x7fb27407dda0 2024-04-26T01:40:55.419 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:54.420+0000 7fb273e00700 1 --2- 172.21.15.151:0/479451524 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb27407dda0 0x7fb27407c200 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb2640098a0 tx=0x7fb264009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f56ee300b0513ea server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:40:55.419 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:54.420+0000 7fb272a00700 1 -- 172.21.15.151:0/479451524 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb264004160 con 0x7fb27407dda0 2024-04-26T01:40:55.419 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:54.421+0000 7fb272a00700 1 -- 172.21.15.151:0/479451524 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb264004300 con 0x7fb27407dda0 2024-04-26T01:40:55.419 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:54.421+0000 7fb272a00700 1 -- 172.21.15.151:0/479451524 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb264011af0 con 0x7fb27407dda0 2024-04-26T01:40:55.419 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:54.421+0000 7fb27ac00700 1 -- 172.21.15.151:0/479451524 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb27407dda0 msgr2=0x7fb27407c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:55.419 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:54.421+0000 7fb27ac00700 1 --2- 172.21.15.151:0/479451524 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb27407dda0 0x7fb27407c200 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb2640098a0 tx=0x7fb264009bb0 comp rx=0 tx=0).stop 2024-04-26T01:40:55.419 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:54.421+0000 7fb27ac00700 1 -- 172.21.15.151:0/479451524 shutdown_connections 2024-04-26T01:40:55.419 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:54.421+0000 7fb27ac00700 1 --2- 172.21.15.151:0/479451524 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb27407dda0 0x7fb27407c200 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:55.419 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:54.421+0000 7fb27ac00700 1 -- 172.21.15.151:0/479451524 >> 172.21.15.151:0/479451524 conn(0x7fb274103c50 msgr2=0x7fb274106070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:40:55.420 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:54.421+0000 7fb27ac00700 1 -- 172.21.15.151:0/479451524 shutdown_connections 2024-04-26T01:40:55.420 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:54.422+0000 7fb27ac00700 1 -- 172.21.15.151:0/479451524 wait complete. 2024-04-26T01:40:55.420 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:54.422+0000 7fb27ac00700 1 Processor -- start 2024-04-26T01:40:55.420 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:54.423+0000 7fb27ac00700 1 -- start start 2024-04-26T01:40:55.420 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:54.423+0000 7fb27ac00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb27407dda0 0x7fb27419e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:55.420 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:54.423+0000 7fb27ac00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb27410eb70 con 0x7fb27407dda0 2024-04-26T01:40:55.420 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:54.423+0000 7fb273e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb27407dda0 0x7fb27419e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:55.420 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:54.423+0000 7fb273e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb27407dda0 0x7fb27419e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37264/0 (socket says 172.21.15.151:37264) 2024-04-26T01:40:55.420 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:54.423+0000 7fb273e00700 1 -- 172.21.15.151:0/1006035544 learned_addr learned my addr 172.21.15.151:0/1006035544 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:40:55.420 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:54.424+0000 7fb273e00700 1 -- 172.21.15.151:0/1006035544 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb27419ecc0 con 0x7fb27407dda0 2024-04-26T01:40:55.420 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:54.424+0000 7fb273e00700 1 --2- 172.21.15.151:0/1006035544 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb27407dda0 0x7fb27419e780 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb264009fb0 tx=0x7fb264011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:40:55.420 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:54.424+0000 7fb270c00700 1 -- 172.21.15.151:0/1006035544 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2640036a0 con 0x7fb27407dda0 2024-04-26T01:40:55.420 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:54.424+0000 7fb270c00700 1 -- 172.21.15.151:0/1006035544 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb26401a430 con 0x7fb27407dda0 2024-04-26T01:40:55.421 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:54.424+0000 7fb27ac00700 1 -- 172.21.15.151:0/1006035544 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb27419ef50 con 0x7fb27407dda0 2024-04-26T01:40:55.421 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:54.424+0000 7fb270c00700 1 -- 172.21.15.151:0/1006035544 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb264019550 con 0x7fb27407dda0 2024-04-26T01:40:55.421 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:54.424+0000 7fb27ac00700 1 -- 172.21.15.151:0/1006035544 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2741a1c40 con 0x7fb27407dda0 2024-04-26T01:40:55.421 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:54.425+0000 7fb27ac00700 1 -- 172.21.15.151:0/1006035544 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2740640a0 con 0x7fb27407dda0 2024-04-26T01:40:55.421 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:54.425+0000 7fb270c00700 1 -- 172.21.15.151:0/1006035544 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fb26401a8e0 con 0x7fb27407dda0 2024-04-26T01:40:55.421 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:54.426+0000 7fb270c00700 1 --2- 172.21.15.151:0/1006035544 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7fb2580395b0 0x7fb25803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:55.421 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:54.426+0000 7fb270c00700 1 -- 172.21.15.151:0/1006035544 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb26404d8d0 con 0x7fb27407dda0 2024-04-26T01:40:55.421 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:54.427+0000 7fb273400700 1 --2- 172.21.15.151:0/1006035544 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7fb2580395b0 0x7fb25803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:55.421 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:54.427+0000 7fb273400700 1 --2- 172.21.15.151:0/1006035544 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7fb2580395b0 0x7fb25803ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb260006fd0 tx=0x7fb260006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:40:55.422 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:54.432+0000 7fb270c00700 1 -- 172.21.15.151:0/1006035544 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb26401a690 con 0x7fb27407dda0 2024-04-26T01:40:55.422 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:54.661+0000 7fb27ac00700 1 -- 172.21.15.151:0/1006035544 --> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fb274104fa0 con 0x7fb2580395b0 2024-04-26T01:40:55.422 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:54.662+0000 7fb270c00700 1 -- 172.21.15.151:0/1006035544 <== mgr.14120 v2:172.21.15.151:6800/796314699 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fb274104fa0 con 0x7fb2580395b0 2024-04-26T01:40:55.422 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:54.665+0000 7fb27ac00700 1 -- 172.21.15.151:0/1006035544 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7fb2580395b0 msgr2=0x7fb25803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:55.422 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:54.665+0000 7fb27ac00700 1 --2- 172.21.15.151:0/1006035544 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7fb2580395b0 0x7fb25803ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb260006fd0 tx=0x7fb260006e40 comp rx=0 tx=0).stop 2024-04-26T01:40:55.422 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:54.665+0000 7fb27ac00700 1 -- 172.21.15.151:0/1006035544 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb27407dda0 msgr2=0x7fb27419e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:55.422 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:54.665+0000 7fb27ac00700 1 --2- 172.21.15.151:0/1006035544 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb27407dda0 0x7fb27419e780 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb264009fb0 tx=0x7fb264011940 comp rx=0 tx=0).stop 2024-04-26T01:40:55.422 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:54.665+0000 7fb27ac00700 1 -- 172.21.15.151:0/1006035544 shutdown_connections 2024-04-26T01:40:55.422 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:54.665+0000 7fb27ac00700 1 --2- 172.21.15.151:0/1006035544 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7fb2580395b0 0x7fb25803ba70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:55.422 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:54.665+0000 7fb27ac00700 1 --2- 172.21.15.151:0/1006035544 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb27407dda0 0x7fb27419e780 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:55.422 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:54.665+0000 7fb27ac00700 1 -- 172.21.15.151:0/1006035544 >> 172.21.15.151:0/1006035544 conn(0x7fb274103c50 msgr2=0x7fb274104850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:40:55.422 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:54.665+0000 7fb27ac00700 1 -- 172.21.15.151:0/1006035544 shutdown_connections 2024-04-26T01:40:55.422 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:54.665+0000 7fb27ac00700 1 -- 172.21.15.151:0/1006035544 wait complete. 2024-04-26T01:40:55.422 INFO:teuthology.orchestra.run.smithi151.stdout:Generating ssh key... 2024-04-26T01:40:55.709 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:55 smithi151 ceph-mon[28057]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:40:57.073 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:56 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: Generating public/private rsa key pair. 2024-04-26T01:40:57.073 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:56 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: Your identification has been saved in /tmp/tmpotz24iuu/key. 2024-04-26T01:40:57.073 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:56 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: Your public key has been saved in /tmp/tmpotz24iuu/key.pub. 2024-04-26T01:40:57.073 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:56 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: The key fingerprint is: 2024-04-26T01:40:57.073 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:56 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: SHA256:JzoyFIFd6QIPigiQgncnqEB9njRd3tVWLJ5hdEi3y1o ceph-c0e49e64-036d-11ef-bc93-c7b262605968 2024-04-26T01:40:57.074 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:56 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: The key's randomart image is: 2024-04-26T01:40:57.074 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:56 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: +---[RSA 3072]----+ 2024-04-26T01:40:57.074 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:56 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: |=o.+..o... ++o=| 2024-04-26T01:40:57.074 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:56 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: |* *.+=o.. . . ==o| 2024-04-26T01:40:57.074 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:56 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: |*+ ==+o . . o.= | 2024-04-26T01:40:57.074 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:56 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: |= o+. + .| 2024-04-26T01:40:57.074 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:56 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: | .. S . E | 2024-04-26T01:40:57.074 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:56 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: | . . o o | 2024-04-26T01:40:57.074 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:56 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: | o o . | 2024-04-26T01:40:57.074 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:56 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: | o . | 2024-04-26T01:40:57.074 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:56 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: | | 2024-04-26T01:40:57.074 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:40:56 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: +----[SHA256]-----+ 2024-04-26T01:40:57.411 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:56.005+0000 7fd6c4400700 1 Processor -- start 2024-04-26T01:40:57.415 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:56.005+0000 7fd6c4400700 1 -- start start 2024-04-26T01:40:57.415 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:56.005+0000 7fd6c4400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd6bc07eed0 0x7fd6bc07d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:57.415 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:56.005+0000 7fd6c4400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd6bc07d900 con 0x7fd6bc07eed0 2024-04-26T01:40:57.415 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:56.006+0000 7fd6c1a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd6bc07eed0 0x7fd6bc07d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:57.415 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:56.006+0000 7fd6c1a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd6bc07eed0 0x7fd6bc07d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37276/0 (socket says 172.21.15.151:37276) 2024-04-26T01:40:57.415 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:56.006+0000 7fd6c1a00700 1 -- 172.21.15.151:0/1184658522 learned_addr learned my addr 172.21.15.151:0/1184658522 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:40:57.416 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:56.006+0000 7fd6c1a00700 1 -- 172.21.15.151:0/1184658522 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6bc07da80 con 0x7fd6bc07eed0 2024-04-26T01:40:57.416 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:56.007+0000 7fd6c1a00700 1 --2- 172.21.15.151:0/1184658522 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd6bc07eed0 0x7fd6bc07d330 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd6b00098a0 tx=0x7fd6b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87831f86450b0547 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:40:57.416 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:56.007+0000 7fd6bbe00700 1 -- 172.21.15.151:0/1184658522 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6b0004160 con 0x7fd6bc07eed0 2024-04-26T01:40:57.416 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:56.007+0000 7fd6bbe00700 1 -- 172.21.15.151:0/1184658522 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd6b0004300 con 0x7fd6bc07eed0 2024-04-26T01:40:57.416 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:56.007+0000 7fd6bbe00700 1 -- 172.21.15.151:0/1184658522 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6b0011af0 con 0x7fd6bc07eed0 2024-04-26T01:40:57.416 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:56.008+0000 7fd6c4400700 1 -- 172.21.15.151:0/1184658522 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd6bc07eed0 msgr2=0x7fd6bc07d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:57.416 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:56.008+0000 7fd6c4400700 1 --2- 172.21.15.151:0/1184658522 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd6bc07eed0 0x7fd6bc07d330 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd6b00098a0 tx=0x7fd6b0009bb0 comp rx=0 tx=0).stop 2024-04-26T01:40:57.416 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:56.008+0000 7fd6c4400700 1 -- 172.21.15.151:0/1184658522 shutdown_connections 2024-04-26T01:40:57.416 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:56.008+0000 7fd6c4400700 1 --2- 172.21.15.151:0/1184658522 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd6bc07eed0 0x7fd6bc07d330 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:57.416 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:56.008+0000 7fd6c4400700 1 -- 172.21.15.151:0/1184658522 >> 172.21.15.151:0/1184658522 conn(0x7fd6bc103cf0 msgr2=0x7fd6bc106110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:40:57.416 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:56.008+0000 7fd6c4400700 1 -- 172.21.15.151:0/1184658522 shutdown_connections 2024-04-26T01:40:57.416 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:56.008+0000 7fd6c4400700 1 -- 172.21.15.151:0/1184658522 wait complete. 2024-04-26T01:40:57.416 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:56.009+0000 7fd6c4400700 1 Processor -- start 2024-04-26T01:40:57.416 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:56.009+0000 7fd6c4400700 1 -- start start 2024-04-26T01:40:57.417 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:56.010+0000 7fd6c4400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd6bc07eed0 0x7fd6bc19e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:57.417 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:56.010+0000 7fd6c4400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd6bc10ebd0 con 0x7fd6bc07eed0 2024-04-26T01:40:57.417 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:56.010+0000 7fd6c1a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd6bc07eed0 0x7fd6bc19e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:57.417 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:56.010+0000 7fd6c1a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd6bc07eed0 0x7fd6bc19e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37290/0 (socket says 172.21.15.151:37290) 2024-04-26T01:40:57.417 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:56.010+0000 7fd6c1a00700 1 -- 172.21.15.151:0/969151937 learned_addr learned my addr 172.21.15.151:0/969151937 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:40:57.417 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:56.010+0000 7fd6c1a00700 1 -- 172.21.15.151:0/969151937 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6bc19ec80 con 0x7fd6bc07eed0 2024-04-26T01:40:57.417 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:56.011+0000 7fd6c1a00700 1 --2- 172.21.15.151:0/969151937 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd6bc07eed0 0x7fd6bc19e740 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd6b00117b0 tx=0x7fd6b0011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:40:57.417 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:56.011+0000 7fd6ba000700 1 -- 172.21.15.151:0/969151937 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6b0003af0 con 0x7fd6bc07eed0 2024-04-26T01:40:57.417 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:56.011+0000 7fd6ba000700 1 -- 172.21.15.151:0/969151937 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd6b0020070 con 0x7fd6bc07eed0 2024-04-26T01:40:57.417 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:56.011+0000 7fd6c4400700 1 -- 172.21.15.151:0/969151937 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6bc19ef10 con 0x7fd6bc07eed0 2024-04-26T01:40:57.417 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:56.011+0000 7fd6ba000700 1 -- 172.21.15.151:0/969151937 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6b0019630 con 0x7fd6bc07eed0 2024-04-26T01:40:57.417 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:56.011+0000 7fd6c4400700 1 -- 172.21.15.151:0/969151937 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6bc1a1c00 con 0x7fd6bc07eed0 2024-04-26T01:40:57.417 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:56.012+0000 7fd6ba000700 1 -- 172.21.15.151:0/969151937 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fd6b001a9c0 con 0x7fd6bc07eed0 2024-04-26T01:40:57.417 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:56.012+0000 7fd6c4400700 1 -- 172.21.15.151:0/969151937 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6bc0640a0 con 0x7fd6bc07eed0 2024-04-26T01:40:57.418 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:56.013+0000 7fd6ba000700 1 --2- 172.21.15.151:0/969151937 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7fd6a40395b0 0x7fd6a403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:57.419 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:56.013+0000 7fd6ba000700 1 -- 172.21.15.151:0/969151937 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd6b004db40 con 0x7fd6bc07eed0 2024-04-26T01:40:57.419 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:56.013+0000 7fd6c1000700 1 --2- 172.21.15.151:0/969151937 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7fd6a40395b0 0x7fd6a403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:57.420 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:56.014+0000 7fd6c1000700 1 --2- 172.21.15.151:0/969151937 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7fd6a40395b0 0x7fd6a403ba70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8006fd0 tx=0x7fd6a8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:40:57.420 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:56.019+0000 7fd6ba000700 1 -- 172.21.15.151:0/969151937 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd6b000e260 con 0x7fd6bc07eed0 2024-04-26T01:40:57.420 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:56.245+0000 7fd6c4400700 1 -- 172.21.15.151:0/969151937 --> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fd6bc104f90 con 0x7fd6a40395b0 2024-04-26T01:40:57.420 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:56.702+0000 7fd6ba000700 1 -- 172.21.15.151:0/969151937 <== mgr.14120 v2:172.21.15.151:6800/796314699 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd6bc104f90 con 0x7fd6a40395b0 2024-04-26T01:40:57.420 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:56.705+0000 7fd6c4400700 1 -- 172.21.15.151:0/969151937 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7fd6a40395b0 msgr2=0x7fd6a403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:57.420 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:56.705+0000 7fd6c4400700 1 --2- 172.21.15.151:0/969151937 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7fd6a40395b0 0x7fd6a403ba70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8006fd0 tx=0x7fd6a8006e40 comp rx=0 tx=0).stop 2024-04-26T01:40:57.420 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:56.705+0000 7fd6c4400700 1 -- 172.21.15.151:0/969151937 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd6bc07eed0 msgr2=0x7fd6bc19e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:57.420 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:56.705+0000 7fd6c4400700 1 --2- 172.21.15.151:0/969151937 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd6bc07eed0 0x7fd6bc19e740 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd6b00117b0 tx=0x7fd6b0011b70 comp rx=0 tx=0).stop 2024-04-26T01:40:57.420 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:56.705+0000 7fd6c4400700 1 -- 172.21.15.151:0/969151937 shutdown_connections 2024-04-26T01:40:57.420 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:56.705+0000 7fd6c4400700 1 --2- 172.21.15.151:0/969151937 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7fd6a40395b0 0x7fd6a403ba70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:57.420 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:56.705+0000 7fd6c4400700 1 --2- 172.21.15.151:0/969151937 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd6bc07eed0 0x7fd6bc19e740 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:57.420 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:56.705+0000 7fd6c4400700 1 -- 172.21.15.151:0/969151937 >> 172.21.15.151:0/969151937 conn(0x7fd6bc103cf0 msgr2=0x7fd6bc104880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:40:57.421 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:56.705+0000 7fd6c4400700 1 -- 172.21.15.151:0/969151937 shutdown_connections 2024-04-26T01:40:57.421 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:56.705+0000 7fd6c4400700 1 -- 172.21.15.151:0/969151937 wait complete. 2024-04-26T01:40:58.003 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:57 smithi151 ceph-mon[28057]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:40:58.004 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:57 smithi151 ceph-mon[28057]: Generating ssh key... 2024-04-26T01:40:58.004 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:57 smithi151 ceph-mon[28057]: from='mgr.14120 172.21.15.151:0/3675138561' entity='mgr.x' 2024-04-26T01:40:58.004 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:57 smithi151 ceph-mon[28057]: from='mgr.14120 172.21.15.151:0/3675138561' entity='mgr.x' 2024-04-26T01:40:59.065 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:40:58 smithi151 ceph-mon[28057]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:40:59.084 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDNzd7KvbZNAU5d2PFfDejLyUsjAG4bUhv/GPWBuqIU9noj88W2T3gv7BkTF9R6/PaR51EPF4OHwFcBnz43ZB8WliajUE3Wk6sWWUgzFecylREAcIaPDvVBUFXAY6f3XTrdv++MwW4Hu2uOFkdrxJd3NXGxUVFsY7qidjTEXO+SQZUVj0aUanVE5UnwXoliiEMR/d1rO2fCQfNAe5Fjn88OoBGqAuXzw+ucltPTRjtD+8NKY5bP/jnuILiR1QPsG+VpJsk3HlRUW4UM55hTcpys7Px/L7f+BSUnQcoxOiODSG+8wl4tgULHqgrpdG6Rzdf0VB0DMkqtM2x9KqPA3GwAXG0VvH+zr/zlFgPUQ7c1wOspXYC0TSUMA7MlEjY6HauiYWT9FhZ/Jwp4F/Mawz4NW4XFYg7Y55p/C5T501vY7lE/GfXY/e3/QaHAxrNrtTfpuY0L8/f/D9Kk3U3iLHwkgWp4xv7fG4/hq7o72GeXcsKXX1NbH4R3Xh1frxFyUBE= ceph-c0e49e64-036d-11ef-bc93-c7b262605968 2024-04-26T01:40:59.084 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:58.087+0000 7f00e9400700 1 Processor -- start 2024-04-26T01:40:59.084 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:58.087+0000 7f00e9400700 1 -- start start 2024-04-26T01:40:59.084 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:58.088+0000 7f00e9400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f00e410a980 0x7f00e410ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:59.084 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:58.088+0000 7f00e9400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f00e410b370 con 0x7f00e410a980 2024-04-26T01:40:59.084 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:58.088+0000 7f00e2800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f00e410a980 0x7f00e410ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:59.084 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:58.088+0000 7f00e2800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f00e410a980 0x7f00e410ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37296/0 (socket says 172.21.15.151:37296) 2024-04-26T01:40:59.085 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:58.088+0000 7f00e2800700 1 -- 172.21.15.151:0/2635473678 learned_addr learned my addr 172.21.15.151:0/2635473678 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:40:59.085 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:58.089+0000 7f00e2800700 1 -- 172.21.15.151:0/2635473678 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e410bb80 con 0x7f00e410a980 2024-04-26T01:40:59.085 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:58.089+0000 7f00e2800700 1 --2- 172.21.15.151:0/2635473678 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f00e410a980 0x7f00e410ada0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f00d40098a0 tx=0x7f00d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a95970eab8a55e2f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:40:59.085 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:58.089+0000 7f00e1400700 1 -- 172.21.15.151:0/2635473678 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00d4004160 con 0x7f00e410a980 2024-04-26T01:40:59.085 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:58.089+0000 7f00e1400700 1 -- 172.21.15.151:0/2635473678 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f00d4004300 con 0x7f00e410a980 2024-04-26T01:40:59.085 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:58.089+0000 7f00e1400700 1 -- 172.21.15.151:0/2635473678 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00d4011af0 con 0x7f00e410a980 2024-04-26T01:40:59.085 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:58.090+0000 7f00e9400700 1 -- 172.21.15.151:0/2635473678 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f00e410a980 msgr2=0x7f00e410ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:59.085 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:58.090+0000 7f00e9400700 1 --2- 172.21.15.151:0/2635473678 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f00e410a980 0x7f00e410ada0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f00d40098a0 tx=0x7f00d4009bb0 comp rx=0 tx=0).stop 2024-04-26T01:40:59.085 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:58.090+0000 7f00e9400700 1 -- 172.21.15.151:0/2635473678 shutdown_connections 2024-04-26T01:40:59.085 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:58.090+0000 7f00e9400700 1 --2- 172.21.15.151:0/2635473678 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f00e410a980 0x7f00e410ada0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:59.085 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:58.090+0000 7f00e9400700 1 -- 172.21.15.151:0/2635473678 >> 172.21.15.151:0/2635473678 conn(0x7f00e407dd70 msgr2=0x7f00e41083d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:40:59.085 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:58.090+0000 7f00e9400700 1 -- 172.21.15.151:0/2635473678 shutdown_connections 2024-04-26T01:40:59.085 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:58.090+0000 7f00e9400700 1 -- 172.21.15.151:0/2635473678 wait complete. 2024-04-26T01:40:59.086 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:58.091+0000 7f00e9400700 1 Processor -- start 2024-04-26T01:40:59.086 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:58.092+0000 7f00e9400700 1 -- start start 2024-04-26T01:40:59.086 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:58.092+0000 7f00e9400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f00e410a980 0x7f00e419e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:59.086 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:58.092+0000 7f00e9400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f00e410edb0 con 0x7f00e410a980 2024-04-26T01:40:59.086 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:58.092+0000 7f00e2800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f00e410a980 0x7f00e419e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:59.086 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:58.092+0000 7f00e2800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f00e410a980 0x7f00e419e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37312/0 (socket says 172.21.15.151:37312) 2024-04-26T01:40:59.086 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:58.092+0000 7f00e2800700 1 -- 172.21.15.151:0/1363682927 learned_addr learned my addr 172.21.15.151:0/1363682927 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:40:59.086 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:58.093+0000 7f00e2800700 1 -- 172.21.15.151:0/1363682927 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e419eca0 con 0x7f00e410a980 2024-04-26T01:40:59.086 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:58.093+0000 7f00e2800700 1 --2- 172.21.15.151:0/1363682927 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f00e410a980 0x7f00e419e760 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f00d4009fb0 tx=0x7f00d4011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:40:59.086 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:58.093+0000 7f00db400700 1 -- 172.21.15.151:0/1363682927 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00d40036a0 con 0x7f00e410a980 2024-04-26T01:40:59.086 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:58.093+0000 7f00e9400700 1 -- 172.21.15.151:0/1363682927 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00e419ef30 con 0x7f00e410a980 2024-04-26T01:40:59.086 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:58.093+0000 7f00db400700 1 -- 172.21.15.151:0/1363682927 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f00d401a430 con 0x7f00e410a980 2024-04-26T01:40:59.086 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:58.093+0000 7f00db400700 1 -- 172.21.15.151:0/1363682927 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00d4019550 con 0x7f00e410a980 2024-04-26T01:40:59.087 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:58.093+0000 7f00e9400700 1 -- 172.21.15.151:0/1363682927 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00e41a1c20 con 0x7f00e410a980 2024-04-26T01:40:59.087 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:58.094+0000 7f00e9400700 1 -- 172.21.15.151:0/1363682927 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00e40640a0 con 0x7f00e410a980 2024-04-26T01:40:59.087 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:58.094+0000 7f00db400700 1 -- 172.21.15.151:0/1363682927 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f00d401a8e0 con 0x7f00e410a980 2024-04-26T01:40:59.087 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:58.095+0000 7f00db400700 1 --2- 172.21.15.151:0/1363682927 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7f00c80395b0 0x7f00c803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:59.087 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:58.095+0000 7f00db400700 1 -- 172.21.15.151:0/1363682927 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f00d404d990 con 0x7f00e410a980 2024-04-26T01:40:59.087 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:58.095+0000 7f00e1e00700 1 --2- 172.21.15.151:0/1363682927 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7f00c80395b0 0x7f00c803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:59.087 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:58.096+0000 7f00e1e00700 1 --2- 172.21.15.151:0/1363682927 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7f00c80395b0 0x7f00c803ba70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f00d0006fd0 tx=0x7f00d0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:40:59.087 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:58.101+0000 7f00db400700 1 -- 172.21.15.151:0/1363682927 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f00d401a690 con 0x7f00e410a980 2024-04-26T01:40:59.087 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:58.331+0000 7f00e9400700 1 -- 172.21.15.151:0/1363682927 --> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f00e41071e0 con 0x7f00c80395b0 2024-04-26T01:40:59.087 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:58.333+0000 7f00db400700 1 -- 172.21.15.151:0/1363682927 <== mgr.14120 v2:172.21.15.151:6800/796314699 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f00e41071e0 con 0x7f00c80395b0 2024-04-26T01:40:59.087 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:58.337+0000 7f00e9400700 1 -- 172.21.15.151:0/1363682927 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7f00c80395b0 msgr2=0x7f00c803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:59.087 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:58.337+0000 7f00e9400700 1 --2- 172.21.15.151:0/1363682927 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7f00c80395b0 0x7f00c803ba70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f00d0006fd0 tx=0x7f00d0006e40 comp rx=0 tx=0).stop 2024-04-26T01:40:59.087 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:58.337+0000 7f00e9400700 1 -- 172.21.15.151:0/1363682927 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f00e410a980 msgr2=0x7f00e419e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:59.088 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:58.337+0000 7f00e9400700 1 --2- 172.21.15.151:0/1363682927 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f00e410a980 0x7f00e419e760 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f00d4009fb0 tx=0x7f00d4011940 comp rx=0 tx=0).stop 2024-04-26T01:40:59.088 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:58.337+0000 7f00e9400700 1 -- 172.21.15.151:0/1363682927 shutdown_connections 2024-04-26T01:40:59.088 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:58.337+0000 7f00e9400700 1 --2- 172.21.15.151:0/1363682927 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7f00c80395b0 0x7f00c803ba70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:59.088 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:58.337+0000 7f00e9400700 1 --2- 172.21.15.151:0/1363682927 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f00e410a980 0x7f00e419e760 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:59.088 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:58.337+0000 7f00e9400700 1 -- 172.21.15.151:0/1363682927 >> 172.21.15.151:0/1363682927 conn(0x7f00e407dd70 msgr2=0x7f00e4106ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:40:59.088 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:58.337+0000 7f00e9400700 1 -- 172.21.15.151:0/1363682927 shutdown_connections 2024-04-26T01:40:59.088 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:58.337+0000 7f00e9400700 1 -- 172.21.15.151:0/1363682927 wait complete. 2024-04-26T01:40:59.088 INFO:teuthology.orchestra.run.smithi151.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-26T01:40:59.088 INFO:teuthology.orchestra.run.smithi151.stdout:Adding key to root@localhost authorized_keys... 2024-04-26T01:40:59.095 INFO:teuthology.orchestra.run.smithi151.stdout:Adding host smithi151... 2024-04-26T01:41:00.870 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:00 smithi151 ceph-mon[28057]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi151", "addr": "172.21.15.151", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:41:02.109 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:01 smithi151 ceph-mon[28057]: Deploying cephadm binary to smithi151 2024-04-26T01:41:06.668 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout Added host 'smithi151' with addr '172.21.15.151' 2024-04-26T01:41:06.669 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:59.713+0000 7f7238800700 1 Processor -- start 2024-04-26T01:41:06.669 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:59.713+0000 7f7238800700 1 -- start start 2024-04-26T01:41:06.669 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:59.713+0000 7f7238800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72301086c0 0x7f7230108ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:41:06.669 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:59.713+0000 7f7238800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f72301090b0 con 0x7f72301086c0 2024-04-26T01:41:06.669 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:59.714+0000 7f7235e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72301086c0 0x7f7230108ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:41:06.669 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:59.714+0000 7f7235e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72301086c0 0x7f7230108ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37326/0 (socket says 172.21.15.151:37326) 2024-04-26T01:41:06.669 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:59.714+0000 7f7235e00700 1 -- 172.21.15.151:0/4147370815 learned_addr learned my addr 172.21.15.151:0/4147370815 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:41:06.669 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:59.714+0000 7f7235e00700 1 -- 172.21.15.151:0/4147370815 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72301098c0 con 0x7f72301086c0 2024-04-26T01:41:06.669 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:59.714+0000 7f7235e00700 1 --2- 172.21.15.151:0/4147370815 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72301086c0 0x7f7230108ae0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f72240098a0 tx=0x7f7224009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fd01d085104724d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:41:06.669 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:59.715+0000 7f7234a00700 1 -- 172.21.15.151:0/4147370815 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7224004160 con 0x7f72301086c0 2024-04-26T01:41:06.669 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:59.715+0000 7f7234a00700 1 -- 172.21.15.151:0/4147370815 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7224004300 con 0x7f72301086c0 2024-04-26T01:41:06.670 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:59.715+0000 7f7234a00700 1 -- 172.21.15.151:0/4147370815 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7224011af0 con 0x7f72301086c0 2024-04-26T01:41:06.670 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:59.715+0000 7f7238800700 1 -- 172.21.15.151:0/4147370815 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72301086c0 msgr2=0x7f7230108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:41:06.670 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:59.715+0000 7f7238800700 1 --2- 172.21.15.151:0/4147370815 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72301086c0 0x7f7230108ae0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f72240098a0 tx=0x7f7224009bb0 comp rx=0 tx=0).stop 2024-04-26T01:41:06.670 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:59.716+0000 7f7238800700 1 -- 172.21.15.151:0/4147370815 shutdown_connections 2024-04-26T01:41:06.670 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:59.716+0000 7f7238800700 1 --2- 172.21.15.151:0/4147370815 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72301086c0 0x7f7230108ae0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:06.670 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:59.716+0000 7f7238800700 1 -- 172.21.15.151:0/4147370815 >> 172.21.15.151:0/4147370815 conn(0x7f7230103c40 msgr2=0x7f72301060a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:41:06.670 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:59.716+0000 7f7238800700 1 -- 172.21.15.151:0/4147370815 shutdown_connections 2024-04-26T01:41:06.670 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:59.716+0000 7f7238800700 1 -- 172.21.15.151:0/4147370815 wait complete. 2024-04-26T01:41:06.670 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:59.717+0000 7f7238800700 1 Processor -- start 2024-04-26T01:41:06.670 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:59.717+0000 7f7238800700 1 -- start start 2024-04-26T01:41:06.671 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:59.717+0000 7f7238800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72301086c0 0x7f723019e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:41:06.671 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:59.717+0000 7f7238800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f723010caf0 con 0x7f72301086c0 2024-04-26T01:41:06.671 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:59.718+0000 7f7235e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72301086c0 0x7f723019e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:41:06.671 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:59.718+0000 7f7235e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72301086c0 0x7f723019e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37342/0 (socket says 172.21.15.151:37342) 2024-04-26T01:41:06.671 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:59.718+0000 7f7235e00700 1 -- 172.21.15.151:0/1424933964 learned_addr learned my addr 172.21.15.151:0/1424933964 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:41:06.671 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:59.718+0000 7f7235e00700 1 -- 172.21.15.151:0/1424933964 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f723019ec70 con 0x7f72301086c0 2024-04-26T01:41:06.671 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:59.718+0000 7f7235e00700 1 --2- 172.21.15.151:0/1424933964 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72301086c0 0x7f723019e730 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7224000c00 tx=0x7f7224011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:41:06.671 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:59.719+0000 7f722aa00700 1 -- 172.21.15.151:0/1424933964 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7224003b20 con 0x7f72301086c0 2024-04-26T01:41:06.671 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:59.719+0000 7f722aa00700 1 -- 172.21.15.151:0/1424933964 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f722401a8c0 con 0x7f72301086c0 2024-04-26T01:41:06.671 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:59.719+0000 7f722aa00700 1 -- 172.21.15.151:0/1424933964 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7224024940 con 0x7f72301086c0 2024-04-26T01:41:06.671 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:59.719+0000 7f7238800700 1 -- 172.21.15.151:0/1424933964 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f723019ef00 con 0x7f72301086c0 2024-04-26T01:41:06.671 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:59.719+0000 7f7238800700 1 -- 172.21.15.151:0/1424933964 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72301a1bf0 con 0x7f72301086c0 2024-04-26T01:41:06.671 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:59.720+0000 7f7238800700 1 -- 172.21.15.151:0/1424933964 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72300640a0 con 0x7f72301086c0 2024-04-26T01:41:06.672 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:59.720+0000 7f722aa00700 1 -- 172.21.15.151:0/1424933964 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f7224022020 con 0x7f72301086c0 2024-04-26T01:41:06.672 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:59.720+0000 7f722aa00700 1 --2- 172.21.15.151:0/1424933964 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7f7218039600 0x7f721803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:41:06.672 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:59.720+0000 7f722aa00700 1 -- 172.21.15.151:0/1424933964 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f722404d7e0 con 0x7f72301086c0 2024-04-26T01:41:06.672 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:59.721+0000 7f7235400700 1 --2- 172.21.15.151:0/1424933964 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7f7218039600 0x7f721803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:41:06.672 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:59.721+0000 7f7235400700 1 --2- 172.21.15.151:0/1424933964 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7f7218039600 0x7f721803bac0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7220006fd0 tx=0x7f7220006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:41:06.672 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:59.726+0000 7f722aa00700 1 -- 172.21.15.151:0/1424933964 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f722400be50 con 0x7f72301086c0 2024-04-26T01:41:06.672 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:40:59.957+0000 7f7238800700 1 -- 172.21.15.151:0/1424933964 --> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi151", "addr": "172.21.15.151", "target": ["mon-mgr", ""]}) -- 0x7f7230104e30 con 0x7f7218039600 2024-04-26T01:41:06.672 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:05.900+0000 7f722aa00700 1 -- 172.21.15.151:0/1424933964 <== mgr.14120 v2:172.21.15.151:6800/796314699 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f7230104e30 con 0x7f7218039600 2024-04-26T01:41:06.672 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:05.903+0000 7f7238800700 1 -- 172.21.15.151:0/1424933964 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7f7218039600 msgr2=0x7f721803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:41:06.672 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:05.903+0000 7f7238800700 1 --2- 172.21.15.151:0/1424933964 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7f7218039600 0x7f721803bac0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7220006fd0 tx=0x7f7220006e40 comp rx=0 tx=0).stop 2024-04-26T01:41:06.672 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:05.903+0000 7f7238800700 1 -- 172.21.15.151:0/1424933964 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72301086c0 msgr2=0x7f723019e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:41:06.672 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:05.903+0000 7f7238800700 1 --2- 172.21.15.151:0/1424933964 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72301086c0 0x7f723019e730 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7224000c00 tx=0x7f7224011b70 comp rx=0 tx=0).stop 2024-04-26T01:41:06.672 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:05.904+0000 7f7238800700 1 -- 172.21.15.151:0/1424933964 shutdown_connections 2024-04-26T01:41:06.673 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:05.904+0000 7f7238800700 1 --2- 172.21.15.151:0/1424933964 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7f7218039600 0x7f721803bac0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:06.673 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:05.904+0000 7f7238800700 1 --2- 172.21.15.151:0/1424933964 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72301086c0 0x7f723019e730 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:06.673 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:05.904+0000 7f7238800700 1 -- 172.21.15.151:0/1424933964 >> 172.21.15.151:0/1424933964 conn(0x7f7230103c40 msgr2=0x7f7230104720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:41:06.673 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:05.904+0000 7f7238800700 1 -- 172.21.15.151:0/1424933964 shutdown_connections 2024-04-26T01:41:06.673 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:05.904+0000 7f7238800700 1 -- 172.21.15.151:0/1424933964 wait complete. 2024-04-26T01:41:06.673 INFO:teuthology.orchestra.run.smithi151.stdout:Deploying unmanaged mon service... 2024-04-26T01:41:06.977 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:06 smithi151 ceph-mon[28057]: from='mgr.14120 172.21.15.151:0/3675138561' entity='mgr.x' 2024-04-26T01:41:06.977 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:06 smithi151 ceph-mon[28057]: Added host smithi151 2024-04-26T01:41:06.977 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:06 smithi151 ceph-mon[28057]: from='mgr.14120 172.21.15.151:0/3675138561' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:41:08.774 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-26T01:41:08.774 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:07.305+0000 7f79dde00700 1 Processor -- start 2024-04-26T01:41:08.775 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:07.305+0000 7f79dde00700 1 -- start start 2024-04-26T01:41:08.775 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:07.306+0000 7f79dde00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79d81086c0 0x7f79d8108ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:41:08.775 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:07.306+0000 7f79dde00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f79d81090b0 con 0x7f79d81086c0 2024-04-26T01:41:08.775 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:07.306+0000 7f79d6a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79d81086c0 0x7f79d8108ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:41:08.775 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:07.307+0000 7f79d6a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79d81086c0 0x7f79d8108ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52980/0 (socket says 172.21.15.151:52980) 2024-04-26T01:41:08.775 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:07.307+0000 7f79d6a00700 1 -- 172.21.15.151:0/2873079850 learned_addr learned my addr 172.21.15.151:0/2873079850 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:41:08.775 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:07.307+0000 7f79d6a00700 1 -- 172.21.15.151:0/2873079850 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79d81098c0 con 0x7f79d81086c0 2024-04-26T01:41:08.775 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:07.307+0000 7f79d6a00700 1 --2- 172.21.15.151:0/2873079850 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79d81086c0 0x7f79d8108ae0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f79c80098a0 tx=0x7f79c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=800b3586bfa39b43 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:41:08.775 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:07.308+0000 7f79d5600700 1 -- 172.21.15.151:0/2873079850 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f79c8004160 con 0x7f79d81086c0 2024-04-26T01:41:08.775 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:07.308+0000 7f79d5600700 1 -- 172.21.15.151:0/2873079850 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f79c8004300 con 0x7f79d81086c0 2024-04-26T01:41:08.775 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:07.308+0000 7f79d5600700 1 -- 172.21.15.151:0/2873079850 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f79c8011af0 con 0x7f79d81086c0 2024-04-26T01:41:08.775 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:07.308+0000 7f79dde00700 1 -- 172.21.15.151:0/2873079850 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79d81086c0 msgr2=0x7f79d8108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:41:08.776 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:07.308+0000 7f79dde00700 1 --2- 172.21.15.151:0/2873079850 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79d81086c0 0x7f79d8108ae0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f79c80098a0 tx=0x7f79c8009bb0 comp rx=0 tx=0).stop 2024-04-26T01:41:08.776 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:07.308+0000 7f79dde00700 1 -- 172.21.15.151:0/2873079850 shutdown_connections 2024-04-26T01:41:08.776 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:07.308+0000 7f79dde00700 1 --2- 172.21.15.151:0/2873079850 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79d81086c0 0x7f79d8108ae0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:08.776 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:07.308+0000 7f79dde00700 1 -- 172.21.15.151:0/2873079850 >> 172.21.15.151:0/2873079850 conn(0x7f79d8103c40 msgr2=0x7f79d81060a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:41:08.776 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:07.309+0000 7f79dde00700 1 -- 172.21.15.151:0/2873079850 shutdown_connections 2024-04-26T01:41:08.776 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:07.309+0000 7f79dde00700 1 -- 172.21.15.151:0/2873079850 wait complete. 2024-04-26T01:41:08.776 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:07.310+0000 7f79dde00700 1 Processor -- start 2024-04-26T01:41:08.776 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:07.310+0000 7f79dde00700 1 -- start start 2024-04-26T01:41:08.776 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:07.310+0000 7f79dde00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79d81086c0 0x7f79d819c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:41:08.776 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:07.310+0000 7f79dde00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f79d810caf0 con 0x7f79d81086c0 2024-04-26T01:41:08.776 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:07.310+0000 7f79d6a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79d81086c0 0x7f79d819c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:41:08.776 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:07.310+0000 7f79d6a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79d81086c0 0x7f79d819c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52984/0 (socket says 172.21.15.151:52984) 2024-04-26T01:41:08.776 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:07.310+0000 7f79d6a00700 1 -- 172.21.15.151:0/2614938878 learned_addr learned my addr 172.21.15.151:0/2614938878 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:41:08.776 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:07.311+0000 7f79d6a00700 1 -- 172.21.15.151:0/2614938878 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79d819ca50 con 0x7f79d81086c0 2024-04-26T01:41:08.777 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:07.311+0000 7f79d6a00700 1 --2- 172.21.15.151:0/2614938878 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79d81086c0 0x7f79d819c510 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f79c8000c00 tx=0x7f79c8011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:41:08.777 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:07.311+0000 7f79dca00700 1 -- 172.21.15.151:0/2614938878 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f79c8003b20 con 0x7f79d81086c0 2024-04-26T01:41:08.777 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:07.311+0000 7f79dca00700 1 -- 172.21.15.151:0/2614938878 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f79c801a8c0 con 0x7f79d81086c0 2024-04-26T01:41:08.777 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:07.311+0000 7f79dca00700 1 -- 172.21.15.151:0/2614938878 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f79c8024940 con 0x7f79d81086c0 2024-04-26T01:41:08.777 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:07.311+0000 7f79dde00700 1 -- 172.21.15.151:0/2614938878 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79d819cce0 con 0x7f79d81086c0 2024-04-26T01:41:08.777 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:07.311+0000 7f79dde00700 1 -- 172.21.15.151:0/2614938878 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79d819f9d0 con 0x7f79d81086c0 2024-04-26T01:41:08.777 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:07.313+0000 7f79dca00700 1 -- 172.21.15.151:0/2614938878 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f79c8022020 con 0x7f79d81086c0 2024-04-26T01:41:08.777 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:07.312+0000 7f79dde00700 1 -- 172.21.15.151:0/2614938878 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79d80640a0 con 0x7f79d81086c0 2024-04-26T01:41:08.777 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:07.313+0000 7f79dca00700 1 --2- 172.21.15.151:0/2614938878 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7f79bc039600 0x7f79bc03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:41:08.777 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:07.313+0000 7f79dca00700 1 -- 172.21.15.151:0/2614938878 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f79c804d8a0 con 0x7f79d81086c0 2024-04-26T01:41:08.777 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:07.313+0000 7f79d6000700 1 --2- 172.21.15.151:0/2614938878 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7f79bc039600 0x7f79bc03bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:41:08.777 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:07.314+0000 7f79d6000700 1 --2- 172.21.15.151:0/2614938878 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7f79bc039600 0x7f79bc03bac0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f79c4006fd0 tx=0x7f79c4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:41:08.778 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:07.319+0000 7f79dca00700 1 -- 172.21.15.151:0/2614938878 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f79c800be50 con 0x7f79d81086c0 2024-04-26T01:41:08.778 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:07.570+0000 7f79dde00700 1 -- 172.21.15.151:0/2614938878 --> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f79d8104f30 con 0x7f79bc039600 2024-04-26T01:41:08.778 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:07.577+0000 7f79dca00700 1 -- 172.21.15.151:0/2614938878 <== mgr.14120 v2:172.21.15.151:6800/796314699 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f79d8104f30 con 0x7f79bc039600 2024-04-26T01:41:08.778 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:07.579+0000 7f79dde00700 1 -- 172.21.15.151:0/2614938878 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7f79bc039600 msgr2=0x7f79bc03bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:41:08.778 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:07.579+0000 7f79dde00700 1 --2- 172.21.15.151:0/2614938878 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7f79bc039600 0x7f79bc03bac0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f79c4006fd0 tx=0x7f79c4006e40 comp rx=0 tx=0).stop 2024-04-26T01:41:08.778 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:07.579+0000 7f79dde00700 1 -- 172.21.15.151:0/2614938878 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79d81086c0 msgr2=0x7f79d819c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:41:08.778 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:07.579+0000 7f79dde00700 1 --2- 172.21.15.151:0/2614938878 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79d81086c0 0x7f79d819c510 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f79c8000c00 tx=0x7f79c8011b70 comp rx=0 tx=0).stop 2024-04-26T01:41:08.778 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:07.580+0000 7f79dde00700 1 -- 172.21.15.151:0/2614938878 shutdown_connections 2024-04-26T01:41:08.778 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:07.580+0000 7f79dde00700 1 --2- 172.21.15.151:0/2614938878 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7f79bc039600 0x7f79bc03bac0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:08.778 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:07.580+0000 7f79dde00700 1 --2- 172.21.15.151:0/2614938878 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79d81086c0 0x7f79d819c510 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:08.778 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:07.580+0000 7f79dde00700 1 -- 172.21.15.151:0/2614938878 >> 172.21.15.151:0/2614938878 conn(0x7f79d8103c40 msgr2=0x7f79d8104820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:41:08.778 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:07.580+0000 7f79dde00700 1 -- 172.21.15.151:0/2614938878 shutdown_connections 2024-04-26T01:41:08.778 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:07.580+0000 7f79dde00700 1 -- 172.21.15.151:0/2614938878 wait complete. 2024-04-26T01:41:08.778 INFO:teuthology.orchestra.run.smithi151.stdout:Deploying unmanaged mgr service... 2024-04-26T01:41:08.870 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:08 smithi151 ceph-mon[28057]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:41:08.870 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:08 smithi151 ceph-mon[28057]: Saving service mon spec with placement count:5 2024-04-26T01:41:08.870 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:08 smithi151 ceph-mon[28057]: from='mgr.14120 172.21.15.151:0/3675138561' entity='mgr.x' 2024-04-26T01:41:10.810 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:10 smithi151 ceph-mon[28057]: from='mgr.14120 172.21.15.151:0/3675138561' entity='mgr.x' 2024-04-26T01:41:10.810 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:10 smithi151 ceph-mon[28057]: from='mgr.14120 172.21.15.151:0/3675138561' entity='mgr.x' 2024-04-26T01:41:10.821 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-26T01:41:10.822 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:09.877+0000 7fe9e0600700 1 Processor -- start 2024-04-26T01:41:10.822 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:09.877+0000 7fe9e0600700 1 -- start start 2024-04-26T01:41:10.822 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:09.878+0000 7fe9e0600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9d807eeb0 0x7fe9d807d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:41:10.823 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:09.878+0000 7fe9e0600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d807d8e0 con 0x7fe9d807eeb0 2024-04-26T01:41:10.823 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:09.878+0000 7fe9ddc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9d807eeb0 0x7fe9d807d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:41:10.823 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:09.878+0000 7fe9ddc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9d807eeb0 0x7fe9d807d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53936/0 (socket says 172.21.15.151:53936) 2024-04-26T01:41:10.823 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:09.878+0000 7fe9ddc00700 1 -- 172.21.15.151:0/169406320 learned_addr learned my addr 172.21.15.151:0/169406320 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:41:10.823 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:09.879+0000 7fe9ddc00700 1 -- 172.21.15.151:0/169406320 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9d807da60 con 0x7fe9d807eeb0 2024-04-26T01:41:10.824 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:09.879+0000 7fe9ddc00700 1 --2- 172.21.15.151:0/169406320 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9d807eeb0 0x7fe9d807d310 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc0098a0 tx=0x7fe9cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1100940eb5ee8ac8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:41:10.825 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:09.879+0000 7fe9d7e00700 1 -- 172.21.15.151:0/169406320 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9cc004160 con 0x7fe9d807eeb0 2024-04-26T01:41:10.825 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:09.880+0000 7fe9d7e00700 1 -- 172.21.15.151:0/169406320 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9cc004300 con 0x7fe9d807eeb0 2024-04-26T01:41:10.825 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:09.880+0000 7fe9d7e00700 1 -- 172.21.15.151:0/169406320 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9cc011a20 con 0x7fe9d807eeb0 2024-04-26T01:41:10.826 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:09.880+0000 7fe9e0600700 1 -- 172.21.15.151:0/169406320 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9d807eeb0 msgr2=0x7fe9d807d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:41:10.826 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:09.880+0000 7fe9e0600700 1 --2- 172.21.15.151:0/169406320 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9d807eeb0 0x7fe9d807d310 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc0098a0 tx=0x7fe9cc009bb0 comp rx=0 tx=0).stop 2024-04-26T01:41:10.826 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:09.880+0000 7fe9e0600700 1 -- 172.21.15.151:0/169406320 shutdown_connections 2024-04-26T01:41:10.826 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:09.880+0000 7fe9e0600700 1 --2- 172.21.15.151:0/169406320 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9d807eeb0 0x7fe9d807d310 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:10.826 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:09.880+0000 7fe9e0600700 1 -- 172.21.15.151:0/169406320 >> 172.21.15.151:0/169406320 conn(0x7fe9d8103ce0 msgr2=0x7fe9d8106100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:41:10.826 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:09.880+0000 7fe9e0600700 1 -- 172.21.15.151:0/169406320 shutdown_connections 2024-04-26T01:41:10.826 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:09.881+0000 7fe9e0600700 1 -- 172.21.15.151:0/169406320 wait complete. 2024-04-26T01:41:10.826 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:09.882+0000 7fe9e0600700 1 Processor -- start 2024-04-26T01:41:10.826 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:09.882+0000 7fe9e0600700 1 -- start start 2024-04-26T01:41:10.826 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:09.882+0000 7fe9e0600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9d807eeb0 0x7fe9d819e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:41:10.826 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:09.882+0000 7fe9e0600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d810ed00 con 0x7fe9d807eeb0 2024-04-26T01:41:10.826 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:09.882+0000 7fe9ddc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9d807eeb0 0x7fe9d819e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:41:10.829 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:09.882+0000 7fe9ddc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9d807eeb0 0x7fe9d819e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53940/0 (socket says 172.21.15.151:53940) 2024-04-26T01:41:10.830 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:09.882+0000 7fe9ddc00700 1 -- 172.21.15.151:0/392142791 learned_addr learned my addr 172.21.15.151:0/392142791 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:41:10.830 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:09.883+0000 7fe9ddc00700 1 -- 172.21.15.151:0/392142791 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9d819eca0 con 0x7fe9d807eeb0 2024-04-26T01:41:10.830 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:09.883+0000 7fe9ddc00700 1 --2- 172.21.15.151:0/392142791 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9d807eeb0 0x7fe9d819e760 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc009520 tx=0x7fe9cc011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:41:10.830 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:09.883+0000 7fe9d6000700 1 -- 172.21.15.151:0/392142791 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9cc0039e0 con 0x7fe9d807eeb0 2024-04-26T01:41:10.830 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:09.883+0000 7fe9d6000700 1 -- 172.21.15.151:0/392142791 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9cc01b8f0 con 0x7fe9d807eeb0 2024-04-26T01:41:10.830 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:09.883+0000 7fe9d6000700 1 -- 172.21.15.151:0/392142791 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9cc019770 con 0x7fe9d807eeb0 2024-04-26T01:41:10.831 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:09.883+0000 7fe9e0600700 1 -- 172.21.15.151:0/392142791 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9d819ef30 con 0x7fe9d807eeb0 2024-04-26T01:41:10.831 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:09.883+0000 7fe9e0600700 1 -- 172.21.15.151:0/392142791 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9d81a1c20 con 0x7fe9d807eeb0 2024-04-26T01:41:10.831 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:09.885+0000 7fe9e0600700 1 -- 172.21.15.151:0/392142791 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9d80640a0 con 0x7fe9d807eeb0 2024-04-26T01:41:10.831 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:09.885+0000 7fe9d6000700 1 -- 172.21.15.151:0/392142791 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fe9cc01b440 con 0x7fe9d807eeb0 2024-04-26T01:41:10.831 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:09.885+0000 7fe9d6000700 1 --2- 172.21.15.151:0/392142791 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7fe9c0039600 0x7fe9c003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:41:10.831 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:09.885+0000 7fe9d6000700 1 -- 172.21.15.151:0/392142791 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe9cc04de70 con 0x7fe9d807eeb0 2024-04-26T01:41:10.831 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:09.886+0000 7fe9dd200700 1 --2- 172.21.15.151:0/392142791 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7fe9c0039600 0x7fe9c003bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:41:10.831 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:09.886+0000 7fe9dd200700 1 --2- 172.21.15.151:0/392142791 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7fe9c0039600 0x7fe9c003bac0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4006fd0 tx=0x7fe9c4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:41:10.832 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:09.891+0000 7fe9d6000700 1 -- 172.21.15.151:0/392142791 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe9cc01f030 con 0x7fe9d807eeb0 2024-04-26T01:41:10.832 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:10.132+0000 7fe9e0600700 1 -- 172.21.15.151:0/392142791 --> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fe9d8104f50 con 0x7fe9c0039600 2024-04-26T01:41:10.832 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:10.139+0000 7fe9d6000700 1 -- 172.21.15.151:0/392142791 <== mgr.14120 v2:172.21.15.151:6800/796314699 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fe9d8104f50 con 0x7fe9c0039600 2024-04-26T01:41:10.832 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:10.142+0000 7fe9e0600700 1 -- 172.21.15.151:0/392142791 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7fe9c0039600 msgr2=0x7fe9c003bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:41:10.832 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:10.142+0000 7fe9e0600700 1 --2- 172.21.15.151:0/392142791 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7fe9c0039600 0x7fe9c003bac0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4006fd0 tx=0x7fe9c4006e40 comp rx=0 tx=0).stop 2024-04-26T01:41:10.832 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:10.142+0000 7fe9e0600700 1 -- 172.21.15.151:0/392142791 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9d807eeb0 msgr2=0x7fe9d819e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:41:10.832 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:10.142+0000 7fe9e0600700 1 --2- 172.21.15.151:0/392142791 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9d807eeb0 0x7fe9d819e760 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc009520 tx=0x7fe9cc011aa0 comp rx=0 tx=0).stop 2024-04-26T01:41:10.832 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:10.142+0000 7fe9e0600700 1 -- 172.21.15.151:0/392142791 shutdown_connections 2024-04-26T01:41:10.832 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:10.142+0000 7fe9e0600700 1 --2- 172.21.15.151:0/392142791 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7fe9c0039600 0x7fe9c003bac0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:10.832 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:10.142+0000 7fe9e0600700 1 --2- 172.21.15.151:0/392142791 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9d807eeb0 0x7fe9d819e760 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:10.832 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:10.142+0000 7fe9e0600700 1 -- 172.21.15.151:0/392142791 >> 172.21.15.151:0/392142791 conn(0x7fe9d8103ce0 msgr2=0x7fe9d8104840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:41:10.832 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:10.142+0000 7fe9e0600700 1 -- 172.21.15.151:0/392142791 shutdown_connections 2024-04-26T01:41:10.832 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:10.142+0000 7fe9e0600700 1 -- 172.21.15.151:0/392142791 wait complete. 2024-04-26T01:41:11.676 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:11 smithi151 ceph-mon[28057]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:41:11.676 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:11 smithi151 ceph-mon[28057]: Saving service mgr spec with placement count:2 2024-04-26T01:41:11.676 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:11 smithi151 ceph-mon[28057]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:41:11.676 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:11 smithi151 ceph-mon[28057]: from='mgr.14120 172.21.15.151:0/3675138561' entity='mgr.x' 2024-04-26T01:41:13.010 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:12 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/2734521725' entity='client.admin' 2024-04-26T01:41:13.023 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:11.424+0000 7f373a600700 1 Processor -- start 2024-04-26T01:41:13.024 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:11.424+0000 7f373a600700 1 -- start start 2024-04-26T01:41:13.024 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:11.425+0000 7f373a600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37341086c0 0x7f3734108ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:41:13.024 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:11.425+0000 7f373a600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f37341090b0 con 0x7f37341086c0 2024-04-26T01:41:13.024 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:11.425+0000 7f3733400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37341086c0 0x7f3734108ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:41:13.024 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:11.425+0000 7f3733400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37341086c0 0x7f3734108ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53944/0 (socket says 172.21.15.151:53944) 2024-04-26T01:41:13.024 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:11.426+0000 7f3733400700 1 -- 172.21.15.151:0/4091366078 learned_addr learned my addr 172.21.15.151:0/4091366078 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:41:13.024 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:11.426+0000 7f3733400700 1 -- 172.21.15.151:0/4091366078 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37341098c0 con 0x7f37341086c0 2024-04-26T01:41:13.024 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:11.426+0000 7f3733400700 1 --2- 172.21.15.151:0/4091366078 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37341086c0 0x7f3734108ae0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f37240098a0 tx=0x7f3724009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1d658da977148dc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:41:13.024 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:11.427+0000 7f3732000700 1 -- 172.21.15.151:0/4091366078 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3724004160 con 0x7f37341086c0 2024-04-26T01:41:13.024 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:11.427+0000 7f3732000700 1 -- 172.21.15.151:0/4091366078 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3724004300 con 0x7f37341086c0 2024-04-26T01:41:13.024 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:11.427+0000 7f3732000700 1 -- 172.21.15.151:0/4091366078 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3724011a20 con 0x7f37341086c0 2024-04-26T01:41:13.024 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:11.427+0000 7f373a600700 1 -- 172.21.15.151:0/4091366078 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37341086c0 msgr2=0x7f3734108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:41:13.025 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:11.427+0000 7f373a600700 1 --2- 172.21.15.151:0/4091366078 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37341086c0 0x7f3734108ae0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f37240098a0 tx=0x7f3724009bb0 comp rx=0 tx=0).stop 2024-04-26T01:41:13.025 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:11.427+0000 7f373a600700 1 -- 172.21.15.151:0/4091366078 shutdown_connections 2024-04-26T01:41:13.025 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:11.427+0000 7f373a600700 1 --2- 172.21.15.151:0/4091366078 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37341086c0 0x7f3734108ae0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:13.025 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:11.427+0000 7f373a600700 1 -- 172.21.15.151:0/4091366078 >> 172.21.15.151:0/4091366078 conn(0x7f3734103c40 msgr2=0x7f37341060a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:41:13.025 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:11.428+0000 7f373a600700 1 -- 172.21.15.151:0/4091366078 shutdown_connections 2024-04-26T01:41:13.025 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:11.428+0000 7f373a600700 1 -- 172.21.15.151:0/4091366078 wait complete. 2024-04-26T01:41:13.025 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:11.429+0000 7f373a600700 1 Processor -- start 2024-04-26T01:41:13.025 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:11.429+0000 7f373a600700 1 -- start start 2024-04-26T01:41:13.025 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:11.429+0000 7f373a600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37341086c0 0x7f373419e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:41:13.025 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:11.429+0000 7f373a600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f373410cc70 con 0x7f37341086c0 2024-04-26T01:41:13.025 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:11.429+0000 7f3733400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37341086c0 0x7f373419e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:41:13.025 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:11.429+0000 7f3733400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37341086c0 0x7f373419e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53950/0 (socket says 172.21.15.151:53950) 2024-04-26T01:41:13.025 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:11.429+0000 7f3733400700 1 -- 172.21.15.151:0/2734521725 learned_addr learned my addr 172.21.15.151:0/2734521725 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:41:13.025 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:11.430+0000 7f3733400700 1 -- 172.21.15.151:0/2734521725 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f373419ecd0 con 0x7f37341086c0 2024-04-26T01:41:13.026 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:11.430+0000 7f3733400700 1 --2- 172.21.15.151:0/2734521725 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37341086c0 0x7f373419e790 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3724009520 tx=0x7f3724011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:41:13.026 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:11.430+0000 7f372be00700 1 -- 172.21.15.151:0/2734521725 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37240039e0 con 0x7f37341086c0 2024-04-26T01:41:13.026 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:11.430+0000 7f372be00700 1 -- 172.21.15.151:0/2734521725 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f372401b8f0 con 0x7f37341086c0 2024-04-26T01:41:13.026 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:11.430+0000 7f373a600700 1 -- 172.21.15.151:0/2734521725 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f373419ef60 con 0x7f37341086c0 2024-04-26T01:41:13.026 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:11.430+0000 7f372be00700 1 -- 172.21.15.151:0/2734521725 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3724019770 con 0x7f37341086c0 2024-04-26T01:41:13.026 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:11.431+0000 7f373a600700 1 -- 172.21.15.151:0/2734521725 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37341a1c50 con 0x7f37341086c0 2024-04-26T01:41:13.026 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:11.432+0000 7f373a600700 1 -- 172.21.15.151:0/2734521725 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37340640a0 con 0x7f37341086c0 2024-04-26T01:41:13.026 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:11.432+0000 7f372be00700 1 -- 172.21.15.151:0/2734521725 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f372401b440 con 0x7f37341086c0 2024-04-26T01:41:13.026 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:11.433+0000 7f372be00700 1 --2- 172.21.15.151:0/2734521725 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7f3718039600 0x7f371803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:41:13.026 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:11.433+0000 7f372be00700 1 -- 172.21.15.151:0/2734521725 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f372404df40 con 0x7f37341086c0 2024-04-26T01:41:13.026 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:11.435+0000 7f3732a00700 1 --2- 172.21.15.151:0/2734521725 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7f3718039600 0x7f371803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:41:13.026 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:11.436+0000 7f3732a00700 1 --2- 172.21.15.151:0/2734521725 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7f3718039600 0x7f371803bac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3720006fd0 tx=0x7f3720006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:41:13.026 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:11.438+0000 7f372be00700 1 -- 172.21.15.151:0/2734521725 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f372401f030 con 0x7f37341086c0 2024-04-26T01:41:13.027 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:11.672+0000 7f373a600700 1 -- 172.21.15.151:0/2734521725 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f37341a2240 con 0x7f37341086c0 2024-04-26T01:41:13.027 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:11.678+0000 7f372be00700 1 -- 172.21.15.151:0/2734521725 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f372404d890 con 0x7f37341086c0 2024-04-26T01:41:13.027 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:11.682+0000 7f373a600700 1 -- 172.21.15.151:0/2734521725 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7f3718039600 msgr2=0x7f371803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:41:13.027 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:11.682+0000 7f373a600700 1 --2- 172.21.15.151:0/2734521725 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7f3718039600 0x7f371803bac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3720006fd0 tx=0x7f3720006e40 comp rx=0 tx=0).stop 2024-04-26T01:41:13.027 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:11.682+0000 7f373a600700 1 -- 172.21.15.151:0/2734521725 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37341086c0 msgr2=0x7f373419e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:41:13.027 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:11.682+0000 7f373a600700 1 --2- 172.21.15.151:0/2734521725 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37341086c0 0x7f373419e790 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3724009520 tx=0x7f3724011aa0 comp rx=0 tx=0).stop 2024-04-26T01:41:13.027 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:11.682+0000 7f373a600700 1 -- 172.21.15.151:0/2734521725 shutdown_connections 2024-04-26T01:41:13.027 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:11.682+0000 7f373a600700 1 --2- 172.21.15.151:0/2734521725 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7f3718039600 0x7f371803bac0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:13.027 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:11.682+0000 7f373a600700 1 --2- 172.21.15.151:0/2734521725 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37341086c0 0x7f373419e790 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:13.027 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:11.682+0000 7f373a600700 1 -- 172.21.15.151:0/2734521725 >> 172.21.15.151:0/2734521725 conn(0x7f3734103c40 msgr2=0x7f3734104700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:41:13.027 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:11.682+0000 7f373a600700 1 -- 172.21.15.151:0/2734521725 shutdown_connections 2024-04-26T01:41:13.027 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:11.682+0000 7f373a600700 1 -- 172.21.15.151:0/2734521725 wait complete. 2024-04-26T01:41:13.942 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:13 smithi151 ceph-mon[28057]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:41:13.942 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:13 smithi151 ceph-mon[28057]: from='mgr.14120 172.21.15.151:0/3675138561' entity='mgr.x' 2024-04-26T01:41:14.715 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:13.706+0000 7f7a8c400700 1 Processor -- start 2024-04-26T01:41:14.715 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:13.706+0000 7f7a8c400700 1 -- start start 2024-04-26T01:41:14.715 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:13.707+0000 7f7a8c400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7a841086c0 0x7f7a84108ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:41:14.716 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:13.707+0000 7f7a8c400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7a841090b0 con 0x7f7a841086c0 2024-04-26T01:41:14.716 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:13.707+0000 7f7a89a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7a841086c0 0x7f7a84108ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:41:14.716 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:13.707+0000 7f7a89a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7a841086c0 0x7f7a84108ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53966/0 (socket says 172.21.15.151:53966) 2024-04-26T01:41:14.716 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:13.707+0000 7f7a89a00700 1 -- 172.21.15.151:0/2395279601 learned_addr learned my addr 172.21.15.151:0/2395279601 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:41:14.716 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:13.707+0000 7f7a89a00700 1 -- 172.21.15.151:0/2395279601 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a841098c0 con 0x7f7a841086c0 2024-04-26T01:41:14.716 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:13.708+0000 7f7a89a00700 1 --2- 172.21.15.151:0/2395279601 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7a841086c0 0x7f7a84108ae0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7a780098a0 tx=0x7f7a78009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60b24abaafc8b4d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:41:14.716 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:13.708+0000 7f7a83e00700 1 -- 172.21.15.151:0/2395279601 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a78004160 con 0x7f7a841086c0 2024-04-26T01:41:14.716 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:13.708+0000 7f7a83e00700 1 -- 172.21.15.151:0/2395279601 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a78004300 con 0x7f7a841086c0 2024-04-26T01:41:14.716 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:13.708+0000 7f7a83e00700 1 -- 172.21.15.151:0/2395279601 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a78011a20 con 0x7f7a841086c0 2024-04-26T01:41:14.716 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:13.709+0000 7f7a8c400700 1 -- 172.21.15.151:0/2395279601 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7a841086c0 msgr2=0x7f7a84108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:41:14.716 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:13.709+0000 7f7a8c400700 1 --2- 172.21.15.151:0/2395279601 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7a841086c0 0x7f7a84108ae0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7a780098a0 tx=0x7f7a78009bb0 comp rx=0 tx=0).stop 2024-04-26T01:41:14.716 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:13.709+0000 7f7a8c400700 1 -- 172.21.15.151:0/2395279601 shutdown_connections 2024-04-26T01:41:14.716 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:13.709+0000 7f7a8c400700 1 --2- 172.21.15.151:0/2395279601 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7a841086c0 0x7f7a84108ae0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:14.717 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:13.709+0000 7f7a8c400700 1 -- 172.21.15.151:0/2395279601 >> 172.21.15.151:0/2395279601 conn(0x7f7a84103c40 msgr2=0x7f7a841060a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:41:14.717 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:13.709+0000 7f7a8c400700 1 -- 172.21.15.151:0/2395279601 shutdown_connections 2024-04-26T01:41:14.717 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:13.709+0000 7f7a8c400700 1 -- 172.21.15.151:0/2395279601 wait complete. 2024-04-26T01:41:14.717 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:13.710+0000 7f7a8c400700 1 Processor -- start 2024-04-26T01:41:14.717 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:13.710+0000 7f7a8c400700 1 -- start start 2024-04-26T01:41:14.717 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:13.711+0000 7f7a8c400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7a841086c0 0x7f7a8419c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:41:14.717 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:13.711+0000 7f7a8c400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8410cc70 con 0x7f7a841086c0 2024-04-26T01:41:14.717 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:13.711+0000 7f7a89a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7a841086c0 0x7f7a8419c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:41:14.717 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:13.711+0000 7f7a89a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7a841086c0 0x7f7a8419c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53972/0 (socket says 172.21.15.151:53972) 2024-04-26T01:41:14.717 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:13.711+0000 7f7a89a00700 1 -- 172.21.15.151:0/2697650175 learned_addr learned my addr 172.21.15.151:0/2697650175 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:41:14.717 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:13.711+0000 7f7a89a00700 1 -- 172.21.15.151:0/2697650175 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a8419ca90 con 0x7f7a841086c0 2024-04-26T01:41:14.717 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:13.712+0000 7f7a89a00700 1 --2- 172.21.15.151:0/2697650175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7a841086c0 0x7f7a8419c550 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7a78009520 tx=0x7f7a78011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:41:14.717 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:13.712+0000 7f7a82000700 1 -- 172.21.15.151:0/2697650175 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a780039e0 con 0x7f7a841086c0 2024-04-26T01:41:14.717 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:13.712+0000 7f7a82000700 1 -- 172.21.15.151:0/2697650175 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a7801b8f0 con 0x7f7a841086c0 2024-04-26T01:41:14.718 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:13.712+0000 7f7a82000700 1 -- 172.21.15.151:0/2697650175 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a78019770 con 0x7f7a841086c0 2024-04-26T01:41:14.718 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:13.712+0000 7f7a8c400700 1 -- 172.21.15.151:0/2697650175 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a8419cd20 con 0x7f7a841086c0 2024-04-26T01:41:14.718 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:13.712+0000 7f7a8c400700 1 -- 172.21.15.151:0/2697650175 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a8419fa10 con 0x7f7a841086c0 2024-04-26T01:41:14.718 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:13.713+0000 7f7a82000700 1 -- 172.21.15.151:0/2697650175 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f7a78019a20 con 0x7f7a841086c0 2024-04-26T01:41:14.718 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:13.714+0000 7f7a8c400700 1 -- 172.21.15.151:0/2697650175 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a840640a0 con 0x7f7a841086c0 2024-04-26T01:41:14.718 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:13.714+0000 7f7a82000700 1 --2- 172.21.15.151:0/2697650175 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7f7a6c039600 0x7f7a6c03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:41:14.718 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:13.714+0000 7f7a82000700 1 -- 172.21.15.151:0/2697650175 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7a7804dfb0 con 0x7f7a841086c0 2024-04-26T01:41:14.718 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:13.714+0000 7f7a89000700 1 --2- 172.21.15.151:0/2697650175 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7f7a6c039600 0x7f7a6c03bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:41:14.718 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:13.715+0000 7f7a89000700 1 --2- 172.21.15.151:0/2697650175 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7f7a6c039600 0x7f7a6c03bac0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7a70006fd0 tx=0x7f7a70006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:41:14.718 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:13.719+0000 7f7a82000700 1 -- 172.21.15.151:0/2697650175 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7a7801f030 con 0x7f7a841086c0 2024-04-26T01:41:14.718 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:13.944+0000 7f7a8c400700 1 -- 172.21.15.151:0/2697650175 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f7a841a0070 con 0x7f7a841086c0 2024-04-26T01:41:14.718 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:13.948+0000 7f7a82000700 1 -- 172.21.15.151:0/2697650175 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f7a7801baa0 con 0x7f7a841086c0 2024-04-26T01:41:14.718 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:13.951+0000 7f7a8c400700 1 -- 172.21.15.151:0/2697650175 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7f7a6c039600 msgr2=0x7f7a6c03bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:41:14.719 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:13.951+0000 7f7a8c400700 1 --2- 172.21.15.151:0/2697650175 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7f7a6c039600 0x7f7a6c03bac0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7a70006fd0 tx=0x7f7a70006e40 comp rx=0 tx=0).stop 2024-04-26T01:41:14.719 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:13.951+0000 7f7a8c400700 1 -- 172.21.15.151:0/2697650175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7a841086c0 msgr2=0x7f7a8419c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:41:14.719 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:13.951+0000 7f7a8c400700 1 --2- 172.21.15.151:0/2697650175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7a841086c0 0x7f7a8419c550 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7a78009520 tx=0x7f7a78011aa0 comp rx=0 tx=0).stop 2024-04-26T01:41:14.719 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:13.952+0000 7f7a8c400700 1 -- 172.21.15.151:0/2697650175 shutdown_connections 2024-04-26T01:41:14.719 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:13.952+0000 7f7a8c400700 1 --2- 172.21.15.151:0/2697650175 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7f7a6c039600 0x7f7a6c03bac0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:14.719 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:13.952+0000 7f7a8c400700 1 --2- 172.21.15.151:0/2697650175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7a841086c0 0x7f7a8419c550 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:14.719 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:13.952+0000 7f7a8c400700 1 -- 172.21.15.151:0/2697650175 >> 172.21.15.151:0/2697650175 conn(0x7f7a84103c40 msgr2=0x7f7a841047c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:41:14.719 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:13.952+0000 7f7a8c400700 1 -- 172.21.15.151:0/2697650175 shutdown_connections 2024-04-26T01:41:14.719 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:13.952+0000 7f7a8c400700 1 -- 172.21.15.151:0/2697650175 wait complete. 2024-04-26T01:41:14.719 INFO:teuthology.orchestra.run.smithi151.stdout:Enabling the dashboard module... 2024-04-26T01:41:14.951 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:14 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/2697650175' entity='client.admin' 2024-04-26T01:41:14.951 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:14 smithi151 ceph-mon[28057]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:41:16.255 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:16 smithi151 ceph-mon[28057]: from='mgr.14120 172.21.15.151:0/3675138561' entity='mgr.x' 2024-04-26T01:41:16.255 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:16 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/528747922' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-26T01:41:16.558 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:41:16 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: ignoring --setuser ceph since I am not root 2024-04-26T01:41:16.558 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:41:16 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: ignoring --setgroup ceph since I am not root 2024-04-26T01:41:16.558 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:41:16 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:41:16.435+0000 7f683c6e4200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T01:41:16.822 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:41:16 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:41:16.685+0000 7f683c6e4200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T01:41:16.822 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:41:16 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:41:16.824+0000 7f683c6e4200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T01:41:17.210 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:15.355+0000 7f89e5c00700 1 Processor -- start 2024-04-26T01:41:17.211 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:15.355+0000 7f89e5c00700 1 -- start start 2024-04-26T01:41:17.211 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:15.355+0000 7f89e5c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89e007dda0 0x7f89e007c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:41:17.211 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:15.356+0000 7f89e5c00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f89e007c7d0 con 0x7f89e007dda0 2024-04-26T01:41:17.212 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:15.356+0000 7f89dea00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89e007dda0 0x7f89e007c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:41:17.212 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:15.356+0000 7f89dea00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89e007dda0 0x7f89e007c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53978/0 (socket says 172.21.15.151:53978) 2024-04-26T01:41:17.212 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:15.356+0000 7f89dea00700 1 -- 172.21.15.151:0/3482262665 learned_addr learned my addr 172.21.15.151:0/3482262665 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:41:17.212 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:15.357+0000 7f89dea00700 1 -- 172.21.15.151:0/3482262665 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89e007c950 con 0x7f89e007dda0 2024-04-26T01:41:17.212 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:15.357+0000 7f89dea00700 1 --2- 172.21.15.151:0/3482262665 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89e007dda0 0x7f89e007c200 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f89d00098a0 tx=0x7f89d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad189acca056235b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:41:17.213 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:15.357+0000 7f89dd600700 1 -- 172.21.15.151:0/3482262665 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89d0004160 con 0x7f89e007dda0 2024-04-26T01:41:17.213 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:15.357+0000 7f89dd600700 1 -- 172.21.15.151:0/3482262665 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89d0004300 con 0x7f89e007dda0 2024-04-26T01:41:17.213 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:15.357+0000 7f89dd600700 1 -- 172.21.15.151:0/3482262665 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89d0011a20 con 0x7f89e007dda0 2024-04-26T01:41:17.214 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:15.358+0000 7f89e5c00700 1 -- 172.21.15.151:0/3482262665 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89e007dda0 msgr2=0x7f89e007c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:41:17.214 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:15.358+0000 7f89e5c00700 1 --2- 172.21.15.151:0/3482262665 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89e007dda0 0x7f89e007c200 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f89d00098a0 tx=0x7f89d0009bb0 comp rx=0 tx=0).stop 2024-04-26T01:41:17.214 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:15.358+0000 7f89e5c00700 1 -- 172.21.15.151:0/3482262665 shutdown_connections 2024-04-26T01:41:17.214 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:15.358+0000 7f89e5c00700 1 --2- 172.21.15.151:0/3482262665 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89e007dda0 0x7f89e007c200 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:17.214 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:15.358+0000 7f89e5c00700 1 -- 172.21.15.151:0/3482262665 >> 172.21.15.151:0/3482262665 conn(0x7f89e0103c40 msgr2=0x7f89e01060a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:41:17.214 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:15.358+0000 7f89e5c00700 1 -- 172.21.15.151:0/3482262665 shutdown_connections 2024-04-26T01:41:17.214 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:15.358+0000 7f89e5c00700 1 -- 172.21.15.151:0/3482262665 wait complete. 2024-04-26T01:41:17.214 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:15.359+0000 7f89e5c00700 1 Processor -- start 2024-04-26T01:41:17.217 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:15.360+0000 7f89e5c00700 1 -- start start 2024-04-26T01:41:17.217 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:15.360+0000 7f89e5c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89e007dda0 0x7f89e019e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:41:17.217 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:15.360+0000 7f89e5c00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f89e010ecc0 con 0x7f89e007dda0 2024-04-26T01:41:17.217 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:15.360+0000 7f89dea00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89e007dda0 0x7f89e019e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:41:17.217 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:15.360+0000 7f89dea00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89e007dda0 0x7f89e019e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53988/0 (socket says 172.21.15.151:53988) 2024-04-26T01:41:17.218 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:15.360+0000 7f89dea00700 1 -- 172.21.15.151:0/528747922 learned_addr learned my addr 172.21.15.151:0/528747922 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:41:17.218 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:15.361+0000 7f89dea00700 1 -- 172.21.15.151:0/528747922 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89e019ec60 con 0x7f89e007dda0 2024-04-26T01:41:17.218 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:15.361+0000 7f89dea00700 1 --2- 172.21.15.151:0/528747922 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89e007dda0 0x7f89e019e720 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f89d0000c00 tx=0x7f89d0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:41:17.218 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:15.361+0000 7f89d7400700 1 -- 172.21.15.151:0/528747922 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89d0003aa0 con 0x7f89e007dda0 2024-04-26T01:41:17.218 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:15.361+0000 7f89d7400700 1 -- 172.21.15.151:0/528747922 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89d001b8f0 con 0x7f89e007dda0 2024-04-26T01:41:17.218 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:15.361+0000 7f89e5c00700 1 -- 172.21.15.151:0/528747922 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89e019eef0 con 0x7f89e007dda0 2024-04-26T01:41:17.218 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:15.361+0000 7f89d7400700 1 -- 172.21.15.151:0/528747922 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89d0023890 con 0x7f89e007dda0 2024-04-26T01:41:17.218 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:15.361+0000 7f89e5c00700 1 -- 172.21.15.151:0/528747922 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89e01a1be0 con 0x7f89e007dda0 2024-04-26T01:41:17.218 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:15.363+0000 7f89d7400700 1 -- 172.21.15.151:0/528747922 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f89d002c3f0 con 0x7f89e007dda0 2024-04-26T01:41:17.218 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:15.363+0000 7f89e5c00700 1 -- 172.21.15.151:0/528747922 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89e00640a0 con 0x7f89e007dda0 2024-04-26T01:41:17.218 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:15.363+0000 7f89d7400700 1 --2- 172.21.15.151:0/528747922 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7f89c4039600 0x7f89c403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:41:17.218 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:15.363+0000 7f89d7400700 1 -- 172.21.15.151:0/528747922 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f89d004de70 con 0x7f89e007dda0 2024-04-26T01:41:17.219 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:15.364+0000 7f89de000700 1 --2- 172.21.15.151:0/528747922 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7f89c4039600 0x7f89c403bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:41:17.219 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:15.364+0000 7f89de000700 1 --2- 172.21.15.151:0/528747922 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7f89c4039600 0x7f89c403bac0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f89cc006fd0 tx=0x7f89cc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:41:17.219 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:15.369+0000 7f89d7400700 1 -- 172.21.15.151:0/528747922 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f89d001f030 con 0x7f89e007dda0 2024-04-26T01:41:17.219 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:15.664+0000 7f89e5c00700 1 -- 172.21.15.151:0/528747922 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f89e01a21d0 con 0x7f89e007dda0 2024-04-26T01:41:17.219 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:16.110+0000 7f89d7400700 1 -- 172.21.15.151:0/528747922 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f89d004d800 con 0x7f89e007dda0 2024-04-26T01:41:17.219 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:16.114+0000 7f89d7400700 1 -- 172.21.15.151:0/528747922 <== mon.0 v2:172.21.15.151:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f89d000bcb0 con 0x7f89e007dda0 2024-04-26T01:41:17.219 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:16.117+0000 7f89e5c00700 1 -- 172.21.15.151:0/528747922 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7f89c4039600 msgr2=0x7f89c403bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:41:17.219 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:16.117+0000 7f89e5c00700 1 --2- 172.21.15.151:0/528747922 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7f89c4039600 0x7f89c403bac0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f89cc006fd0 tx=0x7f89cc006e40 comp rx=0 tx=0).stop 2024-04-26T01:41:17.219 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:16.117+0000 7f89e5c00700 1 -- 172.21.15.151:0/528747922 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89e007dda0 msgr2=0x7f89e019e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:41:17.219 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:16.117+0000 7f89e5c00700 1 --2- 172.21.15.151:0/528747922 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89e007dda0 0x7f89e019e720 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f89d0000c00 tx=0x7f89d0011aa0 comp rx=0 tx=0).stop 2024-04-26T01:41:17.220 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:16.117+0000 7f89e5c00700 1 -- 172.21.15.151:0/528747922 shutdown_connections 2024-04-26T01:41:17.220 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:16.117+0000 7f89e5c00700 1 --2- 172.21.15.151:0/528747922 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7f89c4039600 0x7f89c403bac0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:17.220 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:16.117+0000 7f89e5c00700 1 --2- 172.21.15.151:0/528747922 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89e007dda0 0x7f89e019e720 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:17.220 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:16.117+0000 7f89e5c00700 1 -- 172.21.15.151:0/528747922 >> 172.21.15.151:0/528747922 conn(0x7f89e0103c40 msgr2=0x7f89e01047a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:41:17.220 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:16.117+0000 7f89e5c00700 1 -- 172.21.15.151:0/528747922 shutdown_connections 2024-04-26T01:41:17.220 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:16.117+0000 7f89e5c00700 1 -- 172.21.15.151:0/528747922 wait complete. 2024-04-26T01:41:17.370 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:17 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/528747922' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-26T01:41:17.370 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:17 smithi151 ceph-mon[28057]: mgrmap e9: x(active, since 25s) 2024-04-26T01:41:17.370 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:41:17 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:41:17.226+0000 7f683c6e4200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T01:41:17.370 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:41:17 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:41:17.342+0000 7f683c6e4200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T01:41:18.870 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:18 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/435286340' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-26T01:41:19.144 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:41:18 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:41:18.916+0000 7f683c6e4200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T01:41:19.144 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:41:19 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:41:19.146+0000 7f683c6e4200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T01:41:19.255 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout { 2024-04-26T01:41:19.255 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-26T01:41:19.255 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-26T01:41:19.255 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-26T01:41:19.255 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-26T01:41:19.255 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout } 2024-04-26T01:41:19.255 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:18.225+0000 7fc092e00700 1 Processor -- start 2024-04-26T01:41:19.258 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:18.225+0000 7fc092e00700 1 -- start start 2024-04-26T01:41:19.258 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:18.226+0000 7fc092e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc08c106d80 0x7fc08c1091b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:41:19.258 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:18.226+0000 7fc092e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c1096f0 con 0x7fc08c106d80 2024-04-26T01:41:19.259 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:18.226+0000 7fc091a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc08c106d80 0x7fc08c1091b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:41:19.259 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:18.226+0000 7fc091a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc08c106d80 0x7fc08c1091b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:54016/0 (socket says 172.21.15.151:54016) 2024-04-26T01:41:19.259 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:18.226+0000 7fc091a00700 1 -- 172.21.15.151:0/1374912621 learned_addr learned my addr 172.21.15.151:0/1374912621 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:41:19.259 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:18.227+0000 7fc091a00700 1 -- 172.21.15.151:0/1374912621 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc08c109870 con 0x7fc08c106d80 2024-04-26T01:41:19.259 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:18.227+0000 7fc091a00700 1 --2- 172.21.15.151:0/1374912621 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc08c106d80 0x7fc08c1091b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc0800098a0 tx=0x7fc080009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca12babc7efbcfc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:41:19.259 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:18.227+0000 7fc08be00700 1 -- 172.21.15.151:0/1374912621 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc080004160 con 0x7fc08c106d80 2024-04-26T01:41:19.259 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:18.227+0000 7fc08be00700 1 -- 172.21.15.151:0/1374912621 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc080004300 con 0x7fc08c106d80 2024-04-26T01:41:19.259 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:18.227+0000 7fc08be00700 1 -- 172.21.15.151:0/1374912621 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc080011a20 con 0x7fc08c106d80 2024-04-26T01:41:19.259 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:18.228+0000 7fc092e00700 1 -- 172.21.15.151:0/1374912621 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc08c106d80 msgr2=0x7fc08c1091b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:41:19.259 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:18.228+0000 7fc092e00700 1 --2- 172.21.15.151:0/1374912621 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc08c106d80 0x7fc08c1091b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc0800098a0 tx=0x7fc080009bb0 comp rx=0 tx=0).stop 2024-04-26T01:41:19.259 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:18.228+0000 7fc092e00700 1 -- 172.21.15.151:0/1374912621 shutdown_connections 2024-04-26T01:41:19.260 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:18.228+0000 7fc092e00700 1 --2- 172.21.15.151:0/1374912621 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc08c106d80 0x7fc08c1091b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:19.260 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:18.228+0000 7fc092e00700 1 -- 172.21.15.151:0/1374912621 >> 172.21.15.151:0/1374912621 conn(0x7fc08c1029c0 msgr2=0x7fc08c104e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:41:19.260 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:18.228+0000 7fc092e00700 1 -- 172.21.15.151:0/1374912621 shutdown_connections 2024-04-26T01:41:19.260 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:18.228+0000 7fc092e00700 1 -- 172.21.15.151:0/1374912621 wait complete. 2024-04-26T01:41:19.260 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:18.229+0000 7fc092e00700 1 Processor -- start 2024-04-26T01:41:19.260 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:18.229+0000 7fc092e00700 1 -- start start 2024-04-26T01:41:19.260 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:18.230+0000 7fc092e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc08c106d80 0x7fc08c1a2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:41:19.260 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:18.230+0000 7fc092e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c10a8e0 con 0x7fc08c106d80 2024-04-26T01:41:19.260 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:18.230+0000 7fc091a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc08c106d80 0x7fc08c1a2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:41:19.260 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:18.230+0000 7fc091a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc08c106d80 0x7fc08c1a2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:54026/0 (socket says 172.21.15.151:54026) 2024-04-26T01:41:19.260 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:18.230+0000 7fc091a00700 1 -- 172.21.15.151:0/435286340 learned_addr learned my addr 172.21.15.151:0/435286340 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:41:19.260 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:18.230+0000 7fc091a00700 1 -- 172.21.15.151:0/435286340 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc08c1a2fd0 con 0x7fc08c106d80 2024-04-26T01:41:19.260 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:18.231+0000 7fc091a00700 1 --2- 172.21.15.151:0/435286340 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc08c106d80 0x7fc08c1a2a90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc080000c00 tx=0x7fc080011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:41:19.263 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:18.231+0000 7fc08a000700 1 -- 172.21.15.151:0/435286340 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc080003aa0 con 0x7fc08c106d80 2024-04-26T01:41:19.263 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:18.231+0000 7fc08a000700 1 -- 172.21.15.151:0/435286340 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc08001b8f0 con 0x7fc08c106d80 2024-04-26T01:41:19.263 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:18.231+0000 7fc08a000700 1 -- 172.21.15.151:0/435286340 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc080023870 con 0x7fc08c106d80 2024-04-26T01:41:19.264 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:18.231+0000 7fc092e00700 1 -- 172.21.15.151:0/435286340 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc08c1a3260 con 0x7fc08c106d80 2024-04-26T01:41:19.264 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:18.231+0000 7fc092e00700 1 -- 172.21.15.151:0/435286340 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc08c1a5f50 con 0x7fc08c106d80 2024-04-26T01:41:19.264 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:18.232+0000 7fc08a000700 1 -- 172.21.15.151:0/435286340 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7fc08001b440 con 0x7fc08c106d80 2024-04-26T01:41:19.264 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:18.232+0000 7fc092e00700 1 -- 172.21.15.151:0/435286340 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc08c0640a0 con 0x7fc08c106d80 2024-04-26T01:41:19.264 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:18.233+0000 7fc08a000700 1 --2- 172.21.15.151:0/435286340 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7fc074039660 0x7fc07403bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:41:19.264 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:18.233+0000 7fc08a000700 1 -- 172.21.15.151:0/435286340 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc08004df80 con 0x7fc08c106d80 2024-04-26T01:41:19.264 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:18.233+0000 7fc091000700 1 -- 172.21.15.151:0/435286340 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7fc074039660 msgr2=0x7fc07403bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.151:6800/796314699 2024-04-26T01:41:19.264 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:18.233+0000 7fc091000700 1 --2- 172.21.15.151:0/435286340 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7fc074039660 0x7fc07403bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T01:41:19.264 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:18.238+0000 7fc08a000700 1 -- 172.21.15.151:0/435286340 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc08001f030 con 0x7fc08c106d80 2024-04-26T01:41:19.264 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:18.434+0000 7fc091000700 1 -- 172.21.15.151:0/435286340 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7fc074039660 msgr2=0x7fc07403bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.151:6800/796314699 2024-04-26T01:41:19.264 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:18.434+0000 7fc091000700 1 --2- 172.21.15.151:0/435286340 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7fc074039660 0x7fc07403bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T01:41:19.264 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:18.517+0000 7fc092e00700 1 -- 172.21.15.151:0/435286340 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fc08c02e100 con 0x7fc08c106d80 2024-04-26T01:41:19.264 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:18.518+0000 7fc08a000700 1 -- 172.21.15.151:0/435286340 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fc0800292a0 con 0x7fc08c106d80 2024-04-26T01:41:19.264 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:18.521+0000 7fc092e00700 1 -- 172.21.15.151:0/435286340 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7fc074039660 msgr2=0x7fc07403bb20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T01:41:19.265 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:18.521+0000 7fc092e00700 1 --2- 172.21.15.151:0/435286340 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7fc074039660 0x7fc07403bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:19.265 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:18.521+0000 7fc092e00700 1 -- 172.21.15.151:0/435286340 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc08c106d80 msgr2=0x7fc08c1a2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:41:19.265 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:18.521+0000 7fc092e00700 1 --2- 172.21.15.151:0/435286340 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc08c106d80 0x7fc08c1a2a90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc080000c00 tx=0x7fc080011aa0 comp rx=0 tx=0).stop 2024-04-26T01:41:19.265 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:18.521+0000 7fc092e00700 1 -- 172.21.15.151:0/435286340 shutdown_connections 2024-04-26T01:41:19.265 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:18.521+0000 7fc092e00700 1 --2- 172.21.15.151:0/435286340 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7fc074039660 0x7fc07403bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:19.265 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:18.521+0000 7fc092e00700 1 --2- 172.21.15.151:0/435286340 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc08c106d80 0x7fc08c1a2a90 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:19.265 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:18.521+0000 7fc092e00700 1 -- 172.21.15.151:0/435286340 >> 172.21.15.151:0/435286340 conn(0x7fc08c1029c0 msgr2=0x7fc08c104e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:41:19.265 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:18.521+0000 7fc092e00700 1 -- 172.21.15.151:0/435286340 shutdown_connections 2024-04-26T01:41:19.265 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:18.521+0000 7fc092e00700 1 -- 172.21.15.151:0/435286340 wait complete. 2024-04-26T01:41:19.265 INFO:teuthology.orchestra.run.smithi151.stdout:Waiting for the mgr to restart... 2024-04-26T01:41:19.265 INFO:teuthology.orchestra.run.smithi151.stdout:Waiting for mgr epoch 9... 2024-04-26T01:41:19.419 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:41:19 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:41:19.383+0000 7f683c6e4200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T01:41:20.039 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:41:19 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:41:19.800+0000 7f683c6e4200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T01:41:20.039 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:41:19 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:41:19.925+0000 7f683c6e4200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T01:41:20.370 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:41:20 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:41:20.183+0000 7f683c6e4200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T01:41:20.870 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:41:20 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:41:20.437+0000 7f683c6e4200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T01:41:21.620 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:41:21 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:41:21.223+0000 7f683c6e4200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T01:41:22.120 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:41:21 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:41:21.773+0000 7f683c6e4200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T01:41:22.120 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:41:21 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:41:21.901+0000 7f683c6e4200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T01:41:22.620 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:41:22 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:41:22.169+0000 7f683c6e4200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T01:41:23.120 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:41:22 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:41:22.819+0000 7f683c6e4200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T01:41:23.120 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:41:22 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:41:22.945+0000 7f683c6e4200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T01:41:23.461 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:41:23 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:41:23.275+0000 7f683c6e4200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T01:41:23.759 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:41:23 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:41:23.463+0000 7f683c6e4200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T01:41:23.759 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:41:23 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:41:23.599+0000 7f683c6e4200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T01:41:23.759 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:41:23 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:41:23.761+0000 7f683c6e4200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T01:41:24.620 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:41:24 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:41:24.154+0000 7f683c6e4200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T01:41:25.120 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:41:24 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:41:24.942+0000 7f683c6e4200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T01:41:25.620 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:41:25 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:41:25.230+0000 7f683c6e4200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T01:41:25.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:25 smithi151 ceph-mon[28057]: Active manager daemon x restarted 2024-04-26T01:41:25.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:25 smithi151 ceph-mon[28057]: Activating manager daemon x 2024-04-26T01:41:25.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:25 smithi151 ceph-mon[28057]: osdmap e3: 0 total, 0 up, 0 in 2024-04-26T01:41:25.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:25 smithi151 ceph-mon[28057]: mgrmap e10: x(active, starting, since 0.00827076s) 2024-04-26T01:41:25.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:25 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T01:41:25.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:25 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T01:41:25.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:25 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T01:41:25.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:25 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T01:41:25.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:25 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T01:41:25.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:25 smithi151 ceph-mon[28057]: Manager daemon x is now available 2024-04-26T01:41:26.603 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:26 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:41:26.603 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:26 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T01:41:26.604 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:26 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T01:41:26.604 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:26 smithi151 ceph-mon[28057]: [26/Apr/2024:01:41:25] ENGINE Bus STARTING 2024-04-26T01:41:26.604 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:26 smithi151 ceph-mon[28057]: [26/Apr/2024:01:41:25] ENGINE Serving on http://172.21.15.151:8765 2024-04-26T01:41:26.604 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:26 smithi151 ceph-mon[28057]: [26/Apr/2024:01:41:25] ENGINE Serving on https://172.21.15.151:7150 2024-04-26T01:41:26.604 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:26 smithi151 ceph-mon[28057]: [26/Apr/2024:01:41:25] ENGINE Bus STARTED 2024-04-26T01:41:26.604 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:26 smithi151 ceph-mon[28057]: mgrmap e11: x(active, since 1.01098s) 2024-04-26T01:41:26.604 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:26 smithi151 ceph-mon[28057]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-26T01:41:26.604 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:26 smithi151 ceph-mon[28057]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-26T01:41:26.883 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout { 2024-04-26T01:41:26.883 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-26T01:41:26.883 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-26T01:41:26.884 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout } 2024-04-26T01:41:26.884 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:19.831+0000 7feb12e00700 1 Processor -- start 2024-04-26T01:41:26.884 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:19.832+0000 7feb12e00700 1 -- start start 2024-04-26T01:41:26.884 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:19.832+0000 7feb12e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb0c07dda0 0x7feb0c07c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:41:26.884 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:19.832+0000 7feb12e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7feb0c07c7d0 con 0x7feb0c07dda0 2024-04-26T01:41:26.884 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:19.833+0000 7feb0be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb0c07dda0 0x7feb0c07c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:41:26.884 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:19.833+0000 7feb0be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb0c07dda0 0x7feb0c07c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:56732/0 (socket says 172.21.15.151:56732) 2024-04-26T01:41:26.884 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:19.833+0000 7feb0be00700 1 -- 172.21.15.151:0/2660836254 learned_addr learned my addr 172.21.15.151:0/2660836254 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:41:26.884 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:19.833+0000 7feb0be00700 1 -- 172.21.15.151:0/2660836254 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb0c07c950 con 0x7feb0c07dda0 2024-04-26T01:41:26.884 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:19.833+0000 7feb0be00700 1 --2- 172.21.15.151:0/2660836254 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb0c07dda0 0x7feb0c07c200 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7feafc0098a0 tx=0x7feafc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=672ac25c33d2f4a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:41:26.884 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:19.834+0000 7feb0aa00700 1 -- 172.21.15.151:0/2660836254 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feafc004160 con 0x7feb0c07dda0 2024-04-26T01:41:26.884 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:19.834+0000 7feb0aa00700 1 -- 172.21.15.151:0/2660836254 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feafc004300 con 0x7feb0c07dda0 2024-04-26T01:41:26.885 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:19.834+0000 7feb0aa00700 1 -- 172.21.15.151:0/2660836254 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feafc011a20 con 0x7feb0c07dda0 2024-04-26T01:41:26.885 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:19.834+0000 7feb12e00700 1 -- 172.21.15.151:0/2660836254 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb0c07dda0 msgr2=0x7feb0c07c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:41:26.885 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:19.834+0000 7feb12e00700 1 --2- 172.21.15.151:0/2660836254 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb0c07dda0 0x7feb0c07c200 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7feafc0098a0 tx=0x7feafc009bb0 comp rx=0 tx=0).stop 2024-04-26T01:41:26.885 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:19.835+0000 7feb12e00700 1 -- 172.21.15.151:0/2660836254 shutdown_connections 2024-04-26T01:41:26.885 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:19.835+0000 7feb12e00700 1 --2- 172.21.15.151:0/2660836254 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb0c07dda0 0x7feb0c07c200 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:26.885 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:19.835+0000 7feb12e00700 1 -- 172.21.15.151:0/2660836254 >> 172.21.15.151:0/2660836254 conn(0x7feb0c103c50 msgr2=0x7feb0c106070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:41:26.885 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:19.835+0000 7feb12e00700 1 -- 172.21.15.151:0/2660836254 shutdown_connections 2024-04-26T01:41:26.885 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:19.835+0000 7feb12e00700 1 -- 172.21.15.151:0/2660836254 wait complete. 2024-04-26T01:41:26.885 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:19.836+0000 7feb12e00700 1 Processor -- start 2024-04-26T01:41:26.885 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:19.836+0000 7feb12e00700 1 -- start start 2024-04-26T01:41:26.885 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:19.836+0000 7feb12e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb0c07dda0 0x7feb0c19e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:41:26.885 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:19.836+0000 7feb12e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7feb0c10ec90 con 0x7feb0c07dda0 2024-04-26T01:41:26.885 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:19.836+0000 7feb0be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb0c07dda0 0x7feb0c19e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:41:26.886 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:19.836+0000 7feb0be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb0c07dda0 0x7feb0c19e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:56734/0 (socket says 172.21.15.151:56734) 2024-04-26T01:41:26.886 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:19.836+0000 7feb0be00700 1 -- 172.21.15.151:0/3747529274 learned_addr learned my addr 172.21.15.151:0/3747529274 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:41:26.886 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:19.837+0000 7feb0be00700 1 -- 172.21.15.151:0/3747529274 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb0c19ecd0 con 0x7feb0c07dda0 2024-04-26T01:41:26.886 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:19.837+0000 7feb0be00700 1 --2- 172.21.15.151:0/3747529274 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb0c07dda0 0x7feb0c19e790 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7feafc011770 tx=0x7feafc011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:41:26.886 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:19.837+0000 7feb08c00700 1 -- 172.21.15.151:0/3747529274 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feafc0039e0 con 0x7feb0c07dda0 2024-04-26T01:41:26.886 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:19.837+0000 7feb12e00700 1 -- 172.21.15.151:0/3747529274 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb0c19ef60 con 0x7feb0c07dda0 2024-04-26T01:41:26.886 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:19.837+0000 7feb08c00700 1 -- 172.21.15.151:0/3747529274 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feafc01b8f0 con 0x7feb0c07dda0 2024-04-26T01:41:26.886 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:19.837+0000 7feb08c00700 1 -- 172.21.15.151:0/3747529274 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feafc023760 con 0x7feb0c07dda0 2024-04-26T01:41:26.886 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:19.838+0000 7feb12e00700 1 -- 172.21.15.151:0/3747529274 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb0c1a1c50 con 0x7feb0c07dda0 2024-04-26T01:41:26.886 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:19.839+0000 7feb08c00700 1 -- 172.21.15.151:0/3747529274 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7feafc01b440 con 0x7feb0c07dda0 2024-04-26T01:41:26.886 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:19.840+0000 7feb08c00700 1 --2- 172.21.15.151:0/3747529274 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7feaf00395b0 0x7feaf003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:41:26.886 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:19.840+0000 7feb08c00700 1 -- 172.21.15.151:0/3747529274 --> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7feaf003c180 con 0x7feaf00395b0 2024-04-26T01:41:26.886 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:19.840+0000 7feb08c00700 1 -- 172.21.15.151:0/3747529274 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7feafc04e010 con 0x7feb0c07dda0 2024-04-26T01:41:26.887 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:19.840+0000 7feb0b400700 1 -- 172.21.15.151:0/3747529274 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7feaf00395b0 msgr2=0x7feaf003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.151:6800/796314699 2024-04-26T01:41:26.887 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:19.840+0000 7feb0b400700 1 --2- 172.21.15.151:0/3747529274 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7feaf00395b0 0x7feaf003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T01:41:26.887 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:20.041+0000 7feb0b400700 1 -- 172.21.15.151:0/3747529274 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7feaf00395b0 msgr2=0x7feaf003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.151:6800/796314699 2024-04-26T01:41:26.887 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:20.041+0000 7feb0b400700 1 --2- 172.21.15.151:0/3747529274 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7feaf00395b0 0x7feaf003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T01:41:26.887 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:20.442+0000 7feb0b400700 1 -- 172.21.15.151:0/3747529274 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7feaf00395b0 msgr2=0x7feaf003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.151:6800/796314699 2024-04-26T01:41:26.887 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:20.442+0000 7feb0b400700 1 --2- 172.21.15.151:0/3747529274 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7feaf00395b0 0x7feaf003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-26T01:41:26.887 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:21.243+0000 7feb0b400700 1 -- 172.21.15.151:0/3747529274 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7feaf00395b0 msgr2=0x7feaf003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.151:6800/796314699 2024-04-26T01:41:26.887 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:21.243+0000 7feb0b400700 1 --2- 172.21.15.151:0/3747529274 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7feaf00395b0 0x7feaf003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-26T01:41:26.887 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:22.845+0000 7feb0b400700 1 -- 172.21.15.151:0/3747529274 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7feaf00395b0 msgr2=0x7feaf003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.151:6800/796314699 2024-04-26T01:41:26.887 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:22.845+0000 7feb0b400700 1 --2- 172.21.15.151:0/3747529274 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7feaf00395b0 0x7feaf003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-26T01:41:26.887 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:25.240+0000 7feb08c00700 1 -- 172.21.15.151:0/3747529274 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7feafc0239d0 con 0x7feb0c07dda0 2024-04-26T01:41:26.887 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:25.240+0000 7feb08c00700 1 -- 172.21.15.151:0/3747529274 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7feaf00395b0 msgr2=0x7feaf003ba70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T01:41:26.887 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:25.240+0000 7feb08c00700 1 --2- 172.21.15.151:0/3747529274 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7feaf00395b0 0x7feaf003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:26.888 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:26.242+0000 7feb08c00700 1 -- 172.21.15.151:0/3747529274 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7feafc050270 con 0x7feb0c07dda0 2024-04-26T01:41:26.888 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:26.242+0000 7feb08c00700 1 --2- 172.21.15.151:0/3747529274 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7feaf003cfe0 0x7feaf003f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:41:26.888 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:26.242+0000 7feb08c00700 1 -- 172.21.15.151:0/3747529274 --> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7feaf003c180 con 0x7feaf003cfe0 2024-04-26T01:41:26.888 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:26.243+0000 7feb0b400700 1 --2- 172.21.15.151:0/3747529274 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7feaf003cfe0 0x7feaf003f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:41:26.888 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:26.245+0000 7feb0b400700 1 --2- 172.21.15.151:0/3747529274 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7feaf003cfe0 0x7feaf003f3d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7feaf8003a10 tx=0x7feaf80092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:41:26.888 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:26.245+0000 7feb08c00700 1 -- 172.21.15.151:0/3747529274 <== mgr.14152 v2:172.21.15.151:6800/2251256505 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7feaf003c180 con 0x7feaf003cfe0 2024-04-26T01:41:26.888 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:26.253+0000 7feb12e00700 1 -- 172.21.15.151:0/3747529274 --> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7feb0c0640a0 con 0x7feaf003cfe0 2024-04-26T01:41:26.888 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:26.254+0000 7feb08c00700 1 -- 172.21.15.151:0/3747529274 <== mgr.14152 v2:172.21.15.151:6800/2251256505 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7feb0c0640a0 con 0x7feaf003cfe0 2024-04-26T01:41:26.888 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:26.254+0000 7feb12e00700 1 -- 172.21.15.151:0/3747529274 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7feaf003cfe0 msgr2=0x7feaf003f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:41:26.888 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:26.254+0000 7feb12e00700 1 --2- 172.21.15.151:0/3747529274 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7feaf003cfe0 0x7feaf003f3d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7feaf8003a10 tx=0x7feaf80092b0 comp rx=0 tx=0).stop 2024-04-26T01:41:26.888 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:26.254+0000 7feb12e00700 1 -- 172.21.15.151:0/3747529274 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb0c07dda0 msgr2=0x7feb0c19e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:41:26.888 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:26.254+0000 7feb12e00700 1 --2- 172.21.15.151:0/3747529274 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb0c07dda0 0x7feb0c19e790 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7feafc011770 tx=0x7feafc011aa0 comp rx=0 tx=0).stop 2024-04-26T01:41:26.888 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:26.254+0000 7feb12e00700 1 -- 172.21.15.151:0/3747529274 shutdown_connections 2024-04-26T01:41:26.889 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:26.254+0000 7feb12e00700 1 --2- 172.21.15.151:0/3747529274 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7feaf003cfe0 0x7feaf003f3d0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:26.889 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:26.254+0000 7feb12e00700 1 --2- 172.21.15.151:0/3747529274 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb0c07dda0 0x7feb0c19e790 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:26.889 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:26.255+0000 7feb12e00700 1 --2- 172.21.15.151:0/3747529274 >> [v2:172.21.15.151:6800/796314699,v1:172.21.15.151:6801/796314699] conn(0x7feaf00395b0 0x7feaf003ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:26.889 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:26.255+0000 7feb12e00700 1 -- 172.21.15.151:0/3747529274 >> 172.21.15.151:0/3747529274 conn(0x7feb0c103c50 msgr2=0x7feb0c104800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:41:26.889 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:26.255+0000 7feb12e00700 1 -- 172.21.15.151:0/3747529274 shutdown_connections 2024-04-26T01:41:26.889 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:26.255+0000 7feb12e00700 1 -- 172.21.15.151:0/3747529274 wait complete. 2024-04-26T01:41:26.889 INFO:teuthology.orchestra.run.smithi151.stdout:mgr epoch 9 is available 2024-04-26T01:41:26.889 INFO:teuthology.orchestra.run.smithi151.stdout:Generating a dashboard self-signed certificate... 2024-04-26T01:41:28.370 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:28 smithi151 ceph-mon[28057]: mgrmap e12: x(active, since 2s) 2024-04-26T01:41:28.370 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:28 smithi151 ceph-mon[28057]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:41:28.370 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:28 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:41:28.370 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:28 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:41:28.370 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:28 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:41:29.054 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-26T01:41:29.055 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:27.864+0000 7f60abe00700 1 Processor -- start 2024-04-26T01:41:29.056 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:27.864+0000 7f60abe00700 1 -- start start 2024-04-26T01:41:29.056 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:27.864+0000 7f60abe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60a4109200 0x7f60a410b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:41:29.056 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:27.864+0000 7f60abe00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f60a407ce20 con 0x7f60a4109200 2024-04-26T01:41:29.056 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:27.865+0000 7f60a9400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60a4109200 0x7f60a410b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:41:29.057 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:27.865+0000 7f60a9400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60a4109200 0x7f60a410b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:56796/0 (socket says 172.21.15.151:56796) 2024-04-26T01:41:29.057 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:27.865+0000 7f60a9400700 1 -- 172.21.15.151:0/2000161454 learned_addr learned my addr 172.21.15.151:0/2000161454 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:41:29.057 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:27.865+0000 7f60a9400700 1 -- 172.21.15.151:0/2000161454 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60a410bb30 con 0x7f60a4109200 2024-04-26T01:41:29.057 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:27.866+0000 7f60a9400700 1 --2- 172.21.15.151:0/2000161454 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60a4109200 0x7f60a410b5f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f60980098a0 tx=0x7f6098009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68988acf4b21c05e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:41:29.057 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:27.866+0000 7f60a3e00700 1 -- 172.21.15.151:0/2000161454 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6098004160 con 0x7f60a4109200 2024-04-26T01:41:29.057 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:27.866+0000 7f60a3e00700 1 -- 172.21.15.151:0/2000161454 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6098004300 con 0x7f60a4109200 2024-04-26T01:41:29.057 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:27.866+0000 7f60a3e00700 1 -- 172.21.15.151:0/2000161454 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6098011a20 con 0x7f60a4109200 2024-04-26T01:41:29.057 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:27.867+0000 7f60abe00700 1 -- 172.21.15.151:0/2000161454 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60a4109200 msgr2=0x7f60a410b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:41:29.060 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:27.867+0000 7f60abe00700 1 --2- 172.21.15.151:0/2000161454 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60a4109200 0x7f60a410b5f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f60980098a0 tx=0x7f6098009bb0 comp rx=0 tx=0).stop 2024-04-26T01:41:29.060 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:27.867+0000 7f60abe00700 1 -- 172.21.15.151:0/2000161454 shutdown_connections 2024-04-26T01:41:29.060 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:27.867+0000 7f60abe00700 1 --2- 172.21.15.151:0/2000161454 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60a4109200 0x7f60a410b5f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:29.060 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:27.867+0000 7f60abe00700 1 -- 172.21.15.151:0/2000161454 >> 172.21.15.151:0/2000161454 conn(0x7f60a4102bb0 msgr2=0x7f60a4104fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:41:29.060 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:27.867+0000 7f60abe00700 1 -- 172.21.15.151:0/2000161454 shutdown_connections 2024-04-26T01:41:29.060 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:27.867+0000 7f60abe00700 1 -- 172.21.15.151:0/2000161454 wait complete. 2024-04-26T01:41:29.061 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:27.868+0000 7f60abe00700 1 Processor -- start 2024-04-26T01:41:29.061 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:27.868+0000 7f60abe00700 1 -- start start 2024-04-26T01:41:29.061 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:27.869+0000 7f60abe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60a4109200 0x7f60a4108a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:41:29.061 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:27.869+0000 7f60abe00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f60a410cad0 con 0x7f60a4109200 2024-04-26T01:41:29.061 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:27.869+0000 7f60a9400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60a4109200 0x7f60a4108a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:41:29.061 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:27.869+0000 7f60a9400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60a4109200 0x7f60a4108a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:56806/0 (socket says 172.21.15.151:56806) 2024-04-26T01:41:29.061 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:27.869+0000 7f60a9400700 1 -- 172.21.15.151:0/1342048786 learned_addr learned my addr 172.21.15.151:0/1342048786 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:41:29.061 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:27.869+0000 7f60a9400700 1 -- 172.21.15.151:0/1342048786 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60a4108fc0 con 0x7f60a4109200 2024-04-26T01:41:29.061 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:27.870+0000 7f60a9400700 1 --2- 172.21.15.151:0/1342048786 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60a4109200 0x7f60a4108a80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6098000c00 tx=0x7f6098011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:41:29.061 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:27.870+0000 7f60a2000700 1 -- 172.21.15.151:0/1342048786 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6098003aa0 con 0x7f60a4109200 2024-04-26T01:41:29.061 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:27.870+0000 7f60a2000700 1 -- 172.21.15.151:0/1342048786 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f609801b8f0 con 0x7f60a4109200 2024-04-26T01:41:29.061 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:27.870+0000 7f60abe00700 1 -- 172.21.15.151:0/1342048786 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60a41071b0 con 0x7f60a4109200 2024-04-26T01:41:29.061 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:27.870+0000 7f60a2000700 1 -- 172.21.15.151:0/1342048786 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6098023870 con 0x7f60a4109200 2024-04-26T01:41:29.062 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:27.870+0000 7f60abe00700 1 -- 172.21.15.151:0/1342048786 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60a41076f0 con 0x7f60a4109200 2024-04-26T01:41:29.062 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:27.871+0000 7f60a2000700 1 -- 172.21.15.151:0/1342048786 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f609801b440 con 0x7f60a4109200 2024-04-26T01:41:29.062 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:27.871+0000 7f6097400700 1 -- 172.21.15.151:0/1342048786 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6080005330 con 0x7f60a4109200 2024-04-26T01:41:29.064 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:27.872+0000 7f60a2000700 1 --2- 172.21.15.151:0/1342048786 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f608c039660 0x7f608c03bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:41:29.064 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:27.872+0000 7f60a2000700 1 -- 172.21.15.151:0/1342048786 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f609804e090 con 0x7f60a4109200 2024-04-26T01:41:29.064 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:27.872+0000 7f60a8a00700 1 --2- 172.21.15.151:0/1342048786 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f608c039660 0x7f608c03bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:41:29.064 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:27.873+0000 7f60a8a00700 1 --2- 172.21.15.151:0/1342048786 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f608c039660 0x7f608c03bb20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6090006fd0 tx=0x7f6090006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:41:29.064 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:27.878+0000 7f60a2000700 1 -- 172.21.15.151:0/1342048786 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f609801f030 con 0x7f60a4109200 2024-04-26T01:41:29.064 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:28.116+0000 7f6097400700 1 -- 172.21.15.151:0/1342048786 --> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f6080000bc0 con 0x7f608c039660 2024-04-26T01:41:29.064 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:28.270+0000 7f60a2000700 1 -- 172.21.15.151:0/1342048786 <== mgr.14152 v2:172.21.15.151:6800/2251256505 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f6080000bc0 con 0x7f608c039660 2024-04-26T01:41:29.064 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:28.273+0000 7f6097400700 1 -- 172.21.15.151:0/1342048786 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f608c039660 msgr2=0x7f608c03bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:41:29.064 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:28.273+0000 7f6097400700 1 --2- 172.21.15.151:0/1342048786 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f608c039660 0x7f608c03bb20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6090006fd0 tx=0x7f6090006e40 comp rx=0 tx=0).stop 2024-04-26T01:41:29.064 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:28.273+0000 7f6097400700 1 -- 172.21.15.151:0/1342048786 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60a4109200 msgr2=0x7f60a4108a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:41:29.064 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:28.273+0000 7f6097400700 1 --2- 172.21.15.151:0/1342048786 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60a4109200 0x7f60a4108a80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6098000c00 tx=0x7f6098011aa0 comp rx=0 tx=0).stop 2024-04-26T01:41:29.064 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:28.273+0000 7f6097400700 1 -- 172.21.15.151:0/1342048786 shutdown_connections 2024-04-26T01:41:29.065 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:28.273+0000 7f6097400700 1 --2- 172.21.15.151:0/1342048786 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f608c039660 0x7f608c03bb20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:29.065 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:28.273+0000 7f6097400700 1 --2- 172.21.15.151:0/1342048786 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60a4109200 0x7f60a4108a80 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:29.065 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:28.273+0000 7f6097400700 1 -- 172.21.15.151:0/1342048786 >> 172.21.15.151:0/1342048786 conn(0x7f60a4102bb0 msgr2=0x7f60a4103650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:41:29.065 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:28.273+0000 7f6097400700 1 -- 172.21.15.151:0/1342048786 shutdown_connections 2024-04-26T01:41:29.065 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:28.274+0000 7f6097400700 1 -- 172.21.15.151:0/1342048786 wait complete. 2024-04-26T01:41:29.065 INFO:teuthology.orchestra.run.smithi151.stdout:Creating initial admin user... 2024-04-26T01:41:30.990 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$q7vMWHd4JswINF0L3YA3b.MbUNmmMC3venWlddBnbyPHK.VS4hfwC", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1714095690, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-26T01:41:30.990 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:29.730+0000 7f76e5c00700 1 Processor -- start 2024-04-26T01:41:30.990 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:29.730+0000 7f76e5c00700 1 -- start start 2024-04-26T01:41:30.990 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:29.730+0000 7f76e5c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76e0109110 0x7f76e010b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:41:30.990 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:29.730+0000 7f76e5c00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f76e0063d60 con 0x7f76e0109110 2024-04-26T01:41:30.990 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:29.731+0000 7f76dea00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76e0109110 0x7f76e010b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:41:30.990 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:29.731+0000 7f76dea00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76e0109110 0x7f76e010b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:56822/0 (socket says 172.21.15.151:56822) 2024-04-26T01:41:30.990 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:29.731+0000 7f76dea00700 1 -- 172.21.15.151:0/1700796760 learned_addr learned my addr 172.21.15.151:0/1700796760 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:41:30.990 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:29.731+0000 7f76dea00700 1 -- 172.21.15.151:0/1700796760 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76e010ba40 con 0x7f76e0109110 2024-04-26T01:41:30.990 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:29.732+0000 7f76dea00700 1 --2- 172.21.15.151:0/1700796760 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76e0109110 0x7f76e010b500 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f76d00098a0 tx=0x7f76d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=681e8a3e399362f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:41:30.990 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:29.732+0000 7f76dd600700 1 -- 172.21.15.151:0/1700796760 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76d0004160 con 0x7f76e0109110 2024-04-26T01:41:30.991 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:29.732+0000 7f76dd600700 1 -- 172.21.15.151:0/1700796760 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76d0004300 con 0x7f76e0109110 2024-04-26T01:41:30.991 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:29.732+0000 7f76dd600700 1 -- 172.21.15.151:0/1700796760 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76d0011a20 con 0x7f76e0109110 2024-04-26T01:41:30.991 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:29.733+0000 7f76e5c00700 1 -- 172.21.15.151:0/1700796760 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76e0109110 msgr2=0x7f76e010b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:41:30.991 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:29.733+0000 7f76e5c00700 1 --2- 172.21.15.151:0/1700796760 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76e0109110 0x7f76e010b500 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f76d00098a0 tx=0x7f76d0009bb0 comp rx=0 tx=0).stop 2024-04-26T01:41:30.991 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:29.733+0000 7f76e5c00700 1 -- 172.21.15.151:0/1700796760 shutdown_connections 2024-04-26T01:41:30.991 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:29.733+0000 7f76e5c00700 1 --2- 172.21.15.151:0/1700796760 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76e0109110 0x7f76e010b500 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:30.991 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:29.733+0000 7f76e5c00700 1 -- 172.21.15.151:0/1700796760 >> 172.21.15.151:0/1700796760 conn(0x7f76e0102ba0 msgr2=0x7f76e0104fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:41:30.991 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:29.733+0000 7f76e5c00700 1 -- 172.21.15.151:0/1700796760 shutdown_connections 2024-04-26T01:41:30.991 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:29.733+0000 7f76e5c00700 1 -- 172.21.15.151:0/1700796760 wait complete. 2024-04-26T01:41:30.991 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:29.734+0000 7f76e5c00700 1 Processor -- start 2024-04-26T01:41:30.991 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:29.735+0000 7f76e5c00700 1 -- start start 2024-04-26T01:41:30.991 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:29.735+0000 7f76e5c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76e0109110 0x7f76e019c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:41:30.991 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:29.735+0000 7f76e5c00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f76e010ca50 con 0x7f76e0109110 2024-04-26T01:41:30.992 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:29.735+0000 7f76dea00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76e0109110 0x7f76e019c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:41:30.992 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:29.735+0000 7f76dea00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76e0109110 0x7f76e019c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:56834/0 (socket says 172.21.15.151:56834) 2024-04-26T01:41:30.992 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:29.735+0000 7f76dea00700 1 -- 172.21.15.151:0/3352764512 learned_addr learned my addr 172.21.15.151:0/3352764512 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:41:30.992 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:29.736+0000 7f76dea00700 1 -- 172.21.15.151:0/3352764512 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76e019c9d0 con 0x7f76e0109110 2024-04-26T01:41:30.992 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:29.736+0000 7f76dea00700 1 --2- 172.21.15.151:0/3352764512 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76e0109110 0x7f76e019c490 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f76d0004660 tx=0x7f76d0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:41:30.992 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:29.736+0000 7f76d7400700 1 -- 172.21.15.151:0/3352764512 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76d0003aa0 con 0x7f76e0109110 2024-04-26T01:41:30.992 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:29.736+0000 7f76d7400700 1 -- 172.21.15.151:0/3352764512 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76d001b8f0 con 0x7f76e0109110 2024-04-26T01:41:30.992 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:29.736+0000 7f76d7400700 1 -- 172.21.15.151:0/3352764512 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76d0023900 con 0x7f76e0109110 2024-04-26T01:41:30.992 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:29.736+0000 7f76e5c00700 1 -- 172.21.15.151:0/3352764512 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76e019cc60 con 0x7f76e0109110 2024-04-26T01:41:30.992 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:29.737+0000 7f76e5c00700 1 -- 172.21.15.151:0/3352764512 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76e019f950 con 0x7f76e0109110 2024-04-26T01:41:30.992 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:29.738+0000 7f76e5c00700 1 -- 172.21.15.151:0/3352764512 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76e00640a0 con 0x7f76e0109110 2024-04-26T01:41:30.992 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:29.738+0000 7f76d7400700 1 -- 172.21.15.151:0/3352764512 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f76d001baa0 con 0x7f76e0109110 2024-04-26T01:41:30.992 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:29.738+0000 7f76d7400700 1 --2- 172.21.15.151:0/3352764512 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f76c4039660 0x7f76c403bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:41:30.992 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:29.739+0000 7f76d7400700 1 -- 172.21.15.151:0/3352764512 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f76d004dfb0 con 0x7f76e0109110 2024-04-26T01:41:30.993 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:29.739+0000 7f76de000700 1 --2- 172.21.15.151:0/3352764512 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f76c4039660 0x7f76c403bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:41:30.993 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:29.740+0000 7f76de000700 1 --2- 172.21.15.151:0/3352764512 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f76c4039660 0x7f76c403bb20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f76cc006fd0 tx=0x7f76cc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:41:30.993 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:29.745+0000 7f76d7400700 1 -- 172.21.15.151:0/3352764512 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f76d001f030 con 0x7f76e0109110 2024-04-26T01:41:30.993 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:29.988+0000 7f76e5c00700 1 -- 172.21.15.151:0/3352764512 --> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f76e019fbe0 con 0x7f76c4039660 2024-04-26T01:41:30.993 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:30.226+0000 7f76d7400700 1 -- 172.21.15.151:0/3352764512 <== mgr.14152 v2:172.21.15.151:6800/2251256505 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f76e019fbe0 con 0x7f76c4039660 2024-04-26T01:41:30.993 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:30.229+0000 7f76e5c00700 1 -- 172.21.15.151:0/3352764512 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f76c4039660 msgr2=0x7f76c403bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:41:30.993 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:30.229+0000 7f76e5c00700 1 --2- 172.21.15.151:0/3352764512 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f76c4039660 0x7f76c403bb20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f76cc006fd0 tx=0x7f76cc006e40 comp rx=0 tx=0).stop 2024-04-26T01:41:30.993 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:30.229+0000 7f76e5c00700 1 -- 172.21.15.151:0/3352764512 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76e0109110 msgr2=0x7f76e019c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:41:30.993 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:30.229+0000 7f76e5c00700 1 --2- 172.21.15.151:0/3352764512 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76e0109110 0x7f76e019c490 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f76d0004660 tx=0x7f76d0011aa0 comp rx=0 tx=0).stop 2024-04-26T01:41:30.993 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:30.229+0000 7f76e5c00700 1 -- 172.21.15.151:0/3352764512 shutdown_connections 2024-04-26T01:41:30.993 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:30.229+0000 7f76e5c00700 1 --2- 172.21.15.151:0/3352764512 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f76c4039660 0x7f76c403bb20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:30.993 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:30.229+0000 7f76e5c00700 1 --2- 172.21.15.151:0/3352764512 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76e0109110 0x7f76e019c490 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:30.993 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:30.229+0000 7f76e5c00700 1 -- 172.21.15.151:0/3352764512 >> 172.21.15.151:0/3352764512 conn(0x7f76e0102ba0 msgr2=0x7f76e01035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:41:30.993 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:30.229+0000 7f76e5c00700 1 -- 172.21.15.151:0/3352764512 shutdown_connections 2024-04-26T01:41:30.994 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:30.229+0000 7f76e5c00700 1 -- 172.21.15.151:0/3352764512 wait complete. 2024-04-26T01:41:30.994 INFO:teuthology.orchestra.run.smithi151.stdout:Fetching dashboard port number... 2024-04-26T01:41:31.301 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:31 smithi151 ceph-mon[28057]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:41:31.301 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:31 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:41:31.302 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:31 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:41:32.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:32 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/1374963013' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-26T01:41:33.029 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout 8443 2024-04-26T01:41:33.029 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:31.605+0000 7f92cb600700 1 Processor -- start 2024-04-26T01:41:33.029 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:31.605+0000 7f92cb600700 1 -- start start 2024-04-26T01:41:33.029 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:31.605+0000 7f92cb600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f92c41091d0 0x7f92c410b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:41:33.029 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:31.605+0000 7f92cb600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f92c407ce40 con 0x7f92c41091d0 2024-04-26T01:41:33.029 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:31.606+0000 7f92c8c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f92c41091d0 0x7f92c410b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:41:33.030 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:31.606+0000 7f92c8c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f92c41091d0 0x7f92c410b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36544/0 (socket says 172.21.15.151:36544) 2024-04-26T01:41:33.030 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:31.606+0000 7f92c8c00700 1 -- 172.21.15.151:0/2325972620 learned_addr learned my addr 172.21.15.151:0/2325972620 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:41:33.030 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:31.606+0000 7f92c8c00700 1 -- 172.21.15.151:0/2325972620 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92c410bb00 con 0x7f92c41091d0 2024-04-26T01:41:33.030 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:31.607+0000 7f92c8c00700 1 --2- 172.21.15.151:0/2325972620 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f92c41091d0 0x7f92c410b5c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f92b80098a0 tx=0x7f92b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a92c41eb8c894b4b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:41:33.030 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:31.607+0000 7f92c3400700 1 -- 172.21.15.151:0/2325972620 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92b8004160 con 0x7f92c41091d0 2024-04-26T01:41:33.030 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:31.607+0000 7f92c3400700 1 -- 172.21.15.151:0/2325972620 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92b8004300 con 0x7f92c41091d0 2024-04-26T01:41:33.030 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:31.607+0000 7f92c3400700 1 -- 172.21.15.151:0/2325972620 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92b8011a20 con 0x7f92c41091d0 2024-04-26T01:41:33.030 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:31.608+0000 7f92cb600700 1 -- 172.21.15.151:0/2325972620 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f92c41091d0 msgr2=0x7f92c410b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:41:33.030 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:31.608+0000 7f92cb600700 1 --2- 172.21.15.151:0/2325972620 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f92c41091d0 0x7f92c410b5c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f92b80098a0 tx=0x7f92b8009bb0 comp rx=0 tx=0).stop 2024-04-26T01:41:33.030 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:31.608+0000 7f92cb600700 1 -- 172.21.15.151:0/2325972620 shutdown_connections 2024-04-26T01:41:33.030 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:31.608+0000 7f92cb600700 1 --2- 172.21.15.151:0/2325972620 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f92c41091d0 0x7f92c410b5c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:33.030 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:31.608+0000 7f92cb600700 1 -- 172.21.15.151:0/2325972620 >> 172.21.15.151:0/2325972620 conn(0x7f92c4102be0 msgr2=0x7f92c4105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:41:33.030 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:31.608+0000 7f92cb600700 1 -- 172.21.15.151:0/2325972620 shutdown_connections 2024-04-26T01:41:33.031 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:31.608+0000 7f92cb600700 1 -- 172.21.15.151:0/2325972620 wait complete. 2024-04-26T01:41:33.031 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:31.609+0000 7f92cb600700 1 Processor -- start 2024-04-26T01:41:33.031 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:31.610+0000 7f92cb600700 1 -- start start 2024-04-26T01:41:33.031 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:31.610+0000 7f92cb600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f92c41091d0 0x7f92c419c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:41:33.031 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:31.610+0000 7f92cb600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f92c410caa0 con 0x7f92c41091d0 2024-04-26T01:41:33.031 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:31.610+0000 7f92c8c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f92c41091d0 0x7f92c419c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:41:33.031 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:31.610+0000 7f92c8c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f92c41091d0 0x7f92c419c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36554/0 (socket says 172.21.15.151:36554) 2024-04-26T01:41:33.031 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:31.610+0000 7f92c8c00700 1 -- 172.21.15.151:0/1374963013 learned_addr learned my addr 172.21.15.151:0/1374963013 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:41:33.031 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:31.611+0000 7f92c8c00700 1 -- 172.21.15.151:0/1374963013 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92c419ca60 con 0x7f92c41091d0 2024-04-26T01:41:33.031 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:31.611+0000 7f92c8c00700 1 --2- 172.21.15.151:0/1374963013 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f92c41091d0 0x7f92c419c520 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f92b8000c00 tx=0x7f92b8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:41:33.031 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:31.611+0000 7f92c1600700 1 -- 172.21.15.151:0/1374963013 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92b8003aa0 con 0x7f92c41091d0 2024-04-26T01:41:33.031 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:31.611+0000 7f92c1600700 1 -- 172.21.15.151:0/1374963013 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92b801b8f0 con 0x7f92c41091d0 2024-04-26T01:41:33.031 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:31.611+0000 7f92c1600700 1 -- 172.21.15.151:0/1374963013 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92b8023870 con 0x7f92c41091d0 2024-04-26T01:41:33.032 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:31.611+0000 7f92cb600700 1 -- 172.21.15.151:0/1374963013 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92c419ccf0 con 0x7f92c41091d0 2024-04-26T01:41:33.032 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:31.611+0000 7f92cb600700 1 -- 172.21.15.151:0/1374963013 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92c419f9e0 con 0x7f92c41091d0 2024-04-26T01:41:33.032 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:31.613+0000 7f92c1600700 1 -- 172.21.15.151:0/1374963013 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f92b801b440 con 0x7f92c41091d0 2024-04-26T01:41:33.032 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:31.613+0000 7f92cb600700 1 -- 172.21.15.151:0/1374963013 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92c40640a0 con 0x7f92c41091d0 2024-04-26T01:41:33.032 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:31.613+0000 7f92c1600700 1 --2- 172.21.15.151:0/1374963013 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f92ac039660 0x7f92ac03bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:41:33.032 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:31.613+0000 7f92c1600700 1 -- 172.21.15.151:0/1374963013 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f92b804df80 con 0x7f92c41091d0 2024-04-26T01:41:33.032 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:31.614+0000 7f92c3e00700 1 --2- 172.21.15.151:0/1374963013 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f92ac039660 0x7f92ac03bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:41:33.032 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:31.615+0000 7f92c3e00700 1 --2- 172.21.15.151:0/1374963013 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f92ac039660 0x7f92ac03bb20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f92b0006fd0 tx=0x7f92b0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:41:33.032 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:31.619+0000 7f92c1600700 1 -- 172.21.15.151:0/1374963013 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f92b801f030 con 0x7f92c41091d0 2024-04-26T01:41:33.032 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:31.850+0000 7f92cb600700 1 -- 172.21.15.151:0/1374963013 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f92c419fbb0 con 0x7f92c41091d0 2024-04-26T01:41:33.032 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:31.851+0000 7f92c1600700 1 -- 172.21.15.151:0/1374963013 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f92b80202b0 con 0x7f92c41091d0 2024-04-26T01:41:33.032 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:31.855+0000 7f92cb600700 1 -- 172.21.15.151:0/1374963013 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f92ac039660 msgr2=0x7f92ac03bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:41:33.032 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:31.855+0000 7f92cb600700 1 --2- 172.21.15.151:0/1374963013 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f92ac039660 0x7f92ac03bb20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f92b0006fd0 tx=0x7f92b0006e40 comp rx=0 tx=0).stop 2024-04-26T01:41:33.033 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:31.855+0000 7f92cb600700 1 -- 172.21.15.151:0/1374963013 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f92c41091d0 msgr2=0x7f92c419c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:41:33.033 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:31.855+0000 7f92cb600700 1 --2- 172.21.15.151:0/1374963013 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f92c41091d0 0x7f92c419c520 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f92b8000c00 tx=0x7f92b8011aa0 comp rx=0 tx=0).stop 2024-04-26T01:41:33.033 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:31.855+0000 7f92cb600700 1 -- 172.21.15.151:0/1374963013 shutdown_connections 2024-04-26T01:41:33.033 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:31.855+0000 7f92cb600700 1 --2- 172.21.15.151:0/1374963013 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f92ac039660 0x7f92ac03bb20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:33.033 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:31.855+0000 7f92cb600700 1 --2- 172.21.15.151:0/1374963013 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f92c41091d0 0x7f92c419c520 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:33.033 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:31.855+0000 7f92cb600700 1 -- 172.21.15.151:0/1374963013 >> 172.21.15.151:0/1374963013 conn(0x7f92c4102be0 msgr2=0x7f92c41036d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:41:33.033 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:31.855+0000 7f92cb600700 1 -- 172.21.15.151:0/1374963013 shutdown_connections 2024-04-26T01:41:33.033 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:31.855+0000 7f92cb600700 1 -- 172.21.15.151:0/1374963013 wait complete. 2024-04-26T01:41:33.055 INFO:teuthology.orchestra.run.smithi151.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T01:41:33.055 INFO:teuthology.orchestra.run.smithi151.stdout:systemctl: stdout disabled 2024-04-26T01:41:33.078 INFO:teuthology.orchestra.run.smithi151.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T01:41:33.078 INFO:teuthology.orchestra.run.smithi151.stdout:systemctl: stdout inactive 2024-04-26T01:41:33.078 INFO:teuthology.orchestra.run.smithi151.stdout:firewalld.service is not enabled 2024-04-26T01:41:33.078 INFO:teuthology.orchestra.run.smithi151.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-26T01:41:33.082 INFO:teuthology.orchestra.run.smithi151.stdout:Ceph Dashboard is now available at: 2024-04-26T01:41:33.082 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:41:33.082 INFO:teuthology.orchestra.run.smithi151.stdout: URL: https://smithi151.front.sepia.ceph.com:8443/ 2024-04-26T01:41:33.083 INFO:teuthology.orchestra.run.smithi151.stdout: User: admin 2024-04-26T01:41:33.083 INFO:teuthology.orchestra.run.smithi151.stdout: Password: 1yo3jzn5fd 2024-04-26T01:41:33.083 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:41:33.084 INFO:teuthology.orchestra.run.smithi151.stdout:Saving cluster configuration to /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/config directory 2024-04-26T01:41:33.085 INFO:teuthology.orchestra.run.smithi151.stdout:Enabling autotune for osd_memory_target 2024-04-26T01:41:33.275 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:33 smithi151 ceph-mon[28057]: mgrmap e13: x(active, since 6s) 2024-04-26T01:41:35.047 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:34.128+0000 7f36a0200700 1 Processor -- start 2024-04-26T01:41:35.048 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:34.128+0000 7f36a0200700 1 -- start start 2024-04-26T01:41:35.048 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:34.129+0000 7f36a0200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f369810a920 0x7f369810ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:41:35.048 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:34.129+0000 7f36a0200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f369810b310 con 0x7f369810a920 2024-04-26T01:41:35.048 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:34.129+0000 7f369d800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f369810a920 0x7f369810ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:41:35.048 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:34.129+0000 7f369d800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f369810a920 0x7f369810ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36568/0 (socket says 172.21.15.151:36568) 2024-04-26T01:41:35.048 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:34.129+0000 7f369d800700 1 -- 172.21.15.151:0/3368654451 learned_addr learned my addr 172.21.15.151:0/3368654451 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:41:35.048 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:34.130+0000 7f369d800700 1 -- 172.21.15.151:0/3368654451 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f369810bb10 con 0x7f369810a920 2024-04-26T01:41:35.048 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:34.130+0000 7f369d800700 1 --2- 172.21.15.151:0/3368654451 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f369810a920 0x7f369810ad40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f368c0098a0 tx=0x7f368c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e0e1db16948d77a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:41:35.048 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:34.130+0000 7f3697e00700 1 -- 172.21.15.151:0/3368654451 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f368c004160 con 0x7f369810a920 2024-04-26T01:41:35.048 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:34.130+0000 7f3697e00700 1 -- 172.21.15.151:0/3368654451 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f368c004300 con 0x7f369810a920 2024-04-26T01:41:35.048 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:34.130+0000 7f3697e00700 1 -- 172.21.15.151:0/3368654451 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f368c011a20 con 0x7f369810a920 2024-04-26T01:41:35.048 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:34.131+0000 7f36a0200700 1 -- 172.21.15.151:0/3368654451 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f369810a920 msgr2=0x7f369810ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:41:35.049 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:34.131+0000 7f36a0200700 1 --2- 172.21.15.151:0/3368654451 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f369810a920 0x7f369810ad40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f368c0098a0 tx=0x7f368c009bb0 comp rx=0 tx=0).stop 2024-04-26T01:41:35.049 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:34.131+0000 7f36a0200700 1 -- 172.21.15.151:0/3368654451 shutdown_connections 2024-04-26T01:41:35.049 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:34.131+0000 7f36a0200700 1 --2- 172.21.15.151:0/3368654451 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f369810a920 0x7f369810ad40 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:35.049 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:34.131+0000 7f36a0200700 1 -- 172.21.15.151:0/3368654451 >> 172.21.15.151:0/3368654451 conn(0x7f369807dce0 msgr2=0x7f369807e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:41:35.049 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:34.131+0000 7f36a0200700 1 -- 172.21.15.151:0/3368654451 shutdown_connections 2024-04-26T01:41:35.049 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:34.131+0000 7f36a0200700 1 -- 172.21.15.151:0/3368654451 wait complete. 2024-04-26T01:41:35.049 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:34.132+0000 7f36a0200700 1 Processor -- start 2024-04-26T01:41:35.049 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:34.132+0000 7f36a0200700 1 -- start start 2024-04-26T01:41:35.049 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:34.133+0000 7f36a0200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f369810a920 0x7f369819e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:41:35.049 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:34.133+0000 7f36a0200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f369810eec0 con 0x7f369810a920 2024-04-26T01:41:35.049 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:34.133+0000 7f369d800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f369810a920 0x7f369819e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:41:35.049 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:34.133+0000 7f369d800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f369810a920 0x7f369819e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36578/0 (socket says 172.21.15.151:36578) 2024-04-26T01:41:35.049 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:34.133+0000 7f369d800700 1 -- 172.21.15.151:0/596905629 learned_addr learned my addr 172.21.15.151:0/596905629 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:41:35.050 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:34.133+0000 7f369d800700 1 -- 172.21.15.151:0/596905629 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f369819ec60 con 0x7f369810a920 2024-04-26T01:41:35.050 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:34.134+0000 7f369d800700 1 --2- 172.21.15.151:0/596905629 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f369810a920 0x7f369819e720 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f368c000c00 tx=0x7f368c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:41:35.050 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:34.134+0000 7f3696000700 1 -- 172.21.15.151:0/596905629 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f368c003aa0 con 0x7f369810a920 2024-04-26T01:41:35.050 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:34.134+0000 7f3696000700 1 -- 172.21.15.151:0/596905629 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f368c01b8f0 con 0x7f369810a920 2024-04-26T01:41:35.050 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:34.134+0000 7f36a0200700 1 -- 172.21.15.151:0/596905629 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f369819eef0 con 0x7f369810a920 2024-04-26T01:41:35.050 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:34.134+0000 7f3696000700 1 -- 172.21.15.151:0/596905629 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f368c023870 con 0x7f369810a920 2024-04-26T01:41:35.050 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:34.134+0000 7f36a0200700 1 -- 172.21.15.151:0/596905629 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36981a1be0 con 0x7f369810a920 2024-04-26T01:41:35.050 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:34.135+0000 7f3696000700 1 -- 172.21.15.151:0/596905629 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f368c023b70 con 0x7f369810a920 2024-04-26T01:41:35.050 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:34.135+0000 7f36a0200700 1 -- 172.21.15.151:0/596905629 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36980640a0 con 0x7f369810a920 2024-04-26T01:41:35.050 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:34.136+0000 7f3696000700 1 --2- 172.21.15.151:0/596905629 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f36800396a0 0x7f368003bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:41:35.050 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:34.136+0000 7f3696000700 1 -- 172.21.15.151:0/596905629 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f368c04e0b0 con 0x7f369810a920 2024-04-26T01:41:35.050 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:34.136+0000 7f369ce00700 1 --2- 172.21.15.151:0/596905629 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f36800396a0 0x7f368003bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:41:35.050 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:34.137+0000 7f369ce00700 1 --2- 172.21.15.151:0/596905629 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f36800396a0 0x7f368003bb60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3684006fd0 tx=0x7f3684006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:41:35.050 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:34.142+0000 7f3696000700 1 -- 172.21.15.151:0/596905629 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f368c01f030 con 0x7f369810a920 2024-04-26T01:41:35.051 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:34.365+0000 7f36a0200700 1 -- 172.21.15.151:0/596905629 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f36981a2240 con 0x7f369810a920 2024-04-26T01:41:35.051 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:34.366+0000 7f3696000700 1 -- 172.21.15.151:0/596905629 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f368c0292a0 con 0x7f369810a920 2024-04-26T01:41:35.051 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:34.369+0000 7f36a0200700 1 -- 172.21.15.151:0/596905629 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f36800396a0 msgr2=0x7f368003bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:41:35.051 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:34.369+0000 7f36a0200700 1 --2- 172.21.15.151:0/596905629 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f36800396a0 0x7f368003bb60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3684006fd0 tx=0x7f3684006e40 comp rx=0 tx=0).stop 2024-04-26T01:41:35.051 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:34.369+0000 7f36a0200700 1 -- 172.21.15.151:0/596905629 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f369810a920 msgr2=0x7f369819e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:41:35.051 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:34.369+0000 7f36a0200700 1 --2- 172.21.15.151:0/596905629 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f369810a920 0x7f369819e720 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f368c000c00 tx=0x7f368c011aa0 comp rx=0 tx=0).stop 2024-04-26T01:41:35.051 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:34.370+0000 7f36a0200700 1 -- 172.21.15.151:0/596905629 shutdown_connections 2024-04-26T01:41:35.051 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:34.370+0000 7f36a0200700 1 --2- 172.21.15.151:0/596905629 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f36800396a0 0x7f368003bb60 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:35.051 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:34.370+0000 7f36a0200700 1 --2- 172.21.15.151:0/596905629 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f369810a920 0x7f369819e720 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:35.051 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:34.370+0000 7f36a0200700 1 -- 172.21.15.151:0/596905629 >> 172.21.15.151:0/596905629 conn(0x7f369807dce0 msgr2=0x7f3698108480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:41:35.051 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:34.370+0000 7f36a0200700 1 -- 172.21.15.151:0/596905629 shutdown_connections 2024-04-26T01:41:35.051 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:34.370+0000 7f36a0200700 1 -- 172.21.15.151:0/596905629 wait complete. 2024-04-26T01:41:36.645 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:35.615+0000 7f2cb7400700 1 Processor -- start 2024-04-26T01:41:36.645 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:35.615+0000 7f2cb7400700 1 -- start start 2024-04-26T01:41:36.645 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:35.615+0000 7f2cb7400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2cb010a920 0x7f2cb010ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:41:36.646 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:35.615+0000 7f2cb7400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb010b310 con 0x7f2cb010a920 2024-04-26T01:41:36.646 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:35.616+0000 7f2cb4a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2cb010a920 0x7f2cb010ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:41:36.646 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:35.616+0000 7f2cb4a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2cb010a920 0x7f2cb010ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36580/0 (socket says 172.21.15.151:36580) 2024-04-26T01:41:36.650 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:35.616+0000 7f2cb4a00700 1 -- 172.21.15.151:0/653780483 learned_addr learned my addr 172.21.15.151:0/653780483 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:41:36.650 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:35.616+0000 7f2cb4a00700 1 -- 172.21.15.151:0/653780483 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cb010bb10 con 0x7f2cb010a920 2024-04-26T01:41:36.650 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:35.617+0000 7f2cb4a00700 1 --2- 172.21.15.151:0/653780483 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2cb010a920 0x7f2cb010ad40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2ca40098a0 tx=0x7f2ca4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1a83a067054dccd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:41:36.650 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:35.617+0000 7f2caf400700 1 -- 172.21.15.151:0/653780483 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ca4004160 con 0x7f2cb010a920 2024-04-26T01:41:36.650 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:35.617+0000 7f2caf400700 1 -- 172.21.15.151:0/653780483 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ca4004300 con 0x7f2cb010a920 2024-04-26T01:41:36.651 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:35.617+0000 7f2caf400700 1 -- 172.21.15.151:0/653780483 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ca4011a20 con 0x7f2cb010a920 2024-04-26T01:41:36.651 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:35.618+0000 7f2cb7400700 1 -- 172.21.15.151:0/653780483 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2cb010a920 msgr2=0x7f2cb010ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:41:36.651 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:35.618+0000 7f2cb7400700 1 --2- 172.21.15.151:0/653780483 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2cb010a920 0x7f2cb010ad40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2ca40098a0 tx=0x7f2ca4009bb0 comp rx=0 tx=0).stop 2024-04-26T01:41:36.651 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:35.618+0000 7f2cb7400700 1 -- 172.21.15.151:0/653780483 shutdown_connections 2024-04-26T01:41:36.651 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:35.618+0000 7f2cb7400700 1 --2- 172.21.15.151:0/653780483 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2cb010a920 0x7f2cb010ad40 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:36.651 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:35.618+0000 7f2cb7400700 1 -- 172.21.15.151:0/653780483 >> 172.21.15.151:0/653780483 conn(0x7f2cb007dce0 msgr2=0x7f2cb007e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:41:36.651 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:35.618+0000 7f2cb7400700 1 -- 172.21.15.151:0/653780483 shutdown_connections 2024-04-26T01:41:36.651 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:35.618+0000 7f2cb7400700 1 -- 172.21.15.151:0/653780483 wait complete. 2024-04-26T01:41:36.651 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:35.619+0000 7f2cb7400700 1 Processor -- start 2024-04-26T01:41:36.651 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:35.619+0000 7f2cb7400700 1 -- start start 2024-04-26T01:41:36.651 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:35.619+0000 7f2cb7400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2cb010a920 0x7f2cb019e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:41:36.651 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:35.619+0000 7f2cb7400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb010eec0 con 0x7f2cb010a920 2024-04-26T01:41:36.651 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:35.620+0000 7f2cb4a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2cb010a920 0x7f2cb019e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:41:36.651 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:35.620+0000 7f2cb4a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2cb010a920 0x7f2cb019e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36592/0 (socket says 172.21.15.151:36592) 2024-04-26T01:41:36.652 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:35.620+0000 7f2cb4a00700 1 -- 172.21.15.151:0/1506656377 learned_addr learned my addr 172.21.15.151:0/1506656377 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:41:36.652 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:35.620+0000 7f2cb4a00700 1 -- 172.21.15.151:0/1506656377 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cb019ec60 con 0x7f2cb010a920 2024-04-26T01:41:36.652 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:35.620+0000 7f2cb4a00700 1 --2- 172.21.15.151:0/1506656377 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2cb010a920 0x7f2cb019e720 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4011770 tx=0x7f2ca4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:41:36.652 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:35.621+0000 7f2cad600700 1 -- 172.21.15.151:0/1506656377 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ca4003a60 con 0x7f2cb010a920 2024-04-26T01:41:36.652 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:35.621+0000 7f2cad600700 1 -- 172.21.15.151:0/1506656377 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ca401b8f0 con 0x7f2cb010a920 2024-04-26T01:41:36.652 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:35.621+0000 7f2cb7400700 1 -- 172.21.15.151:0/1506656377 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cb019eef0 con 0x7f2cb010a920 2024-04-26T01:41:36.652 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:35.621+0000 7f2cad600700 1 -- 172.21.15.151:0/1506656377 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ca4019860 con 0x7f2cb010a920 2024-04-26T01:41:36.652 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:35.621+0000 7f2cb7400700 1 -- 172.21.15.151:0/1506656377 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cb01a1be0 con 0x7f2cb010a920 2024-04-26T01:41:36.652 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:35.622+0000 7f2cad600700 1 -- 172.21.15.151:0/1506656377 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f2ca401b440 con 0x7f2cb010a920 2024-04-26T01:41:36.652 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:35.622+0000 7f2cb7400700 1 -- 172.21.15.151:0/1506656377 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cb00640a0 con 0x7f2cb010a920 2024-04-26T01:41:36.652 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:35.623+0000 7f2cad600700 1 --2- 172.21.15.151:0/1506656377 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f2c980396b0 0x7f2c9803bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:41:36.652 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:35.623+0000 7f2cad600700 1 -- 172.21.15.151:0/1506656377 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2ca404de90 con 0x7f2cb010a920 2024-04-26T01:41:36.652 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:35.623+0000 7f2cafe00700 1 --2- 172.21.15.151:0/1506656377 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f2c980396b0 0x7f2c9803bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:41:36.652 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:35.623+0000 7f2cafe00700 1 --2- 172.21.15.151:0/1506656377 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f2c980396b0 0x7f2c9803bb70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c006fd0 tx=0x7f2c9c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:41:36.653 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:35.628+0000 7f2cad600700 1 -- 172.21.15.151:0/1506656377 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2ca401f030 con 0x7f2cb010a920 2024-04-26T01:41:36.653 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:35.971+0000 7f2cb7400700 1 -- 172.21.15.151:0/1506656377 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f2cb01a2190 con 0x7f2cb010a920 2024-04-26T01:41:36.653 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:35.977+0000 7f2cad600700 1 -- 172.21.15.151:0/1506656377 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f2ca40223b0 con 0x7f2cb010a920 2024-04-26T01:41:36.653 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:35.980+0000 7f2cb7400700 1 -- 172.21.15.151:0/1506656377 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f2c980396b0 msgr2=0x7f2c9803bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:41:36.653 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:35.980+0000 7f2cb7400700 1 --2- 172.21.15.151:0/1506656377 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f2c980396b0 0x7f2c9803bb70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c006fd0 tx=0x7f2c9c006e40 comp rx=0 tx=0).stop 2024-04-26T01:41:36.653 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:35.980+0000 7f2cb7400700 1 -- 172.21.15.151:0/1506656377 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2cb010a920 msgr2=0x7f2cb019e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:41:36.653 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:35.980+0000 7f2cb7400700 1 --2- 172.21.15.151:0/1506656377 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2cb010a920 0x7f2cb019e720 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4011770 tx=0x7f2ca4011aa0 comp rx=0 tx=0).stop 2024-04-26T01:41:36.653 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:35.981+0000 7f2cb7400700 1 -- 172.21.15.151:0/1506656377 shutdown_connections 2024-04-26T01:41:36.653 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:35.981+0000 7f2cb7400700 1 --2- 172.21.15.151:0/1506656377 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f2c980396b0 0x7f2c9803bb70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:36.653 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:35.981+0000 7f2cb7400700 1 --2- 172.21.15.151:0/1506656377 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2cb010a920 0x7f2cb019e720 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:36.653 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:35.981+0000 7f2cb7400700 1 -- 172.21.15.151:0/1506656377 >> 172.21.15.151:0/1506656377 conn(0x7f2cb007dce0 msgr2=0x7f2cb0108480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:41:36.653 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:35.981+0000 7f2cb7400700 1 -- 172.21.15.151:0/1506656377 shutdown_connections 2024-04-26T01:41:36.653 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-04-26T01:41:35.981+0000 7f2cb7400700 1 -- 172.21.15.151:0/1506656377 wait complete. 2024-04-26T01:41:36.653 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-26T01:41:36.654 INFO:teuthology.orchestra.run.smithi151.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-26T01:41:36.654 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:41:36.654 INFO:teuthology.orchestra.run.smithi151.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-26T01:41:36.654 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:41:36.654 INFO:teuthology.orchestra.run.smithi151.stdout:Or, if you are only running a single cluster on this host: 2024-04-26T01:41:36.655 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:41:36.655 INFO:teuthology.orchestra.run.smithi151.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-26T01:41:36.655 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:41:36.655 INFO:teuthology.orchestra.run.smithi151.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-26T01:41:36.655 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:41:36.655 INFO:teuthology.orchestra.run.smithi151.stdout: ceph telemetry on 2024-04-26T01:41:36.655 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:41:36.655 INFO:teuthology.orchestra.run.smithi151.stdout:For more information see: 2024-04-26T01:41:36.655 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:41:36.655 INFO:teuthology.orchestra.run.smithi151.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-26T01:41:36.655 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:41:36.655 INFO:teuthology.orchestra.run.smithi151.stdout:Bootstrap complete. 2024-04-26T01:41:36.749 INFO:tasks.cephadm:Fetching config... 2024-04-26T01:41:36.749 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-26T01:41:36.750 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-26T01:41:36.799 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-26T01:41:36.799 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-26T01:41:36.799 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-26T01:41:36.883 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-26T01:41:36.883 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-26T01:41:36.884 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-26T01:41:36.998 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-26T01:41:36.998 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-26T01:41:36.998 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-26T01:41:37.088 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-26T01:41:37.088 DEBUG:teuthology.orchestra.run.smithi151:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDNzd7KvbZNAU5d2PFfDejLyUsjAG4bUhv/GPWBuqIU9noj88W2T3gv7BkTF9R6/PaR51EPF4OHwFcBnz43ZB8WliajUE3Wk6sWWUgzFecylREAcIaPDvVBUFXAY6f3XTrdv++MwW4Hu2uOFkdrxJd3NXGxUVFsY7qidjTEXO+SQZUVj0aUanVE5UnwXoliiEMR/d1rO2fCQfNAe5Fjn88OoBGqAuXzw+ucltPTRjtD+8NKY5bP/jnuILiR1QPsG+VpJsk3HlRUW4UM55hTcpys7Px/L7f+BSUnQcoxOiODSG+8wl4tgULHqgrpdG6Rzdf0VB0DMkqtM2x9KqPA3GwAXG0VvH+zr/zlFgPUQ7c1wOspXYC0TSUMA7MlEjY6HauiYWT9FhZ/Jwp4F/Mawz4NW4XFYg7Y55p/C5T501vY7lE/GfXY/e3/QaHAxrNrtTfpuY0L8/f/D9Kk3U3iLHwkgWp4xv7fG4/hq7o72GeXcsKXX1NbH4R3Xh1frxFyUBE= ceph-c0e49e64-036d-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-26T01:41:37.210 INFO:teuthology.orchestra.run.smithi151.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDNzd7KvbZNAU5d2PFfDejLyUsjAG4bUhv/GPWBuqIU9noj88W2T3gv7BkTF9R6/PaR51EPF4OHwFcBnz43ZB8WliajUE3Wk6sWWUgzFecylREAcIaPDvVBUFXAY6f3XTrdv++MwW4Hu2uOFkdrxJd3NXGxUVFsY7qidjTEXO+SQZUVj0aUanVE5UnwXoliiEMR/d1rO2fCQfNAe5Fjn88OoBGqAuXzw+ucltPTRjtD+8NKY5bP/jnuILiR1QPsG+VpJsk3HlRUW4UM55hTcpys7Px/L7f+BSUnQcoxOiODSG+8wl4tgULHqgrpdG6Rzdf0VB0DMkqtM2x9KqPA3GwAXG0VvH+zr/zlFgPUQ7c1wOspXYC0TSUMA7MlEjY6HauiYWT9FhZ/Jwp4F/Mawz4NW4XFYg7Y55p/C5T501vY7lE/GfXY/e3/QaHAxrNrtTfpuY0L8/f/D9Kk3U3iLHwkgWp4xv7fG4/hq7o72GeXcsKXX1NbH4R3Xh1frxFyUBE= ceph-c0e49e64-036d-11ef-bc93-c7b262605968 2024-04-26T01:41:37.233 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:36 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/1506656377' entity='client.admin' 2024-04-26T01:41:37.244 DEBUG:teuthology.orchestra.run.smithi165:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDNzd7KvbZNAU5d2PFfDejLyUsjAG4bUhv/GPWBuqIU9noj88W2T3gv7BkTF9R6/PaR51EPF4OHwFcBnz43ZB8WliajUE3Wk6sWWUgzFecylREAcIaPDvVBUFXAY6f3XTrdv++MwW4Hu2uOFkdrxJd3NXGxUVFsY7qidjTEXO+SQZUVj0aUanVE5UnwXoliiEMR/d1rO2fCQfNAe5Fjn88OoBGqAuXzw+ucltPTRjtD+8NKY5bP/jnuILiR1QPsG+VpJsk3HlRUW4UM55hTcpys7Px/L7f+BSUnQcoxOiODSG+8wl4tgULHqgrpdG6Rzdf0VB0DMkqtM2x9KqPA3GwAXG0VvH+zr/zlFgPUQ7c1wOspXYC0TSUMA7MlEjY6HauiYWT9FhZ/Jwp4F/Mawz4NW4XFYg7Y55p/C5T501vY7lE/GfXY/e3/QaHAxrNrtTfpuY0L8/f/D9Kk3U3iLHwkgWp4xv7fG4/hq7o72GeXcsKXX1NbH4R3Xh1frxFyUBE= ceph-c0e49e64-036d-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-26T01:41:37.287 INFO:teuthology.orchestra.run.smithi165.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDNzd7KvbZNAU5d2PFfDejLyUsjAG4bUhv/GPWBuqIU9noj88W2T3gv7BkTF9R6/PaR51EPF4OHwFcBnz43ZB8WliajUE3Wk6sWWUgzFecylREAcIaPDvVBUFXAY6f3XTrdv++MwW4Hu2uOFkdrxJd3NXGxUVFsY7qidjTEXO+SQZUVj0aUanVE5UnwXoliiEMR/d1rO2fCQfNAe5Fjn88OoBGqAuXzw+ucltPTRjtD+8NKY5bP/jnuILiR1QPsG+VpJsk3HlRUW4UM55hTcpys7Px/L7f+BSUnQcoxOiODSG+8wl4tgULHqgrpdG6Rzdf0VB0DMkqtM2x9KqPA3GwAXG0VvH+zr/zlFgPUQ7c1wOspXYC0TSUMA7MlEjY6HauiYWT9FhZ/Jwp4F/Mawz4NW4XFYg7Y55p/C5T501vY7lE/GfXY/e3/QaHAxrNrtTfpuY0L8/f/D9Kk3U3iLHwkgWp4xv7fG4/hq7o72GeXcsKXX1NbH4R3Xh1frxFyUBE= ceph-c0e49e64-036d-11ef-bc93-c7b262605968 2024-04-26T01:41:37.298 DEBUG:teuthology.orchestra.run.smithi172:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDNzd7KvbZNAU5d2PFfDejLyUsjAG4bUhv/GPWBuqIU9noj88W2T3gv7BkTF9R6/PaR51EPF4OHwFcBnz43ZB8WliajUE3Wk6sWWUgzFecylREAcIaPDvVBUFXAY6f3XTrdv++MwW4Hu2uOFkdrxJd3NXGxUVFsY7qidjTEXO+SQZUVj0aUanVE5UnwXoliiEMR/d1rO2fCQfNAe5Fjn88OoBGqAuXzw+ucltPTRjtD+8NKY5bP/jnuILiR1QPsG+VpJsk3HlRUW4UM55hTcpys7Px/L7f+BSUnQcoxOiODSG+8wl4tgULHqgrpdG6Rzdf0VB0DMkqtM2x9KqPA3GwAXG0VvH+zr/zlFgPUQ7c1wOspXYC0TSUMA7MlEjY6HauiYWT9FhZ/Jwp4F/Mawz4NW4XFYg7Y55p/C5T501vY7lE/GfXY/e3/QaHAxrNrtTfpuY0L8/f/D9Kk3U3iLHwkgWp4xv7fG4/hq7o72GeXcsKXX1NbH4R3Xh1frxFyUBE= ceph-c0e49e64-036d-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-26T01:41:37.342 INFO:teuthology.orchestra.run.smithi172.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDNzd7KvbZNAU5d2PFfDejLyUsjAG4bUhv/GPWBuqIU9noj88W2T3gv7BkTF9R6/PaR51EPF4OHwFcBnz43ZB8WliajUE3Wk6sWWUgzFecylREAcIaPDvVBUFXAY6f3XTrdv++MwW4Hu2uOFkdrxJd3NXGxUVFsY7qidjTEXO+SQZUVj0aUanVE5UnwXoliiEMR/d1rO2fCQfNAe5Fjn88OoBGqAuXzw+ucltPTRjtD+8NKY5bP/jnuILiR1QPsG+VpJsk3HlRUW4UM55hTcpys7Px/L7f+BSUnQcoxOiODSG+8wl4tgULHqgrpdG6Rzdf0VB0DMkqtM2x9KqPA3GwAXG0VvH+zr/zlFgPUQ7c1wOspXYC0TSUMA7MlEjY6HauiYWT9FhZ/Jwp4F/Mawz4NW4XFYg7Y55p/C5T501vY7lE/GfXY/e3/QaHAxrNrtTfpuY0L8/f/D9Kk3U3iLHwkgWp4xv7fG4/hq7o72GeXcsKXX1NbH4R3Xh1frxFyUBE= ceph-c0e49e64-036d-11ef-bc93-c7b262605968 2024-04-26T01:41:37.353 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-26T01:41:37.763 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:41:40.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:40.637+0000 7f6174a00700 1 -- 172.21.15.151:0/1374848074 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6170076a10 msgr2=0x7f6170074e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:41:40.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:40.637+0000 7f6174a00700 1 --2- 172.21.15.151:0/1374848074 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6170076a10 0x7f6170074e70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6160009910 tx=0x7f6160009c20 comp rx=0 tx=0).stop 2024-04-26T01:41:40.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:40.637+0000 7f6174a00700 1 -- 172.21.15.151:0/1374848074 shutdown_connections 2024-04-26T01:41:40.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:40.637+0000 7f6174a00700 1 --2- 172.21.15.151:0/1374848074 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6170076a10 0x7f6170074e70 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:40.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:40.637+0000 7f6174a00700 1 -- 172.21.15.151:0/1374848074 >> 172.21.15.151:0/1374848074 conn(0x7f61700fd880 msgr2=0x7f61700ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:41:40.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:40.637+0000 7f6174a00700 1 -- 172.21.15.151:0/1374848074 shutdown_connections 2024-04-26T01:41:40.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:40.637+0000 7f6174a00700 1 -- 172.21.15.151:0/1374848074 wait complete. 2024-04-26T01:41:40.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:40.638+0000 7f6174a00700 1 Processor -- start 2024-04-26T01:41:40.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:40.638+0000 7f6174a00700 1 -- start start 2024-04-26T01:41:40.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:40.639+0000 7f6174a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6170076a10 0x7f6170199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:41:40.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:40.639+0000 7f6174a00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f617010a410 con 0x7f6170076a10 2024-04-26T01:41:40.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:40.639+0000 7f616e800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6170076a10 0x7f6170199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:41:40.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:40.639+0000 7f616e800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6170076a10 0x7f6170199b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:56988/0 (socket says 172.21.15.151:56988) 2024-04-26T01:41:40.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:40.639+0000 7f616e800700 1 -- 172.21.15.151:0/3033922471 learned_addr learned my addr 172.21.15.151:0/3033922471 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:41:40.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:40.639+0000 7f616e800700 1 -- 172.21.15.151:0/3033922471 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f617019a040 con 0x7f6170076a10 2024-04-26T01:41:40.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:40.640+0000 7f616e800700 1 --2- 172.21.15.151:0/3033922471 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6170076a10 0x7f6170199b00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6160005190 tx=0x7f6160004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:41:40.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:40.640+0000 7f6167400700 1 -- 172.21.15.151:0/3033922471 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6160027070 con 0x7f6170076a10 2024-04-26T01:41:40.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:40.640+0000 7f6167400700 1 -- 172.21.15.151:0/3033922471 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6160022470 con 0x7f6170076a10 2024-04-26T01:41:40.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:40.640+0000 7f6174a00700 1 -- 172.21.15.151:0/3033922471 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f617019a2d0 con 0x7f6170076a10 2024-04-26T01:41:40.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:40.640+0000 7f6167400700 1 -- 172.21.15.151:0/3033922471 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f616000f5f0 con 0x7f6170076a10 2024-04-26T01:41:40.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:40.640+0000 7f6174a00700 1 -- 172.21.15.151:0/3033922471 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f617019cfc0 con 0x7f6170076a10 2024-04-26T01:41:40.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:40.641+0000 7f6167400700 1 -- 172.21.15.151:0/3033922471 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f616002a070 con 0x7f6170076a10 2024-04-26T01:41:40.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:40.641+0000 7f6174a00700 1 -- 172.21.15.151:0/3033922471 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f617010ab90 con 0x7f6170076a10 2024-04-26T01:41:40.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:40.642+0000 7f6167400700 1 --2- 172.21.15.151:0/3033922471 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f6154039650 0x7f615403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:41:40.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:40.642+0000 7f6167400700 1 -- 172.21.15.151:0/3033922471 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f616001e030 con 0x7f6170076a10 2024-04-26T01:41:40.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:40.642+0000 7f616de00700 1 --2- 172.21.15.151:0/3033922471 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f6154039650 0x7f615403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:41:40.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:40.643+0000 7f616de00700 1 --2- 172.21.15.151:0/3033922471 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f6154039650 0x7f615403bb10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f615c006fd0 tx=0x7f615c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:41:40.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:40.648+0000 7f6167400700 1 -- 172.21.15.151:0/3033922471 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6160013070 con 0x7f6170076a10 2024-04-26T01:41:40.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:40.875+0000 7f6174a00700 1 -- 172.21.15.151:0/3033922471 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f617002e090 con 0x7f6170076a10 2024-04-26T01:41:40.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:40.879+0000 7f6167400700 1 -- 172.21.15.151:0/3033922471 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f61600310c0 con 0x7f6170076a10 2024-04-26T01:41:40.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:40.882+0000 7f6174a00700 1 -- 172.21.15.151:0/3033922471 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f6154039650 msgr2=0x7f615403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:41:40.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:40.882+0000 7f6174a00700 1 --2- 172.21.15.151:0/3033922471 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f6154039650 0x7f615403bb10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f615c006fd0 tx=0x7f615c006e40 comp rx=0 tx=0).stop 2024-04-26T01:41:40.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:40.882+0000 7f6174a00700 1 -- 172.21.15.151:0/3033922471 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6170076a10 msgr2=0x7f6170199b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:41:40.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:40.882+0000 7f6174a00700 1 --2- 172.21.15.151:0/3033922471 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6170076a10 0x7f6170199b00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6160005190 tx=0x7f6160004f80 comp rx=0 tx=0).stop 2024-04-26T01:41:40.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:40.883+0000 7f6174a00700 1 -- 172.21.15.151:0/3033922471 shutdown_connections 2024-04-26T01:41:40.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:40.883+0000 7f6174a00700 1 --2- 172.21.15.151:0/3033922471 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f6154039650 0x7f615403bb10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:40.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:40.883+0000 7f6174a00700 1 --2- 172.21.15.151:0/3033922471 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6170076a10 0x7f6170199b00 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:40.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:40.883+0000 7f6174a00700 1 -- 172.21.15.151:0/3033922471 >> 172.21.15.151:0/3033922471 conn(0x7f61700fd880 msgr2=0x7f61700fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:41:40.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:40.884+0000 7f6174a00700 1 -- 172.21.15.151:0/3033922471 shutdown_connections 2024-04-26T01:41:40.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:40.884+0000 7f6174a00700 1 -- 172.21.15.151:0/3033922471 wait complete. 2024-04-26T01:41:41.829 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-26T01:41:41.829 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-26T01:41:42.120 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:41 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/3033922471' entity='client.admin' 2024-04-26T01:41:42.237 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:41:44.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:44.989+0000 7fd509600700 1 -- 172.21.15.151:0/1471665325 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd504108c50 msgr2=0x7fd504109070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:41:44.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:44.989+0000 7fd509600700 1 --2- 172.21.15.151:0/1471665325 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd504108c50 0x7fd504109070 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd4f4009910 tx=0x7fd4f4009c20 comp rx=0 tx=0).stop 2024-04-26T01:41:44.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:44.990+0000 7fd509600700 1 -- 172.21.15.151:0/1471665325 shutdown_connections 2024-04-26T01:41:44.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:44.990+0000 7fd509600700 1 --2- 172.21.15.151:0/1471665325 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd504108c50 0x7fd504109070 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:44.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:44.990+0000 7fd509600700 1 -- 172.21.15.151:0/1471665325 >> 172.21.15.151:0/1471665325 conn(0x7fd504077a00 msgr2=0x7fd504079e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:41:44.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:44.990+0000 7fd509600700 1 -- 172.21.15.151:0/1471665325 shutdown_connections 2024-04-26T01:41:44.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:44.990+0000 7fd509600700 1 -- 172.21.15.151:0/1471665325 wait complete. 2024-04-26T01:41:44.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:44.991+0000 7fd509600700 1 Processor -- start 2024-04-26T01:41:44.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:44.991+0000 7fd509600700 1 -- start start 2024-04-26T01:41:44.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:44.992+0000 7fd509600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd504108c50 0x7fd50419df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:41:44.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:44.992+0000 7fd509600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd50410e820 con 0x7fd504108c50 2024-04-26T01:41:44.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:44.992+0000 7fd503e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd504108c50 0x7fd50419df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:41:44.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:44.992+0000 7fd503e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd504108c50 0x7fd50419df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:57008/0 (socket says 172.21.15.151:57008) 2024-04-26T01:41:44.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:44.992+0000 7fd503e00700 1 -- 172.21.15.151:0/4007551336 learned_addr learned my addr 172.21.15.151:0/4007551336 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:41:44.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:44.992+0000 7fd503e00700 1 -- 172.21.15.151:0/4007551336 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd50419e470 con 0x7fd504108c50 2024-04-26T01:41:44.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:44.992+0000 7fd503e00700 1 --2- 172.21.15.151:0/4007551336 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd504108c50 0x7fd50419df30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd4f4005950 tx=0x7fd4f4004fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:41:44.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:44.993+0000 7fd500c00700 1 -- 172.21.15.151:0/4007551336 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4f4026070 con 0x7fd504108c50 2024-04-26T01:41:44.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:44.993+0000 7fd500c00700 1 -- 172.21.15.151:0/4007551336 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4f400bc00 con 0x7fd504108c50 2024-04-26T01:41:44.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:44.993+0000 7fd509600700 1 -- 172.21.15.151:0/4007551336 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd50419e700 con 0x7fd504108c50 2024-04-26T01:41:44.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:44.993+0000 7fd500c00700 1 -- 172.21.15.151:0/4007551336 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4f4003a10 con 0x7fd504108c50 2024-04-26T01:41:44.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:44.993+0000 7fd509600700 1 -- 172.21.15.151:0/4007551336 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5041a13f0 con 0x7fd504108c50 2024-04-26T01:41:44.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:44.994+0000 7fd509600700 1 -- 172.21.15.151:0/4007551336 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd50410efa0 con 0x7fd504108c50 2024-04-26T01:41:44.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:44.994+0000 7fd500c00700 1 -- 172.21.15.151:0/4007551336 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7fd4f4029070 con 0x7fd504108c50 2024-04-26T01:41:44.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:44.995+0000 7fd500c00700 1 --2- 172.21.15.151:0/4007551336 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fd4e80396b0 0x7fd4e803bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:41:44.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:44.995+0000 7fd500c00700 1 -- 172.21.15.151:0/4007551336 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd4f401e070 con 0x7fd504108c50 2024-04-26T01:41:44.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:44.995+0000 7fd503400700 1 --2- 172.21.15.151:0/4007551336 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fd4e80396b0 0x7fd4e803bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:41:44.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:44.996+0000 7fd503400700 1 --2- 172.21.15.151:0/4007551336 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fd4e80396b0 0x7fd4e803bb70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0006fd0 tx=0x7fd4f0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:41:44.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:45.001+0000 7fd500c00700 1 -- 172.21.15.151:0/4007551336 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd4f4013070 con 0x7fd504108c50 2024-04-26T01:41:44.998 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:44 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:41:44.998 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:44 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:41:44.998 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:44 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi151", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:41:44.998 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:44 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:41:44.998 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:44 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:41:44.998 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:44 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:41:45.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:45.229+0000 7fd509600700 1 -- 172.21.15.151:0/4007551336 --> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fd50402e090 con 0x7fd4e80396b0 2024-04-26T01:41:45.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:45.238+0000 7fd500c00700 1 -- 172.21.15.151:0/4007551336 <== mgr.14152 v2:172.21.15.151:6800/2251256505 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd50402e090 con 0x7fd4e80396b0 2024-04-26T01:41:45.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:45.241+0000 7fd509600700 1 -- 172.21.15.151:0/4007551336 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fd4e80396b0 msgr2=0x7fd4e803bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:41:45.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:45.241+0000 7fd509600700 1 --2- 172.21.15.151:0/4007551336 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fd4e80396b0 0x7fd4e803bb70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0006fd0 tx=0x7fd4f0006e40 comp rx=0 tx=0).stop 2024-04-26T01:41:45.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:45.241+0000 7fd509600700 1 -- 172.21.15.151:0/4007551336 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd504108c50 msgr2=0x7fd50419df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:41:45.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:45.241+0000 7fd509600700 1 --2- 172.21.15.151:0/4007551336 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd504108c50 0x7fd50419df30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd4f4005950 tx=0x7fd4f4004fa0 comp rx=0 tx=0).stop 2024-04-26T01:41:45.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:45.241+0000 7fd509600700 1 -- 172.21.15.151:0/4007551336 shutdown_connections 2024-04-26T01:41:45.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:45.241+0000 7fd509600700 1 --2- 172.21.15.151:0/4007551336 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fd4e80396b0 0x7fd4e803bb70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:45.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:45.241+0000 7fd509600700 1 --2- 172.21.15.151:0/4007551336 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd504108c50 0x7fd50419df30 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:45.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:45.241+0000 7fd509600700 1 -- 172.21.15.151:0/4007551336 >> 172.21.15.151:0/4007551336 conn(0x7fd504077a00 msgr2=0x7fd5040784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:41:45.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:45.242+0000 7fd509600700 1 -- 172.21.15.151:0/4007551336 shutdown_connections 2024-04-26T01:41:45.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:45.242+0000 7fd509600700 1 -- 172.21.15.151:0/4007551336 wait complete. 2024-04-26T01:41:46.011 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi165 2024-04-26T01:41:46.011 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-26T01:41:46.011 DEBUG:teuthology.orchestra.run.smithi165:> dd of=/etc/ceph/ceph.conf 2024-04-26T01:41:46.030 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-26T01:41:46.030 DEBUG:teuthology.orchestra.run.smithi165:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T01:41:46.087 INFO:tasks.cephadm:Adding host smithi165 to orchestrator... 2024-04-26T01:41:46.087 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph orch host add smithi165 2024-04-26T01:41:46.501 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:41:46.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:46 smithi151 ceph-mon[28057]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:41:46.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:46 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:41:46.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:46 smithi151 ceph-mon[28057]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:41:46.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:46 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:41:46.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:46 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:41:46.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:46 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:41:47.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:47 smithi151 ceph-mon[28057]: Updating smithi151:/etc/ceph/ceph.conf 2024-04-26T01:41:47.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:47 smithi151 ceph-mon[28057]: Updating smithi151:/var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:41:48.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:48 smithi151 ceph-mon[28057]: Updating smithi151:/etc/ceph/ceph.client.admin.keyring 2024-04-26T01:41:48.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:48 smithi151 ceph-mon[28057]: Updating smithi151:/var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T01:41:48.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:48 smithi151 ceph-mon[28057]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:41:48.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:48 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:41:48.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:48 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:41:48.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:48 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:41:48.548 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:48.550+0000 7fce1f600700 1 -- 172.21.15.151:0/1207793711 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce181084b0 msgr2=0x7fce18108890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:41:48.548 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:48.550+0000 7fce1f600700 1 --2- 172.21.15.151:0/1207793711 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce181084b0 0x7fce18108890 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fce08009910 tx=0x7fce08009c20 comp rx=0 tx=0).stop 2024-04-26T01:41:48.549 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:48.551+0000 7fce1f600700 1 -- 172.21.15.151:0/1207793711 shutdown_connections 2024-04-26T01:41:48.549 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:48.551+0000 7fce1f600700 1 --2- 172.21.15.151:0/1207793711 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce181084b0 0x7fce18108890 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:48.549 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:48.551+0000 7fce1f600700 1 -- 172.21.15.151:0/1207793711 >> 172.21.15.151:0/1207793711 conn(0x7fce180fdf30 msgr2=0x7fce18100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:41:48.549 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:48.551+0000 7fce1f600700 1 -- 172.21.15.151:0/1207793711 shutdown_connections 2024-04-26T01:41:48.549 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:48.551+0000 7fce1f600700 1 -- 172.21.15.151:0/1207793711 wait complete. 2024-04-26T01:41:48.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:48.552+0000 7fce1f600700 1 Processor -- start 2024-04-26T01:41:48.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:48.552+0000 7fce1f600700 1 -- start start 2024-04-26T01:41:48.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:48.552+0000 7fce1f600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce181084b0 0x7fce1819a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:41:48.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:48.552+0000 7fce1f600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fce1810c910 con 0x7fce181084b0 2024-04-26T01:41:48.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:48.553+0000 7fce1cc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce181084b0 0x7fce1819a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:41:48.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:48.553+0000 7fce1cc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce181084b0 0x7fce1819a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:57018/0 (socket says 172.21.15.151:57018) 2024-04-26T01:41:48.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:48.553+0000 7fce1cc00700 1 -- 172.21.15.151:0/3461009031 learned_addr learned my addr 172.21.15.151:0/3461009031 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:41:48.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:48.553+0000 7fce1cc00700 1 -- 172.21.15.151:0/3461009031 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce1819a5d0 con 0x7fce181084b0 2024-04-26T01:41:48.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:48.553+0000 7fce1cc00700 1 --2- 172.21.15.151:0/3461009031 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce181084b0 0x7fce1819a090 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fce08006010 tx=0x7fce08005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:41:48.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:48.554+0000 7fce16000700 1 -- 172.21.15.151:0/3461009031 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce08026070 con 0x7fce181084b0 2024-04-26T01:41:48.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:48.554+0000 7fce1f600700 1 -- 172.21.15.151:0/3461009031 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce1819a8c0 con 0x7fce181084b0 2024-04-26T01:41:48.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:48.554+0000 7fce16000700 1 -- 172.21.15.151:0/3461009031 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce0800bba0 con 0x7fce181084b0 2024-04-26T01:41:48.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:48.554+0000 7fce16000700 1 -- 172.21.15.151:0/3461009031 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce08003910 con 0x7fce181084b0 2024-04-26T01:41:48.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:48.554+0000 7fce1f600700 1 -- 172.21.15.151:0/3461009031 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce1819e960 con 0x7fce181084b0 2024-04-26T01:41:48.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:48.555+0000 7fce1f600700 1 -- 172.21.15.151:0/3461009031 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce18103b60 con 0x7fce181084b0 2024-04-26T01:41:48.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:48.555+0000 7fce16000700 1 -- 172.21.15.151:0/3461009031 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7fce08029070 con 0x7fce181084b0 2024-04-26T01:41:48.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:48.555+0000 7fce16000700 1 --2- 172.21.15.151:0/3461009031 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fcdf80396b0 0x7fcdf803bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:41:48.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:48.555+0000 7fce16000700 1 -- 172.21.15.151:0/3461009031 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fce0801e070 con 0x7fce181084b0 2024-04-26T01:41:48.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:48.556+0000 7fce0fe00700 1 --2- 172.21.15.151:0/3461009031 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fcdf80396b0 0x7fcdf803bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:41:48.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:48.556+0000 7fce0fe00700 1 --2- 172.21.15.151:0/3461009031 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fcdf80396b0 0x7fcdf803bb70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fce00006fd0 tx=0x7fce00006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:41:48.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:48.561+0000 7fce16000700 1 -- 172.21.15.151:0/3461009031 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fce08013070 con 0x7fce181084b0 2024-04-26T01:41:48.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:48.790+0000 7fce1f600700 1 -- 172.21.15.151:0/3461009031 --> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi165", "target": ["mon-mgr", ""]}) -- 0x7fce1819b3f0 con 0x7fcdf80396b0 2024-04-26T01:41:49.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:49 smithi151 ceph-mon[28057]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi165", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:41:49.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:49 smithi151 ceph-mon[28057]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:41:50.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:50 smithi151 ceph-mon[28057]: Deploying cephadm binary to smithi165 2024-04-26T01:41:51.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:51 smithi151 ceph-mon[28057]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:41:53.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:53.545+0000 7fce16000700 1 -- 172.21.15.151:0/3461009031 <== mgr.14152 v2:172.21.15.151:6800/2251256505 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fce1819b3f0 con 0x7fcdf80396b0 2024-04-26T01:41:53.542 INFO:teuthology.orchestra.run.smithi151.stdout:Added host 'smithi165' with addr '172.21.15.165' 2024-04-26T01:41:53.546 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:53.548+0000 7fce1f600700 1 -- 172.21.15.151:0/3461009031 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fcdf80396b0 msgr2=0x7fcdf803bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:41:53.546 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:53.548+0000 7fce1f600700 1 --2- 172.21.15.151:0/3461009031 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fcdf80396b0 0x7fcdf803bb70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fce00006fd0 tx=0x7fce00006e40 comp rx=0 tx=0).stop 2024-04-26T01:41:53.546 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:53.548+0000 7fce1f600700 1 -- 172.21.15.151:0/3461009031 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce181084b0 msgr2=0x7fce1819a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:41:53.546 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:53.548+0000 7fce1f600700 1 --2- 172.21.15.151:0/3461009031 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce181084b0 0x7fce1819a090 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fce08006010 tx=0x7fce08005780 comp rx=0 tx=0).stop 2024-04-26T01:41:53.546 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:53.549+0000 7fce1f600700 1 -- 172.21.15.151:0/3461009031 shutdown_connections 2024-04-26T01:41:53.546 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:53.549+0000 7fce1f600700 1 --2- 172.21.15.151:0/3461009031 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fcdf80396b0 0x7fcdf803bb70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:53.546 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:53.549+0000 7fce1f600700 1 --2- 172.21.15.151:0/3461009031 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce181084b0 0x7fce1819a090 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:53.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:53.549+0000 7fce1f600700 1 -- 172.21.15.151:0/3461009031 >> 172.21.15.151:0/3461009031 conn(0x7fce180fdf30 msgr2=0x7fce1810a030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:41:53.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:53.549+0000 7fce1f600700 1 -- 172.21.15.151:0/3461009031 shutdown_connections 2024-04-26T01:41:53.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:53.549+0000 7fce1f600700 1 -- 172.21.15.151:0/3461009031 wait complete. 2024-04-26T01:41:53.601 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:53 smithi151 ceph-mon[28057]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:41:54.321 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-26T01:41:54.739 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:41:54.870 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:54 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:41:54.870 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:54 smithi151 ceph-mon[28057]: Added host smithi165 2024-04-26T01:41:54.870 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:54 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:41:54.870 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:54 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:41:56.006 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:55 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:41:56.006 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:55 smithi151 ceph-mon[28057]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:41:56.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:56.782+0000 7fe660400700 1 -- 172.21.15.151:0/3065320355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe6581084b0 msgr2=0x7fe658108890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:41:56.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:56.782+0000 7fe660400700 1 --2- 172.21.15.151:0/3065320355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe6581084b0 0x7fe658108890 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe64c009910 tx=0x7fe64c009c20 comp rx=0 tx=0).stop 2024-04-26T01:41:56.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:56.783+0000 7fe660400700 1 -- 172.21.15.151:0/3065320355 shutdown_connections 2024-04-26T01:41:56.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:56.783+0000 7fe660400700 1 --2- 172.21.15.151:0/3065320355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe6581084b0 0x7fe658108890 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:56.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:56.783+0000 7fe660400700 1 -- 172.21.15.151:0/3065320355 >> 172.21.15.151:0/3065320355 conn(0x7fe6580fdf30 msgr2=0x7fe658100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:41:56.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:56.783+0000 7fe660400700 1 -- 172.21.15.151:0/3065320355 shutdown_connections 2024-04-26T01:41:56.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:56.783+0000 7fe660400700 1 -- 172.21.15.151:0/3065320355 wait complete. 2024-04-26T01:41:56.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:56.784+0000 7fe660400700 1 Processor -- start 2024-04-26T01:41:56.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:56.784+0000 7fe660400700 1 -- start start 2024-04-26T01:41:56.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:56.784+0000 7fe660400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe6581084b0 0x7fe65819a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:41:56.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:56.784+0000 7fe660400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe65810c910 con 0x7fe6581084b0 2024-04-26T01:41:56.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:56.784+0000 7fe65da00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe6581084b0 0x7fe65819a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:41:56.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:56.785+0000 7fe65da00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe6581084b0 0x7fe65819a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:57370/0 (socket says 172.21.15.151:57370) 2024-04-26T01:41:56.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:56.785+0000 7fe65da00700 1 -- 172.21.15.151:0/2441020616 learned_addr learned my addr 172.21.15.151:0/2441020616 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:41:56.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:56.785+0000 7fe65da00700 1 -- 172.21.15.151:0/2441020616 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe65819a5d0 con 0x7fe6581084b0 2024-04-26T01:41:56.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:56.785+0000 7fe65da00700 1 --2- 172.21.15.151:0/2441020616 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe6581084b0 0x7fe65819a090 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe64c006010 tx=0x7fe64c005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:41:56.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:56.785+0000 7fe656000700 1 -- 172.21.15.151:0/2441020616 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe64c026070 con 0x7fe6581084b0 2024-04-26T01:41:56.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:56.786+0000 7fe656000700 1 -- 172.21.15.151:0/2441020616 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe64c00bba0 con 0x7fe6581084b0 2024-04-26T01:41:56.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:56.786+0000 7fe660400700 1 -- 172.21.15.151:0/2441020616 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe65819a8c0 con 0x7fe6581084b0 2024-04-26T01:41:56.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:56.786+0000 7fe656000700 1 -- 172.21.15.151:0/2441020616 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe64c003910 con 0x7fe6581084b0 2024-04-26T01:41:56.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:56.786+0000 7fe660400700 1 -- 172.21.15.151:0/2441020616 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe65819e960 con 0x7fe6581084b0 2024-04-26T01:41:56.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:56.787+0000 7fe656000700 1 -- 172.21.15.151:0/2441020616 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7fe64c029070 con 0x7fe6581084b0 2024-04-26T01:41:56.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:56.787+0000 7fe660400700 1 -- 172.21.15.151:0/2441020616 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe658103b60 con 0x7fe6581084b0 2024-04-26T01:41:56.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:56.787+0000 7fe656000700 1 --2- 172.21.15.151:0/2441020616 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fe6400396b0 0x7fe64003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:41:56.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:56.787+0000 7fe656000700 1 -- 172.21.15.151:0/2441020616 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe64c01e070 con 0x7fe6581084b0 2024-04-26T01:41:56.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:56.788+0000 7fe65d000700 1 --2- 172.21.15.151:0/2441020616 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fe6400396b0 0x7fe64003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:41:56.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:56.788+0000 7fe65d000700 1 --2- 172.21.15.151:0/2441020616 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fe6400396b0 0x7fe64003bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe644006fd0 tx=0x7fe644006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:41:56.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:56.793+0000 7fe656000700 1 -- 172.21.15.151:0/2441020616 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe64c013070 con 0x7fe6581084b0 2024-04-26T01:41:57.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:57.030+0000 7fe660400700 1 -- 172.21.15.151:0/2441020616 --> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe65819b3f0 con 0x7fe6400396b0 2024-04-26T01:41:57.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:57.031+0000 7fe656000700 1 -- 172.21.15.151:0/2441020616 <== mgr.14152 v2:172.21.15.151:6800/2251256505 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+161 (secure 0 0 0) 0x7fe65819b3f0 con 0x7fe6400396b0 2024-04-26T01:41:57.029 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:41:57.029 INFO:teuthology.orchestra.run.smithi151.stdout:[{"addr": "172.21.15.151", "hostname": "smithi151", "labels": [], "status": ""}, {"addr": "172.21.15.165", "hostname": "smithi165", "labels": [], "status": ""}] 2024-04-26T01:41:57.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:57.034+0000 7fe660400700 1 -- 172.21.15.151:0/2441020616 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fe6400396b0 msgr2=0x7fe64003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:41:57.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:57.034+0000 7fe660400700 1 --2- 172.21.15.151:0/2441020616 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fe6400396b0 0x7fe64003bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe644006fd0 tx=0x7fe644006e40 comp rx=0 tx=0).stop 2024-04-26T01:41:57.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:57.034+0000 7fe660400700 1 -- 172.21.15.151:0/2441020616 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe6581084b0 msgr2=0x7fe65819a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:41:57.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:57.034+0000 7fe660400700 1 --2- 172.21.15.151:0/2441020616 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe6581084b0 0x7fe65819a090 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe64c006010 tx=0x7fe64c005780 comp rx=0 tx=0).stop 2024-04-26T01:41:57.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:57.034+0000 7fe660400700 1 -- 172.21.15.151:0/2441020616 shutdown_connections 2024-04-26T01:41:57.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:57.034+0000 7fe660400700 1 --2- 172.21.15.151:0/2441020616 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fe6400396b0 0x7fe64003bb70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:57.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:57.034+0000 7fe660400700 1 --2- 172.21.15.151:0/2441020616 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe6581084b0 0x7fe65819a090 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:57.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:57.035+0000 7fe660400700 1 -- 172.21.15.151:0/2441020616 >> 172.21.15.151:0/2441020616 conn(0x7fe6580fdf30 msgr2=0x7fe65810a030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:41:57.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:57.035+0000 7fe660400700 1 -- 172.21.15.151:0/2441020616 shutdown_connections 2024-04-26T01:41:57.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:41:57.035+0000 7fe660400700 1 -- 172.21.15.151:0/2441020616 wait complete. 2024-04-26T01:41:57.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:57 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:41:57.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:57 smithi151 ceph-mon[28057]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:41:57.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:57 smithi151 ceph-mon[28057]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:41:57.789 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi172 2024-04-26T01:41:57.789 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-26T01:41:57.789 DEBUG:teuthology.orchestra.run.smithi172:> dd of=/etc/ceph/ceph.conf 2024-04-26T01:41:57.807 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-26T01:41:57.807 DEBUG:teuthology.orchestra.run.smithi172:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T01:41:57.865 INFO:tasks.cephadm:Adding host smithi172 to orchestrator... 2024-04-26T01:41:57.865 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph orch host add smithi172 2024-04-26T01:41:58.299 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:41:59.535 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:59 smithi151 ceph-mon[28057]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:41:59.535 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:59 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:41:59.535 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:59 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:41:59.535 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:41:59 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:00.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:00.340+0000 7f08de800700 1 -- 172.21.15.151:0/1774827721 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f08d81084b0 msgr2=0x7f08d8108890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:42:00.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:00.340+0000 7f08de800700 1 --2- 172.21.15.151:0/1774827721 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f08d81084b0 0x7f08d8108890 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f08c8009910 tx=0x7f08c8009c20 comp rx=0 tx=0).stop 2024-04-26T01:42:00.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:00.340+0000 7f08de800700 1 -- 172.21.15.151:0/1774827721 shutdown_connections 2024-04-26T01:42:00.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:00.340+0000 7f08de800700 1 --2- 172.21.15.151:0/1774827721 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f08d81084b0 0x7f08d8108890 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:00.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:00.340+0000 7f08de800700 1 -- 172.21.15.151:0/1774827721 >> 172.21.15.151:0/1774827721 conn(0x7f08d80fdf30 msgr2=0x7f08d8100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:42:00.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:00.340+0000 7f08de800700 1 -- 172.21.15.151:0/1774827721 shutdown_connections 2024-04-26T01:42:00.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:00.340+0000 7f08de800700 1 -- 172.21.15.151:0/1774827721 wait complete. 2024-04-26T01:42:00.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:00.341+0000 7f08de800700 1 Processor -- start 2024-04-26T01:42:00.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:00.342+0000 7f08de800700 1 -- start start 2024-04-26T01:42:00.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:00.342+0000 7f08de800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f08d81084b0 0x7f08d819eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:42:00.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:00.342+0000 7f08de800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f08d810c910 con 0x7f08d81084b0 2024-04-26T01:42:00.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:00.342+0000 7f08d7400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f08d81084b0 0x7f08d819eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:42:00.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:00.342+0000 7f08d7400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f08d81084b0 0x7f08d819eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:54998/0 (socket says 172.21.15.151:54998) 2024-04-26T01:42:00.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:00.342+0000 7f08d7400700 1 -- 172.21.15.151:0/3602713682 learned_addr learned my addr 172.21.15.151:0/3602713682 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:42:00.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:00.343+0000 7f08d7400700 1 -- 172.21.15.151:0/3602713682 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08d819f0d0 con 0x7f08d81084b0 2024-04-26T01:42:00.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:00.343+0000 7f08d7400700 1 --2- 172.21.15.151:0/3602713682 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f08d81084b0 0x7f08d819eb90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f08c8006010 tx=0x7f08c8005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:42:00.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:00.344+0000 7f08cfe00700 1 -- 172.21.15.151:0/3602713682 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08c8026070 con 0x7f08d81084b0 2024-04-26T01:42:00.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:00.344+0000 7f08cfe00700 1 -- 172.21.15.151:0/3602713682 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08c800bba0 con 0x7f08d81084b0 2024-04-26T01:42:00.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:00.344+0000 7f08de800700 1 -- 172.21.15.151:0/3602713682 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08d819f3c0 con 0x7f08d81084b0 2024-04-26T01:42:00.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:00.344+0000 7f08cfe00700 1 -- 172.21.15.151:0/3602713682 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08c8003910 con 0x7f08d81084b0 2024-04-26T01:42:00.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:00.344+0000 7f08de800700 1 -- 172.21.15.151:0/3602713682 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08d81a2d90 con 0x7f08d81084b0 2024-04-26T01:42:00.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:00.345+0000 7f08de800700 1 -- 172.21.15.151:0/3602713682 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08d8103b60 con 0x7f08d81084b0 2024-04-26T01:42:00.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:00.345+0000 7f08cfe00700 1 -- 172.21.15.151:0/3602713682 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f08c8029070 con 0x7f08d81084b0 2024-04-26T01:42:00.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:00.345+0000 7f08cfe00700 1 --2- 172.21.15.151:0/3602713682 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f08bc0396b0 0x7f08bc03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:42:00.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:00.346+0000 7f08cfe00700 1 -- 172.21.15.151:0/3602713682 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f08c801e070 con 0x7f08d81084b0 2024-04-26T01:42:00.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:00.346+0000 7f08d6a00700 1 --2- 172.21.15.151:0/3602713682 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f08bc0396b0 0x7f08bc03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:42:00.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:00.346+0000 7f08d6a00700 1 --2- 172.21.15.151:0/3602713682 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f08bc0396b0 0x7f08bc03bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f08c4006fd0 tx=0x7f08c4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:42:00.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:00.351+0000 7f08cfe00700 1 -- 172.21.15.151:0/3602713682 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f08c8013070 con 0x7f08d81084b0 2024-04-26T01:42:00.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:00.585+0000 7f08de800700 1 -- 172.21.15.151:0/3602713682 --> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi172", "target": ["mon-mgr", ""]}) -- 0x7f08d819f990 con 0x7f08bc0396b0 2024-04-26T01:42:00.582 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:00 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:00.582 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:00 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi165", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:42:00.582 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:00 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:00.583 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:00 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:42:00.583 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:00 smithi151 ceph-mon[28057]: Updating smithi165:/etc/ceph/ceph.conf 2024-04-26T01:42:00.583 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:00 smithi151 ceph-mon[28057]: Updating smithi165:/var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:42:00.583 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:00 smithi151 ceph-mon[28057]: Updating smithi165:/etc/ceph/ceph.client.admin.keyring 2024-04-26T01:42:00.583 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:00 smithi151 ceph-mon[28057]: Updating smithi165:/var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T01:42:01.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:01 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:01.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:01 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:01.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:01 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:01.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:01 smithi151 ceph-mon[28057]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi172", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:42:01.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:01 smithi151 ceph-mon[28057]: Deploying cephadm binary to smithi172 2024-04-26T01:42:01.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:01 smithi151 ceph-mon[28057]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:03.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:03 smithi151 ceph-mon[28057]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:05.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:05.197+0000 7f08cfe00700 1 -- 172.21.15.151:0/3602713682 <== mgr.14152 v2:172.21.15.151:6800/2251256505 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f08d819f990 con 0x7f08bc0396b0 2024-04-26T01:42:05.195 INFO:teuthology.orchestra.run.smithi151.stdout:Added host 'smithi172' with addr '172.21.15.172' 2024-04-26T01:42:05.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:05.200+0000 7f08de800700 1 -- 172.21.15.151:0/3602713682 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f08bc0396b0 msgr2=0x7f08bc03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:42:05.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:05.200+0000 7f08de800700 1 --2- 172.21.15.151:0/3602713682 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f08bc0396b0 0x7f08bc03bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f08c4006fd0 tx=0x7f08c4006e40 comp rx=0 tx=0).stop 2024-04-26T01:42:05.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:05.200+0000 7f08de800700 1 -- 172.21.15.151:0/3602713682 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f08d81084b0 msgr2=0x7f08d819eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:42:05.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:05.200+0000 7f08de800700 1 --2- 172.21.15.151:0/3602713682 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f08d81084b0 0x7f08d819eb90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f08c8006010 tx=0x7f08c8005780 comp rx=0 tx=0).stop 2024-04-26T01:42:05.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:05.201+0000 7f08de800700 1 -- 172.21.15.151:0/3602713682 shutdown_connections 2024-04-26T01:42:05.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:05.201+0000 7f08de800700 1 --2- 172.21.15.151:0/3602713682 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f08bc0396b0 0x7f08bc03bb70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:05.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:05.201+0000 7f08de800700 1 --2- 172.21.15.151:0/3602713682 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f08d81084b0 0x7f08d819eb90 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:05.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:05.201+0000 7f08de800700 1 -- 172.21.15.151:0/3602713682 >> 172.21.15.151:0/3602713682 conn(0x7f08d80fdf30 msgr2=0x7f08d8109fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:42:05.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:05.201+0000 7f08de800700 1 -- 172.21.15.151:0/3602713682 shutdown_connections 2024-04-26T01:42:05.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:05.201+0000 7f08de800700 1 -- 172.21.15.151:0/3602713682 wait complete. 2024-04-26T01:42:05.969 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-26T01:42:06.392 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:42:06.537 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:06 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:06.537 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:06 smithi151 ceph-mon[28057]: Added host smithi172 2024-04-26T01:42:06.538 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:06 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:42:06.538 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:06 smithi151 ceph-mon[28057]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:06.538 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:06 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:07.395 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:07 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:07.395 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:07 smithi151 ceph-mon[28057]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:08.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:08.560+0000 7f06f9800700 1 -- 172.21.15.151:0/2059929850 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f06f41084c0 msgr2=0x7f06f41088a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:42:08.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:08.560+0000 7f06f9800700 1 --2- 172.21.15.151:0/2059929850 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f06f41084c0 0x7f06f41088a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f06e4009910 tx=0x7f06e4009c20 comp rx=0 tx=0).stop 2024-04-26T01:42:08.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:08.560+0000 7f06f9800700 1 -- 172.21.15.151:0/2059929850 shutdown_connections 2024-04-26T01:42:08.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:08.560+0000 7f06f9800700 1 --2- 172.21.15.151:0/2059929850 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f06f41084c0 0x7f06f41088a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:08.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:08.560+0000 7f06f9800700 1 -- 172.21.15.151:0/2059929850 >> 172.21.15.151:0/2059929850 conn(0x7f06f40fdf60 msgr2=0x7f06f4100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:42:08.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:08.560+0000 7f06f9800700 1 -- 172.21.15.151:0/2059929850 shutdown_connections 2024-04-26T01:42:08.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:08.560+0000 7f06f9800700 1 -- 172.21.15.151:0/2059929850 wait complete. 2024-04-26T01:42:08.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:08.561+0000 7f06f9800700 1 Processor -- start 2024-04-26T01:42:08.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:08.562+0000 7f06f9800700 1 -- start start 2024-04-26T01:42:08.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:08.562+0000 7f06f9800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f06f41084c0 0x7f06f4074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:42:08.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:08.562+0000 7f06f9800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f06f410c920 con 0x7f06f41084c0 2024-04-26T01:42:08.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:08.562+0000 7f06f2a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f06f41084c0 0x7f06f4074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:42:08.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:08.562+0000 7f06f2a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f06f41084c0 0x7f06f4074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:55020/0 (socket says 172.21.15.151:55020) 2024-04-26T01:42:08.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:08.562+0000 7f06f2a00700 1 -- 172.21.15.151:0/3691909257 learned_addr learned my addr 172.21.15.151:0/3691909257 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:42:08.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:08.563+0000 7f06f2a00700 1 -- 172.21.15.151:0/3691909257 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06f4076c30 con 0x7f06f41084c0 2024-04-26T01:42:08.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:08.563+0000 7f06f2a00700 1 --2- 172.21.15.151:0/3691909257 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f06f41084c0 0x7f06f4074d40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f06e4006010 tx=0x7f06e4005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:42:08.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:08.563+0000 7f06eb400700 1 -- 172.21.15.151:0/3691909257 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06e4026070 con 0x7f06f41084c0 2024-04-26T01:42:08.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:08.563+0000 7f06eb400700 1 -- 172.21.15.151:0/3691909257 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06e400bc00 con 0x7f06f41084c0 2024-04-26T01:42:08.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:08.563+0000 7f06eb400700 1 -- 172.21.15.151:0/3691909257 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06e4003910 con 0x7f06f41084c0 2024-04-26T01:42:08.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:08.563+0000 7f06f9800700 1 -- 172.21.15.151:0/3691909257 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06f40752e0 con 0x7f06f41084c0 2024-04-26T01:42:08.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:08.563+0000 7f06f9800700 1 -- 172.21.15.151:0/3691909257 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06f40757a0 con 0x7f06f41084c0 2024-04-26T01:42:08.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:08.564+0000 7f06f9800700 1 -- 172.21.15.151:0/3691909257 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06f4103b70 con 0x7f06f41084c0 2024-04-26T01:42:08.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:08.565+0000 7f06eb400700 1 -- 172.21.15.151:0/3691909257 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f06e4029070 con 0x7f06f41084c0 2024-04-26T01:42:08.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:08.565+0000 7f06eb400700 1 --2- 172.21.15.151:0/3691909257 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f06d80396b0 0x7f06d803bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:42:08.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:08.565+0000 7f06eb400700 1 -- 172.21.15.151:0/3691909257 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f06e401e070 con 0x7f06f41084c0 2024-04-26T01:42:08.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:08.565+0000 7f06ebe00700 1 --2- 172.21.15.151:0/3691909257 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f06d80396b0 0x7f06d803bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:42:08.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:08.566+0000 7f06ebe00700 1 --2- 172.21.15.151:0/3691909257 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f06d80396b0 0x7f06d803bb70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f06dc006fd0 tx=0x7f06dc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:42:08.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:08.571+0000 7f06eb400700 1 -- 172.21.15.151:0/3691909257 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f06e4013070 con 0x7f06f41084c0 2024-04-26T01:42:08.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:08.805+0000 7f06f9800700 1 -- 172.21.15.151:0/3691909257 --> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f06f40762e0 con 0x7f06d80396b0 2024-04-26T01:42:08.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:08.807+0000 7f06eb400700 1 -- 172.21.15.151:0/3691909257 <== mgr.14152 v2:172.21.15.151:6800/2251256505 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+241 (secure 0 0 0) 0x7f06f40762e0 con 0x7f06d80396b0 2024-04-26T01:42:08.805 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:42:08.805 INFO:teuthology.orchestra.run.smithi151.stdout:[{"addr": "172.21.15.151", "hostname": "smithi151", "labels": [], "status": ""}, {"addr": "172.21.15.165", "hostname": "smithi165", "labels": [], "status": ""}, {"addr": "172.21.15.172", "hostname": "smithi172", "labels": [], "status": ""}] 2024-04-26T01:42:08.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:08.811+0000 7f06f9800700 1 -- 172.21.15.151:0/3691909257 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f06d80396b0 msgr2=0x7f06d803bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:42:08.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:08.811+0000 7f06f9800700 1 --2- 172.21.15.151:0/3691909257 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f06d80396b0 0x7f06d803bb70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f06dc006fd0 tx=0x7f06dc006e40 comp rx=0 tx=0).stop 2024-04-26T01:42:08.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:08.811+0000 7f06f9800700 1 -- 172.21.15.151:0/3691909257 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f06f41084c0 msgr2=0x7f06f4074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:42:08.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:08.811+0000 7f06f9800700 1 --2- 172.21.15.151:0/3691909257 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f06f41084c0 0x7f06f4074d40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f06e4006010 tx=0x7f06e4005780 comp rx=0 tx=0).stop 2024-04-26T01:42:08.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:08.811+0000 7f06f9800700 1 -- 172.21.15.151:0/3691909257 shutdown_connections 2024-04-26T01:42:08.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:08.811+0000 7f06f9800700 1 --2- 172.21.15.151:0/3691909257 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f06d80396b0 0x7f06d803bb70 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:08.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:08.811+0000 7f06f9800700 1 --2- 172.21.15.151:0/3691909257 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f06f41084c0 0x7f06f4074d40 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:08.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:08.811+0000 7f06f9800700 1 -- 172.21.15.151:0/3691909257 >> 172.21.15.151:0/3691909257 conn(0x7f06f40fdf60 msgr2=0x7f06f410a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:42:08.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:08.811+0000 7f06f9800700 1 -- 172.21.15.151:0/3691909257 shutdown_connections 2024-04-26T01:42:08.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:08.811+0000 7f06f9800700 1 -- 172.21.15.151:0/3691909257 wait complete. 2024-04-26T01:42:09.120 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:08 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:09.501 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-26T01:42:09.502 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-26T01:42:09.937 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:42:10.274 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:09 smithi151 ceph-mon[28057]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:42:10.274 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:09 smithi151 ceph-mon[28057]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:12.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:12.080+0000 7f3dc6600700 1 -- 172.21.15.151:0/613675567 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3dc0074e70 msgr2=0x7f3dc0075250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:42:12.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:12.080+0000 7f3dc6600700 1 --2- 172.21.15.151:0/613675567 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3dc0074e70 0x7f3dc0075250 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3db000b0a0 tx=0x7f3db000b3b0 comp rx=0 tx=0).stop 2024-04-26T01:42:12.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:12.080+0000 7f3dc6600700 1 -- 172.21.15.151:0/613675567 shutdown_connections 2024-04-26T01:42:12.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:12.080+0000 7f3dc6600700 1 --2- 172.21.15.151:0/613675567 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3dc0074e70 0x7f3dc0075250 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:12.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:12.080+0000 7f3dc6600700 1 -- 172.21.15.151:0/613675567 >> 172.21.15.151:0/613675567 conn(0x7f3dc00fdf30 msgr2=0x7f3dc0100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:42:12.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:12.080+0000 7f3dc6600700 1 -- 172.21.15.151:0/613675567 shutdown_connections 2024-04-26T01:42:12.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:12.080+0000 7f3dc6600700 1 -- 172.21.15.151:0/613675567 wait complete. 2024-04-26T01:42:12.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:12.081+0000 7f3dc6600700 1 Processor -- start 2024-04-26T01:42:12.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:12.082+0000 7f3dc6600700 1 -- start start 2024-04-26T01:42:12.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:12.082+0000 7f3dc6600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3dc0074e70 0x7f3dc019a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:42:12.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:12.082+0000 7f3dc6600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc010eb80 con 0x7f3dc0074e70 2024-04-26T01:42:12.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:12.082+0000 7f3dbf400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3dc0074e70 0x7f3dc019a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:42:12.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:12.082+0000 7f3dbf400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3dc0074e70 0x7f3dc019a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:55064/0 (socket says 172.21.15.151:55064) 2024-04-26T01:42:12.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:12.082+0000 7f3dbf400700 1 -- 172.21.15.151:0/3754291241 learned_addr learned my addr 172.21.15.151:0/3754291241 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:42:12.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:12.083+0000 7f3dbf400700 1 -- 172.21.15.151:0/3754291241 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dc019a5a0 con 0x7f3dc0074e70 2024-04-26T01:42:12.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:12.083+0000 7f3dbf400700 1 --2- 172.21.15.151:0/3754291241 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3dc0074e70 0x7f3dc019a060 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3db0015040 tx=0x7f3db0009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:42:12.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:12.083+0000 7f3db7e00700 1 -- 172.21.15.151:0/3754291241 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3db0021020 con 0x7f3dc0074e70 2024-04-26T01:42:12.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:12.083+0000 7f3db7e00700 1 -- 172.21.15.151:0/3754291241 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3db0004000 con 0x7f3dc0074e70 2024-04-26T01:42:12.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:12.083+0000 7f3dc6600700 1 -- 172.21.15.151:0/3754291241 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dc019a830 con 0x7f3dc0074e70 2024-04-26T01:42:12.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:12.083+0000 7f3db7e00700 1 -- 172.21.15.151:0/3754291241 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3db0017040 con 0x7f3dc0074e70 2024-04-26T01:42:12.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:12.084+0000 7f3dc6600700 1 -- 172.21.15.151:0/3754291241 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dc019e930 con 0x7f3dc0074e70 2024-04-26T01:42:12.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:12.085+0000 7f3db7e00700 1 -- 172.21.15.151:0/3754291241 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f3db0024020 con 0x7f3dc0074e70 2024-04-26T01:42:12.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:12.085+0000 7f3dc6600700 1 -- 172.21.15.151:0/3754291241 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dc0076520 con 0x7f3dc0074e70 2024-04-26T01:42:12.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:12.085+0000 7f3db7e00700 1 --2- 172.21.15.151:0/3754291241 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f3da40396b0 0x7f3da403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:42:12.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:12.086+0000 7f3db7e00700 1 -- 172.21.15.151:0/3754291241 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3db000f050 con 0x7f3dc0074e70 2024-04-26T01:42:12.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:12.086+0000 7f3dbea00700 1 --2- 172.21.15.151:0/3754291241 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f3da40396b0 0x7f3da403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:42:12.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:12.086+0000 7f3dbea00700 1 --2- 172.21.15.151:0/3754291241 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f3da40396b0 0x7f3da403bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3dac006fd0 tx=0x7f3dac006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:42:12.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:12.091+0000 7f3db7e00700 1 -- 172.21.15.151:0/3754291241 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3db002b020 con 0x7f3dc0074e70 2024-04-26T01:42:12.321 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:12.323+0000 7f3dc6600700 1 -- 172.21.15.151:0/3754291241 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f3dc019b3c0 con 0x7f3dc0074e70 2024-04-26T01:42:12.321 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:12 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:12.321 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:12 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:12.321 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:12 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:12.321 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:12 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:12.321 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:12 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi172", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:42:12.321 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:12 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:12.321 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:12 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:42:12.322 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:12 smithi151 ceph-mon[28057]: Updating smithi172:/etc/ceph/ceph.conf 2024-04-26T01:42:12.322 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:12 smithi151 ceph-mon[28057]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:13.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:13.328+0000 7f3db7e00700 1 -- 172.21.15.151:0/3754291241 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f3db002ee20 con 0x7f3dc0074e70 2024-04-26T01:42:13.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:13.332+0000 7f3dc6600700 1 -- 172.21.15.151:0/3754291241 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f3da40396b0 msgr2=0x7f3da403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:42:13.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:13.332+0000 7f3dc6600700 1 --2- 172.21.15.151:0/3754291241 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f3da40396b0 0x7f3da403bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3dac006fd0 tx=0x7f3dac006e40 comp rx=0 tx=0).stop 2024-04-26T01:42:13.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:13.332+0000 7f3dc6600700 1 -- 172.21.15.151:0/3754291241 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3dc0074e70 msgr2=0x7f3dc019a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:42:13.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:13.332+0000 7f3dc6600700 1 --2- 172.21.15.151:0/3754291241 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3dc0074e70 0x7f3dc019a060 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3db0015040 tx=0x7f3db0009800 comp rx=0 tx=0).stop 2024-04-26T01:42:13.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:13.332+0000 7f3dc6600700 1 -- 172.21.15.151:0/3754291241 shutdown_connections 2024-04-26T01:42:13.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:13.332+0000 7f3dc6600700 1 --2- 172.21.15.151:0/3754291241 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f3da40396b0 0x7f3da403bb70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:13.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:13.332+0000 7f3dc6600700 1 --2- 172.21.15.151:0/3754291241 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3dc0074e70 0x7f3dc019a060 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:13.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:13.332+0000 7f3dc6600700 1 -- 172.21.15.151:0/3754291241 >> 172.21.15.151:0/3754291241 conn(0x7f3dc00fdf30 msgr2=0x7f3dc01066c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:42:13.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:13.332+0000 7f3dc6600700 1 -- 172.21.15.151:0/3754291241 shutdown_connections 2024-04-26T01:42:13.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:13.332+0000 7f3dc6600700 1 -- 172.21.15.151:0/3754291241 wait complete. 2024-04-26T01:42:13.332 INFO:teuthology.orchestra.run.smithi151.stderr:adjusted tunables profile to default 2024-04-26T01:42:13.348 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:13 smithi151 ceph-mon[28057]: Updating smithi172:/var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:42:13.348 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:13 smithi151 ceph-mon[28057]: Updating smithi172:/etc/ceph/ceph.client.admin.keyring 2024-04-26T01:42:13.348 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:13 smithi151 ceph-mon[28057]: Updating smithi172:/var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T01:42:13.348 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:13 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:13.349 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:13 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:13.349 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:13 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/3754291241' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-26T01:42:13.349 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:13 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:14.115 INFO:tasks.cephadm:Adding mon.a on smithi151 2024-04-26T01:42:14.116 INFO:tasks.cephadm:Adding mon.b on smithi165 2024-04-26T01:42:14.116 INFO:tasks.cephadm:Adding mon.c on smithi172 2024-04-26T01:42:14.116 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi151:172.21.15.151=a;smithi165:172.21.15.165=b;smithi172:172.21.15.172=c' 2024-04-26T01:42:14.365 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:42:14.370 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:14 smithi151 ceph-mon[28057]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:14.370 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:14 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/3754291241' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-26T01:42:14.370 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:14 smithi151 ceph-mon[28057]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T01:42:15.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:15 smithi151 ceph-mon[28057]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:15.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:15.641+0000 7f397e2da700 1 -- 172.21.15.172:0/492631489 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39781084c0 msgr2=0x7f39781088a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:42:15.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:15.641+0000 7f397e2da700 1 --2- 172.21.15.172:0/492631489 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39781084c0 0x7f39781088a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3968009910 tx=0x7f3968009c20 comp rx=0 tx=0).stop 2024-04-26T01:42:15.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:15.641+0000 7f397e2da700 1 -- 172.21.15.172:0/492631489 shutdown_connections 2024-04-26T01:42:15.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:15.641+0000 7f397e2da700 1 --2- 172.21.15.172:0/492631489 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39781084c0 0x7f39781088a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:15.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:15.641+0000 7f397e2da700 1 -- 172.21.15.172:0/492631489 >> 172.21.15.172:0/492631489 conn(0x7f39780fdf60 msgr2=0x7f3978100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:42:15.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:15.641+0000 7f397e2da700 1 -- 172.21.15.172:0/492631489 shutdown_connections 2024-04-26T01:42:15.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:15.641+0000 7f397e2da700 1 -- 172.21.15.172:0/492631489 wait complete. 2024-04-26T01:42:15.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:15.641+0000 7f397e2da700 1 Processor -- start 2024-04-26T01:42:15.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:15.641+0000 7f397e2da700 1 -- start start 2024-04-26T01:42:15.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:15.642+0000 7f397e2da700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39781084c0 0x7f3978197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:42:15.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:15.642+0000 7f397e2da700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f397810c920 con 0x7f39781084c0 2024-04-26T01:42:15.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:15.642+0000 7f3977fff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39781084c0 0x7f3978197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:42:15.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:15.642+0000 7f3977fff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39781084c0 0x7f3978197e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.172:35566/0 (socket says 172.21.15.172:35566) 2024-04-26T01:42:15.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:15.643+0000 7f3977fff700 1 -- 172.21.15.172:0/887363327 learned_addr learned my addr 172.21.15.172:0/887363327 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-26T01:42:15.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:15.643+0000 7f3977fff700 1 -- 172.21.15.172:0/887363327 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39781983b0 con 0x7f39781084c0 2024-04-26T01:42:15.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:15.643+0000 7f3977fff700 1 --2- 172.21.15.172:0/887363327 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39781084c0 0x7f3978197e70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f39680056d0 tx=0x7f39680057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:42:15.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:15.644+0000 7f39757fa700 1 -- 172.21.15.172:0/887363327 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3968026070 con 0x7f39781084c0 2024-04-26T01:42:15.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:15.644+0000 7f397e2da700 1 -- 172.21.15.172:0/887363327 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39781986a0 con 0x7f39781084c0 2024-04-26T01:42:15.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:15.644+0000 7f397e2da700 1 -- 172.21.15.172:0/887363327 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f397819c740 con 0x7f39781084c0 2024-04-26T01:42:15.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:15.644+0000 7f39757fa700 1 -- 172.21.15.172:0/887363327 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f396800b6f0 con 0x7f39781084c0 2024-04-26T01:42:15.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:15.644+0000 7f39757fa700 1 -- 172.21.15.172:0/887363327 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3968030060 con 0x7f39781084c0 2024-04-26T01:42:15.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:15.645+0000 7f397e2da700 1 -- 172.21.15.172:0/887363327 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3964005330 con 0x7f39781084c0 2024-04-26T01:42:15.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:15.645+0000 7f39757fa700 1 -- 172.21.15.172:0/887363327 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f3968029070 con 0x7f39781084c0 2024-04-26T01:42:15.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:15.645+0000 7f39757fa700 1 --2- 172.21.15.172:0/887363327 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f39600396c0 0x7f396003bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:42:15.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:15.645+0000 7f39757fa700 1 -- 172.21.15.172:0/887363327 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f396801e070 con 0x7f39781084c0 2024-04-26T01:42:15.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:15.646+0000 7f39777fe700 1 --2- 172.21.15.172:0/887363327 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f39600396c0 0x7f396003bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:42:15.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:15.647+0000 7f39777fe700 1 --2- 172.21.15.172:0/887363327 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f39600396c0 0x7f396003bb80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f396c006fd0 tx=0x7f396c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:42:15.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:15.662+0000 7f39757fa700 1 -- 172.21.15.172:0/887363327 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3968013070 con 0x7f39781084c0 2024-04-26T01:42:15.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:15.873+0000 7f397e2da700 1 -- 172.21.15.172:0/887363327 --> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi151:172.21.15.151=a;smithi165:172.21.15.165=b;smithi172:172.21.15.172=c", "target": ["mon-mgr", ""]}) -- 0x7f3964000c80 con 0x7f39600396c0 2024-04-26T01:42:15.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:15.889+0000 7f39757fa700 1 -- 172.21.15.172:0/887363327 <== mgr.14152 v2:172.21.15.151:6800/2251256505 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f3964000c80 con 0x7f39600396c0 2024-04-26T01:42:15.886 INFO:teuthology.orchestra.run.smithi172.stdout:Scheduled mon update... 2024-04-26T01:42:15.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:15.892+0000 7f397e2da700 1 -- 172.21.15.172:0/887363327 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f39600396c0 msgr2=0x7f396003bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:42:15.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:15.892+0000 7f397e2da700 1 --2- 172.21.15.172:0/887363327 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f39600396c0 0x7f396003bb80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f396c006fd0 tx=0x7f396c006e40 comp rx=0 tx=0).stop 2024-04-26T01:42:15.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:15.892+0000 7f397e2da700 1 -- 172.21.15.172:0/887363327 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39781084c0 msgr2=0x7f3978197e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:42:15.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:15.892+0000 7f397e2da700 1 --2- 172.21.15.172:0/887363327 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39781084c0 0x7f3978197e70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f39680056d0 tx=0x7f39680057b0 comp rx=0 tx=0).stop 2024-04-26T01:42:15.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:15.892+0000 7f397e2da700 1 -- 172.21.15.172:0/887363327 shutdown_connections 2024-04-26T01:42:15.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:15.892+0000 7f397e2da700 1 --2- 172.21.15.172:0/887363327 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f39600396c0 0x7f396003bb80 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:15.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:15.892+0000 7f397e2da700 1 --2- 172.21.15.172:0/887363327 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39781084c0 0x7f3978197e70 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:15.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:15.892+0000 7f397e2da700 1 -- 172.21.15.172:0/887363327 >> 172.21.15.172:0/887363327 conn(0x7f39780fdf60 msgr2=0x7f39780fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:42:15.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:15.892+0000 7f397e2da700 1 -- 172.21.15.172:0/887363327 shutdown_connections 2024-04-26T01:42:15.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:15.892+0000 7f397e2da700 1 -- 172.21.15.172:0/887363327 wait complete. 2024-04-26T01:42:16.503 DEBUG:teuthology.orchestra.run.smithi165:mon.b> sudo journalctl -f -n 0 -u ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mon.b.service 2024-04-26T01:42:16.506 DEBUG:teuthology.orchestra.run.smithi172:mon.c> sudo journalctl -f -n 0 -u ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mon.c.service 2024-04-26T01:42:16.508 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-26T01:42:16.508 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T01:42:17.187 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:42:17.370 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:16 smithi151 ceph-mon[28057]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi151:172.21.15.151=a;smithi165:172.21.15.165=b;smithi172:172.21.15.172=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:42:17.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:16 smithi151 ceph-mon[28057]: Saving service mon spec with placement smithi151:172.21.15.151=a;smithi165:172.21.15.165=b;smithi172:172.21.15.172=c;count:3 2024-04-26T01:42:17.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:16 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:17.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:16 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:42:17.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:16 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:17.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:16 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:42:17.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:16 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:17.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:16 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T01:42:17.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:16 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:17.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:16 smithi151 ceph-mon[28057]: Deploying daemon mon.c on smithi172 2024-04-26T01:42:18.370 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:17 smithi151 ceph-mon[28057]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:19.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:19 smithi151 ceph-mon[28057]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:20.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:20.345+0000 7f659997f700 1 -- 172.21.15.172:0/2899141479 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6594108430 msgr2=0x7f6594108810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:42:20.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:20.345+0000 7f659997f700 1 --2- 172.21.15.172:0/2899141479 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6594108430 0x7f6594108810 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6588009910 tx=0x7f6588009c20 comp rx=0 tx=0).stop 2024-04-26T01:42:20.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:20.345+0000 7f659997f700 1 -- 172.21.15.172:0/2899141479 shutdown_connections 2024-04-26T01:42:20.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:20.345+0000 7f659997f700 1 --2- 172.21.15.172:0/2899141479 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6594108430 0x7f6594108810 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:20.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:20.345+0000 7f659997f700 1 -- 172.21.15.172:0/2899141479 >> 172.21.15.172:0/2899141479 conn(0x7f6594078940 msgr2=0x7f6594078d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:42:20.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:20.345+0000 7f659997f700 1 -- 172.21.15.172:0/2899141479 shutdown_connections 2024-04-26T01:42:20.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:20.345+0000 7f659997f700 1 -- 172.21.15.172:0/2899141479 wait complete. 2024-04-26T01:42:20.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:20.345+0000 7f659997f700 1 Processor -- start 2024-04-26T01:42:20.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:20.346+0000 7f659997f700 1 -- start start 2024-04-26T01:42:20.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:20.346+0000 7f659997f700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6594108430 0x7f6594197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:42:20.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:20.346+0000 7f659997f700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f659410c920 con 0x7f6594108430 2024-04-26T01:42:20.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:20.346+0000 7f6592ffd700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6594108430 0x7f6594197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:42:20.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:20.347+0000 7f6592ffd700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6594108430 0x7f6594197de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.172:45446/0 (socket says 172.21.15.172:45446) 2024-04-26T01:42:20.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:20.347+0000 7f6592ffd700 1 -- 172.21.15.172:0/1755717173 learned_addr learned my addr 172.21.15.172:0/1755717173 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-26T01:42:20.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:20.347+0000 7f6592ffd700 1 -- 172.21.15.172:0/1755717173 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6594198320 con 0x7f6594108430 2024-04-26T01:42:20.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:20.347+0000 7f6592ffd700 1 --2- 172.21.15.172:0/1755717173 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6594108430 0x7f6594197de0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f65880056d0 tx=0x7f65880057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:42:20.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:20.348+0000 7f659897d700 1 -- 172.21.15.172:0/1755717173 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6588026070 con 0x7f6594108430 2024-04-26T01:42:20.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:20.348+0000 7f659997f700 1 -- 172.21.15.172:0/1755717173 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6594198610 con 0x7f6594108430 2024-04-26T01:42:20.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:20.348+0000 7f659997f700 1 -- 172.21.15.172:0/1755717173 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f659419c6b0 con 0x7f6594108430 2024-04-26T01:42:20.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:20.348+0000 7f659897d700 1 -- 172.21.15.172:0/1755717173 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f658800bba0 con 0x7f6594108430 2024-04-26T01:42:20.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:20.348+0000 7f659897d700 1 -- 172.21.15.172:0/1755717173 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6588030060 con 0x7f6594108430 2024-04-26T01:42:20.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:20.349+0000 7f659997f700 1 -- 172.21.15.172:0/1755717173 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6580005330 con 0x7f6594108430 2024-04-26T01:42:20.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:20.349+0000 7f659897d700 1 -- 172.21.15.172:0/1755717173 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f6588029070 con 0x7f6594108430 2024-04-26T01:42:20.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:20.349+0000 7f659897d700 1 --2- 172.21.15.172:0/1755717173 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f657c039650 0x7f657c03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:42:20.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:20.349+0000 7f659897d700 1 -- 172.21.15.172:0/1755717173 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f658801e070 con 0x7f6594108430 2024-04-26T01:42:20.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:20.350+0000 7f65927fc700 1 --2- 172.21.15.172:0/1755717173 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f657c039650 0x7f657c03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:42:20.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:20.351+0000 7f65927fc700 1 --2- 172.21.15.172:0/1755717173 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f657c039650 0x7f657c03bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6584006fd0 tx=0x7f6584006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:42:20.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:20.365+0000 7f659897d700 1 -- 172.21.15.172:0/1755717173 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6588013070 con 0x7f6594108430 2024-04-26T01:42:20.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:20.660+0000 7f659997f700 1 -- 172.21.15.172:0/1755717173 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f6580005620 con 0x7f6594108430 2024-04-26T01:42:20.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:20.661+0000 7f659897d700 1 -- 172.21.15.172:0/1755717173 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+749 (secure 0 0 0) 0x7f6588029c70 con 0x7f6594108430 2024-04-26T01:42:20.658 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-26T01:42:20.658 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":1,"fsid":"c0e49e64-036d-11ef-bc93-c7b262605968","modified":"2024-04-26T01:40:03.179682Z","created":"2024-04-26T01:40:03.179682Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:3300","nonce":0},{"type":"v1","addr":"172.21.15.151:6789","nonce":0}]},"addr":"172.21.15.151:6789/0","public_addr":"172.21.15.151:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-26T01:42:20.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:20.663+0000 7f659997f700 1 -- 172.21.15.172:0/1755717173 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f657c039650 msgr2=0x7f657c03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:42:20.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:20.663+0000 7f659997f700 1 --2- 172.21.15.172:0/1755717173 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f657c039650 0x7f657c03bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6584006fd0 tx=0x7f6584006e40 comp rx=0 tx=0).stop 2024-04-26T01:42:20.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:20.663+0000 7f659997f700 1 -- 172.21.15.172:0/1755717173 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6594108430 msgr2=0x7f6594197de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:42:20.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:20.663+0000 7f659997f700 1 --2- 172.21.15.172:0/1755717173 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6594108430 0x7f6594197de0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f65880056d0 tx=0x7f65880057b0 comp rx=0 tx=0).stop 2024-04-26T01:42:20.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:20.663+0000 7f659997f700 1 -- 172.21.15.172:0/1755717173 shutdown_connections 2024-04-26T01:42:20.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:20.663+0000 7f659997f700 1 --2- 172.21.15.172:0/1755717173 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f657c039650 0x7f657c03bb10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:20.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:20.663+0000 7f659997f700 1 --2- 172.21.15.172:0/1755717173 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6594108430 0x7f6594197de0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:20.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:20.663+0000 7f659997f700 1 -- 172.21.15.172:0/1755717173 >> 172.21.15.172:0/1755717173 conn(0x7f6594078940 msgr2=0x7f65940fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:42:20.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:20.664+0000 7f659997f700 1 -- 172.21.15.172:0/1755717173 shutdown_connections 2024-04-26T01:42:20.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:20.664+0000 7f659997f700 1 -- 172.21.15.172:0/1755717173 wait complete. 2024-04-26T01:42:20.661 INFO:teuthology.orchestra.run.smithi172.stderr:dumped monmap epoch 1 2024-04-26T01:42:20.709 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: load: jerasure load: lrc 2024-04-26T01:42:20.986 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: RocksDB version: 7.9.2 2024-04-26T01:42:20.987 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Git sha 0 2024-04-26T01:42:20.987 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Compile date 2024-04-22 10:57:25 2024-04-26T01:42:20.987 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: DB SUMMARY 2024-04-26T01:42:20.987 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: DB Session ID: ANI4D2UJ3SJNN23C116L 2024-04-26T01:42:20.987 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: CURRENT file: CURRENT 2024-04-26T01:42:20.987 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: IDENTITY file: IDENTITY 2024-04-26T01:42:20.987 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-26T01:42:20.987 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-04-26T01:42:20.987 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-04-26T01:42:20.987 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.error_if_exists: 0 2024-04-26T01:42:20.987 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.create_if_missing: 0 2024-04-26T01:42:20.988 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.paranoid_checks: 1 2024-04-26T01:42:20.988 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-26T01:42:20.988 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-26T01:42:20.988 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-26T01:42:20.988 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.env: 0x56170df71cc0 2024-04-26T01:42:20.988 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.fs: PosixFileSystem 2024-04-26T01:42:20.988 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.info_log: 0x56170f26fbe0 2024-04-26T01:42:20.988 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.max_file_opening_threads: 16 2024-04-26T01:42:20.988 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.statistics: (nil) 2024-04-26T01:42:20.988 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.use_fsync: 0 2024-04-26T01:42:20.988 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.max_log_file_size: 0 2024-04-26T01:42:20.988 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-26T01:42:20.989 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-26T01:42:20.989 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.keep_log_file_num: 1000 2024-04-26T01:42:20.989 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.recycle_log_file_num: 0 2024-04-26T01:42:20.989 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.allow_fallocate: 1 2024-04-26T01:42:20.989 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.allow_mmap_reads: 0 2024-04-26T01:42:20.989 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.allow_mmap_writes: 0 2024-04-26T01:42:20.989 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.use_direct_reads: 0 2024-04-26T01:42:20.989 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-26T01:42:20.989 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.create_missing_column_families: 0 2024-04-26T01:42:20.989 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.db_log_dir: 2024-04-26T01:42:20.989 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.wal_dir: 2024-04-26T01:42:20.990 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-26T01:42:20.990 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-26T01:42:20.990 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-26T01:42:20.990 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-26T01:42:20.990 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-26T01:42:20.990 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-26T01:42:20.990 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.advise_random_on_open: 1 2024-04-26T01:42:20.990 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.db_write_buffer_size: 0 2024-04-26T01:42:20.990 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.write_buffer_manager: 0x56170f2030e0 2024-04-26T01:42:20.990 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-26T01:42:20.990 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-26T01:42:20.990 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-26T01:42:20.991 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.rate_limiter: (nil) 2024-04-26T01:42:20.991 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-26T01:42:20.991 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.wal_recovery_mode: 2 2024-04-26T01:42:20.991 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.enable_thread_tracking: 0 2024-04-26T01:42:20.991 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.enable_pipelined_write: 0 2024-04-26T01:42:20.991 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.unordered_write: 0 2024-04-26T01:42:20.991 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-26T01:42:20.991 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-26T01:42:20.991 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-26T01:42:20.991 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-26T01:42:20.991 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.row_cache: None 2024-04-26T01:42:20.991 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.wal_filter: None 2024-04-26T01:42:20.991 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-26T01:42:20.992 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.allow_ingest_behind: 0 2024-04-26T01:42:20.992 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.two_write_queues: 0 2024-04-26T01:42:20.992 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.manual_wal_flush: 0 2024-04-26T01:42:20.992 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.wal_compression: 0 2024-04-26T01:42:20.992 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.atomic_flush: 0 2024-04-26T01:42:20.992 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-26T01:42:20.992 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-26T01:42:20.992 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-26T01:42:20.992 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.log_readahead_size: 0 2024-04-26T01:42:20.992 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-26T01:42:20.992 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.best_efforts_recovery: 0 2024-04-26T01:42:20.992 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-26T01:42:20.992 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-26T01:42:20.992 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.allow_data_in_errors: 0 2024-04-26T01:42:20.993 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.db_host_id: __hostname__ 2024-04-26T01:42:20.993 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-26T01:42:20.993 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.max_background_jobs: 2 2024-04-26T01:42:20.993 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.max_background_compactions: -1 2024-04-26T01:42:20.993 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.max_subcompactions: 1 2024-04-26T01:42:20.993 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-26T01:42:20.995 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-26T01:42:20.995 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-26T01:42:20.995 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.max_total_wal_size: 0 2024-04-26T01:42:20.995 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-26T01:42:20.995 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-26T01:42:20.995 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-26T01:42:20.995 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-26T01:42:20.995 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.max_open_files: -1 2024-04-26T01:42:20.995 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.bytes_per_sync: 0 2024-04-26T01:42:20.996 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-26T01:42:20.996 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-26T01:42:20.996 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.compaction_readahead_size: 0 2024-04-26T01:42:20.996 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.max_background_flushes: -1 2024-04-26T01:42:20.996 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Compression algorithms supported: 2024-04-26T01:42:20.996 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-26T01:42:20.996 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: kZSTD supported: 0 2024-04-26T01:42:20.996 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: kXpressCompression supported: 0 2024-04-26T01:42:20.996 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: kLZ4HCCompression supported: 1 2024-04-26T01:42:20.996 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: kZlibCompression supported: 1 2024-04-26T01:42:20.996 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: kSnappyCompression supported: 1 2024-04-26T01:42:20.997 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: kLZ4Compression supported: 1 2024-04-26T01:42:20.997 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: kBZip2Compression supported: 0 2024-04-26T01:42:20.997 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-26T01:42:20.997 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-26T01:42:20.997 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-04-26T01:42:20.997 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-26T01:42:20.997 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-26T01:42:20.997 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.merge_operator: 2024-04-26T01:42:20.997 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.compaction_filter: None 2024-04-26T01:42:20.997 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.compaction_filter_factory: None 2024-04-26T01:42:20.997 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.sst_partitioner_factory: None 2024-04-26T01:42:20.997 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-26T01:42:20.997 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-26T01:42:20.997 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56170f26fdc0) 2024-04-26T01:42:20.998 INFO:journalctl@ceph.mon.c.smithi172.stdout: cache_index_and_filter_blocks: 1 2024-04-26T01:42:20.998 INFO:journalctl@ceph.mon.c.smithi172.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-26T01:42:20.998 INFO:journalctl@ceph.mon.c.smithi172.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-26T01:42:20.998 INFO:journalctl@ceph.mon.c.smithi172.stdout: pin_top_level_index_and_filter: 1 2024-04-26T01:42:20.998 INFO:journalctl@ceph.mon.c.smithi172.stdout: index_type: 0 2024-04-26T01:42:20.998 INFO:journalctl@ceph.mon.c.smithi172.stdout: data_block_index_type: 0 2024-04-26T01:42:20.998 INFO:journalctl@ceph.mon.c.smithi172.stdout: index_shortening: 1 2024-04-26T01:42:20.998 INFO:journalctl@ceph.mon.c.smithi172.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-26T01:42:20.998 INFO:journalctl@ceph.mon.c.smithi172.stdout: checksum: 4 2024-04-26T01:42:20.998 INFO:journalctl@ceph.mon.c.smithi172.stdout: no_block_cache: 0 2024-04-26T01:42:20.998 INFO:journalctl@ceph.mon.c.smithi172.stdout: block_cache: 0x56170f2351f0 2024-04-26T01:42:20.998 INFO:journalctl@ceph.mon.c.smithi172.stdout: block_cache_name: BinnedLRUCache 2024-04-26T01:42:20.998 INFO:journalctl@ceph.mon.c.smithi172.stdout: block_cache_options: 2024-04-26T01:42:20.999 INFO:journalctl@ceph.mon.c.smithi172.stdout: capacity : 536870912 2024-04-26T01:42:20.999 INFO:journalctl@ceph.mon.c.smithi172.stdout: num_shard_bits : 4 2024-04-26T01:42:20.999 INFO:journalctl@ceph.mon.c.smithi172.stdout: strict_capacity_limit : 0 2024-04-26T01:42:20.999 INFO:journalctl@ceph.mon.c.smithi172.stdout: high_pri_pool_ratio: 0.000 2024-04-26T01:42:20.999 INFO:journalctl@ceph.mon.c.smithi172.stdout: block_cache_compressed: (nil) 2024-04-26T01:42:20.999 INFO:journalctl@ceph.mon.c.smithi172.stdout: persistent_cache: (nil) 2024-04-26T01:42:20.999 INFO:journalctl@ceph.mon.c.smithi172.stdout: block_size: 4096 2024-04-26T01:42:20.999 INFO:journalctl@ceph.mon.c.smithi172.stdout: block_size_deviation: 10 2024-04-26T01:42:20.999 INFO:journalctl@ceph.mon.c.smithi172.stdout: block_restart_interval: 16 2024-04-26T01:42:20.999 INFO:journalctl@ceph.mon.c.smithi172.stdout: index_block_restart_interval: 1 2024-04-26T01:42:20.999 INFO:journalctl@ceph.mon.c.smithi172.stdout: metadata_block_size: 4096 2024-04-26T01:42:20.999 INFO:journalctl@ceph.mon.c.smithi172.stdout: partition_filters: 0 2024-04-26T01:42:20.999 INFO:journalctl@ceph.mon.c.smithi172.stdout: use_delta_encoding: 1 2024-04-26T01:42:20.999 INFO:journalctl@ceph.mon.c.smithi172.stdout: filter_policy: bloomfilter 2024-04-26T01:42:21.000 INFO:journalctl@ceph.mon.c.smithi172.stdout: whole_key_filtering: 1 2024-04-26T01:42:21.000 INFO:journalctl@ceph.mon.c.smithi172.stdout: verify_compression: 0 2024-04-26T01:42:21.000 INFO:journalctl@ceph.mon.c.smithi172.stdout: read_amp_bytes_per_bit: 0 2024-04-26T01:42:21.000 INFO:journalctl@ceph.mon.c.smithi172.stdout: format_version: 5 2024-04-26T01:42:21.000 INFO:journalctl@ceph.mon.c.smithi172.stdout: enable_index_compression: 1 2024-04-26T01:42:21.000 INFO:journalctl@ceph.mon.c.smithi172.stdout: block_align: 0 2024-04-26T01:42:21.000 INFO:journalctl@ceph.mon.c.smithi172.stdout: max_auto_readahead_size: 262144 2024-04-26T01:42:21.000 INFO:journalctl@ceph.mon.c.smithi172.stdout: prepopulate_block_cache: 0 2024-04-26T01:42:21.000 INFO:journalctl@ceph.mon.c.smithi172.stdout: initial_auto_readahead_size: 8192 2024-04-26T01:42:21.000 INFO:journalctl@ceph.mon.c.smithi172.stdout: num_file_reads_for_auto_readahead: 2 2024-04-26T01:42:21.000 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.write_buffer_size: 33554432 2024-04-26T01:42:21.000 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.max_write_buffer_number: 2 2024-04-26T01:42:21.000 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.compression: NoCompression 2024-04-26T01:42:21.000 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.bottommost_compression: Disabled 2024-04-26T01:42:21.001 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.prefix_extractor: nullptr 2024-04-26T01:42:21.001 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-26T01:42:21.001 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.num_levels: 7 2024-04-26T01:42:21.001 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-26T01:42:21.001 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-26T01:42:21.001 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-26T01:42:21.001 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-26T01:42:21.001 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-26T01:42:21.001 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-26T01:42:21.001 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-26T01:42:21.002 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-26T01:42:21.002 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-26T01:42:21.002 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-26T01:42:21.002 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-26T01:42:21.002 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-26T01:42:21.002 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-26T01:42:21.002 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.compression_opts.level: 32767 2024-04-26T01:42:21.002 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.compression_opts.strategy: 0 2024-04-26T01:42:21.002 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-26T01:42:21.002 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-26T01:42:21.002 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-26T01:42:21.002 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-26T01:42:21.002 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.compression_opts.enabled: false 2024-04-26T01:42:21.003 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-26T01:42:21.003 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-26T01:42:21.003 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-26T01:42:21.003 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-26T01:42:21.003 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.target_file_size_base: 67108864 2024-04-26T01:42:21.003 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-26T01:42:21.003 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-26T01:42:21.003 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-26T01:42:21.003 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-26T01:42:21.003 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-26T01:42:21.003 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-26T01:42:21.003 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-26T01:42:21.003 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-26T01:42:21.004 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-26T01:42:21.004 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-26T01:42:21.004 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-26T01:42:21.004 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-26T01:42:21.004 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-26T01:42:21.004 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-26T01:42:21.004 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.arena_block_size: 1048576 2024-04-26T01:42:21.004 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-26T01:42:21.004 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-26T01:42:21.004 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.disable_auto_compactions: 0 2024-04-26T01:42:21.004 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-26T01:42:21.004 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-26T01:42:21.004 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-26T01:42:21.005 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-26T01:42:21.005 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-26T01:42:21.005 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-26T01:42:21.005 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-26T01:42:21.005 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-26T01:42:21.005 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-26T01:42:21.005 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-26T01:42:21.005 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-26T01:42:21.005 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.inplace_update_support: 0 2024-04-26T01:42:21.005 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-26T01:42:21.005 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-26T01:42:21.005 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-26T01:42:21.005 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-26T01:42:21.005 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.bloom_locality: 0 2024-04-26T01:42:21.006 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.max_successive_merges: 0 2024-04-26T01:42:21.006 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-26T01:42:21.006 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.paranoid_file_checks: 0 2024-04-26T01:42:21.006 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.force_consistency_checks: 1 2024-04-26T01:42:21.006 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.report_bg_io_stats: 0 2024-04-26T01:42:21.006 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.ttl: 2592000 2024-04-26T01:42:21.006 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-26T01:42:21.006 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-26T01:42:21.006 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-26T01:42:21.006 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.enable_blob_files: false 2024-04-26T01:42:21.006 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.min_blob_size: 0 2024-04-26T01:42:21.006 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.blob_file_size: 268435456 2024-04-26T01:42:21.006 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-26T01:42:21.007 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-26T01:42:21.007 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-26T01:42:21.007 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-26T01:42:21.007 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-26T01:42:21.007 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.blob_file_starting_level: 0 2024-04-26T01:42:21.007 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-26T01:42:21.007 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-26T01:42:21.007 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-26T01:42:21.007 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 5f40b4f2-f592-4131-b028-158b36a8da35 2024-04-26T01:42:21.007 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714095740715131, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-26T01:42:21.007 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-26T01:42:21.007 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714095740715587, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714095740, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "5f40b4f2-f592-4131-b028-158b36a8da35", "db_session_id": "ANI4D2UJ3SJNN23C116L", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-26T01:42:21.008 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714095740715665, "job": 1, "event": "recovery_finished"} 2024-04-26T01:42:21.008 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-26T01:42:21.008 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-26T01:42:21.008 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56170f34a000 2024-04-26T01:42:21.008 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: DB pointer 0x56170f334000 2024-04-26T01:42:21.008 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-26T01:42:21.008 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-26T01:42:21.008 INFO:journalctl@ceph.mon.c.smithi172.stdout: ** DB Stats ** 2024-04-26T01:42:21.008 INFO:journalctl@ceph.mon.c.smithi172.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T01:42:21.008 INFO:journalctl@ceph.mon.c.smithi172.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-26T01:42:21.008 INFO:journalctl@ceph.mon.c.smithi172.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T01:42:21.008 INFO:journalctl@ceph.mon.c.smithi172.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T01:42:21.009 INFO:journalctl@ceph.mon.c.smithi172.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-26T01:42:21.009 INFO:journalctl@ceph.mon.c.smithi172.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T01:42:21.009 INFO:journalctl@ceph.mon.c.smithi172.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T01:42:21.009 INFO:journalctl@ceph.mon.c.smithi172.stdout: 2024-04-26T01:42:21.009 INFO:journalctl@ceph.mon.c.smithi172.stdout: ** Compaction Stats [default] ** 2024-04-26T01:42:21.009 INFO:journalctl@ceph.mon.c.smithi172.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T01:42:21.009 INFO:journalctl@ceph.mon.c.smithi172.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-26T01:42:21.009 INFO:journalctl@ceph.mon.c.smithi172.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-26T01:42:21.009 INFO:journalctl@ceph.mon.c.smithi172.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-26T01:42:21.009 INFO:journalctl@ceph.mon.c.smithi172.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-26T01:42:21.009 INFO:journalctl@ceph.mon.c.smithi172.stdout: 2024-04-26T01:42:21.009 INFO:journalctl@ceph.mon.c.smithi172.stdout: ** Compaction Stats [default] ** 2024-04-26T01:42:21.010 INFO:journalctl@ceph.mon.c.smithi172.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T01:42:21.010 INFO:journalctl@ceph.mon.c.smithi172.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-26T01:42:21.010 INFO:journalctl@ceph.mon.c.smithi172.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-26T01:42:21.010 INFO:journalctl@ceph.mon.c.smithi172.stdout: 2024-04-26T01:42:21.010 INFO:journalctl@ceph.mon.c.smithi172.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-26T01:42:21.010 INFO:journalctl@ceph.mon.c.smithi172.stdout: 2024-04-26T01:42:21.010 INFO:journalctl@ceph.mon.c.smithi172.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T01:42:21.010 INFO:journalctl@ceph.mon.c.smithi172.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-26T01:42:21.010 INFO:journalctl@ceph.mon.c.smithi172.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-26T01:42:21.010 INFO:journalctl@ceph.mon.c.smithi172.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-26T01:42:21.010 INFO:journalctl@ceph.mon.c.smithi172.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-26T01:42:21.010 INFO:journalctl@ceph.mon.c.smithi172.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-26T01:42:21.010 INFO:journalctl@ceph.mon.c.smithi172.stdout: Cumulative compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T01:42:21.010 INFO:journalctl@ceph.mon.c.smithi172.stdout: Interval compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T01:42:21.011 INFO:journalctl@ceph.mon.c.smithi172.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-26T01:42:21.011 INFO:journalctl@ceph.mon.c.smithi172.stdout: Block cache BinnedLRUCache@0x56170f2351f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-26T01:42:21.011 INFO:journalctl@ceph.mon.c.smithi172.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-26T01:42:21.011 INFO:journalctl@ceph.mon.c.smithi172.stdout: 2024-04-26T01:42:21.011 INFO:journalctl@ceph.mon.c.smithi172.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-26T01:42:21.011 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-04-26T01:42:21.011 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: using public_addr v2:172.21.15.172:0/0 -> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] 2024-04-26T01:42:21.011 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: starting mon.c rank -1 at public addrs [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] at bind addrs [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid c0e49e64-036d-11ef-bc93-c7b262605968 2024-04-26T01:42:21.011 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: mon.c@-1(???) e0 preinit fsid c0e49e64-036d-11ef-bc93-c7b262605968 2024-04-26T01:42:21.011 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: mon.c@-1(synchronizing).mds e1 new map 2024-04-26T01:42:21.011 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: mon.c@-1(synchronizing).mds e1 print_map 2024-04-26T01:42:21.011 INFO:journalctl@ceph.mon.c.smithi172.stdout: e1 2024-04-26T01:42:21.012 INFO:journalctl@ceph.mon.c.smithi172.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T01:42:21.012 INFO:journalctl@ceph.mon.c.smithi172.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T01:42:21.012 INFO:journalctl@ceph.mon.c.smithi172.stdout: legacy client fscid: -1 2024-04-26T01:42:21.012 INFO:journalctl@ceph.mon.c.smithi172.stdout: 2024-04-26T01:42:21.012 INFO:journalctl@ceph.mon.c.smithi172.stdout: No filesystems configured 2024-04-26T01:42:21.012 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-26T01:42:21.012 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-26T01:42:21.012 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-26T01:42:21.012 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-26T01:42:21.012 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-26T01:42:21.012 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-26T01:42:21.012 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-26T01:42:21.012 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T01:42:21.013 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T01:42:21.013 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T01:42:21.013 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:21.013 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:21.013 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:21.013 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: Added host smithi165 2024-04-26T01:42:21.013 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:42:21.013 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:21.013 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:21.013 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:21.013 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:21.013 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:42:21.013 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:21.014 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:21.014 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:21.014 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:21.014 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:21.014 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:21.014 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi165", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:42:21.014 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:21.014 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:42:21.014 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: Updating smithi165:/etc/ceph/ceph.conf 2024-04-26T01:42:21.014 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: Updating smithi165:/var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:42:21.014 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: Updating smithi165:/etc/ceph/ceph.client.admin.keyring 2024-04-26T01:42:21.014 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: Updating smithi165:/var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T01:42:21.014 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:21.015 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:21.015 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:21.015 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi172", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:42:21.015 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: Deploying cephadm binary to smithi172 2024-04-26T01:42:21.015 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:21.015 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:21.015 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:21.015 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: Added host smithi172 2024-04-26T01:42:21.015 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:42:21.015 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:21.015 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:21.015 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:21.015 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:21.016 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:21.016 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:42:21.016 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:21.016 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:21.016 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:21.016 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:21.016 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:21.016 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi172", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:42:21.016 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:21.016 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:42:21.016 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: Updating smithi172:/etc/ceph/ceph.conf 2024-04-26T01:42:21.016 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:21.017 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: Updating smithi172:/var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:42:21.017 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: Updating smithi172:/etc/ceph/ceph.client.admin.keyring 2024-04-26T01:42:21.017 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: Updating smithi172:/var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T01:42:21.017 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:21.017 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:21.017 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/3754291241' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-26T01:42:21.017 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:21.017 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:21.017 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/3754291241' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-26T01:42:21.017 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T01:42:21.017 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:21.017 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi151:172.21.15.151=a;smithi165:172.21.15.165=b;smithi172:172.21.15.172=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:42:21.017 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: Saving service mon spec with placement smithi151:172.21.15.151=a;smithi165:172.21.15.165=b;smithi172:172.21.15.172=c;count:3 2024-04-26T01:42:21.017 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:21.018 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:42:21.018 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:21.018 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:42:21.018 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:21.018 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T01:42:21.018 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:21.018 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: Deploying daemon mon.c on smithi172 2024-04-26T01:42:21.018 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:21.018 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:21.018 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: from='client.? 172.21.15.172:0/1755717173' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T01:42:21.018 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 ceph-mon[32785]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-26T01:42:21.019 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 bash[32758]: 655ff9c7ac337b12b0d565df9db911e444f5a0056a26cfc3ab6d4447cde4b325 2024-04-26T01:42:21.019 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:20 smithi172 systemd[1]: Started Ceph mon.c for c0e49e64-036d-11ef-bc93-c7b262605968. 2024-04-26T01:42:21.120 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:20 smithi151 ceph-mon[28057]: from='client.? 172.21.15.172:0/1755717173' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T01:42:22.418 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-26T01:42:22.418 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T01:42:22.661 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T01:42:26.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:25 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T01:42:26.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:25 smithi151 ceph-mon[28057]: mon.a calling monitor election 2024-04-26T01:42:26.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:25 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:42:26.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:25 smithi151 ceph-mon[28057]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:26.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:25 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:42:26.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:25 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:42:26.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:25 smithi151 ceph-mon[28057]: mon.c calling monitor election 2024-04-26T01:42:26.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:25 smithi151 ceph-mon[28057]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:26.123 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:25 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:42:26.123 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:25 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:42:26.123 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:25 smithi151 ceph-mon[28057]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:26.123 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:25 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:42:26.123 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:25 smithi151 ceph-mon[28057]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T01:42:26.123 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:25 smithi151 ceph-mon[28057]: monmap epoch 2 2024-04-26T01:42:26.123 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:25 smithi151 ceph-mon[28057]: fsid c0e49e64-036d-11ef-bc93-c7b262605968 2024-04-26T01:42:26.123 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:25 smithi151 ceph-mon[28057]: last_changed 2024-04-26T01:42:20.747989+0000 2024-04-26T01:42:26.123 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:25 smithi151 ceph-mon[28057]: created 2024-04-26T01:40:03.179682+0000 2024-04-26T01:42:26.123 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:25 smithi151 ceph-mon[28057]: min_mon_release 19 (squid) 2024-04-26T01:42:26.123 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:25 smithi151 ceph-mon[28057]: election_strategy: 1 2024-04-26T01:42:26.123 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:25 smithi151 ceph-mon[28057]: 0: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.a 2024-04-26T01:42:26.123 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:25 smithi151 ceph-mon[28057]: 1: [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] mon.c 2024-04-26T01:42:26.123 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:25 smithi151 ceph-mon[28057]: fsmap 2024-04-26T01:42:26.124 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:25 smithi151 ceph-mon[28057]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T01:42:26.124 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:25 smithi151 ceph-mon[28057]: mgrmap e13: x(active, since 60s) 2024-04-26T01:42:26.124 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:25 smithi151 ceph-mon[28057]: overall HEALTH_OK 2024-04-26T01:42:26.124 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:25 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:26.124 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:25 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:26.285 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:25 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T01:42:26.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:25 smithi172 ceph-mon[32785]: mon.a calling monitor election 2024-04-26T01:42:26.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:25 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:42:26.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:25 smithi172 ceph-mon[32785]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:26.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:25 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:42:26.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:25 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:42:26.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:25 smithi172 ceph-mon[32785]: mon.c calling monitor election 2024-04-26T01:42:26.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:25 smithi172 ceph-mon[32785]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:26.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:25 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:42:26.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:25 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:42:26.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:25 smithi172 ceph-mon[32785]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:26.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:25 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:42:26.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:25 smithi172 ceph-mon[32785]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T01:42:26.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:25 smithi172 ceph-mon[32785]: monmap epoch 2 2024-04-26T01:42:26.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:25 smithi172 ceph-mon[32785]: fsid c0e49e64-036d-11ef-bc93-c7b262605968 2024-04-26T01:42:26.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:25 smithi172 ceph-mon[32785]: last_changed 2024-04-26T01:42:20.747989+0000 2024-04-26T01:42:26.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:25 smithi172 ceph-mon[32785]: created 2024-04-26T01:40:03.179682+0000 2024-04-26T01:42:26.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:25 smithi172 ceph-mon[32785]: min_mon_release 19 (squid) 2024-04-26T01:42:26.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:25 smithi172 ceph-mon[32785]: election_strategy: 1 2024-04-26T01:42:26.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:25 smithi172 ceph-mon[32785]: 0: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.a 2024-04-26T01:42:26.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:25 smithi172 ceph-mon[32785]: 1: [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] mon.c 2024-04-26T01:42:26.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:25 smithi172 ceph-mon[32785]: fsmap 2024-04-26T01:42:26.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:25 smithi172 ceph-mon[32785]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T01:42:26.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:25 smithi172 ceph-mon[32785]: mgrmap e13: x(active, since 60s) 2024-04-26T01:42:26.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:25 smithi172 ceph-mon[32785]: overall HEALTH_OK 2024-04-26T01:42:26.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:25 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:26.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:25 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:27.035 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:26 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:27.035 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:26 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T01:42:27.035 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:26 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:27.035 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:26 smithi172 ceph-mon[32785]: Deploying daemon mon.b on smithi165 2024-04-26T01:42:27.035 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:26 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:42:27.120 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:26 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:27.120 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:26 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T01:42:27.120 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:26 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:27.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:26 smithi151 ceph-mon[28057]: Deploying daemon mon.b on smithi165 2024-04-26T01:42:27.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:26 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:42:27.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:27.181+0000 7ff74f92e700 1 -- 172.21.15.172:0/2997535747 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff7340035c0 msgr2=0x7ff734005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:42:27.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:27.181+0000 7ff74f92e700 1 --2- 172.21.15.172:0/2997535747 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff7340035c0 0x7ff734005a50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff748101360 tx=0x7ff74400feb0 comp rx=0 tx=0).stop 2024-04-26T01:42:27.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:27.181+0000 7ff74f92e700 1 -- 172.21.15.172:0/2997535747 shutdown_connections 2024-04-26T01:42:27.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:27.181+0000 7ff74f92e700 1 --2- 172.21.15.172:0/2997535747 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff7340035c0 0x7ff734005a50 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:27.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:27.181+0000 7ff74f92e700 1 -- 172.21.15.172:0/2997535747 >> 172.21.15.172:0/2997535747 conn(0x7ff7480fcf20 msgr2=0x7ff7480ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:42:27.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:27.181+0000 7ff74f92e700 1 -- 172.21.15.172:0/2997535747 shutdown_connections 2024-04-26T01:42:27.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:27.181+0000 7ff74f92e700 1 -- 172.21.15.172:0/2997535747 wait complete. 2024-04-26T01:42:27.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:27.182+0000 7ff74f92e700 1 Processor -- start 2024-04-26T01:42:27.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:27.182+0000 7ff74f92e700 1 -- start start 2024-04-26T01:42:27.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:27.182+0000 7ff74f92e700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7340035c0 0x7ff74819a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:42:27.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:27.182+0000 7ff74f92e700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff74819a740 0x7ff74819ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:42:27.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:27.182+0000 7ff74f92e700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff748109380 con 0x7ff74819a740 2024-04-26T01:42:27.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:27.182+0000 7ff74f92e700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff748109200 con 0x7ff7340035c0 2024-04-26T01:42:27.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:27.182+0000 7ff74d6ca700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7340035c0 0x7ff74819a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:42:27.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:27.183+0000 7ff74d6ca700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7340035c0 0x7ff74819a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38838/0 (socket says 172.21.15.172:38838) 2024-04-26T01:42:27.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:27.183+0000 7ff74d6ca700 1 -- 172.21.15.172:0/1366333558 learned_addr learned my addr 172.21.15.172:0/1366333558 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-26T01:42:27.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:27.183+0000 7ff74d6ca700 1 -- 172.21.15.172:0/1366333558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7340035c0 msgr2=0x7ff74819a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-26T01:42:27.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:27.183+0000 7ff74d6ca700 1 -- 172.21.15.172:0/1366333558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7340035c0 msgr2=0x7ff74819a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T01:42:27.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:27.183+0000 7ff74d6ca700 1 --2- 172.21.15.172:0/1366333558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7340035c0 0x7ff74819a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T01:42:27.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:27.183+0000 7ff74d6ca700 1 --2- 172.21.15.172:0/1366333558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7340035c0 0x7ff74819a200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T01:42:27.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:27.183+0000 7ff74cec9700 1 --2- 172.21.15.172:0/1366333558 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff74819a740 0x7ff74819ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:42:27.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:27.184+0000 7ff74cec9700 1 -- 172.21.15.172:0/1366333558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7340035c0 msgr2=0x7ff74819a200 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T01:42:27.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:27.184+0000 7ff74cec9700 1 --2- 172.21.15.172:0/1366333558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7340035c0 0x7ff74819a200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:27.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:27.184+0000 7ff74cec9700 1 -- 172.21.15.172:0/1366333558 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff74819ad80 con 0x7ff74819a740 2024-04-26T01:42:27.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:27.184+0000 7ff74cec9700 1 --2- 172.21.15.172:0/1366333558 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff74819a740 0x7ff74819ead0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff73800b4d0 tx=0x7ff73800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:42:27.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:27.184+0000 7ff73e7fc700 1 -- 172.21.15.172:0/1366333558 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7ff738011630 con 0x7ff74819a740 2024-04-26T01:42:27.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:27.184+0000 7ff74f92e700 1 -- 172.21.15.172:0/1366333558 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff74819f010 con 0x7ff74819a740 2024-04-26T01:42:27.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:27.185+0000 7ff74f92e700 1 -- 172.21.15.172:0/1366333558 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff74819f570 con 0x7ff74819a740 2024-04-26T01:42:27.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:27.185+0000 7ff73e7fc700 1 -- 172.21.15.172:0/1366333558 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff738011c80 con 0x7ff74819a740 2024-04-26T01:42:27.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:27.185+0000 7ff73e7fc700 1 -- 172.21.15.172:0/1366333558 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7ff73801adc0 con 0x7ff74819a740 2024-04-26T01:42:27.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:27.186+0000 7ff74f92e700 1 -- 172.21.15.172:0/1366333558 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff748102b70 con 0x7ff74819a740 2024-04-26T01:42:27.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:27.186+0000 7ff73e7fc700 1 -- 172.21.15.172:0/1366333558 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7ff738018020 con 0x7ff74819a740 2024-04-26T01:42:27.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:27.186+0000 7ff73e7fc700 1 --2- 172.21.15.172:0/1366333558 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7ff72c0393f0 0x7ff72c03b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:42:27.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:27.186+0000 7ff73e7fc700 1 -- 172.21.15.172:0/1366333558 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff73804c210 con 0x7ff74819a740 2024-04-26T01:42:27.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:27.187+0000 7ff74d6ca700 1 --2- 172.21.15.172:0/1366333558 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7ff72c0393f0 0x7ff72c03b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:42:27.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:27.188+0000 7ff74d6ca700 1 --2- 172.21.15.172:0/1366333558 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7ff72c0393f0 0x7ff72c03b8b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff74400fa60 tx=0x7ff744005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:42:27.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:27.201+0000 7ff73e7fc700 1 -- 172.21.15.172:0/1366333558 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff73801f020 con 0x7ff74819a740 2024-04-26T01:42:27.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:27.491+0000 7ff74f92e700 1 -- 172.21.15.172:0/1366333558 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7ff74819f880 con 0x7ff74819a740 2024-04-26T01:42:27.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:27.493+0000 7ff73e7fc700 1 -- 172.21.15.172:0/1366333558 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7ff738017dd0 con 0x7ff74819a740 2024-04-26T01:42:27.489 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-26T01:42:27.489 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":2,"fsid":"c0e49e64-036d-11ef-bc93-c7b262605968","modified":"2024-04-26T01:42:20.747989Z","created":"2024-04-26T01:40:03.179682Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:3300","nonce":0},{"type":"v1","addr":"172.21.15.151:6789","nonce":0}]},"addr":"172.21.15.151:6789/0","public_addr":"172.21.15.151:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:3300","nonce":0},{"type":"v1","addr":"172.21.15.172:6789","nonce":0}]},"addr":"172.21.15.172:6789/0","public_addr":"172.21.15.172:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-26T01:42:27.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:27.495+0000 7ff74f92e700 1 -- 172.21.15.172:0/1366333558 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7ff72c0393f0 msgr2=0x7ff72c03b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:42:27.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:27.495+0000 7ff74f92e700 1 --2- 172.21.15.172:0/1366333558 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7ff72c0393f0 0x7ff72c03b8b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff74400fa60 tx=0x7ff744005a90 comp rx=0 tx=0).stop 2024-04-26T01:42:27.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:27.495+0000 7ff74f92e700 1 -- 172.21.15.172:0/1366333558 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff74819a740 msgr2=0x7ff74819ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:42:27.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:27.495+0000 7ff74f92e700 1 --2- 172.21.15.172:0/1366333558 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff74819a740 0x7ff74819ead0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff73800b4d0 tx=0x7ff73800b890 comp rx=0 tx=0).stop 2024-04-26T01:42:27.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:27.495+0000 7ff74f92e700 1 -- 172.21.15.172:0/1366333558 shutdown_connections 2024-04-26T01:42:27.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:27.495+0000 7ff74f92e700 1 --2- 172.21.15.172:0/1366333558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7340035c0 0x7ff74819a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:27.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:27.495+0000 7ff74f92e700 1 --2- 172.21.15.172:0/1366333558 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7ff72c0393f0 0x7ff72c03b8b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:27.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:27.495+0000 7ff74f92e700 1 --2- 172.21.15.172:0/1366333558 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff74819a740 0x7ff74819ead0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:27.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:27.495+0000 7ff74f92e700 1 -- 172.21.15.172:0/1366333558 >> 172.21.15.172:0/1366333558 conn(0x7ff7480fcf20 msgr2=0x7ff74810aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:42:27.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:27.496+0000 7ff74f92e700 1 -- 172.21.15.172:0/1366333558 shutdown_connections 2024-04-26T01:42:27.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:27.496+0000 7ff74f92e700 1 -- 172.21.15.172:0/1366333558 wait complete. 2024-04-26T01:42:27.493 INFO:teuthology.orchestra.run.smithi172.stderr:dumped monmap epoch 2 2024-04-26T01:42:28.120 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:27 smithi151 ceph-mon[28057]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:28.120 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:27 smithi151 ceph-mon[28057]: from='client.? 172.21.15.172:0/1366333558' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T01:42:28.285 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:27 smithi172 ceph-mon[32785]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:28.285 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:27 smithi172 ceph-mon[32785]: from='client.? 172.21.15.172:0/1366333558' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T01:42:29.050 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-26T01:42:29.051 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T01:42:29.294 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T01:42:29.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:29 smithi151 ceph-mon[28057]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:29.649 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:29 smithi172 ceph-mon[32785]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:30.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:29 smithi165 systemd[1]: Started Ceph mon.b for c0e49e64-036d-11ef-bc93-c7b262605968. 2024-04-26T01:42:35.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:34 smithi151 ceph-mon[28057]: mon.c calling monitor election 2024-04-26T01:42:35.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:34 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T01:42:35.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:34 smithi151 ceph-mon[28057]: mon.a calling monitor election 2024-04-26T01:42:35.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:34 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:42:35.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:34 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:42:35.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:34 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:42:35.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:34 smithi151 ceph-mon[28057]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:35.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:34 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:42:35.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:34 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:42:35.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:34 smithi151 ceph-mon[28057]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:35.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:34 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:42:35.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:34 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:42:35.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:34 smithi151 ceph-mon[28057]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T01:42:35.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:34 smithi151 ceph-mon[28057]: monmap epoch 3 2024-04-26T01:42:35.123 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:34 smithi151 ceph-mon[28057]: fsid c0e49e64-036d-11ef-bc93-c7b262605968 2024-04-26T01:42:35.123 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:34 smithi151 ceph-mon[28057]: last_changed 2024-04-26T01:42:29.750809+0000 2024-04-26T01:42:35.123 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:34 smithi151 ceph-mon[28057]: created 2024-04-26T01:40:03.179682+0000 2024-04-26T01:42:35.123 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:34 smithi151 ceph-mon[28057]: min_mon_release 19 (squid) 2024-04-26T01:42:35.123 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:34 smithi151 ceph-mon[28057]: election_strategy: 1 2024-04-26T01:42:35.123 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:34 smithi151 ceph-mon[28057]: 0: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.a 2024-04-26T01:42:35.123 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:34 smithi151 ceph-mon[28057]: 1: [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] mon.c 2024-04-26T01:42:35.123 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:34 smithi151 ceph-mon[28057]: 2: [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] mon.b 2024-04-26T01:42:35.123 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:34 smithi151 ceph-mon[28057]: fsmap 2024-04-26T01:42:35.123 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:34 smithi151 ceph-mon[28057]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T01:42:35.123 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:34 smithi151 ceph-mon[28057]: mgrmap e13: x(active, since 69s) 2024-04-26T01:42:35.123 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:34 smithi151 ceph-mon[28057]: overall HEALTH_OK 2024-04-26T01:42:35.123 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:34 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:35.123 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:34 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:35.123 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:34 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:35.124 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:34 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:35.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:35.147+0000 7fa283f2c700 1 -- 172.21.15.172:0/3436796400 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa27c108630 msgr2=0x7fa268005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:42:35.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:35.147+0000 7fa283f2c700 1 --2- 172.21.15.172:0/3436796400 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa27c108630 0x7fa268005610 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa26c005fa0 tx=0x7fa26c00ff70 comp rx=0 tx=0).stop 2024-04-26T01:42:35.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:35.148+0000 7fa283f2c700 1 -- 172.21.15.172:0/3436796400 shutdown_connections 2024-04-26T01:42:35.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:35.148+0000 7fa283f2c700 1 --2- 172.21.15.172:0/3436796400 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa27c108630 0x7fa268005610 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:35.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:35.148+0000 7fa283f2c700 1 -- 172.21.15.172:0/3436796400 >> 172.21.15.172:0/3436796400 conn(0x7fa27c0fe050 msgr2=0x7fa27c100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:42:35.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:35.148+0000 7fa283f2c700 1 -- 172.21.15.172:0/3436796400 shutdown_connections 2024-04-26T01:42:35.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:35.148+0000 7fa283f2c700 1 -- 172.21.15.172:0/3436796400 wait complete. 2024-04-26T01:42:35.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:35.149+0000 7fa283f2c700 1 Processor -- start 2024-04-26T01:42:35.147 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:34 smithi172 ceph-mon[32785]: mon.c calling monitor election 2024-04-26T01:42:35.147 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:34 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T01:42:35.147 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:34 smithi172 ceph-mon[32785]: mon.a calling monitor election 2024-04-26T01:42:35.147 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:34 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:42:35.147 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:34 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:42:35.147 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:34 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:42:35.147 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:34 smithi172 ceph-mon[32785]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:35.147 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:34 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:42:35.148 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:34 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:42:35.148 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:34 smithi172 ceph-mon[32785]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:35.148 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:34 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:42:35.148 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:34 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:42:35.148 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:34 smithi172 ceph-mon[32785]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T01:42:35.148 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:34 smithi172 ceph-mon[32785]: monmap epoch 3 2024-04-26T01:42:35.148 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:34 smithi172 ceph-mon[32785]: fsid c0e49e64-036d-11ef-bc93-c7b262605968 2024-04-26T01:42:35.148 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:34 smithi172 ceph-mon[32785]: last_changed 2024-04-26T01:42:29.750809+0000 2024-04-26T01:42:35.148 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:34 smithi172 ceph-mon[32785]: created 2024-04-26T01:40:03.179682+0000 2024-04-26T01:42:35.148 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:34 smithi172 ceph-mon[32785]: min_mon_release 19 (squid) 2024-04-26T01:42:35.148 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:34 smithi172 ceph-mon[32785]: election_strategy: 1 2024-04-26T01:42:35.148 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:34 smithi172 ceph-mon[32785]: 0: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.a 2024-04-26T01:42:35.148 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:34 smithi172 ceph-mon[32785]: 1: [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] mon.c 2024-04-26T01:42:35.148 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:34 smithi172 ceph-mon[32785]: 2: [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] mon.b 2024-04-26T01:42:35.149 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:34 smithi172 ceph-mon[32785]: fsmap 2024-04-26T01:42:35.149 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:34 smithi172 ceph-mon[32785]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T01:42:35.149 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:34 smithi172 ceph-mon[32785]: mgrmap e13: x(active, since 69s) 2024-04-26T01:42:35.149 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:34 smithi172 ceph-mon[32785]: overall HEALTH_OK 2024-04-26T01:42:35.149 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:34 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:35.149 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:34 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:35.149 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:34 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:35.149 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:34 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:35.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:35.149+0000 7fa283f2c700 1 -- start start 2024-04-26T01:42:35.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:35.149+0000 7fa283f2c700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa27c108630 0x7fa27c198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:42:35.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:35.149+0000 7fa283f2c700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa27c198550 0x7fa27c19c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:42:35.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:35.149+0000 7fa283f2c700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa27c19ce20 0x7fa27c19d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:42:35.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:35.150+0000 7fa283f2c700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa27c10a460 con 0x7fa27c198550 2024-04-26T01:42:35.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:35.150+0000 7fa283f2c700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa27c10a2e0 con 0x7fa27c19ce20 2024-04-26T01:42:35.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:35.150+0000 7fa283f2c700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa27c10a5e0 con 0x7fa27c108630 2024-04-26T01:42:35.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:35.150+0000 7fa281cc8700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa27c108630 0x7fa27c198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:42:35.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:35.150+0000 7fa281cc8700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa27c108630 0x7fa27c198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38842/0 (socket says 172.21.15.172:38842) 2024-04-26T01:42:35.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:35.150+0000 7fa281cc8700 1 -- 172.21.15.172:0/2305251607 learned_addr learned my addr 172.21.15.172:0/2305251607 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-26T01:42:35.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:35.150+0000 7fa2814c7700 1 --2- 172.21.15.172:0/2305251607 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa27c198550 0x7fa27c19c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:42:35.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:35.150+0000 7fa2824c9700 1 --2- 172.21.15.172:0/2305251607 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa27c19ce20 0x7fa27c19d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:42:35.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:35.150+0000 7fa281cc8700 1 -- 172.21.15.172:0/2305251607 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa27c108630 msgr2=0x7fa27c198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-26T01:42:35.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:35.150+0000 7fa281cc8700 1 -- 172.21.15.172:0/2305251607 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa27c108630 msgr2=0x7fa27c198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T01:42:35.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:35.150+0000 7fa281cc8700 1 --2- 172.21.15.172:0/2305251607 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa27c108630 0x7fa27c198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T01:42:35.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:35.150+0000 7fa281cc8700 1 --2- 172.21.15.172:0/2305251607 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa27c108630 0x7fa27c198010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T01:42:35.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:35.150+0000 7fa2824c9700 1 -- 172.21.15.172:0/2305251607 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa27c19ce20 msgr2=0x7fa27c19d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-04-26T01:42:35.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:35.150+0000 7fa2824c9700 1 -- 172.21.15.172:0/2305251607 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa27c19ce20 msgr2=0x7fa27c19d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T01:42:35.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:35.150+0000 7fa2824c9700 1 --2- 172.21.15.172:0/2305251607 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa27c19ce20 0x7fa27c19d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T01:42:35.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:35.150+0000 7fa2824c9700 1 --2- 172.21.15.172:0/2305251607 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa27c19ce20 0x7fa27c19d2d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T01:42:35.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:35.150+0000 7fa2814c7700 1 -- 172.21.15.172:0/2305251607 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa27c108630 msgr2=0x7fa27c198010 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T01:42:35.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:35.150+0000 7fa2814c7700 1 --2- 172.21.15.172:0/2305251607 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa27c108630 0x7fa27c198010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:35.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:35.150+0000 7fa2814c7700 1 -- 172.21.15.172:0/2305251607 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa27c19ce20 msgr2=0x7fa27c19d2d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T01:42:35.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:35.150+0000 7fa2814c7700 1 --2- 172.21.15.172:0/2305251607 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa27c19ce20 0x7fa27c19d2d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:35.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:35.150+0000 7fa2814c7700 1 -- 172.21.15.172:0/2305251607 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa27c1a4b90 con 0x7fa27c198550 2024-04-26T01:42:35.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:35.151+0000 7fa2814c7700 1 --2- 172.21.15.172:0/2305251607 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa27c198550 0x7fa27c19c8e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa27800b550 tx=0x7fa27800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:42:35.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:35.151+0000 7fa272ffd700 1 -- 172.21.15.172:0/2305251607 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa278004230 con 0x7fa27c198550 2024-04-26T01:42:35.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:35.151+0000 7fa283f2c700 1 -- 172.21.15.172:0/2305251607 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa27c1a4dc0 con 0x7fa27c198550 2024-04-26T01:42:35.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:35.151+0000 7fa272ffd700 1 -- 172.21.15.172:0/2305251607 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2780043d0 con 0x7fa27c198550 2024-04-26T01:42:35.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:35.151+0000 7fa272ffd700 1 -- 172.21.15.172:0/2305251607 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa278011b90 con 0x7fa27c198550 2024-04-26T01:42:35.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:35.151+0000 7fa283f2c700 1 -- 172.21.15.172:0/2305251607 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa27c1a5300 con 0x7fa27c198550 2024-04-26T01:42:35.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:35.152+0000 7fa283f2c700 1 -- 172.21.15.172:0/2305251607 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa27c0640a0 con 0x7fa27c198550 2024-04-26T01:42:35.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:35.152+0000 7fa272ffd700 1 -- 172.21.15.172:0/2305251607 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7fa2780036a0 con 0x7fa27c198550 2024-04-26T01:42:35.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:35.153+0000 7fa272ffd700 1 --2- 172.21.15.172:0/2305251607 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fa260039ae0 0x7fa26003bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:42:35.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:35.153+0000 7fa272ffd700 1 -- 172.21.15.172:0/2305251607 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa27804c3a0 con 0x7fa27c198550 2024-04-26T01:42:35.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:35.153+0000 7fa281cc8700 1 --2- 172.21.15.172:0/2305251607 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fa260039ae0 0x7fa26003bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:42:35.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:35.154+0000 7fa281cc8700 1 --2- 172.21.15.172:0/2305251607 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fa260039ae0 0x7fa26003bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa26c00f470 tx=0x7fa26c00f3e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:42:35.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:35.160+0000 7fa272ffd700 1 -- 172.21.15.172:0/2305251607 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa278010380 con 0x7fa27c198550 2024-04-26T01:42:35.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:35.455+0000 7fa283f2c700 1 -- 172.21.15.172:0/2305251607 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fa27c1a56a0 con 0x7fa27c198550 2024-04-26T01:42:35.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:35.457+0000 7fa272ffd700 1 -- 172.21.15.172:0/2305251607 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1287 (secure 0 0 0) 0x7fa278003990 con 0x7fa27c198550 2024-04-26T01:42:35.454 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-26T01:42:35.454 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":3,"fsid":"c0e49e64-036d-11ef-bc93-c7b262605968","modified":"2024-04-26T01:42:29.750809Z","created":"2024-04-26T01:40:03.179682Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:3300","nonce":0},{"type":"v1","addr":"172.21.15.151:6789","nonce":0}]},"addr":"172.21.15.151:6789/0","public_addr":"172.21.15.151:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:3300","nonce":0},{"type":"v1","addr":"172.21.15.172:6789","nonce":0}]},"addr":"172.21.15.172:6789/0","public_addr":"172.21.15.172:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:3300","nonce":0},{"type":"v1","addr":"172.21.15.165:6789","nonce":0}]},"addr":"172.21.15.165:6789/0","public_addr":"172.21.15.165:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-26T01:42:35.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:35.460+0000 7fa283f2c700 1 -- 172.21.15.172:0/2305251607 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fa260039ae0 msgr2=0x7fa26003bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:42:35.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:35.460+0000 7fa283f2c700 1 --2- 172.21.15.172:0/2305251607 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fa260039ae0 0x7fa26003bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa26c00f470 tx=0x7fa26c00f3e0 comp rx=0 tx=0).stop 2024-04-26T01:42:35.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:35.460+0000 7fa283f2c700 1 -- 172.21.15.172:0/2305251607 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa27c198550 msgr2=0x7fa27c19c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:42:35.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:35.460+0000 7fa283f2c700 1 --2- 172.21.15.172:0/2305251607 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa27c198550 0x7fa27c19c8e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa27800b550 tx=0x7fa27800b910 comp rx=0 tx=0).stop 2024-04-26T01:42:35.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:35.460+0000 7fa283f2c700 1 -- 172.21.15.172:0/2305251607 shutdown_connections 2024-04-26T01:42:35.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:35.460+0000 7fa283f2c700 1 --2- 172.21.15.172:0/2305251607 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa27c19ce20 0x7fa27c19d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:35.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:35.460+0000 7fa283f2c700 1 --2- 172.21.15.172:0/2305251607 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa27c108630 0x7fa27c198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:35.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:35.460+0000 7fa283f2c700 1 --2- 172.21.15.172:0/2305251607 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fa260039ae0 0x7fa26003bfa0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:35.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:35.460+0000 7fa283f2c700 1 --2- 172.21.15.172:0/2305251607 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa27c198550 0x7fa27c19c8e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:35.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:35.460+0000 7fa283f2c700 1 -- 172.21.15.172:0/2305251607 >> 172.21.15.172:0/2305251607 conn(0x7fa27c0fe050 msgr2=0x7fa27c0fff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:42:35.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:35.460+0000 7fa283f2c700 1 -- 172.21.15.172:0/2305251607 shutdown_connections 2024-04-26T01:42:35.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:35.460+0000 7fa283f2c700 1 -- 172.21.15.172:0/2305251607 wait complete. 2024-04-26T01:42:35.459 INFO:teuthology.orchestra.run.smithi172.stderr:dumped monmap epoch 3 2024-04-26T01:42:36.035 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:35 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:42:36.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:35 smithi172 ceph-mon[32785]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:36.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:35 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:36.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:35 smithi172 ceph-mon[32785]: from='client.? 172.21.15.172:0/2305251607' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T01:42:36.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:35 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:42:36.039 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-26T01:42:36.039 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-26T01:42:36.101 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:35 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:42:36.101 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:35 smithi151 ceph-mon[28057]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:36.101 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:35 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:36.101 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:35 smithi151 ceph-mon[28057]: from='client.? 172.21.15.172:0/2305251607' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T01:42:36.101 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:35 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:42:36.464 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:42:37.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:36 smithi165 ceph-mon[32648]: mon.c calling monitor election 2024-04-26T01:42:37.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:36 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T01:42:37.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:36 smithi165 ceph-mon[32648]: mon.a calling monitor election 2024-04-26T01:42:37.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:36 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:42:37.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:36 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:42:37.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:36 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:42:37.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:36 smithi165 ceph-mon[32648]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:37.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:36 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:42:37.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:36 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:42:37.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:36 smithi165 ceph-mon[32648]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:37.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:36 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:42:37.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:36 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:42:37.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:36 smithi165 ceph-mon[32648]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T01:42:37.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:36 smithi165 ceph-mon[32648]: monmap epoch 3 2024-04-26T01:42:37.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:36 smithi165 ceph-mon[32648]: fsid c0e49e64-036d-11ef-bc93-c7b262605968 2024-04-26T01:42:37.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:36 smithi165 ceph-mon[32648]: last_changed 2024-04-26T01:42:29.750809+0000 2024-04-26T01:42:37.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:36 smithi165 ceph-mon[32648]: created 2024-04-26T01:40:03.179682+0000 2024-04-26T01:42:37.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:36 smithi165 ceph-mon[32648]: min_mon_release 19 (squid) 2024-04-26T01:42:37.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:36 smithi165 ceph-mon[32648]: election_strategy: 1 2024-04-26T01:42:37.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:36 smithi165 ceph-mon[32648]: 0: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.a 2024-04-26T01:42:37.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:36 smithi165 ceph-mon[32648]: 1: [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] mon.c 2024-04-26T01:42:37.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:36 smithi165 ceph-mon[32648]: 2: [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] mon.b 2024-04-26T01:42:37.097 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:36 smithi165 ceph-mon[32648]: fsmap 2024-04-26T01:42:37.097 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:36 smithi165 ceph-mon[32648]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T01:42:37.097 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:36 smithi165 ceph-mon[32648]: mgrmap e13: x(active, since 69s) 2024-04-26T01:42:37.097 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:36 smithi165 ceph-mon[32648]: overall HEALTH_OK 2024-04-26T01:42:37.097 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:36 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:37.097 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:36 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:37.097 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:36 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:37.097 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:36 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:37.097 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:36 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:42:37.097 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:36 smithi165 ceph-mon[32648]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:37.097 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:36 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:37.097 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:36 smithi165 ceph-mon[32648]: from='client.? 172.21.15.172:0/2305251607' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T01:42:37.097 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:36 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:42:37.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:37 smithi165 ceph-mon[32648]: mon.b calling monitor election 2024-04-26T01:42:37.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:37 smithi165 ceph-mon[32648]: mon.b calling monitor election 2024-04-26T01:42:37.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:37 smithi165 ceph-mon[32648]: mon.c calling monitor election 2024-04-26T01:42:37.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:37 smithi165 ceph-mon[32648]: mon.a calling monitor election 2024-04-26T01:42:37.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:37 smithi165 ceph-mon[32648]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-26T01:42:37.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:37 smithi165 ceph-mon[32648]: monmap epoch 3 2024-04-26T01:42:37.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:37 smithi165 ceph-mon[32648]: fsid c0e49e64-036d-11ef-bc93-c7b262605968 2024-04-26T01:42:37.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:37 smithi165 ceph-mon[32648]: last_changed 2024-04-26T01:42:29.750809+0000 2024-04-26T01:42:37.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:37 smithi165 ceph-mon[32648]: created 2024-04-26T01:40:03.179682+0000 2024-04-26T01:42:37.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:37 smithi165 ceph-mon[32648]: min_mon_release 19 (squid) 2024-04-26T01:42:37.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:37 smithi165 ceph-mon[32648]: election_strategy: 1 2024-04-26T01:42:37.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:37 smithi165 ceph-mon[32648]: 0: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.a 2024-04-26T01:42:37.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:37 smithi165 ceph-mon[32648]: 1: [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] mon.c 2024-04-26T01:42:37.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:37 smithi165 ceph-mon[32648]: 2: [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] mon.b 2024-04-26T01:42:37.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:37 smithi165 ceph-mon[32648]: fsmap 2024-04-26T01:42:37.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:37 smithi165 ceph-mon[32648]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T01:42:37.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:37 smithi165 ceph-mon[32648]: mgrmap e13: x(active, since 71s) 2024-04-26T01:42:37.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:37 smithi165 ceph-mon[32648]: overall HEALTH_OK 2024-04-26T01:42:37.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:37 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:37.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:37 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:37.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:37 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:37.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:37 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:42:37.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:37 smithi165 ceph-mon[32648]: Updating smithi151:/etc/ceph/ceph.conf 2024-04-26T01:42:37.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:37 smithi165 ceph-mon[32648]: Updating smithi165:/etc/ceph/ceph.conf 2024-04-26T01:42:37.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:37 smithi165 ceph-mon[32648]: Updating smithi172:/etc/ceph/ceph.conf 2024-04-26T01:42:37.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:37 smithi165 ceph-mon[32648]: Updating smithi172:/var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:42:37.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:37 smithi165 ceph-mon[32648]: Updating smithi165:/var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:42:37.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:37 smithi165 ceph-mon[32648]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:37.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:37 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:37.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:37 smithi151 ceph-mon[28057]: mon.b calling monitor election 2024-04-26T01:42:37.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:37 smithi151 ceph-mon[28057]: mon.b calling monitor election 2024-04-26T01:42:37.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:37 smithi151 ceph-mon[28057]: mon.c calling monitor election 2024-04-26T01:42:37.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:37 smithi151 ceph-mon[28057]: mon.a calling monitor election 2024-04-26T01:42:37.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:37 smithi151 ceph-mon[28057]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-26T01:42:37.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:37 smithi151 ceph-mon[28057]: monmap epoch 3 2024-04-26T01:42:37.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:37 smithi151 ceph-mon[28057]: fsid c0e49e64-036d-11ef-bc93-c7b262605968 2024-04-26T01:42:37.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:37 smithi151 ceph-mon[28057]: last_changed 2024-04-26T01:42:29.750809+0000 2024-04-26T01:42:37.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:37 smithi151 ceph-mon[28057]: created 2024-04-26T01:40:03.179682+0000 2024-04-26T01:42:37.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:37 smithi151 ceph-mon[28057]: min_mon_release 19 (squid) 2024-04-26T01:42:37.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:37 smithi151 ceph-mon[28057]: election_strategy: 1 2024-04-26T01:42:37.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:37 smithi151 ceph-mon[28057]: 0: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.a 2024-04-26T01:42:37.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:37 smithi151 ceph-mon[28057]: 1: [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] mon.c 2024-04-26T01:42:37.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:37 smithi151 ceph-mon[28057]: 2: [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] mon.b 2024-04-26T01:42:37.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:37 smithi151 ceph-mon[28057]: fsmap 2024-04-26T01:42:37.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:37 smithi151 ceph-mon[28057]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T01:42:37.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:37 smithi151 ceph-mon[28057]: mgrmap e13: x(active, since 71s) 2024-04-26T01:42:37.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:37 smithi151 ceph-mon[28057]: overall HEALTH_OK 2024-04-26T01:42:37.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:37 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:37.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:37 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:37.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:37 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:37.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:37 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:42:37.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:37 smithi151 ceph-mon[28057]: Updating smithi151:/etc/ceph/ceph.conf 2024-04-26T01:42:37.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:37 smithi151 ceph-mon[28057]: Updating smithi165:/etc/ceph/ceph.conf 2024-04-26T01:42:37.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:37 smithi151 ceph-mon[28057]: Updating smithi172:/etc/ceph/ceph.conf 2024-04-26T01:42:37.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:37 smithi151 ceph-mon[28057]: Updating smithi172:/var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:42:37.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:37 smithi151 ceph-mon[28057]: Updating smithi165:/var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:42:37.623 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:37 smithi151 ceph-mon[28057]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:37.623 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:37 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:37.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:37 smithi172 ceph-mon[32785]: mon.b calling monitor election 2024-04-26T01:42:37.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:37 smithi172 ceph-mon[32785]: mon.b calling monitor election 2024-04-26T01:42:37.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:37 smithi172 ceph-mon[32785]: mon.c calling monitor election 2024-04-26T01:42:37.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:37 smithi172 ceph-mon[32785]: mon.a calling monitor election 2024-04-26T01:42:37.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:37 smithi172 ceph-mon[32785]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-26T01:42:37.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:37 smithi172 ceph-mon[32785]: monmap epoch 3 2024-04-26T01:42:37.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:37 smithi172 ceph-mon[32785]: fsid c0e49e64-036d-11ef-bc93-c7b262605968 2024-04-26T01:42:37.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:37 smithi172 ceph-mon[32785]: last_changed 2024-04-26T01:42:29.750809+0000 2024-04-26T01:42:37.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:37 smithi172 ceph-mon[32785]: created 2024-04-26T01:40:03.179682+0000 2024-04-26T01:42:37.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:37 smithi172 ceph-mon[32785]: min_mon_release 19 (squid) 2024-04-26T01:42:37.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:37 smithi172 ceph-mon[32785]: election_strategy: 1 2024-04-26T01:42:37.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:37 smithi172 ceph-mon[32785]: 0: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.a 2024-04-26T01:42:37.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:37 smithi172 ceph-mon[32785]: 1: [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] mon.c 2024-04-26T01:42:37.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:37 smithi172 ceph-mon[32785]: 2: [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] mon.b 2024-04-26T01:42:37.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:37 smithi172 ceph-mon[32785]: fsmap 2024-04-26T01:42:37.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:37 smithi172 ceph-mon[32785]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T01:42:37.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:37 smithi172 ceph-mon[32785]: mgrmap e13: x(active, since 71s) 2024-04-26T01:42:37.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:37 smithi172 ceph-mon[32785]: overall HEALTH_OK 2024-04-26T01:42:37.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:37 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:37.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:37 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:37.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:37 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:37.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:37 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:42:37.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:37 smithi172 ceph-mon[32785]: Updating smithi151:/etc/ceph/ceph.conf 2024-04-26T01:42:37.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:37 smithi172 ceph-mon[32785]: Updating smithi165:/etc/ceph/ceph.conf 2024-04-26T01:42:37.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:37 smithi172 ceph-mon[32785]: Updating smithi172:/etc/ceph/ceph.conf 2024-04-26T01:42:37.788 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:37 smithi172 ceph-mon[32785]: Updating smithi172:/var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:42:37.788 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:37 smithi172 ceph-mon[32785]: Updating smithi165:/var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:42:37.788 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:37 smithi172 ceph-mon[32785]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:37.788 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:37 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:38.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:38.545+0000 7fa05da00700 1 -- 172.21.15.151:0/3258197797 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0581084b0 msgr2=0x7fa058108890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:42:38.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:38.545+0000 7fa05da00700 1 --2- 172.21.15.151:0/3258197797 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0581084b0 0x7fa058108890 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa048009910 tx=0x7fa048009c20 comp rx=0 tx=0).stop 2024-04-26T01:42:38.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:38.545+0000 7fa05da00700 1 -- 172.21.15.151:0/3258197797 shutdown_connections 2024-04-26T01:42:38.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:38.545+0000 7fa05da00700 1 --2- 172.21.15.151:0/3258197797 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0581084b0 0x7fa058108890 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:38.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:38.545+0000 7fa05da00700 1 -- 172.21.15.151:0/3258197797 >> 172.21.15.151:0/3258197797 conn(0x7fa0580fdf30 msgr2=0x7fa058100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:42:38.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:38.546+0000 7fa05da00700 1 -- 172.21.15.151:0/3258197797 shutdown_connections 2024-04-26T01:42:38.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:38.546+0000 7fa05da00700 1 -- 172.21.15.151:0/3258197797 wait complete. 2024-04-26T01:42:38.544 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:38 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:38.544 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:38 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:38.545 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:38 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:38.545 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:38 smithi151 ceph-mon[28057]: Updating smithi151:/var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:42:38.545 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:38 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:42:38.545 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:38 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:38.545 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:38 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:38.545 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:38 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:38.545 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:38 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:38.545 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:38 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:38.545 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:38 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:38.545 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:38 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:38.545 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:38 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T01:42:38.545 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:38 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T01:42:38.545 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:38 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:38.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:38.547+0000 7fa05da00700 1 Processor -- start 2024-04-26T01:42:38.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:38.547+0000 7fa05da00700 1 -- start start 2024-04-26T01:42:38.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:38.547+0000 7fa05da00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0581084b0 0x7fa058198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:42:38.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:38.547+0000 7fa05da00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa058198660 0x7fa05819c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:42:38.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:38.548+0000 7fa05da00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa05819cf30 0x7fa05819d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:42:38.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:38.548+0000 7fa05da00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa05810c910 con 0x7fa058198660 2024-04-26T01:42:38.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:38.548+0000 7fa05da00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa05810c790 con 0x7fa0581084b0 2024-04-26T01:42:38.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:38.548+0000 7fa05da00700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa05810ca90 con 0x7fa05819cf30 2024-04-26T01:42:38.548 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:38.548+0000 7fa04fe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa058198660 0x7fa05819c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:42:38.548 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:38.548+0000 7fa056a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0581084b0 0x7fa058198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:42:38.548 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:38.548+0000 7fa056a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0581084b0 0x7fa058198120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:37410/0 (socket says 172.21.15.151:37410) 2024-04-26T01:42:38.548 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:38.548+0000 7fa056a00700 1 -- 172.21.15.151:0/332669224 learned_addr learned my addr 172.21.15.151:0/332669224 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:42:38.548 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:38.548+0000 7fa04fe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa058198660 0x7fa05819c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:46374/0 (socket says 172.21.15.151:46374) 2024-04-26T01:42:38.548 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:38.548+0000 7fa057400700 1 --2- 172.21.15.151:0/332669224 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa05819cf30 0x7fa05819d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:42:38.548 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:38.548+0000 7fa056a00700 1 -- 172.21.15.151:0/332669224 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa05819cf30 msgr2=0x7fa05819d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:42:38.548 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:38.548+0000 7fa056a00700 1 --2- 172.21.15.151:0/332669224 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa05819cf30 0x7fa05819d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:38.548 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:38.548+0000 7fa056a00700 1 -- 172.21.15.151:0/332669224 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa058198660 msgr2=0x7fa05819c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:42:38.548 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:38.548+0000 7fa056a00700 1 --2- 172.21.15.151:0/332669224 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa058198660 0x7fa05819c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:38.548 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:38.548+0000 7fa056a00700 1 -- 172.21.15.151:0/332669224 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0581a4ca0 con 0x7fa0581084b0 2024-04-26T01:42:38.548 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:38.549+0000 7fa056a00700 1 --2- 172.21.15.151:0/332669224 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0581084b0 0x7fa058198120 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa048004f80 tx=0x7fa048004fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:42:38.548 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:38.549+0000 7fa04f400700 1 -- 172.21.15.151:0/332669224 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa048026070 con 0x7fa0581084b0 2024-04-26T01:42:38.549 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:38.549+0000 7fa04f400700 1 -- 172.21.15.151:0/332669224 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa04800b6f0 con 0x7fa0581084b0 2024-04-26T01:42:38.549 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:38.549+0000 7fa05da00700 1 -- 172.21.15.151:0/332669224 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0581a4ed0 con 0x7fa0581084b0 2024-04-26T01:42:38.549 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:38.549+0000 7fa04f400700 1 -- 172.21.15.151:0/332669224 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa048009e60 con 0x7fa0581084b0 2024-04-26T01:42:38.549 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:38.549+0000 7fa05da00700 1 -- 172.21.15.151:0/332669224 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0581a5330 con 0x7fa0581084b0 2024-04-26T01:42:38.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:38.551+0000 7fa05da00700 1 -- 172.21.15.151:0/332669224 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0580640a0 con 0x7fa0581084b0 2024-04-26T01:42:38.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:38.551+0000 7fa04f400700 1 -- 172.21.15.151:0/332669224 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7fa048029070 con 0x7fa0581084b0 2024-04-26T01:42:38.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:38.551+0000 7fa04f400700 1 --2- 172.21.15.151:0/332669224 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fa03c039ae0 0x7fa03c03bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:42:38.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:38.551+0000 7fa04f400700 1 -- 172.21.15.151:0/332669224 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa04804d3c0 con 0x7fa0581084b0 2024-04-26T01:42:38.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:38.552+0000 7fa04fe00700 1 --2- 172.21.15.151:0/332669224 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fa03c039ae0 0x7fa03c03bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:42:38.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:38.552+0000 7fa04fe00700 1 --2- 172.21.15.151:0/332669224 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fa03c039ae0 0x7fa03c03bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa040009ad0 tx=0x7fa040009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:42:38.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:38.556+0000 7fa04f400700 1 -- 172.21.15.151:0/332669224 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa048013070 con 0x7fa0581084b0 2024-04-26T01:42:38.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:38 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:38.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:38 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:38.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:38 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:38.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:38 smithi165 ceph-mon[32648]: Updating smithi151:/var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:42:38.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:38 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:42:38.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:38 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:38.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:38 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:38.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:38 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:38.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:38 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:38.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:38 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:38.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:38 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:38.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:38 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:38.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:38 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T01:42:38.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:38 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T01:42:38.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:38 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:38.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:38.783+0000 7fa05da00700 1 -- 172.21.15.151:0/332669224 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fa0581a5740 con 0x7fa0581084b0 2024-04-26T01:42:38.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:38.784+0000 7fa04f400700 1 -- 172.21.15.151:0/332669224 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+277 (secure 0 0 0) 0x7fa048030090 con 0x7fa0581084b0 2024-04-26T01:42:38.781 INFO:teuthology.orchestra.run.smithi151.stdout:# minimal ceph.conf for c0e49e64-036d-11ef-bc93-c7b262605968 2024-04-26T01:42:38.781 INFO:teuthology.orchestra.run.smithi151.stdout:[global] 2024-04-26T01:42:38.781 INFO:teuthology.orchestra.run.smithi151.stdout: fsid = c0e49e64-036d-11ef-bc93-c7b262605968 2024-04-26T01:42:38.781 INFO:teuthology.orchestra.run.smithi151.stdout: mon_host = [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] 2024-04-26T01:42:38.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:38.787+0000 7fa05da00700 1 -- 172.21.15.151:0/332669224 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fa03c039ae0 msgr2=0x7fa03c03bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:42:38.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:38.787+0000 7fa05da00700 1 --2- 172.21.15.151:0/332669224 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fa03c039ae0 0x7fa03c03bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa040009ad0 tx=0x7fa040009210 comp rx=0 tx=0).stop 2024-04-26T01:42:38.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:38.787+0000 7fa05da00700 1 -- 172.21.15.151:0/332669224 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0581084b0 msgr2=0x7fa058198120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:42:38.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:38.787+0000 7fa05da00700 1 --2- 172.21.15.151:0/332669224 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0581084b0 0x7fa058198120 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa048004f80 tx=0x7fa048004fb0 comp rx=0 tx=0).stop 2024-04-26T01:42:38.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:38.787+0000 7fa05da00700 1 -- 172.21.15.151:0/332669224 shutdown_connections 2024-04-26T01:42:38.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:38.787+0000 7fa05da00700 1 --2- 172.21.15.151:0/332669224 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa05819cf30 0x7fa05819d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:38.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:38.787+0000 7fa05da00700 1 --2- 172.21.15.151:0/332669224 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0581084b0 0x7fa058198120 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:38.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:38.787+0000 7fa05da00700 1 --2- 172.21.15.151:0/332669224 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fa03c039ae0 0x7fa03c03bfa0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:38.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:38.787+0000 7fa05da00700 1 --2- 172.21.15.151:0/332669224 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa058198660 0x7fa05819c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:38.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:38.787+0000 7fa05da00700 1 -- 172.21.15.151:0/332669224 >> 172.21.15.151:0/332669224 conn(0x7fa0580fdf30 msgr2=0x7fa058103fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:42:38.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:38.787+0000 7fa05da00700 1 -- 172.21.15.151:0/332669224 shutdown_connections 2024-04-26T01:42:38.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:38.787+0000 7fa05da00700 1 -- 172.21.15.151:0/332669224 wait complete. 2024-04-26T01:42:38.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:38 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:38.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:38 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:38.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:38 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:38.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:38 smithi172 ceph-mon[32785]: Updating smithi151:/var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:42:38.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:38 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:42:38.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:38 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:38.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:38 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:38.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:38 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:38.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:38 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:38.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:38 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:38.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:38 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:38.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:38 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:38.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:38 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T01:42:38.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:38 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T01:42:38.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:38 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:39.370 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:39 smithi151 ceph-mon[28057]: Reconfiguring mon.a (unknown last config time)... 2024-04-26T01:42:39.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:39 smithi151 ceph-mon[28057]: Reconfiguring daemon mon.a on smithi151 2024-04-26T01:42:39.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:39 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/332669224' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:39.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:39 smithi151 ceph-mon[28057]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:39.450 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-26T01:42:39.451 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-26T01:42:39.451 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/ceph/ceph.conf 2024-04-26T01:42:39.532 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-26T01:42:39.533 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T01:42:39.658 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-26T01:42:39.658 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/etc/ceph/ceph.conf 2024-04-26T01:42:39.685 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:39 smithi165 ceph-mon[32648]: Reconfiguring mon.a (unknown last config time)... 2024-04-26T01:42:39.685 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:39 smithi165 ceph-mon[32648]: Reconfiguring daemon mon.a on smithi151 2024-04-26T01:42:39.685 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:39 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/332669224' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:39.685 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:39 smithi165 ceph-mon[32648]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:39.691 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-26T01:42:39.691 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T01:42:39.758 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-26T01:42:39.759 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/etc/ceph/ceph.conf 2024-04-26T01:42:39.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:39 smithi172 ceph-mon[32785]: Reconfiguring mon.a (unknown last config time)... 2024-04-26T01:42:39.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:39 smithi172 ceph-mon[32785]: Reconfiguring daemon mon.a on smithi151 2024-04-26T01:42:39.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:39 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/332669224' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:39.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:39 smithi172 ceph-mon[32785]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:39.790 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-26T01:42:39.790 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T01:42:39.859 INFO:tasks.cephadm:Adding mgr.x on smithi151 2024-04-26T01:42:39.859 INFO:tasks.cephadm:Adding mgr.y on smithi165 2024-04-26T01:42:39.859 INFO:tasks.cephadm:Adding mgr.z on smithi172 2024-04-26T01:42:39.859 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi151=x;smithi165=y;smithi172=z' 2024-04-26T01:42:40.129 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T01:42:41.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:41.594+0000 7fca3a453700 1 -- 172.21.15.172:0/862192592 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca34100870 msgr2=0x7fca34100c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:42:41.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:41.594+0000 7fca3a453700 1 --2- 172.21.15.172:0/862192592 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca34100870 0x7fca34100c90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fca24009910 tx=0x7fca24009c20 comp rx=0 tx=0).stop 2024-04-26T01:42:41.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:41.594+0000 7fca3a453700 1 -- 172.21.15.172:0/862192592 shutdown_connections 2024-04-26T01:42:41.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:41.594+0000 7fca3a453700 1 --2- 172.21.15.172:0/862192592 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca34100870 0x7fca34100c90 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:41.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:41.594+0000 7fca3a453700 1 -- 172.21.15.172:0/862192592 >> 172.21.15.172:0/862192592 conn(0x7fca340fc1b0 msgr2=0x7fca340fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:42:41.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:41.594+0000 7fca3a453700 1 -- 172.21.15.172:0/862192592 shutdown_connections 2024-04-26T01:42:41.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:41.594+0000 7fca3a453700 1 -- 172.21.15.172:0/862192592 wait complete. 2024-04-26T01:42:41.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:41.594+0000 7fca3a453700 1 Processor -- start 2024-04-26T01:42:41.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:41.594+0000 7fca3a453700 1 -- start start 2024-04-26T01:42:41.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:41.595+0000 7fca3a453700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca34100870 0x7fca34197ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:42:41.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:41.595+0000 7fca3a453700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca34198220 0x7fca3419d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:42:41.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:41.595+0000 7fca3a453700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca341986a0 0x7fca3419f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:42:41.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:41.595+0000 7fca3a453700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fca34108470 con 0x7fca34198220 2024-04-26T01:42:41.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:41.595+0000 7fca3a453700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fca341082f0 con 0x7fca341986a0 2024-04-26T01:42:41.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:41.595+0000 7fca3a453700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fca341085f0 con 0x7fca34100870 2024-04-26T01:42:41.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:41.595+0000 7fca33fff700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca34100870 0x7fca34197ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:42:41.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:41.595+0000 7fca33fff700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca34100870 0x7fca34197ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58840/0 (socket says 172.21.15.172:58840) 2024-04-26T01:42:41.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:41.595+0000 7fca33fff700 1 -- 172.21.15.172:0/3733827458 learned_addr learned my addr 172.21.15.172:0/3733827458 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-26T01:42:41.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:41.595+0000 7fca33fff700 1 -- 172.21.15.172:0/3733827458 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca341986a0 msgr2=0x7fca3419f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:42:41.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:41.595+0000 7fca33fff700 1 --2- 172.21.15.172:0/3733827458 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca341986a0 0x7fca3419f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:41.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:41.595+0000 7fca33fff700 1 -- 172.21.15.172:0/3733827458 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca34198220 msgr2=0x7fca3419d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:42:41.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:41.595+0000 7fca33fff700 1 --2- 172.21.15.172:0/3733827458 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca34198220 0x7fca3419d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:41.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:41.595+0000 7fca33fff700 1 -- 172.21.15.172:0/3733827458 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca341a0100 con 0x7fca34100870 2024-04-26T01:42:41.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:41.595+0000 7fca33fff700 1 --2- 172.21.15.172:0/3733827458 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca34100870 0x7fca34197ce0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fca24005e00 tx=0x7fca24009e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:42:41.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:41.596+0000 7fca317fa700 1 -- 172.21.15.172:0/3733827458 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca24027070 con 0x7fca34100870 2024-04-26T01:42:41.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:41.596+0000 7fca3a453700 1 -- 172.21.15.172:0/3733827458 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca341a03f0 con 0x7fca34100870 2024-04-26T01:42:41.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:41.596+0000 7fca317fa700 1 -- 172.21.15.172:0/3733827458 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca2400be10 con 0x7fca34100870 2024-04-26T01:42:41.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:41.596+0000 7fca317fa700 1 -- 172.21.15.172:0/3733827458 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2400f460 con 0x7fca34100870 2024-04-26T01:42:41.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:41.596+0000 7fca3a453700 1 -- 172.21.15.172:0/3733827458 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca341a09b0 con 0x7fca34100870 2024-04-26T01:42:41.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:41.596+0000 7fca317fa700 1 -- 172.21.15.172:0/3733827458 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7fca2402a070 con 0x7fca34100870 2024-04-26T01:42:41.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:41.597+0000 7fca317fa700 1 --2- 172.21.15.172:0/3733827458 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fca1c039ae0 0x7fca1c03bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:42:41.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:41.597+0000 7fca317fa700 1 -- 172.21.15.172:0/3733827458 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fca2404db30 con 0x7fca34100870 2024-04-26T01:42:41.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:41.597+0000 7fca3a453700 1 -- 172.21.15.172:0/3733827458 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca340640a0 con 0x7fca34100870 2024-04-26T01:42:41.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:41.597+0000 7fca337fe700 1 --2- 172.21.15.172:0/3733827458 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fca1c039ae0 0x7fca1c03bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:42:41.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:41.598+0000 7fca337fe700 1 --2- 172.21.15.172:0/3733827458 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fca1c039ae0 0x7fca1c03bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fca280098e0 tx=0x7fca28006dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:42:41.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:41.602+0000 7fca317fa700 1 -- 172.21.15.172:0/3733827458 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fca24031070 con 0x7fca34100870 2024-04-26T01:42:41.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:41.814+0000 7fca3a453700 1 -- 172.21.15.172:0/3733827458 --> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi151=x;smithi165=y;smithi172=z", "target": ["mon-mgr", ""]}) -- 0x7fca34062e80 con 0x7fca1c039ae0 2024-04-26T01:42:41.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:41.828+0000 7fca317fa700 1 -- 172.21.15.172:0/3733827458 <== mgr.14152 v2:172.21.15.151:6800/2251256505 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fca34062e80 con 0x7fca1c039ae0 2024-04-26T01:42:41.826 INFO:teuthology.orchestra.run.smithi172.stdout:Scheduled mgr update... 2024-04-26T01:42:41.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:41.831+0000 7fca3a453700 1 -- 172.21.15.172:0/3733827458 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fca1c039ae0 msgr2=0x7fca1c03bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:42:41.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:41.831+0000 7fca3a453700 1 --2- 172.21.15.172:0/3733827458 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fca1c039ae0 0x7fca1c03bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fca280098e0 tx=0x7fca28006dd0 comp rx=0 tx=0).stop 2024-04-26T01:42:41.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:41.831+0000 7fca3a453700 1 -- 172.21.15.172:0/3733827458 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca34100870 msgr2=0x7fca34197ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:42:41.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:41.831+0000 7fca3a453700 1 --2- 172.21.15.172:0/3733827458 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca34100870 0x7fca34197ce0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fca24005e00 tx=0x7fca24009e60 comp rx=0 tx=0).stop 2024-04-26T01:42:41.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:41.831+0000 7fca3a453700 1 -- 172.21.15.172:0/3733827458 shutdown_connections 2024-04-26T01:42:41.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:41.831+0000 7fca3a453700 1 --2- 172.21.15.172:0/3733827458 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca341986a0 0x7fca3419f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:41.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:41.831+0000 7fca3a453700 1 --2- 172.21.15.172:0/3733827458 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca34100870 0x7fca34197ce0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:41.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:41.831+0000 7fca3a453700 1 --2- 172.21.15.172:0/3733827458 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fca1c039ae0 0x7fca1c03bfa0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:41.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:41.831+0000 7fca3a453700 1 --2- 172.21.15.172:0/3733827458 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca34198220 0x7fca3419d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:41.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:41.831+0000 7fca3a453700 1 -- 172.21.15.172:0/3733827458 >> 172.21.15.172:0/3733827458 conn(0x7fca340fc1b0 msgr2=0x7fca340fce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:42:41.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:41.831+0000 7fca3a453700 1 -- 172.21.15.172:0/3733827458 shutdown_connections 2024-04-26T01:42:41.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:42:41.831+0000 7fca3a453700 1 -- 172.21.15.172:0/3733827458 wait complete. 2024-04-26T01:42:42.035 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:41 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:42.035 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:41 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:42.035 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:41 smithi172 ceph-mon[32785]: Reconfiguring mon.b (monmap changed)... 2024-04-26T01:42:42.035 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:41 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T01:42:42.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:41 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T01:42:42.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:41 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:42.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:41 smithi172 ceph-mon[32785]: Reconfiguring daemon mon.b on smithi165 2024-04-26T01:42:42.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:41 smithi172 ceph-mon[32785]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:42.065 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:41 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:42.065 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:41 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:42.066 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:41 smithi165 ceph-mon[32648]: Reconfiguring mon.b (monmap changed)... 2024-04-26T01:42:42.066 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:41 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T01:42:42.066 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:41 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T01:42:42.066 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:41 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:42.066 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:41 smithi165 ceph-mon[32648]: Reconfiguring daemon mon.b on smithi165 2024-04-26T01:42:42.066 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:41 smithi165 ceph-mon[32648]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:42.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:41 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:42.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:41 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:42.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:41 smithi151 ceph-mon[28057]: Reconfiguring mon.b (monmap changed)... 2024-04-26T01:42:42.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:41 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T01:42:42.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:41 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T01:42:42.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:41 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:42.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:41 smithi151 ceph-mon[28057]: Reconfiguring daemon mon.b on smithi165 2024-04-26T01:42:42.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:41 smithi151 ceph-mon[28057]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:42.320 DEBUG:teuthology.orchestra.run.smithi165:mgr.y> sudo journalctl -f -n 0 -u ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mgr.y.service 2024-04-26T01:42:42.323 DEBUG:teuthology.orchestra.run.smithi172:mgr.z> sudo journalctl -f -n 0 -u ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mgr.z.service 2024-04-26T01:42:42.326 INFO:tasks.cephadm:Deploying OSDs... 2024-04-26T01:42:42.326 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-26T01:42:42.326 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T01:42:42.386 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T01:42:42.387 DEBUG:teuthology.orchestra.run.smithi151:> stat /dev/vg_nvme/lv_1 2024-04-26T01:42:42.440 INFO:teuthology.orchestra.run.smithi151.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T01:42:42.440 INFO:teuthology.orchestra.run.smithi151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T01:42:42.440 INFO:teuthology.orchestra.run.smithi151.stdout:Device: 6h/6d Inode: 632 Links: 1 2024-04-26T01:42:42.440 INFO:teuthology.orchestra.run.smithi151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T01:42:42.440 INFO:teuthology.orchestra.run.smithi151.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T01:42:42.440 INFO:teuthology.orchestra.run.smithi151.stdout:Access: 2024-04-26 01:41:42.140901294 +0000 2024-04-26T01:42:42.440 INFO:teuthology.orchestra.run.smithi151.stdout:Modify: 2024-04-26 01:35:55.254843982 +0000 2024-04-26T01:42:42.441 INFO:teuthology.orchestra.run.smithi151.stdout:Change: 2024-04-26 01:35:55.254843982 +0000 2024-04-26T01:42:42.441 INFO:teuthology.orchestra.run.smithi151.stdout: Birth: 2024-04-26 01:35:55.254843982 +0000 2024-04-26T01:42:42.441 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T01:42:42.520 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records in 2024-04-26T01:42:42.521 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records out 2024-04-26T01:42:42.521 INFO:teuthology.orchestra.run.smithi151.stderr:512 bytes copied, 0.00039495 s, 1.3 MB/s 2024-04-26T01:42:42.524 DEBUG:teuthology.orchestra.run.smithi151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T01:42:42.578 DEBUG:teuthology.orchestra.run.smithi151:> stat /dev/vg_nvme/lv_2 2024-04-26T01:42:42.628 INFO:teuthology.orchestra.run.smithi151.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T01:42:42.628 INFO:teuthology.orchestra.run.smithi151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T01:42:42.628 INFO:teuthology.orchestra.run.smithi151.stdout:Device: 6h/6d Inode: 653 Links: 1 2024-04-26T01:42:42.628 INFO:teuthology.orchestra.run.smithi151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T01:42:42.628 INFO:teuthology.orchestra.run.smithi151.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T01:42:42.628 INFO:teuthology.orchestra.run.smithi151.stdout:Access: 2024-04-26 01:41:42.185900769 +0000 2024-04-26T01:42:42.628 INFO:teuthology.orchestra.run.smithi151.stdout:Modify: 2024-04-26 01:35:55.859836926 +0000 2024-04-26T01:42:42.628 INFO:teuthology.orchestra.run.smithi151.stdout:Change: 2024-04-26 01:35:55.859836926 +0000 2024-04-26T01:42:42.628 INFO:teuthology.orchestra.run.smithi151.stdout: Birth: 2024-04-26 01:35:55.859836926 +0000 2024-04-26T01:42:42.629 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T01:42:42.705 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records in 2024-04-26T01:42:42.705 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records out 2024-04-26T01:42:42.705 INFO:teuthology.orchestra.run.smithi151.stderr:512 bytes copied, 0.000538992 s, 950 kB/s 2024-04-26T01:42:42.709 DEBUG:teuthology.orchestra.run.smithi151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T01:42:42.763 DEBUG:teuthology.orchestra.run.smithi151:> stat /dev/vg_nvme/lv_3 2024-04-26T01:42:42.817 INFO:teuthology.orchestra.run.smithi151.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T01:42:42.817 INFO:teuthology.orchestra.run.smithi151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T01:42:42.817 INFO:teuthology.orchestra.run.smithi151.stdout:Device: 6h/6d Inode: 671 Links: 1 2024-04-26T01:42:42.817 INFO:teuthology.orchestra.run.smithi151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T01:42:42.817 INFO:teuthology.orchestra.run.smithi151.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T01:42:42.817 INFO:teuthology.orchestra.run.smithi151.stdout:Access: 2024-04-26 01:41:42.231900233 +0000 2024-04-26T01:42:42.818 INFO:teuthology.orchestra.run.smithi151.stdout:Modify: 2024-04-26 01:35:56.458829940 +0000 2024-04-26T01:42:42.818 INFO:teuthology.orchestra.run.smithi151.stdout:Change: 2024-04-26 01:35:56.458829940 +0000 2024-04-26T01:42:42.818 INFO:teuthology.orchestra.run.smithi151.stdout: Birth: 2024-04-26 01:35:56.458829940 +0000 2024-04-26T01:42:42.818 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T01:42:42.894 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records in 2024-04-26T01:42:42.894 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records out 2024-04-26T01:42:42.894 INFO:teuthology.orchestra.run.smithi151.stderr:512 bytes copied, 0.000337768 s, 1.5 MB/s 2024-04-26T01:42:42.898 DEBUG:teuthology.orchestra.run.smithi151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T01:42:42.954 DEBUG:teuthology.orchestra.run.smithi151:> stat /dev/vg_nvme/lv_4 2024-04-26T01:42:43.002 INFO:teuthology.orchestra.run.smithi151.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T01:42:43.003 INFO:teuthology.orchestra.run.smithi151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T01:42:43.003 INFO:teuthology.orchestra.run.smithi151.stdout:Device: 6h/6d Inode: 691 Links: 1 2024-04-26T01:42:43.003 INFO:teuthology.orchestra.run.smithi151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T01:42:43.003 INFO:teuthology.orchestra.run.smithi151.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T01:42:43.003 INFO:teuthology.orchestra.run.smithi151.stdout:Access: 2024-04-26 01:41:42.272899755 +0000 2024-04-26T01:42:43.003 INFO:teuthology.orchestra.run.smithi151.stdout:Modify: 2024-04-26 01:35:57.054822988 +0000 2024-04-26T01:42:43.003 INFO:teuthology.orchestra.run.smithi151.stdout:Change: 2024-04-26 01:35:57.054822988 +0000 2024-04-26T01:42:43.003 INFO:teuthology.orchestra.run.smithi151.stdout: Birth: 2024-04-26 01:35:57.054822988 +0000 2024-04-26T01:42:43.003 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T01:42:43.026 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:42 smithi172 ceph-mon[32785]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi151=x;smithi165=y;smithi172=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:42:43.026 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:42 smithi172 ceph-mon[32785]: Saving service mgr spec with placement smithi151=x;smithi165=y;smithi172=z;count:3 2024-04-26T01:42:43.026 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:42 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:43.026 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:42 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:43.026 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:42 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:43.026 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:42 smithi172 ceph-mon[32785]: Reconfiguring mon.c (monmap changed)... 2024-04-26T01:42:43.026 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:42 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T01:42:43.027 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:42 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T01:42:43.027 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:42 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:43.027 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:42 smithi172 ceph-mon[32785]: Reconfiguring daemon mon.c on smithi172 2024-04-26T01:42:43.076 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:42 smithi151 ceph-mon[28057]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi151=x;smithi165=y;smithi172=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:42:43.076 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:42 smithi151 ceph-mon[28057]: Saving service mgr spec with placement smithi151=x;smithi165=y;smithi172=z;count:3 2024-04-26T01:42:43.076 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:42 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:43.076 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:42 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:43.076 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:42 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:43.076 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:42 smithi151 ceph-mon[28057]: Reconfiguring mon.c (monmap changed)... 2024-04-26T01:42:43.076 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:42 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T01:42:43.076 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:42 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T01:42:43.076 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:42 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:43.076 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:42 smithi151 ceph-mon[28057]: Reconfiguring daemon mon.c on smithi172 2024-04-26T01:42:43.079 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records in 2024-04-26T01:42:43.080 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records out 2024-04-26T01:42:43.080 INFO:teuthology.orchestra.run.smithi151.stderr:512 bytes copied, 0.000366297 s, 1.4 MB/s 2024-04-26T01:42:43.083 DEBUG:teuthology.orchestra.run.smithi151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T01:42:43.094 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:42 smithi165 ceph-mon[32648]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi151=x;smithi165=y;smithi172=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:42:43.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:42 smithi165 ceph-mon[32648]: Saving service mgr spec with placement smithi151=x;smithi165=y;smithi172=z;count:3 2024-04-26T01:42:43.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:42 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:43.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:42 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:43.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:42 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:43.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:42 smithi165 ceph-mon[32648]: Reconfiguring mon.c (monmap changed)... 2024-04-26T01:42:43.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:42 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T01:42:43.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:42 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T01:42:43.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:42 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:43.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:42 smithi165 ceph-mon[32648]: Reconfiguring daemon mon.c on smithi172 2024-04-26T01:42:43.138 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-26T01:42:43.138 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T01:42:43.156 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T01:42:43.157 DEBUG:teuthology.orchestra.run.smithi165:> stat /dev/vg_nvme/lv_1 2024-04-26T01:42:43.213 INFO:teuthology.orchestra.run.smithi165.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T01:42:43.213 INFO:teuthology.orchestra.run.smithi165.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T01:42:43.213 INFO:teuthology.orchestra.run.smithi165.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-26T01:42:43.213 INFO:teuthology.orchestra.run.smithi165.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T01:42:43.213 INFO:teuthology.orchestra.run.smithi165.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T01:42:43.213 INFO:teuthology.orchestra.run.smithi165.stdout:Access: 2024-04-26 01:42:29.187154305 +0000 2024-04-26T01:42:43.214 INFO:teuthology.orchestra.run.smithi165.stdout:Modify: 2024-04-26 01:34:50.962325032 +0000 2024-04-26T01:42:43.214 INFO:teuthology.orchestra.run.smithi165.stdout:Change: 2024-04-26 01:34:50.962325032 +0000 2024-04-26T01:42:43.214 INFO:teuthology.orchestra.run.smithi165.stdout: Birth: 2024-04-26 01:34:50.962325032 +0000 2024-04-26T01:42:43.214 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T01:42:43.278 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-26T01:42:43.278 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-26T01:42:43.278 INFO:teuthology.orchestra.run.smithi165.stderr:512 bytes copied, 0.000354307 s, 1.4 MB/s 2024-04-26T01:42:43.279 DEBUG:teuthology.orchestra.run.smithi165:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T01:42:43.336 DEBUG:teuthology.orchestra.run.smithi165:> stat /dev/vg_nvme/lv_2 2024-04-26T01:42:43.392 INFO:teuthology.orchestra.run.smithi165.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T01:42:43.392 INFO:teuthology.orchestra.run.smithi165.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T01:42:43.392 INFO:teuthology.orchestra.run.smithi165.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-26T01:42:43.392 INFO:teuthology.orchestra.run.smithi165.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T01:42:43.392 INFO:teuthology.orchestra.run.smithi165.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T01:42:43.392 INFO:teuthology.orchestra.run.smithi165.stdout:Access: 2024-04-26 01:42:29.187154305 +0000 2024-04-26T01:42:43.393 INFO:teuthology.orchestra.run.smithi165.stdout:Modify: 2024-04-26 01:34:51.300319379 +0000 2024-04-26T01:42:43.393 INFO:teuthology.orchestra.run.smithi165.stdout:Change: 2024-04-26 01:34:51.300319379 +0000 2024-04-26T01:42:43.393 INFO:teuthology.orchestra.run.smithi165.stdout: Birth: 2024-04-26 01:34:51.300319379 +0000 2024-04-26T01:42:43.393 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T01:42:43.456 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-26T01:42:43.456 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-26T01:42:43.456 INFO:teuthology.orchestra.run.smithi165.stderr:512 bytes copied, 0.000233642 s, 2.2 MB/s 2024-04-26T01:42:43.457 DEBUG:teuthology.orchestra.run.smithi165:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T01:42:43.514 DEBUG:teuthology.orchestra.run.smithi165:> stat /dev/vg_nvme/lv_3 2024-04-26T01:42:43.569 INFO:teuthology.orchestra.run.smithi165.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T01:42:43.569 INFO:teuthology.orchestra.run.smithi165.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T01:42:43.569 INFO:teuthology.orchestra.run.smithi165.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-26T01:42:43.569 INFO:teuthology.orchestra.run.smithi165.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T01:42:43.569 INFO:teuthology.orchestra.run.smithi165.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T01:42:43.569 INFO:teuthology.orchestra.run.smithi165.stdout:Access: 2024-04-26 01:42:29.187154305 +0000 2024-04-26T01:42:43.569 INFO:teuthology.orchestra.run.smithi165.stdout:Modify: 2024-04-26 01:34:51.636313760 +0000 2024-04-26T01:42:43.569 INFO:teuthology.orchestra.run.smithi165.stdout:Change: 2024-04-26 01:34:51.636313760 +0000 2024-04-26T01:42:43.569 INFO:teuthology.orchestra.run.smithi165.stdout: Birth: 2024-04-26 01:34:51.636313760 +0000 2024-04-26T01:42:43.570 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T01:42:43.632 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-26T01:42:43.632 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-26T01:42:43.632 INFO:teuthology.orchestra.run.smithi165.stderr:512 bytes copied, 0.000225133 s, 2.3 MB/s 2024-04-26T01:42:43.633 DEBUG:teuthology.orchestra.run.smithi165:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T01:42:43.690 DEBUG:teuthology.orchestra.run.smithi165:> stat /dev/vg_nvme/lv_4 2024-04-26T01:42:43.745 INFO:teuthology.orchestra.run.smithi165.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T01:42:43.745 INFO:teuthology.orchestra.run.smithi165.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T01:42:43.746 INFO:teuthology.orchestra.run.smithi165.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-26T01:42:43.746 INFO:teuthology.orchestra.run.smithi165.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T01:42:43.746 INFO:teuthology.orchestra.run.smithi165.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T01:42:43.746 INFO:teuthology.orchestra.run.smithi165.stdout:Access: 2024-04-26 01:42:29.188154289 +0000 2024-04-26T01:42:43.746 INFO:teuthology.orchestra.run.smithi165.stdout:Modify: 2024-04-26 01:34:51.970308174 +0000 2024-04-26T01:42:43.746 INFO:teuthology.orchestra.run.smithi165.stdout:Change: 2024-04-26 01:34:51.970308174 +0000 2024-04-26T01:42:43.746 INFO:teuthology.orchestra.run.smithi165.stdout: Birth: 2024-04-26 01:34:51.970308174 +0000 2024-04-26T01:42:43.746 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T01:42:43.811 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-26T01:42:43.811 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-26T01:42:43.811 INFO:teuthology.orchestra.run.smithi165.stderr:512 bytes copied, 0.00022338 s, 2.3 MB/s 2024-04-26T01:42:43.811 DEBUG:teuthology.orchestra.run.smithi165:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T01:42:43.867 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-26T01:42:43.867 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T01:42:43.883 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T01:42:43.883 DEBUG:teuthology.orchestra.run.smithi172:> stat /dev/vg_nvme/lv_1 2024-04-26T01:42:43.939 INFO:teuthology.orchestra.run.smithi172.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T01:42:43.939 INFO:teuthology.orchestra.run.smithi172.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T01:42:43.939 INFO:teuthology.orchestra.run.smithi172.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-26T01:42:43.939 INFO:teuthology.orchestra.run.smithi172.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T01:42:43.940 INFO:teuthology.orchestra.run.smithi172.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T01:42:43.940 INFO:teuthology.orchestra.run.smithi172.stdout:Access: 2024-04-26 01:42:19.495199690 +0000 2024-04-26T01:42:43.940 INFO:teuthology.orchestra.run.smithi172.stdout:Modify: 2024-04-26 01:34:50.957717650 +0000 2024-04-26T01:42:43.940 INFO:teuthology.orchestra.run.smithi172.stdout:Change: 2024-04-26 01:34:50.957717650 +0000 2024-04-26T01:42:43.940 INFO:teuthology.orchestra.run.smithi172.stdout: Birth: 2024-04-26 01:34:50.957717650 +0000 2024-04-26T01:42:43.940 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T01:42:44.000 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:43 smithi172 ceph-mon[32785]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:44.000 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:43 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:44.000 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:43 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:44.000 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:43 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:42:44.000 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:43 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:44.000 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:43 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:42:44.000 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:43 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:44.001 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:43 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T01:42:44.001 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:43 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T01:42:44.001 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:43 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T01:42:44.001 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:43 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:44.002 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records in 2024-04-26T01:42:44.003 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records out 2024-04-26T01:42:44.003 INFO:teuthology.orchestra.run.smithi172.stderr:512 bytes copied, 0.000210739 s, 2.4 MB/s 2024-04-26T01:42:44.004 DEBUG:teuthology.orchestra.run.smithi172:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T01:42:44.060 DEBUG:teuthology.orchestra.run.smithi172:> stat /dev/vg_nvme/lv_2 2024-04-26T01:42:44.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:43 smithi165 ceph-mon[32648]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:44.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:43 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:44.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:43 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:44.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:43 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:42:44.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:43 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:44.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:43 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:42:44.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:43 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:44.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:43 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T01:42:44.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:43 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T01:42:44.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:43 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T01:42:44.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:43 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:44.116 INFO:teuthology.orchestra.run.smithi172.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T01:42:44.116 INFO:teuthology.orchestra.run.smithi172.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T01:42:44.116 INFO:teuthology.orchestra.run.smithi172.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-26T01:42:44.116 INFO:teuthology.orchestra.run.smithi172.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T01:42:44.116 INFO:teuthology.orchestra.run.smithi172.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T01:42:44.116 INFO:teuthology.orchestra.run.smithi172.stdout:Access: 2024-04-26 01:42:19.496199666 +0000 2024-04-26T01:42:44.116 INFO:teuthology.orchestra.run.smithi172.stdout:Modify: 2024-04-26 01:34:51.292709384 +0000 2024-04-26T01:42:44.116 INFO:teuthology.orchestra.run.smithi172.stdout:Change: 2024-04-26 01:34:51.292709384 +0000 2024-04-26T01:42:44.116 INFO:teuthology.orchestra.run.smithi172.stdout: Birth: 2024-04-26 01:34:51.292709384 +0000 2024-04-26T01:42:44.117 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T01:42:44.120 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:43 smithi151 ceph-mon[28057]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:44.120 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:43 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:44.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:43 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:44.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:43 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:42:44.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:43 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:44.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:43 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:42:44.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:43 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:44.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:43 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T01:42:44.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:43 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T01:42:44.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:43 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T01:42:44.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:43 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:44.179 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records in 2024-04-26T01:42:44.179 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records out 2024-04-26T01:42:44.179 INFO:teuthology.orchestra.run.smithi172.stderr:512 bytes copied, 0.00021781 s, 2.4 MB/s 2024-04-26T01:42:44.180 DEBUG:teuthology.orchestra.run.smithi172:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T01:42:44.237 DEBUG:teuthology.orchestra.run.smithi172:> stat /dev/vg_nvme/lv_3 2024-04-26T01:42:44.293 INFO:teuthology.orchestra.run.smithi172.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T01:42:44.294 INFO:teuthology.orchestra.run.smithi172.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T01:42:44.294 INFO:teuthology.orchestra.run.smithi172.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-04-26T01:42:44.294 INFO:teuthology.orchestra.run.smithi172.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T01:42:44.294 INFO:teuthology.orchestra.run.smithi172.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T01:42:44.294 INFO:teuthology.orchestra.run.smithi172.stdout:Access: 2024-04-26 01:42:19.496199666 +0000 2024-04-26T01:42:44.294 INFO:teuthology.orchestra.run.smithi172.stdout:Modify: 2024-04-26 01:34:51.630701043 +0000 2024-04-26T01:42:44.294 INFO:teuthology.orchestra.run.smithi172.stdout:Change: 2024-04-26 01:34:51.630701043 +0000 2024-04-26T01:42:44.294 INFO:teuthology.orchestra.run.smithi172.stdout: Birth: 2024-04-26 01:34:51.630701043 +0000 2024-04-26T01:42:44.294 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T01:42:44.357 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records in 2024-04-26T01:42:44.357 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records out 2024-04-26T01:42:44.357 INFO:teuthology.orchestra.run.smithi172.stderr:512 bytes copied, 0.000197389 s, 2.6 MB/s 2024-04-26T01:42:44.358 DEBUG:teuthology.orchestra.run.smithi172:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T01:42:44.417 DEBUG:teuthology.orchestra.run.smithi172:> stat /dev/vg_nvme/lv_4 2024-04-26T01:42:44.472 INFO:teuthology.orchestra.run.smithi172.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T01:42:44.472 INFO:teuthology.orchestra.run.smithi172.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T01:42:44.472 INFO:teuthology.orchestra.run.smithi172.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-04-26T01:42:44.472 INFO:teuthology.orchestra.run.smithi172.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T01:42:44.472 INFO:teuthology.orchestra.run.smithi172.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T01:42:44.472 INFO:teuthology.orchestra.run.smithi172.stdout:Access: 2024-04-26 01:42:19.496199666 +0000 2024-04-26T01:42:44.472 INFO:teuthology.orchestra.run.smithi172.stdout:Modify: 2024-04-26 01:34:51.963692826 +0000 2024-04-26T01:42:44.472 INFO:teuthology.orchestra.run.smithi172.stdout:Change: 2024-04-26 01:34:51.963692826 +0000 2024-04-26T01:42:44.472 INFO:teuthology.orchestra.run.smithi172.stdout: Birth: 2024-04-26 01:34:51.963692826 +0000 2024-04-26T01:42:44.473 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T01:42:44.535 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records in 2024-04-26T01:42:44.535 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records out 2024-04-26T01:42:44.535 INFO:teuthology.orchestra.run.smithi172.stderr:512 bytes copied, 0.00019606 s, 2.6 MB/s 2024-04-26T01:42:44.536 DEBUG:teuthology.orchestra.run.smithi172:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T01:42:44.591 INFO:tasks.cephadm:Deploying osd.0 on smithi151 with /dev/vg_nvme/lv_4... 2024-04-26T01:42:44.592 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-26T01:42:45.024 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:42:45.035 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:44 smithi172 ceph-mon[32785]: Deploying daemon mgr.z on smithi172 2024-04-26T01:42:45.094 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:44 smithi165 ceph-mon[32648]: Deploying daemon mgr.z on smithi172 2024-04-26T01:42:45.120 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:44 smithi151 ceph-mon[28057]: Deploying daemon mgr.z on smithi172 2024-04-26T01:42:45.843 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 01:42:45 smithi172 systemd[1]: Starting Ceph mgr.z for c0e49e64-036d-11ef-bc93-c7b262605968... 2024-04-26T01:42:46.094 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:45 smithi165 ceph-mon[32648]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:46.120 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:45 smithi151 ceph-mon[28057]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:46.164 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:45 smithi172 ceph-mon[32785]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:46.164 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 01:42:45 smithi172 podman[34638]: 2024-04-26 01:42:45.856184397 +0000 UTC m=+0.018395584 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T01:42:46.164 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 01:42:45 smithi172 podman[34638]: 2024-04-26 01:42:45.991029437 +0000 UTC m=+0.153240621 container create 0171154f9dfead9162455177ee255154c83c87799c120c454f2c8261a48c9203 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-z, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-26T01:42:46.164 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 01:42:46 smithi172 podman[34638]: 2024-04-26 01:42:46.168127071 +0000 UTC m=+0.330338255 container init 0171154f9dfead9162455177ee255154c83c87799c120c454f2c8261a48c9203 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-26T01:42:46.509 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 01:42:46 smithi172 podman[34638]: 2024-04-26 01:42:46.170769244 +0000 UTC m=+0.332980437 container start 0171154f9dfead9162455177ee255154c83c87799c120c454f2c8261a48c9203 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-z, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-26T01:42:46.509 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 01:42:46 smithi172 ceph-mgr[34656]: -- 172.21.15.172:0/824792296 <== mon.1 v2:172.21.15.172:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cf2b26c5a0 con 0x55cf2b241000 2024-04-26T01:42:46.509 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 01:42:46 smithi172 bash[34638]: 0171154f9dfead9162455177ee255154c83c87799c120c454f2c8261a48c9203 2024-04-26T01:42:46.509 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 01:42:46 smithi172 systemd[1]: Started Ceph mgr.z for c0e49e64-036d-11ef-bc93-c7b262605968. 2024-04-26T01:42:46.509 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 01:42:46 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-z[34652]: 2024-04-26T01:42:46.328+0000 7f3f5f708200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T01:42:46.510 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 01:42:46 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-z[34652]: 2024-04-26T01:42:46.511+0000 7f3f5f708200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T01:42:46.785 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 01:42:46 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-z[34652]: 2024-04-26T01:42:46.612+0000 7f3f5f708200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T01:42:47.285 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 01:42:46 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-z[34652]: 2024-04-26T01:42:46.922+0000 7f3f5f708200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T01:42:47.285 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 01:42:47 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-z[34652]: 2024-04-26T01:42:47.006+0000 7f3f5f708200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T01:42:47.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:47 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:47.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:47 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:47.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:47 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:47.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:47 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T01:42:47.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:47 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T01:42:47.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:47 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T01:42:47.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:47 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:47.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:47 smithi165 ceph-mon[32648]: Deploying daemon mgr.y on smithi165 2024-04-26T01:42:47.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:47 smithi165 ceph-mon[32648]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:47.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:47 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:47.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:47 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:47.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:47 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:47.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:47 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T01:42:47.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:47 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T01:42:47.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:47 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T01:42:47.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:47 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:47.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:47 smithi172 ceph-mon[32785]: Deploying daemon mgr.y on smithi165 2024-04-26T01:42:47.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:47 smithi172 ceph-mon[32785]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:47.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:47 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:47.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:47 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:47.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:47 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:47.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:47 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T01:42:47.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:47 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T01:42:47.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:47 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T01:42:47.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:47 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:47.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:47 smithi151 ceph-mon[28057]: Deploying daemon mgr.y on smithi165 2024-04-26T01:42:47.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:47 smithi151 ceph-mon[28057]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:47.926 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-26T01:42:47.926 INFO:teuthology.orchestra.run.smithi151.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-26T01:42:47.926 INFO:teuthology.orchestra.run.smithi151.stderr: stderr: 10+0 records in 2024-04-26T01:42:47.926 INFO:teuthology.orchestra.run.smithi151.stderr:10+0 records out 2024-04-26T01:42:47.926 INFO:teuthology.orchestra.run.smithi151.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0353232 s, 297 MB/s 2024-04-26T01:42:47.926 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping successful for: 2024-04-26T01:42:48.529 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 01:42:48 smithi165 systemd[1]: Starting Ceph mgr.y for c0e49e64-036d-11ef-bc93-c7b262605968... 2024-04-26T01:42:48.535 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 01:42:48 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-z[34652]: 2024-04-26T01:42:48.239+0000 7f3f5f708200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T01:42:48.535 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 01:42:48 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-z[34652]: 2024-04-26T01:42:48.409+0000 7f3f5f708200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T01:42:48.655 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi151:vg_nvme/lv_4 2024-04-26T01:42:48.838 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 01:42:48 smithi165 podman[34013]: 2024-04-26 01:42:48.435039495 +0000 UTC m=+0.017679561 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T01:42:48.838 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 01:42:48 smithi165 podman[34013]: 2024-04-26 01:42:48.544756818 +0000 UTC m=+0.127396857 container create 1faa05cec8e2000966940378229b9014670d653393f861025e5dbaa1a2d4dfe1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-y, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-26T01:42:48.838 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 01:42:48 smithi165 podman[34013]: 2024-04-26 01:42:48.749500942 +0000 UTC m=+0.332140991 container init 1faa05cec8e2000966940378229b9014670d653393f861025e5dbaa1a2d4dfe1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-y, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-26T01:42:48.838 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 01:42:48 smithi165 podman[34013]: 2024-04-26 01:42:48.752000833 +0000 UTC m=+0.334640871 container start 1faa05cec8e2000966940378229b9014670d653393f861025e5dbaa1a2d4dfe1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T01:42:48.838 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 01:42:48 smithi165 ceph-mgr[34033]: -- 172.21.15.165:0/4027506597 <== mon.2 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55779c6c05a0 con 0x55779c695000 2024-04-26T01:42:48.838 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 01:42:48 smithi165 bash[34013]: 1faa05cec8e2000966940378229b9014670d653393f861025e5dbaa1a2d4dfe1 2024-04-26T01:42:48.869 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 01:42:48 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-z[34652]: 2024-04-26T01:42:48.583+0000 7f3f5f708200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T01:42:48.869 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 01:42:48 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-z[34652]: 2024-04-26T01:42:48.871+0000 7f3f5f708200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T01:42:49.083 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:42:49.094 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 01:42:48 smithi165 systemd[1]: Started Ceph mgr.y for c0e49e64-036d-11ef-bc93-c7b262605968. 2024-04-26T01:42:49.094 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 01:42:48 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-y[34029]: 2024-04-26T01:42:48.915+0000 7f7153c8b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T01:42:49.166 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 01:42:48 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-z[34652]: 2024-04-26T01:42:48.965+0000 7f3f5f708200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T01:42:49.166 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 01:42:49 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-z[34652]: 2024-04-26T01:42:49.169+0000 7f3f5f708200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T01:42:49.533 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 01:42:49 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-y[34029]: 2024-04-26T01:42:49.101+0000 7f7153c8b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T01:42:49.534 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 01:42:49 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-y[34029]: 2024-04-26T01:42:49.206+0000 7f7153c8b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T01:42:49.534 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 01:42:49 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-y[34029]: 2024-04-26T01:42:49.536+0000 7f7153c8b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T01:42:49.535 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 01:42:49 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-z[34652]: 2024-04-26T01:42:49.369+0000 7f3f5f708200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T01:42:49.844 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 01:42:49 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-y[34029]: 2024-04-26T01:42:49.625+0000 7f7153c8b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T01:42:50.225 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 01:42:50 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-z[34652]: 2024-04-26T01:42:50.001+0000 7f3f5f708200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T01:42:50.226 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:49 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:50.226 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:49 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:50.226 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:49 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:50.226 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:49 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:50.226 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:49 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:42:50.226 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:49 smithi172 ceph-mon[32785]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:50.244 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:49 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:50.244 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:49 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:50.244 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:49 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:50.244 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:49 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:50.244 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:49 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:42:50.244 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:49 smithi151 ceph-mon[28057]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:50.250 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:49 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:50.250 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:49 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:50.250 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:49 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:50.250 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:49 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:50.250 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:49 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:42:50.250 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:49 smithi165 ceph-mon[32648]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:50.535 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 01:42:50 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-z[34652]: 2024-04-26T01:42:50.439+0000 7f3f5f708200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T01:42:51.035 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 01:42:50 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-z[34652]: 2024-04-26T01:42:50.541+0000 7f3f5f708200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T01:42:51.035 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 01:42:50 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-z[34652]: 2024-04-26T01:42:50.745+0000 7f3f5f708200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T01:42:51.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:51.207+0000 7f8d17000700 1 -- 172.21.15.151:0/2796669596 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8d10105e60 msgr2=0x7f8d10108250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:42:51.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:51.207+0000 7f8d17000700 1 --2- 172.21.15.151:0/2796669596 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8d10105e60 0x7f8d10108250 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc0098f0 tx=0x7f8cfc009c00 comp rx=0 tx=0).stop 2024-04-26T01:42:51.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:51.207+0000 7f8d17000700 1 -- 172.21.15.151:0/2796669596 shutdown_connections 2024-04-26T01:42:51.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:51.207+0000 7f8d17000700 1 --2- 172.21.15.151:0/2796669596 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8d10108790 0x7f8d1010ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:51.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:51.207+0000 7f8d17000700 1 --2- 172.21.15.151:0/2796669596 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d10103530 0x7f8d10105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:51.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:51.207+0000 7f8d17000700 1 --2- 172.21.15.151:0/2796669596 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8d10105e60 0x7f8d10108250 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:51.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:51.207+0000 7f8d17000700 1 -- 172.21.15.151:0/2796669596 >> 172.21.15.151:0/2796669596 conn(0x7f8d100fcf40 msgr2=0x7f8d100ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:42:51.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:51.207+0000 7f8d17000700 1 -- 172.21.15.151:0/2796669596 shutdown_connections 2024-04-26T01:42:51.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:51.207+0000 7f8d17000700 1 -- 172.21.15.151:0/2796669596 wait complete. 2024-04-26T01:42:51.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:51.208+0000 7f8d17000700 1 Processor -- start 2024-04-26T01:42:51.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:51.208+0000 7f8d17000700 1 -- start start 2024-04-26T01:42:51.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:51.209+0000 7f8d17000700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d10103530 0x7f8d10198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:42:51.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:51.209+0000 7f8d17000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8d10105e60 0x7f8d10199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:42:51.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:51.209+0000 7f8d17000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8d10108790 0x7f8d101a0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:42:51.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:51.209+0000 7f8d17000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1010e580 con 0x7f8d10108790 2024-04-26T01:42:51.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:51.209+0000 7f8d17000700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1010e400 con 0x7f8d10105e60 2024-04-26T01:42:51.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:51.209+0000 7f8d17000700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1010e700 con 0x7f8d10103530 2024-04-26T01:42:51.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:51.209+0000 7f8d0f400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8d10105e60 0x7f8d10199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:42:51.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:51.209+0000 7f8d0fe00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d10103530 0x7f8d10198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:42:51.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:51.209+0000 7f8d0f400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8d10105e60 0x7f8d10199230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:49564/0 (socket says 172.21.15.151:49564) 2024-04-26T01:42:51.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:51.209+0000 7f8d0f400700 1 -- 172.21.15.151:0/3966348325 learned_addr learned my addr 172.21.15.151:0/3966348325 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:42:51.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:51.209+0000 7f8d15000700 1 --2- 172.21.15.151:0/3966348325 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8d10108790 0x7f8d101a0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:42:51.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:51.209+0000 7f8d0f400700 1 -- 172.21.15.151:0/3966348325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d10103530 msgr2=0x7f8d10198cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:42:51.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:51.209+0000 7f8d0f400700 1 --2- 172.21.15.151:0/3966348325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d10103530 0x7f8d10198cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:51.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:51.209+0000 7f8d0f400700 1 -- 172.21.15.151:0/3966348325 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8d10108790 msgr2=0x7f8d101a0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:42:51.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:51.209+0000 7f8d0f400700 1 --2- 172.21.15.151:0/3966348325 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8d10108790 0x7f8d101a0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:51.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:51.209+0000 7f8d0f400700 1 -- 172.21.15.151:0/3966348325 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d101a0920 con 0x7f8d10105e60 2024-04-26T01:42:51.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:51.209+0000 7f8d15000700 1 --2- 172.21.15.151:0/3966348325 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8d10108790 0x7f8d101a0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:42:51.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:51.210+0000 7f8d0f400700 1 --2- 172.21.15.151:0/3966348325 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8d10105e60 0x7f8d10199230 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc00c010 tx=0x7f8cfc005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:42:51.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:51.210+0000 7f8d0cc00700 1 -- 172.21.15.151:0/3966348325 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc005510 con 0x7f8d10105e60 2024-04-26T01:42:51.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:51.210+0000 7f8d0cc00700 1 -- 172.21.15.151:0/3966348325 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cfc0056b0 con 0x7f8d10105e60 2024-04-26T01:42:51.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:51.210+0000 7f8d17000700 1 -- 172.21.15.151:0/3966348325 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d101a0bb0 con 0x7f8d10105e60 2024-04-26T01:42:51.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:51.210+0000 7f8d0cc00700 1 -- 172.21.15.151:0/3966348325 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc003c10 con 0x7f8d10105e60 2024-04-26T01:42:51.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:51.210+0000 7f8d17000700 1 -- 172.21.15.151:0/3966348325 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d101a1060 con 0x7f8d10105e60 2024-04-26T01:42:51.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:51.211+0000 7f8d0cc00700 1 -- 172.21.15.151:0/3966348325 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f8cfc02c440 con 0x7f8d10105e60 2024-04-26T01:42:51.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:51.211+0000 7f8d17000700 1 -- 172.21.15.151:0/3966348325 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d10069730 con 0x7f8d10105e60 2024-04-26T01:42:51.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:51.212+0000 7f8d0cc00700 1 --2- 172.21.15.151:0/3966348325 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f8cf4039890 0x7f8cf403bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:42:51.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:51.212+0000 7f8d0cc00700 1 -- 172.21.15.151:0/3966348325 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f8cfc04dc70 con 0x7f8d10105e60 2024-04-26T01:42:51.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:51.212+0000 7f8d0fe00700 1 --2- 172.21.15.151:0/3966348325 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f8cf4039890 0x7f8cf403bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:42:51.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:51.213+0000 7f8d0fe00700 1 --2- 172.21.15.151:0/3966348325 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f8cf4039890 0x7f8cf403bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8d10105e30 tx=0x7f8d00009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:42:51.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:51.217+0000 7f8d0cc00700 1 -- 172.21.15.151:0/3966348325 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8cfc013030 con 0x7f8d10105e60 2024-04-26T01:42:51.266 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 01:42:50 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-y[34029]: 2024-04-26T01:42:50.927+0000 7f7153c8b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T01:42:51.267 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 01:42:51 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-y[34029]: 2024-04-26T01:42:51.097+0000 7f7153c8b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T01:42:51.267 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 01:42:51 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-y[34029]: 2024-04-26T01:42:51.269+0000 7f7153c8b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T01:42:51.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:51.459+0000 7f8d17000700 1 -- 172.21.15.151:0/3966348325 --> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f8d10062e80 con 0x7f8cf4039890 2024-04-26T01:42:51.535 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 01:42:51 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-z[34652]: 2024-04-26T01:42:51.269+0000 7f3f5f708200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T01:42:51.535 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 01:42:51 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-z[34652]: 2024-04-26T01:42:51.361+0000 7f3f5f708200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T01:42:51.535 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:51 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:51.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:51 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:51.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:51 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:51.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:51 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:51.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:51 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:51.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:51 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:51.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:51 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:42:51.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:51 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:51.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:51 smithi172 ceph-mon[32785]: Reconfiguring mgr.x (unknown last config time)... 2024-04-26T01:42:51.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:51 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T01:42:51.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:51 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T01:42:51.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:51 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:51.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:51 smithi172 ceph-mon[32785]: Reconfiguring daemon mgr.x on smithi151 2024-04-26T01:42:51.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:51 smithi172 ceph-mon[32785]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:51.556 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:51 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:51.556 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:51 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:51.557 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:51 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:51.557 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:51 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:51.557 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:51 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:51.557 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:51 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:51.557 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:51 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:42:51.557 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:51 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:51.557 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:51 smithi165 ceph-mon[32648]: Reconfiguring mgr.x (unknown last config time)... 2024-04-26T01:42:51.557 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:51 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T01:42:51.557 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:51 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T01:42:51.557 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:51 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:51.557 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:51 smithi165 ceph-mon[32648]: Reconfiguring daemon mgr.x on smithi151 2024-04-26T01:42:51.557 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:51 smithi165 ceph-mon[32648]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:51.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:51 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:51.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:51 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:51.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:51 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:51.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:51 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:51.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:51 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:51.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:51 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:51.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:51 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:42:51.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:51 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:51.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:51 smithi151 ceph-mon[28057]: Reconfiguring mgr.x (unknown last config time)... 2024-04-26T01:42:51.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:51 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T01:42:51.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:51 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T01:42:51.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:51 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:51.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:51 smithi151 ceph-mon[28057]: Reconfiguring daemon mgr.x on smithi151 2024-04-26T01:42:51.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:51 smithi151 ceph-mon[28057]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:51.844 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 01:42:51 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-y[34029]: 2024-04-26T01:42:51.559+0000 7f7153c8b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T01:42:51.845 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 01:42:51 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-y[34029]: 2024-04-26T01:42:51.652+0000 7f7153c8b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T01:42:51.945 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 01:42:51 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-z[34652]: 2024-04-26T01:42:51.604+0000 7f3f5f708200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T01:42:51.945 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 01:42:51 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-z[34652]: 2024-04-26T01:42:51.736+0000 7f3f5f708200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T01:42:51.945 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 01:42:51 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-z[34652]: 2024-04-26T01:42:51.832+0000 7f3f5f708200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T01:42:51.946 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 01:42:51 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-z[34652]: 2024-04-26T01:42:51.948+0000 7f3f5f708200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T01:42:52.333 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 01:42:51 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-y[34029]: 2024-04-26T01:42:51.858+0000 7f7153c8b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T01:42:52.334 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 01:42:52 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-y[34029]: 2024-04-26T01:42:52.050+0000 7f7153c8b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T01:42:52.535 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 01:42:52 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-z[34652]: 2024-04-26T01:42:52.252+0000 7f3f5f708200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T01:42:52.535 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:52 smithi172 ceph-mon[32785]: from='client.24113 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:42:52.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:52 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:42:52.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:52 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:42:52.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:52 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:52.594 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:52 smithi165 ceph-mon[32648]: from='client.24113 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:42:52.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:52 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:42:52.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:52 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:42:52.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:52 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:52.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:52 smithi151 ceph-mon[28057]: from='client.24113 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:42:52.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:52 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:42:52.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:52 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:42:52.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:52 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:53.076 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 01:42:52 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-y[34029]: 2024-04-26T01:42:52.655+0000 7f7153c8b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T01:42:53.285 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 01:42:52 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-z[34652]: 2024-04-26T01:42:52.850+0000 7f3f5f708200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T01:42:53.285 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 01:42:53 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-z[34652]: 2024-04-26T01:42:53.067+0000 7f3f5f708200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T01:42:53.344 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 01:42:53 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-y[34029]: 2024-04-26T01:42:53.079+0000 7f7153c8b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T01:42:53.344 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 01:42:53 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-y[34029]: 2024-04-26T01:42:53.178+0000 7f7153c8b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T01:42:53.844 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 01:42:53 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-y[34029]: 2024-04-26T01:42:53.382+0000 7f7153c8b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T01:42:53.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:53.963+0000 7f8d0cc00700 1 -- 172.21.15.151:0/3966348325 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mgrmap(e 14) ==== 92603+0+0 (secure 0 0 0) 0x7f8cfc00baa0 con 0x7f8d10105e60 2024-04-26T01:42:54.056 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:53 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:54.057 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:53 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:54.057 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:53 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:42:54.057 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:53 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:54.057 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:53 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:42:54.057 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:53 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:54.057 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:53 smithi151 ceph-mon[28057]: Standby manager daemon z started 2024-04-26T01:42:54.057 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:53 smithi151 ceph-mon[28057]: from='mgr.? 172.21.15.172:0/113357222' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T01:42:54.057 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:53 smithi151 ceph-mon[28057]: from='mgr.? 172.21.15.172:0/113357222' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T01:42:54.057 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:53 smithi151 ceph-mon[28057]: from='mgr.? 172.21.15.172:0/113357222' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T01:42:54.057 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:53 smithi151 ceph-mon[28057]: from='mgr.? 172.21.15.172:0/113357222' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T01:42:54.057 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:53 smithi151 ceph-mon[28057]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:54.244 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 01:42:53 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-y[34029]: 2024-04-26T01:42:53.906+0000 7f7153c8b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T01:42:54.244 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 01:42:54 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-y[34029]: 2024-04-26T01:42:54.001+0000 7f7153c8b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T01:42:54.245 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:53 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:54.245 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:53 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:54.245 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:53 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:42:54.245 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:53 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:54.245 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:53 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:42:54.245 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:53 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:54.245 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:53 smithi165 ceph-mon[32648]: Standby manager daemon z started 2024-04-26T01:42:54.245 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:53 smithi165 ceph-mon[32648]: from='mgr.? 172.21.15.172:0/113357222' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T01:42:54.245 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:53 smithi165 ceph-mon[32648]: from='mgr.? 172.21.15.172:0/113357222' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T01:42:54.245 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:53 smithi165 ceph-mon[32648]: from='mgr.? 172.21.15.172:0/113357222' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T01:42:54.245 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:53 smithi165 ceph-mon[32648]: from='mgr.? 172.21.15.172:0/113357222' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T01:42:54.245 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:53 smithi165 ceph-mon[32648]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:54.285 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:53 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:54.285 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:53 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:54.285 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:53 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:42:54.285 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:53 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:54.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:53 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:42:54.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:53 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:42:54.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:53 smithi172 ceph-mon[32785]: Standby manager daemon z started 2024-04-26T01:42:54.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:53 smithi172 ceph-mon[32785]: from='mgr.? 172.21.15.172:0/113357222' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T01:42:54.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:53 smithi172 ceph-mon[32785]: from='mgr.? 172.21.15.172:0/113357222' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T01:42:54.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:53 smithi172 ceph-mon[32785]: from='mgr.? 172.21.15.172:0/113357222' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T01:42:54.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:53 smithi172 ceph-mon[32785]: from='mgr.? 172.21.15.172:0/113357222' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T01:42:54.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:53 smithi172 ceph-mon[32785]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:54.592 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 01:42:54 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-y[34029]: 2024-04-26T01:42:54.247+0000 7f7153c8b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T01:42:54.593 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 01:42:54 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-y[34029]: 2024-04-26T01:42:54.381+0000 7f7153c8b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T01:42:54.593 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 01:42:54 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-y[34029]: 2024-04-26T01:42:54.478+0000 7f7153c8b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T01:42:54.593 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 01:42:54 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-y[34029]: 2024-04-26T01:42:54.595+0000 7f7153c8b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T01:42:55.285 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:54 smithi172 ceph-mon[32785]: mgrmap e14: x(active, since 88s), standbys: z 2024-04-26T01:42:55.285 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:54 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T01:42:55.344 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:54 smithi165 ceph-mon[32648]: mgrmap e14: x(active, since 88s), standbys: z 2024-04-26T01:42:55.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:54 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T01:42:55.345 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 01:42:54 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-y[34029]: 2024-04-26T01:42:54.909+0000 7f7153c8b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T01:42:55.370 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:54 smithi151 ceph-mon[28057]: mgrmap e14: x(active, since 88s), standbys: z 2024-04-26T01:42:55.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:54 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T01:42:55.845 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 01:42:55 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-y[34029]: 2024-04-26T01:42:55.514+0000 7f7153c8b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T01:42:55.845 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 01:42:55 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-y[34029]: 2024-04-26T01:42:55.731+0000 7f7153c8b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T01:42:55.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:42:55.984+0000 7f8d0cc00700 1 -- 172.21.15.151:0/3966348325 <== mon.2 v2:172.21.15.165:3300/0 8 ==== mgrmap(e 15) ==== 138591+0+0 (secure 0 0 0) 0x7f8cfc02c730 con 0x7f8d10105e60 2024-04-26T01:42:56.272 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:55 smithi151 ceph-mon[28057]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:56.272 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:55 smithi151 ceph-mon[28057]: from='mgr.? 172.21.15.165:0/3820634735' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T01:42:56.272 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:55 smithi151 ceph-mon[28057]: Standby manager daemon y started 2024-04-26T01:42:56.273 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:55 smithi151 ceph-mon[28057]: from='mgr.? 172.21.15.165:0/3820634735' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T01:42:56.273 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:55 smithi151 ceph-mon[28057]: from='mgr.? 172.21.15.165:0/3820634735' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T01:42:56.273 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:55 smithi151 ceph-mon[28057]: from='mgr.? 172.21.15.165:0/3820634735' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T01:42:56.273 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:55 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/2728785813' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ca27e814-a982-482c-ae1a-e596aac5c59a"}]: dispatch 2024-04-26T01:42:56.273 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:55 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/2728785813' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ca27e814-a982-482c-ae1a-e596aac5c59a"}]': finished 2024-04-26T01:42:56.273 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:55 smithi151 ceph-mon[28057]: osdmap e5: 1 total, 0 up, 1 in 2024-04-26T01:42:56.273 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:55 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:42:56.285 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:55 smithi172 ceph-mon[32785]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:56.285 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:55 smithi172 ceph-mon[32785]: from='mgr.? 172.21.15.165:0/3820634735' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T01:42:56.285 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:55 smithi172 ceph-mon[32785]: Standby manager daemon y started 2024-04-26T01:42:56.285 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:55 smithi172 ceph-mon[32785]: from='mgr.? 172.21.15.165:0/3820634735' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T01:42:56.285 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:55 smithi172 ceph-mon[32785]: from='mgr.? 172.21.15.165:0/3820634735' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T01:42:56.285 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:55 smithi172 ceph-mon[32785]: from='mgr.? 172.21.15.165:0/3820634735' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T01:42:56.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:55 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/2728785813' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ca27e814-a982-482c-ae1a-e596aac5c59a"}]: dispatch 2024-04-26T01:42:56.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:55 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/2728785813' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ca27e814-a982-482c-ae1a-e596aac5c59a"}]': finished 2024-04-26T01:42:56.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:55 smithi172 ceph-mon[32785]: osdmap e5: 1 total, 0 up, 1 in 2024-04-26T01:42:56.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:55 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:42:56.344 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:55 smithi165 ceph-mon[32648]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:56.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:55 smithi165 ceph-mon[32648]: from='mgr.? 172.21.15.165:0/3820634735' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T01:42:56.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:55 smithi165 ceph-mon[32648]: Standby manager daemon y started 2024-04-26T01:42:56.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:55 smithi165 ceph-mon[32648]: from='mgr.? 172.21.15.165:0/3820634735' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T01:42:56.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:55 smithi165 ceph-mon[32648]: from='mgr.? 172.21.15.165:0/3820634735' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T01:42:56.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:55 smithi165 ceph-mon[32648]: from='mgr.? 172.21.15.165:0/3820634735' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T01:42:56.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:55 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/2728785813' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ca27e814-a982-482c-ae1a-e596aac5c59a"}]: dispatch 2024-04-26T01:42:56.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:55 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/2728785813' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ca27e814-a982-482c-ae1a-e596aac5c59a"}]': finished 2024-04-26T01:42:56.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:55 smithi165 ceph-mon[32648]: osdmap e5: 1 total, 0 up, 1 in 2024-04-26T01:42:56.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:55 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:42:57.285 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:57 smithi172 ceph-mon[32785]: mgrmap e15: x(active, since 90s), standbys: y, z 2024-04-26T01:42:57.285 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:57 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T01:42:57.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:57 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/2237259102' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:42:57.300 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:57 smithi151 ceph-mon[28057]: mgrmap e15: x(active, since 90s), standbys: y, z 2024-04-26T01:42:57.301 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:57 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T01:42:57.301 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:57 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/2237259102' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:42:57.344 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:57 smithi165 ceph-mon[32648]: mgrmap e15: x(active, since 90s), standbys: y, z 2024-04-26T01:42:57.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:57 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T01:42:57.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:57 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/2237259102' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:42:58.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:58 smithi165 ceph-mon[32648]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:58.370 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:58 smithi151 ceph-mon[28057]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:58.535 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:58 smithi172 ceph-mon[32785]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:59.594 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:42:59 smithi165 ceph-mon[32648]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:59.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:42:59 smithi151 ceph-mon[28057]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:42:59.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:42:59 smithi172 ceph-mon[32785]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:43:01.588 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:01 smithi172 ceph-mon[32785]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:43:01.594 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:01 smithi165 ceph-mon[32648]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:43:01.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:01 smithi151 ceph-mon[28057]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:43:03.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:03 smithi165 ceph-mon[32648]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:43:03.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:03 smithi151 ceph-mon[28057]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:43:03.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:03 smithi172 ceph-mon[32785]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:43:05.370 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:05 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-26T01:43:05.370 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:05 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:43:05.535 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:05 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-26T01:43:05.535 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:05 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:43:05.594 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:05 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-26T01:43:05.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:05 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:43:06.370 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:06 smithi151 ceph-mon[28057]: Deploying daemon osd.0 on smithi151 2024-04-26T01:43:06.370 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:06 smithi151 ceph-mon[28057]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:43:06.535 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:06 smithi172 ceph-mon[32785]: Deploying daemon osd.0 on smithi151 2024-04-26T01:43:06.535 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:06 smithi172 ceph-mon[32785]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:43:06.594 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:06 smithi165 ceph-mon[32648]: Deploying daemon osd.0 on smithi151 2024-04-26T01:43:06.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:06 smithi165 ceph-mon[32648]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:43:07.594 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:07 smithi165 ceph-mon[32648]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:43:07.606 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:07 smithi151 ceph-mon[28057]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:43:07.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:07 smithi172 ceph-mon[32785]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:43:09.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:09 smithi165 ceph-mon[32648]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:43:09.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:09 smithi151 ceph-mon[28057]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:43:09.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:09 smithi172 ceph-mon[32785]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:43:11.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:11 smithi165 ceph-mon[32648]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:43:11.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:11 smithi151 ceph-mon[28057]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:43:11.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:11 smithi172 ceph-mon[32785]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:43:13.594 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:13 smithi165 ceph-mon[32648]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:43:13.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:13 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:13.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:13 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:13.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:13 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:43:13.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:13 smithi151 ceph-mon[28057]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:43:13.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:13 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:13.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:13 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:13.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:13 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:43:13.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:13 smithi172 ceph-mon[32785]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:43:13.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:13 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:13.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:13 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:13.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:13 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:43:15.436 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:15 smithi151 ceph-mon[28057]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:43:15.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:15 smithi165 ceph-mon[32648]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:43:15.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:15 smithi172 ceph-mon[32785]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:43:17.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:43:17.750+0000 7f8d0cc00700 1 -- 172.21.15.151:0/3966348325 <== mgr.14152 v2:172.21.15.151:6800/2251256505 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8d10062e80 con 0x7f8cf4039890 2024-04-26T01:43:17.748 INFO:teuthology.orchestra.run.smithi151.stdout:Created osd(s) 0 on host 'smithi151' 2024-04-26T01:43:17.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:43:17.754+0000 7f8d17000700 1 -- 172.21.15.151:0/3966348325 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f8cf4039890 msgr2=0x7f8cf403bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:43:17.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:43:17.754+0000 7f8d17000700 1 --2- 172.21.15.151:0/3966348325 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f8cf4039890 0x7f8cf403bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8d10105e30 tx=0x7f8d00009260 comp rx=0 tx=0).stop 2024-04-26T01:43:17.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:43:17.754+0000 7f8d17000700 1 -- 172.21.15.151:0/3966348325 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8d10105e60 msgr2=0x7f8d10199230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:43:17.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:43:17.754+0000 7f8d17000700 1 --2- 172.21.15.151:0/3966348325 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8d10105e60 0x7f8d10199230 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc00c010 tx=0x7f8cfc005270 comp rx=0 tx=0).stop 2024-04-26T01:43:17.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:43:17.754+0000 7f8d17000700 1 -- 172.21.15.151:0/3966348325 shutdown_connections 2024-04-26T01:43:17.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:43:17.754+0000 7f8d17000700 1 --2- 172.21.15.151:0/3966348325 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f8cf4039890 0x7f8cf403bd50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:43:17.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:43:17.754+0000 7f8d17000700 1 --2- 172.21.15.151:0/3966348325 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8d10108790 0x7f8d101a0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:43:17.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:43:17.754+0000 7f8d17000700 1 --2- 172.21.15.151:0/3966348325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d10103530 0x7f8d10198cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:43:17.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:43:17.754+0000 7f8d17000700 1 --2- 172.21.15.151:0/3966348325 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8d10105e60 0x7f8d10199230 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:43:17.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:43:17.754+0000 7f8d17000700 1 -- 172.21.15.151:0/3966348325 >> 172.21.15.151:0/3966348325 conn(0x7f8d100fcf40 msgr2=0x7f8d10105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:43:17.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:43:17.755+0000 7f8d17000700 1 -- 172.21.15.151:0/3966348325 shutdown_connections 2024-04-26T01:43:17.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:43:17.755+0000 7f8d17000700 1 -- 172.21.15.151:0/3966348325 wait complete. 2024-04-26T01:43:18.285 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:18 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:18.285 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:18 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:18.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:18 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:43:18.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:18 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:43:18.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:18 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:18.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:18 smithi172 ceph-mon[32785]: from='osd.0 [v2:172.21.15.151:6802/669937998,v1:172.21.15.151:6803/669937998]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T01:43:18.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:18 smithi172 ceph-mon[32785]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T01:43:18.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:18 smithi172 ceph-mon[32785]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:43:18.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:18 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:18.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:18 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:18.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:18 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:43:18.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:18 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:18.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:18 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:18.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:18 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:43:18.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:18 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:43:18.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:18 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:18.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:18 smithi165 ceph-mon[32648]: from='osd.0 [v2:172.21.15.151:6802/669937998,v1:172.21.15.151:6803/669937998]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T01:43:18.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:18 smithi165 ceph-mon[32648]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T01:43:18.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:18 smithi165 ceph-mon[32648]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:43:18.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:18 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:18.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:18 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:18.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:18 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:43:18.370 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:18 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:18.370 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:18 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:18.370 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:18 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:43:18.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:18 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:43:18.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:18 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:18.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:18 smithi151 ceph-mon[28057]: from='osd.0 [v2:172.21.15.151:6802/669937998,v1:172.21.15.151:6803/669937998]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T01:43:18.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:18 smithi151 ceph-mon[28057]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T01:43:18.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:18 smithi151 ceph-mon[28057]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:43:18.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:18 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:18.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:18 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:18.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:18 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:43:18.479 DEBUG:teuthology.orchestra.run.smithi151:osd.0> sudo journalctl -f -n 0 -u ceph-c0e49e64-036d-11ef-bc93-c7b262605968@osd.0.service 2024-04-26T01:43:18.484 INFO:tasks.cephadm:Deploying osd.1 on smithi165 with /dev/vg_nvme/lv_4... 2024-04-26T01:43:18.484 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-26T01:43:18.726 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T01:43:19.109 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:19 smithi165 ceph-mon[32648]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-26T01:43:19.110 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:19 smithi165 ceph-mon[32648]: from='osd.0 [v2:172.21.15.151:6802/669937998,v1:172.21.15.151:6803/669937998]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-04-26T01:43:19.110 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:19 smithi165 ceph-mon[32648]: osdmap e6: 1 total, 0 up, 1 in 2024-04-26T01:43:19.110 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:19 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:43:19.110 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:19 smithi165 ceph-mon[32648]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-04-26T01:43:19.370 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:19 smithi151 ceph-mon[28057]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-26T01:43:19.370 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:19 smithi151 ceph-mon[28057]: from='osd.0 [v2:172.21.15.151:6802/669937998,v1:172.21.15.151:6803/669937998]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-04-26T01:43:19.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:19 smithi151 ceph-mon[28057]: osdmap e6: 1 total, 0 up, 1 in 2024-04-26T01:43:19.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:19 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:43:19.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:19 smithi151 ceph-mon[28057]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-04-26T01:43:19.535 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:19 smithi172 ceph-mon[32785]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-26T01:43:19.535 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:19 smithi172 ceph-mon[32785]: from='osd.0 [v2:172.21.15.151:6802/669937998,v1:172.21.15.151:6803/669937998]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-04-26T01:43:19.535 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:19 smithi172 ceph-mon[32785]: osdmap e6: 1 total, 0 up, 1 in 2024-04-26T01:43:19.535 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:19 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:43:19.535 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:19 smithi172 ceph-mon[32785]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-04-26T01:43:20.324 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:20 smithi151 ceph-mon[28057]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-04-26T01:43:20.324 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:20 smithi151 ceph-mon[28057]: osdmap e7: 1 total, 0 up, 1 in 2024-04-26T01:43:20.324 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:20 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:43:20.325 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:20 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:43:20.325 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:20 smithi151 ceph-mon[28057]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:43:20.325 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:20 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:43:20.325 INFO:journalctl@ceph.osd.0.smithi151.stdout:Apr 26 01:43:20 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-0[42271]: 2024-04-26T01:43:20.079+0000 7f1d0a000700 -1 osd.0 0 waiting for initial osdmap 2024-04-26T01:43:20.325 INFO:journalctl@ceph.osd.0.smithi151.stdout:Apr 26 01:43:20 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-0[42271]: 2024-04-26T01:43:20.116+0000 7f1cff400700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T01:43:20.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:20 smithi165 ceph-mon[32648]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-04-26T01:43:20.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:20 smithi165 ceph-mon[32648]: osdmap e7: 1 total, 0 up, 1 in 2024-04-26T01:43:20.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:20 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:43:20.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:20 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:43:20.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:20 smithi165 ceph-mon[32648]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:43:20.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:20 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:43:20.404 INFO:teuthology.orchestra.run.smithi165.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-26T01:43:20.405 INFO:teuthology.orchestra.run.smithi165.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-26T01:43:20.405 INFO:teuthology.orchestra.run.smithi165.stderr: stderr: 10+0 records in 2024-04-26T01:43:20.405 INFO:teuthology.orchestra.run.smithi165.stderr:10+0 records out 2024-04-26T01:43:20.405 INFO:teuthology.orchestra.run.smithi165.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113829 s, 921 MB/s 2024-04-26T01:43:20.405 INFO:teuthology.orchestra.run.smithi165.stderr:--> Zapping successful for: 2024-04-26T01:43:20.535 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:20 smithi172 ceph-mon[32785]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-04-26T01:43:20.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:20 smithi172 ceph-mon[32785]: osdmap e7: 1 total, 0 up, 1 in 2024-04-26T01:43:20.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:20 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:43:20.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:20 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:43:20.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:20 smithi172 ceph-mon[32785]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:43:20.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:20 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:43:20.915 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi165:vg_nvme/lv_4 2024-04-26T01:43:21.148 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T01:43:21.344 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:21 smithi165 ceph-mon[32648]: purged_snaps scrub starts 2024-04-26T01:43:21.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:21 smithi165 ceph-mon[32648]: purged_snaps scrub ok 2024-04-26T01:43:21.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:21 smithi165 ceph-mon[32648]: from='osd.0 ' entity='osd.0' 2024-04-26T01:43:21.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:21 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:43:21.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:21 smithi151 ceph-mon[28057]: purged_snaps scrub starts 2024-04-26T01:43:21.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:21 smithi151 ceph-mon[28057]: purged_snaps scrub ok 2024-04-26T01:43:21.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:21 smithi151 ceph-mon[28057]: from='osd.0 ' entity='osd.0' 2024-04-26T01:43:21.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:21 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:43:21.535 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:21 smithi172 ceph-mon[32785]: purged_snaps scrub starts 2024-04-26T01:43:21.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:21 smithi172 ceph-mon[32785]: purged_snaps scrub ok 2024-04-26T01:43:21.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:21 smithi172 ceph-mon[32785]: from='osd.0 ' entity='osd.0' 2024-04-26T01:43:21.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:21 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:43:22.370 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:22 smithi151 ceph-mon[28057]: osd.0 [v2:172.21.15.151:6802/669937998,v1:172.21.15.151:6803/669937998] boot 2024-04-26T01:43:22.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:22 smithi151 ceph-mon[28057]: osdmap e8: 1 total, 1 up, 1 in 2024-04-26T01:43:22.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:22 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:43:22.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:22 smithi151 ceph-mon[28057]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:43:22.387 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:22 smithi165 ceph-mon[32648]: osd.0 [v2:172.21.15.151:6802/669937998,v1:172.21.15.151:6803/669937998] boot 2024-04-26T01:43:22.387 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:22 smithi165 ceph-mon[32648]: osdmap e8: 1 total, 1 up, 1 in 2024-04-26T01:43:22.387 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:22 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:43:22.387 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:22 smithi165 ceph-mon[32648]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:43:22.535 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:22 smithi172 ceph-mon[32785]: osd.0 [v2:172.21.15.151:6802/669937998,v1:172.21.15.151:6803/669937998] boot 2024-04-26T01:43:22.535 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:22 smithi172 ceph-mon[32785]: osdmap e8: 1 total, 1 up, 1 in 2024-04-26T01:43:22.535 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:22 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:43:22.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:22 smithi172 ceph-mon[32785]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:43:22.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:43:22.704+0000 7fa141512700 1 -- 172.21.15.165:0/1882814584 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa13c103540 msgr2=0x7fa13c109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:43:22.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:43:22.704+0000 7fa141512700 1 --2- 172.21.15.165:0/1882814584 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa13c103540 0x7fa13c109df0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa13400b0d0 tx=0x7fa13400b3e0 comp rx=0 tx=0).stop 2024-04-26T01:43:22.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:43:22.704+0000 7fa141512700 1 -- 172.21.15.165:0/1882814584 shutdown_connections 2024-04-26T01:43:22.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:43:22.704+0000 7fa141512700 1 --2- 172.21.15.165:0/1882814584 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa13c103540 0x7fa13c109df0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:43:22.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:43:22.704+0000 7fa141512700 1 --2- 172.21.15.165:0/1882814584 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa13c1019a0 0x7fa13c101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:43:22.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:43:22.704+0000 7fa141512700 1 --2- 172.21.15.165:0/1882814584 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa13c102ba0 0x7fa13c103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:43:22.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:43:22.704+0000 7fa141512700 1 -- 172.21.15.165:0/1882814584 >> 172.21.15.165:0/1882814584 conn(0x7fa13c0fcf60 msgr2=0x7fa13c0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:43:22.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:43:22.704+0000 7fa141512700 1 -- 172.21.15.165:0/1882814584 shutdown_connections 2024-04-26T01:43:22.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:43:22.704+0000 7fa141512700 1 -- 172.21.15.165:0/1882814584 wait complete. 2024-04-26T01:43:22.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:43:22.704+0000 7fa141512700 1 Processor -- start 2024-04-26T01:43:22.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:43:22.704+0000 7fa141512700 1 -- start start 2024-04-26T01:43:22.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:43:22.705+0000 7fa141512700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa13c1019a0 0x7fa13c198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:43:22.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:43:22.705+0000 7fa141512700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa13c102ba0 0x7fa13c199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:43:22.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:43:22.705+0000 7fa141512700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa13c103540 0x7fa13c1a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:43:22.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:43:22.705+0000 7fa141512700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa13c10d4b0 con 0x7fa13c1019a0 2024-04-26T01:43:22.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:43:22.705+0000 7fa141512700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa13c10d330 con 0x7fa13c102ba0 2024-04-26T01:43:22.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:43:22.705+0000 7fa141512700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa13c10d630 con 0x7fa13c103540 2024-04-26T01:43:22.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:43:22.705+0000 7fa13a7fc700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa13c102ba0 0x7fa13c199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:43:22.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:43:22.705+0000 7fa13a7fc700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa13c102ba0 0x7fa13c199120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48346/0 (socket says 172.21.15.165:48346) 2024-04-26T01:43:22.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:43:22.705+0000 7fa13a7fc700 1 -- 172.21.15.165:0/1237765670 learned_addr learned my addr 172.21.15.165:0/1237765670 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-26T01:43:22.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:43:22.705+0000 7fa13b7fe700 1 --2- 172.21.15.165:0/1237765670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa13c103540 0x7fa13c1a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:43:22.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:43:22.705+0000 7fa13a7fc700 1 -- 172.21.15.165:0/1237765670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa13c103540 msgr2=0x7fa13c1a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:43:22.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:43:22.705+0000 7fa13a7fc700 1 --2- 172.21.15.165:0/1237765670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa13c103540 0x7fa13c1a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:43:22.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:43:22.705+0000 7fa13a7fc700 1 -- 172.21.15.165:0/1237765670 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa13c1019a0 msgr2=0x7fa13c198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:43:22.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:43:22.705+0000 7fa13a7fc700 1 --2- 172.21.15.165:0/1237765670 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa13c1019a0 0x7fa13c198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:43:22.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:43:22.705+0000 7fa13a7fc700 1 -- 172.21.15.165:0/1237765670 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa13c1a0840 con 0x7fa13c102ba0 2024-04-26T01:43:22.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:43:22.705+0000 7fa13b7fe700 1 --2- 172.21.15.165:0/1237765670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa13c103540 0x7fa13c1a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:43:22.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:43:22.705+0000 7fa13a7fc700 1 --2- 172.21.15.165:0/1237765670 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa13c102ba0 0x7fa13c199120 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa13000b530 tx=0x7fa13000b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:43:22.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:43:22.705+0000 7fa123fff700 1 -- 172.21.15.165:0/1237765670 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa130015020 con 0x7fa13c102ba0 2024-04-26T01:43:22.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:43:22.705+0000 7fa123fff700 1 -- 172.21.15.165:0/1237765670 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa130011ce0 con 0x7fa13c102ba0 2024-04-26T01:43:22.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:43:22.705+0000 7fa141512700 1 -- 172.21.15.165:0/1237765670 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa13c1a0b30 con 0x7fa13c102ba0 2024-04-26T01:43:22.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:43:22.705+0000 7fa123fff700 1 -- 172.21.15.165:0/1237765670 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa130010d50 con 0x7fa13c102ba0 2024-04-26T01:43:22.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:43:22.705+0000 7fa141512700 1 -- 172.21.15.165:0/1237765670 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa13c1a1070 con 0x7fa13c102ba0 2024-04-26T01:43:22.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:43:22.706+0000 7fa141512700 1 -- 172.21.15.165:0/1237765670 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa13c069760 con 0x7fa13c102ba0 2024-04-26T01:43:22.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:43:22.707+0000 7fa123fff700 1 -- 172.21.15.165:0/1237765670 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 15) ==== 138591+0+0 (secure 0 0 0) 0x7fa130018020 con 0x7fa13c102ba0 2024-04-26T01:43:22.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:43:22.708+0000 7fa123fff700 1 --2- 172.21.15.165:0/1237765670 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fa1240a3bb0 0x7fa1240a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:43:22.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:43:22.708+0000 7fa123fff700 1 -- 172.21.15.165:0/1237765670 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7fa1300ce150 con 0x7fa13c102ba0 2024-04-26T01:43:22.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:43:22.708+0000 7fa13affd700 1 --2- 172.21.15.165:0/1237765670 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fa1240a3bb0 0x7fa1240a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:43:22.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:43:22.709+0000 7fa13affd700 1 --2- 172.21.15.165:0/1237765670 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fa1240a3bb0 0x7fa1240a6070 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa13c102a00 tx=0x7fa12c009470 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:43:22.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:43:22.713+0000 7fa123fff700 1 -- 172.21.15.165:0/1237765670 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa13c069760 con 0x7fa13c102ba0 2024-04-26T01:43:22.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:43:22.919+0000 7fa141512700 1 -- 172.21.15.165:0/1237765670 --> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fa13c0ff140 con 0x7fa1240a3bb0 2024-04-26T01:43:23.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:23 smithi151 ceph-mon[28057]: osdmap e9: 1 total, 1 up, 1 in 2024-04-26T01:43:23.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:23 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:43:23.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:23 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:43:23.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:23 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:43:23.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:23 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:23.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:23 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:23.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:23 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:43:23.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:23 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:23.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:23 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:43:23.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:23 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:43:23.535 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:23 smithi172 ceph-mon[32785]: osdmap e9: 1 total, 1 up, 1 in 2024-04-26T01:43:23.535 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:23 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:43:23.535 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:23 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:43:23.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:23 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:43:23.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:23 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:23.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:23 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:23.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:23 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:43:23.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:23 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:23.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:23 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:43:23.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:23 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:43:23.561 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:23 smithi165 ceph-mon[32648]: osdmap e9: 1 total, 1 up, 1 in 2024-04-26T01:43:23.561 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:23 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:43:23.561 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:23 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:43:23.561 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:23 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:43:23.561 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:23 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:23.562 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:23 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:23.562 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:23 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:43:23.562 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:23 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:23.562 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:23 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:43:23.562 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:23 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:43:24.429 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:24 smithi165 ceph-mon[32648]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:43:24.429 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:24 smithi165 ceph-mon[32648]: Detected new or changed devices on smithi151 2024-04-26T01:43:24.430 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:24 smithi165 ceph-mon[32648]: Adjusting osd_memory_target on smithi151 to 17089M 2024-04-26T01:43:24.430 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:24 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:24.430 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:24 smithi165 ceph-mon[32648]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:43:24.535 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:24 smithi172 ceph-mon[32785]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:43:24.535 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:24 smithi172 ceph-mon[32785]: Detected new or changed devices on smithi151 2024-04-26T01:43:24.535 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:24 smithi172 ceph-mon[32785]: Adjusting osd_memory_target on smithi151 to 17089M 2024-04-26T01:43:24.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:24 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:24.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:24 smithi172 ceph-mon[32785]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:43:24.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:24 smithi151 ceph-mon[28057]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:43:24.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:24 smithi151 ceph-mon[28057]: Detected new or changed devices on smithi151 2024-04-26T01:43:24.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:24 smithi151 ceph-mon[28057]: Adjusting osd_memory_target on smithi151 to 17089M 2024-04-26T01:43:24.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:24 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:24.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:24 smithi151 ceph-mon[28057]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:43:25.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:25 smithi165 ceph-mon[32648]: pgmap v59: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:43:25.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:25 smithi151 ceph-mon[28057]: pgmap v59: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:43:25.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:25 smithi172 ceph-mon[32785]: pgmap v59: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:43:26.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:26 smithi151 ceph-mon[28057]: from='client.? 172.21.15.165:0/2568752342' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7d192c3-9624-4625-9ccb-a76c114d8aad"}]: dispatch 2024-04-26T01:43:26.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:26 smithi151 ceph-mon[28057]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7d192c3-9624-4625-9ccb-a76c114d8aad"}]: dispatch 2024-04-26T01:43:26.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:26 smithi151 ceph-mon[28057]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7d192c3-9624-4625-9ccb-a76c114d8aad"}]': finished 2024-04-26T01:43:26.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:26 smithi151 ceph-mon[28057]: osdmap e10: 2 total, 1 up, 2 in 2024-04-26T01:43:26.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:26 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:43:26.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:26 smithi151 ceph-mon[28057]: from='client.? 172.21.15.165:0/18062449' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:43:26.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:26 smithi172 ceph-mon[32785]: from='client.? 172.21.15.165:0/2568752342' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7d192c3-9624-4625-9ccb-a76c114d8aad"}]: dispatch 2024-04-26T01:43:26.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:26 smithi172 ceph-mon[32785]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7d192c3-9624-4625-9ccb-a76c114d8aad"}]: dispatch 2024-04-26T01:43:26.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:26 smithi172 ceph-mon[32785]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7d192c3-9624-4625-9ccb-a76c114d8aad"}]': finished 2024-04-26T01:43:26.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:26 smithi172 ceph-mon[32785]: osdmap e10: 2 total, 1 up, 2 in 2024-04-26T01:43:26.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:26 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:43:26.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:26 smithi172 ceph-mon[32785]: from='client.? 172.21.15.165:0/18062449' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:43:26.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:26 smithi165 ceph-mon[32648]: from='client.? 172.21.15.165:0/2568752342' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7d192c3-9624-4625-9ccb-a76c114d8aad"}]: dispatch 2024-04-26T01:43:26.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:26 smithi165 ceph-mon[32648]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7d192c3-9624-4625-9ccb-a76c114d8aad"}]: dispatch 2024-04-26T01:43:26.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:26 smithi165 ceph-mon[32648]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7d192c3-9624-4625-9ccb-a76c114d8aad"}]': finished 2024-04-26T01:43:26.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:26 smithi165 ceph-mon[32648]: osdmap e10: 2 total, 1 up, 2 in 2024-04-26T01:43:26.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:26 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:43:26.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:26 smithi165 ceph-mon[32648]: from='client.? 172.21.15.165:0/18062449' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:43:27.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:27 smithi151 ceph-mon[28057]: pgmap v61: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:43:27.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:27 smithi172 ceph-mon[32785]: pgmap v61: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:43:27.844 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:27 smithi165 ceph-mon[32648]: pgmap v61: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:43:29.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:29 smithi165 ceph-mon[32648]: pgmap v62: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:43:29.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:29 smithi151 ceph-mon[28057]: pgmap v62: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:43:29.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:29 smithi172 ceph-mon[32785]: pgmap v62: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:43:31.594 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:31 smithi165 ceph-mon[32648]: pgmap v63: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:43:31.601 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:31 smithi172 ceph-mon[32785]: pgmap v63: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:43:31.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:31 smithi151 ceph-mon[28057]: pgmap v63: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:43:33.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:32 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-26T01:43:33.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:32 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:43:33.094 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:32 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-26T01:43:33.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:32 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:43:33.120 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:32 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-26T01:43:33.120 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:32 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:43:33.844 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:33 smithi165 ceph-mon[32648]: Deploying daemon osd.1 on smithi165 2024-04-26T01:43:33.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:33 smithi165 ceph-mon[32648]: pgmap v64: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:43:34.035 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:33 smithi172 ceph-mon[32785]: Deploying daemon osd.1 on smithi165 2024-04-26T01:43:34.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:33 smithi172 ceph-mon[32785]: pgmap v64: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:43:34.120 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:33 smithi151 ceph-mon[28057]: Deploying daemon osd.1 on smithi165 2024-04-26T01:43:34.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:33 smithi151 ceph-mon[28057]: pgmap v64: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:43:35.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:35 smithi165 ceph-mon[32648]: pgmap v65: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:43:35.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:35 smithi151 ceph-mon[28057]: pgmap v65: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:43:35.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:35 smithi172 ceph-mon[32785]: pgmap v65: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:43:37.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:37 smithi165 ceph-mon[32648]: pgmap v66: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:43:37.601 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:37 smithi151 ceph-mon[28057]: pgmap v66: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:43:37.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:37 smithi172 ceph-mon[32785]: pgmap v66: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:43:39.285 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:38 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:39.285 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:38 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:39.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:38 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:43:39.299 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:38 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:39.299 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:38 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:39.299 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:38 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:43:39.370 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:38 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:39.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:38 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:39.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:38 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:43:40.010 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:39 smithi165 ceph-mon[32648]: pgmap v67: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:43:40.285 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:39 smithi172 ceph-mon[32785]: pgmap v67: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:43:40.370 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:39 smithi151 ceph-mon[28057]: pgmap v67: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:43:41.373 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:41 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:41.373 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:41 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:41.374 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:41 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:43:41.374 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:41 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:43:41.374 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:41 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:41.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:43:41.463+0000 7fa123fff700 1 -- 172.21.15.165:0/1237765670 <== mgr.14152 v2:172.21.15.151:6800/2251256505 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa13c0ff140 con 0x7fa1240a3bb0 2024-04-26T01:43:41.462 INFO:teuthology.orchestra.run.smithi165.stdout:Created osd(s) 1 on host 'smithi165' 2024-04-26T01:43:41.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:43:41.465+0000 7fa141512700 1 -- 172.21.15.165:0/1237765670 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fa1240a3bb0 msgr2=0x7fa1240a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:43:41.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:43:41.465+0000 7fa141512700 1 --2- 172.21.15.165:0/1237765670 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fa1240a3bb0 0x7fa1240a6070 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa13c102a00 tx=0x7fa12c009470 comp rx=0 tx=0).stop 2024-04-26T01:43:41.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:43:41.465+0000 7fa141512700 1 -- 172.21.15.165:0/1237765670 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa13c102ba0 msgr2=0x7fa13c199120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:43:41.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:43:41.465+0000 7fa141512700 1 --2- 172.21.15.165:0/1237765670 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa13c102ba0 0x7fa13c199120 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa13000b530 tx=0x7fa13000b8f0 comp rx=0 tx=0).stop 2024-04-26T01:43:41.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:43:41.466+0000 7fa141512700 1 -- 172.21.15.165:0/1237765670 shutdown_connections 2024-04-26T01:43:41.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:43:41.466+0000 7fa141512700 1 --2- 172.21.15.165:0/1237765670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa13c103540 0x7fa13c1a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:43:41.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:43:41.466+0000 7fa141512700 1 --2- 172.21.15.165:0/1237765670 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fa1240a3bb0 0x7fa1240a6070 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:43:41.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:43:41.466+0000 7fa141512700 1 --2- 172.21.15.165:0/1237765670 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa13c1019a0 0x7fa13c198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:43:41.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:43:41.466+0000 7fa141512700 1 --2- 172.21.15.165:0/1237765670 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa13c102ba0 0x7fa13c199120 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:43:41.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:43:41.466+0000 7fa141512700 1 -- 172.21.15.165:0/1237765670 >> 172.21.15.165:0/1237765670 conn(0x7fa13c0fcf60 msgr2=0x7fa13c0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:43:41.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:43:41.466+0000 7fa141512700 1 -- 172.21.15.165:0/1237765670 shutdown_connections 2024-04-26T01:43:41.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:43:41.466+0000 7fa141512700 1 -- 172.21.15.165:0/1237765670 wait complete. 2024-04-26T01:43:41.535 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:41 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:41.535 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:41 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:41.535 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:41 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:43:41.535 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:41 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:43:41.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:41 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:41.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:41 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:41.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:41 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:41.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:41 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:43:41.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:41 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:43:41.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:41 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:42.046 DEBUG:teuthology.orchestra.run.smithi165:osd.1> sudo journalctl -f -n 0 -u ceph-c0e49e64-036d-11ef-bc93-c7b262605968@osd.1.service 2024-04-26T01:43:42.049 INFO:tasks.cephadm:Deploying osd.2 on smithi172 with /dev/vg_nvme/lv_4... 2024-04-26T01:43:42.049 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-26T01:43:42.297 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T01:43:42.344 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:42 smithi165 ceph-mon[32648]: pgmap v68: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:43:42.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:42 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:42.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:42 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:42.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:42 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:43:42.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:42 smithi165 ceph-mon[32648]: from='osd.1 [v2:172.21.15.165:6800/1356168171,v1:172.21.15.165:6801/1356168171]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T01:43:42.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:42 smithi165 ceph-mon[32648]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T01:43:42.413 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:42 smithi172 ceph-mon[32785]: pgmap v68: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:43:42.413 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:42 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:42.413 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:42 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:42.413 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:42 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:43:42.413 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:42 smithi172 ceph-mon[32785]: from='osd.1 [v2:172.21.15.165:6800/1356168171,v1:172.21.15.165:6801/1356168171]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T01:43:42.413 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:42 smithi172 ceph-mon[32785]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T01:43:42.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:42 smithi151 ceph-mon[28057]: pgmap v68: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:43:42.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:42 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:42.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:42 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:42.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:42 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:43:42.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:42 smithi151 ceph-mon[28057]: from='osd.1 [v2:172.21.15.165:6800/1356168171,v1:172.21.15.165:6801/1356168171]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T01:43:42.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:42 smithi151 ceph-mon[28057]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T01:43:43.637 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:43 smithi172 ceph-mon[32785]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-26T01:43:43.638 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:43 smithi172 ceph-mon[32785]: from='osd.1 [v2:172.21.15.165:6800/1356168171,v1:172.21.15.165:6801/1356168171]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-26T01:43:43.638 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:43 smithi172 ceph-mon[32785]: osdmap e11: 2 total, 1 up, 2 in 2024-04-26T01:43:43.638 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:43 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:43:43.638 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:43 smithi172 ceph-mon[32785]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-26T01:43:43.638 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:43 smithi172 ceph-mon[32785]: pgmap v70: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:43:43.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:43 smithi165 ceph-mon[32648]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-26T01:43:43.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:43 smithi165 ceph-mon[32648]: from='osd.1 [v2:172.21.15.165:6800/1356168171,v1:172.21.15.165:6801/1356168171]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-26T01:43:43.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:43 smithi165 ceph-mon[32648]: osdmap e11: 2 total, 1 up, 2 in 2024-04-26T01:43:43.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:43 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:43:43.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:43 smithi165 ceph-mon[32648]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-26T01:43:43.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:43 smithi165 ceph-mon[32648]: pgmap v70: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:43:43.870 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:43 smithi151 ceph-mon[28057]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-26T01:43:43.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:43 smithi151 ceph-mon[28057]: from='osd.1 [v2:172.21.15.165:6800/1356168171,v1:172.21.15.165:6801/1356168171]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-26T01:43:43.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:43 smithi151 ceph-mon[28057]: osdmap e11: 2 total, 1 up, 2 in 2024-04-26T01:43:43.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:43 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:43:43.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:43 smithi151 ceph-mon[28057]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-26T01:43:43.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:43 smithi151 ceph-mon[28057]: pgmap v70: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:43:44.190 INFO:teuthology.orchestra.run.smithi172.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-26T01:43:44.190 INFO:teuthology.orchestra.run.smithi172.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-26T01:43:44.190 INFO:teuthology.orchestra.run.smithi172.stderr: stderr: 10+0 records in 2024-04-26T01:43:44.190 INFO:teuthology.orchestra.run.smithi172.stderr:10+0 records out 2024-04-26T01:43:44.190 INFO:teuthology.orchestra.run.smithi172.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0133037 s, 788 MB/s 2024-04-26T01:43:44.190 INFO:teuthology.orchestra.run.smithi172.stderr:--> Zapping successful for: 2024-04-26T01:43:44.576 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:44 smithi165 ceph-mon[32648]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-04-26T01:43:44.576 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:44 smithi165 ceph-mon[32648]: osdmap e12: 2 total, 1 up, 2 in 2024-04-26T01:43:44.576 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:44 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:43:44.576 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:44 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:43:44.576 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:44 smithi165 ceph-mon[32648]: from='osd.1 ' entity='osd.1' 2024-04-26T01:43:44.576 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:44 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:43:44.576 INFO:journalctl@ceph.osd.1.smithi165.stdout:Apr 26 01:43:44 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-1[37251]: 2024-04-26T01:43:44.354+0000 7f0f53cfc700 -1 osd.1 0 waiting for initial osdmap 2024-04-26T01:43:44.577 INFO:journalctl@ceph.osd.1.smithi165.stdout:Apr 26 01:43:44 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-1[37251]: 2024-04-26T01:43:44.479+0000 7f0f4b2e6700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T01:43:44.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:44 smithi172 ceph-mon[32785]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-04-26T01:43:44.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:44 smithi172 ceph-mon[32785]: osdmap e12: 2 total, 1 up, 2 in 2024-04-26T01:43:44.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:44 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:43:44.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:44 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:43:44.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:44 smithi172 ceph-mon[32785]: from='osd.1 ' entity='osd.1' 2024-04-26T01:43:44.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:44 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:43:44.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:44 smithi151 ceph-mon[28057]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-04-26T01:43:44.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:44 smithi151 ceph-mon[28057]: osdmap e12: 2 total, 1 up, 2 in 2024-04-26T01:43:44.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:44 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:43:44.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:44 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:43:44.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:44 smithi151 ceph-mon[28057]: from='osd.1 ' entity='osd.1' 2024-04-26T01:43:44.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:44 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:43:44.903 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi172:vg_nvme/lv_4 2024-04-26T01:43:45.149 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T01:43:45.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:45 smithi172 ceph-mon[32785]: purged_snaps scrub starts 2024-04-26T01:43:45.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:45 smithi172 ceph-mon[32785]: purged_snaps scrub ok 2024-04-26T01:43:45.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:45 smithi172 ceph-mon[32785]: Detected new or changed devices on smithi165 2024-04-26T01:43:45.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:45 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:45.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:45 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:45.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:45 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:43:45.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:45 smithi172 ceph-mon[32785]: Adjusting osd_memory_target on smithi165 to 17088M 2024-04-26T01:43:45.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:45 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:45.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:45 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:43:45.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:45 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:43:45.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:45 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:45.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:45 smithi172 ceph-mon[32785]: pgmap v72: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:43:45.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:45 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:43:45.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:45 smithi165 ceph-mon[32648]: purged_snaps scrub starts 2024-04-26T01:43:45.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:45 smithi165 ceph-mon[32648]: purged_snaps scrub ok 2024-04-26T01:43:45.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:45 smithi165 ceph-mon[32648]: Detected new or changed devices on smithi165 2024-04-26T01:43:45.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:45 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:45.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:45 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:45.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:45 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:43:45.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:45 smithi165 ceph-mon[32648]: Adjusting osd_memory_target on smithi165 to 17088M 2024-04-26T01:43:45.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:45 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:45.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:45 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:43:45.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:45 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:43:45.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:45 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:45.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:45 smithi165 ceph-mon[32648]: pgmap v72: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:43:45.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:45 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:43:45.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:45 smithi151 ceph-mon[28057]: purged_snaps scrub starts 2024-04-26T01:43:45.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:45 smithi151 ceph-mon[28057]: purged_snaps scrub ok 2024-04-26T01:43:45.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:45 smithi151 ceph-mon[28057]: Detected new or changed devices on smithi165 2024-04-26T01:43:45.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:45 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:45.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:45 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:45.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:45 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:43:45.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:45 smithi151 ceph-mon[28057]: Adjusting osd_memory_target on smithi165 to 17088M 2024-04-26T01:43:45.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:45 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:45.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:45 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:43:45.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:45 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:43:45.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:45 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:43:45.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:45 smithi151 ceph-mon[28057]: pgmap v72: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:43:45.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:45 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:43:46.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:43:46.656+0000 7fc40bf89700 1 -- 172.21.15.172:0/2940718630 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4041019b0 msgr2=0x7fc404101dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:43:46.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:43:46.656+0000 7fc40bf89700 1 --2- 172.21.15.172:0/2940718630 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4041019b0 0x7fc404101dd0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc400009990 tx=0x7fc400009ca0 comp rx=0 tx=0).stop 2024-04-26T01:43:46.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:43:46.657+0000 7fc40bf89700 1 -- 172.21.15.172:0/2940718630 shutdown_connections 2024-04-26T01:43:46.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:43:46.657+0000 7fc40bf89700 1 --2- 172.21.15.172:0/2940718630 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc404103550 0x7fc404109e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:43:46.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:43:46.657+0000 7fc40bf89700 1 --2- 172.21.15.172:0/2940718630 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4041019b0 0x7fc404101dd0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:43:46.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:43:46.657+0000 7fc40bf89700 1 --2- 172.21.15.172:0/2940718630 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc404102bb0 0x7fc404103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:43:46.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:43:46.657+0000 7fc40bf89700 1 -- 172.21.15.172:0/2940718630 >> 172.21.15.172:0/2940718630 conn(0x7fc4040fcf70 msgr2=0x7fc4040ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:43:46.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:43:46.657+0000 7fc40bf89700 1 -- 172.21.15.172:0/2940718630 shutdown_connections 2024-04-26T01:43:46.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:43:46.657+0000 7fc40bf89700 1 -- 172.21.15.172:0/2940718630 wait complete. 2024-04-26T01:43:46.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:43:46.657+0000 7fc40bf89700 1 Processor -- start 2024-04-26T01:43:46.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:43:46.657+0000 7fc40bf89700 1 -- start start 2024-04-26T01:43:46.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:43:46.657+0000 7fc40bf89700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4041019b0 0x7fc404105940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:43:46.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:43:46.657+0000 7fc40bf89700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc404102bb0 0x7fc404103f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:43:46.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:43:46.658+0000 7fc40bf89700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc404103550 0x7fc4041044d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:43:46.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:43:46.658+0000 7fc40bf89700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc40410d4a0 con 0x7fc404103550 2024-04-26T01:43:46.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:43:46.658+0000 7fc40bf89700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc40410d320 con 0x7fc404102bb0 2024-04-26T01:43:46.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:43:46.658+0000 7fc40bf89700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc40410d620 con 0x7fc4041019b0 2024-04-26T01:43:46.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:43:46.658+0000 7fc409d25700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4041019b0 0x7fc404105940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:43:46.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:43:46.658+0000 7fc409d25700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4041019b0 0x7fc404105940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41058/0 (socket says 172.21.15.172:41058) 2024-04-26T01:43:46.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:43:46.658+0000 7fc409d25700 1 -- 172.21.15.172:0/3375859060 learned_addr learned my addr 172.21.15.172:0/3375859060 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-26T01:43:46.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:43:46.658+0000 7fc409524700 1 --2- 172.21.15.172:0/3375859060 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc404102bb0 0x7fc404103f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:43:46.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:43:46.658+0000 7fc409d25700 1 -- 172.21.15.172:0/3375859060 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc404102bb0 msgr2=0x7fc404103f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:43:46.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:43:46.658+0000 7fc409d25700 1 --2- 172.21.15.172:0/3375859060 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc404102bb0 0x7fc404103f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:43:46.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:43:46.658+0000 7fc409d25700 1 -- 172.21.15.172:0/3375859060 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc404103550 msgr2=0x7fc4041044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:43:46.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:43:46.658+0000 7fc409d25700 1 --2- 172.21.15.172:0/3375859060 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc404103550 0x7fc4041044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:43:46.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:43:46.658+0000 7fc409d25700 1 -- 172.21.15.172:0/3375859060 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc404104d80 con 0x7fc4041019b0 2024-04-26T01:43:46.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:43:46.658+0000 7fc409524700 1 --2- 172.21.15.172:0/3375859060 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc404102bb0 0x7fc404103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:43:46.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:43:46.658+0000 7fc409d25700 1 --2- 172.21.15.172:0/3375859060 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4041019b0 0x7fc404105940 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc400006010 tx=0x7fc4000036f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:43:46.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:43:46.658+0000 7fc3faffd700 1 -- 172.21.15.172:0/3375859060 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc400027070 con 0x7fc4041019b0 2024-04-26T01:43:46.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:43:46.658+0000 7fc3faffd700 1 -- 172.21.15.172:0/3375859060 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc400022470 con 0x7fc4041019b0 2024-04-26T01:43:46.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:43:46.658+0000 7fc40bf89700 1 -- 172.21.15.172:0/3375859060 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4041a2d60 con 0x7fc4041019b0 2024-04-26T01:43:46.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:43:46.658+0000 7fc3faffd700 1 -- 172.21.15.172:0/3375859060 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc400004230 con 0x7fc4041019b0 2024-04-26T01:43:46.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:43:46.658+0000 7fc40bf89700 1 -- 172.21.15.172:0/3375859060 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4041a31b0 con 0x7fc4041019b0 2024-04-26T01:43:46.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:43:46.659+0000 7fc40bf89700 1 -- 172.21.15.172:0/3375859060 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc404069760 con 0x7fc4041019b0 2024-04-26T01:43:46.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:43:46.660+0000 7fc3faffd700 1 -- 172.21.15.172:0/3375859060 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 15) ==== 138591+0+0 (secure 0 0 0) 0x7fc40002a070 con 0x7fc4041019b0 2024-04-26T01:43:46.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:43:46.661+0000 7fc3faffd700 1 --2- 172.21.15.172:0/3375859060 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fc3f00a3bb0 0x7fc3f00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:43:46.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:43:46.661+0000 7fc3faffd700 1 -- 172.21.15.172:0/3375859060 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7fc4000d02b0 con 0x7fc4041019b0 2024-04-26T01:43:46.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:43:46.662+0000 7fc409524700 1 --2- 172.21.15.172:0/3375859060 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fc3f00a3bb0 0x7fc3f00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:43:46.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:43:46.662+0000 7fc409524700 1 --2- 172.21.15.172:0/3375859060 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fc3f00a3bb0 0x7fc3f00a6070 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4005fd0 tx=0x7fc3f4005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:43:46.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:43:46.665+0000 7fc3faffd700 1 -- 172.21.15.172:0/3375859060 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc404069760 con 0x7fc4041019b0 2024-04-26T01:43:46.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:46 smithi172 ceph-mon[32785]: osd.1 [v2:172.21.15.165:6800/1356168171,v1:172.21.15.165:6801/1356168171] boot 2024-04-26T01:43:46.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:46 smithi172 ceph-mon[32785]: osdmap e13: 2 total, 2 up, 2 in 2024-04-26T01:43:46.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:46 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:43:46.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:46 smithi165 ceph-mon[32648]: osd.1 [v2:172.21.15.165:6800/1356168171,v1:172.21.15.165:6801/1356168171] boot 2024-04-26T01:43:46.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:46 smithi165 ceph-mon[32648]: osdmap e13: 2 total, 2 up, 2 in 2024-04-26T01:43:46.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:46 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:43:46.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:43:46.872+0000 7fc40bf89700 1 -- 172.21.15.172:0/3375859060 --> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fc4040ff180 con 0x7fc3f00a3bb0 2024-04-26T01:43:46.870 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:46 smithi151 ceph-mon[28057]: osd.1 [v2:172.21.15.165:6800/1356168171,v1:172.21.15.165:6801/1356168171] boot 2024-04-26T01:43:46.870 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:46 smithi151 ceph-mon[28057]: osdmap e13: 2 total, 2 up, 2 in 2024-04-26T01:43:46.870 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:46 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:43:47.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:47 smithi172 ceph-mon[32785]: osdmap e14: 2 total, 2 up, 2 in 2024-04-26T01:43:47.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:47 smithi172 ceph-mon[32785]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:43:47.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:47 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:43:47.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:47 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:43:47.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:47 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:43:47.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:47 smithi172 ceph-mon[32785]: pgmap v75: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:43:47.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:47 smithi165 ceph-mon[32648]: osdmap e14: 2 total, 2 up, 2 in 2024-04-26T01:43:47.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:47 smithi165 ceph-mon[32648]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:43:47.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:47 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:43:47.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:47 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:43:47.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:47 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:43:47.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:47 smithi165 ceph-mon[32648]: pgmap v75: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:43:47.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:47 smithi151 ceph-mon[28057]: osdmap e14: 2 total, 2 up, 2 in 2024-04-26T01:43:47.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:47 smithi151 ceph-mon[28057]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:43:47.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:47 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:43:47.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:47 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:43:47.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:47 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:43:47.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:47 smithi151 ceph-mon[28057]: pgmap v75: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:43:49.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:49 smithi151 ceph-mon[28057]: pgmap v76: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:43:49.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:49 smithi172 ceph-mon[32785]: pgmap v76: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:43:49.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:49 smithi165 ceph-mon[32648]: pgmap v76: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:43:50.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:50 smithi151 ceph-mon[28057]: from='client.? 172.21.15.172:0/1766239310' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0631df59-1ecc-48b4-b00a-48b0118dfc24"}]: dispatch 2024-04-26T01:43:50.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:50 smithi151 ceph-mon[28057]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0631df59-1ecc-48b4-b00a-48b0118dfc24"}]: dispatch 2024-04-26T01:43:50.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:50 smithi151 ceph-mon[28057]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0631df59-1ecc-48b4-b00a-48b0118dfc24"}]': finished 2024-04-26T01:43:50.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:50 smithi151 ceph-mon[28057]: osdmap e15: 3 total, 2 up, 3 in 2024-04-26T01:43:50.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:50 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T01:43:50.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:50 smithi151 ceph-mon[28057]: from='client.? 172.21.15.172:0/2544555131' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:43:50.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:50 smithi172 ceph-mon[32785]: from='client.? 172.21.15.172:0/1766239310' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0631df59-1ecc-48b4-b00a-48b0118dfc24"}]: dispatch 2024-04-26T01:43:50.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:50 smithi172 ceph-mon[32785]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0631df59-1ecc-48b4-b00a-48b0118dfc24"}]: dispatch 2024-04-26T01:43:50.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:50 smithi172 ceph-mon[32785]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0631df59-1ecc-48b4-b00a-48b0118dfc24"}]': finished 2024-04-26T01:43:50.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:50 smithi172 ceph-mon[32785]: osdmap e15: 3 total, 2 up, 3 in 2024-04-26T01:43:50.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:50 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T01:43:50.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:50 smithi172 ceph-mon[32785]: from='client.? 172.21.15.172:0/2544555131' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:43:50.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:50 smithi165 ceph-mon[32648]: from='client.? 172.21.15.172:0/1766239310' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0631df59-1ecc-48b4-b00a-48b0118dfc24"}]: dispatch 2024-04-26T01:43:50.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:50 smithi165 ceph-mon[32648]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0631df59-1ecc-48b4-b00a-48b0118dfc24"}]: dispatch 2024-04-26T01:43:50.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:50 smithi165 ceph-mon[32648]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0631df59-1ecc-48b4-b00a-48b0118dfc24"}]': finished 2024-04-26T01:43:50.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:50 smithi165 ceph-mon[32648]: osdmap e15: 3 total, 2 up, 3 in 2024-04-26T01:43:50.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:50 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T01:43:50.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:50 smithi165 ceph-mon[32648]: from='client.? 172.21.15.172:0/2544555131' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:43:51.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:51 smithi151 ceph-mon[28057]: pgmap v78: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:43:51.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:51 smithi172 ceph-mon[32785]: pgmap v78: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:43:51.844 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:51 smithi165 ceph-mon[32648]: pgmap v78: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:43:53.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:53 smithi151 ceph-mon[28057]: pgmap v79: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:43:53.746 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:53 smithi172 ceph-mon[32785]: pgmap v79: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:43:53.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:53 smithi165 ceph-mon[32648]: pgmap v79: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:43:55.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:55 smithi151 ceph-mon[28057]: pgmap v80: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:43:55.627 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:55 smithi172 ceph-mon[32785]: pgmap v80: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:43:55.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:55 smithi165 ceph-mon[32648]: pgmap v80: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:43:57.035 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:56 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-26T01:43:57.035 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:56 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:43:57.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:56 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-26T01:43:57.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:56 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:43:57.120 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:56 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-26T01:43:57.120 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:56 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:43:57.972 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:57 smithi172 ceph-mon[32785]: Deploying daemon osd.2 on smithi172 2024-04-26T01:43:57.972 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:57 smithi172 ceph-mon[32785]: pgmap v81: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:43:58.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:57 smithi165 ceph-mon[32648]: Deploying daemon osd.2 on smithi172 2024-04-26T01:43:58.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:57 smithi165 ceph-mon[32648]: pgmap v81: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:43:58.120 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:57 smithi151 ceph-mon[28057]: Deploying daemon osd.2 on smithi172 2024-04-26T01:43:58.120 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:57 smithi151 ceph-mon[28057]: pgmap v81: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:43:59.528 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:43:59 smithi172 ceph-mon[32785]: pgmap v82: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:43:59.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:43:59 smithi165 ceph-mon[32648]: pgmap v82: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:43:59.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:43:59 smithi151 ceph-mon[28057]: pgmap v82: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:44:01.570 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:01 smithi172 ceph-mon[32785]: pgmap v83: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:44:01.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:01 smithi151 ceph-mon[28057]: pgmap v83: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:44:01.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:01 smithi165 ceph-mon[32648]: pgmap v83: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:44:03.513 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:03 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:03.513 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:03 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:44:03.513 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:03 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:03.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:03 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:03.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:03 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:44:03.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:03 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:03.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:03 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:03.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:03 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:44:03.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:03 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:04.430 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:04 smithi172 ceph-mon[32785]: pgmap v84: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:44:04.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:04 smithi165 ceph-mon[32648]: pgmap v84: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:44:04.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:04 smithi151 ceph-mon[28057]: pgmap v84: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:44:05.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:44:05.203+0000 7fc3faffd700 1 -- 172.21.15.172:0/3375859060 <== mgr.14152 v2:172.21.15.151:6800/2251256505 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc4040ff180 con 0x7fc3f00a3bb0 2024-04-26T01:44:05.201 INFO:teuthology.orchestra.run.smithi172.stdout:Created osd(s) 2 on host 'smithi172' 2024-04-26T01:44:05.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:44:05.205+0000 7fc40bf89700 1 -- 172.21.15.172:0/3375859060 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fc3f00a3bb0 msgr2=0x7fc3f00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:44:05.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:44:05.205+0000 7fc40bf89700 1 --2- 172.21.15.172:0/3375859060 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fc3f00a3bb0 0x7fc3f00a6070 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4005fd0 tx=0x7fc3f4005e30 comp rx=0 tx=0).stop 2024-04-26T01:44:05.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:44:05.205+0000 7fc40bf89700 1 -- 172.21.15.172:0/3375859060 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4041019b0 msgr2=0x7fc404105940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:44:05.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:44:05.205+0000 7fc40bf89700 1 --2- 172.21.15.172:0/3375859060 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4041019b0 0x7fc404105940 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc400006010 tx=0x7fc4000036f0 comp rx=0 tx=0).stop 2024-04-26T01:44:05.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:44:05.205+0000 7fc40bf89700 1 -- 172.21.15.172:0/3375859060 shutdown_connections 2024-04-26T01:44:05.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:44:05.205+0000 7fc40bf89700 1 --2- 172.21.15.172:0/3375859060 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fc3f00a3bb0 0x7fc3f00a6070 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:05.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:44:05.205+0000 7fc40bf89700 1 --2- 172.21.15.172:0/3375859060 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc404103550 0x7fc4041044d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:05.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:44:05.205+0000 7fc40bf89700 1 --2- 172.21.15.172:0/3375859060 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4041019b0 0x7fc404105940 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:05.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:44:05.205+0000 7fc40bf89700 1 --2- 172.21.15.172:0/3375859060 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc404102bb0 0x7fc404103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:05.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:44:05.205+0000 7fc40bf89700 1 -- 172.21.15.172:0/3375859060 >> 172.21.15.172:0/3375859060 conn(0x7fc4040fcf70 msgr2=0x7fc4040fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:44:05.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:44:05.205+0000 7fc40bf89700 1 -- 172.21.15.172:0/3375859060 shutdown_connections 2024-04-26T01:44:05.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:44:05.205+0000 7fc40bf89700 1 -- 172.21.15.172:0/3375859060 wait complete. 2024-04-26T01:44:06.158 DEBUG:teuthology.orchestra.run.smithi172:osd.2> sudo journalctl -f -n 0 -u ceph-c0e49e64-036d-11ef-bc93-c7b262605968@osd.2.service 2024-04-26T01:44:06.161 INFO:tasks.cephadm:Deploying osd.3 on smithi151 with /dev/vg_nvme/lv_3... 2024-04-26T01:44:06.161 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-26T01:44:06.408 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:06 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:06.408 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:06 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:06.408 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:06 smithi172 ceph-mon[32785]: pgmap v85: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:44:06.408 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:06 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:06.408 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:06 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:06.408 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:06 smithi172 ceph-mon[32785]: from='osd.2 [v2:172.21.15.172:6800/237502635,v1:172.21.15.172:6801/237502635]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T01:44:06.408 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:06 smithi172 ceph-mon[32785]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T01:44:06.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:06 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:06.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:06 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:06.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:06 smithi165 ceph-mon[32648]: pgmap v85: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:44:06.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:06 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:06.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:06 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:06.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:06 smithi165 ceph-mon[32648]: from='osd.2 [v2:172.21.15.172:6800/237502635,v1:172.21.15.172:6801/237502635]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T01:44:06.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:06 smithi165 ceph-mon[32648]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T01:44:06.614 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:44:06.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:06 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:06.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:06 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:06.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:06 smithi151 ceph-mon[28057]: pgmap v85: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:44:06.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:06 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:06.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:06 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:06.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:06 smithi151 ceph-mon[28057]: from='osd.2 [v2:172.21.15.172:6800/237502635,v1:172.21.15.172:6801/237502635]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T01:44:06.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:06 smithi151 ceph-mon[28057]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T01:44:07.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:07 smithi172 ceph-mon[32785]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-26T01:44:07.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:07 smithi172 ceph-mon[32785]: from='osd.2 [v2:172.21.15.172:6800/237502635,v1:172.21.15.172:6801/237502635]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-04-26T01:44:07.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:07 smithi172 ceph-mon[32785]: osdmap e16: 3 total, 2 up, 3 in 2024-04-26T01:44:07.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:07 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T01:44:07.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:07 smithi172 ceph-mon[32785]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-04-26T01:44:07.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:07 smithi172 ceph-mon[32785]: pgmap v87: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:44:07.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:07 smithi151 ceph-mon[28057]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-26T01:44:07.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:07 smithi151 ceph-mon[28057]: from='osd.2 [v2:172.21.15.172:6800/237502635,v1:172.21.15.172:6801/237502635]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-04-26T01:44:07.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:07 smithi151 ceph-mon[28057]: osdmap e16: 3 total, 2 up, 3 in 2024-04-26T01:44:07.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:07 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T01:44:07.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:07 smithi151 ceph-mon[28057]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-04-26T01:44:07.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:07 smithi151 ceph-mon[28057]: pgmap v87: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:44:08.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:07 smithi165 ceph-mon[32648]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-26T01:44:08.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:07 smithi165 ceph-mon[32648]: from='osd.2 [v2:172.21.15.172:6800/237502635,v1:172.21.15.172:6801/237502635]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-04-26T01:44:08.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:07 smithi165 ceph-mon[32648]: osdmap e16: 3 total, 2 up, 3 in 2024-04-26T01:44:08.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:07 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T01:44:08.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:07 smithi165 ceph-mon[32648]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-04-26T01:44:08.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:07 smithi165 ceph-mon[32648]: pgmap v87: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:44:08.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:08 smithi172 ceph-mon[32785]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-04-26T01:44:08.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:08 smithi172 ceph-mon[32785]: osdmap e17: 3 total, 2 up, 3 in 2024-04-26T01:44:08.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:08 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T01:44:08.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:08 smithi172 ceph-mon[32785]: from='osd.2 ' entity='osd.2' 2024-04-26T01:44:08.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:08 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T01:44:08.786 INFO:journalctl@ceph.osd.2.smithi172.stdout:Apr 26 01:44:08 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-2[37925]: 2024-04-26T01:44:08.530+0000 7f95c7991700 -1 osd.2 0 waiting for initial osdmap 2024-04-26T01:44:08.786 INFO:journalctl@ceph.osd.2.smithi172.stdout:Apr 26 01:44:08 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-2[37925]: 2024-04-26T01:44:08.629+0000 7f95bf77c700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T01:44:08.870 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:08 smithi151 ceph-mon[28057]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-04-26T01:44:08.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:08 smithi151 ceph-mon[28057]: osdmap e17: 3 total, 2 up, 3 in 2024-04-26T01:44:08.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:08 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T01:44:08.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:08 smithi151 ceph-mon[28057]: from='osd.2 ' entity='osd.2' 2024-04-26T01:44:08.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:08 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T01:44:09.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:08 smithi165 ceph-mon[32648]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-04-26T01:44:09.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:08 smithi165 ceph-mon[32648]: osdmap e17: 3 total, 2 up, 3 in 2024-04-26T01:44:09.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:08 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T01:44:09.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:08 smithi165 ceph-mon[32648]: from='osd.2 ' entity='osd.2' 2024-04-26T01:44:09.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:08 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T01:44:09.385 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-26T01:44:09.385 INFO:teuthology.orchestra.run.smithi151.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-26T01:44:09.385 INFO:teuthology.orchestra.run.smithi151.stderr: stderr: 10+0 records in 2024-04-26T01:44:09.385 INFO:teuthology.orchestra.run.smithi151.stderr:10+0 records out 2024-04-26T01:44:09.385 INFO:teuthology.orchestra.run.smithi151.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0376564 s, 278 MB/s 2024-04-26T01:44:09.385 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping successful for: 2024-04-26T01:44:09.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:09 smithi151 ceph-mon[28057]: purged_snaps scrub starts 2024-04-26T01:44:09.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:09 smithi151 ceph-mon[28057]: purged_snaps scrub ok 2024-04-26T01:44:09.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:09 smithi151 ceph-mon[28057]: Detected new or changed devices on smithi172 2024-04-26T01:44:09.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:09 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:09.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:09 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:09.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:09 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:44:09.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:09 smithi151 ceph-mon[28057]: Adjusting osd_memory_target on smithi172 to 17088M 2024-04-26T01:44:09.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:09 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:09.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:09 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:44:09.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:09 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:44:09.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:09 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:09.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:09 smithi151 ceph-mon[28057]: pgmap v89: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:44:09.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:09 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:44:09.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:09 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:44:09.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:09 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:44:09.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:09 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:09.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:09 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T01:44:10.035 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:09 smithi172 ceph-mon[32785]: purged_snaps scrub starts 2024-04-26T01:44:10.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:09 smithi172 ceph-mon[32785]: purged_snaps scrub ok 2024-04-26T01:44:10.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:09 smithi172 ceph-mon[32785]: Detected new or changed devices on smithi172 2024-04-26T01:44:10.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:09 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:10.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:09 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:10.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:09 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:44:10.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:09 smithi172 ceph-mon[32785]: Adjusting osd_memory_target on smithi172 to 17088M 2024-04-26T01:44:10.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:09 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:10.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:09 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:44:10.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:09 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:44:10.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:09 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:10.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:09 smithi172 ceph-mon[32785]: pgmap v89: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:44:10.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:09 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:44:10.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:09 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:44:10.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:09 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:44:10.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:09 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:10.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:09 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T01:44:10.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:09 smithi165 ceph-mon[32648]: purged_snaps scrub starts 2024-04-26T01:44:10.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:09 smithi165 ceph-mon[32648]: purged_snaps scrub ok 2024-04-26T01:44:10.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:09 smithi165 ceph-mon[32648]: Detected new or changed devices on smithi172 2024-04-26T01:44:10.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:09 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:10.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:09 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:10.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:09 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:44:10.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:09 smithi165 ceph-mon[32648]: Adjusting osd_memory_target on smithi172 to 17088M 2024-04-26T01:44:10.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:09 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:10.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:09 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:44:10.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:09 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:44:10.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:09 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:10.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:09 smithi165 ceph-mon[32648]: pgmap v89: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:44:10.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:09 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:44:10.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:09 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:44:10.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:09 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:44:10.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:09 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:10.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:09 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T01:44:10.187 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi151:vg_nvme/lv_3 2024-04-26T01:44:10.613 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:44:10.929 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:10 smithi151 ceph-mon[28057]: osd.2 [v2:172.21.15.172:6800/237502635,v1:172.21.15.172:6801/237502635] boot 2024-04-26T01:44:10.929 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:10 smithi151 ceph-mon[28057]: osdmap e18: 3 total, 3 up, 3 in 2024-04-26T01:44:10.929 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:10 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T01:44:11.035 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:10 smithi172 ceph-mon[32785]: osd.2 [v2:172.21.15.172:6800/237502635,v1:172.21.15.172:6801/237502635] boot 2024-04-26T01:44:11.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:10 smithi172 ceph-mon[32785]: osdmap e18: 3 total, 3 up, 3 in 2024-04-26T01:44:11.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:10 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T01:44:11.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:10 smithi165 ceph-mon[32648]: osd.2 [v2:172.21.15.172:6800/237502635,v1:172.21.15.172:6801/237502635] boot 2024-04-26T01:44:11.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:10 smithi165 ceph-mon[32648]: osdmap e18: 3 total, 3 up, 3 in 2024-04-26T01:44:11.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:10 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T01:44:11.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:11 smithi151 ceph-mon[28057]: osdmap e19: 3 total, 3 up, 3 in 2024-04-26T01:44:11.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:11 smithi151 ceph-mon[28057]: pgmap v92: 0 pgs: ; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:44:11.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:11 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-26T01:44:12.035 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:11 smithi172 ceph-mon[32785]: osdmap e19: 3 total, 3 up, 3 in 2024-04-26T01:44:12.035 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:11 smithi172 ceph-mon[32785]: pgmap v92: 0 pgs: ; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:44:12.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:11 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-26T01:44:12.094 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:11 smithi165 ceph-mon[32648]: osdmap e19: 3 total, 3 up, 3 in 2024-04-26T01:44:12.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:11 smithi165 ceph-mon[32648]: pgmap v92: 0 pgs: ; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:44:12.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:11 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-26T01:44:12.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:44:12.757+0000 7f5afac00700 1 -- 172.21.15.151:0/2250333456 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5af4076a10 msgr2=0x7f5af4074e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:44:12.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:44:12.757+0000 7f5afac00700 1 --2- 172.21.15.151:0/2250333456 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5af4076a10 0x7f5af4074e70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4009990 tx=0x7f5ae4009ca0 comp rx=0 tx=0).stop 2024-04-26T01:44:12.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:44:12.757+0000 7f5afac00700 1 -- 172.21.15.151:0/2250333456 shutdown_connections 2024-04-26T01:44:12.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:44:12.757+0000 7f5afac00700 1 --2- 172.21.15.151:0/2250333456 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5af410a980 0x7f5af410ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:12.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:44:12.757+0000 7f5afac00700 1 --2- 172.21.15.151:0/2250333456 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5af4076a10 0x7f5af4074e70 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:12.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:44:12.757+0000 7f5afac00700 1 --2- 172.21.15.151:0/2250333456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5af4075440 0x7f5af40758c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:12.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:44:12.757+0000 7f5afac00700 1 -- 172.21.15.151:0/2250333456 >> 172.21.15.151:0/2250333456 conn(0x7f5af40fe040 msgr2=0x7f5af4100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:44:12.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:44:12.757+0000 7f5afac00700 1 -- 172.21.15.151:0/2250333456 shutdown_connections 2024-04-26T01:44:12.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:44:12.757+0000 7f5afac00700 1 -- 172.21.15.151:0/2250333456 wait complete. 2024-04-26T01:44:12.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:44:12.759+0000 7f5afac00700 1 Processor -- start 2024-04-26T01:44:12.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:44:12.759+0000 7f5afac00700 1 -- start start 2024-04-26T01:44:12.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:44:12.759+0000 7f5afac00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5af4075440 0x7f5af419ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:44:12.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:44:12.759+0000 7f5afac00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5af4076a10 0x7f5af419b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:44:12.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:44:12.759+0000 7f5afac00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5af410a980 0x7f5af41a23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:44:12.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:44:12.759+0000 7f5afac00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5af4110740 con 0x7f5af4076a10 2024-04-26T01:44:12.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:44:12.759+0000 7f5afac00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5af41105c0 con 0x7f5af4075440 2024-04-26T01:44:12.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:44:12.759+0000 7f5afac00700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5af41108c0 con 0x7f5af410a980 2024-04-26T01:44:12.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:44:12.760+0000 7f5af3400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5af4076a10 0x7f5af419b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:44:12.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:44:12.760+0000 7f5af3e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5af4075440 0x7f5af419ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:44:12.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:44:12.760+0000 7f5af3400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5af4076a10 0x7f5af419b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:48902/0 (socket says 172.21.15.151:48902) 2024-04-26T01:44:12.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:44:12.760+0000 7f5af3e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5af4075440 0x7f5af419ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:44400/0 (socket says 172.21.15.151:44400) 2024-04-26T01:44:12.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:44:12.760+0000 7f5af3400700 1 -- 172.21.15.151:0/1608163849 learned_addr learned my addr 172.21.15.151:0/1608163849 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:44:12.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:44:12.760+0000 7f5af8c00700 1 --2- 172.21.15.151:0/1608163849 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5af410a980 0x7f5af41a23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:44:12.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:44:12.760+0000 7f5af3e00700 1 -- 172.21.15.151:0/1608163849 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5af410a980 msgr2=0x7f5af41a23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:44:12.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:44:12.760+0000 7f5af3e00700 1 --2- 172.21.15.151:0/1608163849 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5af410a980 0x7f5af41a23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:12.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:44:12.760+0000 7f5af3e00700 1 -- 172.21.15.151:0/1608163849 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5af4076a10 msgr2=0x7f5af419b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:44:12.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:44:12.760+0000 7f5af3e00700 1 --2- 172.21.15.151:0/1608163849 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5af4076a10 0x7f5af419b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:12.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:44:12.760+0000 7f5af3e00700 1 -- 172.21.15.151:0/1608163849 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5af41a2a60 con 0x7f5af4075440 2024-04-26T01:44:12.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:44:12.760+0000 7f5af3e00700 1 --2- 172.21.15.151:0/1608163849 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5af4075440 0x7f5af419ae00 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5ae40098e0 tx=0x7f5ae40047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:44:12.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:44:12.761+0000 7f5af0c00700 1 -- 172.21.15.151:0/1608163849 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae4026070 con 0x7f5af4075440 2024-04-26T01:44:12.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:44:12.761+0000 7f5afac00700 1 -- 172.21.15.151:0/1608163849 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5af41a2cf0 con 0x7f5af4075440 2024-04-26T01:44:12.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:44:12.761+0000 7f5af0c00700 1 -- 172.21.15.151:0/1608163849 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ae4021410 con 0x7f5af4075440 2024-04-26T01:44:12.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:44:12.761+0000 7f5af0c00700 1 -- 172.21.15.151:0/1608163849 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae4017650 con 0x7f5af4075440 2024-04-26T01:44:12.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:44:12.761+0000 7f5afac00700 1 -- 172.21.15.151:0/1608163849 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5af41a31a0 con 0x7f5af4075440 2024-04-26T01:44:12.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:44:12.762+0000 7f5afac00700 1 -- 172.21.15.151:0/1608163849 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5af4069730 con 0x7f5af4075440 2024-04-26T01:44:12.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:44:12.764+0000 7f5af0c00700 1 -- 172.21.15.151:0/1608163849 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 15) ==== 138591+0+0 (secure 0 0 0) 0x7f5ae4029070 con 0x7f5af4075440 2024-04-26T01:44:12.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:44:12.765+0000 7f5af0c00700 1 --2- 172.21.15.151:0/1608163849 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f5ad80a3bb0 0x7f5ad80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:44:12.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:44:12.766+0000 7f5af0c00700 1 -- 172.21.15.151:0/1608163849 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 3237+0+0 (secure 0 0 0) 0x7f5ae40d0360 con 0x7f5af4075440 2024-04-26T01:44:12.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:44:12.766+0000 7f5af3400700 1 --2- 172.21.15.151:0/1608163849 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f5ad80a3bb0 0x7f5ad80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:44:12.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:44:12.766+0000 7f5af3400700 1 --2- 172.21.15.151:0/1608163849 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f5ad80a3bb0 0x7f5ad80a6070 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0009d20 tx=0x7f5ae0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:44:12.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:44:12.768+0000 7f5af0c00700 1 -- 172.21.15.151:0/1608163849 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5ae4030070 con 0x7f5af4075440 2024-04-26T01:44:12.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:12 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-26T01:44:12.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:12 smithi151 ceph-mon[28057]: osdmap e20: 3 total, 3 up, 3 in 2024-04-26T01:44:12.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:12 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T01:44:13.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:44:13.008+0000 7f5afac00700 1 -- 172.21.15.151:0/1608163849 --> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f5af4062f40 con 0x7f5ad80a3bb0 2024-04-26T01:44:13.035 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:12 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-26T01:44:13.035 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:12 smithi172 ceph-mon[32785]: osdmap e20: 3 total, 3 up, 3 in 2024-04-26T01:44:13.035 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:12 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T01:44:13.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:12 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-26T01:44:13.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:12 smithi165 ceph-mon[32648]: osdmap e20: 3 total, 3 up, 3 in 2024-04-26T01:44:13.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:12 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T01:44:13.745 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:13 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-26T01:44:13.745 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:13 smithi151 ceph-mon[28057]: osdmap e21: 3 total, 3 up, 3 in 2024-04-26T01:44:13.745 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:13 smithi151 ceph-mon[28057]: from='client.24185 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:44:13.745 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:13 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:44:13.745 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:13 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:44:13.745 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:13 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:44:13.745 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:13 smithi151 ceph-mon[28057]: pgmap v95: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:44:13.745 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:13 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:13.745 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:13 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:13.746 INFO:journalctl@ceph.osd.0.smithi151.stdout:Apr 26 01:44:13 smithi151 sudo[46998]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T01:44:13.746 INFO:journalctl@ceph.osd.0.smithi151.stdout:Apr 26 01:44:13 smithi151 sudo[46998]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T01:44:13.746 INFO:journalctl@ceph.osd.0.smithi151.stdout:Apr 26 01:44:13 smithi151 sudo[46998]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T01:44:13.746 INFO:journalctl@ceph.osd.0.smithi151.stdout:Apr 26 01:44:13 smithi151 sudo[46998]: pam_unix(sudo:session): session closed for user root 2024-04-26T01:44:13.746 INFO:journalctl@ceph.osd.0.smithi151.stdout:Apr 26 01:44:13 smithi151 sudo[47001]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T01:44:13.746 INFO:journalctl@ceph.osd.0.smithi151.stdout:Apr 26 01:44:13 smithi151 sudo[47001]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T01:44:13.746 INFO:journalctl@ceph.osd.0.smithi151.stdout:Apr 26 01:44:13 smithi151 sudo[47001]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T01:44:13.746 INFO:journalctl@ceph.osd.0.smithi151.stdout:Apr 26 01:44:13 smithi151 sudo[47001]: pam_unix(sudo:session): session closed for user root 2024-04-26T01:44:13.845 INFO:journalctl@ceph.osd.1.smithi165.stdout:Apr 26 01:44:13 smithi165 sudo[41085]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T01:44:13.845 INFO:journalctl@ceph.osd.1.smithi165.stdout:Apr 26 01:44:13 smithi165 sudo[41085]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T01:44:13.845 INFO:journalctl@ceph.osd.1.smithi165.stdout:Apr 26 01:44:13 smithi165 sudo[41085]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T01:44:13.845 INFO:journalctl@ceph.osd.1.smithi165.stdout:Apr 26 01:44:13 smithi165 sudo[41085]: pam_unix(sudo:session): session closed for user root 2024-04-26T01:44:13.845 INFO:journalctl@ceph.osd.1.smithi165.stdout:Apr 26 01:44:13 smithi165 sudo[41088]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T01:44:13.845 INFO:journalctl@ceph.osd.1.smithi165.stdout:Apr 26 01:44:13 smithi165 sudo[41088]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T01:44:13.845 INFO:journalctl@ceph.osd.1.smithi165.stdout:Apr 26 01:44:13 smithi165 sudo[41088]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T01:44:13.845 INFO:journalctl@ceph.osd.1.smithi165.stdout:Apr 26 01:44:13 smithi165 sudo[41088]: pam_unix(sudo:session): session closed for user root 2024-04-26T01:44:13.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:13 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-26T01:44:13.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:13 smithi165 ceph-mon[32648]: osdmap e21: 3 total, 3 up, 3 in 2024-04-26T01:44:13.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:13 smithi165 ceph-mon[32648]: from='client.24185 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:44:13.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:13 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:44:13.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:13 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:44:13.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:13 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:44:13.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:13 smithi165 ceph-mon[32648]: pgmap v95: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:44:13.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:13 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:13.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:13 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:14.036 INFO:journalctl@ceph.osd.2.smithi172.stdout:Apr 26 01:44:13 smithi172 sudo[41690]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T01:44:14.036 INFO:journalctl@ceph.osd.2.smithi172.stdout:Apr 26 01:44:13 smithi172 sudo[41690]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T01:44:14.036 INFO:journalctl@ceph.osd.2.smithi172.stdout:Apr 26 01:44:13 smithi172 sudo[41690]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T01:44:14.036 INFO:journalctl@ceph.osd.2.smithi172.stdout:Apr 26 01:44:13 smithi172 sudo[41690]: pam_unix(sudo:session): session closed for user root 2024-04-26T01:44:14.036 INFO:journalctl@ceph.osd.2.smithi172.stdout:Apr 26 01:44:13 smithi172 sudo[41693]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T01:44:14.036 INFO:journalctl@ceph.osd.2.smithi172.stdout:Apr 26 01:44:13 smithi172 sudo[41693]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T01:44:14.036 INFO:journalctl@ceph.osd.2.smithi172.stdout:Apr 26 01:44:13 smithi172 sudo[41693]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T01:44:14.036 INFO:journalctl@ceph.osd.2.smithi172.stdout:Apr 26 01:44:13 smithi172 sudo[41693]: pam_unix(sudo:session): session closed for user root 2024-04-26T01:44:14.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:13 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-26T01:44:14.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:13 smithi172 ceph-mon[32785]: osdmap e21: 3 total, 3 up, 3 in 2024-04-26T01:44:14.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:13 smithi172 ceph-mon[32785]: from='client.24185 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:44:14.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:13 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:44:14.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:13 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:44:14.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:13 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:44:14.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:13 smithi172 ceph-mon[32785]: pgmap v95: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:44:14.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:13 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:14.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:13 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:14.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:13 smithi172 sudo[41696]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T01:44:14.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:13 smithi172 sudo[41696]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T01:44:14.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:13 smithi172 sudo[41696]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T01:44:14.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:13 smithi172 sudo[41696]: pam_unix(sudo:session): session closed for user root 2024-04-26T01:44:14.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:13 smithi172 sudo[41700]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T01:44:14.038 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:13 smithi172 sudo[41700]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T01:44:14.038 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:13 smithi172 sudo[41700]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T01:44:14.038 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:13 smithi172 sudo[41700]: pam_unix(sudo:session): session closed for user root 2024-04-26T01:44:14.120 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:13 smithi151 sudo[47023]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T01:44:14.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:13 smithi151 sudo[47023]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T01:44:14.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:13 smithi151 sudo[47023]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T01:44:14.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:13 smithi151 sudo[47023]: pam_unix(sudo:session): session closed for user root 2024-04-26T01:44:14.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:13 smithi151 sudo[47034]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T01:44:14.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:13 smithi151 sudo[47034]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T01:44:14.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:13 smithi151 sudo[47034]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T01:44:14.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:13 smithi151 sudo[47034]: pam_unix(sudo:session): session closed for user root 2024-04-26T01:44:14.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:13 smithi165 sudo[41091]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T01:44:14.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:13 smithi165 sudo[41091]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T01:44:14.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:13 smithi165 sudo[41091]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T01:44:14.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:13 smithi165 sudo[41091]: pam_unix(sudo:session): session closed for user root 2024-04-26T01:44:14.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:13 smithi165 sudo[41094]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T01:44:14.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:13 smithi165 sudo[41094]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T01:44:14.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:13 smithi165 sudo[41094]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T01:44:14.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:13 smithi165 sudo[41094]: pam_unix(sudo:session): session closed for user root 2024-04-26T01:44:15.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:14 smithi172 ceph-mon[32785]: osdmap e22: 3 total, 3 up, 3 in 2024-04-26T01:44:15.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:14 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:15.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:14 smithi172 ceph-mon[32785]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T01:44:15.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:14 smithi172 ceph-mon[32785]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T01:44:15.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:14 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T01:44:15.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:14 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:44:15.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:14 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:44:15.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:14 smithi172 ceph-mon[32785]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T01:44:15.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:14 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T01:44:15.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:14 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:44:15.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:14 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:44:15.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:14 smithi172 ceph-mon[32785]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T01:44:15.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:14 smithi172 ceph-mon[32785]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T01:44:15.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:14 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T01:44:15.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:14 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:44:15.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:14 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:44:15.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:14 smithi172 ceph-mon[32785]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T01:44:15.041 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:14 smithi151 ceph-mon[28057]: osdmap e22: 3 total, 3 up, 3 in 2024-04-26T01:44:15.041 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:14 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:15.041 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:14 smithi151 ceph-mon[28057]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T01:44:15.041 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:14 smithi151 ceph-mon[28057]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T01:44:15.041 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:14 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T01:44:15.041 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:14 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:44:15.041 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:14 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:44:15.041 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:14 smithi151 ceph-mon[28057]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T01:44:15.041 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:14 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T01:44:15.041 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:14 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:44:15.041 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:14 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:44:15.041 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:14 smithi151 ceph-mon[28057]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T01:44:15.042 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:14 smithi151 ceph-mon[28057]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T01:44:15.042 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:14 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T01:44:15.042 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:14 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:44:15.042 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:14 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:44:15.042 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:14 smithi151 ceph-mon[28057]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T01:44:15.099 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:14 smithi165 ceph-mon[32648]: osdmap e22: 3 total, 3 up, 3 in 2024-04-26T01:44:15.100 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:14 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:15.100 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:14 smithi165 ceph-mon[32648]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T01:44:15.100 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:14 smithi165 ceph-mon[32648]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T01:44:15.100 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:14 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T01:44:15.100 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:14 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:44:15.100 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:14 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:44:15.100 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:14 smithi165 ceph-mon[32648]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T01:44:15.100 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:14 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T01:44:15.100 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:14 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:44:15.100 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:14 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:44:15.100 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:14 smithi165 ceph-mon[32648]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T01:44:15.101 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:14 smithi165 ceph-mon[32648]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T01:44:15.101 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:14 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T01:44:15.101 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:14 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:44:15.101 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:14 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:44:15.101 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:14 smithi165 ceph-mon[32648]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T01:44:15.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:44:15.717+0000 7f5af0c00700 1 -- 172.21.15.151:0/1608163849 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f5ae4009d10 con 0x7f5af4075440 2024-04-26T01:44:15.870 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:15 smithi151 ceph-mon[28057]: pgmap v97: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:44:16.035 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:15 smithi172 ceph-mon[32785]: pgmap v97: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:44:16.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:15 smithi165 ceph-mon[32648]: pgmap v97: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:44:17.035 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:16 smithi172 ceph-mon[32785]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-04-26T01:44:17.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:16 smithi165 ceph-mon[32648]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-04-26T01:44:17.120 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:16 smithi151 ceph-mon[28057]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-04-26T01:44:18.035 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:17 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/167083029' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73076e49-2f8a-4a3d-99eb-7b50aff448aa"}]: dispatch 2024-04-26T01:44:18.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:17 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/167083029' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "73076e49-2f8a-4a3d-99eb-7b50aff448aa"}]': finished 2024-04-26T01:44:18.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:17 smithi172 ceph-mon[32785]: osdmap e23: 4 total, 3 up, 4 in 2024-04-26T01:44:18.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:17 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:44:18.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:17 smithi172 ceph-mon[32785]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:44:18.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:17 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/167083029' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73076e49-2f8a-4a3d-99eb-7b50aff448aa"}]: dispatch 2024-04-26T01:44:18.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:17 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/167083029' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "73076e49-2f8a-4a3d-99eb-7b50aff448aa"}]': finished 2024-04-26T01:44:18.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:17 smithi165 ceph-mon[32648]: osdmap e23: 4 total, 3 up, 4 in 2024-04-26T01:44:18.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:17 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:44:18.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:17 smithi165 ceph-mon[32648]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:44:18.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:17 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/167083029' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73076e49-2f8a-4a3d-99eb-7b50aff448aa"}]: dispatch 2024-04-26T01:44:18.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:17 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/167083029' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "73076e49-2f8a-4a3d-99eb-7b50aff448aa"}]': finished 2024-04-26T01:44:18.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:17 smithi151 ceph-mon[28057]: osdmap e23: 4 total, 3 up, 4 in 2024-04-26T01:44:18.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:17 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:44:18.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:17 smithi151 ceph-mon[28057]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:44:19.035 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:18 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/1831224538' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:44:19.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:18 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/1831224538' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:44:19.120 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:18 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/1831224538' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:44:20.035 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:19 smithi172 ceph-mon[32785]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:44:20.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:19 smithi165 ceph-mon[32648]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:44:20.120 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:19 smithi151 ceph-mon[28057]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:44:21.604 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:21 smithi151 ceph-mon[28057]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:44:21.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:21 smithi172 ceph-mon[32785]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:44:21.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:21 smithi165 ceph-mon[32648]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:44:23.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:23 smithi151 ceph-mon[28057]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:44:23.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:23 smithi172 ceph-mon[32785]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:44:23.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:23 smithi165 ceph-mon[32648]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:44:25.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:25 smithi151 ceph-mon[28057]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:44:25.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:25 smithi172 ceph-mon[32785]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:44:25.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:25 smithi165 ceph-mon[32648]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:44:26.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:26 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-26T01:44:26.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:26 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:44:26.870 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:26 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-26T01:44:26.870 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:26 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:44:27.035 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:26 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-26T01:44:27.035 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:26 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:44:27.773 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:27 smithi151 ceph-mon[28057]: Deploying daemon osd.3 on smithi151 2024-04-26T01:44:27.773 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:27 smithi151 ceph-mon[28057]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:44:27.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:27 smithi165 ceph-mon[32648]: Deploying daemon osd.3 on smithi151 2024-04-26T01:44:27.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:27 smithi165 ceph-mon[32648]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:44:28.035 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:27 smithi172 ceph-mon[32785]: Deploying daemon osd.3 on smithi151 2024-04-26T01:44:28.035 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:27 smithi172 ceph-mon[32785]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:44:29.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:29 smithi151 ceph-mon[28057]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:44:29.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:29 smithi172 ceph-mon[32785]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:44:29.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:29 smithi165 ceph-mon[32648]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:44:31.434 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:31 smithi151 ceph-mon[28057]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:44:31.606 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:31 smithi172 ceph-mon[32785]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:44:31.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:31 smithi165 ceph-mon[32648]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:44:33.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:33 smithi151 ceph-mon[28057]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:44:33.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:33 smithi172 ceph-mon[32785]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:44:33.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:33 smithi165 ceph-mon[32648]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:44:35.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:35 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:35.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:35 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:35.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:35 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:44:35.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:35 smithi151 ceph-mon[28057]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:44:36.035 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:35 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:36.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:35 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:36.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:35 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:44:36.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:35 smithi172 ceph-mon[32785]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:44:36.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:35 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:36.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:35 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:36.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:35 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:44:36.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:35 smithi165 ceph-mon[32648]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:44:37.615 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:37 smithi151 ceph-mon[28057]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:44:37.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:37 smithi172 ceph-mon[32785]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:44:37.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:37 smithi165 ceph-mon[32648]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:44:39.277 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:39 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:39.277 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:39 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:39.277 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:39 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:44:39.277 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:39 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:44:39.277 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:39 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:39.277 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:39 smithi151 ceph-mon[28057]: from='osd.3 [v2:172.21.15.151:6810/3442279476,v1:172.21.15.151:6811/3442279476]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T01:44:39.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:39 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:39.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:39 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:39.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:39 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:44:39.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:39 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:44:39.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:39 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:39.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:39 smithi165 ceph-mon[32648]: from='osd.3 [v2:172.21.15.151:6810/3442279476,v1:172.21.15.151:6811/3442279476]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T01:44:39.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:44:39.411+0000 7f5af0c00700 1 -- 172.21.15.151:0/1608163849 <== mgr.14152 v2:172.21.15.151:6800/2251256505 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5af4062f40 con 0x7f5ad80a3bb0 2024-04-26T01:44:39.409 INFO:teuthology.orchestra.run.smithi151.stdout:Created osd(s) 3 on host 'smithi151' 2024-04-26T01:44:39.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:44:39.415+0000 7f5afac00700 1 -- 172.21.15.151:0/1608163849 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f5ad80a3bb0 msgr2=0x7f5ad80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:44:39.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:44:39.415+0000 7f5afac00700 1 --2- 172.21.15.151:0/1608163849 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f5ad80a3bb0 0x7f5ad80a6070 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0009d20 tx=0x7f5ae0009260 comp rx=0 tx=0).stop 2024-04-26T01:44:39.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:44:39.415+0000 7f5afac00700 1 -- 172.21.15.151:0/1608163849 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5af4075440 msgr2=0x7f5af419ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:44:39.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:44:39.415+0000 7f5afac00700 1 --2- 172.21.15.151:0/1608163849 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5af4075440 0x7f5af419ae00 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5ae40098e0 tx=0x7f5ae40047b0 comp rx=0 tx=0).stop 2024-04-26T01:44:39.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:44:39.415+0000 7f5afac00700 1 -- 172.21.15.151:0/1608163849 shutdown_connections 2024-04-26T01:44:39.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:44:39.415+0000 7f5afac00700 1 --2- 172.21.15.151:0/1608163849 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5af410a980 0x7f5af41a23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:39.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:44:39.415+0000 7f5afac00700 1 --2- 172.21.15.151:0/1608163849 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5af4075440 0x7f5af419ae00 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:39.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:44:39.415+0000 7f5afac00700 1 --2- 172.21.15.151:0/1608163849 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f5ad80a3bb0 0x7f5ad80a6070 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:39.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:44:39.415+0000 7f5afac00700 1 --2- 172.21.15.151:0/1608163849 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5af4076a10 0x7f5af419b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:39.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:44:39.415+0000 7f5afac00700 1 -- 172.21.15.151:0/1608163849 >> 172.21.15.151:0/1608163849 conn(0x7f5af40fe040 msgr2=0x7f5af41068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:44:39.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:44:39.415+0000 7f5afac00700 1 -- 172.21.15.151:0/1608163849 shutdown_connections 2024-04-26T01:44:39.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:44:39.415+0000 7f5afac00700 1 -- 172.21.15.151:0/1608163849 wait complete. 2024-04-26T01:44:39.535 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:39 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:39.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:39 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:39.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:39 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:44:39.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:39 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:44:39.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:39 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:39.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:39 smithi172 ceph-mon[32785]: from='osd.3 [v2:172.21.15.151:6810/3442279476,v1:172.21.15.151:6811/3442279476]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T01:44:40.154 DEBUG:teuthology.orchestra.run.smithi151:osd.3> sudo journalctl -f -n 0 -u ceph-c0e49e64-036d-11ef-bc93-c7b262605968@osd.3.service 2024-04-26T01:44:40.160 INFO:tasks.cephadm:Deploying osd.4 on smithi165 with /dev/vg_nvme/lv_3... 2024-04-26T01:44:40.160 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-26T01:44:40.370 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:40 smithi151 ceph-mon[28057]: from='osd.3 [v2:172.21.15.151:6810/3442279476,v1:172.21.15.151:6811/3442279476]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-26T01:44:40.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:40 smithi151 ceph-mon[28057]: osdmap e24: 4 total, 3 up, 4 in 2024-04-26T01:44:40.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:40 smithi151 ceph-mon[28057]: from='osd.3 [v2:172.21.15.151:6810/3442279476,v1:172.21.15.151:6811/3442279476]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-04-26T01:44:40.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:40 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:44:40.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:40 smithi151 ceph-mon[28057]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:44:40.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:40 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:44:40.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:40 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:40.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:40 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:40.404 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T01:44:40.506 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:40 smithi165 ceph-mon[32648]: from='osd.3 [v2:172.21.15.151:6810/3442279476,v1:172.21.15.151:6811/3442279476]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-26T01:44:40.506 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:40 smithi165 ceph-mon[32648]: osdmap e24: 4 total, 3 up, 4 in 2024-04-26T01:44:40.506 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:40 smithi165 ceph-mon[32648]: from='osd.3 [v2:172.21.15.151:6810/3442279476,v1:172.21.15.151:6811/3442279476]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-04-26T01:44:40.506 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:40 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:44:40.506 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:40 smithi165 ceph-mon[32648]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:44:40.506 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:40 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:44:40.506 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:40 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:40.506 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:40 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:40.535 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:40 smithi172 ceph-mon[32785]: from='osd.3 [v2:172.21.15.151:6810/3442279476,v1:172.21.15.151:6811/3442279476]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-26T01:44:40.535 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:40 smithi172 ceph-mon[32785]: osdmap e24: 4 total, 3 up, 4 in 2024-04-26T01:44:40.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:40 smithi172 ceph-mon[32785]: from='osd.3 [v2:172.21.15.151:6810/3442279476,v1:172.21.15.151:6811/3442279476]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-04-26T01:44:40.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:40 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:44:40.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:40 smithi172 ceph-mon[32785]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:44:40.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:40 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:44:40.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:40 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:40.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:40 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:41.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:41 smithi151 ceph-mon[28057]: from='osd.3 [v2:172.21.15.151:6810/3442279476,v1:172.21.15.151:6811/3442279476]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-04-26T01:44:41.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:41 smithi151 ceph-mon[28057]: osdmap e25: 4 total, 3 up, 4 in 2024-04-26T01:44:41.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:41 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:44:41.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:41 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:44:41.371 INFO:journalctl@ceph.osd.3.smithi151.stdout:Apr 26 01:44:41 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-3[49178]: 2024-04-26T01:44:41.217+0000 7fe1d0200700 -1 osd.3 0 waiting for initial osdmap 2024-04-26T01:44:41.371 INFO:journalctl@ceph.osd.3.smithi151.stdout:Apr 26 01:44:41 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-3[49178]: 2024-04-26T01:44:41.257+0000 7fe1c5600700 -1 osd.3 26 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T01:44:41.486 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:41 smithi165 ceph-mon[32648]: from='osd.3 [v2:172.21.15.151:6810/3442279476,v1:172.21.15.151:6811/3442279476]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-04-26T01:44:41.486 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:41 smithi165 ceph-mon[32648]: osdmap e25: 4 total, 3 up, 4 in 2024-04-26T01:44:41.486 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:41 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:44:41.486 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:41 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:44:41.535 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:41 smithi172 ceph-mon[32785]: from='osd.3 [v2:172.21.15.151:6810/3442279476,v1:172.21.15.151:6811/3442279476]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-04-26T01:44:41.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:41 smithi172 ceph-mon[32785]: osdmap e25: 4 total, 3 up, 4 in 2024-04-26T01:44:41.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:41 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:44:41.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:41 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:44:42.333 INFO:teuthology.orchestra.run.smithi165.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-26T01:44:42.333 INFO:teuthology.orchestra.run.smithi165.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-26T01:44:42.333 INFO:teuthology.orchestra.run.smithi165.stderr: stderr: 10+0 records in 2024-04-26T01:44:42.334 INFO:teuthology.orchestra.run.smithi165.stderr:10+0 records out 2024-04-26T01:44:42.334 INFO:teuthology.orchestra.run.smithi165.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115622 s, 907 MB/s 2024-04-26T01:44:42.334 INFO:teuthology.orchestra.run.smithi165.stderr:--> Zapping successful for: 2024-04-26T01:44:42.342 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:42 smithi151 ceph-mon[28057]: purged_snaps scrub starts 2024-04-26T01:44:42.342 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:42 smithi151 ceph-mon[28057]: purged_snaps scrub ok 2024-04-26T01:44:42.342 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:42 smithi151 ceph-mon[28057]: osdmap e26: 4 total, 3 up, 4 in 2024-04-26T01:44:42.342 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:42 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:44:42.342 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:42 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:44:42.342 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:42 smithi151 ceph-mon[28057]: from='osd.3 [v2:172.21.15.151:6810/3442279476,v1:172.21.15.151:6811/3442279476]' entity='osd.3' 2024-04-26T01:44:42.343 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:42 smithi151 ceph-mon[28057]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:44:42.357 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:42 smithi165 ceph-mon[32648]: purged_snaps scrub starts 2024-04-26T01:44:42.357 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:42 smithi165 ceph-mon[32648]: purged_snaps scrub ok 2024-04-26T01:44:42.357 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:42 smithi165 ceph-mon[32648]: osdmap e26: 4 total, 3 up, 4 in 2024-04-26T01:44:42.357 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:42 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:44:42.357 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:42 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:44:42.357 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:42 smithi165 ceph-mon[32648]: from='osd.3 [v2:172.21.15.151:6810/3442279476,v1:172.21.15.151:6811/3442279476]' entity='osd.3' 2024-04-26T01:44:42.357 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:42 smithi165 ceph-mon[32648]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:44:42.535 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:42 smithi172 ceph-mon[32785]: purged_snaps scrub starts 2024-04-26T01:44:42.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:42 smithi172 ceph-mon[32785]: purged_snaps scrub ok 2024-04-26T01:44:42.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:42 smithi172 ceph-mon[32785]: osdmap e26: 4 total, 3 up, 4 in 2024-04-26T01:44:42.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:42 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:44:42.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:42 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:44:42.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:42 smithi172 ceph-mon[32785]: from='osd.3 [v2:172.21.15.151:6810/3442279476,v1:172.21.15.151:6811/3442279476]' entity='osd.3' 2024-04-26T01:44:42.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:42 smithi172 ceph-mon[32785]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:44:42.929 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi165:vg_nvme/lv_3 2024-04-26T01:44:43.174 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T01:44:43.232 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:43 smithi151 ceph-mon[28057]: osd.3 [v2:172.21.15.151:6810/3442279476,v1:172.21.15.151:6811/3442279476] boot 2024-04-26T01:44:43.232 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:43 smithi151 ceph-mon[28057]: osdmap e27: 4 total, 4 up, 4 in 2024-04-26T01:44:43.232 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:43 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:44:43.232 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:43 smithi151 ceph-mon[28057]: osdmap e28: 4 total, 4 up, 4 in 2024-04-26T01:44:43.296 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:43 smithi165 ceph-mon[32648]: osd.3 [v2:172.21.15.151:6810/3442279476,v1:172.21.15.151:6811/3442279476] boot 2024-04-26T01:44:43.296 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:43 smithi165 ceph-mon[32648]: osdmap e27: 4 total, 4 up, 4 in 2024-04-26T01:44:43.296 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:43 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:44:43.296 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:43 smithi165 ceph-mon[32648]: osdmap e28: 4 total, 4 up, 4 in 2024-04-26T01:44:43.535 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:43 smithi172 ceph-mon[32785]: osd.3 [v2:172.21.15.151:6810/3442279476,v1:172.21.15.151:6811/3442279476] boot 2024-04-26T01:44:43.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:43 smithi172 ceph-mon[32785]: osdmap e27: 4 total, 4 up, 4 in 2024-04-26T01:44:43.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:43 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:44:43.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:43 smithi172 ceph-mon[32785]: osdmap e28: 4 total, 4 up, 4 in 2024-04-26T01:44:44.280 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:44 smithi165 ceph-mon[32648]: pgmap v117: 1 pgs: 1 remapped+peering; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:44:44.281 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:44 smithi165 ceph-mon[32648]: osdmap e29: 4 total, 4 up, 4 in 2024-04-26T01:44:44.535 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:44 smithi172 ceph-mon[32785]: pgmap v117: 1 pgs: 1 remapped+peering; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:44:44.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:44 smithi172 ceph-mon[32785]: osdmap e29: 4 total, 4 up, 4 in 2024-04-26T01:44:44.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:44:44.583+0000 7f0517865700 1 -- 172.21.15.165:0/3006573087 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0510102b90 msgr2=0x7f0510102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:44:44.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:44:44.583+0000 7f0517865700 1 --2- 172.21.15.165:0/3006573087 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0510102b90 0x7f0510102ff0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f05000099c0 tx=0x7f0500009cd0 comp rx=0 tx=0).stop 2024-04-26T01:44:44.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:44:44.583+0000 7f0517865700 1 -- 172.21.15.165:0/3006573087 shutdown_connections 2024-04-26T01:44:44.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:44:44.583+0000 7f0517865700 1 --2- 172.21.15.165:0/3006573087 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0510103530 0x7f0510109de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:44.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:44:44.583+0000 7f0517865700 1 --2- 172.21.15.165:0/3006573087 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0510101990 0x7f0510101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:44.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:44:44.583+0000 7f0517865700 1 --2- 172.21.15.165:0/3006573087 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0510102b90 0x7f0510102ff0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:44.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:44:44.583+0000 7f0517865700 1 -- 172.21.15.165:0/3006573087 >> 172.21.15.165:0/3006573087 conn(0x7f05100fcf30 msgr2=0x7f05100ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:44:44.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:44:44.583+0000 7f0517865700 1 -- 172.21.15.165:0/3006573087 shutdown_connections 2024-04-26T01:44:44.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:44:44.583+0000 7f0517865700 1 -- 172.21.15.165:0/3006573087 wait complete. 2024-04-26T01:44:44.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:44:44.584+0000 7f0517865700 1 Processor -- start 2024-04-26T01:44:44.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:44:44.584+0000 7f0517865700 1 -- start start 2024-04-26T01:44:44.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:44:44.584+0000 7f0517865700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0510101990 0x7f051006bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:44:44.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:44:44.584+0000 7f0517865700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0510102b90 0x7f051019d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:44:44.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:44:44.584+0000 7f0517865700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0510103530 0x7f051019fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:44:44.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:44:44.584+0000 7f0517865700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f051010d400 con 0x7f0510102b90 2024-04-26T01:44:44.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:44:44.584+0000 7f0517865700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f051010d280 con 0x7f0510103530 2024-04-26T01:44:44.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:44:44.584+0000 7f0517865700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f051010d580 con 0x7f0510101990 2024-04-26T01:44:44.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:44:44.584+0000 7f0515e02700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0510103530 0x7f051019fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:44:44.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:44:44.584+0000 7f0515e02700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0510103530 0x7f051019fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49562/0 (socket says 172.21.15.165:49562) 2024-04-26T01:44:44.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:44:44.584+0000 7f0515e02700 1 -- 172.21.15.165:0/3900503351 learned_addr learned my addr 172.21.15.165:0/3900503351 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-26T01:44:44.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:44:44.584+0000 7f0515601700 1 --2- 172.21.15.165:0/3900503351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0510101990 0x7f051006bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:44:44.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:44:44.585+0000 7f0514e00700 1 --2- 172.21.15.165:0/3900503351 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0510102b90 0x7f051019d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:44:44.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:44:44.585+0000 7f0515e02700 1 -- 172.21.15.165:0/3900503351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0510101990 msgr2=0x7f051006bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:44:44.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:44:44.585+0000 7f0515e02700 1 --2- 172.21.15.165:0/3900503351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0510101990 0x7f051006bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:44.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:44:44.585+0000 7f0515e02700 1 -- 172.21.15.165:0/3900503351 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0510102b90 msgr2=0x7f051019d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:44:44.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:44:44.585+0000 7f0515e02700 1 --2- 172.21.15.165:0/3900503351 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0510102b90 0x7f051019d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:44.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:44:44.585+0000 7f0515e02700 1 -- 172.21.15.165:0/3900503351 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05101a07a0 con 0x7f0510103530 2024-04-26T01:44:44.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:44:44.585+0000 7f0515e02700 1 --2- 172.21.15.165:0/3900503351 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0510103530 0x7f051019fed0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f050800c970 tx=0x7f050800cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:44:44.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:44:44.585+0000 7f05067fc700 1 -- 172.21.15.165:0/3900503351 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050800ce60 con 0x7f0510103530 2024-04-26T01:44:44.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:44:44.585+0000 7f0517865700 1 -- 172.21.15.165:0/3900503351 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05101a0a90 con 0x7f0510103530 2024-04-26T01:44:44.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:44:44.585+0000 7f05067fc700 1 -- 172.21.15.165:0/3900503351 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0508004830 con 0x7f0510103530 2024-04-26T01:44:44.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:44:44.585+0000 7f05067fc700 1 -- 172.21.15.165:0/3900503351 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05080054c0 con 0x7f0510103530 2024-04-26T01:44:44.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:44:44.585+0000 7f0517865700 1 -- 172.21.15.165:0/3900503351 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05101a0f50 con 0x7f0510103530 2024-04-26T01:44:44.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:44:44.586+0000 7f0517865700 1 -- 172.21.15.165:0/3900503351 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0510069730 con 0x7f0510103530 2024-04-26T01:44:44.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:44:44.587+0000 7f05067fc700 1 -- 172.21.15.165:0/3900503351 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f05080049e0 con 0x7f0510103530 2024-04-26T01:44:44.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:44:44.588+0000 7f05067fc700 1 --2- 172.21.15.165:0/3900503351 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f04fc0a3c40 0x7f04fc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:44:44.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:44:44.588+0000 7f05067fc700 1 -- 172.21.15.165:0/3900503351 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 3698+0+0 (secure 0 0 0) 0x7f05080cf140 con 0x7f0510103530 2024-04-26T01:44:44.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:44:44.588+0000 7f0515601700 1 --2- 172.21.15.165:0/3900503351 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f04fc0a3c40 0x7f04fc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:44:44.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:44:44.589+0000 7f0515601700 1 --2- 172.21.15.165:0/3900503351 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f04fc0a3c40 0x7f04fc0a6100 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f05101029f0 tx=0x7f050c007a30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:44:44.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:44:44.592+0000 7f05067fc700 1 -- 172.21.15.165:0/3900503351 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0508014030 con 0x7f0510103530 2024-04-26T01:44:44.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:44 smithi151 ceph-mon[28057]: pgmap v117: 1 pgs: 1 remapped+peering; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:44:44.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:44 smithi151 ceph-mon[28057]: osdmap e29: 4 total, 4 up, 4 in 2024-04-26T01:44:44.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:44:44.796+0000 7f0517865700 1 -- 172.21.15.165:0/3900503351 --> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f0510062e80 con 0x7f04fc0a3c40 2024-04-26T01:44:45.408 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:45 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:44:45.408 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:45 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:44:45.408 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:45 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:44:45.529 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:45 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:44:45.529 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:45 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:44:45.530 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:45 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:44:45.535 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:45 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:44:45.535 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:45 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:44:45.535 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:45 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:44:46.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:46 smithi165 ceph-mon[32648]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:44:46.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:46 smithi165 ceph-mon[32648]: osdmap e30: 4 total, 4 up, 4 in 2024-04-26T01:44:46.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:46 smithi165 ceph-mon[32648]: pgmap v120: 1 pgs: 1 remapped+peering; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:44:46.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:46 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:46.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:46 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:46.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:46 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:44:46.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:46 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:44:46.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:46 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:46.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:46 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:44:46.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:46 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:44:46.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:46 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:46.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:46 smithi172 ceph-mon[32785]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:44:46.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:46 smithi172 ceph-mon[32785]: osdmap e30: 4 total, 4 up, 4 in 2024-04-26T01:44:46.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:46 smithi172 ceph-mon[32785]: pgmap v120: 1 pgs: 1 remapped+peering; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:44:46.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:46 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:46.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:46 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:46.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:46 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:44:46.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:46 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:44:46.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:46 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:46.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:46 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:44:46.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:46 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:44:46.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:46 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:46.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:46 smithi151 ceph-mon[28057]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:44:46.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:46 smithi151 ceph-mon[28057]: osdmap e30: 4 total, 4 up, 4 in 2024-04-26T01:44:46.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:46 smithi151 ceph-mon[28057]: pgmap v120: 1 pgs: 1 remapped+peering; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:44:46.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:46 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:46.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:46 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:46.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:46 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:44:46.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:46 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:44:46.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:46 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:46.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:46 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:44:46.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:46 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:44:46.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:46 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:44:47.535 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:47 smithi172 ceph-mon[32785]: Detected new or changed devices on smithi151 2024-04-26T01:44:47.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:47 smithi172 ceph-mon[32785]: Adjusting osd_memory_target on smithi151 to 8544M 2024-04-26T01:44:47.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:47 smithi172 ceph-mon[32785]: osdmap e31: 4 total, 4 up, 4 in 2024-04-26T01:44:47.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:47 smithi165 ceph-mon[32648]: Detected new or changed devices on smithi151 2024-04-26T01:44:47.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:47 smithi165 ceph-mon[32648]: Adjusting osd_memory_target on smithi151 to 8544M 2024-04-26T01:44:47.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:47 smithi165 ceph-mon[32648]: osdmap e31: 4 total, 4 up, 4 in 2024-04-26T01:44:47.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:47 smithi151 ceph-mon[28057]: Detected new or changed devices on smithi151 2024-04-26T01:44:47.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:47 smithi151 ceph-mon[28057]: Adjusting osd_memory_target on smithi151 to 8544M 2024-04-26T01:44:47.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:47 smithi151 ceph-mon[28057]: osdmap e31: 4 total, 4 up, 4 in 2024-04-26T01:44:48.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:48 smithi172 ceph-mon[32785]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 138 KiB/s, 0 objects/s recovering 2024-04-26T01:44:48.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:48 smithi172 ceph-mon[32785]: from='client.? 172.21.15.165:0/2463327595' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0230d615-548c-43f0-b8aa-7130f4cbbc1f"}]: dispatch 2024-04-26T01:44:48.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:48 smithi172 ceph-mon[32785]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0230d615-548c-43f0-b8aa-7130f4cbbc1f"}]: dispatch 2024-04-26T01:44:48.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:48 smithi172 ceph-mon[32785]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0230d615-548c-43f0-b8aa-7130f4cbbc1f"}]': finished 2024-04-26T01:44:48.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:48 smithi172 ceph-mon[32785]: osdmap e32: 5 total, 4 up, 5 in 2024-04-26T01:44:48.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:48 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T01:44:48.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:48 smithi172 ceph-mon[32785]: from='client.? 172.21.15.165:0/3264077038' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:44:48.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:48 smithi165 ceph-mon[32648]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 138 KiB/s, 0 objects/s recovering 2024-04-26T01:44:48.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:48 smithi165 ceph-mon[32648]: from='client.? 172.21.15.165:0/2463327595' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0230d615-548c-43f0-b8aa-7130f4cbbc1f"}]: dispatch 2024-04-26T01:44:48.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:48 smithi165 ceph-mon[32648]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0230d615-548c-43f0-b8aa-7130f4cbbc1f"}]: dispatch 2024-04-26T01:44:48.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:48 smithi165 ceph-mon[32648]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0230d615-548c-43f0-b8aa-7130f4cbbc1f"}]': finished 2024-04-26T01:44:48.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:48 smithi165 ceph-mon[32648]: osdmap e32: 5 total, 4 up, 5 in 2024-04-26T01:44:48.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:48 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T01:44:48.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:48 smithi165 ceph-mon[32648]: from='client.? 172.21.15.165:0/3264077038' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:44:48.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:48 smithi151 ceph-mon[28057]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 138 KiB/s, 0 objects/s recovering 2024-04-26T01:44:48.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:48 smithi151 ceph-mon[28057]: from='client.? 172.21.15.165:0/2463327595' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0230d615-548c-43f0-b8aa-7130f4cbbc1f"}]: dispatch 2024-04-26T01:44:48.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:48 smithi151 ceph-mon[28057]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0230d615-548c-43f0-b8aa-7130f4cbbc1f"}]: dispatch 2024-04-26T01:44:48.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:48 smithi151 ceph-mon[28057]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0230d615-548c-43f0-b8aa-7130f4cbbc1f"}]': finished 2024-04-26T01:44:48.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:48 smithi151 ceph-mon[28057]: osdmap e32: 5 total, 4 up, 5 in 2024-04-26T01:44:48.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:48 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T01:44:48.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:48 smithi151 ceph-mon[28057]: from='client.? 172.21.15.165:0/3264077038' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:44:49.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:49 smithi151 ceph-mon[28057]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-04-26T01:44:49.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:49 smithi172 ceph-mon[32785]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-04-26T01:44:49.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:49 smithi165 ceph-mon[32648]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-04-26T01:44:51.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:51 smithi165 ceph-mon[32648]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-04-26T01:44:51.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:51 smithi151 ceph-mon[28057]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-04-26T01:44:51.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:51 smithi172 ceph-mon[32785]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-04-26T01:44:53.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:53 smithi151 ceph-mon[28057]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T01:44:53.633 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:53 smithi165 ceph-mon[32648]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T01:44:53.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:53 smithi172 ceph-mon[32785]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T01:44:54.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:54 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-26T01:44:54.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:54 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:44:55.035 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:54 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-26T01:44:55.035 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:54 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:44:55.120 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:54 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-26T01:44:55.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:54 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:44:55.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:55 smithi165 ceph-mon[32648]: Deploying daemon osd.4 on smithi165 2024-04-26T01:44:55.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:55 smithi165 ceph-mon[32648]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-04-26T01:44:56.035 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:55 smithi172 ceph-mon[32785]: Deploying daemon osd.4 on smithi165 2024-04-26T01:44:56.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:55 smithi172 ceph-mon[32785]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-04-26T01:44:56.120 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:55 smithi151 ceph-mon[28057]: Deploying daemon osd.4 on smithi165 2024-04-26T01:44:56.120 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:55 smithi151 ceph-mon[28057]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-04-26T01:44:57.606 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:57 smithi165 ceph-mon[32648]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:44:57.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:57 smithi151 ceph-mon[28057]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:44:57.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:57 smithi172 ceph-mon[32785]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:44:59.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:44:59 smithi165 ceph-mon[32648]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:44:59.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:44:59 smithi151 ceph-mon[28057]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:44:59.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:44:59 smithi172 ceph-mon[32785]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:45:01.120 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:00 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:01.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:00 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:01.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:00 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:45:01.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:00 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:01.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:00 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:01.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:00 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:45:01.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:00 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:01.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:00 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:01.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:00 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:45:02.094 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:01 smithi165 ceph-mon[32648]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:45:02.120 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:01 smithi151 ceph-mon[28057]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:45:02.285 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:01 smithi172 ceph-mon[32785]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:45:03.396 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:03 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:03.396 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:03 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:03.396 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:03 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:45:03.396 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:03 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:45:03.396 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:03 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:03.396 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:03 smithi165 ceph-mon[32648]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:45:03.470 INFO:teuthology.orchestra.run.smithi165.stdout:Created osd(s) 4 on host 'smithi165' 2024-04-26T01:45:03.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:45:03.472+0000 7f05067fc700 1 -- 172.21.15.165:0/3900503351 <== mgr.14152 v2:172.21.15.151:6800/2251256505 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0510062e80 con 0x7f04fc0a3c40 2024-04-26T01:45:03.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:45:03.476+0000 7f0517865700 1 -- 172.21.15.165:0/3900503351 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f04fc0a3c40 msgr2=0x7f04fc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:03.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:45:03.476+0000 7f0517865700 1 --2- 172.21.15.165:0/3900503351 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f04fc0a3c40 0x7f04fc0a6100 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f05101029f0 tx=0x7f050c007a30 comp rx=0 tx=0).stop 2024-04-26T01:45:03.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:45:03.476+0000 7f0517865700 1 -- 172.21.15.165:0/3900503351 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0510103530 msgr2=0x7f051019fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:03.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:45:03.476+0000 7f0517865700 1 --2- 172.21.15.165:0/3900503351 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0510103530 0x7f051019fed0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f050800c970 tx=0x7f050800cc80 comp rx=0 tx=0).stop 2024-04-26T01:45:03.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:45:03.476+0000 7f0517865700 1 -- 172.21.15.165:0/3900503351 shutdown_connections 2024-04-26T01:45:03.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:45:03.476+0000 7f0517865700 1 --2- 172.21.15.165:0/3900503351 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0510103530 0x7f051019fed0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:03.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:45:03.476+0000 7f0517865700 1 --2- 172.21.15.165:0/3900503351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0510101990 0x7f051006bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:03.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:45:03.476+0000 7f0517865700 1 --2- 172.21.15.165:0/3900503351 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f04fc0a3c40 0x7f04fc0a6100 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:03.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:45:03.476+0000 7f0517865700 1 --2- 172.21.15.165:0/3900503351 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0510102b90 0x7f051019d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:03.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:45:03.476+0000 7f0517865700 1 -- 172.21.15.165:0/3900503351 >> 172.21.15.165:0/3900503351 conn(0x7f05100fcf30 msgr2=0x7f0510105f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:45:03.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:45:03.476+0000 7f0517865700 1 -- 172.21.15.165:0/3900503351 shutdown_connections 2024-04-26T01:45:03.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:45:03.476+0000 7f0517865700 1 -- 172.21.15.165:0/3900503351 wait complete. 2024-04-26T01:45:03.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:03 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:03.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:03 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:03.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:03 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:45:03.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:03 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:45:03.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:03 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:03.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:03 smithi151 ceph-mon[28057]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:45:03.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:03 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:03.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:03 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:03.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:03 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:45:03.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:03 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:45:03.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:03 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:03.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:03 smithi172 ceph-mon[32785]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:45:04.042 DEBUG:teuthology.orchestra.run.smithi165:osd.4> sudo journalctl -f -n 0 -u ceph-c0e49e64-036d-11ef-bc93-c7b262605968@osd.4.service 2024-04-26T01:45:04.044 INFO:tasks.cephadm:Deploying osd.5 on smithi172 with /dev/vg_nvme/lv_3... 2024-04-26T01:45:04.044 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-26T01:45:04.288 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T01:45:04.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:04 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:04.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:04 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:04.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:04 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:45:04.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:04 smithi165 ceph-mon[32648]: from='osd.4 [v2:172.21.15.165:6808/4029980723,v1:172.21.15.165:6809/4029980723]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T01:45:04.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:04 smithi165 ceph-mon[32648]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T01:45:04.727 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:04 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:04.727 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:04 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:04.727 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:04 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:45:04.727 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:04 smithi172 ceph-mon[32785]: from='osd.4 [v2:172.21.15.165:6808/4029980723,v1:172.21.15.165:6809/4029980723]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T01:45:04.727 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:04 smithi172 ceph-mon[32785]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T01:45:04.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:04 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:04.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:04 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:04.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:04 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:45:04.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:04 smithi151 ceph-mon[28057]: from='osd.4 [v2:172.21.15.165:6808/4029980723,v1:172.21.15.165:6809/4029980723]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T01:45:04.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:04 smithi151 ceph-mon[28057]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T01:45:05.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:05 smithi165 ceph-mon[32648]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-26T01:45:05.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:05 smithi165 ceph-mon[32648]: from='osd.4 [v2:172.21.15.165:6808/4029980723,v1:172.21.15.165:6809/4029980723]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-26T01:45:05.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:05 smithi165 ceph-mon[32648]: osdmap e33: 5 total, 4 up, 5 in 2024-04-26T01:45:05.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:05 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T01:45:05.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:05 smithi165 ceph-mon[32648]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-26T01:45:05.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:05 smithi165 ceph-mon[32648]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:45:05.633 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:05 smithi172 ceph-mon[32785]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-26T01:45:05.633 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:05 smithi172 ceph-mon[32785]: from='osd.4 [v2:172.21.15.165:6808/4029980723,v1:172.21.15.165:6809/4029980723]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-26T01:45:05.633 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:05 smithi172 ceph-mon[32785]: osdmap e33: 5 total, 4 up, 5 in 2024-04-26T01:45:05.633 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:05 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T01:45:05.633 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:05 smithi172 ceph-mon[32785]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-26T01:45:05.633 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:05 smithi172 ceph-mon[32785]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:45:05.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:05 smithi151 ceph-mon[28057]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-26T01:45:05.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:05 smithi151 ceph-mon[28057]: from='osd.4 [v2:172.21.15.165:6808/4029980723,v1:172.21.15.165:6809/4029980723]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-26T01:45:05.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:05 smithi151 ceph-mon[28057]: osdmap e33: 5 total, 4 up, 5 in 2024-04-26T01:45:05.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:05 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T01:45:05.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:05 smithi151 ceph-mon[28057]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-26T01:45:05.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:05 smithi151 ceph-mon[28057]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:45:06.225 INFO:teuthology.orchestra.run.smithi172.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-26T01:45:06.225 INFO:teuthology.orchestra.run.smithi172.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-26T01:45:06.225 INFO:teuthology.orchestra.run.smithi172.stderr: stderr: 10+0 records in 2024-04-26T01:45:06.225 INFO:teuthology.orchestra.run.smithi172.stderr:10+0 records out 2024-04-26T01:45:06.225 INFO:teuthology.orchestra.run.smithi172.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114558 s, 915 MB/s 2024-04-26T01:45:06.225 INFO:teuthology.orchestra.run.smithi172.stderr:--> Zapping successful for: 2024-04-26T01:45:06.564 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:06 smithi165 ceph-mon[32648]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-04-26T01:45:06.564 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:06 smithi165 ceph-mon[32648]: osdmap e34: 5 total, 4 up, 5 in 2024-04-26T01:45:06.564 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:06 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T01:45:06.564 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:06 smithi165 ceph-mon[32648]: from='osd.4 ' entity='osd.4' 2024-04-26T01:45:06.565 INFO:journalctl@ceph.osd.4.smithi165.stdout:Apr 26 01:45:06 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-4[43947]: 2024-04-26T01:45:06.400+0000 7f9e0ea30700 -1 osd.4 0 waiting for initial osdmap 2024-04-26T01:45:06.565 INFO:journalctl@ceph.osd.4.smithi165.stdout:Apr 26 01:45:06 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-4[43947]: 2024-04-26T01:45:06.566+0000 7f9e0601a700 -1 osd.4 34 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T01:45:06.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:06 smithi172 ceph-mon[32785]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-04-26T01:45:06.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:06 smithi172 ceph-mon[32785]: osdmap e34: 5 total, 4 up, 5 in 2024-04-26T01:45:06.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:06 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T01:45:06.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:06 smithi172 ceph-mon[32785]: from='osd.4 ' entity='osd.4' 2024-04-26T01:45:06.870 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:06 smithi151 ceph-mon[28057]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-04-26T01:45:06.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:06 smithi151 ceph-mon[28057]: osdmap e34: 5 total, 4 up, 5 in 2024-04-26T01:45:06.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:06 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T01:45:06.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:06 smithi151 ceph-mon[28057]: from='osd.4 ' entity='osd.4' 2024-04-26T01:45:06.899 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi172:vg_nvme/lv_3 2024-04-26T01:45:07.135 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T01:45:07.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:07 smithi172 ceph-mon[32785]: purged_snaps scrub starts 2024-04-26T01:45:07.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:07 smithi172 ceph-mon[32785]: purged_snaps scrub ok 2024-04-26T01:45:07.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:07 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T01:45:07.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:07 smithi172 ceph-mon[32785]: Detected new or changed devices on smithi165 2024-04-26T01:45:07.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:07 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:07.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:07 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:07.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:07 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:45:07.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:07 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:45:07.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:07 smithi172 ceph-mon[32785]: Adjusting osd_memory_target on smithi165 to 8544M 2024-04-26T01:45:07.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:07 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:07.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:07 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:45:07.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:07 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:45:07.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:07 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:07.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:07 smithi172 ceph-mon[32785]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:45:07.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:07 smithi165 ceph-mon[32648]: purged_snaps scrub starts 2024-04-26T01:45:07.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:07 smithi165 ceph-mon[32648]: purged_snaps scrub ok 2024-04-26T01:45:07.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:07 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T01:45:07.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:07 smithi165 ceph-mon[32648]: Detected new or changed devices on smithi165 2024-04-26T01:45:07.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:07 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:07.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:07 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:07.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:07 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:45:07.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:07 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:45:07.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:07 smithi165 ceph-mon[32648]: Adjusting osd_memory_target on smithi165 to 8544M 2024-04-26T01:45:07.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:07 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:07.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:07 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:45:07.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:07 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:45:07.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:07 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:07.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:07 smithi165 ceph-mon[32648]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:45:07.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:07 smithi151 ceph-mon[28057]: purged_snaps scrub starts 2024-04-26T01:45:07.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:07 smithi151 ceph-mon[28057]: purged_snaps scrub ok 2024-04-26T01:45:07.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:07 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T01:45:07.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:07 smithi151 ceph-mon[28057]: Detected new or changed devices on smithi165 2024-04-26T01:45:07.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:07 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:07.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:07 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:07.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:07 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:45:07.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:07 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:45:07.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:07 smithi151 ceph-mon[28057]: Adjusting osd_memory_target on smithi165 to 8544M 2024-04-26T01:45:07.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:07 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:07.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:07 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:45:07.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:07 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:45:07.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:07 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:07.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:07 smithi151 ceph-mon[28057]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:45:08.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:45:08.678+0000 7fd647fde700 1 -- 172.21.15.172:0/2458997024 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd640103530 msgr2=0x7fd640109de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:08.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:45:08.678+0000 7fd647fde700 1 --2- 172.21.15.172:0/2458997024 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd640103530 0x7fd640109de0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd63800b0a0 tx=0x7fd63800b3b0 comp rx=0 tx=0).stop 2024-04-26T01:45:08.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:45:08.678+0000 7fd647fde700 1 -- 172.21.15.172:0/2458997024 shutdown_connections 2024-04-26T01:45:08.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:45:08.678+0000 7fd647fde700 1 --2- 172.21.15.172:0/2458997024 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd640103530 0x7fd640109de0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:08.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:45:08.678+0000 7fd647fde700 1 --2- 172.21.15.172:0/2458997024 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd640101990 0x7fd640101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:08.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:45:08.678+0000 7fd647fde700 1 --2- 172.21.15.172:0/2458997024 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd640102b90 0x7fd640102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:08.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:45:08.678+0000 7fd647fde700 1 -- 172.21.15.172:0/2458997024 >> 172.21.15.172:0/2458997024 conn(0x7fd6400fcf30 msgr2=0x7fd6400ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:45:08.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:45:08.679+0000 7fd647fde700 1 -- 172.21.15.172:0/2458997024 shutdown_connections 2024-04-26T01:45:08.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:45:08.679+0000 7fd647fde700 1 -- 172.21.15.172:0/2458997024 wait complete. 2024-04-26T01:45:08.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:45:08.679+0000 7fd647fde700 1 Processor -- start 2024-04-26T01:45:08.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:45:08.679+0000 7fd647fde700 1 -- start start 2024-04-26T01:45:08.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:45:08.680+0000 7fd647fde700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd640101990 0x7fd640197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:08.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:45:08.680+0000 7fd647fde700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd640102b90 0x7fd640198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:08.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:45:08.680+0000 7fd647fde700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd640103530 0x7fd64019f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:08.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:45:08.680+0000 7fd647fde700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd64010d520 con 0x7fd640103530 2024-04-26T01:45:08.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:45:08.680+0000 7fd647fde700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd64010d3a0 con 0x7fd640102b90 2024-04-26T01:45:08.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:45:08.680+0000 7fd647fde700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd64010d6a0 con 0x7fd640101990 2024-04-26T01:45:08.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:45:08.680+0000 7fd645d7a700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd640101990 0x7fd640197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:08.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:45:08.680+0000 7fd645d7a700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd640101990 0x7fd640197be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40234/0 (socket says 172.21.15.172:40234) 2024-04-26T01:45:08.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:45:08.680+0000 7fd645d7a700 1 -- 172.21.15.172:0/2281735574 learned_addr learned my addr 172.21.15.172:0/2281735574 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-26T01:45:08.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:45:08.680+0000 7fd645579700 1 --2- 172.21.15.172:0/2281735574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd640102b90 0x7fd640198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:08.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:45:08.680+0000 7fd645d7a700 1 -- 172.21.15.172:0/2281735574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd640102b90 msgr2=0x7fd640198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:08.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:45:08.680+0000 7fd645d7a700 1 --2- 172.21.15.172:0/2281735574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd640102b90 0x7fd640198120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:08.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:45:08.680+0000 7fd645d7a700 1 -- 172.21.15.172:0/2281735574 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd640103530 msgr2=0x7fd64019f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:08.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:45:08.680+0000 7fd645d7a700 1 --2- 172.21.15.172:0/2281735574 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd640103530 0x7fd64019f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:08.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:45:08.680+0000 7fd645d7a700 1 -- 172.21.15.172:0/2281735574 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd64019f840 con 0x7fd640101990 2024-04-26T01:45:08.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:45:08.680+0000 7fd645d7a700 1 --2- 172.21.15.172:0/2281735574 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd640101990 0x7fd640197be0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd63c00b530 tx=0x7fd63c00b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:08.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:45:08.680+0000 7fd636ffd700 1 -- 172.21.15.172:0/2281735574 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63c015020 con 0x7fd640101990 2024-04-26T01:45:08.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:45:08.680+0000 7fd647fde700 1 -- 172.21.15.172:0/2281735574 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd64019fb30 con 0x7fd640101990 2024-04-26T01:45:08.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:45:08.680+0000 7fd636ffd700 1 -- 172.21.15.172:0/2281735574 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd63c011ce0 con 0x7fd640101990 2024-04-26T01:45:08.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:45:08.680+0000 7fd636ffd700 1 -- 172.21.15.172:0/2281735574 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63c010d60 con 0x7fd640101990 2024-04-26T01:45:08.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:45:08.681+0000 7fd647fde700 1 -- 172.21.15.172:0/2281735574 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6401a00c0 con 0x7fd640101990 2024-04-26T01:45:08.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:45:08.681+0000 7fd647fde700 1 -- 172.21.15.172:0/2281735574 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd640069730 con 0x7fd640101990 2024-04-26T01:45:08.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:45:08.683+0000 7fd636ffd700 1 -- 172.21.15.172:0/2281735574 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fd63c018020 con 0x7fd640101990 2024-04-26T01:45:08.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:45:08.683+0000 7fd636ffd700 1 --2- 172.21.15.172:0/2281735574 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fd62c0a3c40 0x7fd62c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:08.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:45:08.683+0000 7fd636ffd700 1 -- 172.21.15.172:0/2281735574 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4101+0+0 (secure 0 0 0) 0x7fd63c0cf120 con 0x7fd640101990 2024-04-26T01:45:08.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:45:08.684+0000 7fd645579700 1 --2- 172.21.15.172:0/2281735574 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fd62c0a3c40 0x7fd62c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:08.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:45:08.685+0000 7fd645579700 1 --2- 172.21.15.172:0/2281735574 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fd62c0a3c40 0x7fd62c0a6100 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd630006fd0 tx=0x7fd630008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:08.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:45:08.687+0000 7fd636ffd700 1 -- 172.21.15.172:0/2281735574 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd63c01f020 con 0x7fd640101990 2024-04-26T01:45:08.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:08 smithi172 ceph-mon[32785]: osd.4 [v2:172.21.15.165:6808/4029980723,v1:172.21.15.165:6809/4029980723] boot 2024-04-26T01:45:08.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:08 smithi172 ceph-mon[32785]: osdmap e35: 5 total, 5 up, 5 in 2024-04-26T01:45:08.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:08 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T01:45:08.837 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:08 smithi165 ceph-mon[32648]: osd.4 [v2:172.21.15.165:6808/4029980723,v1:172.21.15.165:6809/4029980723] boot 2024-04-26T01:45:08.837 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:08 smithi165 ceph-mon[32648]: osdmap e35: 5 total, 5 up, 5 in 2024-04-26T01:45:08.837 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:08 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T01:45:08.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:08 smithi151 ceph-mon[28057]: osd.4 [v2:172.21.15.165:6808/4029980723,v1:172.21.15.165:6809/4029980723] boot 2024-04-26T01:45:08.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:08 smithi151 ceph-mon[28057]: osdmap e35: 5 total, 5 up, 5 in 2024-04-26T01:45:08.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:08 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T01:45:08.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:45:08.894+0000 7fd647fde700 1 -- 172.21.15.172:0/2281735574 --> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fd640062e80 con 0x7fd62c0a3c40 2024-04-26T01:45:09.756 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:09 smithi172 ceph-mon[32785]: osdmap e36: 5 total, 5 up, 5 in 2024-04-26T01:45:09.756 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:09 smithi172 ceph-mon[32785]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:45:09.756 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:09 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:45:09.756 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:09 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:45:09.757 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:09 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:45:09.757 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:09 smithi172 ceph-mon[32785]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:45:09.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:09 smithi165 ceph-mon[32648]: osdmap e36: 5 total, 5 up, 5 in 2024-04-26T01:45:09.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:09 smithi165 ceph-mon[32648]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:45:09.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:09 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:45:09.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:09 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:45:09.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:09 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:45:09.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:09 smithi165 ceph-mon[32648]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:45:09.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:09 smithi151 ceph-mon[28057]: osdmap e36: 5 total, 5 up, 5 in 2024-04-26T01:45:09.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:09 smithi151 ceph-mon[28057]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:45:09.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:09 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:45:09.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:09 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:45:09.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:09 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:45:09.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:09 smithi151 ceph-mon[28057]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:45:11.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:11 smithi151 ceph-mon[28057]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:45:11.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:11 smithi172 ceph-mon[32785]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:45:11.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:11 smithi165 ceph-mon[32648]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:45:12.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:12 smithi151 ceph-mon[28057]: from='client.? 172.21.15.172:0/173607227' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9739f44b-2605-47dd-82fb-d30034742bf3"}]: dispatch 2024-04-26T01:45:12.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:12 smithi151 ceph-mon[28057]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9739f44b-2605-47dd-82fb-d30034742bf3"}]: dispatch 2024-04-26T01:45:12.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:12 smithi151 ceph-mon[28057]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9739f44b-2605-47dd-82fb-d30034742bf3"}]': finished 2024-04-26T01:45:12.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:12 smithi151 ceph-mon[28057]: osdmap e37: 6 total, 5 up, 6 in 2024-04-26T01:45:12.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:12 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T01:45:12.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:12 smithi151 ceph-mon[28057]: from='client.? 172.21.15.172:0/350288566' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:45:12.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:12 smithi172 ceph-mon[32785]: from='client.? 172.21.15.172:0/173607227' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9739f44b-2605-47dd-82fb-d30034742bf3"}]: dispatch 2024-04-26T01:45:12.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:12 smithi172 ceph-mon[32785]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9739f44b-2605-47dd-82fb-d30034742bf3"}]: dispatch 2024-04-26T01:45:12.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:12 smithi172 ceph-mon[32785]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9739f44b-2605-47dd-82fb-d30034742bf3"}]': finished 2024-04-26T01:45:12.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:12 smithi172 ceph-mon[32785]: osdmap e37: 6 total, 5 up, 6 in 2024-04-26T01:45:12.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:12 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T01:45:12.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:12 smithi172 ceph-mon[32785]: from='client.? 172.21.15.172:0/350288566' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:45:12.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:12 smithi165 ceph-mon[32648]: from='client.? 172.21.15.172:0/173607227' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9739f44b-2605-47dd-82fb-d30034742bf3"}]: dispatch 2024-04-26T01:45:12.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:12 smithi165 ceph-mon[32648]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9739f44b-2605-47dd-82fb-d30034742bf3"}]: dispatch 2024-04-26T01:45:12.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:12 smithi165 ceph-mon[32648]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9739f44b-2605-47dd-82fb-d30034742bf3"}]': finished 2024-04-26T01:45:12.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:12 smithi165 ceph-mon[32648]: osdmap e37: 6 total, 5 up, 6 in 2024-04-26T01:45:12.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:12 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T01:45:12.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:12 smithi165 ceph-mon[32648]: from='client.? 172.21.15.172:0/350288566' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:45:13.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:13 smithi172 ceph-mon[32785]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:45:13.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:13 smithi165 ceph-mon[32648]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:45:13.870 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:13 smithi151 ceph-mon[28057]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:45:15.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:15 smithi151 ceph-mon[28057]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:45:15.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:15 smithi172 ceph-mon[32785]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:45:15.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:15 smithi165 ceph-mon[32648]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:45:17.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:17 smithi151 ceph-mon[28057]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:45:17.700 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:17 smithi172 ceph-mon[32785]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:45:17.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:17 smithi165 ceph-mon[32648]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:45:19.535 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:19 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-26T01:45:19.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:19 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:45:19.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:19 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-26T01:45:19.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:19 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:45:19.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:19 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-26T01:45:19.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:19 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:45:20.404 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:20 smithi172 ceph-mon[32785]: Deploying daemon osd.5 on smithi172 2024-04-26T01:45:20.405 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:20 smithi172 ceph-mon[32785]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:45:20.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:20 smithi165 ceph-mon[32648]: Deploying daemon osd.5 on smithi172 2024-04-26T01:45:20.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:20 smithi165 ceph-mon[32648]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:45:20.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:20 smithi151 ceph-mon[28057]: Deploying daemon osd.5 on smithi172 2024-04-26T01:45:20.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:20 smithi151 ceph-mon[28057]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:45:21.473 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:21 smithi172 ceph-mon[32785]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:45:21.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:21 smithi151 ceph-mon[28057]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:45:21.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:21 smithi165 ceph-mon[32648]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:45:23.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:23 smithi151 ceph-mon[28057]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:45:23.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:23 smithi172 ceph-mon[32785]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:45:23.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:23 smithi165 ceph-mon[32648]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:45:26.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:25 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:26.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:25 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:26.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:25 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:45:26.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:25 smithi172 ceph-mon[32785]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:45:26.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:25 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:26.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:25 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:26.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:25 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:45:26.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:25 smithi165 ceph-mon[32648]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:45:26.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:25 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:26.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:25 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:26.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:25 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:45:26.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:25 smithi151 ceph-mon[28057]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:45:27.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:27 smithi151 ceph-mon[28057]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:45:27.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:27 smithi172 ceph-mon[32785]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:45:27.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:27 smithi165 ceph-mon[32648]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:45:28.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:45:28.137+0000 7fd636ffd700 1 -- 172.21.15.172:0/2281735574 <== mgr.14152 v2:172.21.15.151:6800/2251256505 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd640062e80 con 0x7fd62c0a3c40 2024-04-26T01:45:28.135 INFO:teuthology.orchestra.run.smithi172.stdout:Created osd(s) 5 on host 'smithi172' 2024-04-26T01:45:28.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:45:28.139+0000 7fd647fde700 1 -- 172.21.15.172:0/2281735574 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fd62c0a3c40 msgr2=0x7fd62c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:28.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:45:28.139+0000 7fd647fde700 1 --2- 172.21.15.172:0/2281735574 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fd62c0a3c40 0x7fd62c0a6100 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd630006fd0 tx=0x7fd630008040 comp rx=0 tx=0).stop 2024-04-26T01:45:28.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:45:28.139+0000 7fd647fde700 1 -- 172.21.15.172:0/2281735574 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd640101990 msgr2=0x7fd640197be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:28.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:45:28.139+0000 7fd647fde700 1 --2- 172.21.15.172:0/2281735574 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd640101990 0x7fd640197be0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd63c00b530 tx=0x7fd63c00b8f0 comp rx=0 tx=0).stop 2024-04-26T01:45:28.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:45:28.139+0000 7fd647fde700 1 -- 172.21.15.172:0/2281735574 shutdown_connections 2024-04-26T01:45:28.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:45:28.139+0000 7fd647fde700 1 --2- 172.21.15.172:0/2281735574 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fd62c0a3c40 0x7fd62c0a6100 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:28.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:45:28.139+0000 7fd647fde700 1 --2- 172.21.15.172:0/2281735574 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd640103530 0x7fd64019f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:28.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:45:28.139+0000 7fd647fde700 1 --2- 172.21.15.172:0/2281735574 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd640101990 0x7fd640197be0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:28.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:45:28.139+0000 7fd647fde700 1 --2- 172.21.15.172:0/2281735574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd640102b90 0x7fd640198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:28.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:45:28.139+0000 7fd647fde700 1 -- 172.21.15.172:0/2281735574 >> 172.21.15.172:0/2281735574 conn(0x7fd6400fcf30 msgr2=0x7fd640105f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:45:28.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:45:28.139+0000 7fd647fde700 1 -- 172.21.15.172:0/2281735574 shutdown_connections 2024-04-26T01:45:28.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:45:28.139+0000 7fd647fde700 1 -- 172.21.15.172:0/2281735574 wait complete. 2024-04-26T01:45:28.794 DEBUG:teuthology.orchestra.run.smithi172:osd.5> sudo journalctl -f -n 0 -u ceph-c0e49e64-036d-11ef-bc93-c7b262605968@osd.5.service 2024-04-26T01:45:28.796 INFO:tasks.cephadm:Deploying osd.6 on smithi151 with /dev/vg_nvme/lv_2... 2024-04-26T01:45:28.796 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-26T01:45:29.254 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:45:29.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:29 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:29.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:29 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:29.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:29 smithi165 ceph-mon[32648]: from='osd.5 [v2:172.21.15.172:6808/2216578687,v1:172.21.15.172:6809/2216578687]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T01:45:29.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:29 smithi165 ceph-mon[32648]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T01:45:29.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:29 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:29.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:29 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:29.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:29 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:29.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:29 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:29.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:29 smithi151 ceph-mon[28057]: from='osd.5 [v2:172.21.15.172:6808/2216578687,v1:172.21.15.172:6809/2216578687]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T01:45:29.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:29 smithi151 ceph-mon[28057]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T01:45:29.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:29 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:29.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:29 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:29.535 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:29 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:29.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:29 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:29.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:29 smithi172 ceph-mon[32785]: from='osd.5 [v2:172.21.15.172:6808/2216578687,v1:172.21.15.172:6809/2216578687]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T01:45:29.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:29 smithi172 ceph-mon[32785]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T01:45:29.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:29 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:29.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:29 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:30.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:30 smithi172 ceph-mon[32785]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-26T01:45:30.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:30 smithi172 ceph-mon[32785]: osdmap e38: 6 total, 5 up, 6 in 2024-04-26T01:45:30.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:30 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T01:45:30.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:30 smithi172 ceph-mon[32785]: from='osd.5 [v2:172.21.15.172:6808/2216578687,v1:172.21.15.172:6809/2216578687]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-04-26T01:45:30.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:30 smithi172 ceph-mon[32785]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-04-26T01:45:30.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:30 smithi172 ceph-mon[32785]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:45:30.287 INFO:journalctl@ceph.osd.5.smithi172.stdout:Apr 26 01:45:30 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-5[44593]: 2024-04-26T01:45:30.139+0000 7f60a159c700 -1 osd.5 0 waiting for initial osdmap 2024-04-26T01:45:30.287 INFO:journalctl@ceph.osd.5.smithi172.stdout:Apr 26 01:45:30 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-5[44593]: 2024-04-26T01:45:30.147+0000 7f609d38f700 -1 osd.5 39 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T01:45:30.461 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:30 smithi151 ceph-mon[28057]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-26T01:45:30.461 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:30 smithi151 ceph-mon[28057]: osdmap e38: 6 total, 5 up, 6 in 2024-04-26T01:45:30.461 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:30 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T01:45:30.461 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:30 smithi151 ceph-mon[28057]: from='osd.5 [v2:172.21.15.172:6808/2216578687,v1:172.21.15.172:6809/2216578687]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-04-26T01:45:30.461 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:30 smithi151 ceph-mon[28057]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-04-26T01:45:30.462 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:30 smithi151 ceph-mon[28057]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:45:30.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:30 smithi165 ceph-mon[32648]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-26T01:45:30.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:30 smithi165 ceph-mon[32648]: osdmap e38: 6 total, 5 up, 6 in 2024-04-26T01:45:30.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:30 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T01:45:30.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:30 smithi165 ceph-mon[32648]: from='osd.5 [v2:172.21.15.172:6808/2216578687,v1:172.21.15.172:6809/2216578687]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-04-26T01:45:30.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:30 smithi165 ceph-mon[32648]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-04-26T01:45:30.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:30 smithi165 ceph-mon[32648]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:45:31.285 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:31 smithi151 ceph-mon[28057]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-04-26T01:45:31.286 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:31 smithi151 ceph-mon[28057]: osdmap e39: 6 total, 5 up, 6 in 2024-04-26T01:45:31.286 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:31 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T01:45:31.286 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:31 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T01:45:31.310 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:31 smithi172 ceph-mon[32785]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-04-26T01:45:31.310 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:31 smithi172 ceph-mon[32785]: osdmap e39: 6 total, 5 up, 6 in 2024-04-26T01:45:31.310 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:31 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T01:45:31.310 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:31 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T01:45:31.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:31 smithi165 ceph-mon[32648]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-04-26T01:45:31.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:31 smithi165 ceph-mon[32648]: osdmap e39: 6 total, 5 up, 6 in 2024-04-26T01:45:31.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:31 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T01:45:31.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:31 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T01:45:32.208 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-26T01:45:32.208 INFO:teuthology.orchestra.run.smithi151.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-26T01:45:32.208 INFO:teuthology.orchestra.run.smithi151.stderr: stderr: 10+0 records in 2024-04-26T01:45:32.208 INFO:teuthology.orchestra.run.smithi151.stderr:10+0 records out 2024-04-26T01:45:32.209 INFO:teuthology.orchestra.run.smithi151.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0408834 s, 256 MB/s 2024-04-26T01:45:32.209 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping successful for: 2024-04-26T01:45:32.231 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:32 smithi172 ceph-mon[32785]: purged_snaps scrub starts 2024-04-26T01:45:32.231 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:32 smithi172 ceph-mon[32785]: purged_snaps scrub ok 2024-04-26T01:45:32.231 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:32 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T01:45:32.232 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:32 smithi172 ceph-mon[32785]: osd.5 [v2:172.21.15.172:6808/2216578687,v1:172.21.15.172:6809/2216578687] boot 2024-04-26T01:45:32.232 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:32 smithi172 ceph-mon[32785]: osdmap e40: 6 total, 6 up, 6 in 2024-04-26T01:45:32.232 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:32 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T01:45:32.232 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:32 smithi172 ceph-mon[32785]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:45:32.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:32 smithi165 ceph-mon[32648]: purged_snaps scrub starts 2024-04-26T01:45:32.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:32 smithi165 ceph-mon[32648]: purged_snaps scrub ok 2024-04-26T01:45:32.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:32 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T01:45:32.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:32 smithi165 ceph-mon[32648]: osd.5 [v2:172.21.15.172:6808/2216578687,v1:172.21.15.172:6809/2216578687] boot 2024-04-26T01:45:32.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:32 smithi165 ceph-mon[32648]: osdmap e40: 6 total, 6 up, 6 in 2024-04-26T01:45:32.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:32 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T01:45:32.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:32 smithi165 ceph-mon[32648]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:45:32.606 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:32 smithi151 ceph-mon[28057]: purged_snaps scrub starts 2024-04-26T01:45:32.606 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:32 smithi151 ceph-mon[28057]: purged_snaps scrub ok 2024-04-26T01:45:32.606 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:32 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T01:45:32.606 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:32 smithi151 ceph-mon[28057]: osd.5 [v2:172.21.15.172:6808/2216578687,v1:172.21.15.172:6809/2216578687] boot 2024-04-26T01:45:32.606 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:32 smithi151 ceph-mon[28057]: osdmap e40: 6 total, 6 up, 6 in 2024-04-26T01:45:32.606 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:32 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T01:45:32.606 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:32 smithi151 ceph-mon[28057]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:45:33.087 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi151:vg_nvme/lv_2 2024-04-26T01:45:33.515 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:45:33.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:33 smithi172 ceph-mon[32785]: osdmap e41: 6 total, 6 up, 6 in 2024-04-26T01:45:33.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:33 smithi172 ceph-mon[32785]: Detected new or changed devices on smithi172 2024-04-26T01:45:33.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:33 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:33.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:33 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:33.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:33 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:45:33.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:33 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:45:33.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:33 smithi172 ceph-mon[32785]: Adjusting osd_memory_target on smithi172 to 8544M 2024-04-26T01:45:33.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:33 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:33.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:33 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:45:33.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:33 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:45:33.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:33 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:33.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:33 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:45:33.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:33 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:45:33.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:33 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:45:33.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:33 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:33.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:33 smithi165 ceph-mon[32648]: osdmap e41: 6 total, 6 up, 6 in 2024-04-26T01:45:33.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:33 smithi165 ceph-mon[32648]: Detected new or changed devices on smithi172 2024-04-26T01:45:33.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:33 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:33.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:33 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:33.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:33 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:45:33.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:33 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:45:33.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:33 smithi165 ceph-mon[32648]: Adjusting osd_memory_target on smithi172 to 8544M 2024-04-26T01:45:33.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:33 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:33.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:33 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:45:33.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:33 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:45:33.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:33 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:33.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:33 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:45:33.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:33 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:45:33.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:33 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:45:33.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:33 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:33.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:33 smithi151 ceph-mon[28057]: osdmap e41: 6 total, 6 up, 6 in 2024-04-26T01:45:33.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:33 smithi151 ceph-mon[28057]: Detected new or changed devices on smithi172 2024-04-26T01:45:33.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:33 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:33.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:33 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:33.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:33 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:45:33.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:33 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:45:33.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:33 smithi151 ceph-mon[28057]: Adjusting osd_memory_target on smithi172 to 8544M 2024-04-26T01:45:33.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:33 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:33.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:33 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:45:33.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:33 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:45:33.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:33 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:33.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:33 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:45:33.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:33 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:45:33.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:33 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:45:33.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:33 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:34.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:34 smithi151 ceph-mon[28057]: osdmap e42: 6 total, 6 up, 6 in 2024-04-26T01:45:34.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:34 smithi151 ceph-mon[28057]: pgmap v156: 1 pgs: 1 remapped+peering; 577 KiB data, 194 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:45:34.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:34 smithi151 ceph-mon[28057]: osdmap e43: 6 total, 6 up, 6 in 2024-04-26T01:45:34.535 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:34 smithi172 ceph-mon[32785]: osdmap e42: 6 total, 6 up, 6 in 2024-04-26T01:45:34.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:34 smithi172 ceph-mon[32785]: pgmap v156: 1 pgs: 1 remapped+peering; 577 KiB data, 194 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:45:34.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:34 smithi172 ceph-mon[32785]: osdmap e43: 6 total, 6 up, 6 in 2024-04-26T01:45:34.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:34 smithi165 ceph-mon[32648]: osdmap e42: 6 total, 6 up, 6 in 2024-04-26T01:45:34.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:34 smithi165 ceph-mon[32648]: pgmap v156: 1 pgs: 1 remapped+peering; 577 KiB data, 194 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:45:34.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:34 smithi165 ceph-mon[32648]: osdmap e43: 6 total, 6 up, 6 in 2024-04-26T01:45:35.690 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:45:35.691+0000 7f496e600700 1 -- 172.21.15.151:0/1622688371 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49681019a0 msgr2=0x7f4968101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:35.690 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:45:35.691+0000 7f496e600700 1 --2- 172.21.15.151:0/1622688371 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49681019a0 0x7f4968101dc0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f49580099c0 tx=0x7f4958009cd0 comp rx=0 tx=0).stop 2024-04-26T01:45:35.690 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:45:35.691+0000 7f496e600700 1 -- 172.21.15.151:0/1622688371 shutdown_connections 2024-04-26T01:45:35.690 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:45:35.691+0000 7f496e600700 1 --2- 172.21.15.151:0/1622688371 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4968103540 0x7f4968109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:35.690 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:45:35.691+0000 7f496e600700 1 --2- 172.21.15.151:0/1622688371 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49681019a0 0x7f4968101dc0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:35.690 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:45:35.691+0000 7f496e600700 1 --2- 172.21.15.151:0/1622688371 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4968102ba0 0x7f4968103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:35.690 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:45:35.691+0000 7f496e600700 1 -- 172.21.15.151:0/1622688371 >> 172.21.15.151:0/1622688371 conn(0x7f49680fcf40 msgr2=0x7f49680ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:45:35.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:45:35.692+0000 7f496e600700 1 -- 172.21.15.151:0/1622688371 shutdown_connections 2024-04-26T01:45:35.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:45:35.692+0000 7f496e600700 1 -- 172.21.15.151:0/1622688371 wait complete. 2024-04-26T01:45:35.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:45:35.693+0000 7f496e600700 1 Processor -- start 2024-04-26T01:45:35.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:45:35.693+0000 7f496e600700 1 -- start start 2024-04-26T01:45:35.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:45:35.694+0000 7f496e600700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49681019a0 0x7f4968198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:35.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:45:35.694+0000 7f496e600700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4968102ba0 0x7f4968199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:35.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:45:35.694+0000 7f496e600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4968103540 0x7f49681a0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:35.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:45:35.694+0000 7f496e600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f496810d480 con 0x7f4968103540 2024-04-26T01:45:35.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:45:35.694+0000 7f496e600700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f496810d300 con 0x7f4968102ba0 2024-04-26T01:45:35.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:45:35.694+0000 7f496e600700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f496810d600 con 0x7f49681019a0 2024-04-26T01:45:35.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:45:35.694+0000 7f4967400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49681019a0 0x7f4968198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:35.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:45:35.694+0000 7f4967400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49681019a0 0x7f4968198d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:43188/0 (socket says 172.21.15.151:43188) 2024-04-26T01:45:35.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:45:35.694+0000 7f4967400700 1 -- 172.21.15.151:0/4038080947 learned_addr learned my addr 172.21.15.151:0/4038080947 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:45:35.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:45:35.694+0000 7f4967e00700 1 --2- 172.21.15.151:0/4038080947 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4968103540 0x7f49681a0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:35.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:45:35.694+0000 7f4966a00700 1 --2- 172.21.15.151:0/4038080947 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4968102ba0 0x7f4968199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:35.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:45:35.694+0000 7f4967400700 1 -- 172.21.15.151:0/4038080947 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4968102ba0 msgr2=0x7f4968199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:35.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:45:35.694+0000 7f4967400700 1 --2- 172.21.15.151:0/4038080947 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4968102ba0 0x7f4968199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:35.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:45:35.694+0000 7f4967400700 1 -- 172.21.15.151:0/4038080947 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4968103540 msgr2=0x7f49681a0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:35.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:45:35.694+0000 7f4967400700 1 --2- 172.21.15.151:0/4038080947 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4968103540 0x7f49681a0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:35.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:45:35.694+0000 7f4967400700 1 -- 172.21.15.151:0/4038080947 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49681a0940 con 0x7f49681019a0 2024-04-26T01:45:35.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:45:35.695+0000 7f4967400700 1 --2- 172.21.15.151:0/4038080947 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49681019a0 0x7f4968198d40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f495800b580 tx=0x7f4958003af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:35.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:45:35.695+0000 7f495fe00700 1 -- 172.21.15.151:0/4038080947 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4958009e60 con 0x7f49681019a0 2024-04-26T01:45:35.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:45:35.695+0000 7f495fe00700 1 -- 172.21.15.151:0/4038080947 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4958005570 con 0x7f49681019a0 2024-04-26T01:45:35.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:45:35.695+0000 7f496e600700 1 -- 172.21.15.151:0/4038080947 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4968103dc0 con 0x7f49681019a0 2024-04-26T01:45:35.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:45:35.695+0000 7f495fe00700 1 -- 172.21.15.151:0/4038080947 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495800fd90 con 0x7f49681019a0 2024-04-26T01:45:35.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:45:35.695+0000 7f496e600700 1 -- 172.21.15.151:0/4038080947 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49681042a0 con 0x7f49681019a0 2024-04-26T01:45:35.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:45:35.696+0000 7f496e600700 1 -- 172.21.15.151:0/4038080947 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4968069730 con 0x7f49681019a0 2024-04-26T01:45:35.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:45:35.698+0000 7f495fe00700 1 -- 172.21.15.151:0/4038080947 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f495800f4f0 con 0x7f49681019a0 2024-04-26T01:45:35.697 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:45:35.700+0000 7f495fe00700 1 --2- 172.21.15.151:0/4038080947 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f494c0a3c40 0x7f494c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:35.697 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:45:35.700+0000 7f495fe00700 1 -- 172.21.15.151:0/4038080947 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 4533+0+0 (secure 0 0 0) 0x7f49580d0560 con 0x7f49681019a0 2024-04-26T01:45:35.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:45:35.700+0000 7f4966a00700 1 --2- 172.21.15.151:0/4038080947 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f494c0a3c40 0x7f494c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:35.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:45:35.701+0000 7f4966a00700 1 --2- 172.21.15.151:0/4038080947 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f494c0a3c40 0x7f494c0a6100 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4954010d10 tx=0x7f4954009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:35.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:45:35.702+0000 7f495fe00700 1 -- 172.21.15.151:0/4038080947 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4958013070 con 0x7f49681019a0 2024-04-26T01:45:35.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:45:35.939+0000 7f496e600700 1 -- 172.21.15.151:0/4038080947 --> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f496806aea0 con 0x7f494c0a3c40 2024-04-26T01:45:36.120 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:35 smithi151 ceph-mon[28057]: osdmap e44: 6 total, 6 up, 6 in 2024-04-26T01:45:36.120 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:35 smithi151 ceph-mon[28057]: pgmap v159: 1 pgs: 1 remapped+peering; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:45:36.285 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:35 smithi172 ceph-mon[32785]: osdmap e44: 6 total, 6 up, 6 in 2024-04-26T01:45:36.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:35 smithi172 ceph-mon[32785]: pgmap v159: 1 pgs: 1 remapped+peering; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:45:36.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:35 smithi165 ceph-mon[32648]: osdmap e44: 6 total, 6 up, 6 in 2024-04-26T01:45:36.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:35 smithi165 ceph-mon[32648]: pgmap v159: 1 pgs: 1 remapped+peering; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:45:37.106 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:36 smithi151 ceph-mon[28057]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:45:37.106 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:36 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:45:37.106 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:36 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:45:37.107 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:36 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:45:37.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:36 smithi172 ceph-mon[32785]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:45:37.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:36 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:45:37.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:36 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:45:37.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:36 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:45:37.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:36 smithi165 ceph-mon[32648]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:45:37.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:36 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:45:37.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:36 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:45:37.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:36 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:45:38.054 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:37 smithi151 ceph-mon[28057]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-04-26T01:45:38.285 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:37 smithi172 ceph-mon[32785]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-04-26T01:45:38.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:37 smithi165 ceph-mon[32648]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-04-26T01:45:39.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:39 smithi151 ceph-mon[28057]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-04-26T01:45:39.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:39 smithi172 ceph-mon[32785]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-04-26T01:45:39.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:39 smithi165 ceph-mon[32648]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-04-26T01:45:40.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:40 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/3844629652' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4290baaa-6160-49cb-a4e0-c96f070f2ef5"}]: dispatch 2024-04-26T01:45:40.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:40 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/3844629652' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4290baaa-6160-49cb-a4e0-c96f070f2ef5"}]': finished 2024-04-26T01:45:40.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:40 smithi151 ceph-mon[28057]: osdmap e45: 7 total, 6 up, 7 in 2024-04-26T01:45:40.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:40 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:45:40.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:40 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/3844629652' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4290baaa-6160-49cb-a4e0-c96f070f2ef5"}]: dispatch 2024-04-26T01:45:40.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:40 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/3844629652' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4290baaa-6160-49cb-a4e0-c96f070f2ef5"}]': finished 2024-04-26T01:45:40.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:40 smithi172 ceph-mon[32785]: osdmap e45: 7 total, 6 up, 7 in 2024-04-26T01:45:40.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:40 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:45:40.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:40 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/3844629652' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4290baaa-6160-49cb-a4e0-c96f070f2ef5"}]: dispatch 2024-04-26T01:45:40.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:40 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/3844629652' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4290baaa-6160-49cb-a4e0-c96f070f2ef5"}]': finished 2024-04-26T01:45:40.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:40 smithi165 ceph-mon[32648]: osdmap e45: 7 total, 6 up, 7 in 2024-04-26T01:45:40.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:40 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:45:41.754 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:41 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/608909796' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:45:41.754 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:41 smithi172 ceph-mon[32785]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-04-26T01:45:41.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:41 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/608909796' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:45:41.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:41 smithi165 ceph-mon[32648]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-04-26T01:45:41.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:41 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/608909796' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:45:41.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:41 smithi151 ceph-mon[28057]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-04-26T01:45:43.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:43 smithi151 ceph-mon[28057]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-04-26T01:45:43.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:43 smithi172 ceph-mon[32785]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-04-26T01:45:43.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:43 smithi165 ceph-mon[32648]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-04-26T01:45:45.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:45 smithi151 ceph-mon[28057]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T01:45:45.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:45 smithi172 ceph-mon[32785]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T01:45:45.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:45 smithi165 ceph-mon[32648]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T01:45:47.485 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:47 smithi151 ceph-mon[28057]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:45:47.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:47 smithi172 ceph-mon[32785]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:45:47.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:47 smithi165 ceph-mon[32648]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:45:49.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:49 smithi151 ceph-mon[28057]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:45:49.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:49 smithi172 ceph-mon[32785]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:45:49.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:49 smithi165 ceph-mon[32648]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:45:50.651 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:50 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-26T01:45:50.651 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:50 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:45:50.652 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:50 smithi151 ceph-mon[28057]: Deploying daemon osd.6 on smithi151 2024-04-26T01:45:50.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:50 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-26T01:45:50.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:50 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:45:50.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:50 smithi172 ceph-mon[32785]: Deploying daemon osd.6 on smithi151 2024-04-26T01:45:50.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:50 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-26T01:45:50.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:50 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:45:50.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:50 smithi165 ceph-mon[32648]: Deploying daemon osd.6 on smithi151 2024-04-26T01:45:51.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:51 smithi151 ceph-mon[28057]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:45:51.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:51 smithi172 ceph-mon[32785]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:45:51.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:51 smithi165 ceph-mon[32648]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:45:53.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:53 smithi151 ceph-mon[28057]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:45:53.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:53 smithi172 ceph-mon[32785]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:45:53.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:53 smithi165 ceph-mon[32648]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:45:55.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:55 smithi151 ceph-mon[28057]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:45:55.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:55 smithi172 ceph-mon[32785]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:45:55.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:55 smithi165 ceph-mon[32648]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:45:57.590 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:57 smithi151 ceph-mon[28057]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:45:57.647 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:57 smithi165 ceph-mon[32648]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:45:57.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:57 smithi172 ceph-mon[32785]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:45:59.370 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:59 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:59.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:59 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:59.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:45:59 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:45:59.535 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:59 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:59.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:59 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:59.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:45:59 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:45:59.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:59 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:59.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:59 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:45:59.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:45:59 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:46:00.535 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:00 smithi172 ceph-mon[32785]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:46:00.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:00 smithi165 ceph-mon[32648]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:46:00.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:00 smithi151 ceph-mon[28057]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:46:01.412 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:01 smithi151 ceph-mon[28057]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:46:01.713 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:01 smithi172 ceph-mon[32785]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:46:01.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:01 smithi165 ceph-mon[32648]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:46:02.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:46:02.987+0000 7f495fe00700 1 -- 172.21.15.151:0/4038080947 <== mgr.14152 v2:172.21.15.151:6800/2251256505 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f496806aea0 con 0x7f494c0a3c40 2024-04-26T01:46:02.986 INFO:teuthology.orchestra.run.smithi151.stdout:Created osd(s) 6 on host 'smithi151' 2024-04-26T01:46:02.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:46:02.991+0000 7f496e600700 1 -- 172.21.15.151:0/4038080947 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f494c0a3c40 msgr2=0x7f494c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:02.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:46:02.991+0000 7f496e600700 1 --2- 172.21.15.151:0/4038080947 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f494c0a3c40 0x7f494c0a6100 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4954010d10 tx=0x7f4954009260 comp rx=0 tx=0).stop 2024-04-26T01:46:02.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:46:02.991+0000 7f496e600700 1 -- 172.21.15.151:0/4038080947 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49681019a0 msgr2=0x7f4968198d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:02.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:46:02.991+0000 7f496e600700 1 --2- 172.21.15.151:0/4038080947 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49681019a0 0x7f4968198d40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f495800b580 tx=0x7f4958003af0 comp rx=0 tx=0).stop 2024-04-26T01:46:02.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:46:02.991+0000 7f496e600700 1 -- 172.21.15.151:0/4038080947 shutdown_connections 2024-04-26T01:46:02.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:46:02.991+0000 7f496e600700 1 --2- 172.21.15.151:0/4038080947 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f494c0a3c40 0x7f494c0a6100 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:02.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:46:02.991+0000 7f496e600700 1 --2- 172.21.15.151:0/4038080947 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4968103540 0x7f49681a0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:02.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:46:02.991+0000 7f496e600700 1 --2- 172.21.15.151:0/4038080947 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49681019a0 0x7f4968198d40 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:02.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:46:02.991+0000 7f496e600700 1 --2- 172.21.15.151:0/4038080947 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4968102ba0 0x7f4968199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:02.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:46:02.991+0000 7f496e600700 1 -- 172.21.15.151:0/4038080947 >> 172.21.15.151:0/4038080947 conn(0x7f49680fcf40 msgr2=0x7f496806a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:46:02.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:46:02.992+0000 7f496e600700 1 -- 172.21.15.151:0/4038080947 shutdown_connections 2024-04-26T01:46:02.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:46:02.992+0000 7f496e600700 1 -- 172.21.15.151:0/4038080947 wait complete. 2024-04-26T01:46:03.035 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:02 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:03.035 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:02 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:03.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:02 smithi172 ceph-mon[32785]: from='osd.6 [v2:172.21.15.151:6818/1723252315,v1:172.21.15.151:6819/1723252315]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-26T01:46:03.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:02 smithi172 ceph-mon[32785]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-26T01:46:03.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:02 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:03.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:02 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:03.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:02 smithi165 ceph-mon[32648]: from='osd.6 [v2:172.21.15.151:6818/1723252315,v1:172.21.15.151:6819/1723252315]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-26T01:46:03.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:02 smithi165 ceph-mon[32648]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-26T01:46:03.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:02 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:03.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:02 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:03.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:02 smithi151 ceph-mon[28057]: from='osd.6 [v2:172.21.15.151:6818/1723252315,v1:172.21.15.151:6819/1723252315]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-26T01:46:03.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:02 smithi151 ceph-mon[28057]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-26T01:46:03.800 DEBUG:teuthology.orchestra.run.smithi151:osd.6> sudo journalctl -f -n 0 -u ceph-c0e49e64-036d-11ef-bc93-c7b262605968@osd.6.service 2024-04-26T01:46:03.804 INFO:tasks.cephadm:Deploying osd.7 on smithi165 with /dev/vg_nvme/lv_2... 2024-04-26T01:46:03.804 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-26T01:46:04.050 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T01:46:04.051 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:03 smithi151 ceph-mon[28057]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-26T01:46:04.051 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:03 smithi151 ceph-mon[28057]: from='osd.6 [v2:172.21.15.151:6818/1723252315,v1:172.21.15.151:6819/1723252315]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-04-26T01:46:04.051 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:03 smithi151 ceph-mon[28057]: osdmap e46: 7 total, 6 up, 7 in 2024-04-26T01:46:04.052 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:03 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:46:04.052 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:03 smithi151 ceph-mon[28057]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-04-26T01:46:04.052 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:03 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:04.052 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:03 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:04.052 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:03 smithi151 ceph-mon[28057]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:46:04.052 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 01:46:03 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T01:46:03.881+0000 7f5f58200700 -1 osd.6 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T01:46:04.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:03 smithi165 ceph-mon[32648]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-26T01:46:04.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:03 smithi165 ceph-mon[32648]: from='osd.6 [v2:172.21.15.151:6818/1723252315,v1:172.21.15.151:6819/1723252315]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-04-26T01:46:04.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:03 smithi165 ceph-mon[32648]: osdmap e46: 7 total, 6 up, 7 in 2024-04-26T01:46:04.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:03 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:46:04.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:03 smithi165 ceph-mon[32648]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-04-26T01:46:04.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:03 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:04.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:03 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:04.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:03 smithi165 ceph-mon[32648]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:46:04.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:03 smithi172 ceph-mon[32785]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-26T01:46:04.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:03 smithi172 ceph-mon[32785]: from='osd.6 [v2:172.21.15.151:6818/1723252315,v1:172.21.15.151:6819/1723252315]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-04-26T01:46:04.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:03 smithi172 ceph-mon[32785]: osdmap e46: 7 total, 6 up, 7 in 2024-04-26T01:46:04.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:03 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:46:04.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:03 smithi172 ceph-mon[32785]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-04-26T01:46:04.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:03 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:04.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:03 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:04.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:03 smithi172 ceph-mon[32785]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:46:05.091 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:04 smithi151 ceph-mon[28057]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-04-26T01:46:05.092 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:04 smithi151 ceph-mon[28057]: osdmap e47: 7 total, 6 up, 7 in 2024-04-26T01:46:05.092 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:04 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:46:05.092 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:04 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:46:05.135 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:04 smithi165 ceph-mon[32648]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-04-26T01:46:05.135 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:04 smithi165 ceph-mon[32648]: osdmap e47: 7 total, 6 up, 7 in 2024-04-26T01:46:05.135 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:04 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:46:05.135 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:04 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:46:05.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:04 smithi172 ceph-mon[32785]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-04-26T01:46:05.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:04 smithi172 ceph-mon[32785]: osdmap e47: 7 total, 6 up, 7 in 2024-04-26T01:46:05.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:04 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:46:05.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:04 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:46:05.829 INFO:teuthology.orchestra.run.smithi165.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-26T01:46:05.829 INFO:teuthology.orchestra.run.smithi165.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-26T01:46:05.829 INFO:teuthology.orchestra.run.smithi165.stderr: stderr: 10+0 records in 2024-04-26T01:46:05.829 INFO:teuthology.orchestra.run.smithi165.stderr:10+0 records out 2024-04-26T01:46:05.829 INFO:teuthology.orchestra.run.smithi165.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113958 s, 920 MB/s 2024-04-26T01:46:05.829 INFO:teuthology.orchestra.run.smithi165.stderr:--> Zapping successful for: 2024-04-26T01:46:05.957 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:05 smithi151 ceph-mon[28057]: purged_snaps scrub starts 2024-04-26T01:46:05.957 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:05 smithi151 ceph-mon[28057]: purged_snaps scrub ok 2024-04-26T01:46:05.957 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:05 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:46:05.957 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:05 smithi151 ceph-mon[28057]: osd.6 [v2:172.21.15.151:6818/1723252315,v1:172.21.15.151:6819/1723252315] boot 2024-04-26T01:46:05.957 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:05 smithi151 ceph-mon[28057]: osdmap e48: 7 total, 7 up, 7 in 2024-04-26T01:46:05.957 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:05 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:46:05.957 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:05 smithi151 ceph-mon[28057]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:46:06.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:05 smithi165 ceph-mon[32648]: purged_snaps scrub starts 2024-04-26T01:46:06.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:05 smithi165 ceph-mon[32648]: purged_snaps scrub ok 2024-04-26T01:46:06.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:05 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:46:06.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:05 smithi165 ceph-mon[32648]: osd.6 [v2:172.21.15.151:6818/1723252315,v1:172.21.15.151:6819/1723252315] boot 2024-04-26T01:46:06.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:05 smithi165 ceph-mon[32648]: osdmap e48: 7 total, 7 up, 7 in 2024-04-26T01:46:06.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:05 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:46:06.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:05 smithi165 ceph-mon[32648]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:46:06.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:05 smithi172 ceph-mon[32785]: purged_snaps scrub starts 2024-04-26T01:46:06.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:05 smithi172 ceph-mon[32785]: purged_snaps scrub ok 2024-04-26T01:46:06.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:05 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:46:06.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:05 smithi172 ceph-mon[32785]: osd.6 [v2:172.21.15.151:6818/1723252315,v1:172.21.15.151:6819/1723252315] boot 2024-04-26T01:46:06.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:05 smithi172 ceph-mon[32785]: osdmap e48: 7 total, 7 up, 7 in 2024-04-26T01:46:06.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:05 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:46:06.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:05 smithi172 ceph-mon[32785]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:46:06.420 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi165:vg_nvme/lv_2 2024-04-26T01:46:06.663 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T01:46:06.970 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:06 smithi165 ceph-mon[32648]: osdmap e49: 7 total, 7 up, 7 in 2024-04-26T01:46:07.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:06 smithi172 ceph-mon[32785]: osdmap e49: 7 total, 7 up, 7 in 2024-04-26T01:46:07.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:06 smithi151 ceph-mon[28057]: osdmap e49: 7 total, 7 up, 7 in 2024-04-26T01:46:07.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:46:07.989+0000 7fde26564700 1 -- 172.21.15.165:0/593817434 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fde20102ba0 msgr2=0x7fde20103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:07.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:46:07.989+0000 7fde26564700 1 --2- 172.21.15.165:0/593817434 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fde20102ba0 0x7fde20103000 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fde140099c0 tx=0x7fde14009cd0 comp rx=0 tx=0).stop 2024-04-26T01:46:07.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:46:07.989+0000 7fde26564700 1 -- 172.21.15.165:0/593817434 shutdown_connections 2024-04-26T01:46:07.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:46:07.989+0000 7fde26564700 1 --2- 172.21.15.165:0/593817434 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde20103540 0x7fde20109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:07.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:46:07.989+0000 7fde26564700 1 --2- 172.21.15.165:0/593817434 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde201019a0 0x7fde20101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:07.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:46:07.989+0000 7fde26564700 1 --2- 172.21.15.165:0/593817434 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fde20102ba0 0x7fde20103000 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:07.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:46:07.989+0000 7fde26564700 1 -- 172.21.15.165:0/593817434 >> 172.21.15.165:0/593817434 conn(0x7fde200fcf40 msgr2=0x7fde200ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:46:07.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:46:07.989+0000 7fde26564700 1 -- 172.21.15.165:0/593817434 shutdown_connections 2024-04-26T01:46:07.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:46:07.989+0000 7fde26564700 1 -- 172.21.15.165:0/593817434 wait complete. 2024-04-26T01:46:07.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:46:07.990+0000 7fde26564700 1 Processor -- start 2024-04-26T01:46:07.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:46:07.990+0000 7fde26564700 1 -- start start 2024-04-26T01:46:07.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:46:07.990+0000 7fde26564700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde201019a0 0x7fde20105930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:07.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:46:07.990+0000 7fde26564700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fde20102ba0 0x7fde20103f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:07.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:46:07.990+0000 7fde26564700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde20103540 0x7fde201044c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:07.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:46:07.990+0000 7fde26564700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fde2010d530 con 0x7fde20103540 2024-04-26T01:46:07.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:46:07.990+0000 7fde26564700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fde2010d3b0 con 0x7fde20102ba0 2024-04-26T01:46:07.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:46:07.990+0000 7fde26564700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fde2010d6b0 con 0x7fde201019a0 2024-04-26T01:46:07.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:46:07.990+0000 7fde1f7fe700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fde20102ba0 0x7fde20103f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:07.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:46:07.990+0000 7fde1f7fe700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fde20102ba0 0x7fde20103f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:47902/0 (socket says 172.21.15.165:47902) 2024-04-26T01:46:07.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:46:07.990+0000 7fde1f7fe700 1 -- 172.21.15.165:0/1098847537 learned_addr learned my addr 172.21.15.165:0/1098847537 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-26T01:46:07.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:46:07.991+0000 7fde1ffff700 1 --2- 172.21.15.165:0/1098847537 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde201019a0 0x7fde20105930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:07.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:46:07.991+0000 7fde1f7fe700 1 -- 172.21.15.165:0/1098847537 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde201019a0 msgr2=0x7fde20105930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:07.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:46:07.991+0000 7fde1f7fe700 1 --2- 172.21.15.165:0/1098847537 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde201019a0 0x7fde20105930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:07.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:46:07.991+0000 7fde1f7fe700 1 -- 172.21.15.165:0/1098847537 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde20103540 msgr2=0x7fde201044c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:07.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:46:07.991+0000 7fde1f7fe700 1 --2- 172.21.15.165:0/1098847537 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde20103540 0x7fde201044c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:07.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:46:07.991+0000 7fde1f7fe700 1 -- 172.21.15.165:0/1098847537 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde20104d70 con 0x7fde20102ba0 2024-04-26T01:46:07.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:46:07.991+0000 7fde1f7fe700 1 --2- 172.21.15.165:0/1098847537 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fde20102ba0 0x7fde20103f80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fde14003790 tx=0x7fde14003a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:46:07.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:46:07.991+0000 7fde1d7fa700 1 -- 172.21.15.165:0/1098847537 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde14027070 con 0x7fde20102ba0 2024-04-26T01:46:07.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:46:07.991+0000 7fde26564700 1 -- 172.21.15.165:0/1098847537 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde201a2e10 con 0x7fde20102ba0 2024-04-26T01:46:07.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:46:07.991+0000 7fde1d7fa700 1 -- 172.21.15.165:0/1098847537 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde14003f80 con 0x7fde20102ba0 2024-04-26T01:46:07.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:46:07.991+0000 7fde1d7fa700 1 -- 172.21.15.165:0/1098847537 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1400f460 con 0x7fde20102ba0 2024-04-26T01:46:07.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:46:07.991+0000 7fde26564700 1 -- 172.21.15.165:0/1098847537 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde201a32e0 con 0x7fde20102ba0 2024-04-26T01:46:07.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:46:07.992+0000 7fde26564700 1 -- 172.21.15.165:0/1098847537 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde201a3510 con 0x7fde20102ba0 2024-04-26T01:46:07.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:46:07.994+0000 7fde1d7fa700 1 -- 172.21.15.165:0/1098847537 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fde1402a070 con 0x7fde20102ba0 2024-04-26T01:46:07.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:46:07.994+0000 7fde1d7fa700 1 --2- 172.21.15.165:0/1098847537 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fde080a3c40 0x7fde080a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:07.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:46:07.995+0000 7fde1d7fa700 1 -- 172.21.15.165:0/1098847537 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4965+0+0 (secure 0 0 0) 0x7fde140d0b40 con 0x7fde20102ba0 2024-04-26T01:46:07.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:46:07.995+0000 7fde1ffff700 1 --2- 172.21.15.165:0/1098847537 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fde080a3c40 0x7fde080a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:07.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:46:07.996+0000 7fde1ffff700 1 --2- 172.21.15.165:0/1098847537 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fde080a3c40 0x7fde080a6100 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fde10006fd0 tx=0x7fde10008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:46:07.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:46:07.998+0000 7fde1d7fa700 1 -- 172.21.15.165:0/1098847537 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fde14013070 con 0x7fde20102ba0 2024-04-26T01:46:08.064 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:07 smithi151 ceph-mon[28057]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:46:08.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:07 smithi165 ceph-mon[32648]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:46:08.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:46:08.206+0000 7fde26564700 1 -- 172.21.15.165:0/1098847537 --> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fde200ff170 con 0x7fde080a3c40 2024-04-26T01:46:08.285 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:07 smithi172 ceph-mon[32785]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:46:09.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:08 smithi165 ceph-mon[32648]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:46:09.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:08 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:46:09.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:08 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:46:09.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:08 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:09.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:08 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:09.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:08 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:09.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:08 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:46:09.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:08 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:46:09.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:08 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:46:09.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:08 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:09.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:08 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:09.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:08 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:46:09.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:08 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:09.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:08 smithi172 ceph-mon[32785]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:46:09.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:08 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:46:09.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:08 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:46:09.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:08 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:09.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:08 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:09.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:08 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:09.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:08 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:46:09.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:08 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:46:09.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:08 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:46:09.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:08 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:09.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:08 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:09.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:08 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:46:09.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:08 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:09.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:08 smithi151 ceph-mon[28057]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:46:09.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:08 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:46:09.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:08 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:46:09.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:08 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:09.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:08 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:09.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:08 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:09.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:08 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:46:09.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:08 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:46:09.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:08 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:46:09.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:08 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:09.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:08 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:09.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:08 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:46:09.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:08 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:10.060 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:09 smithi165 ceph-mon[32648]: Detected new or changed devices on smithi151 2024-04-26T01:46:10.060 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:09 smithi165 ceph-mon[32648]: Adjusting osd_memory_target on smithi151 to 5696M 2024-04-26T01:46:10.060 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:09 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:46:10.060 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:09 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:10.060 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:09 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:46:10.060 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:09 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:10.060 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:09 smithi165 ceph-mon[32648]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:46:10.285 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:09 smithi172 ceph-mon[32785]: Detected new or changed devices on smithi151 2024-04-26T01:46:10.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:09 smithi172 ceph-mon[32785]: Adjusting osd_memory_target on smithi151 to 5696M 2024-04-26T01:46:10.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:09 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:46:10.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:09 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:10.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:09 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:46:10.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:09 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:10.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:09 smithi172 ceph-mon[32785]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:46:10.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:09 smithi151 ceph-mon[28057]: Detected new or changed devices on smithi151 2024-04-26T01:46:10.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:09 smithi151 ceph-mon[28057]: Adjusting osd_memory_target on smithi151 to 5696M 2024-04-26T01:46:10.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:09 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:46:10.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:09 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:10.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:09 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:46:10.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:09 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:10.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:09 smithi151 ceph-mon[28057]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:46:11.230 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:10 smithi165 ceph-mon[32648]: from='client.? 172.21.15.165:0/784800096' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f5315f2-036c-4e2f-92ed-3321a9e3c860"}]: dispatch 2024-04-26T01:46:11.230 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:10 smithi165 ceph-mon[32648]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f5315f2-036c-4e2f-92ed-3321a9e3c860"}]: dispatch 2024-04-26T01:46:11.230 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:10 smithi165 ceph-mon[32648]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9f5315f2-036c-4e2f-92ed-3321a9e3c860"}]': finished 2024-04-26T01:46:11.231 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:10 smithi165 ceph-mon[32648]: osdmap e50: 8 total, 7 up, 8 in 2024-04-26T01:46:11.231 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:10 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T01:46:11.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:10 smithi172 ceph-mon[32785]: from='client.? 172.21.15.165:0/784800096' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f5315f2-036c-4e2f-92ed-3321a9e3c860"}]: dispatch 2024-04-26T01:46:11.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:10 smithi172 ceph-mon[32785]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f5315f2-036c-4e2f-92ed-3321a9e3c860"}]: dispatch 2024-04-26T01:46:11.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:10 smithi172 ceph-mon[32785]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9f5315f2-036c-4e2f-92ed-3321a9e3c860"}]': finished 2024-04-26T01:46:11.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:10 smithi172 ceph-mon[32785]: osdmap e50: 8 total, 7 up, 8 in 2024-04-26T01:46:11.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:10 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T01:46:11.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:10 smithi151 ceph-mon[28057]: from='client.? 172.21.15.165:0/784800096' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f5315f2-036c-4e2f-92ed-3321a9e3c860"}]: dispatch 2024-04-26T01:46:11.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:10 smithi151 ceph-mon[28057]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f5315f2-036c-4e2f-92ed-3321a9e3c860"}]: dispatch 2024-04-26T01:46:11.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:10 smithi151 ceph-mon[28057]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9f5315f2-036c-4e2f-92ed-3321a9e3c860"}]': finished 2024-04-26T01:46:11.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:10 smithi151 ceph-mon[28057]: osdmap e50: 8 total, 7 up, 8 in 2024-04-26T01:46:11.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:10 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T01:46:12.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:11 smithi172 ceph-mon[32785]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:46:12.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:11 smithi172 ceph-mon[32785]: from='client.? 172.21.15.165:0/1891914397' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:46:12.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:11 smithi165 ceph-mon[32648]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:46:12.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:11 smithi165 ceph-mon[32648]: from='client.? 172.21.15.165:0/1891914397' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:46:12.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:11 smithi151 ceph-mon[28057]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:46:12.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:11 smithi151 ceph-mon[28057]: from='client.? 172.21.15.165:0/1891914397' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:46:13.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:13 smithi172 ceph-mon[32785]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:46:13.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:13 smithi165 ceph-mon[32648]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:46:13.870 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:13 smithi151 ceph-mon[28057]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:46:15.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:15 smithi165 ceph-mon[32648]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:46:15.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:15 smithi172 ceph-mon[32785]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:46:15.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:15 smithi151 ceph-mon[28057]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:46:17.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:17 smithi165 ceph-mon[32648]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:46:17.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:17 smithi172 ceph-mon[32785]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:46:17.870 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:17 smithi151 ceph-mon[28057]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:46:18.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:18 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-26T01:46:18.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:18 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:18.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:18 smithi172 ceph-mon[32785]: Deploying daemon osd.7 on smithi165 2024-04-26T01:46:18.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:18 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-26T01:46:18.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:18 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:18.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:18 smithi165 ceph-mon[32648]: Deploying daemon osd.7 on smithi165 2024-04-26T01:46:18.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:18 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-26T01:46:18.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:18 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:18.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:18 smithi151 ceph-mon[28057]: Deploying daemon osd.7 on smithi165 2024-04-26T01:46:19.723 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:19 smithi165 ceph-mon[32648]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:46:19.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:19 smithi172 ceph-mon[32785]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:46:19.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:19 smithi151 ceph-mon[28057]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:46:21.693 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:21 smithi165 ceph-mon[32648]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:46:21.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:21 smithi172 ceph-mon[32785]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:46:21.870 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:21 smithi151 ceph-mon[28057]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:46:23.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:23 smithi165 ceph-mon[32648]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:46:23.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:23 smithi172 ceph-mon[32785]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:46:23.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:23 smithi151 ceph-mon[28057]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:46:24.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:24 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:24.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:24 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:24.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:24 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:46:24.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:24 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:24.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:24 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:24.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:24 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:46:24.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:24 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:24.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:24 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:24.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:24 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:46:25.714 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:25 smithi165 ceph-mon[32648]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:46:25.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:25 smithi172 ceph-mon[32785]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:46:25.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:25 smithi151 ceph-mon[28057]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:46:27.042 INFO:teuthology.orchestra.run.smithi165.stdout:Created osd(s) 7 on host 'smithi165' 2024-04-26T01:46:27.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:46:27.044+0000 7fde1d7fa700 1 -- 172.21.15.165:0/1098847537 <== mgr.14152 v2:172.21.15.151:6800/2251256505 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fde200ff170 con 0x7fde080a3c40 2024-04-26T01:46:27.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:46:27.047+0000 7fde26564700 1 -- 172.21.15.165:0/1098847537 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fde080a3c40 msgr2=0x7fde080a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:27.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:46:27.047+0000 7fde26564700 1 --2- 172.21.15.165:0/1098847537 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fde080a3c40 0x7fde080a6100 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fde10006fd0 tx=0x7fde10008040 comp rx=0 tx=0).stop 2024-04-26T01:46:27.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:46:27.047+0000 7fde26564700 1 -- 172.21.15.165:0/1098847537 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fde20102ba0 msgr2=0x7fde20103f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:27.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:46:27.047+0000 7fde26564700 1 --2- 172.21.15.165:0/1098847537 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fde20102ba0 0x7fde20103f80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fde14003790 tx=0x7fde14003a10 comp rx=0 tx=0).stop 2024-04-26T01:46:27.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:46:27.047+0000 7fde26564700 1 -- 172.21.15.165:0/1098847537 shutdown_connections 2024-04-26T01:46:27.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:46:27.047+0000 7fde26564700 1 --2- 172.21.15.165:0/1098847537 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fde080a3c40 0x7fde080a6100 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:27.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:46:27.047+0000 7fde26564700 1 --2- 172.21.15.165:0/1098847537 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde20103540 0x7fde201044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:27.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:46:27.047+0000 7fde26564700 1 --2- 172.21.15.165:0/1098847537 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde201019a0 0x7fde20105930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:27.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:46:27.047+0000 7fde26564700 1 --2- 172.21.15.165:0/1098847537 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fde20102ba0 0x7fde20103f80 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:27.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:46:27.047+0000 7fde26564700 1 -- 172.21.15.165:0/1098847537 >> 172.21.15.165:0/1098847537 conn(0x7fde200fcf40 msgr2=0x7fde200fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:46:27.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:46:27.047+0000 7fde26564700 1 -- 172.21.15.165:0/1098847537 shutdown_connections 2024-04-26T01:46:27.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:46:27.047+0000 7fde26564700 1 -- 172.21.15.165:0/1098847537 wait complete. 2024-04-26T01:46:27.646 DEBUG:teuthology.orchestra.run.smithi165:osd.7> sudo journalctl -f -n 0 -u ceph-c0e49e64-036d-11ef-bc93-c7b262605968@osd.7.service 2024-04-26T01:46:27.649 INFO:tasks.cephadm:Deploying osd.8 on smithi172 with /dev/vg_nvme/lv_2... 2024-04-26T01:46:27.649 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-26T01:46:27.897 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T01:46:28.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:28 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:28.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:28 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:28.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:28 smithi172 ceph-mon[32785]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:46:28.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:28 smithi172 ceph-mon[32785]: from='osd.7 [v2:172.21.15.165:6816/1977803781,v1:172.21.15.165:6817/1977803781]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T01:46:28.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:28 smithi172 ceph-mon[32785]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T01:46:28.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:28 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:28.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:28 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:28.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:28 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:28.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:28 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:28.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:28 smithi165 ceph-mon[32648]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:46:28.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:28 smithi165 ceph-mon[32648]: from='osd.7 [v2:172.21.15.165:6816/1977803781,v1:172.21.15.165:6817/1977803781]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T01:46:28.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:28 smithi165 ceph-mon[32648]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T01:46:28.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:28 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:28.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:28 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:28.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:28 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:28.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:28 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:28.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:28 smithi151 ceph-mon[28057]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:46:28.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:28 smithi151 ceph-mon[28057]: from='osd.7 [v2:172.21.15.165:6816/1977803781,v1:172.21.15.165:6817/1977803781]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T01:46:28.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:28 smithi151 ceph-mon[28057]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T01:46:28.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:28 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:28.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:28 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:29.282 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:29 smithi172 ceph-mon[32785]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-26T01:46:29.282 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:29 smithi172 ceph-mon[32785]: from='osd.7 [v2:172.21.15.165:6816/1977803781,v1:172.21.15.165:6817/1977803781]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-26T01:46:29.282 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:29 smithi172 ceph-mon[32785]: osdmap e51: 8 total, 7 up, 8 in 2024-04-26T01:46:29.282 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:29 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T01:46:29.283 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:29 smithi172 ceph-mon[32785]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-26T01:46:29.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:29 smithi165 ceph-mon[32648]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-26T01:46:29.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:29 smithi165 ceph-mon[32648]: from='osd.7 [v2:172.21.15.165:6816/1977803781,v1:172.21.15.165:6817/1977803781]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-26T01:46:29.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:29 smithi165 ceph-mon[32648]: osdmap e51: 8 total, 7 up, 8 in 2024-04-26T01:46:29.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:29 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T01:46:29.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:29 smithi165 ceph-mon[32648]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-26T01:46:29.345 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 01:46:29 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T01:46:29.032+0000 7fb40b8c1700 -1 osd.7 0 waiting for initial osdmap 2024-04-26T01:46:29.346 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 01:46:29 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T01:46:29.043+0000 7fb4076b4700 -1 osd.7 52 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T01:46:29.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:29 smithi151 ceph-mon[28057]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-26T01:46:29.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:29 smithi151 ceph-mon[28057]: from='osd.7 [v2:172.21.15.165:6816/1977803781,v1:172.21.15.165:6817/1977803781]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-26T01:46:29.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:29 smithi151 ceph-mon[28057]: osdmap e51: 8 total, 7 up, 8 in 2024-04-26T01:46:29.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:29 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T01:46:29.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:29 smithi151 ceph-mon[28057]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-26T01:46:29.864 INFO:teuthology.orchestra.run.smithi172.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-26T01:46:29.864 INFO:teuthology.orchestra.run.smithi172.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-26T01:46:29.864 INFO:teuthology.orchestra.run.smithi172.stderr: stderr: 10+0 records in 2024-04-26T01:46:29.864 INFO:teuthology.orchestra.run.smithi172.stderr:10+0 records out 2024-04-26T01:46:29.864 INFO:teuthology.orchestra.run.smithi172.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113455 s, 924 MB/s 2024-04-26T01:46:29.864 INFO:teuthology.orchestra.run.smithi172.stderr:--> Zapping successful for: 2024-04-26T01:46:30.185 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:30 smithi172 ceph-mon[32785]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-04-26T01:46:30.185 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:30 smithi172 ceph-mon[32785]: osdmap e52: 8 total, 7 up, 8 in 2024-04-26T01:46:30.185 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:30 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T01:46:30.186 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:30 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T01:46:30.186 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:30 smithi172 ceph-mon[32785]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:46:30.186 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:30 smithi172 ceph-mon[32785]: osd.7 [v2:172.21.15.165:6816/1977803781,v1:172.21.15.165:6817/1977803781] boot 2024-04-26T01:46:30.186 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:30 smithi172 ceph-mon[32785]: osdmap e53: 8 total, 8 up, 8 in 2024-04-26T01:46:30.297 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:30 smithi165 ceph-mon[32648]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-04-26T01:46:30.297 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:30 smithi165 ceph-mon[32648]: osdmap e52: 8 total, 7 up, 8 in 2024-04-26T01:46:30.297 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:30 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T01:46:30.297 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:30 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T01:46:30.298 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:30 smithi165 ceph-mon[32648]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:46:30.298 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:30 smithi165 ceph-mon[32648]: osd.7 [v2:172.21.15.165:6816/1977803781,v1:172.21.15.165:6817/1977803781] boot 2024-04-26T01:46:30.298 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:30 smithi165 ceph-mon[32648]: osdmap e53: 8 total, 8 up, 8 in 2024-04-26T01:46:30.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:30 smithi151 ceph-mon[28057]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-04-26T01:46:30.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:30 smithi151 ceph-mon[28057]: osdmap e52: 8 total, 7 up, 8 in 2024-04-26T01:46:30.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:30 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T01:46:30.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:30 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T01:46:30.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:30 smithi151 ceph-mon[28057]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:46:30.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:30 smithi151 ceph-mon[28057]: osd.7 [v2:172.21.15.165:6816/1977803781,v1:172.21.15.165:6817/1977803781] boot 2024-04-26T01:46:30.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:30 smithi151 ceph-mon[28057]: osdmap e53: 8 total, 8 up, 8 in 2024-04-26T01:46:30.498 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi172:vg_nvme/lv_2 2024-04-26T01:46:30.738 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T01:46:31.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:31 smithi165 ceph-mon[32648]: purged_snaps scrub starts 2024-04-26T01:46:31.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:31 smithi165 ceph-mon[32648]: purged_snaps scrub ok 2024-04-26T01:46:31.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:31 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T01:46:31.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:31 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:31.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:31 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:31.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:31 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:46:31.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:31 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:46:31.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:31 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:46:31.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:31 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:31.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:31 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:31.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:31 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:46:31.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:31 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:31.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:31 smithi165 ceph-mon[32648]: osdmap e54: 8 total, 8 up, 8 in 2024-04-26T01:46:31.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:31 smithi151 ceph-mon[28057]: purged_snaps scrub starts 2024-04-26T01:46:31.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:31 smithi151 ceph-mon[28057]: purged_snaps scrub ok 2024-04-26T01:46:31.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:31 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T01:46:31.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:31 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:31.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:31 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:31.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:31 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:46:31.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:31 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:46:31.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:31 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:46:31.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:31 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:31.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:31 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:31.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:31 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:46:31.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:31 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:31.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:31 smithi151 ceph-mon[28057]: osdmap e54: 8 total, 8 up, 8 in 2024-04-26T01:46:31.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:31 smithi172 ceph-mon[32785]: purged_snaps scrub starts 2024-04-26T01:46:31.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:31 smithi172 ceph-mon[32785]: purged_snaps scrub ok 2024-04-26T01:46:31.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:31 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T01:46:31.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:31 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:31.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:31 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:31.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:31 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:46:31.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:31 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:46:31.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:31 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:46:31.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:31 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:31.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:31 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:31.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:31 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:46:31.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:31 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:31.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:31 smithi172 ceph-mon[32785]: osdmap e54: 8 total, 8 up, 8 in 2024-04-26T01:46:32.099 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:32 smithi172 ceph-mon[32785]: Detected new or changed devices on smithi165 2024-04-26T01:46:32.099 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:32 smithi172 ceph-mon[32785]: Adjusting osd_memory_target on smithi165 to 5696M 2024-04-26T01:46:32.099 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:32 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:46:32.099 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:32 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:32.099 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:32 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:46:32.099 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:32 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:32.099 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:32 smithi172 ceph-mon[32785]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:46:32.099 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:32 smithi172 ceph-mon[32785]: osdmap e55: 8 total, 8 up, 8 in 2024-04-26T01:46:32.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:46:32.323+0000 7fc30697f700 1 -- 172.21.15.172:0/2578545448 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc300076a70 msgr2=0x7fc300074ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:32.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:46:32.323+0000 7fc30697f700 1 --2- 172.21.15.172:0/2578545448 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc300076a70 0x7fc300074ed0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0009990 tx=0x7fc2f0009ca0 comp rx=0 tx=0).stop 2024-04-26T01:46:32.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:46:32.323+0000 7fc30697f700 1 -- 172.21.15.172:0/2578545448 shutdown_connections 2024-04-26T01:46:32.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:46:32.323+0000 7fc30697f700 1 --2- 172.21.15.172:0/2578545448 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc30010f080 0x7fc300111470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:32.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:46:32.323+0000 7fc30697f700 1 --2- 172.21.15.172:0/2578545448 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc300076a70 0x7fc300074ed0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:32.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:46:32.323+0000 7fc30697f700 1 --2- 172.21.15.172:0/2578545448 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc3000754a0 0x7fc300075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:32.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:46:32.323+0000 7fc30697f700 1 -- 172.21.15.172:0/2578545448 >> 172.21.15.172:0/2578545448 conn(0x7fc3000fe040 msgr2=0x7fc300100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:46:32.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:46:32.323+0000 7fc30697f700 1 -- 172.21.15.172:0/2578545448 shutdown_connections 2024-04-26T01:46:32.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:46:32.323+0000 7fc30697f700 1 -- 172.21.15.172:0/2578545448 wait complete. 2024-04-26T01:46:32.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:46:32.323+0000 7fc30697f700 1 Processor -- start 2024-04-26T01:46:32.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:46:32.323+0000 7fc30697f700 1 -- start start 2024-04-26T01:46:32.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:46:32.324+0000 7fc30697f700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc3000754a0 0x7fc30019f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:32.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:46:32.324+0000 7fc30697f700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc300076a70 0x7fc30019f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:32.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:46:32.324+0000 7fc30697f700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc30010f080 0x7fc3001a69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:32.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:46:32.324+0000 7fc30697f700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc300114c80 con 0x7fc300076a70 2024-04-26T01:46:32.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:46:32.324+0000 7fc30697f700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc300114b00 con 0x7fc30010f080 2024-04-26T01:46:32.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:46:32.324+0000 7fc30697f700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc300114e00 con 0x7fc3000754a0 2024-04-26T01:46:32.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:46:32.324+0000 7fc2fffff700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc3000754a0 0x7fc30019f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:32.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:46:32.324+0000 7fc2fffff700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc3000754a0 0x7fc30019f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49068/0 (socket says 172.21.15.172:49068) 2024-04-26T01:46:32.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:46:32.324+0000 7fc2fffff700 1 -- 172.21.15.172:0/3168138429 learned_addr learned my addr 172.21.15.172:0/3168138429 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-26T01:46:32.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:46:32.324+0000 7fc2fffff700 1 -- 172.21.15.172:0/3168138429 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc30010f080 msgr2=0x7fc3001a69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:32.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:46:32.324+0000 7fc304f1c700 1 --2- 172.21.15.172:0/3168138429 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc30010f080 0x7fc3001a69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:32.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:46:32.324+0000 7fc2fffff700 1 --2- 172.21.15.172:0/3168138429 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc30010f080 0x7fc3001a69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:32.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:46:32.324+0000 7fc2fffff700 1 -- 172.21.15.172:0/3168138429 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc300076a70 msgr2=0x7fc30019f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:32.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:46:32.324+0000 7fc2fffff700 1 --2- 172.21.15.172:0/3168138429 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc300076a70 0x7fc30019f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:32.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:46:32.324+0000 7fc2fffff700 1 -- 172.21.15.172:0/3168138429 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3001a7040 con 0x7fc3000754a0 2024-04-26T01:46:32.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:46:32.324+0000 7fc304f1c700 1 --2- 172.21.15.172:0/3168138429 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc30010f080 0x7fc3001a69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T01:46:32.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:46:32.324+0000 7fc2fffff700 1 --2- 172.21.15.172:0/3168138429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc3000754a0 0x7fc30019f3e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc2f00047b0 tx=0x7fc2f0009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:46:32.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:46:32.325+0000 7fc2fd7fa700 1 -- 172.21.15.172:0/3168138429 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f0027070 con 0x7fc3000754a0 2024-04-26T01:46:32.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:46:32.325+0000 7fc30697f700 1 -- 172.21.15.172:0/3168138429 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3001a72d0 con 0x7fc3000754a0 2024-04-26T01:46:32.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:46:32.325+0000 7fc2fd7fa700 1 -- 172.21.15.172:0/3168138429 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2f0022470 con 0x7fc3000754a0 2024-04-26T01:46:32.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:46:32.325+0000 7fc2fd7fa700 1 -- 172.21.15.172:0/3168138429 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f000f6c0 con 0x7fc3000754a0 2024-04-26T01:46:32.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:46:32.325+0000 7fc30697f700 1 -- 172.21.15.172:0/3168138429 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3001a7800 con 0x7fc3000754a0 2024-04-26T01:46:32.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:46:32.326+0000 7fc30697f700 1 -- 172.21.15.172:0/3168138429 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc300069730 con 0x7fc3000754a0 2024-04-26T01:46:32.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:46:32.327+0000 7fc2fd7fa700 1 -- 172.21.15.172:0/3168138429 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fc2f002a070 con 0x7fc3000754a0 2024-04-26T01:46:32.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:46:32.327+0000 7fc2fd7fa700 1 --2- 172.21.15.172:0/3168138429 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fc2e80a3c40 0x7fc2e80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:32.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:46:32.327+0000 7fc2fd7fa700 1 -- 172.21.15.172:0/3168138429 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5426+0+0 (secure 0 0 0) 0x7fc2f00d0780 con 0x7fc3000754a0 2024-04-26T01:46:32.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:46:32.328+0000 7fc2ff7fe700 1 --2- 172.21.15.172:0/3168138429 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fc2e80a3c40 0x7fc2e80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:32.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:46:32.329+0000 7fc2ff7fe700 1 --2- 172.21.15.172:0/3168138429 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fc2e80a3c40 0x7fc2e80a6100 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc3000767a0 tx=0x7fc2f4006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:46:32.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:46:32.332+0000 7fc2fd7fa700 1 -- 172.21.15.172:0/3168138429 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc2f0013070 con 0x7fc3000754a0 2024-04-26T01:46:32.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:32 smithi165 ceph-mon[32648]: Detected new or changed devices on smithi165 2024-04-26T01:46:32.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:32 smithi165 ceph-mon[32648]: Adjusting osd_memory_target on smithi165 to 5696M 2024-04-26T01:46:32.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:32 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:46:32.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:32 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:32.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:32 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:46:32.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:32 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:32.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:32 smithi165 ceph-mon[32648]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:46:32.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:32 smithi165 ceph-mon[32648]: osdmap e55: 8 total, 8 up, 8 in 2024-04-26T01:46:32.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:32 smithi151 ceph-mon[28057]: Detected new or changed devices on smithi165 2024-04-26T01:46:32.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:32 smithi151 ceph-mon[28057]: Adjusting osd_memory_target on smithi165 to 5696M 2024-04-26T01:46:32.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:32 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:46:32.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:32 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:32.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:32 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:46:32.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:32 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:32.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:32 smithi151 ceph-mon[28057]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:46:32.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:32 smithi151 ceph-mon[28057]: osdmap e55: 8 total, 8 up, 8 in 2024-04-26T01:46:32.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:46:32.539+0000 7fc30697f700 1 -- 172.21.15.172:0/3168138429 --> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fc300062e80 con 0x7fc2e80a3c40 2024-04-26T01:46:33.469 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:33 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:46:33.469 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:33 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:46:33.469 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:33 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:33.469 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:33 smithi172 ceph-mon[32785]: osdmap e56: 8 total, 8 up, 8 in 2024-04-26T01:46:33.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:33 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:46:33.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:33 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:46:33.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:33 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:33.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:33 smithi165 ceph-mon[32648]: osdmap e56: 8 total, 8 up, 8 in 2024-04-26T01:46:33.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:33 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:46:33.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:33 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:46:33.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:33 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:33.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:33 smithi151 ceph-mon[28057]: osdmap e56: 8 total, 8 up, 8 in 2024-04-26T01:46:34.479 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:34 smithi172 ceph-mon[32785]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:46:34.479 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:34 smithi172 ceph-mon[32785]: pgmap v200: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:46:34.479 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:34 smithi172 ceph-mon[32785]: osdmap e57: 8 total, 8 up, 8 in 2024-04-26T01:46:34.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:34 smithi165 ceph-mon[32648]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:46:34.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:34 smithi165 ceph-mon[32648]: pgmap v200: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:46:34.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:34 smithi165 ceph-mon[32648]: osdmap e57: 8 total, 8 up, 8 in 2024-04-26T01:46:34.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:34 smithi151 ceph-mon[28057]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:46:34.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:34 smithi151 ceph-mon[28057]: pgmap v200: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:46:34.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:34 smithi151 ceph-mon[28057]: osdmap e57: 8 total, 8 up, 8 in 2024-04-26T01:46:35.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:35 smithi151 ceph-mon[28057]: from='client.? 172.21.15.172:0/473112675' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ddbe36d-cea7-45ef-a3ca-044a0e64ff85"}]: dispatch 2024-04-26T01:46:35.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:35 smithi151 ceph-mon[28057]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ddbe36d-cea7-45ef-a3ca-044a0e64ff85"}]: dispatch 2024-04-26T01:46:35.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:35 smithi151 ceph-mon[28057]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8ddbe36d-cea7-45ef-a3ca-044a0e64ff85"}]': finished 2024-04-26T01:46:35.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:35 smithi151 ceph-mon[28057]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:46:35.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:35 smithi151 ceph-mon[28057]: osdmap e58: 9 total, 8 up, 9 in 2024-04-26T01:46:35.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:35 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T01:46:35.670 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:35 smithi172 ceph-mon[32785]: from='client.? 172.21.15.172:0/473112675' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ddbe36d-cea7-45ef-a3ca-044a0e64ff85"}]: dispatch 2024-04-26T01:46:35.670 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:35 smithi172 ceph-mon[32785]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ddbe36d-cea7-45ef-a3ca-044a0e64ff85"}]: dispatch 2024-04-26T01:46:35.671 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:35 smithi172 ceph-mon[32785]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8ddbe36d-cea7-45ef-a3ca-044a0e64ff85"}]': finished 2024-04-26T01:46:35.671 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:35 smithi172 ceph-mon[32785]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:46:35.671 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:35 smithi172 ceph-mon[32785]: osdmap e58: 9 total, 8 up, 9 in 2024-04-26T01:46:35.671 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:35 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T01:46:35.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:35 smithi165 ceph-mon[32648]: from='client.? 172.21.15.172:0/473112675' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ddbe36d-cea7-45ef-a3ca-044a0e64ff85"}]: dispatch 2024-04-26T01:46:35.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:35 smithi165 ceph-mon[32648]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ddbe36d-cea7-45ef-a3ca-044a0e64ff85"}]: dispatch 2024-04-26T01:46:35.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:35 smithi165 ceph-mon[32648]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8ddbe36d-cea7-45ef-a3ca-044a0e64ff85"}]': finished 2024-04-26T01:46:35.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:35 smithi165 ceph-mon[32648]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:46:35.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:35 smithi165 ceph-mon[32648]: osdmap e58: 9 total, 8 up, 9 in 2024-04-26T01:46:35.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:35 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T01:46:36.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:36 smithi172 ceph-mon[32785]: from='client.? 172.21.15.172:0/1243073452' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:46:36.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:36 smithi165 ceph-mon[32648]: from='client.? 172.21.15.172:0/1243073452' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:46:36.870 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:36 smithi151 ceph-mon[28057]: from='client.? 172.21.15.172:0/1243073452' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:46:37.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:37 smithi172 ceph-mon[32785]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:46:37.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:37 smithi165 ceph-mon[32648]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:46:37.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:37 smithi151 ceph-mon[28057]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:46:39.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:39 smithi172 ceph-mon[32785]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:46:39.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:39 smithi165 ceph-mon[32648]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:46:39.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:39 smithi151 ceph-mon[28057]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:46:41.700 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:41 smithi172 ceph-mon[32785]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:46:41.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:41 smithi165 ceph-mon[32648]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:46:41.870 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:41 smithi151 ceph-mon[28057]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:46:43.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:43 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-26T01:46:43.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:43 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:43.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:43 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-26T01:46:43.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:43 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:43.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:43 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-26T01:46:43.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:43 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:44.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:44 smithi172 ceph-mon[32785]: Deploying daemon osd.8 on smithi172 2024-04-26T01:46:44.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:44 smithi172 ceph-mon[32785]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:46:44.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:44 smithi165 ceph-mon[32648]: Deploying daemon osd.8 on smithi172 2024-04-26T01:46:44.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:44 smithi165 ceph-mon[32648]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:46:44.370 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:44 smithi151 ceph-mon[28057]: Deploying daemon osd.8 on smithi172 2024-04-26T01:46:44.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:44 smithi151 ceph-mon[28057]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:46:45.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:45 smithi172 ceph-mon[32785]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:46:45.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:45 smithi165 ceph-mon[32648]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:46:45.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:45 smithi151 ceph-mon[28057]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:46:47.688 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:47 smithi172 ceph-mon[32785]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:46:47.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:47 smithi165 ceph-mon[32648]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:46:47.870 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:47 smithi151 ceph-mon[28057]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:46:49.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:49 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:49.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:49 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:46:49.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:49 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:49.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:49 smithi165 ceph-mon[32648]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:46:49.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:49 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:49.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:49 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:46:49.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:49 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:49.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:49 smithi151 ceph-mon[28057]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:46:50.032 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:49 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:50.032 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:49 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:46:50.032 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:49 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:50.032 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:49 smithi172 ceph-mon[32785]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:46:51.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:51 smithi172 ceph-mon[32785]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:46:51.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:51 smithi165 ceph-mon[32648]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:46:51.870 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:51 smithi151 ceph-mon[28057]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:46:52.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:46:52.524+0000 7fc2fd7fa700 1 -- 172.21.15.172:0/3168138429 <== mgr.14152 v2:172.21.15.151:6800/2251256505 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc300062e80 con 0x7fc2e80a3c40 2024-04-26T01:46:52.522 INFO:teuthology.orchestra.run.smithi172.stdout:Created osd(s) 8 on host 'smithi172' 2024-04-26T01:46:52.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:46:52.526+0000 7fc30697f700 1 -- 172.21.15.172:0/3168138429 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fc2e80a3c40 msgr2=0x7fc2e80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:52.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:46:52.526+0000 7fc30697f700 1 --2- 172.21.15.172:0/3168138429 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fc2e80a3c40 0x7fc2e80a6100 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc3000767a0 tx=0x7fc2f4006d20 comp rx=0 tx=0).stop 2024-04-26T01:46:52.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:46:52.526+0000 7fc30697f700 1 -- 172.21.15.172:0/3168138429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc3000754a0 msgr2=0x7fc30019f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:52.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:46:52.526+0000 7fc30697f700 1 --2- 172.21.15.172:0/3168138429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc3000754a0 0x7fc30019f3e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc2f00047b0 tx=0x7fc2f0009f90 comp rx=0 tx=0).stop 2024-04-26T01:46:52.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:46:52.526+0000 7fc30697f700 1 -- 172.21.15.172:0/3168138429 shutdown_connections 2024-04-26T01:46:52.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:46:52.526+0000 7fc30697f700 1 --2- 172.21.15.172:0/3168138429 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc30010f080 0x7fc3001a69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:52.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:46:52.526+0000 7fc30697f700 1 --2- 172.21.15.172:0/3168138429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc3000754a0 0x7fc30019f3e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:52.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:46:52.526+0000 7fc30697f700 1 --2- 172.21.15.172:0/3168138429 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fc2e80a3c40 0x7fc2e80a6100 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:52.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:46:52.526+0000 7fc30697f700 1 --2- 172.21.15.172:0/3168138429 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc300076a70 0x7fc30019f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:52.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:46:52.526+0000 7fc30697f700 1 -- 172.21.15.172:0/3168138429 >> 172.21.15.172:0/3168138429 conn(0x7fc3000fe040 msgr2=0x7fc300100430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:46:52.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:46:52.526+0000 7fc30697f700 1 -- 172.21.15.172:0/3168138429 shutdown_connections 2024-04-26T01:46:52.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:46:52.526+0000 7fc30697f700 1 -- 172.21.15.172:0/3168138429 wait complete. 2024-04-26T01:46:52.785 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:52 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:52.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:52 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:52.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:52 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:52.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:52 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:46:52.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:52 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:52.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:52 smithi172 ceph-mon[32785]: from='osd.8 [v2:172.21.15.172:6816/491199512,v1:172.21.15.172:6817/491199512]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T01:46:52.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:52 smithi172 ceph-mon[32785]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T01:46:52.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:52 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:52.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:52 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:52.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:52 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:46:53.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:52 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:53.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:52 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:53.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:52 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:53.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:52 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:46:53.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:52 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:53.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:52 smithi165 ceph-mon[32648]: from='osd.8 [v2:172.21.15.172:6816/491199512,v1:172.21.15.172:6817/491199512]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T01:46:53.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:52 smithi165 ceph-mon[32648]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T01:46:53.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:52 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:53.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:52 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:53.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:52 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:46:53.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:52 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:53.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:52 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:53.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:52 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:53.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:52 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:46:53.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:52 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:53.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:52 smithi151 ceph-mon[28057]: from='osd.8 [v2:172.21.15.172:6816/491199512,v1:172.21.15.172:6817/491199512]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T01:46:53.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:52 smithi151 ceph-mon[28057]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T01:46:53.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:52 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:53.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:52 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:53.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:52 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:46:53.181 DEBUG:teuthology.orchestra.run.smithi172:osd.8> sudo journalctl -f -n 0 -u ceph-c0e49e64-036d-11ef-bc93-c7b262605968@osd.8.service 2024-04-26T01:46:53.183 INFO:tasks.cephadm:Deploying osd.9 on smithi151 with /dev/vg_nvme/lv_1... 2024-04-26T01:46:53.184 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-26T01:46:53.630 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:46:53.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:53 smithi172 ceph-mon[32785]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-26T01:46:53.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:53 smithi172 ceph-mon[32785]: from='osd.8 [v2:172.21.15.172:6816/491199512,v1:172.21.15.172:6817/491199512]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-04-26T01:46:53.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:53 smithi172 ceph-mon[32785]: osdmap e59: 9 total, 8 up, 9 in 2024-04-26T01:46:53.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:53 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T01:46:53.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:53 smithi172 ceph-mon[32785]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-04-26T01:46:53.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:53 smithi172 ceph-mon[32785]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:46:53.786 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 01:46:53 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T01:46:53.705+0000 7f86800f0700 -1 osd.8 0 waiting for initial osdmap 2024-04-26T01:46:53.786 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 01:46:53 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T01:46:53.721+0000 7f867bee3700 -1 osd.8 60 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T01:46:53.975 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:53 smithi151 ceph-mon[28057]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-26T01:46:53.975 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:53 smithi151 ceph-mon[28057]: from='osd.8 [v2:172.21.15.172:6816/491199512,v1:172.21.15.172:6817/491199512]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-04-26T01:46:53.975 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:53 smithi151 ceph-mon[28057]: osdmap e59: 9 total, 8 up, 9 in 2024-04-26T01:46:53.975 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:53 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T01:46:53.975 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:53 smithi151 ceph-mon[28057]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-04-26T01:46:53.975 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:53 smithi151 ceph-mon[28057]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:46:54.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:53 smithi165 ceph-mon[32648]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-26T01:46:54.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:53 smithi165 ceph-mon[32648]: from='osd.8 [v2:172.21.15.172:6816/491199512,v1:172.21.15.172:6817/491199512]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-04-26T01:46:54.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:53 smithi165 ceph-mon[32648]: osdmap e59: 9 total, 8 up, 9 in 2024-04-26T01:46:54.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:53 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T01:46:54.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:53 smithi165 ceph-mon[32648]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-04-26T01:46:54.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:53 smithi165 ceph-mon[32648]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:46:54.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:54 smithi172 ceph-mon[32785]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-04-26T01:46:54.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:54 smithi172 ceph-mon[32785]: osdmap e60: 9 total, 8 up, 9 in 2024-04-26T01:46:54.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:54 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T01:46:54.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:54 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T01:46:54.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:54 smithi172 ceph-mon[32785]: osd.8 [v2:172.21.15.172:6816/491199512,v1:172.21.15.172:6817/491199512] boot 2024-04-26T01:46:54.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:54 smithi172 ceph-mon[32785]: osdmap e61: 9 total, 9 up, 9 in 2024-04-26T01:46:54.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:54 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T01:46:54.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:54 smithi151 ceph-mon[28057]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-04-26T01:46:54.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:54 smithi151 ceph-mon[28057]: osdmap e60: 9 total, 8 up, 9 in 2024-04-26T01:46:54.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:54 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T01:46:54.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:54 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T01:46:54.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:54 smithi151 ceph-mon[28057]: osd.8 [v2:172.21.15.172:6816/491199512,v1:172.21.15.172:6817/491199512] boot 2024-04-26T01:46:54.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:54 smithi151 ceph-mon[28057]: osdmap e61: 9 total, 9 up, 9 in 2024-04-26T01:46:54.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:54 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T01:46:55.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:54 smithi165 ceph-mon[32648]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-04-26T01:46:55.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:54 smithi165 ceph-mon[32648]: osdmap e60: 9 total, 8 up, 9 in 2024-04-26T01:46:55.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:54 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T01:46:55.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:54 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T01:46:55.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:54 smithi165 ceph-mon[32648]: osd.8 [v2:172.21.15.172:6816/491199512,v1:172.21.15.172:6817/491199512] boot 2024-04-26T01:46:55.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:54 smithi165 ceph-mon[32648]: osdmap e61: 9 total, 9 up, 9 in 2024-04-26T01:46:55.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:54 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T01:46:56.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:55 smithi172 ceph-mon[32785]: purged_snaps scrub starts 2024-04-26T01:46:56.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:55 smithi172 ceph-mon[32785]: purged_snaps scrub ok 2024-04-26T01:46:56.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:55 smithi172 ceph-mon[32785]: osdmap e62: 9 total, 9 up, 9 in 2024-04-26T01:46:56.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:55 smithi172 ceph-mon[32785]: pgmap v217: 1 pgs: 1 remapped+peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:46:56.161 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:55 smithi151 ceph-mon[28057]: purged_snaps scrub starts 2024-04-26T01:46:56.161 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:55 smithi151 ceph-mon[28057]: purged_snaps scrub ok 2024-04-26T01:46:56.162 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:55 smithi151 ceph-mon[28057]: osdmap e62: 9 total, 9 up, 9 in 2024-04-26T01:46:56.162 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:55 smithi151 ceph-mon[28057]: pgmap v217: 1 pgs: 1 remapped+peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:46:56.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:55 smithi165 ceph-mon[32648]: purged_snaps scrub starts 2024-04-26T01:46:56.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:55 smithi165 ceph-mon[32648]: purged_snaps scrub ok 2024-04-26T01:46:56.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:55 smithi165 ceph-mon[32648]: osdmap e62: 9 total, 9 up, 9 in 2024-04-26T01:46:56.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:55 smithi165 ceph-mon[32648]: pgmap v217: 1 pgs: 1 remapped+peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:46:56.636 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-26T01:46:56.636 INFO:teuthology.orchestra.run.smithi151.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-26T01:46:56.636 INFO:teuthology.orchestra.run.smithi151.stderr: stderr: 10+0 records in 2024-04-26T01:46:56.636 INFO:teuthology.orchestra.run.smithi151.stderr:10+0 records out 2024-04-26T01:46:56.636 INFO:teuthology.orchestra.run.smithi151.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0601552 s, 174 MB/s 2024-04-26T01:46:56.636 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping successful for: 2024-04-26T01:46:57.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:56 smithi151 ceph-mon[28057]: osdmap e63: 9 total, 9 up, 9 in 2024-04-26T01:46:57.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:56 smithi151 ceph-mon[28057]: Detected new or changed devices on smithi172 2024-04-26T01:46:57.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:56 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:57.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:56 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:57.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:56 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:46:57.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:56 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:46:57.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:56 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:46:57.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:56 smithi151 ceph-mon[28057]: Adjusting osd_memory_target on smithi172 to 5696M 2024-04-26T01:46:57.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:56 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:57.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:56 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:57.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:56 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:46:57.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:56 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:57.246 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:56 smithi165 ceph-mon[32648]: osdmap e63: 9 total, 9 up, 9 in 2024-04-26T01:46:57.247 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:56 smithi165 ceph-mon[32648]: Detected new or changed devices on smithi172 2024-04-26T01:46:57.247 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:56 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:57.247 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:56 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:57.247 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:56 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:46:57.247 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:56 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:46:57.247 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:56 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:46:57.247 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:56 smithi165 ceph-mon[32648]: Adjusting osd_memory_target on smithi172 to 5696M 2024-04-26T01:46:57.247 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:56 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:57.247 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:56 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:57.247 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:56 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:46:57.247 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:56 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:57.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:56 smithi172 ceph-mon[32785]: osdmap e63: 9 total, 9 up, 9 in 2024-04-26T01:46:57.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:56 smithi172 ceph-mon[32785]: Detected new or changed devices on smithi172 2024-04-26T01:46:57.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:56 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:57.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:56 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:57.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:56 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:46:57.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:56 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:46:57.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:56 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:46:57.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:56 smithi172 ceph-mon[32785]: Adjusting osd_memory_target on smithi172 to 5696M 2024-04-26T01:46:57.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:56 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:57.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:56 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:57.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:56 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:46:57.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:56 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:46:57.547 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi151:vg_nvme/lv_1 2024-04-26T01:46:57.979 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:46:58.268 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:57 smithi151 ceph-mon[28057]: osdmap e64: 9 total, 9 up, 9 in 2024-04-26T01:46:58.268 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:57 smithi151 ceph-mon[28057]: pgmap v220: 1 pgs: 1 remapped+peering; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:46:58.268 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:57 smithi151 ceph-mon[28057]: osdmap e65: 9 total, 9 up, 9 in 2024-04-26T01:46:58.285 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:57 smithi172 ceph-mon[32785]: osdmap e64: 9 total, 9 up, 9 in 2024-04-26T01:46:58.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:57 smithi172 ceph-mon[32785]: pgmap v220: 1 pgs: 1 remapped+peering; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:46:58.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:57 smithi172 ceph-mon[32785]: osdmap e65: 9 total, 9 up, 9 in 2024-04-26T01:46:58.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:57 smithi165 ceph-mon[32648]: osdmap e64: 9 total, 9 up, 9 in 2024-04-26T01:46:58.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:57 smithi165 ceph-mon[32648]: pgmap v220: 1 pgs: 1 remapped+peering; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:46:58.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:57 smithi165 ceph-mon[32648]: osdmap e65: 9 total, 9 up, 9 in 2024-04-26T01:46:59.729 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:46:59 smithi151 ceph-mon[28057]: pgmap v222: 1 pgs: 1 remapped+peering; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:46:59.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:46:59 smithi172 ceph-mon[32785]: pgmap v222: 1 pgs: 1 remapped+peering; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:46:59.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:46:59 smithi165 ceph-mon[32648]: pgmap v222: 1 pgs: 1 remapped+peering; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:47:00.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:47:00.265+0000 7f4c2ba00700 1 -- 172.21.15.151:0/723195017 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4c24101320 msgr2=0x7f4c24101740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:00.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:47:00.265+0000 7f4c2ba00700 1 --2- 172.21.15.151:0/723195017 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4c24101320 0x7f4c24101740 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4c18009990 tx=0x7f4c18009ca0 comp rx=0 tx=0).stop 2024-04-26T01:47:00.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:47:00.265+0000 7f4c2ba00700 1 -- 172.21.15.151:0/723195017 shutdown_connections 2024-04-26T01:47:00.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:47:00.265+0000 7f4c2ba00700 1 --2- 172.21.15.151:0/723195017 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c24102640 0x7f4c24102ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:00.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:47:00.265+0000 7f4c2ba00700 1 --2- 172.21.15.151:0/723195017 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4c24101320 0x7f4c24101740 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:00.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:47:00.265+0000 7f4c2ba00700 1 --2- 172.21.15.151:0/723195017 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c24101c80 0x7f4c24102100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:00.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:47:00.265+0000 7f4c2ba00700 1 -- 172.21.15.151:0/723195017 >> 172.21.15.151:0/723195017 conn(0x7f4c240fcf40 msgr2=0x7f4c240ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:47:00.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:47:00.265+0000 7f4c2ba00700 1 -- 172.21.15.151:0/723195017 shutdown_connections 2024-04-26T01:47:00.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:47:00.266+0000 7f4c2ba00700 1 -- 172.21.15.151:0/723195017 wait complete. 2024-04-26T01:47:00.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:47:00.267+0000 7f4c2ba00700 1 Processor -- start 2024-04-26T01:47:00.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:47:00.267+0000 7f4c2ba00700 1 -- start start 2024-04-26T01:47:00.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:47:00.267+0000 7f4c2ba00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c24101320 0x7f4c2419aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:00.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:47:00.267+0000 7f4c2ba00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c24101c80 0x7f4c2419b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:00.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:47:00.267+0000 7f4c2ba00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4c24102640 0x7f4c241a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:00.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:47:00.267+0000 7f4c2ba00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4c24110740 con 0x7f4c24102640 2024-04-26T01:47:00.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:47:00.267+0000 7f4c2ba00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4c241105c0 con 0x7f4c24101c80 2024-04-26T01:47:00.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:47:00.267+0000 7f4c2ba00700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4c241108c0 con 0x7f4c24101320 2024-04-26T01:47:00.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:47:00.268+0000 7f4c29a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4c24102640 0x7f4c241a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:00.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:47:00.268+0000 7f4c29000700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c24101320 0x7f4c2419aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:00.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:47:00.268+0000 7f4c29a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4c24102640 0x7f4c241a2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:60942/0 (socket says 172.21.15.151:60942) 2024-04-26T01:47:00.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:47:00.268+0000 7f4c23e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c24101c80 0x7f4c2419b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:00.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:47:00.268+0000 7f4c29a00700 1 -- 172.21.15.151:0/782476789 learned_addr learned my addr 172.21.15.151:0/782476789 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:47:00.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:47:00.268+0000 7f4c23e00700 1 -- 172.21.15.151:0/782476789 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c24101320 msgr2=0x7f4c2419aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:00.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:47:00.268+0000 7f4c23e00700 1 --2- 172.21.15.151:0/782476789 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c24101320 0x7f4c2419aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:00.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:47:00.268+0000 7f4c23e00700 1 -- 172.21.15.151:0/782476789 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4c24102640 msgr2=0x7f4c241a2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:00.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:47:00.268+0000 7f4c23e00700 1 --2- 172.21.15.151:0/782476789 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4c24102640 0x7f4c241a2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:00.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:47:00.268+0000 7f4c23e00700 1 -- 172.21.15.151:0/782476789 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c241a2b00 con 0x7f4c24101c80 2024-04-26T01:47:00.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:47:00.268+0000 7f4c29000700 1 --2- 172.21.15.151:0/782476789 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c24101320 0x7f4c2419aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:47:00.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:47:00.268+0000 7f4c29a00700 1 --2- 172.21.15.151:0/782476789 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4c24102640 0x7f4c241a2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:47:00.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:47:00.268+0000 7f4c23e00700 1 --2- 172.21.15.151:0/782476789 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c24101c80 0x7f4c2419b3e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4c1000e9f0 tx=0x7f4c1000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:47:00.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:47:00.269+0000 7f4c21600700 1 -- 172.21.15.151:0/782476789 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c1000c9c0 con 0x7f4c24101c80 2024-04-26T01:47:00.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:47:00.269+0000 7f4c21600700 1 -- 172.21.15.151:0/782476789 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c1000cb60 con 0x7f4c24101c80 2024-04-26T01:47:00.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:47:00.269+0000 7f4c2ba00700 1 -- 172.21.15.151:0/782476789 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c241a2df0 con 0x7f4c24101c80 2024-04-26T01:47:00.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:47:00.269+0000 7f4c21600700 1 -- 172.21.15.151:0/782476789 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c10018680 con 0x7f4c24101c80 2024-04-26T01:47:00.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:47:00.269+0000 7f4c2ba00700 1 -- 172.21.15.151:0/782476789 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c241a32f0 con 0x7f4c24101c80 2024-04-26T01:47:00.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:47:00.270+0000 7f4c2ba00700 1 -- 172.21.15.151:0/782476789 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c24069730 con 0x7f4c24101c80 2024-04-26T01:47:00.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:47:00.272+0000 7f4c21600700 1 -- 172.21.15.151:0/782476789 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f4c10010430 con 0x7f4c24101c80 2024-04-26T01:47:00.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:47:00.274+0000 7f4c21600700 1 --2- 172.21.15.151:0/782476789 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f4c0c0a3cc0 0x7f4c0c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:00.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:47:00.275+0000 7f4c21600700 1 -- 172.21.15.151:0/782476789 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5829+0+0 (secure 0 0 0) 0x7f4c100cfee0 con 0x7f4c24101c80 2024-04-26T01:47:00.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:47:00.275+0000 7f4c29000700 1 --2- 172.21.15.151:0/782476789 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f4c0c0a3cc0 0x7f4c0c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:00.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:47:00.275+0000 7f4c29000700 1 --2- 172.21.15.151:0/782476789 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f4c0c0a3cc0 0x7f4c0c0a6180 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4c1800b760 tx=0x7f4c1800b2a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:47:00.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:47:00.276+0000 7f4c21600700 1 -- 172.21.15.151:0/782476789 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4c1009d1d0 con 0x7f4c24101c80 2024-04-26T01:47:00.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:47:00.512+0000 7f4c2ba00700 1 -- 172.21.15.151:0/782476789 --> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f4c24062e80 con 0x7f4c0c0a3cc0 2024-04-26T01:47:00.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:00 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:47:00.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:00 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:47:00.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:00 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:00.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:00 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:47:00.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:00 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:47:00.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:00 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:01.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:00 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:47:01.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:00 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:47:01.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:00 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:01.761 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:01 smithi151 ceph-mon[28057]: from='client.24323 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:47:01.761 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:01 smithi151 ceph-mon[28057]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T01:47:01.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:01 smithi165 ceph-mon[32648]: from='client.24323 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:47:01.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:01 smithi165 ceph-mon[32648]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T01:47:01.937 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:01 smithi172 ceph-mon[32785]: from='client.24323 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:47:01.937 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:01 smithi172 ceph-mon[32785]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T01:47:03.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:03 smithi172 ceph-mon[32785]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-04-26T01:47:03.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:03 smithi151 ceph-mon[28057]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-04-26T01:47:03.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:03 smithi165 ceph-mon[32648]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-04-26T01:47:04.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:04 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/2185929401' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40654ba0-1bd9-4b54-97ae-f1ed6c4785d3"}]: dispatch 2024-04-26T01:47:04.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:04 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/2185929401' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "40654ba0-1bd9-4b54-97ae-f1ed6c4785d3"}]': finished 2024-04-26T01:47:04.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:04 smithi172 ceph-mon[32785]: osdmap e66: 10 total, 9 up, 10 in 2024-04-26T01:47:04.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:04 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:47:04.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:04 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/2185929401' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40654ba0-1bd9-4b54-97ae-f1ed6c4785d3"}]: dispatch 2024-04-26T01:47:04.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:04 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/2185929401' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "40654ba0-1bd9-4b54-97ae-f1ed6c4785d3"}]': finished 2024-04-26T01:47:04.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:04 smithi165 ceph-mon[32648]: osdmap e66: 10 total, 9 up, 10 in 2024-04-26T01:47:04.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:04 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:47:04.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:04 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/2185929401' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40654ba0-1bd9-4b54-97ae-f1ed6c4785d3"}]: dispatch 2024-04-26T01:47:04.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:04 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/2185929401' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "40654ba0-1bd9-4b54-97ae-f1ed6c4785d3"}]': finished 2024-04-26T01:47:04.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:04 smithi151 ceph-mon[28057]: osdmap e66: 10 total, 9 up, 10 in 2024-04-26T01:47:04.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:04 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:47:05.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:05 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/1286151207' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:47:05.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:05 smithi165 ceph-mon[32648]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T01:47:05.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:05 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/1286151207' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:47:05.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:05 smithi151 ceph-mon[28057]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T01:47:06.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:05 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/1286151207' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:47:06.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:05 smithi172 ceph-mon[32785]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T01:47:08.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:08 smithi151 ceph-mon[28057]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-04-26T01:47:08.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:08 smithi172 ceph-mon[32785]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-04-26T01:47:08.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:08 smithi165 ceph-mon[32648]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-04-26T01:47:10.743 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:10 smithi151 ceph-mon[28057]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T01:47:10.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:10 smithi172 ceph-mon[32785]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T01:47:10.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:10 smithi165 ceph-mon[32648]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T01:47:12.531 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:12 smithi151 ceph-mon[28057]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:47:12.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:12 smithi172 ceph-mon[32785]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:47:12.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:12 smithi165 ceph-mon[32648]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:47:14.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:14 smithi172 ceph-mon[32785]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:47:14.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:14 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-26T01:47:14.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:14 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:14.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:14 smithi172 ceph-mon[32785]: Deploying daemon osd.9 on smithi151 2024-04-26T01:47:14.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:14 smithi165 ceph-mon[32648]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:47:14.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:14 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-26T01:47:14.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:14 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:14.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:14 smithi165 ceph-mon[32648]: Deploying daemon osd.9 on smithi151 2024-04-26T01:47:14.870 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:14 smithi151 ceph-mon[28057]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:47:14.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:14 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-26T01:47:14.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:14 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:14.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:14 smithi151 ceph-mon[28057]: Deploying daemon osd.9 on smithi151 2024-04-26T01:47:16.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:16 smithi151 ceph-mon[28057]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:47:16.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:16 smithi172 ceph-mon[32785]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:47:16.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:16 smithi165 ceph-mon[32648]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:47:18.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:18 smithi172 ceph-mon[32785]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:47:18.799 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:18 smithi151 ceph-mon[28057]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:47:18.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:18 smithi165 ceph-mon[32648]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:47:20.675 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:20 smithi151 ceph-mon[28057]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:47:20.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:20 smithi172 ceph-mon[32785]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:47:20.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:20 smithi165 ceph-mon[32648]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:47:22.725 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:22 smithi151 ceph-mon[28057]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:47:22.725 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:22 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:22.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:22 smithi172 ceph-mon[32785]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:47:22.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:22 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:22.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:22 smithi165 ceph-mon[32648]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:47:22.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:22 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:23.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:23 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:23.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:23 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:47:23.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:23 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:23.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:23 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:47:23.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:23 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:23.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:23 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:47:24.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:24 smithi151 ceph-mon[28057]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:47:24.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:24 smithi172 ceph-mon[32785]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:47:24.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:24 smithi165 ceph-mon[32648]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:47:27.295 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:27 smithi151 ceph-mon[28057]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:47:27.295 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:27 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:27.295 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:27 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:27.295 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:27 smithi151 ceph-mon[28057]: from='osd.9 [v2:172.21.15.151:6826/1818923828,v1:172.21.15.151:6827/1818923828]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-26T01:47:27.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:27 smithi172 ceph-mon[32785]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:47:27.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:27 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:27.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:27 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:27.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:27 smithi172 ceph-mon[32785]: from='osd.9 [v2:172.21.15.151:6826/1818923828,v1:172.21.15.151:6827/1818923828]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-26T01:47:27.548 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:27 smithi165 ceph-mon[32648]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:47:27.548 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:27 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:27.548 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:27 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:27.548 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:27 smithi165 ceph-mon[32648]: from='osd.9 [v2:172.21.15.151:6826/1818923828,v1:172.21.15.151:6827/1818923828]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-26T01:47:27.578 INFO:teuthology.orchestra.run.smithi151.stdout:Created osd(s) 9 on host 'smithi151' 2024-04-26T01:47:27.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:47:27.579+0000 7f4c21600700 1 -- 172.21.15.151:0/782476789 <== mgr.14152 v2:172.21.15.151:6800/2251256505 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4c24062e80 con 0x7f4c0c0a3cc0 2024-04-26T01:47:27.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:47:27.583+0000 7f4c2ba00700 1 -- 172.21.15.151:0/782476789 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f4c0c0a3cc0 msgr2=0x7f4c0c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:27.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:47:27.583+0000 7f4c2ba00700 1 --2- 172.21.15.151:0/782476789 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f4c0c0a3cc0 0x7f4c0c0a6180 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4c1800b760 tx=0x7f4c1800b2a0 comp rx=0 tx=0).stop 2024-04-26T01:47:27.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:47:27.583+0000 7f4c2ba00700 1 -- 172.21.15.151:0/782476789 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c24101c80 msgr2=0x7f4c2419b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:27.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:47:27.583+0000 7f4c2ba00700 1 --2- 172.21.15.151:0/782476789 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c24101c80 0x7f4c2419b3e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4c1000e9f0 tx=0x7f4c1000ed00 comp rx=0 tx=0).stop 2024-04-26T01:47:27.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:47:27.583+0000 7f4c2ba00700 1 -- 172.21.15.151:0/782476789 shutdown_connections 2024-04-26T01:47:27.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:47:27.583+0000 7f4c2ba00700 1 --2- 172.21.15.151:0/782476789 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f4c0c0a3cc0 0x7f4c0c0a6180 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:27.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:47:27.583+0000 7f4c2ba00700 1 --2- 172.21.15.151:0/782476789 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4c24102640 0x7f4c241a2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:27.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:47:27.583+0000 7f4c2ba00700 1 --2- 172.21.15.151:0/782476789 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c24101320 0x7f4c2419aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:27.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:47:27.583+0000 7f4c2ba00700 1 --2- 172.21.15.151:0/782476789 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c24101c80 0x7f4c2419b3e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:27.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:47:27.583+0000 7f4c2ba00700 1 -- 172.21.15.151:0/782476789 >> 172.21.15.151:0/782476789 conn(0x7f4c240fcf40 msgr2=0x7f4c240fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:47:27.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:47:27.583+0000 7f4c2ba00700 1 -- 172.21.15.151:0/782476789 shutdown_connections 2024-04-26T01:47:27.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:47:27.584+0000 7f4c2ba00700 1 -- 172.21.15.151:0/782476789 wait complete. 2024-04-26T01:47:28.427 DEBUG:teuthology.orchestra.run.smithi151:osd.9> sudo journalctl -f -n 0 -u ceph-c0e49e64-036d-11ef-bc93-c7b262605968@osd.9.service 2024-04-26T01:47:28.431 INFO:tasks.cephadm:Deploying osd.10 on smithi165 with /dev/vg_nvme/lv_1... 2024-04-26T01:47:28.431 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-26T01:47:28.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:28 smithi172 ceph-mon[32785]: from='osd.9 [v2:172.21.15.151:6826/1818923828,v1:172.21.15.151:6827/1818923828]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-26T01:47:28.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:28 smithi172 ceph-mon[32785]: osdmap e67: 10 total, 9 up, 10 in 2024-04-26T01:47:28.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:28 smithi172 ceph-mon[32785]: from='osd.9 [v2:172.21.15.151:6826/1818923828,v1:172.21.15.151:6827/1818923828]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-04-26T01:47:28.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:28 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:47:28.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:28 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:28.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:28 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:28.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:28 smithi165 ceph-mon[32648]: from='osd.9 [v2:172.21.15.151:6826/1818923828,v1:172.21.15.151:6827/1818923828]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-26T01:47:28.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:28 smithi165 ceph-mon[32648]: osdmap e67: 10 total, 9 up, 10 in 2024-04-26T01:47:28.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:28 smithi165 ceph-mon[32648]: from='osd.9 [v2:172.21.15.151:6826/1818923828,v1:172.21.15.151:6827/1818923828]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-04-26T01:47:28.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:28 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:47:28.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:28 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:28.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:28 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:28.609 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:28 smithi151 ceph-mon[28057]: from='osd.9 [v2:172.21.15.151:6826/1818923828,v1:172.21.15.151:6827/1818923828]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-26T01:47:28.609 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:28 smithi151 ceph-mon[28057]: osdmap e67: 10 total, 9 up, 10 in 2024-04-26T01:47:28.609 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:28 smithi151 ceph-mon[28057]: from='osd.9 [v2:172.21.15.151:6826/1818923828,v1:172.21.15.151:6827/1818923828]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-04-26T01:47:28.609 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:28 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:47:28.609 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:28 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:28.610 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:28 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:28.677 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T01:47:29.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:29 smithi172 ceph-mon[32785]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:47:29.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:29 smithi172 ceph-mon[32785]: from='osd.9 [v2:172.21.15.151:6826/1818923828,v1:172.21.15.151:6827/1818923828]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-04-26T01:47:29.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:29 smithi172 ceph-mon[32785]: osdmap e68: 10 total, 9 up, 10 in 2024-04-26T01:47:29.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:29 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:47:29.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:29 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:47:29.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:29 smithi165 ceph-mon[32648]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:47:29.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:29 smithi165 ceph-mon[32648]: from='osd.9 [v2:172.21.15.151:6826/1818923828,v1:172.21.15.151:6827/1818923828]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-04-26T01:47:29.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:29 smithi165 ceph-mon[32648]: osdmap e68: 10 total, 9 up, 10 in 2024-04-26T01:47:29.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:29 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:47:29.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:29 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:47:29.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:29 smithi151 ceph-mon[28057]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:47:29.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:29 smithi151 ceph-mon[28057]: from='osd.9 [v2:172.21.15.151:6826/1818923828,v1:172.21.15.151:6827/1818923828]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-04-26T01:47:29.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:29 smithi151 ceph-mon[28057]: osdmap e68: 10 total, 9 up, 10 in 2024-04-26T01:47:29.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:29 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:47:29.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:29 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:47:29.622 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 01:47:29 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T01:47:29.298+0000 7ffa99c00700 -1 osd.9 0 waiting for initial osdmap 2024-04-26T01:47:29.622 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 01:47:29 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T01:47:29.343+0000 7ffa8f000700 -1 osd.9 68 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T01:47:30.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:30 smithi151 ceph-mon[28057]: purged_snaps scrub starts 2024-04-26T01:47:30.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:30 smithi151 ceph-mon[28057]: purged_snaps scrub ok 2024-04-26T01:47:30.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:30 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:47:30.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:30 smithi151 ceph-mon[28057]: from='osd.9 [v2:172.21.15.151:6826/1818923828,v1:172.21.15.151:6827/1818923828]' entity='osd.9' 2024-04-26T01:47:30.508 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:30 smithi165 ceph-mon[32648]: purged_snaps scrub starts 2024-04-26T01:47:30.508 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:30 smithi165 ceph-mon[32648]: purged_snaps scrub ok 2024-04-26T01:47:30.508 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:30 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:47:30.509 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:30 smithi165 ceph-mon[32648]: from='osd.9 [v2:172.21.15.151:6826/1818923828,v1:172.21.15.151:6827/1818923828]' entity='osd.9' 2024-04-26T01:47:30.509 INFO:teuthology.orchestra.run.smithi165.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-26T01:47:30.509 INFO:teuthology.orchestra.run.smithi165.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-26T01:47:30.509 INFO:teuthology.orchestra.run.smithi165.stderr: stderr: 10+0 records in 2024-04-26T01:47:30.509 INFO:teuthology.orchestra.run.smithi165.stderr:10+0 records out 2024-04-26T01:47:30.509 INFO:teuthology.orchestra.run.smithi165.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0129186 s, 812 MB/s 2024-04-26T01:47:30.509 INFO:teuthology.orchestra.run.smithi165.stderr:--> Zapping successful for: 2024-04-26T01:47:30.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:30 smithi172 ceph-mon[32785]: purged_snaps scrub starts 2024-04-26T01:47:30.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:30 smithi172 ceph-mon[32785]: purged_snaps scrub ok 2024-04-26T01:47:30.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:30 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:47:30.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:30 smithi172 ceph-mon[32785]: from='osd.9 [v2:172.21.15.151:6826/1818923828,v1:172.21.15.151:6827/1818923828]' entity='osd.9' 2024-04-26T01:47:31.100 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi165:vg_nvme/lv_1 2024-04-26T01:47:31.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:31 smithi165 ceph-mon[32648]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:47:31.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:31 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:47:31.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:31 smithi165 ceph-mon[32648]: osd.9 [v2:172.21.15.151:6826/1818923828,v1:172.21.15.151:6827/1818923828] boot 2024-04-26T01:47:31.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:31 smithi165 ceph-mon[32648]: osdmap e69: 10 total, 10 up, 10 in 2024-04-26T01:47:31.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:31 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:47:31.349 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T01:47:31.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:31 smithi172 ceph-mon[32785]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:47:31.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:31 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:47:31.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:31 smithi172 ceph-mon[32785]: osd.9 [v2:172.21.15.151:6826/1818923828,v1:172.21.15.151:6827/1818923828] boot 2024-04-26T01:47:31.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:31 smithi172 ceph-mon[32785]: osdmap e69: 10 total, 10 up, 10 in 2024-04-26T01:47:31.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:31 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:47:31.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:31 smithi151 ceph-mon[28057]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:47:31.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:31 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:47:31.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:31 smithi151 ceph-mon[28057]: osd.9 [v2:172.21.15.151:6826/1818923828,v1:172.21.15.151:6827/1818923828] boot 2024-04-26T01:47:31.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:31 smithi151 ceph-mon[28057]: osdmap e69: 10 total, 10 up, 10 in 2024-04-26T01:47:31.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:31 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:47:32.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:47:32.809+0000 7f165d904700 1 -- 172.21.15.165:0/887015419 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16581035d0 msgr2=0x7f16581059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:32.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:47:32.809+0000 7f165d904700 1 --2- 172.21.15.165:0/887015419 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16581035d0 0x7f16581059c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1648009990 tx=0x7f1648009ca0 comp rx=0 tx=0).stop 2024-04-26T01:47:32.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:47:32.809+0000 7f165d904700 1 -- 172.21.15.165:0/887015419 shutdown_connections 2024-04-26T01:47:32.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:47:32.809+0000 7f165d904700 1 --2- 172.21.15.165:0/887015419 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1658108830 0x7f165810ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:32.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:47:32.809+0000 7f165d904700 1 --2- 172.21.15.165:0/887015419 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16581035d0 0x7f16581059c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:32.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:47:32.809+0000 7f165d904700 1 --2- 172.21.15.165:0/887015419 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1658105f00 0x7f16581082f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:32.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:47:32.809+0000 7f165d904700 1 -- 172.21.15.165:0/887015419 >> 172.21.15.165:0/887015419 conn(0x7f16580fcf40 msgr2=0x7f16580ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:47:32.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:47:32.809+0000 7f165d904700 1 -- 172.21.15.165:0/887015419 shutdown_connections 2024-04-26T01:47:32.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:47:32.809+0000 7f165d904700 1 -- 172.21.15.165:0/887015419 wait complete. 2024-04-26T01:47:32.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:47:32.810+0000 7f165d904700 1 Processor -- start 2024-04-26T01:47:32.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:47:32.810+0000 7f165d904700 1 -- start start 2024-04-26T01:47:32.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:47:32.810+0000 7f165d904700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16581035d0 0x7f1658102f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:32.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:47:32.810+0000 7f165d904700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1658105f00 0x7f1658101580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:32.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:47:32.810+0000 7f165d904700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1658108830 0x7f1658101ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:32.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:47:32.810+0000 7f165d904700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f165810e580 con 0x7f1658108830 2024-04-26T01:47:32.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:47:32.810+0000 7f165d904700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f165810e400 con 0x7f1658105f00 2024-04-26T01:47:32.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:47:32.810+0000 7f165d904700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f165810e700 con 0x7f16581035d0 2024-04-26T01:47:32.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:47:32.810+0000 7f16567fc700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1658105f00 0x7f1658101580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:32.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:47:32.810+0000 7f16567fc700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1658105f00 0x7f1658101580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33378/0 (socket says 172.21.15.165:33378) 2024-04-26T01:47:32.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:47:32.810+0000 7f16567fc700 1 -- 172.21.15.165:0/3527431329 learned_addr learned my addr 172.21.15.165:0/3527431329 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-26T01:47:32.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:47:32.810+0000 7f1656ffd700 1 --2- 172.21.15.165:0/3527431329 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16581035d0 0x7f1658102f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:32.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:47:32.810+0000 7f16567fc700 1 -- 172.21.15.165:0/3527431329 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16581035d0 msgr2=0x7f1658102f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:32.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:47:32.810+0000 7f16567fc700 1 --2- 172.21.15.165:0/3527431329 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16581035d0 0x7f1658102f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:32.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:47:32.810+0000 7f16567fc700 1 -- 172.21.15.165:0/3527431329 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1658108830 msgr2=0x7f1658101ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:32.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:47:32.810+0000 7f16567fc700 1 --2- 172.21.15.165:0/3527431329 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1658108830 0x7f1658101ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:32.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:47:32.810+0000 7f16567fc700 1 -- 172.21.15.165:0/3527431329 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1658102370 con 0x7f1658105f00 2024-04-26T01:47:32.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:47:32.811+0000 7f16567fc700 1 --2- 172.21.15.165:0/3527431329 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1658105f00 0x7f1658101580 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f164c00b530 tx=0x7f164c00b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:47:32.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:47:32.811+0000 7f165c902700 1 -- 172.21.15.165:0/3527431329 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f164c015020 con 0x7f1658105f00 2024-04-26T01:47:32.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:47:32.811+0000 7f165c902700 1 -- 172.21.15.165:0/3527431329 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f164c011ce0 con 0x7f1658105f00 2024-04-26T01:47:32.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:47:32.811+0000 7f165c902700 1 -- 172.21.15.165:0/3527431329 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f164c010e40 con 0x7f1658105f00 2024-04-26T01:47:32.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:47:32.811+0000 7f165d904700 1 -- 172.21.15.165:0/3527431329 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16581a2de0 con 0x7f1658105f00 2024-04-26T01:47:32.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:47:32.811+0000 7f165d904700 1 -- 172.21.15.165:0/3527431329 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16581a32c0 con 0x7f1658105f00 2024-04-26T01:47:32.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:47:32.812+0000 7f165d904700 1 -- 172.21.15.165:0/3527431329 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1658069730 con 0x7f1658105f00 2024-04-26T01:47:32.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:47:32.813+0000 7f165c902700 1 -- 172.21.15.165:0/3527431329 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f164c018020 con 0x7f1658105f00 2024-04-26T01:47:32.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:47:32.813+0000 7f165c902700 1 --2- 172.21.15.165:0/3527431329 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f16400a3c40 0x7f16400a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:32.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:47:32.814+0000 7f165c902700 1 -- 172.21.15.165:0/3527431329 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6261+0+0 (secure 0 0 0) 0x7f164c0cf410 con 0x7f1658105f00 2024-04-26T01:47:32.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:47:32.814+0000 7f1656ffd700 1 --2- 172.21.15.165:0/3527431329 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f16400a3c40 0x7f16400a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:32.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:47:32.815+0000 7f1656ffd700 1 --2- 172.21.15.165:0/3527431329 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f16400a3c40 0x7f16400a6100 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f164800b5c0 tx=0x7f1648005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:47:32.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:47:32.818+0000 7f165c902700 1 -- 172.21.15.165:0/3527431329 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f164c01f020 con 0x7f1658105f00 2024-04-26T01:47:33.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:47:33.027+0000 7f165d904700 1 -- 172.21.15.165:0/3527431329 --> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f1658062e80 con 0x7f16400a3c40 2024-04-26T01:47:33.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:33 smithi172 ceph-mon[32785]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:47:33.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:33 smithi172 ceph-mon[32785]: osdmap e70: 10 total, 10 up, 10 in 2024-04-26T01:47:33.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:33 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:33.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:33 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:33.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:33 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:47:33.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:33 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:47:33.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:33 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:47:33.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:33 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:47:33.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:33 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:33.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:33 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:33.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:33 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:47:33.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:33 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:33.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:33 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:47:33.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:33 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:47:33.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:33 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:33.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:33 smithi165 ceph-mon[32648]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:47:33.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:33 smithi165 ceph-mon[32648]: osdmap e70: 10 total, 10 up, 10 in 2024-04-26T01:47:33.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:33 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:33.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:33 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:33.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:33 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:47:33.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:33 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:47:33.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:33 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:47:33.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:33 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:47:33.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:33 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:33.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:33 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:33.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:33 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:47:33.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:33 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:33.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:33 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:47:33.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:33 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:47:33.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:33 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:33.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:33 smithi151 ceph-mon[28057]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:47:33.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:33 smithi151 ceph-mon[28057]: osdmap e70: 10 total, 10 up, 10 in 2024-04-26T01:47:33.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:33 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:33.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:33 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:33.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:33 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:47:33.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:33 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:47:33.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:33 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:47:33.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:33 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:47:33.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:33 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:33.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:33 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:33.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:33 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:47:33.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:33 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:33.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:33 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:47:33.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:33 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:47:33.623 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:33 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:34.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:34 smithi172 ceph-mon[32785]: Detected new or changed devices on smithi151 2024-04-26T01:47:34.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:34 smithi172 ceph-mon[32785]: Adjusting osd_memory_target on smithi151 to 4272M 2024-04-26T01:47:34.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:34 smithi172 ceph-mon[32785]: from='client.24350 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:47:34.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:34 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:47:34.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:34 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:34.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:34 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:47:34.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:34 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:34.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:34 smithi165 ceph-mon[32648]: Detected new or changed devices on smithi151 2024-04-26T01:47:34.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:34 smithi165 ceph-mon[32648]: Adjusting osd_memory_target on smithi151 to 4272M 2024-04-26T01:47:34.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:34 smithi165 ceph-mon[32648]: from='client.24350 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:47:34.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:34 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:47:34.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:34 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:34.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:34 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:47:34.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:34 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:34.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:34 smithi151 ceph-mon[28057]: Detected new or changed devices on smithi151 2024-04-26T01:47:34.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:34 smithi151 ceph-mon[28057]: Adjusting osd_memory_target on smithi151 to 4272M 2024-04-26T01:47:34.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:34 smithi151 ceph-mon[28057]: from='client.24350 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:47:34.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:34 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:47:34.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:34 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:34.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:34 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:47:34.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:34 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:35.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:35 smithi165 ceph-mon[32648]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:47:35.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:35 smithi151 ceph-mon[28057]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:47:35.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:35 smithi172 ceph-mon[32785]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:47:36.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:36 smithi165 ceph-mon[32648]: from='client.? 172.21.15.165:0/1809839655' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "19e75a1a-7bd0-41d9-b0cc-0fa4d1475dda"}]: dispatch 2024-04-26T01:47:36.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:36 smithi165 ceph-mon[32648]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "19e75a1a-7bd0-41d9-b0cc-0fa4d1475dda"}]: dispatch 2024-04-26T01:47:36.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:36 smithi165 ceph-mon[32648]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "19e75a1a-7bd0-41d9-b0cc-0fa4d1475dda"}]': finished 2024-04-26T01:47:36.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:36 smithi165 ceph-mon[32648]: osdmap e71: 11 total, 10 up, 11 in 2024-04-26T01:47:36.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:36 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T01:47:36.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:36 smithi151 ceph-mon[28057]: from='client.? 172.21.15.165:0/1809839655' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "19e75a1a-7bd0-41d9-b0cc-0fa4d1475dda"}]: dispatch 2024-04-26T01:47:36.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:36 smithi151 ceph-mon[28057]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "19e75a1a-7bd0-41d9-b0cc-0fa4d1475dda"}]: dispatch 2024-04-26T01:47:36.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:36 smithi151 ceph-mon[28057]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "19e75a1a-7bd0-41d9-b0cc-0fa4d1475dda"}]': finished 2024-04-26T01:47:36.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:36 smithi151 ceph-mon[28057]: osdmap e71: 11 total, 10 up, 11 in 2024-04-26T01:47:36.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:36 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T01:47:36.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:36 smithi172 ceph-mon[32785]: from='client.? 172.21.15.165:0/1809839655' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "19e75a1a-7bd0-41d9-b0cc-0fa4d1475dda"}]: dispatch 2024-04-26T01:47:36.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:36 smithi172 ceph-mon[32785]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "19e75a1a-7bd0-41d9-b0cc-0fa4d1475dda"}]: dispatch 2024-04-26T01:47:36.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:36 smithi172 ceph-mon[32785]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "19e75a1a-7bd0-41d9-b0cc-0fa4d1475dda"}]': finished 2024-04-26T01:47:36.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:36 smithi172 ceph-mon[32785]: osdmap e71: 11 total, 10 up, 11 in 2024-04-26T01:47:36.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:36 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T01:47:37.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:37 smithi165 ceph-mon[32648]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:47:37.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:37 smithi165 ceph-mon[32648]: from='client.? 172.21.15.165:0/1755392513' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:47:37.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:37 smithi151 ceph-mon[28057]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:47:37.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:37 smithi151 ceph-mon[28057]: from='client.? 172.21.15.165:0/1755392513' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:47:37.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:37 smithi172 ceph-mon[32785]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:47:37.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:37 smithi172 ceph-mon[32785]: from='client.? 172.21.15.165:0/1755392513' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:47:39.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:39 smithi165 ceph-mon[32648]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:47:39.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:39 smithi151 ceph-mon[28057]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:47:39.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:39 smithi172 ceph-mon[32785]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:47:41.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:41 smithi165 ceph-mon[32648]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:47:41.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:41 smithi151 ceph-mon[28057]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:47:41.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:41 smithi172 ceph-mon[32785]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:47:43.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:43 smithi165 ceph-mon[32648]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:47:43.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:43 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-26T01:47:43.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:43 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:43.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:43 smithi151 ceph-mon[28057]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:47:43.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:43 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-26T01:47:43.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:43 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:43.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:43 smithi172 ceph-mon[32785]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:47:43.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:43 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-26T01:47:43.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:43 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:44.502 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:44 smithi165 ceph-mon[32648]: Deploying daemon osd.10 on smithi165 2024-04-26T01:47:44.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:44 smithi151 ceph-mon[28057]: Deploying daemon osd.10 on smithi165 2024-04-26T01:47:44.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:44 smithi172 ceph-mon[32785]: Deploying daemon osd.10 on smithi165 2024-04-26T01:47:45.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:45 smithi165 ceph-mon[32648]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:47:45.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:45 smithi151 ceph-mon[28057]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:47:45.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:45 smithi172 ceph-mon[32785]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:47:46.485 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:46 smithi165 ceph-mon[32648]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:47:46.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:46 smithi151 ceph-mon[28057]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:47:46.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:46 smithi172 ceph-mon[32785]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:47:48.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:48 smithi151 ceph-mon[28057]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:47:48.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:48 smithi172 ceph-mon[32785]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:47:48.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:48 smithi165 ceph-mon[32648]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:47:50.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:49 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:50.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:49 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:47:50.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:49 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:50.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:49 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:50.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:49 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:47:50.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:49 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:50.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:49 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:50.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:49 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:47:50.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:49 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:51.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:50 smithi172 ceph-mon[32785]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:47:51.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:50 smithi165 ceph-mon[32648]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:47:51.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:50 smithi151 ceph-mon[28057]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:47:52.305 INFO:teuthology.orchestra.run.smithi165.stdout:Created osd(s) 10 on host 'smithi165' 2024-04-26T01:47:52.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:47:52.307+0000 7f165c902700 1 -- 172.21.15.165:0/3527431329 <== mgr.14152 v2:172.21.15.151:6800/2251256505 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f1658062e80 con 0x7f16400a3c40 2024-04-26T01:47:52.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:47:52.310+0000 7f165d904700 1 -- 172.21.15.165:0/3527431329 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f16400a3c40 msgr2=0x7f16400a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:52.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:47:52.310+0000 7f165d904700 1 --2- 172.21.15.165:0/3527431329 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f16400a3c40 0x7f16400a6100 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f164800b5c0 tx=0x7f1648005fb0 comp rx=0 tx=0).stop 2024-04-26T01:47:52.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:47:52.310+0000 7f165d904700 1 -- 172.21.15.165:0/3527431329 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1658105f00 msgr2=0x7f1658101580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:52.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:47:52.310+0000 7f165d904700 1 --2- 172.21.15.165:0/3527431329 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1658105f00 0x7f1658101580 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f164c00b530 tx=0x7f164c00b840 comp rx=0 tx=0).stop 2024-04-26T01:47:52.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:47:52.310+0000 7f165d904700 1 -- 172.21.15.165:0/3527431329 shutdown_connections 2024-04-26T01:47:52.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:47:52.310+0000 7f165d904700 1 --2- 172.21.15.165:0/3527431329 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f16400a3c40 0x7f16400a6100 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:52.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:47:52.310+0000 7f165d904700 1 --2- 172.21.15.165:0/3527431329 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1658108830 0x7f1658101ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:52.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:47:52.310+0000 7f165d904700 1 --2- 172.21.15.165:0/3527431329 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16581035d0 0x7f1658102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:52.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:47:52.310+0000 7f165d904700 1 --2- 172.21.15.165:0/3527431329 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1658105f00 0x7f1658101580 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:52.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:47:52.310+0000 7f165d904700 1 -- 172.21.15.165:0/3527431329 >> 172.21.15.165:0/3527431329 conn(0x7f16580fcf40 msgr2=0x7f1658103ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:47:52.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:47:52.310+0000 7f165d904700 1 -- 172.21.15.165:0/3527431329 shutdown_connections 2024-04-26T01:47:52.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T01:47:52.310+0000 7f165d904700 1 -- 172.21.15.165:0/3527431329 wait complete. 2024-04-26T01:47:52.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:52 smithi172 ceph-mon[32785]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:47:52.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:52 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:52.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:52 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:52.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:52 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:52.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:52 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:47:52.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:52 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:52.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:52 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:52.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:52 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:52.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:52 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:47:52.831 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:52 smithi165 ceph-mon[32648]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:47:52.831 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:52 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:52.832 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:52 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:52.832 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:52 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:52.832 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:52 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:47:52.832 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:52 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:52.832 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:52 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:52.832 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:52 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:52.832 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:52 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:47:52.862 DEBUG:teuthology.orchestra.run.smithi165:osd.10> sudo journalctl -f -n 0 -u ceph-c0e49e64-036d-11ef-bc93-c7b262605968@osd.10.service 2024-04-26T01:47:52.865 INFO:tasks.cephadm:Deploying osd.11 on smithi172 with /dev/vg_nvme/lv_1... 2024-04-26T01:47:52.865 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-26T01:47:52.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:52 smithi151 ceph-mon[28057]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:47:52.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:52 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:52.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:52 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:52.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:52 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:52.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:52 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:47:52.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:52 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:52.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:52 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:52.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:52 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:52.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:52 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:47:53.112 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T01:47:53.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:53 smithi172 ceph-mon[32785]: from='osd.10 [v2:172.21.15.165:6824/2632621809,v1:172.21.15.165:6825/2632621809]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T01:47:53.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:53 smithi172 ceph-mon[32785]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T01:47:53.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:53 smithi165 ceph-mon[32648]: from='osd.10 [v2:172.21.15.165:6824/2632621809,v1:172.21.15.165:6825/2632621809]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T01:47:53.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:53 smithi165 ceph-mon[32648]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T01:47:53.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:53 smithi151 ceph-mon[28057]: from='osd.10 [v2:172.21.15.165:6824/2632621809,v1:172.21.15.165:6825/2632621809]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T01:47:53.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:53 smithi151 ceph-mon[28057]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T01:47:54.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:54 smithi172 ceph-mon[32785]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:47:54.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:54 smithi172 ceph-mon[32785]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-26T01:47:54.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:54 smithi172 ceph-mon[32785]: from='osd.10 [v2:172.21.15.165:6824/2632621809,v1:172.21.15.165:6825/2632621809]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-26T01:47:54.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:54 smithi172 ceph-mon[32785]: osdmap e72: 11 total, 10 up, 11 in 2024-04-26T01:47:54.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:54 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T01:47:54.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:54 smithi172 ceph-mon[32785]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-26T01:47:54.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:54 smithi165 ceph-mon[32648]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:47:54.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:54 smithi165 ceph-mon[32648]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-26T01:47:54.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:54 smithi165 ceph-mon[32648]: from='osd.10 [v2:172.21.15.165:6824/2632621809,v1:172.21.15.165:6825/2632621809]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-26T01:47:54.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:54 smithi165 ceph-mon[32648]: osdmap e72: 11 total, 10 up, 11 in 2024-04-26T01:47:54.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:54 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T01:47:54.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:54 smithi165 ceph-mon[32648]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-26T01:47:54.846 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 01:47:54 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T01:47:54.537+0000 7fae1b43f700 -1 osd.10 0 waiting for initial osdmap 2024-04-26T01:47:54.846 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 01:47:54 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T01:47:54.551+0000 7fae17232700 -1 osd.10 73 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T01:47:54.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:54 smithi151 ceph-mon[28057]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:47:54.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:54 smithi151 ceph-mon[28057]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-26T01:47:54.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:54 smithi151 ceph-mon[28057]: from='osd.10 [v2:172.21.15.165:6824/2632621809,v1:172.21.15.165:6825/2632621809]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-26T01:47:54.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:54 smithi151 ceph-mon[28057]: osdmap e72: 11 total, 10 up, 11 in 2024-04-26T01:47:54.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:54 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T01:47:54.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:54 smithi151 ceph-mon[28057]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-26T01:47:55.061 INFO:teuthology.orchestra.run.smithi172.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-26T01:47:55.061 INFO:teuthology.orchestra.run.smithi172.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-26T01:47:55.061 INFO:teuthology.orchestra.run.smithi172.stderr: stderr: 10+0 records in 2024-04-26T01:47:55.061 INFO:teuthology.orchestra.run.smithi172.stderr:10+0 records out 2024-04-26T01:47:55.061 INFO:teuthology.orchestra.run.smithi172.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0179766 s, 583 MB/s 2024-04-26T01:47:55.061 INFO:teuthology.orchestra.run.smithi172.stderr:--> Zapping successful for: 2024-04-26T01:47:55.620 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:55 smithi172 ceph-mon[32785]: purged_snaps scrub starts 2024-04-26T01:47:55.620 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:55 smithi172 ceph-mon[32785]: purged_snaps scrub ok 2024-04-26T01:47:55.620 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:55 smithi172 ceph-mon[32785]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-04-26T01:47:55.620 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:55 smithi172 ceph-mon[32785]: osdmap e73: 11 total, 10 up, 11 in 2024-04-26T01:47:55.620 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:55 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T01:47:55.647 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi172:vg_nvme/lv_1 2024-04-26T01:47:55.711 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:55 smithi165 ceph-mon[32648]: purged_snaps scrub starts 2024-04-26T01:47:55.711 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:55 smithi165 ceph-mon[32648]: purged_snaps scrub ok 2024-04-26T01:47:55.711 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:55 smithi165 ceph-mon[32648]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-04-26T01:47:55.711 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:55 smithi165 ceph-mon[32648]: osdmap e73: 11 total, 10 up, 11 in 2024-04-26T01:47:55.711 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:55 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T01:47:55.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:55 smithi151 ceph-mon[28057]: purged_snaps scrub starts 2024-04-26T01:47:55.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:55 smithi151 ceph-mon[28057]: purged_snaps scrub ok 2024-04-26T01:47:55.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:55 smithi151 ceph-mon[28057]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-04-26T01:47:55.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:55 smithi151 ceph-mon[28057]: osdmap e73: 11 total, 10 up, 11 in 2024-04-26T01:47:55.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:55 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T01:47:55.888 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T01:47:56.717 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:56 smithi172 ceph-mon[32785]: pgmap v258: 1 pgs: 1 peering; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:47:56.718 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:56 smithi172 ceph-mon[32785]: osd.10 [v2:172.21.15.165:6824/2632621809,v1:172.21.15.165:6825/2632621809] boot 2024-04-26T01:47:56.718 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:56 smithi172 ceph-mon[32785]: osdmap e74: 11 total, 11 up, 11 in 2024-04-26T01:47:56.718 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:56 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T01:47:56.718 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:56 smithi172 ceph-mon[32785]: Detected new or changed devices on smithi165 2024-04-26T01:47:56.718 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:56 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:56.718 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:56 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:56.718 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:56 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:47:56.718 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:56 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:47:56.718 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:56 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:47:56.718 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:56 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:47:56.718 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:56 smithi172 ceph-mon[32785]: Adjusting osd_memory_target on smithi165 to 4272M 2024-04-26T01:47:56.718 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:56 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:56.719 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:56 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:56.719 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:56 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:47:56.719 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:56 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:56.719 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:56 smithi172 ceph-mon[32785]: osdmap e75: 11 total, 11 up, 11 in 2024-04-26T01:47:56.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:56 smithi165 ceph-mon[32648]: pgmap v258: 1 pgs: 1 peering; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:47:56.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:56 smithi165 ceph-mon[32648]: osd.10 [v2:172.21.15.165:6824/2632621809,v1:172.21.15.165:6825/2632621809] boot 2024-04-26T01:47:56.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:56 smithi165 ceph-mon[32648]: osdmap e74: 11 total, 11 up, 11 in 2024-04-26T01:47:56.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:56 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T01:47:56.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:56 smithi165 ceph-mon[32648]: Detected new or changed devices on smithi165 2024-04-26T01:47:56.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:56 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:56.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:56 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:56.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:56 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:47:56.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:56 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:47:56.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:56 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:47:56.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:56 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:47:56.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:56 smithi165 ceph-mon[32648]: Adjusting osd_memory_target on smithi165 to 4272M 2024-04-26T01:47:56.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:56 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:56.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:56 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:56.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:56 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:47:56.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:56 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:56.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:56 smithi165 ceph-mon[32648]: osdmap e75: 11 total, 11 up, 11 in 2024-04-26T01:47:56.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:56 smithi151 ceph-mon[28057]: pgmap v258: 1 pgs: 1 peering; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:47:56.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:56 smithi151 ceph-mon[28057]: osd.10 [v2:172.21.15.165:6824/2632621809,v1:172.21.15.165:6825/2632621809] boot 2024-04-26T01:47:56.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:56 smithi151 ceph-mon[28057]: osdmap e74: 11 total, 11 up, 11 in 2024-04-26T01:47:56.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:56 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T01:47:56.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:56 smithi151 ceph-mon[28057]: Detected new or changed devices on smithi165 2024-04-26T01:47:56.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:56 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:56.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:56 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:56.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:56 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:47:56.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:56 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:47:56.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:56 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:47:56.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:56 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:47:56.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:56 smithi151 ceph-mon[28057]: Adjusting osd_memory_target on smithi165 to 4272M 2024-04-26T01:47:56.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:56 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:56.873 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:56 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:56.873 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:56 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:47:56.873 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:56 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:47:56.873 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:56 smithi151 ceph-mon[28057]: osdmap e75: 11 total, 11 up, 11 in 2024-04-26T01:47:57.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:47:57.500+0000 7fefa000e700 1 -- 172.21.15.172:0/2757929483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef981040f0 msgr2=0x7fef98104510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:57.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:47:57.500+0000 7fefa000e700 1 --2- 172.21.15.172:0/2757929483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef981040f0 0x7fef98104510 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fef88009990 tx=0x7fef88009ca0 comp rx=0 tx=0).stop 2024-04-26T01:47:57.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:47:57.500+0000 7fefa000e700 1 -- 172.21.15.172:0/2757929483 shutdown_connections 2024-04-26T01:47:57.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:47:57.500+0000 7fefa000e700 1 --2- 172.21.15.172:0/2757929483 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef98101510 0x7fef9810b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:57.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:47:57.500+0000 7fefa000e700 1 --2- 172.21.15.172:0/2757929483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef981040f0 0x7fef98104510 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:57.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:47:57.500+0000 7fefa000e700 1 --2- 172.21.15.172:0/2757929483 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fef98100b70 0x7fef98100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:57.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:47:57.500+0000 7fefa000e700 1 -- 172.21.15.172:0/2757929483 >> 172.21.15.172:0/2757929483 conn(0x7fef980fc6f0 msgr2=0x7fef980feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:47:57.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:47:57.500+0000 7fefa000e700 1 -- 172.21.15.172:0/2757929483 shutdown_connections 2024-04-26T01:47:57.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:47:57.500+0000 7fefa000e700 1 -- 172.21.15.172:0/2757929483 wait complete. 2024-04-26T01:47:57.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:47:57.501+0000 7fefa000e700 1 Processor -- start 2024-04-26T01:47:57.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:47:57.501+0000 7fefa000e700 1 -- start start 2024-04-26T01:47:57.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:47:57.501+0000 7fefa000e700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fef98100b70 0x7fef9819af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:57.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:47:57.501+0000 7fefa000e700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef98101510 0x7fef9819b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:57.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:47:57.501+0000 7fefa000e700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef981040f0 0x7fef981a2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:57.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:47:57.501+0000 7fefa000e700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fef9810e630 con 0x7fef98101510 2024-04-26T01:47:57.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:47:57.501+0000 7fefa000e700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fef9810e4b0 con 0x7fef98100b70 2024-04-26T01:47:57.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:47:57.501+0000 7fefa000e700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fef9810e7b0 con 0x7fef981040f0 2024-04-26T01:47:57.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:47:57.501+0000 7fef9e5ab700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef981040f0 0x7fef981a2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:57.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:47:57.501+0000 7fef9e5ab700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef981040f0 0x7fef981a2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55544/0 (socket says 172.21.15.172:55544) 2024-04-26T01:47:57.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:47:57.501+0000 7fef9e5ab700 1 -- 172.21.15.172:0/2848809822 learned_addr learned my addr 172.21.15.172:0/2848809822 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-26T01:47:57.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:47:57.502+0000 7fef9ddaa700 1 --2- 172.21.15.172:0/2848809822 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fef98100b70 0x7fef9819af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:57.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:47:57.502+0000 7fef9e5ab700 1 -- 172.21.15.172:0/2848809822 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fef98100b70 msgr2=0x7fef9819af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:57.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:47:57.502+0000 7fef9e5ab700 1 --2- 172.21.15.172:0/2848809822 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fef98100b70 0x7fef9819af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:57.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:47:57.502+0000 7fef9e5ab700 1 -- 172.21.15.172:0/2848809822 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef98101510 msgr2=0x7fef9819b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:57.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:47:57.502+0000 7fef9e5ab700 1 --2- 172.21.15.172:0/2848809822 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef98101510 0x7fef9819b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:57.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:47:57.502+0000 7fef9e5ab700 1 -- 172.21.15.172:0/2848809822 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef981a2ba0 con 0x7fef981040f0 2024-04-26T01:47:57.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:47:57.502+0000 7fef9e5ab700 1 --2- 172.21.15.172:0/2848809822 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef981040f0 0x7fef981a2500 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fef9000b9a0 tx=0x7fef9000bcb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:47:57.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:47:57.502+0000 7fef8effd700 1 -- 172.21.15.172:0/2848809822 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef90019070 con 0x7fef981040f0 2024-04-26T01:47:57.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:47:57.502+0000 7fef8effd700 1 -- 172.21.15.172:0/2848809822 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef9000cde0 con 0x7fef981040f0 2024-04-26T01:47:57.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:47:57.502+0000 7fefa000e700 1 -- 172.21.15.172:0/2848809822 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef981a2e90 con 0x7fef981040f0 2024-04-26T01:47:57.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:47:57.502+0000 7fef8effd700 1 -- 172.21.15.172:0/2848809822 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef90007370 con 0x7fef981040f0 2024-04-26T01:47:57.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:47:57.502+0000 7fefa000e700 1 -- 172.21.15.172:0/2848809822 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef981a33c0 con 0x7fef981040f0 2024-04-26T01:47:57.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:47:57.503+0000 7fefa000e700 1 -- 172.21.15.172:0/2848809822 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef98069730 con 0x7fef981040f0 2024-04-26T01:47:57.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:47:57.505+0000 7fef8effd700 1 -- 172.21.15.172:0/2848809822 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fef9001c070 con 0x7fef981040f0 2024-04-26T01:47:57.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:47:57.505+0000 7fef8effd700 1 --2- 172.21.15.172:0/2848809822 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fef840a3c40 0x7fef840a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:57.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:47:57.505+0000 7fef8effd700 1 -- 172.21.15.172:0/2848809822 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6723+0+0 (secure 0 0 0) 0x7fef900cfab0 con 0x7fef981040f0 2024-04-26T01:47:57.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:47:57.506+0000 7fef9ddaa700 1 --2- 172.21.15.172:0/2848809822 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fef840a3c40 0x7fef840a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:57.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:47:57.507+0000 7fef9ddaa700 1 --2- 172.21.15.172:0/2848809822 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fef840a3c40 0x7fef840a6100 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fef880038a0 tx=0x7fef88005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:47:57.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:47:57.509+0000 7fef8effd700 1 -- 172.21.15.172:0/2848809822 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fef90023070 con 0x7fef981040f0 2024-04-26T01:47:57.715 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:47:57.717+0000 7fefa000e700 1 -- 172.21.15.172:0/2848809822 --> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fef981033a0 con 0x7fef840a3c40 2024-04-26T01:47:58.604 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:58 smithi172 ceph-mon[32785]: pgmap v261: 1 pgs: 1 peering; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:47:58.604 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:58 smithi172 ceph-mon[32785]: osdmap e76: 11 total, 11 up, 11 in 2024-04-26T01:47:58.604 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:58 smithi172 ceph-mon[32785]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:47:58.604 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:58 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:47:58.604 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:58 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:47:58.604 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:58 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:58.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:58 smithi165 ceph-mon[32648]: pgmap v261: 1 pgs: 1 peering; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:47:58.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:58 smithi165 ceph-mon[32648]: osdmap e76: 11 total, 11 up, 11 in 2024-04-26T01:47:58.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:58 smithi165 ceph-mon[32648]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:47:58.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:58 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:47:58.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:58 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:47:58.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:58 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:58.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:58 smithi151 ceph-mon[28057]: pgmap v261: 1 pgs: 1 peering; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:47:58.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:58 smithi151 ceph-mon[28057]: osdmap e76: 11 total, 11 up, 11 in 2024-04-26T01:47:58.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:58 smithi151 ceph-mon[28057]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:47:58.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:58 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:47:58.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:58 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:47:58.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:58 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:59.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:47:59 smithi151 ceph-mon[28057]: osdmap e77: 11 total, 11 up, 11 in 2024-04-26T01:47:59.660 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:47:59 smithi172 ceph-mon[32785]: osdmap e77: 11 total, 11 up, 11 in 2024-04-26T01:47:59.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:47:59 smithi165 ceph-mon[32648]: osdmap e77: 11 total, 11 up, 11 in 2024-04-26T01:48:00.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:00 smithi165 ceph-mon[32648]: pgmap v264: 1 pgs: 1 peering; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:48:00.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:00 smithi165 ceph-mon[32648]: osdmap e78: 11 total, 11 up, 11 in 2024-04-26T01:48:00.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:00 smithi165 ceph-mon[32648]: from='client.? 172.21.15.172:0/680752553' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aa84e5f5-93a3-4375-8544-a1175f0eb06e"}]: dispatch 2024-04-26T01:48:00.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:00 smithi165 ceph-mon[32648]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aa84e5f5-93a3-4375-8544-a1175f0eb06e"}]: dispatch 2024-04-26T01:48:00.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:00 smithi165 ceph-mon[32648]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aa84e5f5-93a3-4375-8544-a1175f0eb06e"}]': finished 2024-04-26T01:48:00.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:00 smithi165 ceph-mon[32648]: osdmap e79: 12 total, 11 up, 12 in 2024-04-26T01:48:00.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:00 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T01:48:00.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:00 smithi151 ceph-mon[28057]: pgmap v264: 1 pgs: 1 peering; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:48:00.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:00 smithi151 ceph-mon[28057]: osdmap e78: 11 total, 11 up, 11 in 2024-04-26T01:48:00.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:00 smithi151 ceph-mon[28057]: from='client.? 172.21.15.172:0/680752553' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aa84e5f5-93a3-4375-8544-a1175f0eb06e"}]: dispatch 2024-04-26T01:48:00.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:00 smithi151 ceph-mon[28057]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aa84e5f5-93a3-4375-8544-a1175f0eb06e"}]: dispatch 2024-04-26T01:48:00.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:00 smithi151 ceph-mon[28057]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aa84e5f5-93a3-4375-8544-a1175f0eb06e"}]': finished 2024-04-26T01:48:00.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:00 smithi151 ceph-mon[28057]: osdmap e79: 12 total, 11 up, 12 in 2024-04-26T01:48:00.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:00 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T01:48:00.923 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:00 smithi172 ceph-mon[32785]: pgmap v264: 1 pgs: 1 peering; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:48:00.923 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:00 smithi172 ceph-mon[32785]: osdmap e78: 11 total, 11 up, 11 in 2024-04-26T01:48:00.923 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:00 smithi172 ceph-mon[32785]: from='client.? 172.21.15.172:0/680752553' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aa84e5f5-93a3-4375-8544-a1175f0eb06e"}]: dispatch 2024-04-26T01:48:00.923 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:00 smithi172 ceph-mon[32785]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aa84e5f5-93a3-4375-8544-a1175f0eb06e"}]: dispatch 2024-04-26T01:48:00.923 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:00 smithi172 ceph-mon[32785]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aa84e5f5-93a3-4375-8544-a1175f0eb06e"}]': finished 2024-04-26T01:48:00.923 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:00 smithi172 ceph-mon[32785]: osdmap e79: 12 total, 11 up, 12 in 2024-04-26T01:48:00.923 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:00 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T01:48:01.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:01 smithi151 ceph-mon[28057]: from='client.? 172.21.15.172:0/1141821692' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:48:02.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:01 smithi172 ceph-mon[32785]: from='client.? 172.21.15.172:0/1141821692' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:48:02.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:01 smithi165 ceph-mon[32648]: from='client.? 172.21.15.172:0/1141821692' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:48:02.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:02 smithi151 ceph-mon[28057]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-26T01:48:03.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:02 smithi172 ceph-mon[32785]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-26T01:48:03.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:02 smithi165 ceph-mon[32648]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-26T01:48:04.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:04 smithi172 ceph-mon[32785]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 983 GiB / 983 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-04-26T01:48:04.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:04 smithi151 ceph-mon[28057]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 983 GiB / 983 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-04-26T01:48:05.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:04 smithi165 ceph-mon[32648]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 983 GiB / 983 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-04-26T01:48:06.948 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:06 smithi172 ceph-mon[32785]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-04-26T01:48:07.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:06 smithi165 ceph-mon[32648]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-04-26T01:48:07.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:06 smithi151 ceph-mon[28057]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-04-26T01:48:08.931 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:08 smithi172 ceph-mon[32785]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T01:48:08.931 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:08 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-26T01:48:08.931 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:08 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:48:08.931 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:08 smithi172 ceph-mon[32785]: Deploying daemon osd.11 on smithi172 2024-04-26T01:48:09.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:08 smithi165 ceph-mon[32648]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T01:48:09.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:08 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-26T01:48:09.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:08 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:48:09.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:08 smithi165 ceph-mon[32648]: Deploying daemon osd.11 on smithi172 2024-04-26T01:48:09.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:08 smithi151 ceph-mon[28057]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T01:48:09.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:08 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-26T01:48:09.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:08 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:48:09.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:08 smithi151 ceph-mon[28057]: Deploying daemon osd.11 on smithi172 2024-04-26T01:48:11.205 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:10 smithi172 ceph-mon[32785]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-04-26T01:48:11.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:10 smithi165 ceph-mon[32648]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-04-26T01:48:11.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:10 smithi151 ceph-mon[28057]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-04-26T01:48:13.132 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:12 smithi172 ceph-mon[32785]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:48:13.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:12 smithi165 ceph-mon[32648]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:48:13.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:12 smithi151 ceph-mon[28057]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:48:15.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:14 smithi172 ceph-mon[32785]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:48:15.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:14 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:15.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:14 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:48:15.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:14 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:15.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:14 smithi165 ceph-mon[32648]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:48:15.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:14 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:15.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:14 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:48:15.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:14 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:15.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:14 smithi151 ceph-mon[28057]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:48:15.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:14 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:15.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:14 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:48:15.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:14 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:17.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:16 smithi172 ceph-mon[32785]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:48:17.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:16 smithi165 ceph-mon[32648]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:48:17.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:16 smithi151 ceph-mon[28057]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:48:17.150 INFO:teuthology.orchestra.run.smithi172.stdout:Created osd(s) 11 on host 'smithi172' 2024-04-26T01:48:17.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:17.151+0000 7fef8effd700 1 -- 172.21.15.172:0/2848809822 <== mgr.14152 v2:172.21.15.151:6800/2251256505 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fef981033a0 con 0x7fef840a3c40 2024-04-26T01:48:17.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:17.153+0000 7fefa000e700 1 -- 172.21.15.172:0/2848809822 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fef840a3c40 msgr2=0x7fef840a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:17.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:17.153+0000 7fefa000e700 1 --2- 172.21.15.172:0/2848809822 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fef840a3c40 0x7fef840a6100 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fef880038a0 tx=0x7fef88005fb0 comp rx=0 tx=0).stop 2024-04-26T01:48:17.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:17.153+0000 7fefa000e700 1 -- 172.21.15.172:0/2848809822 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef981040f0 msgr2=0x7fef981a2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:17.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:17.153+0000 7fefa000e700 1 --2- 172.21.15.172:0/2848809822 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef981040f0 0x7fef981a2500 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fef9000b9a0 tx=0x7fef9000bcb0 comp rx=0 tx=0).stop 2024-04-26T01:48:17.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:17.153+0000 7fefa000e700 1 -- 172.21.15.172:0/2848809822 shutdown_connections 2024-04-26T01:48:17.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:17.153+0000 7fefa000e700 1 --2- 172.21.15.172:0/2848809822 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef981040f0 0x7fef981a2500 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:17.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:17.153+0000 7fefa000e700 1 --2- 172.21.15.172:0/2848809822 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fef98100b70 0x7fef9819af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:17.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:17.153+0000 7fefa000e700 1 --2- 172.21.15.172:0/2848809822 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fef840a3c40 0x7fef840a6100 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:17.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:17.153+0000 7fefa000e700 1 --2- 172.21.15.172:0/2848809822 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef98101510 0x7fef9819b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:17.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:17.153+0000 7fefa000e700 1 -- 172.21.15.172:0/2848809822 >> 172.21.15.172:0/2848809822 conn(0x7fef980fc6f0 msgr2=0x7fef98101c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:48:17.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:17.153+0000 7fefa000e700 1 -- 172.21.15.172:0/2848809822 shutdown_connections 2024-04-26T01:48:17.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:17.153+0000 7fefa000e700 1 -- 172.21.15.172:0/2848809822 wait complete. 2024-04-26T01:48:17.768 DEBUG:teuthology.orchestra.run.smithi172:osd.11> sudo journalctl -f -n 0 -u ceph-c0e49e64-036d-11ef-bc93-c7b262605968@osd.11.service 2024-04-26T01:48:17.770 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-26T01:48:17.770 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-26T01:48:18.218 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:48:18.391 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:18 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:18.391 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:18 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:18.391 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:18 smithi151 ceph-mon[28057]: from='osd.11 [v2:172.21.15.172:6824/886604365,v1:172.21.15.172:6825/886604365]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T01:48:18.391 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:18 smithi151 ceph-mon[28057]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T01:48:18.392 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:18 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:18.392 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:18 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:18.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:18 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:18.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:18 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:18.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:18 smithi172 ceph-mon[32785]: from='osd.11 [v2:172.21.15.172:6824/886604365,v1:172.21.15.172:6825/886604365]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T01:48:18.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:18 smithi172 ceph-mon[32785]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T01:48:18.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:18 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:18.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:18 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:18.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:18 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:18.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:18 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:18.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:18 smithi165 ceph-mon[32648]: from='osd.11 [v2:172.21.15.172:6824/886604365,v1:172.21.15.172:6825/886604365]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T01:48:18.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:18 smithi165 ceph-mon[32648]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T01:48:18.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:18 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:18.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:18 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:19.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:19 smithi172 ceph-mon[32785]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:48:19.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:19 smithi172 ceph-mon[32785]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-26T01:48:19.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:19 smithi172 ceph-mon[32785]: osdmap e80: 12 total, 11 up, 12 in 2024-04-26T01:48:19.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:19 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T01:48:19.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:19 smithi172 ceph-mon[32785]: from='osd.11 [v2:172.21.15.172:6824/886604365,v1:172.21.15.172:6825/886604365]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-04-26T01:48:19.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:19 smithi172 ceph-mon[32785]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-04-26T01:48:19.401 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:19 smithi151 ceph-mon[28057]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:48:19.401 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:19 smithi151 ceph-mon[28057]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-26T01:48:19.401 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:19 smithi151 ceph-mon[28057]: osdmap e80: 12 total, 11 up, 12 in 2024-04-26T01:48:19.401 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:19 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T01:48:19.401 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:19 smithi151 ceph-mon[28057]: from='osd.11 [v2:172.21.15.172:6824/886604365,v1:172.21.15.172:6825/886604365]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-04-26T01:48:19.401 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:19 smithi151 ceph-mon[28057]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-04-26T01:48:19.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:19 smithi165 ceph-mon[32648]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:48:19.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:19 smithi165 ceph-mon[32648]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-26T01:48:19.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:19 smithi165 ceph-mon[32648]: osdmap e80: 12 total, 11 up, 12 in 2024-04-26T01:48:19.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:19 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T01:48:19.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:19 smithi165 ceph-mon[32648]: from='osd.11 [v2:172.21.15.172:6824/886604365,v1:172.21.15.172:6825/886604365]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-04-26T01:48:19.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:19 smithi165 ceph-mon[32648]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-04-26T01:48:20.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:20 smithi172 ceph-mon[32785]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-04-26T01:48:20.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:20 smithi172 ceph-mon[32785]: osdmap e81: 12 total, 11 up, 12 in 2024-04-26T01:48:20.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:20 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T01:48:20.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:20 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T01:48:20.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:20 smithi172 ceph-mon[32785]: from='osd.11 ' entity='osd.11' 2024-04-26T01:48:20.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 01:48:20 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T01:48:20.105+0000 7f9cf4520700 -1 osd.11 0 waiting for initial osdmap 2024-04-26T01:48:20.302 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:20 smithi151 ceph-mon[28057]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-04-26T01:48:20.302 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:20 smithi151 ceph-mon[28057]: osdmap e81: 12 total, 11 up, 12 in 2024-04-26T01:48:20.302 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:20 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T01:48:20.302 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:20 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T01:48:20.302 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:20 smithi151 ceph-mon[28057]: from='osd.11 ' entity='osd.11' 2024-04-26T01:48:20.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:20.554+0000 7ff5b6c00700 1 -- 172.21.15.151:0/1321555219 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5b0105e50 msgr2=0x7ff5b0108240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:20.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:20.554+0000 7ff5b6c00700 1 --2- 172.21.15.151:0/1321555219 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5b0105e50 0x7ff5b0108240 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff59c009990 tx=0x7ff59c009ca0 comp rx=0 tx=0).stop 2024-04-26T01:48:20.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:20.555+0000 7ff5b6c00700 1 -- 172.21.15.151:0/1321555219 shutdown_connections 2024-04-26T01:48:20.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:20.555+0000 7ff5b6c00700 1 --2- 172.21.15.151:0/1321555219 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff5b0108780 0x7ff5b010abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:20.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:20.555+0000 7ff5b6c00700 1 --2- 172.21.15.151:0/1321555219 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5b0103520 0x7ff5b0105910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:20.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:20.555+0000 7ff5b6c00700 1 --2- 172.21.15.151:0/1321555219 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5b0105e50 0x7ff5b0108240 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:20.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:20.555+0000 7ff5b6c00700 1 -- 172.21.15.151:0/1321555219 >> 172.21.15.151:0/1321555219 conn(0x7ff5b00fcf30 msgr2=0x7ff5b00ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:48:20.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:20.555+0000 7ff5b6c00700 1 -- 172.21.15.151:0/1321555219 shutdown_connections 2024-04-26T01:48:20.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:20.555+0000 7ff5b6c00700 1 -- 172.21.15.151:0/1321555219 wait complete. 2024-04-26T01:48:20.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:20.556+0000 7ff5b6c00700 1 Processor -- start 2024-04-26T01:48:20.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:20.556+0000 7ff5b6c00700 1 -- start start 2024-04-26T01:48:20.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:20.556+0000 7ff5b6c00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5b0103520 0x7ff5b019ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:20.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:20.556+0000 7ff5b6c00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5b0105e50 0x7ff5b019b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:20.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:20.557+0000 7ff5b6c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff5b0108780 0x7ff5b01a2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:20.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:20.557+0000 7ff5b6c00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b010e570 con 0x7ff5b0108780 2024-04-26T01:48:20.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:20.557+0000 7ff5b6c00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b010e3f0 con 0x7ff5b0105e50 2024-04-26T01:48:20.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:20.557+0000 7ff5b6c00700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b010e6f0 con 0x7ff5b0103520 2024-04-26T01:48:20.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:20.557+0000 7ff5afe00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5b0103520 0x7ff5b019ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:20.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:20.557+0000 7ff5afe00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5b0103520 0x7ff5b019ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:60652/0 (socket says 172.21.15.151:60652) 2024-04-26T01:48:20.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:20.557+0000 7ff5b4c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff5b0108780 0x7ff5b01a2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:20.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:20.557+0000 7ff5afe00700 1 -- 172.21.15.151:0/1407724855 learned_addr learned my addr 172.21.15.151:0/1407724855 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:48:20.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:20.557+0000 7ff5af400700 1 --2- 172.21.15.151:0/1407724855 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5b0105e50 0x7ff5b019b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:20.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:20.557+0000 7ff5afe00700 1 -- 172.21.15.151:0/1407724855 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5b0105e50 msgr2=0x7ff5b019b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:20.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:20.557+0000 7ff5afe00700 1 --2- 172.21.15.151:0/1407724855 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5b0105e50 0x7ff5b019b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:20.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:20.557+0000 7ff5afe00700 1 -- 172.21.15.151:0/1407724855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff5b0108780 msgr2=0x7ff5b01a2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:20.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:20.557+0000 7ff5afe00700 1 --2- 172.21.15.151:0/1407724855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff5b0108780 0x7ff5b01a2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:20.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:20.557+0000 7ff5afe00700 1 -- 172.21.15.151:0/1407724855 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5b01a2ad0 con 0x7ff5b0103520 2024-04-26T01:48:20.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:20.558+0000 7ff5afe00700 1 --2- 172.21.15.151:0/1407724855 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5b0103520 0x7ff5b019ae70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff5a000e9d0 tx=0x7ff5a000ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:48:20.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:20.558+0000 7ff5acc00700 1 -- 172.21.15.151:0/1407724855 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a000cbd0 con 0x7ff5b0103520 2024-04-26T01:48:20.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:20.558+0000 7ff5acc00700 1 -- 172.21.15.151:0/1407724855 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5a000ee70 con 0x7ff5b0103520 2024-04-26T01:48:20.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:20.558+0000 7ff5acc00700 1 -- 172.21.15.151:0/1407724855 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a0018680 con 0x7ff5b0103520 2024-04-26T01:48:20.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:20.558+0000 7ff5b6c00700 1 -- 172.21.15.151:0/1407724855 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5b01a2dc0 con 0x7ff5b0103520 2024-04-26T01:48:20.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:20.558+0000 7ff5b6c00700 1 -- 172.21.15.151:0/1407724855 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5b01a3220 con 0x7ff5b0103520 2024-04-26T01:48:20.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:20.560+0000 7ff5b6c00700 1 -- 172.21.15.151:0/1407724855 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5b002df80 con 0x7ff5b0103520 2024-04-26T01:48:20.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:20.561+0000 7ff5acc00700 1 -- 172.21.15.151:0/1407724855 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7ff5a0010430 con 0x7ff5b0103520 2024-04-26T01:48:20.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:20.563+0000 7ff5acc00700 1 --2- 172.21.15.151:0/1407724855 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7ff5940a3cc0 0x7ff5940a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:20.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:20.563+0000 7ff5acc00700 1 -- 172.21.15.151:0/1407724855 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6847+0+0 (secure 0 0 0) 0x7ff5a00cffd0 con 0x7ff5b0103520 2024-04-26T01:48:20.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:20.563+0000 7ff5af400700 1 --2- 172.21.15.151:0/1407724855 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7ff5940a3cc0 0x7ff5940a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:20.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:20.564+0000 7ff5af400700 1 --2- 172.21.15.151:0/1407724855 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7ff5940a3cc0 0x7ff5940a6180 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff59c00b2e0 tx=0x7ff59c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:48:20.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:20.567+0000 7ff5acc00700 1 -- 172.21.15.151:0/1407724855 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff5a009ced0 con 0x7ff5b0103520 2024-04-26T01:48:20.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:20 smithi165 ceph-mon[32648]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-04-26T01:48:20.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:20 smithi165 ceph-mon[32648]: osdmap e81: 12 total, 11 up, 12 in 2024-04-26T01:48:20.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:20 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T01:48:20.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:20 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T01:48:20.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:20 smithi165 ceph-mon[32648]: from='osd.11 ' entity='osd.11' 2024-04-26T01:48:20.732 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 01:48:20 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T01:48:20.296+0000 7f9cec30b700 -1 osd.11 81 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T01:48:20.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:20.794+0000 7ff5b6c00700 1 -- 172.21.15.151:0/1407724855 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7ff5b00636f0 con 0x7ff5b0103520 2024-04-26T01:48:20.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:20.795+0000 7ff5acc00700 1 -- 172.21.15.151:0/1407724855 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) ==== 74+0+133 (secure 0 0 0) 0x7ff5a000cd70 con 0x7ff5b0103520 2024-04-26T01:48:20.794 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:48:20.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:20.798+0000 7ff5b6c00700 1 -- 172.21.15.151:0/1407724855 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7ff5940a3cc0 msgr2=0x7ff5940a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:20.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:20.798+0000 7ff5b6c00700 1 --2- 172.21.15.151:0/1407724855 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7ff5940a3cc0 0x7ff5940a6180 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff59c00b2e0 tx=0x7ff59c005fb0 comp rx=0 tx=0).stop 2024-04-26T01:48:20.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:20.798+0000 7ff5b6c00700 1 -- 172.21.15.151:0/1407724855 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5b0103520 msgr2=0x7ff5b019ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:20.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:20.798+0000 7ff5b6c00700 1 --2- 172.21.15.151:0/1407724855 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5b0103520 0x7ff5b019ae70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff5a000e9d0 tx=0x7ff5a000ece0 comp rx=0 tx=0).stop 2024-04-26T01:48:20.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:20.798+0000 7ff5b6c00700 1 -- 172.21.15.151:0/1407724855 shutdown_connections 2024-04-26T01:48:20.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:20.798+0000 7ff5b6c00700 1 --2- 172.21.15.151:0/1407724855 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7ff5940a3cc0 0x7ff5940a6180 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:20.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:20.798+0000 7ff5b6c00700 1 --2- 172.21.15.151:0/1407724855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff5b0108780 0x7ff5b01a2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:20.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:20.798+0000 7ff5b6c00700 1 --2- 172.21.15.151:0/1407724855 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5b0103520 0x7ff5b019ae70 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:20.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:20.798+0000 7ff5b6c00700 1 --2- 172.21.15.151:0/1407724855 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5b0105e50 0x7ff5b019b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:20.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:20.798+0000 7ff5b6c00700 1 -- 172.21.15.151:0/1407724855 >> 172.21.15.151:0/1407724855 conn(0x7ff5b00fcf30 msgr2=0x7ff5b0102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:48:20.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:20.798+0000 7ff5b6c00700 1 -- 172.21.15.151:0/1407724855 shutdown_connections 2024-04-26T01:48:20.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:20.798+0000 7ff5b6c00700 1 -- 172.21.15.151:0/1407724855 wait complete. 2024-04-26T01:48:21.393 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:21 smithi172 ceph-mon[32785]: purged_snaps scrub starts 2024-04-26T01:48:21.393 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:21 smithi172 ceph-mon[32785]: purged_snaps scrub ok 2024-04-26T01:48:21.393 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:21 smithi172 ceph-mon[32785]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:48:21.393 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:21 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T01:48:21.393 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:21 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/1407724855' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-26T01:48:21.519 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:21 smithi151 ceph-mon[28057]: purged_snaps scrub starts 2024-04-26T01:48:21.519 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:21 smithi151 ceph-mon[28057]: purged_snaps scrub ok 2024-04-26T01:48:21.519 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:21 smithi151 ceph-mon[28057]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:48:21.520 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:21 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T01:48:21.520 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:21 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/1407724855' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-26T01:48:21.564 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":81,"num_osds":12,"num_up_osds":11,"osd_up_since":1714096075,"num_in_osds":12,"osd_in_since":1714096080,"num_remapped_pgs":0} 2024-04-26T01:48:21.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:21 smithi165 ceph-mon[32648]: purged_snaps scrub starts 2024-04-26T01:48:21.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:21 smithi165 ceph-mon[32648]: purged_snaps scrub ok 2024-04-26T01:48:21.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:21 smithi165 ceph-mon[32648]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:48:21.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:21 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T01:48:21.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:21 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/1407724855' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-26T01:48:22.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:22 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T01:48:22.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:22 smithi172 ceph-mon[32785]: osd.11 [v2:172.21.15.172:6824/886604365,v1:172.21.15.172:6825/886604365] boot 2024-04-26T01:48:22.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:22 smithi172 ceph-mon[32785]: osdmap e82: 12 total, 12 up, 12 in 2024-04-26T01:48:22.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:22 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T01:48:22.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:22 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:22.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:22 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:22.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:22 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:48:22.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:22 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:48:22.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:22 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:48:22.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:22 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:48:22.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:22 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:22.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:22 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:48:22.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:22 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:48:22.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:22 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:22.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:22 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:48:22.538 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:22 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:48:22.538 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:22 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:48:22.538 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:22 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:22.566 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-26T01:48:22.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:22 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T01:48:22.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:22 smithi165 ceph-mon[32648]: osd.11 [v2:172.21.15.172:6824/886604365,v1:172.21.15.172:6825/886604365] boot 2024-04-26T01:48:22.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:22 smithi165 ceph-mon[32648]: osdmap e82: 12 total, 12 up, 12 in 2024-04-26T01:48:22.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:22 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T01:48:22.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:22 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:22.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:22 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:22.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:22 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:48:22.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:22 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:48:22.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:22 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:48:22.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:22 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:48:22.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:22 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:22.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:22 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:48:22.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:22 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:48:22.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:22 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:22.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:22 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:48:22.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:22 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:48:22.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:22 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:48:22.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:22 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:22.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:22 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T01:48:22.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:22 smithi151 ceph-mon[28057]: osd.11 [v2:172.21.15.172:6824/886604365,v1:172.21.15.172:6825/886604365] boot 2024-04-26T01:48:22.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:22 smithi151 ceph-mon[28057]: osdmap e82: 12 total, 12 up, 12 in 2024-04-26T01:48:22.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:22 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T01:48:22.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:22 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:22.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:22 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:22.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:22 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:48:22.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:22 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:48:22.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:22 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:48:22.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:22 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:48:22.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:22 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:22.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:22 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:48:22.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:22 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:48:22.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:22 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:22.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:22 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:48:22.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:22 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:48:22.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:22 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:48:22.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:22 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:23.010 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:48:23.475 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:23 smithi151 ceph-mon[28057]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:48:23.475 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:23 smithi151 ceph-mon[28057]: Detected new or changed devices on smithi172 2024-04-26T01:48:23.475 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:23 smithi151 ceph-mon[28057]: Adjusting osd_memory_target on smithi172 to 4272M 2024-04-26T01:48:23.475 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:23 smithi151 ceph-mon[28057]: osdmap e83: 12 total, 12 up, 12 in 2024-04-26T01:48:23.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:23 smithi172 ceph-mon[32785]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:48:23.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:23 smithi172 ceph-mon[32785]: Detected new or changed devices on smithi172 2024-04-26T01:48:23.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:23 smithi172 ceph-mon[32785]: Adjusting osd_memory_target on smithi172 to 4272M 2024-04-26T01:48:23.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:23 smithi172 ceph-mon[32785]: osdmap e83: 12 total, 12 up, 12 in 2024-04-26T01:48:23.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:23 smithi165 ceph-mon[32648]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:48:23.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:23 smithi165 ceph-mon[32648]: Detected new or changed devices on smithi172 2024-04-26T01:48:23.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:23 smithi165 ceph-mon[32648]: Adjusting osd_memory_target on smithi172 to 4272M 2024-04-26T01:48:23.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:23 smithi165 ceph-mon[32648]: osdmap e83: 12 total, 12 up, 12 in 2024-04-26T01:48:25.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:25.207+0000 7f1d98a00700 1 -- 172.21.15.151:0/3734147525 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d941029d0 msgr2=0x7f1d94102df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:25.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:25.207+0000 7f1d98a00700 1 --2- 172.21.15.151:0/3734147525 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d941029d0 0x7f1d94102df0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1d840099c0 tx=0x7f1d84009cd0 comp rx=0 tx=0).stop 2024-04-26T01:48:25.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:25.207+0000 7f1d98a00700 1 -- 172.21.15.151:0/3734147525 shutdown_connections 2024-04-26T01:48:25.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:25.207+0000 7f1d98a00700 1 --2- 172.21.15.151:0/3734147525 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1d94104570 0x7f1d9410ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:25.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:25.207+0000 7f1d98a00700 1 --2- 172.21.15.151:0/3734147525 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d941029d0 0x7f1d94102df0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:25.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:25.207+0000 7f1d98a00700 1 --2- 172.21.15.151:0/3734147525 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d94103bd0 0x7f1d94104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:25.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:25.207+0000 7f1d98a00700 1 -- 172.21.15.151:0/3734147525 >> 172.21.15.151:0/3734147525 conn(0x7f1d940fdf70 msgr2=0x7f1d941003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:48:25.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:25.207+0000 7f1d98a00700 1 -- 172.21.15.151:0/3734147525 shutdown_connections 2024-04-26T01:48:25.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:25.207+0000 7f1d98a00700 1 -- 172.21.15.151:0/3734147525 wait complete. 2024-04-26T01:48:25.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:25.209+0000 7f1d98a00700 1 Processor -- start 2024-04-26T01:48:25.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:25.209+0000 7f1d98a00700 1 -- start start 2024-04-26T01:48:25.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:25.209+0000 7f1d98a00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d941029d0 0x7f1d940767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:25.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:25.209+0000 7f1d98a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d94103bd0 0x7f1d94074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:25.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:25.209+0000 7f1d98a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1d94104570 0x7f1d94075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:25.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:25.209+0000 7f1d98a00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9410e490 con 0x7f1d94103bd0 2024-04-26T01:48:25.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:25.209+0000 7f1d98a00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9410e310 con 0x7f1d94104570 2024-04-26T01:48:25.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:25.209+0000 7f1d98a00700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9410e610 con 0x7f1d941029d0 2024-04-26T01:48:25.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:25.210+0000 7f1d91e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d94103bd0 0x7f1d94074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:25.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:25.210+0000 7f1d91e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d94103bd0 0x7f1d94074e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49716/0 (socket says 172.21.15.151:49716) 2024-04-26T01:48:25.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:25.210+0000 7f1d91e00700 1 -- 172.21.15.151:0/1421045211 learned_addr learned my addr 172.21.15.151:0/1421045211 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:48:25.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:25.210+0000 7f1d92800700 1 --2- 172.21.15.151:0/1421045211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d941029d0 0x7f1d940767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:25.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:25.210+0000 7f1d93200700 1 --2- 172.21.15.151:0/1421045211 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1d94104570 0x7f1d94075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:25.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:25.210+0000 7f1d92800700 1 -- 172.21.15.151:0/1421045211 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1d94104570 msgr2=0x7f1d94075370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:25.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:25.210+0000 7f1d92800700 1 --2- 172.21.15.151:0/1421045211 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1d94104570 0x7f1d94075370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:25.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:25.210+0000 7f1d92800700 1 -- 172.21.15.151:0/1421045211 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d94103bd0 msgr2=0x7f1d94074e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:25.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:25.210+0000 7f1d92800700 1 --2- 172.21.15.151:0/1421045211 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d94103bd0 0x7f1d94074e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:25.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:25.210+0000 7f1d92800700 1 -- 172.21.15.151:0/1421045211 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d94075c20 con 0x7f1d941029d0 2024-04-26T01:48:25.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:25.211+0000 7f1d92800700 1 --2- 172.21.15.151:0/1421045211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d941029d0 0x7f1d940767e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1d840098e0 tx=0x7f1d84003af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:48:25.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:25.211+0000 7f1d8b400700 1 -- 172.21.15.151:0/1421045211 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d840050c0 con 0x7f1d941029d0 2024-04-26T01:48:25.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:25.211+0000 7f1d8b400700 1 -- 172.21.15.151:0/1421045211 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d8400f460 con 0x7f1d941029d0 2024-04-26T01:48:25.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:25.211+0000 7f1d98a00700 1 -- 172.21.15.151:0/1421045211 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d941a2c10 con 0x7f1d941029d0 2024-04-26T01:48:25.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:25.211+0000 7f1d8b400700 1 -- 172.21.15.151:0/1421045211 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d84017670 con 0x7f1d941029d0 2024-04-26T01:48:25.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:25.211+0000 7f1d98a00700 1 -- 172.21.15.151:0/1421045211 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d941a3060 con 0x7f1d941029d0 2024-04-26T01:48:25.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:25.212+0000 7f1d98a00700 1 -- 172.21.15.151:0/1421045211 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d94069730 con 0x7f1d941029d0 2024-04-26T01:48:25.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:25.214+0000 7f1d8b400700 1 -- 172.21.15.151:0/1421045211 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f1d8400f610 con 0x7f1d941029d0 2024-04-26T01:48:25.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:25.215+0000 7f1d8b400700 1 --2- 172.21.15.151:0/1421045211 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f1d780a3cc0 0x7f1d780a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:25.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:25.215+0000 7f1d8b400700 1 -- 172.21.15.151:0/1421045211 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f1d840d0340 con 0x7f1d941029d0 2024-04-26T01:48:25.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:25.216+0000 7f1d91e00700 1 --2- 172.21.15.151:0/1421045211 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f1d780a3cc0 0x7f1d780a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:25.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:25.216+0000 7f1d91e00700 1 --2- 172.21.15.151:0/1421045211 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f1d780a3cc0 0x7f1d780a6180 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1d80009d20 tx=0x7f1d80009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:48:25.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:25.221+0000 7f1d8b400700 1 -- 172.21.15.151:0/1421045211 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1d84013070 con 0x7f1d941029d0 2024-04-26T01:48:25.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:25 smithi151 ceph-mon[28057]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:48:25.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:25.447+0000 7f1d98a00700 1 -- 172.21.15.151:0/1421045211 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f1d94063650 con 0x7f1d941029d0 2024-04-26T01:48:25.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:25.448+0000 7f1d8b400700 1 -- 172.21.15.151:0/1421045211 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v83) ==== 74+0+133 (secure 0 0 0) 0x7f1d8409d1b0 con 0x7f1d941029d0 2024-04-26T01:48:25.446 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:48:25.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:25.451+0000 7f1d98a00700 1 -- 172.21.15.151:0/1421045211 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f1d780a3cc0 msgr2=0x7f1d780a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:25.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:25.451+0000 7f1d98a00700 1 --2- 172.21.15.151:0/1421045211 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f1d780a3cc0 0x7f1d780a6180 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1d80009d20 tx=0x7f1d80009260 comp rx=0 tx=0).stop 2024-04-26T01:48:25.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:25.451+0000 7f1d98a00700 1 -- 172.21.15.151:0/1421045211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d941029d0 msgr2=0x7f1d940767e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:25.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:25.451+0000 7f1d98a00700 1 --2- 172.21.15.151:0/1421045211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d941029d0 0x7f1d940767e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1d840098e0 tx=0x7f1d84003af0 comp rx=0 tx=0).stop 2024-04-26T01:48:25.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:25.451+0000 7f1d98a00700 1 -- 172.21.15.151:0/1421045211 shutdown_connections 2024-04-26T01:48:25.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:25.451+0000 7f1d98a00700 1 --2- 172.21.15.151:0/1421045211 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1d94104570 0x7f1d94075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:25.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:25.451+0000 7f1d98a00700 1 --2- 172.21.15.151:0/1421045211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d941029d0 0x7f1d940767e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:25.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:25.451+0000 7f1d98a00700 1 --2- 172.21.15.151:0/1421045211 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f1d780a3cc0 0x7f1d780a6180 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:25.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:25.451+0000 7f1d98a00700 1 --2- 172.21.15.151:0/1421045211 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d94103bd0 0x7f1d94074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:25.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:25.451+0000 7f1d98a00700 1 -- 172.21.15.151:0/1421045211 >> 172.21.15.151:0/1421045211 conn(0x7f1d940fdf70 msgr2=0x7f1d94106ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:48:25.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:25.451+0000 7f1d98a00700 1 -- 172.21.15.151:0/1421045211 shutdown_connections 2024-04-26T01:48:25.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:25.451+0000 7f1d98a00700 1 -- 172.21.15.151:0/1421045211 wait complete. 2024-04-26T01:48:25.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:25 smithi172 ceph-mon[32785]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:48:25.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:25 smithi165 ceph-mon[32648]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:48:26.262 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":83,"num_osds":12,"num_up_osds":12,"osd_up_since":1714096101,"num_in_osds":12,"osd_in_since":1714096080,"num_remapped_pgs":0} 2024-04-26T01:48:26.263 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T01:48:26.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:26 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/1421045211' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-26T01:48:26.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:26 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/1421045211' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-26T01:48:26.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:26 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/1421045211' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-26T01:48:26.688 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:48:27.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:27 smithi151 ceph-mon[28057]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:48:27.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:27 smithi172 ceph-mon[32785]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:48:27.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:27 smithi165 ceph-mon[32648]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:48:28.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:28.789+0000 7fa4f3e00700 1 -- 172.21.15.151:0/1018245368 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4f4109e90 msgr2=0x7fa4f4111a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:28.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:28.789+0000 7fa4f3e00700 1 --2- 172.21.15.151:0/1018245368 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4f4109e90 0x7fa4f4111a60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec00b0d0 tx=0x7fa4ec00b3e0 comp rx=0 tx=0).stop 2024-04-26T01:48:28.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:28.790+0000 7fa4f3e00700 1 -- 172.21.15.151:0/1018245368 shutdown_connections 2024-04-26T01:48:28.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:28.790+0000 7fa4f3e00700 1 --2- 172.21.15.151:0/1018245368 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4f4109e90 0x7fa4f4111a60 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:28.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:28.790+0000 7fa4f3e00700 1 --2- 172.21.15.151:0/1018245368 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4f4104db0 0x7fa4f4105190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:28.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:28.790+0000 7fa4f3e00700 1 --2- 172.21.15.151:0/1018245368 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa4f4105760 0x7fa4f41097b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:28.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:28.790+0000 7fa4f3e00700 1 -- 172.21.15.151:0/1018245368 >> 172.21.15.151:0/1018245368 conn(0x7fa4f4100620 msgr2=0x7fa4f4102a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:48:28.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:28.790+0000 7fa4f3e00700 1 -- 172.21.15.151:0/1018245368 shutdown_connections 2024-04-26T01:48:28.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:28.790+0000 7fa4f3e00700 1 -- 172.21.15.151:0/1018245368 wait complete. 2024-04-26T01:48:28.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:28.791+0000 7fa4f3e00700 1 Processor -- start 2024-04-26T01:48:28.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:28.791+0000 7fa4f3e00700 1 -- start start 2024-04-26T01:48:28.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:28.791+0000 7fa4f3e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa4f4104db0 0x7fa4f419b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:28.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:28.791+0000 7fa4f3e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4f4105760 0x7fa4f419b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:28.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:28.791+0000 7fa4f3e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4f4109e90 0x7fa4f419fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:28.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:28.791+0000 7fa4f3e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f4114d70 con 0x7fa4f4105760 2024-04-26T01:48:28.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:28.791+0000 7fa4f3e00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f4114bf0 con 0x7fa4f4104db0 2024-04-26T01:48:28.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:28.791+0000 7fa4f3e00700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f4114ef0 con 0x7fa4f4109e90 2024-04-26T01:48:28.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:28.792+0000 7fa4f3400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4f4109e90 0x7fa4f419fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:28.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:28.792+0000 7fa4ea000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4f4105760 0x7fa4f419b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:28.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:28.792+0000 7fa4f2a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa4f4104db0 0x7fa4f419b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:28.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:28.792+0000 7fa4f3400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4f4109e90 0x7fa4f419fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:60680/0 (socket says 172.21.15.151:60680) 2024-04-26T01:48:28.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:28.792+0000 7fa4f3400700 1 -- 172.21.15.151:0/1825424526 learned_addr learned my addr 172.21.15.151:0/1825424526 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:48:28.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:28.792+0000 7fa4f2a00700 1 -- 172.21.15.151:0/1825424526 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4f4109e90 msgr2=0x7fa4f419fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:28.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:28.792+0000 7fa4f2a00700 1 --2- 172.21.15.151:0/1825424526 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4f4109e90 0x7fa4f419fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:28.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:28.792+0000 7fa4f2a00700 1 -- 172.21.15.151:0/1825424526 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4f4105760 msgr2=0x7fa4f419b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:28.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:28.792+0000 7fa4f2a00700 1 --2- 172.21.15.151:0/1825424526 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4f4105760 0x7fa4f419b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:28.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:28.792+0000 7fa4f2a00700 1 -- 172.21.15.151:0/1825424526 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4f41a01d0 con 0x7fa4f4104db0 2024-04-26T01:48:28.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:28.792+0000 7fa4f3400700 1 --2- 172.21.15.151:0/1825424526 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4f4109e90 0x7fa4f419fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:48:28.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:28.792+0000 7fa4f2a00700 1 --2- 172.21.15.151:0/1825424526 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa4f4104db0 0x7fa4f419b1c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa4e400e9f0 tx=0x7fa4e400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:48:28.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:28.793+0000 7fa4ebe00700 1 -- 172.21.15.151:0/1825424526 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e400c9c0 con 0x7fa4f4104db0 2024-04-26T01:48:28.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:28.793+0000 7fa4ebe00700 1 -- 172.21.15.151:0/1825424526 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4e400cb60 con 0x7fa4f4104db0 2024-04-26T01:48:28.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:28.793+0000 7fa4ebe00700 1 -- 172.21.15.151:0/1825424526 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e4018680 con 0x7fa4f4104db0 2024-04-26T01:48:28.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:28.793+0000 7fa4f3e00700 1 -- 172.21.15.151:0/1825424526 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4f41a04c0 con 0x7fa4f4104db0 2024-04-26T01:48:28.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:28.793+0000 7fa4f3e00700 1 -- 172.21.15.151:0/1825424526 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4f41a7d40 con 0x7fa4f4104db0 2024-04-26T01:48:28.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:28.794+0000 7fa4f3e00700 1 -- 172.21.15.151:0/1825424526 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4f4069730 con 0x7fa4f4104db0 2024-04-26T01:48:28.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:28.796+0000 7fa4ebe00700 1 -- 172.21.15.151:0/1825424526 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fa4e4010430 con 0x7fa4f4104db0 2024-04-26T01:48:28.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:28.797+0000 7fa4ebe00700 1 --2- 172.21.15.151:0/1825424526 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fa4d80a3c40 0x7fa4d80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:28.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:28.798+0000 7fa4ebe00700 1 -- 172.21.15.151:0/1825424526 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fa4e40d0150 con 0x7fa4f4104db0 2024-04-26T01:48:28.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:28.798+0000 7fa4ea000700 1 --2- 172.21.15.151:0/1825424526 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fa4d80a3c40 0x7fa4d80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:28.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:28.798+0000 7fa4ea000700 1 --2- 172.21.15.151:0/1825424526 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fa4d80a3c40 0x7fa4d80a6100 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa4f410b3f0 tx=0x7fa4dc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:48:28.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:28.802+0000 7fa4ebe00700 1 -- 172.21.15.151:0/1825424526 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa4e409cf30 con 0x7fa4f4104db0 2024-04-26T01:48:29.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:29.029+0000 7fa4f3e00700 1 -- 172.21.15.151:0/1825424526 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa4f402df80 con 0x7fa4f4104db0 2024-04-26T01:48:29.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:29.031+0000 7fa4ebe00700 1 -- 172.21.15.151:0/1825424526 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+18496 (secure 0 0 0) 0x7fa4e409c830 con 0x7fa4f4104db0 2024-04-26T01:48:29.029 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:48:29.029 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":83,"fsid":"c0e49e64-036d-11ef-bc93-c7b262605968","created":"2024-04-26T01:40:07.483197+0000","modified":"2024-04-26T01:48:22.509335+0000","last_up_change":"2024-04-26T01:48:21.167691+0000","last_in_change":"2024-04-26T01:48:00.556990+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T01:44:11.411722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ca27e814-a982-482c-ae1a-e596aac5c59a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":669937998},{"type":"v1","addr":"172.21.15.151:6803","nonce":669937998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":669937998},{"type":"v1","addr":"172.21.15.151:6805","nonce":669937998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":669937998},{"type":"v1","addr":"172.21.15.151:6809","nonce":669937998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":669937998},{"type":"v1","addr":"172.21.15.151:6807","nonce":669937998}]},"public_addr":"172.21.15.151:6803/669937998","cluster_addr":"172.21.15.151:6805/669937998","heartbeat_back_addr":"172.21.15.151:6809/669937998","heartbeat_front_addr":"172.21.15.151:6807/669937998","state":["exists","up"]},{"osd":1,"uuid":"e7d192c3-9624-4625-9ccb-a76c114d8aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":1356168171},{"type":"v1","addr":"172.21.15.165:6801","nonce":1356168171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":1356168171},{"type":"v1","addr":"172.21.15.165:6803","nonce":1356168171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":1356168171},{"type":"v1","addr":"172.21.15.165:6807","nonce":1356168171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":1356168171},{"type":"v1","addr":"172.21.15.165:6805","nonce":1356168171}]},"public_addr":"172.21.15.165:6801/1356168171","cluster_addr":"172.21.15.165:6803/1356168171","heartbeat_back_addr":"172.21.15.165:6807/1356168171","heartbeat_front_addr":"172.21.15.165:6805/1356168171","state":["exists","up"]},{"osd":2,"uuid":"0631df59-1ecc-48b4-b00a-48b0118dfc24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":237502635},{"type":"v1","addr":"172.21.15.172:6801","nonce":237502635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":237502635},{"type":"v1","addr":"172.21.15.172:6803","nonce":237502635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":237502635},{"type":"v1","addr":"172.21.15.172:6807","nonce":237502635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":237502635},{"type":"v1","addr":"172.21.15.172:6805","nonce":237502635}]},"public_addr":"172.21.15.172:6801/237502635","cluster_addr":"172.21.15.172:6803/237502635","heartbeat_back_addr":"172.21.15.172:6807/237502635","heartbeat_front_addr":"172.21.15.172:6805/237502635","state":["exists","up"]},{"osd":3,"uuid":"73076e49-2f8a-4a3d-99eb-7b50aff448aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3442279476},{"type":"v1","addr":"172.21.15.151:6811","nonce":3442279476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3442279476},{"type":"v1","addr":"172.21.15.151:6813","nonce":3442279476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":3442279476},{"type":"v1","addr":"172.21.15.151:6817","nonce":3442279476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3442279476},{"type":"v1","addr":"172.21.15.151:6815","nonce":3442279476}]},"public_addr":"172.21.15.151:6811/3442279476","cluster_addr":"172.21.15.151:6813/3442279476","heartbeat_back_addr":"172.21.15.151:6817/3442279476","heartbeat_front_addr":"172.21.15.151:6815/3442279476","state":["exists","up"]},{"osd":4,"uuid":"0230d615-548c-43f0-b8aa-7130f4cbbc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4029980723},{"type":"v1","addr":"172.21.15.165:6809","nonce":4029980723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4029980723},{"type":"v1","addr":"172.21.15.165:6811","nonce":4029980723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4029980723},{"type":"v1","addr":"172.21.15.165:6815","nonce":4029980723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4029980723},{"type":"v1","addr":"172.21.15.165:6813","nonce":4029980723}]},"public_addr":"172.21.15.165:6809/4029980723","cluster_addr":"172.21.15.165:6811/4029980723","heartbeat_back_addr":"172.21.15.165:6815/4029980723","heartbeat_front_addr":"172.21.15.165:6813/4029980723","state":["exists","up"]},{"osd":5,"uuid":"9739f44b-2605-47dd-82fb-d30034742bf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2216578687},{"type":"v1","addr":"172.21.15.172:6809","nonce":2216578687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2216578687},{"type":"v1","addr":"172.21.15.172:6811","nonce":2216578687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2216578687},{"type":"v1","addr":"172.21.15.172:6815","nonce":2216578687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2216578687},{"type":"v1","addr":"172.21.15.172:6813","nonce":2216578687}]},"public_addr":"172.21.15.172:6809/2216578687","cluster_addr":"172.21.15.172:6811/2216578687","heartbeat_back_addr":"172.21.15.172:6815/2216578687","heartbeat_front_addr":"172.21.15.172:6813/2216578687","state":["exists","up"]},{"osd":6,"uuid":"4290baaa-6160-49cb-a4e0-c96f070f2ef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1723252315},{"type":"v1","addr":"172.21.15.151:6819","nonce":1723252315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1723252315},{"type":"v1","addr":"172.21.15.151:6821","nonce":1723252315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":1723252315},{"type":"v1","addr":"172.21.15.151:6825","nonce":1723252315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1723252315},{"type":"v1","addr":"172.21.15.151:6823","nonce":1723252315}]},"public_addr":"172.21.15.151:6819/1723252315","cluster_addr":"172.21.15.151:6821/1723252315","heartbeat_back_addr":"172.21.15.151:6825/1723252315","heartbeat_front_addr":"172.21.15.151:6823/1723252315","state":["exists","up"]},{"osd":7,"uuid":"9f5315f2-036c-4e2f-92ed-3321a9e3c860","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":1977803781},{"type":"v1","addr":"172.21.15.165:6817","nonce":1977803781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":1977803781},{"type":"v1","addr":"172.21.15.165:6819","nonce":1977803781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":1977803781},{"type":"v1","addr":"172.21.15.165:6823","nonce":1977803781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":1977803781},{"type":"v1","addr":"172.21.15.165:6821","nonce":1977803781}]},"public_addr":"172.21.15.165:6817/1977803781","cluster_addr":"172.21.15.165:6819/1977803781","heartbeat_back_addr":"172.21.15.165:6823/1977803781","heartbeat_front_addr":"172.21.15.165:6821/1977803781","state":["exists","up"]},{"osd":8,"uuid":"8ddbe36d-cea7-45ef-a3ca-044a0e64ff85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":491199512},{"type":"v1","addr":"172.21.15.172:6817","nonce":491199512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":491199512},{"type":"v1","addr":"172.21.15.172:6819","nonce":491199512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":491199512},{"type":"v1","addr":"172.21.15.172:6823","nonce":491199512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":491199512},{"type":"v1","addr":"172.21.15.172:6821","nonce":491199512}]},"public_addr":"172.21.15.172:6817/491199512","cluster_addr":"172.21.15.172:6819/491199512","heartbeat_back_addr":"172.21.15.172:6823/491199512","heartbeat_front_addr":"172.21.15.172:6821/491199512","state":["exists","up"]},{"osd":9,"uuid":"40654ba0-1bd9-4b54-97ae-f1ed6c4785d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":1818923828},{"type":"v1","addr":"172.21.15.151:6827","nonce":1818923828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":1818923828},{"type":"v1","addr":"172.21.15.151:6829","nonce":1818923828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1818923828},{"type":"v1","addr":"172.21.15.151:6833","nonce":1818923828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":1818923828},{"type":"v1","addr":"172.21.15.151:6831","nonce":1818923828}]},"public_addr":"172.21.15.151:6827/1818923828","cluster_addr":"172.21.15.151:6829/1818923828","heartbeat_back_addr":"172.21.15.151:6833/1818923828","heartbeat_front_addr":"172.21.15.151:6831/1818923828","state":["exists","up"]},{"osd":10,"uuid":"19e75a1a-7bd0-41d9-b0cc-0fa4d1475dda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":2632621809},{"type":"v1","addr":"172.21.15.165:6825","nonce":2632621809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":2632621809},{"type":"v1","addr":"172.21.15.165:6827","nonce":2632621809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":2632621809},{"type":"v1","addr":"172.21.15.165:6831","nonce":2632621809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":2632621809},{"type":"v1","addr":"172.21.15.165:6829","nonce":2632621809}]},"public_addr":"172.21.15.165:6825/2632621809","cluster_addr":"172.21.15.165:6827/2632621809","heartbeat_back_addr":"172.21.15.165:6831/2632621809","heartbeat_front_addr":"172.21.15.165:6829/2632621809","state":["exists","up"]},{"osd":11,"uuid":"aa84e5f5-93a3-4375-8544-a1175f0eb06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":886604365},{"type":"v1","addr":"172.21.15.172:6825","nonce":886604365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":886604365},{"type":"v1","addr":"172.21.15.172:6827","nonce":886604365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":886604365},{"type":"v1","addr":"172.21.15.172:6831","nonce":886604365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":886604365},{"type":"v1","addr":"172.21.15.172:6829","nonce":886604365}]},"public_addr":"172.21.15.172:6825/886604365","cluster_addr":"172.21.15.172:6827/886604365","heartbeat_back_addr":"172.21.15.172:6831/886604365","heartbeat_front_addr":"172.21.15.172:6829/886604365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:43:18.296278+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:43:42.860620+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:44:07.073601+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:44:39.631760+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:45:04.771214+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:45:29.606674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:46:03.101851+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:46:28.351995+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:46:53.445435+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:47:27.391179+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:47:53.543638+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:48:18.687811+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.151:6801/796314699":"2024-04-27T01:41:25.234322+0000","172.21.15.151:6800/796314699":"2024-04-27T01:41:25.234322+0000","172.21.15.151:0/1173525554":"2024-04-27T01:41:25.234322+0000","172.21.15.151:0/1342496801":"2024-04-27T01:40:50.492554+0000","172.21.15.151:0/11665926":"2024-04-27T01:40:50.492554+0000","172.21.15.151:0/1765448952":"2024-04-27T01:40:50.492554+0000","172.21.15.151:0/46007430":"2024-04-27T01:41:25.234322+0000","172.21.15.151:6800/244725060":"2024-04-27T01:40:50.492554+0000","172.21.15.151:0/3703862543":"2024-04-27T01:41:25.234322+0000","172.21.15.151:6801/244725060":"2024-04-27T01:40:50.492554+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T01:48:29.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:29.034+0000 7fa4f3e00700 1 -- 172.21.15.151:0/1825424526 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fa4d80a3c40 msgr2=0x7fa4d80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:29.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:29.034+0000 7fa4f3e00700 1 --2- 172.21.15.151:0/1825424526 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fa4d80a3c40 0x7fa4d80a6100 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa4f410b3f0 tx=0x7fa4dc009260 comp rx=0 tx=0).stop 2024-04-26T01:48:29.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:29.034+0000 7fa4f3e00700 1 -- 172.21.15.151:0/1825424526 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa4f4104db0 msgr2=0x7fa4f419b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:29.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:29.034+0000 7fa4f3e00700 1 --2- 172.21.15.151:0/1825424526 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa4f4104db0 0x7fa4f419b1c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa4e400e9f0 tx=0x7fa4e400ed00 comp rx=0 tx=0).stop 2024-04-26T01:48:29.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:29.034+0000 7fa4f3e00700 1 -- 172.21.15.151:0/1825424526 shutdown_connections 2024-04-26T01:48:29.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:29.034+0000 7fa4f3e00700 1 --2- 172.21.15.151:0/1825424526 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4f4109e90 0x7fa4f419fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:29.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:29.034+0000 7fa4f3e00700 1 --2- 172.21.15.151:0/1825424526 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa4f4104db0 0x7fa4f419b1c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:29.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:29.034+0000 7fa4f3e00700 1 --2- 172.21.15.151:0/1825424526 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fa4d80a3c40 0x7fa4d80a6100 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:29.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:29.034+0000 7fa4f3e00700 1 --2- 172.21.15.151:0/1825424526 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4f4105760 0x7fa4f419b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:29.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:29.034+0000 7fa4f3e00700 1 -- 172.21.15.151:0/1825424526 >> 172.21.15.151:0/1825424526 conn(0x7fa4f4100620 msgr2=0x7fa4f41028d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:48:29.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:29.034+0000 7fa4f3e00700 1 -- 172.21.15.151:0/1825424526 shutdown_connections 2024-04-26T01:48:29.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:29.034+0000 7fa4f3e00700 1 -- 172.21.15.151:0/1825424526 wait complete. 2024-04-26T01:48:29.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:29 smithi151 ceph-mon[28057]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:48:29.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:29 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/1825424526' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:48:29.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:29 smithi172 ceph-mon[32785]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:48:29.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:29 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/1825424526' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:48:29.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:29 smithi165 ceph-mon[32648]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:48:29.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:29 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/1825424526' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:48:29.886 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T01:44:11.411722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T01:48:29.887 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-26T01:48:30.330 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:48:31.519 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:31 smithi151 ceph-mon[28057]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:48:31.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:31 smithi172 ceph-mon[32785]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:48:31.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:31 smithi165 ceph-mon[32648]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:48:32.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:32.405+0000 7f1490000700 1 -- 172.21.15.151:0/3292585038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f148806b0f0 msgr2=0x7f148810f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:32.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:32.405+0000 7f1490000700 1 --2- 172.21.15.151:0/3292585038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f148806b0f0 0x7f148810f900 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f148000b0a0 tx=0x7f148000b3b0 comp rx=0 tx=0).stop 2024-04-26T01:48:32.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:32.405+0000 7f1490000700 1 -- 172.21.15.151:0/3292585038 shutdown_connections 2024-04-26T01:48:32.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:32.405+0000 7f1490000700 1 --2- 172.21.15.151:0/3292585038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f148806b0f0 0x7f148810f900 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:32.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:32.405+0000 7f1490000700 1 --2- 172.21.15.151:0/3292585038 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1488108a30 0x7f1488108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:32.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:32.405+0000 7f1490000700 1 --2- 172.21.15.151:0/3292585038 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f148806a750 0x7f148806abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:32.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:32.405+0000 7f1490000700 1 -- 172.21.15.151:0/3292585038 >> 172.21.15.151:0/3292585038 conn(0x7f14880788f0 msgr2=0x7f1488078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:48:32.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:32.405+0000 7f1490000700 1 -- 172.21.15.151:0/3292585038 shutdown_connections 2024-04-26T01:48:32.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:32.405+0000 7f1490000700 1 -- 172.21.15.151:0/3292585038 wait complete. 2024-04-26T01:48:32.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:32.406+0000 7f1490000700 1 Processor -- start 2024-04-26T01:48:32.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:32.407+0000 7f1490000700 1 -- start start 2024-04-26T01:48:32.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:32.407+0000 7f1490000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f148806a750 0x7f1488101a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:32.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:32.407+0000 7f1490000700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f148806b0f0 0x7f1488101f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:32.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:32.407+0000 7f1490000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1488108a30 0x7f1488102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:32.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:32.407+0000 7f1490000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1488112c60 con 0x7f148806a750 2024-04-26T01:48:32.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:32.407+0000 7f1490000700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1488112ae0 con 0x7f1488108a30 2024-04-26T01:48:32.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:32.407+0000 7f1490000700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1488112de0 con 0x7f148806b0f0 2024-04-26T01:48:32.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:32.407+0000 7f148d600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f148806a750 0x7f1488101a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:32.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:32.407+0000 7f148d600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f148806a750 0x7f1488101a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49924/0 (socket says 172.21.15.151:49924) 2024-04-26T01:48:32.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:32.407+0000 7f148d600700 1 -- 172.21.15.151:0/961413715 learned_addr learned my addr 172.21.15.151:0/961413715 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:48:32.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:32.408+0000 7f148cc00700 1 --2- 172.21.15.151:0/961413715 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f148806b0f0 0x7f1488101f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:32.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:32.408+0000 7f148e000700 1 --2- 172.21.15.151:0/961413715 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1488108a30 0x7f1488102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:32.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:32.408+0000 7f148d600700 1 -- 172.21.15.151:0/961413715 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f148806b0f0 msgr2=0x7f1488101f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:32.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:32.408+0000 7f148d600700 1 --2- 172.21.15.151:0/961413715 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f148806b0f0 0x7f1488101f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:32.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:32.408+0000 7f148d600700 1 -- 172.21.15.151:0/961413715 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1488108a30 msgr2=0x7f1488102490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:32.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:32.408+0000 7f148d600700 1 --2- 172.21.15.151:0/961413715 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1488108a30 0x7f1488102490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:32.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:32.408+0000 7f148d600700 1 -- 172.21.15.151:0/961413715 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14881a7720 con 0x7f148806a750 2024-04-26T01:48:32.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:32.408+0000 7f148e000700 1 --2- 172.21.15.151:0/961413715 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1488108a30 0x7f1488102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:48:32.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:32.408+0000 7f148d600700 1 --2- 172.21.15.151:0/961413715 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f148806a750 0x7f1488101a10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f147c00e9a0 tx=0x7f147c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:48:32.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:32.409+0000 7f1486000700 1 -- 172.21.15.151:0/961413715 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147c00ca10 con 0x7f148806a750 2024-04-26T01:48:32.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:32.409+0000 7f1486000700 1 -- 172.21.15.151:0/961413715 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f147c00cbb0 con 0x7f148806a750 2024-04-26T01:48:32.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:32.409+0000 7f1486000700 1 -- 172.21.15.151:0/961413715 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147c018680 con 0x7f148806a750 2024-04-26T01:48:32.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:32.409+0000 7f1490000700 1 -- 172.21.15.151:0/961413715 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14881a7a10 con 0x7f148806a750 2024-04-26T01:48:32.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:32.409+0000 7f1490000700 1 -- 172.21.15.151:0/961413715 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14881a7f00 con 0x7f148806a750 2024-04-26T01:48:32.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:32.410+0000 7f1490000700 1 -- 172.21.15.151:0/961413715 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1488069730 con 0x7f148806a750 2024-04-26T01:48:32.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:32.412+0000 7f1486000700 1 -- 172.21.15.151:0/961413715 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f147c010430 con 0x7f148806a750 2024-04-26T01:48:32.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:32.413+0000 7f1486000700 1 --2- 172.21.15.151:0/961413715 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f14700a3cc0 0x7f14700a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:32.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:32.414+0000 7f1486000700 1 -- 172.21.15.151:0/961413715 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f147c0d0090 con 0x7f148806a750 2024-04-26T01:48:32.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:32.414+0000 7f148cc00700 1 --2- 172.21.15.151:0/961413715 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f14700a3cc0 0x7f14700a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:32.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:32.414+0000 7f148cc00700 1 --2- 172.21.15.151:0/961413715 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f14700a3cc0 0x7f14700a6180 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1488078ed0 tx=0x7f1474009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:48:32.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:32.418+0000 7f1486000700 1 -- 172.21.15.151:0/961413715 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f147c09ce70 con 0x7f148806a750 2024-04-26T01:48:32.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:32.645+0000 7f1490000700 1 -- 172.21.15.151:0/961413715 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f148802e140 con 0x7f148806a750 2024-04-26T01:48:32.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:32.645+0000 7f1486000700 1 -- 172.21.15.151:0/961413715 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7f147c09c770 con 0x7f148806a750 2024-04-26T01:48:32.643 INFO:teuthology.orchestra.run.smithi151.stdout:pg_num: 1 2024-04-26T01:48:32.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:32.648+0000 7f1490000700 1 -- 172.21.15.151:0/961413715 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f14700a3cc0 msgr2=0x7f14700a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:32.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:32.648+0000 7f1490000700 1 --2- 172.21.15.151:0/961413715 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f14700a3cc0 0x7f14700a6180 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1488078ed0 tx=0x7f1474009260 comp rx=0 tx=0).stop 2024-04-26T01:48:32.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:32.648+0000 7f1490000700 1 -- 172.21.15.151:0/961413715 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f148806a750 msgr2=0x7f1488101a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:32.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:32.648+0000 7f1490000700 1 --2- 172.21.15.151:0/961413715 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f148806a750 0x7f1488101a10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f147c00e9a0 tx=0x7f147c00ecb0 comp rx=0 tx=0).stop 2024-04-26T01:48:32.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:32.648+0000 7f1490000700 1 -- 172.21.15.151:0/961413715 shutdown_connections 2024-04-26T01:48:32.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:32.648+0000 7f1490000700 1 --2- 172.21.15.151:0/961413715 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1488108a30 0x7f1488102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:32.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:32.648+0000 7f1490000700 1 --2- 172.21.15.151:0/961413715 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f14700a3cc0 0x7f14700a6180 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:32.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:32.648+0000 7f1490000700 1 --2- 172.21.15.151:0/961413715 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f148806a750 0x7f1488101a10 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:32.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:32.648+0000 7f1490000700 1 --2- 172.21.15.151:0/961413715 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f148806b0f0 0x7f1488101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:32.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:32.648+0000 7f1490000700 1 -- 172.21.15.151:0/961413715 >> 172.21.15.151:0/961413715 conn(0x7f14880788f0 msgr2=0x7f14880ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:48:32.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:32.649+0000 7f1490000700 1 -- 172.21.15.151:0/961413715 shutdown_connections 2024-04-26T01:48:32.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:32.649+0000 7f1490000700 1 -- 172.21.15.151:0/961413715 wait complete. 2024-04-26T01:48:33.358 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:33 smithi151 ceph-mon[28057]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:48:33.359 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:33 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/961413715' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T01:48:33.427 INFO:tasks.cephadm:Adding mds.a on smithi151 2024-04-26T01:48:33.427 INFO:tasks.cephadm:Adding mds.d on smithi151 2024-04-26T01:48:33.427 INFO:tasks.cephadm:Adding mds.g on smithi151 2024-04-26T01:48:33.427 INFO:tasks.cephadm:Adding mds.j on smithi151 2024-04-26T01:48:33.427 INFO:tasks.cephadm:Adding mds.b on smithi165 2024-04-26T01:48:33.427 INFO:tasks.cephadm:Adding mds.e on smithi165 2024-04-26T01:48:33.427 INFO:tasks.cephadm:Adding mds.h on smithi165 2024-04-26T01:48:33.427 INFO:tasks.cephadm:Adding mds.k on smithi165 2024-04-26T01:48:33.427 INFO:tasks.cephadm:Adding mds.c on smithi172 2024-04-26T01:48:33.427 INFO:tasks.cephadm:Adding mds.f on smithi172 2024-04-26T01:48:33.427 INFO:tasks.cephadm:Adding mds.i on smithi172 2024-04-26T01:48:33.427 INFO:tasks.cephadm:Adding mds.l on smithi172 2024-04-26T01:48:33.428 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi151=a;smithi151=d;smithi151=g;smithi151=j;smithi165=b;smithi165=e;smithi165=h;smithi165=k;smithi172=c;smithi172=f;smithi172=i;smithi172=l' 2024-04-26T01:48:33.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:33 smithi172 ceph-mon[32785]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:48:33.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:33 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/961413715' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T01:48:33.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:33 smithi165 ceph-mon[32648]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:48:33.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:33 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/961413715' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T01:48:33.676 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T01:48:35.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:35.244+0000 7f78f0bdf700 1 -- 172.21.15.172:0/754787874 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78ec102ba0 msgr2=0x7f78ec103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:35.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:35.244+0000 7f78f0bdf700 1 --2- 172.21.15.172:0/754787874 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78ec102ba0 0x7f78ec103000 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f78d4009990 tx=0x7f78d4009ca0 comp rx=0 tx=0).stop 2024-04-26T01:48:35.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:35.244+0000 7f78f0bdf700 1 -- 172.21.15.172:0/754787874 shutdown_connections 2024-04-26T01:48:35.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:35.244+0000 7f78f0bdf700 1 --2- 172.21.15.172:0/754787874 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78ec103540 0x7f78ec109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:35.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:35.244+0000 7f78f0bdf700 1 --2- 172.21.15.172:0/754787874 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f78ec1019a0 0x7f78ec101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:35.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:35.244+0000 7f78f0bdf700 1 --2- 172.21.15.172:0/754787874 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78ec102ba0 0x7f78ec103000 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:35.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:35.244+0000 7f78f0bdf700 1 -- 172.21.15.172:0/754787874 >> 172.21.15.172:0/754787874 conn(0x7f78ec0fcf60 msgr2=0x7f78ec0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:48:35.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:35.244+0000 7f78f0bdf700 1 -- 172.21.15.172:0/754787874 shutdown_connections 2024-04-26T01:48:35.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:35.244+0000 7f78f0bdf700 1 -- 172.21.15.172:0/754787874 wait complete. 2024-04-26T01:48:35.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:35.245+0000 7f78f0bdf700 1 Processor -- start 2024-04-26T01:48:35.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:35.245+0000 7f78f0bdf700 1 -- start start 2024-04-26T01:48:35.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:35.245+0000 7f78f0bdf700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78ec1019a0 0x7f78ec198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:35.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:35.245+0000 7f78f0bdf700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f78ec102ba0 0x7f78ec1991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:35.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:35.245+0000 7f78f0bdf700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78ec103540 0x7f78ec1a0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:35.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:35.245+0000 7f78f0bdf700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f78ec10d4d0 con 0x7f78ec102ba0 2024-04-26T01:48:35.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:35.245+0000 7f78f0bdf700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f78ec10d350 con 0x7f78ec103540 2024-04-26T01:48:35.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:35.245+0000 7f78f0bdf700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f78ec10d650 con 0x7f78ec1019a0 2024-04-26T01:48:35.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:35.245+0000 7f78ea59c700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78ec1019a0 0x7f78ec198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:35.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:35.245+0000 7f78ea59c700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78ec1019a0 0x7f78ec198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35630/0 (socket says 172.21.15.172:35630) 2024-04-26T01:48:35.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:35.245+0000 7f78ea59c700 1 -- 172.21.15.172:0/2553448715 learned_addr learned my addr 172.21.15.172:0/2553448715 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-26T01:48:35.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:35.246+0000 7f78ead9d700 1 --2- 172.21.15.172:0/2553448715 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78ec103540 0x7f78ec1a0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:35.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:35.246+0000 7f78ea59c700 1 -- 172.21.15.172:0/2553448715 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78ec103540 msgr2=0x7f78ec1a0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:35.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:35.246+0000 7f78ea59c700 1 --2- 172.21.15.172:0/2553448715 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78ec103540 0x7f78ec1a0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:35.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:35.246+0000 7f78ea59c700 1 -- 172.21.15.172:0/2553448715 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f78ec102ba0 msgr2=0x7f78ec1991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:35.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:35.246+0000 7f78ea59c700 1 --2- 172.21.15.172:0/2553448715 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f78ec102ba0 0x7f78ec1991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:35.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:35.246+0000 7f78ea59c700 1 -- 172.21.15.172:0/2553448715 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78ec1a0900 con 0x7f78ec1019a0 2024-04-26T01:48:35.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:35.246+0000 7f78ea59c700 1 --2- 172.21.15.172:0/2553448715 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78ec1019a0 0x7f78ec198c80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f78e400b9c0 tx=0x7f78e400bd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:48:35.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:35.246+0000 7f78e37fe700 1 -- 172.21.15.172:0/2553448715 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e4019070 con 0x7f78ec1019a0 2024-04-26T01:48:35.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:35.246+0000 7f78e37fe700 1 -- 172.21.15.172:0/2553448715 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78e4014440 con 0x7f78ec1019a0 2024-04-26T01:48:35.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:35.246+0000 7f78f0bdf700 1 -- 172.21.15.172:0/2553448715 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78ec1a0bf0 con 0x7f78ec1019a0 2024-04-26T01:48:35.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:35.246+0000 7f78e37fe700 1 -- 172.21.15.172:0/2553448715 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e40125a0 con 0x7f78ec1019a0 2024-04-26T01:48:35.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:35.246+0000 7f78f0bdf700 1 -- 172.21.15.172:0/2553448715 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78ec1a1130 con 0x7f78ec1019a0 2024-04-26T01:48:35.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:35.247+0000 7f78f0bdf700 1 -- 172.21.15.172:0/2553448715 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78ec069730 con 0x7f78ec1019a0 2024-04-26T01:48:35.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:35.248+0000 7f78e37fe700 1 -- 172.21.15.172:0/2553448715 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f78e401c070 con 0x7f78ec1019a0 2024-04-26T01:48:35.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:35.249+0000 7f78e37fe700 1 --2- 172.21.15.172:0/2553448715 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f78d80a3c40 0x7f78d80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:35.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:35.249+0000 7f78e37fe700 1 -- 172.21.15.172:0/2553448715 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f78e40cf390 con 0x7f78ec1019a0 2024-04-26T01:48:35.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:35.249+0000 7f78e9d9b700 1 --2- 172.21.15.172:0/2553448715 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f78d80a3c40 0x7f78d80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:35.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:35.250+0000 7f78e9d9b700 1 --2- 172.21.15.172:0/2553448715 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f78d80a3c40 0x7f78d80a6100 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f78d400b5c0 tx=0x7f78d4003860 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:48:35.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:35.254+0000 7f78e37fe700 1 -- 172.21.15.172:0/2553448715 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f78e409c210 con 0x7f78ec1019a0 2024-04-26T01:48:35.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:35 smithi172 ceph-mon[32785]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:48:35.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:35.462+0000 7f78f0bdf700 1 -- 172.21.15.172:0/2553448715 --> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi151=a;smithi151=d;smithi151=g;smithi151=j;smithi165=b;smithi165=e;smithi165=h;smithi165=k;smithi172=c;smithi172=f;smithi172=i;smithi172=l", "target": ["mon-mgr", ""]}) -- 0x7f78ec0ff260 con 0x7f78d80a3c40 2024-04-26T01:48:35.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:35.479+0000 7f78e37fe700 1 -- 172.21.15.172:0/2553448715 <== mgr.14152 v2:172.21.15.151:6800/2251256505 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f78ec0ff260 con 0x7f78d80a3c40 2024-04-26T01:48:35.477 INFO:teuthology.orchestra.run.smithi172.stdout:Scheduled mds.all update... 2024-04-26T01:48:35.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:35.481+0000 7f78f0bdf700 1 -- 172.21.15.172:0/2553448715 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f78d80a3c40 msgr2=0x7f78d80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:35.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:35.481+0000 7f78f0bdf700 1 --2- 172.21.15.172:0/2553448715 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f78d80a3c40 0x7f78d80a6100 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f78d400b5c0 tx=0x7f78d4003860 comp rx=0 tx=0).stop 2024-04-26T01:48:35.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:35.481+0000 7f78f0bdf700 1 -- 172.21.15.172:0/2553448715 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78ec1019a0 msgr2=0x7f78ec198c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:35.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:35.481+0000 7f78f0bdf700 1 --2- 172.21.15.172:0/2553448715 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78ec1019a0 0x7f78ec198c80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f78e400b9c0 tx=0x7f78e400bd80 comp rx=0 tx=0).stop 2024-04-26T01:48:35.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:35.482+0000 7f78f0bdf700 1 -- 172.21.15.172:0/2553448715 shutdown_connections 2024-04-26T01:48:35.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:35.482+0000 7f78f0bdf700 1 --2- 172.21.15.172:0/2553448715 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78ec103540 0x7f78ec1a0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:35.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:35.482+0000 7f78f0bdf700 1 --2- 172.21.15.172:0/2553448715 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78ec1019a0 0x7f78ec198c80 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:35.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:35.482+0000 7f78f0bdf700 1 --2- 172.21.15.172:0/2553448715 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f78d80a3c40 0x7f78d80a6100 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:35.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:35.482+0000 7f78f0bdf700 1 --2- 172.21.15.172:0/2553448715 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f78ec102ba0 0x7f78ec1991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:35.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:35.482+0000 7f78f0bdf700 1 -- 172.21.15.172:0/2553448715 >> 172.21.15.172:0/2553448715 conn(0x7f78ec0fcf60 msgr2=0x7f78ec0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:48:35.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:35.482+0000 7f78f0bdf700 1 -- 172.21.15.172:0/2553448715 shutdown_connections 2024-04-26T01:48:35.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-26T01:48:35.482+0000 7f78f0bdf700 1 -- 172.21.15.172:0/2553448715 wait complete. 2024-04-26T01:48:35.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:35 smithi165 ceph-mon[32648]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:48:35.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:35 smithi151 ceph-mon[28057]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:48:36.158 DEBUG:teuthology.orchestra.run.smithi151:mds.a> sudo journalctl -f -n 0 -u ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mds.a.service 2024-04-26T01:48:36.165 DEBUG:teuthology.orchestra.run.smithi151:mds.d> sudo journalctl -f -n 0 -u ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mds.d.service 2024-04-26T01:48:36.209 DEBUG:teuthology.orchestra.run.smithi151:mds.g> sudo journalctl -f -n 0 -u ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mds.g.service 2024-04-26T01:48:36.256 DEBUG:teuthology.orchestra.run.smithi151:mds.j> sudo journalctl -f -n 0 -u ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mds.j.service 2024-04-26T01:48:36.304 DEBUG:teuthology.orchestra.run.smithi165:mds.b> sudo journalctl -f -n 0 -u ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mds.b.service 2024-04-26T01:48:36.307 DEBUG:teuthology.orchestra.run.smithi165:mds.e> sudo journalctl -f -n 0 -u ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mds.e.service 2024-04-26T01:48:36.350 DEBUG:teuthology.orchestra.run.smithi165:mds.h> sudo journalctl -f -n 0 -u ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mds.h.service 2024-04-26T01:48:36.393 DEBUG:teuthology.orchestra.run.smithi165:mds.k> sudo journalctl -f -n 0 -u ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mds.k.service 2024-04-26T01:48:36.435 DEBUG:teuthology.orchestra.run.smithi172:mds.c> sudo journalctl -f -n 0 -u ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mds.c.service 2024-04-26T01:48:36.438 DEBUG:teuthology.orchestra.run.smithi172:mds.f> sudo journalctl -f -n 0 -u ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mds.f.service 2024-04-26T01:48:36.481 DEBUG:teuthology.orchestra.run.smithi172:mds.i> sudo journalctl -f -n 0 -u ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mds.i.service 2024-04-26T01:48:36.523 DEBUG:teuthology.orchestra.run.smithi172:mds.l> sudo journalctl -f -n 0 -u ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mds.l.service 2024-04-26T01:48:36.567 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-26T01:48:36.567 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:48:36.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:36 smithi165 ceph-mon[32648]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:48:36.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:36 smithi165 ceph-mon[32648]: from='client.24430 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi151=a;smithi151=d;smithi151=g;smithi151=j;smithi165=b;smithi165=e;smithi165=h;smithi165=k;smithi172=c;smithi172=f;smithi172=i;smithi172=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:48:36.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:36 smithi165 ceph-mon[32648]: Saving service mds.all spec with placement smithi151=a;smithi151=d;smithi151=g;smithi151=j;smithi165=b;smithi165=e;smithi165=h;smithi165=k;smithi172=c;smithi172=f;smithi172=i;smithi172=l;count:12 2024-04-26T01:48:36.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:36 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:36.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:36 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:48:36.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:36 smithi151 ceph-mon[28057]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:48:36.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:36 smithi151 ceph-mon[28057]: from='client.24430 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi151=a;smithi151=d;smithi151=g;smithi151=j;smithi165=b;smithi165=e;smithi165=h;smithi165=k;smithi172=c;smithi172=f;smithi172=i;smithi172=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:48:36.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:36 smithi151 ceph-mon[28057]: Saving service mds.all spec with placement smithi151=a;smithi151=d;smithi151=g;smithi151=j;smithi165=b;smithi165=e;smithi165=h;smithi165=k;smithi172=c;smithi172=f;smithi172=i;smithi172=l;count:12 2024-04-26T01:48:36.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:36 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:36.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:36 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:48:36.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:36 smithi172 ceph-mon[32785]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:48:36.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:36 smithi172 ceph-mon[32785]: from='client.24430 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi151=a;smithi151=d;smithi151=g;smithi151=j;smithi165=b;smithi165=e;smithi165=h;smithi165=k;smithi172=c;smithi172=f;smithi172=i;smithi172=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:48:36.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:36 smithi172 ceph-mon[32785]: Saving service mds.all spec with placement smithi151=a;smithi151=d;smithi151=g;smithi151=j;smithi165=b;smithi165=e;smithi165=h;smithi165=k;smithi172=c;smithi172=f;smithi172=i;smithi172=l;count:12 2024-04-26T01:48:36.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:36 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:36.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:36 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:48:36.998 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:48:37.660 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:37 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:48:37.660 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:37 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:48:37.661 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:37 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:37.661 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:37 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:48:37.661 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:37 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:48:37.661 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:37 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:48:37.661 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:37 smithi172 ceph-mon[32785]: Deploying daemon mds.l on smithi172 2024-04-26T01:48:37.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:37 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:48:37.847 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:37 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:48:37.847 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:37 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:37.847 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:37 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:48:37.847 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:37 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:48:37.847 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:37 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:48:37.847 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:37 smithi165 ceph-mon[32648]: Deploying daemon mds.l on smithi172 2024-04-26T01:48:37.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:37 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:48:37.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:37 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:48:37.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:37 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:37.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:37 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:48:37.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:37 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:48:37.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:37 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:48:37.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:37 smithi151 ceph-mon[28057]: Deploying daemon mds.l on smithi172 2024-04-26T01:48:38.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:38 smithi151 ceph-mon[28057]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:48:38.983 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:38 smithi172 ceph-mon[32785]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:48:38.984 INFO:journalctl@ceph.mds.l.smithi172.stdout:Apr 26 01:48:38 smithi172 systemd[1]: Starting Ceph mds.l for c0e49e64-036d-11ef-bc93-c7b262605968... 2024-04-26T01:48:39.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:38 smithi165 ceph-mon[32648]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:48:39.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:39.142+0000 7fbb8f800700 1 -- 172.21.15.151:0/2727035255 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb8806a750 msgr2=0x7fbb8806abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:39.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:39.142+0000 7fbb8f800700 1 --2- 172.21.15.151:0/2727035255 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb8806a750 0x7fbb8806abb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbb74009990 tx=0x7fbb74009ca0 comp rx=0 tx=0).stop 2024-04-26T01:48:39.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:39.142+0000 7fbb8f800700 1 -- 172.21.15.151:0/2727035255 shutdown_connections 2024-04-26T01:48:39.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:39.142+0000 7fbb8f800700 1 --2- 172.21.15.151:0/2727035255 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb8806b0f0 0x7fbb8810f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:39.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:39.142+0000 7fbb8f800700 1 --2- 172.21.15.151:0/2727035255 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb88108a30 0x7fbb88108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:39.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:39.142+0000 7fbb8f800700 1 --2- 172.21.15.151:0/2727035255 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb8806a750 0x7fbb8806abb0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:39.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:39.142+0000 7fbb8f800700 1 -- 172.21.15.151:0/2727035255 >> 172.21.15.151:0/2727035255 conn(0x7fbb880788f0 msgr2=0x7fbb88078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:48:39.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:39.142+0000 7fbb8f800700 1 -- 172.21.15.151:0/2727035255 shutdown_connections 2024-04-26T01:48:39.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:39.142+0000 7fbb8f800700 1 -- 172.21.15.151:0/2727035255 wait complete. 2024-04-26T01:48:39.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:39.143+0000 7fbb8f800700 1 Processor -- start 2024-04-26T01:48:39.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:39.143+0000 7fbb8f800700 1 -- start start 2024-04-26T01:48:39.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:39.144+0000 7fbb8f800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb8806a750 0x7fbb88101a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:39.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:39.144+0000 7fbb8f800700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb8806b0f0 0x7fbb88101f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:39.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:39.144+0000 7fbb8f800700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb88108a30 0x7fbb88102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:39.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:39.144+0000 7fbb8f800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbb88112c20 con 0x7fbb8806a750 2024-04-26T01:48:39.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:39.144+0000 7fbb8f800700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbb88112aa0 con 0x7fbb88108a30 2024-04-26T01:48:39.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:39.144+0000 7fbb8f800700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbb88112da0 con 0x7fbb8806b0f0 2024-04-26T01:48:39.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:39.144+0000 7fbb87e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb8806b0f0 0x7fbb88101f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:39.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:39.144+0000 7fbb8ce00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb8806a750 0x7fbb88101a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:39.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:39.144+0000 7fbb87e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb8806b0f0 0x7fbb88101f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:57290/0 (socket says 172.21.15.151:57290) 2024-04-26T01:48:39.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:39.144+0000 7fbb87e00700 1 -- 172.21.15.151:0/2056983671 learned_addr learned my addr 172.21.15.151:0/2056983671 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:48:39.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:39.144+0000 7fbb8d800700 1 --2- 172.21.15.151:0/2056983671 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb88108a30 0x7fbb88102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:39.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:39.144+0000 7fbb87e00700 1 -- 172.21.15.151:0/2056983671 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb88108a30 msgr2=0x7fbb88102490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:39.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:39.144+0000 7fbb87e00700 1 --2- 172.21.15.151:0/2056983671 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb88108a30 0x7fbb88102490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:39.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:39.145+0000 7fbb87e00700 1 -- 172.21.15.151:0/2056983671 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb8806a750 msgr2=0x7fbb88101a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:39.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:39.145+0000 7fbb87e00700 1 --2- 172.21.15.151:0/2056983671 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb8806a750 0x7fbb88101a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:39.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:39.145+0000 7fbb87e00700 1 -- 172.21.15.151:0/2056983671 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb881a7650 con 0x7fbb8806b0f0 2024-04-26T01:48:39.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:39.145+0000 7fbb87e00700 1 --2- 172.21.15.151:0/2056983671 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb8806b0f0 0x7fbb88101f50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbb74005910 tx=0x7fbb7400b670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:48:39.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:39.145+0000 7fbb85600700 1 -- 172.21.15.151:0/2056983671 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb74027070 con 0x7fbb8806b0f0 2024-04-26T01:48:39.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:39.145+0000 7fbb85600700 1 -- 172.21.15.151:0/2056983671 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb74022470 con 0x7fbb8806b0f0 2024-04-26T01:48:39.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:39.145+0000 7fbb8f800700 1 -- 172.21.15.151:0/2056983671 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb881a78e0 con 0x7fbb8806b0f0 2024-04-26T01:48:39.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:39.145+0000 7fbb85600700 1 -- 172.21.15.151:0/2056983671 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb74017650 con 0x7fbb8806b0f0 2024-04-26T01:48:39.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:39.145+0000 7fbb8f800700 1 -- 172.21.15.151:0/2056983671 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb881a7d90 con 0x7fbb8806b0f0 2024-04-26T01:48:39.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:39.147+0000 7fbb8f800700 1 -- 172.21.15.151:0/2056983671 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb88069730 con 0x7fbb8806b0f0 2024-04-26T01:48:39.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:39.149+0000 7fbb85600700 1 -- 172.21.15.151:0/2056983671 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fbb7402a070 con 0x7fbb8806b0f0 2024-04-26T01:48:39.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:39.150+0000 7fbb85600700 1 --2- 172.21.15.151:0/2056983671 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fbb700a3cc0 0x7fbb700a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:39.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:39.150+0000 7fbb85600700 1 -- 172.21.15.151:0/2056983671 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fbb740d0c70 con 0x7fbb8806b0f0 2024-04-26T01:48:39.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:39.150+0000 7fbb8ce00700 1 --2- 172.21.15.151:0/2056983671 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fbb700a3cc0 0x7fbb700a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:39.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:39.151+0000 7fbb8ce00700 1 --2- 172.21.15.151:0/2056983671 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fbb700a3cc0 0x7fbb700a6180 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c009550 tx=0x7fbb7c0091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:48:39.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:39.154+0000 7fbb85600700 1 -- 172.21.15.151:0/2056983671 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbb74013070 con 0x7fbb8806b0f0 2024-04-26T01:48:39.246 INFO:journalctl@ceph.mds.l.smithi172.stdout:Apr 26 01:48:38 smithi172 podman[62860]: 2024-04-26 01:48:38.888764469 +0000 UTC m=+0.017982096 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T01:48:39.246 INFO:journalctl@ceph.mds.l.smithi172.stdout:Apr 26 01:48:39 smithi172 podman[62860]: 2024-04-26 01:48:39.002549644 +0000 UTC m=+0.131767330 container create 5ca0c9a91dcbda1ff7f60396ddd30053c23fc86d50744793d56726bfb3ed0bcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-l, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-26T01:48:39.437 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:39.438+0000 7fbb8f800700 1 -- 172.21.15.151:0/2056983671 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbb8802df80 con 0x7fbb8806b0f0 2024-04-26T01:48:39.437 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:39.439+0000 7fbb85600700 1 -- 172.21.15.151:0/2056983671 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7fbb7409dae0 con 0x7fbb8806b0f0 2024-04-26T01:48:39.437 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:48:39.438 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1207580015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1207580015},{"type":"v1","addr":"172.21.15.172:6833","nonce":1207580015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-26T01:48:39.441 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:39.442+0000 7fbb8f800700 1 -- 172.21.15.151:0/2056983671 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fbb700a3cc0 msgr2=0x7fbb700a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:39.441 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:39.442+0000 7fbb8f800700 1 --2- 172.21.15.151:0/2056983671 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fbb700a3cc0 0x7fbb700a6180 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c009550 tx=0x7fbb7c0091b0 comp rx=0 tx=0).stop 2024-04-26T01:48:39.441 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:39.443+0000 7fbb8f800700 1 -- 172.21.15.151:0/2056983671 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb8806b0f0 msgr2=0x7fbb88101f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:39.441 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:39.443+0000 7fbb8f800700 1 --2- 172.21.15.151:0/2056983671 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb8806b0f0 0x7fbb88101f50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbb74005910 tx=0x7fbb7400b670 comp rx=0 tx=0).stop 2024-04-26T01:48:39.441 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:39.443+0000 7fbb8f800700 1 -- 172.21.15.151:0/2056983671 shutdown_connections 2024-04-26T01:48:39.442 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:39.443+0000 7fbb8f800700 1 --2- 172.21.15.151:0/2056983671 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb88108a30 0x7fbb88102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:39.442 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:39.443+0000 7fbb8f800700 1 --2- 172.21.15.151:0/2056983671 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fbb700a3cc0 0x7fbb700a6180 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:39.442 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:39.443+0000 7fbb8f800700 1 --2- 172.21.15.151:0/2056983671 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb8806a750 0x7fbb88101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:39.442 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:39.443+0000 7fbb8f800700 1 --2- 172.21.15.151:0/2056983671 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb8806b0f0 0x7fbb88101f50 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:39.442 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:39.443+0000 7fbb8f800700 1 -- 172.21.15.151:0/2056983671 >> 172.21.15.151:0/2056983671 conn(0x7fbb880788f0 msgr2=0x7fbb880ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:48:39.442 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:39.443+0000 7fbb8f800700 1 -- 172.21.15.151:0/2056983671 shutdown_connections 2024-04-26T01:48:39.442 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:39.444+0000 7fbb8f800700 1 -- 172.21.15.151:0/2056983671 wait complete. 2024-04-26T01:48:39.443 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 2 2024-04-26T01:48:39.536 INFO:journalctl@ceph.mds.l.smithi172.stdout:Apr 26 01:48:39 smithi172 podman[62860]: 2024-04-26 01:48:39.248639851 +0000 UTC m=+0.377857478 container init 5ca0c9a91dcbda1ff7f60396ddd30053c23fc86d50744793d56726bfb3ed0bcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-l, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=) 2024-04-26T01:48:39.536 INFO:journalctl@ceph.mds.l.smithi172.stdout:Apr 26 01:48:39 smithi172 podman[62860]: 2024-04-26 01:48:39.252228635 +0000 UTC m=+0.381446263 container start 5ca0c9a91dcbda1ff7f60396ddd30053c23fc86d50744793d56726bfb3ed0bcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-26T01:48:39.536 INFO:journalctl@ceph.mds.l.smithi172.stdout:Apr 26 01:48:39 smithi172 ceph-mds[62879]: -- 172.21.15.172:0/1767924305 <== mon.1 v2:172.21.15.172:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5567295ffa40 con 0x5567295d3000 2024-04-26T01:48:39.536 INFO:journalctl@ceph.mds.l.smithi172.stdout:Apr 26 01:48:39 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-l[62875]: starting mds.l at 2024-04-26T01:48:39.536 INFO:journalctl@ceph.mds.l.smithi172.stdout:Apr 26 01:48:39 smithi172 bash[62860]: 5ca0c9a91dcbda1ff7f60396ddd30053c23fc86d50744793d56726bfb3ed0bcb 2024-04-26T01:48:39.537 INFO:journalctl@ceph.mds.l.smithi172.stdout:Apr 26 01:48:39 smithi172 systemd[1]: Started Ceph mds.l for c0e49e64-036d-11ef-bc93-c7b262605968. 2024-04-26T01:48:40.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:40 smithi165 ceph-mon[32648]: mds.? [v2:172.21.15.172:6832/1207580015,v1:172.21.15.172:6833/1207580015] up:boot 2024-04-26T01:48:40.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:40 smithi165 ceph-mon[32648]: fsmap 1 up:standby 2024-04-26T01:48:40.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:40 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T01:48:40.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:40 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T01:48:40.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:40 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:40.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:40 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:40.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:40 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:40.347 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:40 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/2056983671' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:48:40.347 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:40 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:48:40.347 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:40 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:48:40.347 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:40 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:48:40.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:40 smithi172 ceph-mon[32785]: mds.? [v2:172.21.15.172:6832/1207580015,v1:172.21.15.172:6833/1207580015] up:boot 2024-04-26T01:48:40.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:40 smithi172 ceph-mon[32785]: fsmap 1 up:standby 2024-04-26T01:48:40.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:40 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T01:48:40.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:40 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T01:48:40.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:40 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:40.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:40 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:40.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:40 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:40.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:40 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/2056983671' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:48:40.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:40 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:48:40.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:40 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:48:40.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:40 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:48:40.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:40 smithi151 ceph-mon[28057]: mds.? [v2:172.21.15.172:6832/1207580015,v1:172.21.15.172:6833/1207580015] up:boot 2024-04-26T01:48:40.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:40 smithi151 ceph-mon[28057]: fsmap 1 up:standby 2024-04-26T01:48:40.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:40 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T01:48:40.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:40 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T01:48:40.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:40 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:40.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:40 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:40.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:40 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:40.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:40 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/2056983671' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:48:40.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:40 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:48:40.623 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:40 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:48:40.623 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:40 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:48:41.343 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:41 smithi165 ceph-mon[32648]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:48:41.343 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:41 smithi165 ceph-mon[32648]: Deploying daemon mds.k on smithi165 2024-04-26T01:48:41.598 INFO:journalctl@ceph.mds.k.smithi165.stdout:Apr 26 01:48:41 smithi165 systemd[1]: Starting Ceph mds.k for c0e49e64-036d-11ef-bc93-c7b262605968... 2024-04-26T01:48:41.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:41 smithi151 ceph-mon[28057]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:48:41.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:41 smithi151 ceph-mon[28057]: Deploying daemon mds.k on smithi165 2024-04-26T01:48:41.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:41 smithi172 ceph-mon[32785]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:48:41.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:41 smithi172 ceph-mon[32785]: Deploying daemon mds.k on smithi165 2024-04-26T01:48:41.984 INFO:journalctl@ceph.mds.k.smithi165.stdout:Apr 26 01:48:41 smithi165 podman[62366]: 2024-04-26 01:48:41.726170984 +0000 UTC m=+0.101409664 container create 78f7d2a054140649cfde348c3cda960f8a3b7a872c5a3de0dbf0cd0df958023b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-k, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-26T01:48:41.984 INFO:journalctl@ceph.mds.k.smithi165.stdout:Apr 26 01:48:41 smithi165 podman[62366]: 2024-04-26 01:48:41.642490095 +0000 UTC m=+0.017728794 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T01:48:41.984 INFO:journalctl@ceph.mds.k.smithi165.stdout:Apr 26 01:48:41 smithi165 podman[62366]: 2024-04-26 01:48:41.987103465 +0000 UTC m=+0.362342146 container init 78f7d2a054140649cfde348c3cda960f8a3b7a872c5a3de0dbf0cd0df958023b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-k, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-26T01:48:42.227 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:48:42.346 INFO:journalctl@ceph.mds.k.smithi165.stdout:Apr 26 01:48:41 smithi165 podman[62366]: 2024-04-26 01:48:41.990060663 +0000 UTC m=+0.365299344 container start 78f7d2a054140649cfde348c3cda960f8a3b7a872c5a3de0dbf0cd0df958023b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-26T01:48:42.346 INFO:journalctl@ceph.mds.k.smithi165.stdout:Apr 26 01:48:42 smithi165 ceph-mds[62385]: -- 172.21.15.165:0/2152015285 <== mon.2 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561a8a851a40 con 0x561a8a824c00 2024-04-26T01:48:42.346 INFO:journalctl@ceph.mds.k.smithi165.stdout:Apr 26 01:48:42 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-k[62381]: starting mds.k at 2024-04-26T01:48:42.346 INFO:journalctl@ceph.mds.k.smithi165.stdout:Apr 26 01:48:42 smithi165 bash[62366]: 78f7d2a054140649cfde348c3cda960f8a3b7a872c5a3de0dbf0cd0df958023b 2024-04-26T01:48:42.346 INFO:journalctl@ceph.mds.k.smithi165.stdout:Apr 26 01:48:42 smithi165 systemd[1]: Started Ceph mds.k for c0e49e64-036d-11ef-bc93-c7b262605968. 2024-04-26T01:48:42.657 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:48:43.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:43 smithi151 ceph-mon[28057]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:48:43.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:43 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:43.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:43 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:43.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:43 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:43.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:43 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:48:43.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:43 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:48:43.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:43 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:48:43.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:43 smithi151 ceph-mon[28057]: Deploying daemon mds.j on smithi151 2024-04-26T01:48:43.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:43 smithi151 ceph-mon[28057]: mds.? [v2:172.21.15.165:6832/504675912,v1:172.21.15.165:6833/504675912] up:boot 2024-04-26T01:48:43.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:43 smithi151 ceph-mon[28057]: fsmap 2 up:standby 2024-04-26T01:48:43.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:43 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T01:48:43.386 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:43 smithi165 ceph-mon[32648]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:48:43.386 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:43 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:43.386 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:43 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:43.386 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:43 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:43.387 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:43 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:48:43.387 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:43 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:48:43.387 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:43 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:48:43.387 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:43 smithi165 ceph-mon[32648]: Deploying daemon mds.j on smithi151 2024-04-26T01:48:43.387 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:43 smithi165 ceph-mon[32648]: mds.? [v2:172.21.15.165:6832/504675912,v1:172.21.15.165:6833/504675912] up:boot 2024-04-26T01:48:43.387 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:43 smithi165 ceph-mon[32648]: fsmap 2 up:standby 2024-04-26T01:48:43.387 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:43 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T01:48:43.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:43 smithi172 ceph-mon[32785]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:48:43.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:43 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:43.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:43 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:43.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:43 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:43.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:43 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:48:43.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:43 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:48:43.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:43 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:48:43.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:43 smithi172 ceph-mon[32785]: Deploying daemon mds.j on smithi151 2024-04-26T01:48:43.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:43 smithi172 ceph-mon[32785]: mds.? [v2:172.21.15.165:6832/504675912,v1:172.21.15.165:6833/504675912] up:boot 2024-04-26T01:48:43.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:43 smithi172 ceph-mon[32785]: fsmap 2 up:standby 2024-04-26T01:48:43.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:43 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T01:48:45.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:45 smithi151 ceph-mon[28057]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:48:45.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:45 smithi172 ceph-mon[32785]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:48:45.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:45 smithi165 ceph-mon[32648]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:48:45.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:45.375+0000 7f5379600700 1 -- 172.21.15.151:0/1546301642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f537410f320 msgr2=0x7f5374111710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:45.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:45.375+0000 7f5379600700 1 --2- 172.21.15.151:0/1546301642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f537410f320 0x7f5374111710 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f536c009f90 tx=0x7f536c00b330 comp rx=0 tx=0).stop 2024-04-26T01:48:45.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:45.375+0000 7f5379600700 1 -- 172.21.15.151:0/1546301642 shutdown_connections 2024-04-26T01:48:45.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:45.375+0000 7f5379600700 1 --2- 172.21.15.151:0/1546301642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f537410f320 0x7f5374111710 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:45.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:45.375+0000 7f5379600700 1 --2- 172.21.15.151:0/1546301642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5374100de0 0x7f53741011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:45.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:45.375+0000 7f5379600700 1 --2- 172.21.15.151:0/1546301642 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5374101700 0x7f537410ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:45.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:45.375+0000 7f5379600700 1 -- 172.21.15.151:0/1546301642 >> 172.21.15.151:0/1546301642 conn(0x7f53740766e0 msgr2=0x7f5374076af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:48:45.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:45.375+0000 7f5379600700 1 -- 172.21.15.151:0/1546301642 shutdown_connections 2024-04-26T01:48:45.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:45.375+0000 7f5379600700 1 -- 172.21.15.151:0/1546301642 wait complete. 2024-04-26T01:48:45.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:45.376+0000 7f5379600700 1 Processor -- start 2024-04-26T01:48:45.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:45.376+0000 7f5379600700 1 -- start start 2024-04-26T01:48:45.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:45.377+0000 7f5379600700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5374100de0 0x7f537419b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:45.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:45.377+0000 7f5379600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5374101700 0x7f537419b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:45.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:45.377+0000 7f5379600700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f537410f320 0x7f537419fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:45.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:45.377+0000 7f5379600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5374114db0 con 0x7f5374101700 2024-04-26T01:48:45.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:45.377+0000 7f5379600700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5374114c30 con 0x7f5374100de0 2024-04-26T01:48:45.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:45.377+0000 7f5379600700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5374114f30 con 0x7f537410f320 2024-04-26T01:48:45.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:45.377+0000 7f5373400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5374101700 0x7f537419b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:45.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:45.377+0000 7f5373400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5374101700 0x7f537419b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:38614/0 (socket says 172.21.15.151:38614) 2024-04-26T01:48:45.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:45.377+0000 7f5373400700 1 -- 172.21.15.151:0/1376741245 learned_addr learned my addr 172.21.15.151:0/1376741245 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:48:45.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:45.377+0000 7f5378c00700 1 --2- 172.21.15.151:0/1376741245 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f537410f320 0x7f537419fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:45.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:45.377+0000 7f5373e00700 1 --2- 172.21.15.151:0/1376741245 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5374100de0 0x7f537419b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:45.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:45.377+0000 7f5373400700 1 -- 172.21.15.151:0/1376741245 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f537410f320 msgr2=0x7f537419fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:45.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:45.377+0000 7f5373400700 1 --2- 172.21.15.151:0/1376741245 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f537410f320 0x7f537419fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:45.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:45.377+0000 7f5373400700 1 -- 172.21.15.151:0/1376741245 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5374100de0 msgr2=0x7f537419b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:45.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:45.377+0000 7f5373400700 1 --2- 172.21.15.151:0/1376741245 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5374100de0 0x7f537419b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:45.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:45.377+0000 7f5373400700 1 -- 172.21.15.151:0/1376741245 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53741a0260 con 0x7f5374101700 2024-04-26T01:48:45.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:45.378+0000 7f5373400700 1 --2- 172.21.15.151:0/1376741245 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5374101700 0x7f537419b7e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f536000eb30 tx=0x7f536000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:48:45.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:45.378+0000 7f5370c00700 1 -- 172.21.15.151:0/1376741245 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536000cbd0 con 0x7f5374101700 2024-04-26T01:48:45.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:45.378+0000 7f5370c00700 1 -- 172.21.15.151:0/1376741245 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53600044d0 con 0x7f5374101700 2024-04-26T01:48:45.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:45.378+0000 7f5379600700 1 -- 172.21.15.151:0/1376741245 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53741a0550 con 0x7f5374101700 2024-04-26T01:48:45.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:45.378+0000 7f5370c00700 1 -- 172.21.15.151:0/1376741245 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5360003e30 con 0x7f5374101700 2024-04-26T01:48:45.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:45.378+0000 7f5379600700 1 -- 172.21.15.151:0/1376741245 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53741a7dd0 con 0x7f5374101700 2024-04-26T01:48:45.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:45.379+0000 7f5379600700 1 -- 172.21.15.151:0/1376741245 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5374069730 con 0x7f5374101700 2024-04-26T01:48:45.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:45.382+0000 7f5370c00700 1 -- 172.21.15.151:0/1376741245 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f5360004000 con 0x7f5374101700 2024-04-26T01:48:45.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:45.383+0000 7f5370c00700 1 --2- 172.21.15.151:0/1376741245 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f53580a3c40 0x7f53580a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:45.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:45.383+0000 7f5370c00700 1 -- 172.21.15.151:0/1376741245 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f53600d0530 con 0x7f5374101700 2024-04-26T01:48:45.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:45.383+0000 7f5373e00700 1 --2- 172.21.15.151:0/1376741245 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f53580a3c40 0x7f53580a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:45.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:45.384+0000 7f5373e00700 1 --2- 172.21.15.151:0/1376741245 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f53580a3c40 0x7f53580a6100 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5364009750 tx=0x7f5364009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:48:45.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:45.388+0000 7f5370c00700 1 -- 172.21.15.151:0/1376741245 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5360053ff0 con 0x7f5374101700 2024-04-26T01:48:45.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:45.666+0000 7f5379600700 1 -- 172.21.15.151:0/1376741245 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f537402df80 con 0x7f5374101700 2024-04-26T01:48:45.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:45.667+0000 7f5370c00700 1 -- 172.21.15.151:0/1376741245 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f53600042f0 con 0x7f5374101700 2024-04-26T01:48:45.665 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:48:45.666 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/504675912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":504675912},{"type":"v1","addr":"172.21.15.165:6833","nonce":504675912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1207580015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1207580015},{"type":"v1","addr":"172.21.15.172:6833","nonce":1207580015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-26T01:48:45.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:45.671+0000 7f5379600700 1 -- 172.21.15.151:0/1376741245 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f53580a3c40 msgr2=0x7f53580a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:45.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:45.671+0000 7f5379600700 1 --2- 172.21.15.151:0/1376741245 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f53580a3c40 0x7f53580a6100 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5364009750 tx=0x7f5364009310 comp rx=0 tx=0).stop 2024-04-26T01:48:45.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:45.671+0000 7f5379600700 1 -- 172.21.15.151:0/1376741245 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5374101700 msgr2=0x7f537419b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:45.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:45.671+0000 7f5379600700 1 --2- 172.21.15.151:0/1376741245 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5374101700 0x7f537419b7e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f536000eb30 tx=0x7f536000ee40 comp rx=0 tx=0).stop 2024-04-26T01:48:45.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:45.671+0000 7f5379600700 1 -- 172.21.15.151:0/1376741245 shutdown_connections 2024-04-26T01:48:45.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:45.671+0000 7f5379600700 1 --2- 172.21.15.151:0/1376741245 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f537410f320 0x7f537419fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:45.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:45.671+0000 7f5379600700 1 --2- 172.21.15.151:0/1376741245 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5374100de0 0x7f537419b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:45.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:45.671+0000 7f5379600700 1 --2- 172.21.15.151:0/1376741245 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f53580a3c40 0x7f53580a6100 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:45.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:45.671+0000 7f5379600700 1 --2- 172.21.15.151:0/1376741245 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5374101700 0x7f537419b7e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:45.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:45.671+0000 7f5379600700 1 -- 172.21.15.151:0/1376741245 >> 172.21.15.151:0/1376741245 conn(0x7f53740766e0 msgr2=0x7f537410f920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:48:45.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:45.672+0000 7f5379600700 1 -- 172.21.15.151:0/1376741245 shutdown_connections 2024-04-26T01:48:45.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:45.672+0000 7f5379600700 1 -- 172.21.15.151:0/1376741245 wait complete. 2024-04-26T01:48:45.671 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 3 2024-04-26T01:48:46.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:46 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/1376741245' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:48:46.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:46 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/1376741245' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:48:46.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:46 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/1376741245' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:48:47.079 INFO:journalctl@ceph.mds.j.smithi151.stdout:Apr 26 01:48:47 smithi151 podman[68302]: 2024-04-26 01:48:47.079877073 +0000 UTC m=+0.402436698 container init c1527278cf053d24301c64a0d57caef8e83305ebe400151b64e61695a831c8dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-j, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-26T01:48:47.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:47 smithi151 ceph-mon[28057]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:48:47.371 INFO:journalctl@ceph.mds.j.smithi151.stdout:Apr 26 01:48:47 smithi151 podman[68302]: 2024-04-26 01:48:47.092161383 +0000 UTC m=+0.414720973 container start c1527278cf053d24301c64a0d57caef8e83305ebe400151b64e61695a831c8dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-j, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240416) 2024-04-26T01:48:47.372 INFO:journalctl@ceph.mds.j.smithi151.stdout:Apr 26 01:48:47 smithi151 ceph-mds[68321]: -- 172.21.15.151:0/1797149260 <== mon.0 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5650dcc47a40 con 0x5650dcc1a800 2024-04-26T01:48:47.372 INFO:journalctl@ceph.mds.j.smithi151.stdout:Apr 26 01:48:47 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-j[68317]: starting mds.j at 2024-04-26T01:48:47.372 INFO:journalctl@ceph.mds.j.smithi151.stdout:Apr 26 01:48:47 smithi151 bash[68302]: c1527278cf053d24301c64a0d57caef8e83305ebe400151b64e61695a831c8dd 2024-04-26T01:48:47.372 INFO:journalctl@ceph.mds.j.smithi151.stdout:Apr 26 01:48:47 smithi151 systemd[1]: Started Ceph mds.j for c0e49e64-036d-11ef-bc93-c7b262605968. 2024-04-26T01:48:47.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:47 smithi172 ceph-mon[32785]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:48:47.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:47 smithi165 ceph-mon[32648]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:48:48.528 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:48:48.573 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:48 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:48.574 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:48 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:48.574 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:48 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:48.574 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:48 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:48:48.574 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:48 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:48:48.574 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:48 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:48:48.574 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:48 smithi151 ceph-mon[28057]: Deploying daemon mds.d on smithi151 2024-04-26T01:48:48.574 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:48 smithi151 ceph-mon[28057]: mds.? [v2:172.21.15.151:6834/2937864097,v1:172.21.15.151:6835/2937864097] up:boot 2024-04-26T01:48:48.574 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:48 smithi151 ceph-mon[28057]: fsmap 3 up:standby 2024-04-26T01:48:48.574 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:48 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T01:48:48.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:48 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:48.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:48 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:48.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:48 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:48.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:48 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:48:48.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:48 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:48:48.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:48 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:48:48.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:48 smithi165 ceph-mon[32648]: Deploying daemon mds.d on smithi151 2024-04-26T01:48:48.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:48 smithi165 ceph-mon[32648]: mds.? [v2:172.21.15.151:6834/2937864097,v1:172.21.15.151:6835/2937864097] up:boot 2024-04-26T01:48:48.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:48 smithi165 ceph-mon[32648]: fsmap 3 up:standby 2024-04-26T01:48:48.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:48 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T01:48:48.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:48 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:48.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:48 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:48.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:48 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:48.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:48 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:48:48.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:48 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:48:48.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:48 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:48:48.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:48 smithi172 ceph-mon[32785]: Deploying daemon mds.d on smithi151 2024-04-26T01:48:48.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:48 smithi172 ceph-mon[32785]: mds.? [v2:172.21.15.151:6834/2937864097,v1:172.21.15.151:6835/2937864097] up:boot 2024-04-26T01:48:48.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:48 smithi172 ceph-mon[32785]: fsmap 3 up:standby 2024-04-26T01:48:48.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:48 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T01:48:49.545 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:49 smithi151 ceph-mon[28057]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:48:49.558 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:48:49.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:49 smithi165 ceph-mon[32648]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:48:49.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:49 smithi172 ceph-mon[32785]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:48:50.732 INFO:journalctl@ceph.mds.d.smithi151.stdout:Apr 26 01:48:50 smithi151 systemd[1]: Starting Ceph mds.d for c0e49e64-036d-11ef-bc93-c7b262605968... 2024-04-26T01:48:51.511 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:51 smithi151 ceph-mon[28057]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:48:51.511 INFO:journalctl@ceph.mds.d.smithi151.stdout:Apr 26 01:48:51 smithi151 podman[68701]: 2024-04-26 01:48:51.269084977 +0000 UTC m=+0.275833351 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T01:48:51.512 INFO:journalctl@ceph.mds.d.smithi151.stdout:Apr 26 01:48:51 smithi151 podman[68701]: 2024-04-26 01:48:51.383574356 +0000 UTC m=+0.390322762 container create 8d353b91b12ce3434bc347d5b2e0ab58bbf7a08573c8d5fe0091911780d25f8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-26T01:48:51.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:51 smithi165 ceph-mon[32648]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:48:51.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:51 smithi172 ceph-mon[32785]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:48:51.845 INFO:journalctl@ceph.mds.d.smithi151.stdout:Apr 26 01:48:51 smithi151 podman[68701]: 2024-04-26 01:48:51.739237471 +0000 UTC m=+0.745985889 container init 8d353b91b12ce3434bc347d5b2e0ab58bbf7a08573c8d5fe0091911780d25f8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-d, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T01:48:51.845 INFO:journalctl@ceph.mds.d.smithi151.stdout:Apr 26 01:48:51 smithi151 podman[68701]: 2024-04-26 01:48:51.748436939 +0000 UTC m=+0.755185334 container start 8d353b91b12ce3434bc347d5b2e0ab58bbf7a08573c8d5fe0091911780d25f8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-d, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-26T01:48:51.845 INFO:journalctl@ceph.mds.d.smithi151.stdout:Apr 26 01:48:51 smithi151 ceph-mds[68740]: -- 172.21.15.151:0/1522802425 <== mon.0 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f9d639fa40 con 0x55f9d6372c00 2024-04-26T01:48:51.845 INFO:journalctl@ceph.mds.d.smithi151.stdout:Apr 26 01:48:51 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-d[68735]: starting mds.d at 2024-04-26T01:48:52.106 INFO:journalctl@ceph.mds.d.smithi151.stdout:Apr 26 01:48:51 smithi151 bash[68701]: 8d353b91b12ce3434bc347d5b2e0ab58bbf7a08573c8d5fe0091911780d25f8a 2024-04-26T01:48:52.107 INFO:journalctl@ceph.mds.d.smithi151.stdout:Apr 26 01:48:51 smithi151 systemd[1]: Started Ceph mds.d for c0e49e64-036d-11ef-bc93-c7b262605968. 2024-04-26T01:48:52.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:52.455+0000 7f007f200700 1 -- 172.21.15.151:0/959878143 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0078108a20 msgr2=0x7f0078108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:52.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:52.455+0000 7f007f200700 1 --2- 172.21.15.151:0/959878143 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0078108a20 0x7f0078108e00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f006800b0a0 tx=0x7f006800b3b0 comp rx=0 tx=0).stop 2024-04-26T01:48:52.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:52.455+0000 7f007f200700 1 -- 172.21.15.151:0/959878143 shutdown_connections 2024-04-26T01:48:52.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:52.455+0000 7f007f200700 1 --2- 172.21.15.151:0/959878143 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f007806b980 0x7f007810f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:52.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:52.455+0000 7f007f200700 1 --2- 172.21.15.151:0/959878143 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0078108a20 0x7f0078108e00 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:52.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:52.455+0000 7f007f200700 1 --2- 172.21.15.151:0/959878143 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f007806afe0 0x7f007806b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:52.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:52.455+0000 7f007f200700 1 -- 172.21.15.151:0/959878143 >> 172.21.15.151:0/959878143 conn(0x7f00780788b0 msgr2=0x7f0078078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:48:52.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:52.456+0000 7f007f200700 1 -- 172.21.15.151:0/959878143 shutdown_connections 2024-04-26T01:48:52.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:52.456+0000 7f007f200700 1 -- 172.21.15.151:0/959878143 wait complete. 2024-04-26T01:48:52.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:52.457+0000 7f007f200700 1 Processor -- start 2024-04-26T01:48:52.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:52.457+0000 7f007f200700 1 -- start start 2024-04-26T01:48:52.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:52.457+0000 7f007f200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f007806afe0 0x7f007819b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:52.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:52.457+0000 7f007f200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f007806b980 0x7f007819b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:52.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:52.457+0000 7f007f200700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0078108a20 0x7f007819fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:52.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:52.457+0000 7f007f200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0078112c10 con 0x7f007806b980 2024-04-26T01:48:52.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:52.457+0000 7f007f200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0078112a90 con 0x7f007806afe0 2024-04-26T01:48:52.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:52.457+0000 7f007f200700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0078112d90 con 0x7f0078108a20 2024-04-26T01:48:52.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:52.458+0000 7f0077e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f007806afe0 0x7f007819b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:52.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:52.458+0000 7f0077e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f007806afe0 0x7f007819b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:49520/0 (socket says 172.21.15.151:49520) 2024-04-26T01:48:52.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:52.458+0000 7f0077e00700 1 -- 172.21.15.151:0/2668611816 learned_addr learned my addr 172.21.15.151:0/2668611816 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:48:52.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:52.458+0000 7f0077400700 1 --2- 172.21.15.151:0/2668611816 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f007806b980 0x7f007819b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:52.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:52.458+0000 7f007d200700 1 --2- 172.21.15.151:0/2668611816 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0078108a20 0x7f007819fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:52.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:52.458+0000 7f0077e00700 1 -- 172.21.15.151:0/2668611816 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0078108a20 msgr2=0x7f007819fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:52.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:52.458+0000 7f0077e00700 1 --2- 172.21.15.151:0/2668611816 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0078108a20 0x7f007819fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:52.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:52.458+0000 7f0077e00700 1 -- 172.21.15.151:0/2668611816 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f007806b980 msgr2=0x7f007819b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:52.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:52.458+0000 7f0077e00700 1 --2- 172.21.15.151:0/2668611816 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f007806b980 0x7f007819b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:52.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:52.458+0000 7f0077e00700 1 -- 172.21.15.151:0/2668611816 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00781a0380 con 0x7f007806afe0 2024-04-26T01:48:52.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:52.458+0000 7f0077e00700 1 --2- 172.21.15.151:0/2668611816 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f007806afe0 0x7f007819b3c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f006800d010 tx=0x7f0068004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:48:52.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:52.458+0000 7f0074c00700 1 -- 172.21.15.151:0/2668611816 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0068021070 con 0x7f007806afe0 2024-04-26T01:48:52.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:52.458+0000 7f0074c00700 1 -- 172.21.15.151:0/2668611816 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0068007e30 con 0x7f007806afe0 2024-04-26T01:48:52.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:52.458+0000 7f007f200700 1 -- 172.21.15.151:0/2668611816 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00781a0610 con 0x7f007806afe0 2024-04-26T01:48:52.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:52.459+0000 7f0074c00700 1 -- 172.21.15.151:0/2668611816 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00680045f0 con 0x7f007806afe0 2024-04-26T01:48:52.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:52.459+0000 7f007f200700 1 -- 172.21.15.151:0/2668611816 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00781a7ef0 con 0x7f007806afe0 2024-04-26T01:48:52.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:52.460+0000 7f007f200700 1 -- 172.21.15.151:0/2668611816 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0078069730 con 0x7f007806afe0 2024-04-26T01:48:52.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:52.461+0000 7f0074c00700 1 -- 172.21.15.151:0/2668611816 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f0068007980 con 0x7f007806afe0 2024-04-26T01:48:52.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:52.463+0000 7f0074c00700 1 --2- 172.21.15.151:0/2668611816 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f005c0a3cc0 0x7f005c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:52.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:52.463+0000 7f0074c00700 1 -- 172.21.15.151:0/2668611816 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f006800f020 con 0x7f007806afe0 2024-04-26T01:48:52.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:52.463+0000 7f0077400700 1 --2- 172.21.15.151:0/2668611816 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f005c0a3cc0 0x7f005c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:52.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:52.464+0000 7f0077400700 1 --2- 172.21.15.151:0/2668611816 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f005c0a3cc0 0x7f005c0a6180 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0078078f20 tx=0x7f0064009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:48:52.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:52.467+0000 7f0074c00700 1 -- 172.21.15.151:0/2668611816 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f006802a020 con 0x7f007806afe0 2024-04-26T01:48:52.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:52.742+0000 7f007f200700 1 -- 172.21.15.151:0/2668611816 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f007802df80 con 0x7f007806afe0 2024-04-26T01:48:52.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:52.742+0000 7f0074c00700 1 -- 172.21.15.151:0/2668611816 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3877 (secure 0 0 0) 0x7f006809e1c0 con 0x7f007806afe0 2024-04-26T01:48:52.741 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:48:52.741 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2937864097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2937864097},{"type":"v1","addr":"172.21.15.151:6835","nonce":2937864097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14373,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/2947090802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2947090802},{"type":"v1","addr":"172.21.15.151:6837","nonce":2947090802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/504675912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":504675912},{"type":"v1","addr":"172.21.15.165:6833","nonce":504675912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1207580015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1207580015},{"type":"v1","addr":"172.21.15.172:6833","nonce":1207580015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-26T01:48:52.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:52.746+0000 7f007f200700 1 -- 172.21.15.151:0/2668611816 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f005c0a3cc0 msgr2=0x7f005c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:52.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:52.746+0000 7f007f200700 1 --2- 172.21.15.151:0/2668611816 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f005c0a3cc0 0x7f005c0a6180 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0078078f20 tx=0x7f0064009260 comp rx=0 tx=0).stop 2024-04-26T01:48:52.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:52.746+0000 7f007f200700 1 -- 172.21.15.151:0/2668611816 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f007806afe0 msgr2=0x7f007819b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:52.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:52.746+0000 7f007f200700 1 --2- 172.21.15.151:0/2668611816 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f007806afe0 0x7f007819b3c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f006800d010 tx=0x7f0068004d00 comp rx=0 tx=0).stop 2024-04-26T01:48:52.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:52.746+0000 7f007f200700 1 -- 172.21.15.151:0/2668611816 shutdown_connections 2024-04-26T01:48:52.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:52.746+0000 7f007f200700 1 --2- 172.21.15.151:0/2668611816 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0078108a20 0x7f007819fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:52.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:52.746+0000 7f007f200700 1 --2- 172.21.15.151:0/2668611816 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f007806afe0 0x7f007819b3c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:52.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:52.746+0000 7f007f200700 1 --2- 172.21.15.151:0/2668611816 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f005c0a3cc0 0x7f005c0a6180 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:52.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:52.746+0000 7f007f200700 1 --2- 172.21.15.151:0/2668611816 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f007806b980 0x7f007819b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:52.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:52.746+0000 7f007f200700 1 -- 172.21.15.151:0/2668611816 >> 172.21.15.151:0/2668611816 conn(0x7f00780788b0 msgr2=0x7f007810dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:48:52.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:52.746+0000 7f007f200700 1 -- 172.21.15.151:0/2668611816 shutdown_connections 2024-04-26T01:48:52.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:52.746+0000 7f007f200700 1 -- 172.21.15.151:0/2668611816 wait complete. 2024-04-26T01:48:52.746 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 5 2024-04-26T01:48:53.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:52 smithi151 ceph-mon[28057]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:48:53.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:52 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:53.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:52 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:53.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:52 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:53.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:52 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:48:53.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:52 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:48:53.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:52 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:48:53.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:52 smithi151 ceph-mon[28057]: Deploying daemon mds.e on smithi165 2024-04-26T01:48:53.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:52 smithi151 ceph-mon[28057]: mds.? [v2:172.21.15.151:6836/2947090802,v1:172.21.15.151:6837/2947090802] up:boot 2024-04-26T01:48:53.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:52 smithi151 ceph-mon[28057]: fsmap 4 up:standby 2024-04-26T01:48:53.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:52 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T01:48:53.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:52 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/2668611816' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:48:53.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:52 smithi172 ceph-mon[32785]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:48:53.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:52 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:53.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:52 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:53.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:52 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:53.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:52 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:48:53.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:52 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:48:53.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:52 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:48:53.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:52 smithi172 ceph-mon[32785]: Deploying daemon mds.e on smithi165 2024-04-26T01:48:53.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:52 smithi172 ceph-mon[32785]: mds.? [v2:172.21.15.151:6836/2947090802,v1:172.21.15.151:6837/2947090802] up:boot 2024-04-26T01:48:53.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:52 smithi172 ceph-mon[32785]: fsmap 4 up:standby 2024-04-26T01:48:53.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:52 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T01:48:53.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:52 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/2668611816' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:48:53.339 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:52 smithi165 ceph-mon[32648]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:48:53.339 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:52 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:53.339 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:52 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:53.340 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:52 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:53.340 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:52 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:48:53.340 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:52 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:48:53.340 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:52 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:48:53.340 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:52 smithi165 ceph-mon[32648]: Deploying daemon mds.e on smithi165 2024-04-26T01:48:53.340 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:52 smithi165 ceph-mon[32648]: mds.? [v2:172.21.15.151:6836/2947090802,v1:172.21.15.151:6837/2947090802] up:boot 2024-04-26T01:48:53.340 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:52 smithi165 ceph-mon[32648]: fsmap 4 up:standby 2024-04-26T01:48:53.340 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:52 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T01:48:53.340 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:52 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/2668611816' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:48:54.346 INFO:journalctl@ceph.mds.e.smithi165.stdout:Apr 26 01:48:54 smithi165 systemd[1]: Starting Ceph mds.e for c0e49e64-036d-11ef-bc93-c7b262605968... 2024-04-26T01:48:54.665 INFO:journalctl@ceph.mds.e.smithi165.stdout:Apr 26 01:48:54 smithi165 podman[62685]: 2024-04-26 01:48:54.373523137 +0000 UTC m=+0.111165211 container create ddbe4960fbb298b411844d9e64dad98c1560b7a0749fc3e7fb839a8eedadb8b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-e, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-26T01:48:54.665 INFO:journalctl@ceph.mds.e.smithi165.stdout:Apr 26 01:48:54 smithi165 podman[62685]: 2024-04-26 01:48:54.280088572 +0000 UTC m=+0.017730659 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T01:48:54.665 INFO:journalctl@ceph.mds.e.smithi165.stdout:Apr 26 01:48:54 smithi165 podman[62685]: 2024-04-26 01:48:54.592642348 +0000 UTC m=+0.330284422 container init ddbe4960fbb298b411844d9e64dad98c1560b7a0749fc3e7fb839a8eedadb8b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-e, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-26T01:48:54.665 INFO:journalctl@ceph.mds.e.smithi165.stdout:Apr 26 01:48:54 smithi165 podman[62685]: 2024-04-26 01:48:54.595277394 +0000 UTC m=+0.332919478 container start ddbe4960fbb298b411844d9e64dad98c1560b7a0749fc3e7fb839a8eedadb8b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-e, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-26T01:48:54.665 INFO:journalctl@ceph.mds.e.smithi165.stdout:Apr 26 01:48:54 smithi165 ceph-mds[62704]: -- 172.21.15.165:0/544831669 <== mon.2 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564c73743a40 con 0x564c73716800 2024-04-26T01:48:54.665 INFO:journalctl@ceph.mds.e.smithi165.stdout:Apr 26 01:48:54 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-e[62700]: starting mds.e at 2024-04-26T01:48:54.665 INFO:journalctl@ceph.mds.e.smithi165.stdout:Apr 26 01:48:54 smithi165 bash[62685]: ddbe4960fbb298b411844d9e64dad98c1560b7a0749fc3e7fb839a8eedadb8b4 2024-04-26T01:48:54.965 INFO:journalctl@ceph.mds.e.smithi165.stdout:Apr 26 01:48:54 smithi165 systemd[1]: Started Ceph mds.e for c0e49e64-036d-11ef-bc93-c7b262605968. 2024-04-26T01:48:55.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:54 smithi172 ceph-mon[32785]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:48:55.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:54 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:55.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:54 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:55.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:54 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:55.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:54 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:48:55.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:54 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:48:55.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:54 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:48:55.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:54 smithi165 ceph-mon[32648]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:48:55.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:54 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:55.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:54 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:55.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:54 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:55.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:54 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:48:55.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:54 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:48:55.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:54 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:48:55.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:54 smithi151 ceph-mon[28057]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:48:55.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:54 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:55.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:54 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:55.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:54 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:55.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:54 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:48:55.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:54 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:48:55.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:54 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:48:55.624 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:48:56.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:55 smithi172 ceph-mon[32785]: Deploying daemon mds.f on smithi172 2024-04-26T01:48:56.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:55 smithi172 ceph-mon[32785]: mds.? [v2:172.21.15.165:6834/3598138294,v1:172.21.15.165:6835/3598138294] up:boot 2024-04-26T01:48:56.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:55 smithi172 ceph-mon[32785]: fsmap 5 up:standby 2024-04-26T01:48:56.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:55 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T01:48:56.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:55 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T01:48:56.092 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:48:56.244 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:55 smithi151 ceph-mon[28057]: Deploying daemon mds.f on smithi172 2024-04-26T01:48:56.244 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:55 smithi151 ceph-mon[28057]: mds.? [v2:172.21.15.165:6834/3598138294,v1:172.21.15.165:6835/3598138294] up:boot 2024-04-26T01:48:56.244 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:55 smithi151 ceph-mon[28057]: fsmap 5 up:standby 2024-04-26T01:48:56.244 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:55 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T01:48:56.244 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:55 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T01:48:56.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:55 smithi165 ceph-mon[32648]: Deploying daemon mds.f on smithi172 2024-04-26T01:48:56.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:55 smithi165 ceph-mon[32648]: mds.? [v2:172.21.15.165:6834/3598138294,v1:172.21.15.165:6835/3598138294] up:boot 2024-04-26T01:48:56.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:55 smithi165 ceph-mon[32648]: fsmap 5 up:standby 2024-04-26T01:48:56.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:55 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T01:48:56.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:55 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T01:48:57.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:56 smithi172 ceph-mon[32785]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:48:57.267 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:56 smithi151 ceph-mon[28057]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:48:57.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:56 smithi165 ceph-mon[32648]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:48:57.463 INFO:journalctl@ceph.mds.f.smithi172.stdout:Apr 26 01:48:57 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-f[63189]: starting mds.f at 2024-04-26T01:48:57.463 INFO:journalctl@ceph.mds.f.smithi172.stdout:Apr 26 01:48:57 smithi172 bash[63173]: d8269a242bd32bf2b55aef082d70891a92f3d8332fcff5812608152b53ec5b1a 2024-04-26T01:48:57.786 INFO:journalctl@ceph.mds.f.smithi172.stdout:Apr 26 01:48:57 smithi172 systemd[1]: Started Ceph mds.f for c0e49e64-036d-11ef-bc93-c7b262605968. 2024-04-26T01:48:58.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:58.331+0000 7fbead200700 1 -- 172.21.15.151:0/1387307659 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbea81034b0 msgr2=0x7fbea810f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:58.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:58.331+0000 7fbead200700 1 --2- 172.21.15.151:0/1387307659 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbea81034b0 0x7fbea810f920 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbea000b0a0 tx=0x7fbea000b3b0 comp rx=0 tx=0).stop 2024-04-26T01:48:58.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:58.331+0000 7fbead200700 1 -- 172.21.15.151:0/1387307659 shutdown_connections 2024-04-26T01:48:58.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:58.331+0000 7fbead200700 1 --2- 172.21.15.151:0/1387307659 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbea81034b0 0x7fbea810f920 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:58.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:58.331+0000 7fbead200700 1 --2- 172.21.15.151:0/1387307659 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbea8107940 0x7fbea8107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:58.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:58.331+0000 7fbead200700 1 --2- 172.21.15.151:0/1387307659 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbea8102af0 0x7fbea8102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:58.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:58.331+0000 7fbead200700 1 -- 172.21.15.151:0/1387307659 >> 172.21.15.151:0/1387307659 conn(0x7fbea8079ba0 msgr2=0x7fbea8079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:48:58.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:58.332+0000 7fbead200700 1 -- 172.21.15.151:0/1387307659 shutdown_connections 2024-04-26T01:48:58.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:58.332+0000 7fbead200700 1 -- 172.21.15.151:0/1387307659 wait complete. 2024-04-26T01:48:58.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:58.333+0000 7fbead200700 1 Processor -- start 2024-04-26T01:48:58.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:58.333+0000 7fbead200700 1 -- start start 2024-04-26T01:48:58.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:58.334+0000 7fbead200700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbea8102af0 0x7fbea8074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:58.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:58.334+0000 7fbead200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbea81034b0 0x7fbea8075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:58.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:58.334+0000 7fbead200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbea8107940 0x7fbea80757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:58.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:58.334+0000 7fbead200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbea8112c90 con 0x7fbea81034b0 2024-04-26T01:48:58.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:58.334+0000 7fbead200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbea8112b10 con 0x7fbea8107940 2024-04-26T01:48:58.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:58.334+0000 7fbead200700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbea8112e10 con 0x7fbea8102af0 2024-04-26T01:48:58.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:58.334+0000 7fbea5e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbea8102af0 0x7fbea8074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:58.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:58.334+0000 7fbea5e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbea8102af0 0x7fbea8074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:59722/0 (socket says 172.21.15.151:59722) 2024-04-26T01:48:58.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:58.334+0000 7fbea5e00700 1 -- 172.21.15.151:0/228232561 learned_addr learned my addr 172.21.15.151:0/228232561 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:48:58.333 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:58.334+0000 7fbea5400700 1 --2- 172.21.15.151:0/228232561 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbea81034b0 0x7fbea8075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:58.333 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:58.334+0000 7fbea6800700 1 --2- 172.21.15.151:0/228232561 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbea8107940 0x7fbea80757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:58.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:58.334+0000 7fbea5e00700 1 -- 172.21.15.151:0/228232561 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbea8107940 msgr2=0x7fbea80757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:58.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:58.334+0000 7fbea5e00700 1 --2- 172.21.15.151:0/228232561 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbea8107940 0x7fbea80757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:58.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:58.335+0000 7fbea5e00700 1 -- 172.21.15.151:0/228232561 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbea81034b0 msgr2=0x7fbea8075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:58.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:58.335+0000 7fbea5e00700 1 --2- 172.21.15.151:0/228232561 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbea81034b0 0x7fbea8075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:58.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:58.335+0000 7fbea5e00700 1 -- 172.21.15.151:0/228232561 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbea81a7710 con 0x7fbea8102af0 2024-04-26T01:48:58.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:58.335+0000 7fbea5e00700 1 --2- 172.21.15.151:0/228232561 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbea8102af0 0x7fbea8074d40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbe9800e9a0 tx=0x7fbe9800ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:48:58.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:58.335+0000 7fbe9ea00700 1 -- 172.21.15.151:0/228232561 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe9800cbd0 con 0x7fbea8102af0 2024-04-26T01:48:58.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:58.335+0000 7fbe9ea00700 1 -- 172.21.15.151:0/228232561 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe9800cd70 con 0x7fbea8102af0 2024-04-26T01:48:58.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:58.335+0000 7fbead200700 1 -- 172.21.15.151:0/228232561 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbea81a7a00 con 0x7fbea8102af0 2024-04-26T01:48:58.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:58.335+0000 7fbe9ea00700 1 -- 172.21.15.151:0/228232561 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe98018680 con 0x7fbea8102af0 2024-04-26T01:48:58.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:58.335+0000 7fbead200700 1 -- 172.21.15.151:0/228232561 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbea81a7f40 con 0x7fbea8102af0 2024-04-26T01:48:58.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:58.337+0000 7fbead200700 1 -- 172.21.15.151:0/228232561 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbea8069730 con 0x7fbea8102af0 2024-04-26T01:48:58.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:58.338+0000 7fbe9ea00700 1 -- 172.21.15.151:0/228232561 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fbe98010970 con 0x7fbea8102af0 2024-04-26T01:48:58.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:58.339+0000 7fbe9ea00700 1 --2- 172.21.15.151:0/228232561 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fbe8c0a3c40 0x7fbe8c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:58.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:58.340+0000 7fbe9ea00700 1 -- 172.21.15.151:0/228232561 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fbe980d02e0 con 0x7fbea8102af0 2024-04-26T01:48:58.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:58.340+0000 7fbea5400700 1 --2- 172.21.15.151:0/228232561 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fbe8c0a3c40 0x7fbe8c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:58.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:58.340+0000 7fbea5400700 1 --2- 172.21.15.151:0/228232561 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fbe8c0a3c40 0x7fbe8c0a6100 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbea807a180 tx=0x7fbe94009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:48:58.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:58.344+0000 7fbe9ea00700 1 -- 172.21.15.151:0/228232561 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbe9809d0c0 con 0x7fbea8102af0 2024-04-26T01:48:58.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:58.622+0000 7fbead200700 1 -- 172.21.15.151:0/228232561 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbea802df80 con 0x7fbea8102af0 2024-04-26T01:48:58.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:58 smithi151 ceph-mon[28057]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:48:58.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:58 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:58.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:58 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:58.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:58 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:58.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:58 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:48:58.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:58 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:48:58.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:58 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:48:58.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:58 smithi151 ceph-mon[28057]: Deploying daemon mds.c on smithi172 2024-04-26T01:48:58.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:58 smithi151 ceph-mon[28057]: mds.? [v2:172.21.15.172:6834/2309751171,v1:172.21.15.172:6835/2309751171] up:boot 2024-04-26T01:48:58.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:58 smithi151 ceph-mon[28057]: fsmap 6 up:standby 2024-04-26T01:48:58.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:58 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T01:48:58.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:58.623+0000 7fbe9ea00700 1 -- 172.21.15.151:0/228232561 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5533 (secure 0 0 0) 0x7fbe9809c9c0 con 0x7fbea8102af0 2024-04-26T01:48:58.622 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:48:58.622 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2937864097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2937864097},{"type":"v1","addr":"172.21.15.151:6835","nonce":2937864097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14373,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/2947090802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2947090802},{"type":"v1","addr":"172.21.15.151:6837","nonce":2947090802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/504675912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":504675912},{"type":"v1","addr":"172.21.15.165:6833","nonce":504675912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1207580015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1207580015},{"type":"v1","addr":"172.21.15.172:6833","nonce":1207580015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/3598138294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3598138294},{"type":"v1","addr":"172.21.15.165:6835","nonce":3598138294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24481,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2309751171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2309751171},{"type":"v1","addr":"172.21.15.172:6835","nonce":2309751171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-26T01:48:58.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:58.627+0000 7fbead200700 1 -- 172.21.15.151:0/228232561 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fbe8c0a3c40 msgr2=0x7fbe8c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:58.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:58.627+0000 7fbead200700 1 --2- 172.21.15.151:0/228232561 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fbe8c0a3c40 0x7fbe8c0a6100 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbea807a180 tx=0x7fbe94009260 comp rx=0 tx=0).stop 2024-04-26T01:48:58.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:58.627+0000 7fbead200700 1 -- 172.21.15.151:0/228232561 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbea8102af0 msgr2=0x7fbea8074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:58.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:58.627+0000 7fbead200700 1 --2- 172.21.15.151:0/228232561 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbea8102af0 0x7fbea8074d40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbe9800e9a0 tx=0x7fbe9800ed60 comp rx=0 tx=0).stop 2024-04-26T01:48:58.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:58.627+0000 7fbead200700 1 -- 172.21.15.151:0/228232561 shutdown_connections 2024-04-26T01:48:58.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:58.627+0000 7fbead200700 1 --2- 172.21.15.151:0/228232561 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbea8107940 0x7fbea80757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:58.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:58.627+0000 7fbead200700 1 --2- 172.21.15.151:0/228232561 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbea8102af0 0x7fbea8074d40 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:58.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:58.627+0000 7fbead200700 1 --2- 172.21.15.151:0/228232561 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fbe8c0a3c40 0x7fbe8c0a6100 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:58.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:58.627+0000 7fbead200700 1 --2- 172.21.15.151:0/228232561 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbea81034b0 0x7fbea8075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:58.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:58.627+0000 7fbead200700 1 -- 172.21.15.151:0/228232561 >> 172.21.15.151:0/228232561 conn(0x7fbea8079ba0 msgr2=0x7fbea8100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:48:58.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:58.627+0000 7fbead200700 1 -- 172.21.15.151:0/228232561 shutdown_connections 2024-04-26T01:48:58.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:48:58.627+0000 7fbead200700 1 -- 172.21.15.151:0/228232561 wait complete. 2024-04-26T01:48:58.627 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 7 2024-04-26T01:48:58.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:58 smithi172 ceph-mon[32785]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:48:58.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:58 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:58.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:58 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:58.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:58 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:58.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:58 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:48:58.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:58 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:48:58.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:58 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:48:58.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:58 smithi172 ceph-mon[32785]: Deploying daemon mds.c on smithi172 2024-04-26T01:48:58.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:58 smithi172 ceph-mon[32785]: mds.? [v2:172.21.15.172:6834/2309751171,v1:172.21.15.172:6835/2309751171] up:boot 2024-04-26T01:48:58.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:58 smithi172 ceph-mon[32785]: fsmap 6 up:standby 2024-04-26T01:48:58.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:58 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T01:48:58.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:58 smithi165 ceph-mon[32648]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:48:58.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:58 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:58.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:58 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:58.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:58 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:48:58.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:58 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:48:58.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:58 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:48:58.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:58 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:48:58.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:58 smithi165 ceph-mon[32648]: Deploying daemon mds.c on smithi172 2024-04-26T01:48:58.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:58 smithi165 ceph-mon[32648]: mds.? [v2:172.21.15.172:6834/2309751171,v1:172.21.15.172:6835/2309751171] up:boot 2024-04-26T01:48:58.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:58 smithi165 ceph-mon[32648]: fsmap 6 up:standby 2024-04-26T01:48:58.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:58 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T01:48:59.781 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:48:59 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/228232561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:48:59.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:48:59 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/228232561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:48:59.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:48:59 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/228232561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:49:00.036 INFO:journalctl@ceph.mds.c.smithi172.stdout:Apr 26 01:49:00 smithi172 podman[63481]: 2024-04-26 01:49:00.009837762 +0000 UTC m=+0.123106618 container create 4da7b90b621d3980cd26764930148b950e51bae17f8781dfdc3882651008f38d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD) 2024-04-26T01:49:00.531 INFO:journalctl@ceph.mds.c.smithi172.stdout:Apr 26 01:49:00 smithi172 podman[63481]: 2024-04-26 01:49:00.230645073 +0000 UTC m=+0.343913922 container init 4da7b90b621d3980cd26764930148b950e51bae17f8781dfdc3882651008f38d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-c, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-26T01:49:00.531 INFO:journalctl@ceph.mds.c.smithi172.stdout:Apr 26 01:49:00 smithi172 podman[63481]: 2024-04-26 01:49:00.233254644 +0000 UTC m=+0.346523500 container start 4da7b90b621d3980cd26764930148b950e51bae17f8781dfdc3882651008f38d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-c, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-26T01:49:00.531 INFO:journalctl@ceph.mds.c.smithi172.stdout:Apr 26 01:49:00 smithi172 ceph-mds[63500]: -- 172.21.15.172:0/2724167689 <== mon.1 v2:172.21.15.172:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563f310e9a40 con 0x563f310bd000 2024-04-26T01:49:00.531 INFO:journalctl@ceph.mds.c.smithi172.stdout:Apr 26 01:49:00 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-c[63496]: starting mds.c at 2024-04-26T01:49:00.531 INFO:journalctl@ceph.mds.c.smithi172.stdout:Apr 26 01:49:00 smithi172 bash[63481]: 4da7b90b621d3980cd26764930148b950e51bae17f8781dfdc3882651008f38d 2024-04-26T01:49:00.532 INFO:journalctl@ceph.mds.c.smithi172.stdout:Apr 26 01:49:00 smithi172 systemd[1]: Started Ceph mds.c for c0e49e64-036d-11ef-bc93-c7b262605968. 2024-04-26T01:49:00.759 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:00 smithi165 ceph-mon[32648]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:00.759 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:00 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:00.759 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:00 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:00.759 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:00 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:00.760 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:00 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:49:00.760 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:00 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:49:00.760 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:00 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:49:00.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:00 smithi172 ceph-mon[32785]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:00.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:00 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:00.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:00 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:00.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:00 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:00.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:00 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:49:00.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:00 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:49:00.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:00 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:49:00.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:00 smithi151 ceph-mon[28057]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:00.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:00 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:00.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:00 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:00.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:00 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:00.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:00 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:49:00.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:00 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:49:00.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:00 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:49:01.456 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:49:01.729 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:01 smithi165 ceph-mon[32648]: Deploying daemon mds.h on smithi165 2024-04-26T01:49:01.729 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:01 smithi165 ceph-mon[32648]: mds.? [v2:172.21.15.172:6836/55972537,v1:172.21.15.172:6837/55972537] up:boot 2024-04-26T01:49:01.729 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:01 smithi165 ceph-mon[32648]: fsmap 7 up:standby 2024-04-26T01:49:01.729 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:01 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T01:49:01.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:01 smithi151 ceph-mon[28057]: Deploying daemon mds.h on smithi165 2024-04-26T01:49:01.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:01 smithi151 ceph-mon[28057]: mds.? [v2:172.21.15.172:6836/55972537,v1:172.21.15.172:6837/55972537] up:boot 2024-04-26T01:49:01.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:01 smithi151 ceph-mon[28057]: fsmap 7 up:standby 2024-04-26T01:49:01.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:01 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T01:49:01.902 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:49:02.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:01 smithi172 ceph-mon[32785]: Deploying daemon mds.h on smithi165 2024-04-26T01:49:02.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:01 smithi172 ceph-mon[32785]: mds.? [v2:172.21.15.172:6836/55972537,v1:172.21.15.172:6837/55972537] up:boot 2024-04-26T01:49:02.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:01 smithi172 ceph-mon[32785]: fsmap 7 up:standby 2024-04-26T01:49:02.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:01 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T01:49:02.543 INFO:journalctl@ceph.mds.h.smithi165.stdout:Apr 26 01:49:02 smithi165 systemd[1]: Starting Ceph mds.h for c0e49e64-036d-11ef-bc93-c7b262605968... 2024-04-26T01:49:02.543 INFO:journalctl@ceph.mds.h.smithi165.stdout:Apr 26 01:49:02 smithi165 podman[63000]: 2024-04-26 01:49:02.448124569 +0000 UTC m=+0.017762496 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T01:49:02.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:02 smithi151 ceph-mon[28057]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:02.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:02 smithi165 ceph-mon[32648]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:02.846 INFO:journalctl@ceph.mds.h.smithi165.stdout:Apr 26 01:49:02 smithi165 podman[63000]: 2024-04-26 01:49:02.550030811 +0000 UTC m=+0.119668728 container create a37b5d31366444f427078dfc9d889e6aa6d83f116c59eff9af08868f79219b01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-h, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-26T01:49:02.846 INFO:journalctl@ceph.mds.h.smithi165.stdout:Apr 26 01:49:02 smithi165 podman[63000]: 2024-04-26 01:49:02.769060204 +0000 UTC m=+0.338698123 container init a37b5d31366444f427078dfc9d889e6aa6d83f116c59eff9af08868f79219b01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-h, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True) 2024-04-26T01:49:02.846 INFO:journalctl@ceph.mds.h.smithi165.stdout:Apr 26 01:49:02 smithi165 podman[63000]: 2024-04-26 01:49:02.771794817 +0000 UTC m=+0.341432739 container start a37b5d31366444f427078dfc9d889e6aa6d83f116c59eff9af08868f79219b01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-h, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-26T01:49:02.846 INFO:journalctl@ceph.mds.h.smithi165.stdout:Apr 26 01:49:02 smithi165 ceph-mds[63020]: -- 172.21.15.165:0/1986814103 <== mon.2 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556da96bfa40 con 0x556da9693000 2024-04-26T01:49:02.846 INFO:journalctl@ceph.mds.h.smithi165.stdout:Apr 26 01:49:02 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-h[63016]: starting mds.h at 2024-04-26T01:49:03.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:02 smithi172 ceph-mon[32785]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:03.345 INFO:journalctl@ceph.mds.h.smithi165.stdout:Apr 26 01:49:02 smithi165 bash[63000]: a37b5d31366444f427078dfc9d889e6aa6d83f116c59eff9af08868f79219b01 2024-04-26T01:49:03.346 INFO:journalctl@ceph.mds.h.smithi165.stdout:Apr 26 01:49:02 smithi165 systemd[1]: Started Ceph mds.h for c0e49e64-036d-11ef-bc93-c7b262605968. 2024-04-26T01:49:03.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:03.941+0000 7f860c000700 1 -- 172.21.15.151:0/2000517969 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86041092d0 msgr2=0x7f86041096b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:03.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:03.941+0000 7f860c000700 1 --2- 172.21.15.151:0/2000517969 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86041092d0 0x7f86041096b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f85f800b0d0 tx=0x7f85f800b3e0 comp rx=0 tx=0).stop 2024-04-26T01:49:03.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:03.941+0000 7f860c000700 1 -- 172.21.15.151:0/2000517969 shutdown_connections 2024-04-26T01:49:03.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:03.941+0000 7f860c000700 1 --2- 172.21.15.151:0/2000517969 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f860410e3b0 0x7f8604115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:03.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:03.941+0000 7f860c000700 1 --2- 172.21.15.151:0/2000517969 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86041092d0 0x7f86041096b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:03.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:03.941+0000 7f860c000700 1 --2- 172.21.15.151:0/2000517969 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8604109c80 0x7f860410dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:03.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:03.941+0000 7f860c000700 1 -- 172.21.15.151:0/2000517969 >> 172.21.15.151:0/2000517969 conn(0x7f8604076210 msgr2=0x7f8604107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:03.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:03.941+0000 7f860c000700 1 -- 172.21.15.151:0/2000517969 shutdown_connections 2024-04-26T01:49:03.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:03.941+0000 7f860c000700 1 -- 172.21.15.151:0/2000517969 wait complete. 2024-04-26T01:49:03.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:03.942+0000 7f860c000700 1 Processor -- start 2024-04-26T01:49:03.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:03.943+0000 7f860c000700 1 -- start start 2024-04-26T01:49:03.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:03.943+0000 7f860c000700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86041092d0 0x7f86041a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:03.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:03.943+0000 7f860c000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8604109c80 0x7f86041a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:03.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:03.943+0000 7f860c000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f860410e3b0 0x7f860419f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:03.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:03.943+0000 7f860c000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8604119290 con 0x7f860410e3b0 2024-04-26T01:49:03.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:03.943+0000 7f860c000700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8604119110 con 0x7f8604109c80 2024-04-26T01:49:03.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:03.943+0000 7f860c000700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8604119410 con 0x7f86041092d0 2024-04-26T01:49:03.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:03.943+0000 7f860a000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f860410e3b0 0x7f860419f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:03.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:03.943+0000 7f8608c00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8604109c80 0x7f86041a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:03.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:03.943+0000 7f8609600700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86041092d0 0x7f86041a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:03.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:03.943+0000 7f860a000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f860410e3b0 0x7f860419f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:60662/0 (socket says 172.21.15.151:60662) 2024-04-26T01:49:03.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:03.943+0000 7f860a000700 1 -- 172.21.15.151:0/1648393797 learned_addr learned my addr 172.21.15.151:0/1648393797 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:49:03.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:03.944+0000 7f8608c00700 1 -- 172.21.15.151:0/1648393797 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86041092d0 msgr2=0x7f86041a53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:03.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:03.944+0000 7f8608c00700 1 --2- 172.21.15.151:0/1648393797 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86041092d0 0x7f86041a53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:03.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:03.944+0000 7f8608c00700 1 -- 172.21.15.151:0/1648393797 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f860410e3b0 msgr2=0x7f860419f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:03.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:03.944+0000 7f8608c00700 1 --2- 172.21.15.151:0/1648393797 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f860410e3b0 0x7f860419f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:03.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:03.944+0000 7f8608c00700 1 -- 172.21.15.151:0/1648393797 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f860419fcf0 con 0x7f8604109c80 2024-04-26T01:49:03.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:03.944+0000 7f8609600700 1 --2- 172.21.15.151:0/1648393797 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86041092d0 0x7f86041a53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:49:03.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:03.944+0000 7f8608c00700 1 --2- 172.21.15.151:0/1648393797 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8604109c80 0x7f86041a5910 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f85f000e9c0 tx=0x7f85f000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:03.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:03.945+0000 7f8602000700 1 -- 172.21.15.151:0/1648393797 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f000cb70 con 0x7f8604109c80 2024-04-26T01:49:03.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:03.945+0000 7f8602000700 1 -- 172.21.15.151:0/1648393797 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85f000ee60 con 0x7f8604109c80 2024-04-26T01:49:03.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:03.945+0000 7f860c000700 1 -- 172.21.15.151:0/1648393797 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f860419ffe0 con 0x7f8604109c80 2024-04-26T01:49:03.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:03.945+0000 7f8602000700 1 -- 172.21.15.151:0/1648393797 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f0003d60 con 0x7f8604109c80 2024-04-26T01:49:03.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:03.945+0000 7f860c000700 1 -- 172.21.15.151:0/1648393797 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86041ac2a0 con 0x7f8604109c80 2024-04-26T01:49:03.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:03.946+0000 7f860c000700 1 -- 172.21.15.151:0/1648393797 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8604069730 con 0x7f8604109c80 2024-04-26T01:49:03.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:03.948+0000 7f8602000700 1 -- 172.21.15.151:0/1648393797 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f85f000cd10 con 0x7f8604109c80 2024-04-26T01:49:03.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:03.949+0000 7f8602000700 1 --2- 172.21.15.151:0/1648393797 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f85ec0a3cc0 0x7f85ec0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:03.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:03.949+0000 7f8602000700 1 -- 172.21.15.151:0/1648393797 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f85f00d00f0 con 0x7f8604109c80 2024-04-26T01:49:03.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:03.949+0000 7f8609600700 1 --2- 172.21.15.151:0/1648393797 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f85ec0a3cc0 0x7f85ec0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:03.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:03.950+0000 7f8609600700 1 --2- 172.21.15.151:0/1648393797 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f85ec0a3cc0 0x7f85ec0a6180 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f85f800bfd0 tx=0x7f85f800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:03.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:03.954+0000 7f8602000700 1 -- 172.21.15.151:0/1648393797 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f85f009ced0 con 0x7f8604109c80 2024-04-26T01:49:04.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:03 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:04.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:03 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:04.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:03 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:04.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:03 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:49:04.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:03 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:49:04.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:03 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:49:04.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:03 smithi151 ceph-mon[28057]: Deploying daemon mds.i on smithi172 2024-04-26T01:49:04.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:03 smithi151 ceph-mon[28057]: mds.? [v2:172.21.15.165:6836/1847859054,v1:172.21.15.165:6837/1847859054] up:boot 2024-04-26T01:49:04.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:03 smithi151 ceph-mon[28057]: fsmap 8 up:standby 2024-04-26T01:49:04.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:03 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T01:49:04.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:04.231+0000 7f860c000700 1 -- 172.21.15.151:0/1648393797 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f860402df80 con 0x7f8604109c80 2024-04-26T01:49:04.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:04.232+0000 7f8602000700 1 -- 172.21.15.151:0/1648393797 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7183 (secure 0 0 0) 0x7f85f009c7d0 con 0x7f8604109c80 2024-04-26T01:49:04.231 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:49:04.231 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2937864097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2937864097},{"type":"v1","addr":"172.21.15.151:6835","nonce":2937864097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14373,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/2947090802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2947090802},{"type":"v1","addr":"172.21.15.151:6837","nonce":2947090802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/504675912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":504675912},{"type":"v1","addr":"172.21.15.165:6833","nonce":504675912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1207580015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1207580015},{"type":"v1","addr":"172.21.15.172:6833","nonce":1207580015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/3598138294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3598138294},{"type":"v1","addr":"172.21.15.165:6835","nonce":3598138294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/1847859054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":1847859054},{"type":"v1","addr":"172.21.15.165:6837","nonce":1847859054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24481,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2309751171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2309751171},{"type":"v1","addr":"172.21.15.172:6835","nonce":2309751171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24493,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/55972537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":55972537},{"type":"v1","addr":"172.21.15.172:6837","nonce":55972537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-26T01:49:04.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:04.235+0000 7f860c000700 1 -- 172.21.15.151:0/1648393797 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f85ec0a3cc0 msgr2=0x7f85ec0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:04.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:04.235+0000 7f860c000700 1 --2- 172.21.15.151:0/1648393797 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f85ec0a3cc0 0x7f85ec0a6180 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f85f800bfd0 tx=0x7f85f800b040 comp rx=0 tx=0).stop 2024-04-26T01:49:04.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:04.235+0000 7f860c000700 1 -- 172.21.15.151:0/1648393797 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8604109c80 msgr2=0x7f86041a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:04.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:04.235+0000 7f860c000700 1 --2- 172.21.15.151:0/1648393797 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8604109c80 0x7f86041a5910 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f85f000e9c0 tx=0x7f85f000ecd0 comp rx=0 tx=0).stop 2024-04-26T01:49:04.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:04.235+0000 7f860c000700 1 -- 172.21.15.151:0/1648393797 shutdown_connections 2024-04-26T01:49:04.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:04.235+0000 7f860c000700 1 --2- 172.21.15.151:0/1648393797 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f85ec0a3cc0 0x7f85ec0a6180 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:04.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:04.235+0000 7f860c000700 1 --2- 172.21.15.151:0/1648393797 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f860410e3b0 0x7f860419f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:04.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:04.235+0000 7f860c000700 1 --2- 172.21.15.151:0/1648393797 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86041092d0 0x7f86041a53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:04.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:04.235+0000 7f860c000700 1 --2- 172.21.15.151:0/1648393797 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8604109c80 0x7f86041a5910 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:04.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:04.235+0000 7f860c000700 1 -- 172.21.15.151:0/1648393797 >> 172.21.15.151:0/1648393797 conn(0x7f8604076210 msgr2=0x7f860410cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:04.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:04.236+0000 7f860c000700 1 -- 172.21.15.151:0/1648393797 shutdown_connections 2024-04-26T01:49:04.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:04.236+0000 7f860c000700 1 -- 172.21.15.151:0/1648393797 wait complete. 2024-04-26T01:49:04.235 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 9 2024-04-26T01:49:04.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:03 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:04.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:03 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:04.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:03 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:04.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:03 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:49:04.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:03 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:49:04.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:03 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:49:04.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:03 smithi172 ceph-mon[32785]: Deploying daemon mds.i on smithi172 2024-04-26T01:49:04.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:03 smithi172 ceph-mon[32785]: mds.? [v2:172.21.15.165:6836/1847859054,v1:172.21.15.165:6837/1847859054] up:boot 2024-04-26T01:49:04.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:03 smithi172 ceph-mon[32785]: fsmap 8 up:standby 2024-04-26T01:49:04.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:03 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T01:49:04.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:03 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:04.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:03 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:04.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:03 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:04.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:03 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:49:04.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:03 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:49:04.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:03 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:49:04.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:03 smithi165 ceph-mon[32648]: Deploying daemon mds.i on smithi172 2024-04-26T01:49:04.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:03 smithi165 ceph-mon[32648]: mds.? [v2:172.21.15.165:6836/1847859054,v1:172.21.15.165:6837/1847859054] up:boot 2024-04-26T01:49:04.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:03 smithi165 ceph-mon[32648]: fsmap 8 up:standby 2024-04-26T01:49:04.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:03 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T01:49:05.266 INFO:journalctl@ceph.mds.i.smithi172.stdout:Apr 26 01:49:05 smithi172 systemd[1]: Starting Ceph mds.i for c0e49e64-036d-11ef-bc93-c7b262605968... 2024-04-26T01:49:05.267 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:04 smithi172 ceph-mon[32785]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:05.267 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:04 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/1648393797' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:49:05.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:04 smithi165 ceph-mon[32648]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:05.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:04 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/1648393797' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:49:05.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:04 smithi151 ceph-mon[28057]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:05.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:04 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/1648393797' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:49:05.519 INFO:journalctl@ceph.mds.i.smithi172.stdout:Apr 26 01:49:05 smithi172 podman[63790]: 2024-04-26 01:49:05.401774591 +0000 UTC m=+0.017912759 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T01:49:05.519 INFO:journalctl@ceph.mds.i.smithi172.stdout:Apr 26 01:49:05 smithi172 podman[63790]: 2024-04-26 01:49:05.521529299 +0000 UTC m=+0.137667469 container create c32829abcd2d6fcc40d1c33543f3e4cf37d1118090af9329a4190ad245dc67e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-26T01:49:05.780 INFO:journalctl@ceph.mds.i.smithi172.stdout:Apr 26 01:49:05 smithi172 podman[63790]: 2024-04-26 01:49:05.75747535 +0000 UTC m=+0.373613520 container init c32829abcd2d6fcc40d1c33543f3e4cf37d1118090af9329a4190ad245dc67e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-26T01:49:05.781 INFO:journalctl@ceph.mds.i.smithi172.stdout:Apr 26 01:49:05 smithi172 podman[63790]: 2024-04-26 01:49:05.760902797 +0000 UTC m=+0.377040968 container start c32829abcd2d6fcc40d1c33543f3e4cf37d1118090af9329a4190ad245dc67e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-i, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0) 2024-04-26T01:49:06.036 INFO:journalctl@ceph.mds.i.smithi172.stdout:Apr 26 01:49:05 smithi172 ceph-mds[63819]: -- 172.21.15.172:0/3900761700 <== mon.1 v2:172.21.15.172:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c758f49a40 con 0x55c758f1cc00 2024-04-26T01:49:06.036 INFO:journalctl@ceph.mds.i.smithi172.stdout:Apr 26 01:49:05 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-i[63813]: starting mds.i at 2024-04-26T01:49:06.036 INFO:journalctl@ceph.mds.i.smithi172.stdout:Apr 26 01:49:05 smithi172 bash[63790]: c32829abcd2d6fcc40d1c33543f3e4cf37d1118090af9329a4190ad245dc67e2 2024-04-26T01:49:06.036 INFO:journalctl@ceph.mds.i.smithi172.stdout:Apr 26 01:49:05 smithi172 systemd[1]: Started Ceph mds.i for c0e49e64-036d-11ef-bc93-c7b262605968. 2024-04-26T01:49:07.003 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:49:07.150 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:06 smithi151 ceph-mon[28057]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:07.151 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:06 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:07.151 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:06 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:07.151 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:06 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:07.151 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:06 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:49:07.151 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:06 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:49:07.151 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:06 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:49:07.151 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:06 smithi151 ceph-mon[28057]: Deploying daemon mds.g on smithi151 2024-04-26T01:49:07.151 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:06 smithi151 ceph-mon[28057]: mds.? [v2:172.21.15.172:6838/1242302810,v1:172.21.15.172:6839/1242302810] up:boot 2024-04-26T01:49:07.151 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:06 smithi151 ceph-mon[28057]: fsmap 9 up:standby 2024-04-26T01:49:07.151 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:06 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T01:49:07.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:06 smithi172 ceph-mon[32785]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:07.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:06 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:07.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:06 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:07.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:06 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:07.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:06 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:49:07.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:06 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:49:07.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:06 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:49:07.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:06 smithi172 ceph-mon[32785]: Deploying daemon mds.g on smithi151 2024-04-26T01:49:07.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:06 smithi172 ceph-mon[32785]: mds.? [v2:172.21.15.172:6838/1242302810,v1:172.21.15.172:6839/1242302810] up:boot 2024-04-26T01:49:07.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:06 smithi172 ceph-mon[32785]: fsmap 9 up:standby 2024-04-26T01:49:07.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:06 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T01:49:07.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:06 smithi165 ceph-mon[32648]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:07.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:06 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:07.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:06 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:07.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:06 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:07.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:06 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:49:07.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:06 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:49:07.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:06 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:49:07.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:06 smithi165 ceph-mon[32648]: Deploying daemon mds.g on smithi151 2024-04-26T01:49:07.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:06 smithi165 ceph-mon[32648]: mds.? [v2:172.21.15.172:6838/1242302810,v1:172.21.15.172:6839/1242302810] up:boot 2024-04-26T01:49:07.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:06 smithi165 ceph-mon[32648]: fsmap 9 up:standby 2024-04-26T01:49:07.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:06 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T01:49:08.434 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:49:09.155 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:08 smithi151 ceph-mon[28057]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:09.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:08 smithi172 ceph-mon[32785]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:09.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:08 smithi165 ceph-mon[32648]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:10.307 INFO:journalctl@ceph.mds.g.smithi151.stdout:Apr 26 01:49:10 smithi151 podman[69480]: 2024-04-26 01:49:10.156965281 +0000 UTC m=+0.037769125 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T01:49:10.307 INFO:journalctl@ceph.mds.g.smithi151.stdout:Apr 26 01:49:10 smithi151 podman[69480]: 2024-04-26 01:49:10.307988909 +0000 UTC m=+0.188792814 container create aa6c28e3310fb36f51da1f8e52b65e741a71a6a89910f2f7f259bb7e8dd99e98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-g, org.label-schema.build-date=20240416, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-26T01:49:10.856 INFO:journalctl@ceph.mds.g.smithi151.stdout:Apr 26 01:49:10 smithi151 podman[69480]: 2024-04-26 01:49:10.612693027 +0000 UTC m=+0.493496909 container init aa6c28e3310fb36f51da1f8e52b65e741a71a6a89910f2f7f259bb7e8dd99e98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-g, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416) 2024-04-26T01:49:10.856 INFO:journalctl@ceph.mds.g.smithi151.stdout:Apr 26 01:49:10 smithi151 podman[69480]: 2024-04-26 01:49:10.623401638 +0000 UTC m=+0.504205480 container start aa6c28e3310fb36f51da1f8e52b65e741a71a6a89910f2f7f259bb7e8dd99e98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-g, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240416) 2024-04-26T01:49:10.856 INFO:journalctl@ceph.mds.g.smithi151.stdout:Apr 26 01:49:10 smithi151 ceph-mds[69517]: -- 172.21.15.151:0/4070081011 <== mon.2 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b913395a40 con 0x55b913369000 2024-04-26T01:49:10.857 INFO:journalctl@ceph.mds.g.smithi151.stdout:Apr 26 01:49:10 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-g[69512]: starting mds.g at 2024-04-26T01:49:11.112 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:10 smithi151 ceph-mon[28057]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:11.112 INFO:journalctl@ceph.mds.g.smithi151.stdout:Apr 26 01:49:10 smithi151 bash[69480]: aa6c28e3310fb36f51da1f8e52b65e741a71a6a89910f2f7f259bb7e8dd99e98 2024-04-26T01:49:11.112 INFO:journalctl@ceph.mds.g.smithi151.stdout:Apr 26 01:49:10 smithi151 systemd[1]: Started Ceph mds.g for c0e49e64-036d-11ef-bc93-c7b262605968. 2024-04-26T01:49:11.212 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:10 smithi165 ceph-mon[32648]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:11.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:11.272+0000 7f9ee0400700 1 -- 172.21.15.151:0/1974472547 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ed8101940 msgr2=0x7f9ed8101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:11.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:11.272+0000 7f9ee0400700 1 --2- 172.21.15.151:0/1974472547 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ed8101940 0x7f9ed8101dc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4009990 tx=0x7f9ec4009ca0 comp rx=0 tx=0).stop 2024-04-26T01:49:11.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:11.272+0000 7f9ee0400700 1 -- 172.21.15.151:0/1974472547 shutdown_connections 2024-04-26T01:49:11.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:11.272+0000 7f9ee0400700 1 --2- 172.21.15.151:0/1974472547 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ed8102300 0x7f9ed810e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:11.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:11.272+0000 7f9ee0400700 1 --2- 172.21.15.151:0/1974472547 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9ed8107990 0x7f9ed8107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:11.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:11.272+0000 7f9ee0400700 1 --2- 172.21.15.151:0/1974472547 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ed8101940 0x7f9ed8101dc0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:11.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:11.272+0000 7f9ee0400700 1 -- 172.21.15.151:0/1974472547 >> 172.21.15.151:0/1974472547 conn(0x7f9ed80788f0 msgr2=0x7f9ed8078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:11.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:11.273+0000 7f9ee0400700 1 -- 172.21.15.151:0/1974472547 shutdown_connections 2024-04-26T01:49:11.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:11.273+0000 7f9ee0400700 1 -- 172.21.15.151:0/1974472547 wait complete. 2024-04-26T01:49:11.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:11.274+0000 7f9ee0400700 1 Processor -- start 2024-04-26T01:49:11.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:11.274+0000 7f9ee0400700 1 -- start start 2024-04-26T01:49:11.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:11.274+0000 7f9ee0400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ed8101940 0x7f9ed819b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:11.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:11.274+0000 7f9ee0400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ed8102300 0x7f9ed819bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:11.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:11.274+0000 7f9ee0400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9ed8107990 0x7f9ed819fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:11.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:11.274+0000 7f9ee0400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed8111b30 con 0x7f9ed8101940 2024-04-26T01:49:11.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:11.274+0000 7f9ee0400700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed81119b0 con 0x7f9ed8107990 2024-04-26T01:49:11.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:11.274+0000 7f9ee0400700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed8111cb0 con 0x7f9ed8102300 2024-04-26T01:49:11.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:11.275+0000 7f9edda00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ed8101940 0x7f9ed819b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:11.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:11.275+0000 7f9edd000700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ed8102300 0x7f9ed819bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:11.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:11.275+0000 7f9edda00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ed8101940 0x7f9ed819b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:46964/0 (socket says 172.21.15.151:46964) 2024-04-26T01:49:11.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:11.275+0000 7f9edda00700 1 -- 172.21.15.151:0/1089957379 learned_addr learned my addr 172.21.15.151:0/1089957379 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:49:11.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:11.275+0000 7f9ede400700 1 --2- 172.21.15.151:0/1089957379 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9ed8107990 0x7f9ed819fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:11.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:11.275+0000 7f9edda00700 1 -- 172.21.15.151:0/1089957379 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ed8102300 msgr2=0x7f9ed819bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:11.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:11.275+0000 7f9edda00700 1 --2- 172.21.15.151:0/1089957379 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ed8102300 0x7f9ed819bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:11.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:11.275+0000 7f9edda00700 1 -- 172.21.15.151:0/1089957379 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9ed8107990 msgr2=0x7f9ed819fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:11.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:11.275+0000 7f9edda00700 1 --2- 172.21.15.151:0/1089957379 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9ed8107990 0x7f9ed819fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:11.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:11.275+0000 7f9edda00700 1 -- 172.21.15.151:0/1089957379 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ed81a0430 con 0x7f9ed8101940 2024-04-26T01:49:11.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:11.276+0000 7f9edda00700 1 --2- 172.21.15.151:0/1089957379 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ed8101940 0x7f9ed819b5e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc00e9a0 tx=0x7f9ecc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:11.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:11.276+0000 7f9ed6000700 1 -- 172.21.15.151:0/1089957379 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ecc00c9c0 con 0x7f9ed8101940 2024-04-26T01:49:11.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:11.276+0000 7f9ed6000700 1 -- 172.21.15.151:0/1089957379 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ecc00cb60 con 0x7f9ed8101940 2024-04-26T01:49:11.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:11.276+0000 7f9ed6000700 1 -- 172.21.15.151:0/1089957379 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ecc018680 con 0x7f9ed8101940 2024-04-26T01:49:11.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:11.276+0000 7f9ee0400700 1 -- 172.21.15.151:0/1089957379 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ed81a0720 con 0x7f9ed8101940 2024-04-26T01:49:11.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:11.276+0000 7f9ee0400700 1 -- 172.21.15.151:0/1089957379 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ed81a7fa0 con 0x7f9ed8101940 2024-04-26T01:49:11.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:11.277+0000 7f9ee0400700 1 -- 172.21.15.151:0/1089957379 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ed8069730 con 0x7f9ed8101940 2024-04-26T01:49:11.278 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:11.280+0000 7f9ed6000700 1 -- 172.21.15.151:0/1089957379 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f9ecc010430 con 0x7f9ed8101940 2024-04-26T01:49:11.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:11.281+0000 7f9ed6000700 1 --2- 172.21.15.151:0/1089957379 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f9ec00a3cc0 0x7f9ec00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:11.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:11.281+0000 7f9ed6000700 1 -- 172.21.15.151:0/1089957379 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f9ecc0d01c0 con 0x7f9ed8101940 2024-04-26T01:49:11.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:11.281+0000 7f9edd000700 1 --2- 172.21.15.151:0/1089957379 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f9ec00a3cc0 0x7f9ec00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:11.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:11.282+0000 7f9edd000700 1 --2- 172.21.15.151:0/1089957379 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f9ec00a3cc0 0x7f9ec00a6180 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4009750 tx=0x7f9ec400b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:11.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:11.287+0000 7f9ed6000700 1 -- 172.21.15.151:0/1089957379 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9ecc09cfa0 con 0x7f9ed8101940 2024-04-26T01:49:11.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:10 smithi172 ceph-mon[32785]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:11.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:11.568+0000 7f9ee0400700 1 -- 172.21.15.151:0/1089957379 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9ed81a8170 con 0x7f9ed8101940 2024-04-26T01:49:11.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:11.569+0000 7f9ed6000700 1 -- 172.21.15.151:0/1089957379 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8842 (secure 0 0 0) 0x7f9ecc09c8a0 con 0x7f9ed8101940 2024-04-26T01:49:11.567 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:49:11.567 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2937864097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2937864097},{"type":"v1","addr":"172.21.15.151:6835","nonce":2937864097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14373,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/2947090802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2947090802},{"type":"v1","addr":"172.21.15.151:6837","nonce":2947090802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/1356563280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1356563280},{"type":"v1","addr":"172.21.15.151:6839","nonce":1356563280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/504675912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":504675912},{"type":"v1","addr":"172.21.15.165:6833","nonce":504675912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1207580015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1207580015},{"type":"v1","addr":"172.21.15.172:6833","nonce":1207580015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/3598138294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3598138294},{"type":"v1","addr":"172.21.15.165:6835","nonce":3598138294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/1847859054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":1847859054},{"type":"v1","addr":"172.21.15.165:6837","nonce":1847859054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24481,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2309751171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2309751171},{"type":"v1","addr":"172.21.15.172:6835","nonce":2309751171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24493,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/55972537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":55972537},{"type":"v1","addr":"172.21.15.172:6837","nonce":55972537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24505,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6839/1242302810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1242302810},{"type":"v1","addr":"172.21.15.172:6839","nonce":1242302810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-26T01:49:11.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:11.572+0000 7f9ee0400700 1 -- 172.21.15.151:0/1089957379 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f9ec00a3cc0 msgr2=0x7f9ec00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:11.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:11.572+0000 7f9ee0400700 1 --2- 172.21.15.151:0/1089957379 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f9ec00a3cc0 0x7f9ec00a6180 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4009750 tx=0x7f9ec400b260 comp rx=0 tx=0).stop 2024-04-26T01:49:11.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:11.572+0000 7f9ee0400700 1 -- 172.21.15.151:0/1089957379 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ed8101940 msgr2=0x7f9ed819b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:11.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:11.572+0000 7f9ee0400700 1 --2- 172.21.15.151:0/1089957379 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ed8101940 0x7f9ed819b5e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc00e9a0 tx=0x7f9ecc00ecb0 comp rx=0 tx=0).stop 2024-04-26T01:49:11.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:11.572+0000 7f9ee0400700 1 -- 172.21.15.151:0/1089957379 shutdown_connections 2024-04-26T01:49:11.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:11.572+0000 7f9ee0400700 1 --2- 172.21.15.151:0/1089957379 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9ed8107990 0x7f9ed819fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:11.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:11.572+0000 7f9ee0400700 1 --2- 172.21.15.151:0/1089957379 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f9ec00a3cc0 0x7f9ec00a6180 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:11.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:11.572+0000 7f9ee0400700 1 --2- 172.21.15.151:0/1089957379 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ed8101940 0x7f9ed819b5e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:11.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:11.572+0000 7f9ee0400700 1 --2- 172.21.15.151:0/1089957379 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ed8102300 0x7f9ed819bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:11.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:11.572+0000 7f9ee0400700 1 -- 172.21.15.151:0/1089957379 >> 172.21.15.151:0/1089957379 conn(0x7f9ed80788f0 msgr2=0x7f9ed80fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:11.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:11.572+0000 7f9ee0400700 1 -- 172.21.15.151:0/1089957379 shutdown_connections 2024-04-26T01:49:11.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:11.572+0000 7f9ee0400700 1 -- 172.21.15.151:0/1089957379 wait complete. 2024-04-26T01:49:11.572 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 11 2024-04-26T01:49:12.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:11 smithi165 ceph-mon[32648]: mds.? [v2:172.21.15.151:6838/1356563280,v1:172.21.15.151:6839/1356563280] up:boot 2024-04-26T01:49:12.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:11 smithi165 ceph-mon[32648]: fsmap 10 up:standby 2024-04-26T01:49:12.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:11 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T01:49:12.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:11 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:12.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:11 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:12.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:11 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:12.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:11 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:49:12.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:11 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:49:12.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:11 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:49:12.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:11 smithi165 ceph-mon[32648]: Deploying daemon mds.a on smithi151 2024-04-26T01:49:12.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:11 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/1089957379' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:49:12.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:11 smithi172 ceph-mon[32785]: mds.? [v2:172.21.15.151:6838/1356563280,v1:172.21.15.151:6839/1356563280] up:boot 2024-04-26T01:49:12.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:11 smithi172 ceph-mon[32785]: fsmap 10 up:standby 2024-04-26T01:49:12.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:11 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T01:49:12.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:11 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:12.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:11 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:12.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:11 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:12.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:11 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:49:12.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:11 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:49:12.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:11 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:49:12.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:11 smithi172 ceph-mon[32785]: Deploying daemon mds.a on smithi151 2024-04-26T01:49:12.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:11 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/1089957379' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:49:12.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:11 smithi151 ceph-mon[28057]: mds.? [v2:172.21.15.151:6838/1356563280,v1:172.21.15.151:6839/1356563280] up:boot 2024-04-26T01:49:12.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:11 smithi151 ceph-mon[28057]: fsmap 10 up:standby 2024-04-26T01:49:12.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:11 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T01:49:12.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:11 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:12.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:11 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:12.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:11 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:12.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:11 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:49:12.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:11 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:49:12.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:11 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:49:12.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:11 smithi151 ceph-mon[28057]: Deploying daemon mds.a on smithi151 2024-04-26T01:49:12.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:11 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/1089957379' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:49:13.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:13 smithi172 ceph-mon[32785]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:13.567 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:13 smithi151 ceph-mon[28057]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:13.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:13 smithi165 ceph-mon[32648]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:14.522 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:49:15.105 INFO:journalctl@ceph.mds.a.smithi151.stdout:Apr 26 01:49:14 smithi151 systemd[1]: Starting Ceph mds.a for c0e49e64-036d-11ef-bc93-c7b262605968... 2024-04-26T01:49:15.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:15 smithi151 ceph-mon[28057]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:15.372 INFO:journalctl@ceph.mds.a.smithi151.stdout:Apr 26 01:49:15 smithi151 podman[69874]: 2024-04-26 01:49:15.012996448 +0000 UTC m=+0.043243040 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T01:49:15.372 INFO:journalctl@ceph.mds.a.smithi151.stdout:Apr 26 01:49:15 smithi151 podman[69874]: 2024-04-26 01:49:15.108697666 +0000 UTC m=+0.138944289 container create b6ec99fbb17c460fede77cfe9c643c9571428497502e3adec126dab579f0bb82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-a, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416) 2024-04-26T01:49:15.498 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:49:15.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:15 smithi172 ceph-mon[32785]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:15.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:15 smithi165 ceph-mon[32648]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:15.695 INFO:journalctl@ceph.mds.a.smithi151.stdout:Apr 26 01:49:15 smithi151 podman[69874]: 2024-04-26 01:49:15.405292025 +0000 UTC m=+0.435538661 container init b6ec99fbb17c460fede77cfe9c643c9571428497502e3adec126dab579f0bb82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-a, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-26T01:49:15.695 INFO:journalctl@ceph.mds.a.smithi151.stdout:Apr 26 01:49:15 smithi151 podman[69874]: 2024-04-26 01:49:15.415066827 +0000 UTC m=+0.445313427 container start b6ec99fbb17c460fede77cfe9c643c9571428497502e3adec126dab579f0bb82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-a, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-26T01:49:15.695 INFO:journalctl@ceph.mds.a.smithi151.stdout:Apr 26 01:49:15 smithi151 ceph-mds[69917]: -- 172.21.15.151:0/2248950796 <== mon.1 v2:172.21.15.172:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5605a4f8fa40 con 0x5605a4f62c00 2024-04-26T01:49:15.695 INFO:journalctl@ceph.mds.a.smithi151.stdout:Apr 26 01:49:15 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-a[69912]: starting mds.a at 2024-04-26T01:49:15.695 INFO:journalctl@ceph.mds.a.smithi151.stdout:Apr 26 01:49:15 smithi151 bash[69874]: b6ec99fbb17c460fede77cfe9c643c9571428497502e3adec126dab579f0bb82 2024-04-26T01:49:15.695 INFO:journalctl@ceph.mds.a.smithi151.stdout:Apr 26 01:49:15 smithi151 systemd[1]: Started Ceph mds.a for c0e49e64-036d-11ef-bc93-c7b262605968. 2024-04-26T01:49:16.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:16 smithi165 ceph-mon[32648]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:16.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:16 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:16.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:16 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:16.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:16 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:16.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:16 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:49:16.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:16 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:49:16.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:16 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:49:16.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:16 smithi165 ceph-mon[32648]: Deploying daemon mds.b on smithi165 2024-04-26T01:49:16.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:16 smithi165 ceph-mon[32648]: mds.? [v2:172.21.15.151:6840/1075240552,v1:172.21.15.151:6841/1075240552] up:boot 2024-04-26T01:49:16.847 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:16 smithi165 ceph-mon[32648]: fsmap 11 up:standby 2024-04-26T01:49:16.847 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:16 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T01:49:16.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:16 smithi151 ceph-mon[28057]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:16.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:16 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:16.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:16 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:16.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:16 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:16.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:16 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:49:16.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:16 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:49:16.873 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:16 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:49:16.873 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:16 smithi151 ceph-mon[28057]: Deploying daemon mds.b on smithi165 2024-04-26T01:49:16.873 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:16 smithi151 ceph-mon[28057]: mds.? [v2:172.21.15.151:6840/1075240552,v1:172.21.15.151:6841/1075240552] up:boot 2024-04-26T01:49:16.873 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:16 smithi151 ceph-mon[28057]: fsmap 11 up:standby 2024-04-26T01:49:16.873 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:16 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T01:49:17.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:16 smithi172 ceph-mon[32785]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:17.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:16 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:17.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:16 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:17.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:16 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:17.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:16 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:49:17.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:16 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:49:17.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:16 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:49:17.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:16 smithi172 ceph-mon[32785]: Deploying daemon mds.b on smithi165 2024-04-26T01:49:17.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:16 smithi172 ceph-mon[32785]: mds.? [v2:172.21.15.151:6840/1075240552,v1:172.21.15.151:6841/1075240552] up:boot 2024-04-26T01:49:17.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:16 smithi172 ceph-mon[32785]: fsmap 11 up:standby 2024-04-26T01:49:17.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:16 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T01:49:17.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:17.731+0000 7f70e0800700 1 -- 172.21.15.151:0/2838413815 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f70d8109c80 msgr2=0x7f70d810dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:17.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:17.731+0000 7f70e0800700 1 --2- 172.21.15.151:0/2838413815 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f70d8109c80 0x7f70d810dcd0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f70c80099c0 tx=0x7f70c8009cd0 comp rx=0 tx=0).stop 2024-04-26T01:49:17.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:17.731+0000 7f70e0800700 1 -- 172.21.15.151:0/2838413815 shutdown_connections 2024-04-26T01:49:17.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:17.731+0000 7f70e0800700 1 --2- 172.21.15.151:0/2838413815 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70d810e3b0 0x7f70d8115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:17.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:17.731+0000 7f70e0800700 1 --2- 172.21.15.151:0/2838413815 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f70d81092d0 0x7f70d81096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:17.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:17.731+0000 7f70e0800700 1 --2- 172.21.15.151:0/2838413815 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f70d8109c80 0x7f70d810dcd0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:17.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:17.731+0000 7f70e0800700 1 -- 172.21.15.151:0/2838413815 >> 172.21.15.151:0/2838413815 conn(0x7f70d8076210 msgr2=0x7f70d8107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:17.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:17.731+0000 7f70e0800700 1 -- 172.21.15.151:0/2838413815 shutdown_connections 2024-04-26T01:49:17.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:17.731+0000 7f70e0800700 1 -- 172.21.15.151:0/2838413815 wait complete. 2024-04-26T01:49:17.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:17.732+0000 7f70e0800700 1 Processor -- start 2024-04-26T01:49:17.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:17.732+0000 7f70e0800700 1 -- start start 2024-04-26T01:49:17.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:17.732+0000 7f70e0800700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f70d81092d0 0x7f70d819f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:17.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:17.732+0000 7f70e0800700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70d8109c80 0x7f70d819fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:17.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:17.732+0000 7f70e0800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f70d810e3b0 0x7f70d81a3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:17.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:17.732+0000 7f70e0800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f70d8119290 con 0x7f70d810e3b0 2024-04-26T01:49:17.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:17.732+0000 7f70e0800700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f70d8119110 con 0x7f70d81092d0 2024-04-26T01:49:17.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:17.732+0000 7f70e0800700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f70d8119410 con 0x7f70d8109c80 2024-04-26T01:49:17.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:17.733+0000 7f70dde00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f70d81092d0 0x7f70d819f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:17.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:17.733+0000 7f70dde00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f70d81092d0 0x7f70d819f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:46976/0 (socket says 172.21.15.151:46976) 2024-04-26T01:49:17.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:17.733+0000 7f70dde00700 1 -- 172.21.15.151:0/4067870669 learned_addr learned my addr 172.21.15.151:0/4067870669 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:49:17.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:17.733+0000 7f70de800700 1 --2- 172.21.15.151:0/4067870669 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f70d810e3b0 0x7f70d81a3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:17.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:17.733+0000 7f70dd400700 1 --2- 172.21.15.151:0/4067870669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70d8109c80 0x7f70d819fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:17.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:17.733+0000 7f70dde00700 1 -- 172.21.15.151:0/4067870669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70d8109c80 msgr2=0x7f70d819fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:17.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:17.733+0000 7f70dde00700 1 --2- 172.21.15.151:0/4067870669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70d8109c80 0x7f70d819fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:17.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:17.733+0000 7f70dde00700 1 -- 172.21.15.151:0/4067870669 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f70d810e3b0 msgr2=0x7f70d81a3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:17.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:17.733+0000 7f70dde00700 1 --2- 172.21.15.151:0/4067870669 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f70d810e3b0 0x7f70d81a3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:17.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:17.733+0000 7f70dde00700 1 -- 172.21.15.151:0/4067870669 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70d81a4730 con 0x7f70d81092d0 2024-04-26T01:49:17.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:17.733+0000 7f70de800700 1 --2- 172.21.15.151:0/4067870669 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f70d810e3b0 0x7f70d81a3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:49:17.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:17.733+0000 7f70dde00700 1 --2- 172.21.15.151:0/4067870669 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f70d81092d0 0x7f70d819f720 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f70cc00e9a0 tx=0x7f70cc00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:17.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:17.734+0000 7f70d2a00700 1 -- 172.21.15.151:0/4067870669 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70cc00cbd0 con 0x7f70d81092d0 2024-04-26T01:49:17.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:17.734+0000 7f70d2a00700 1 -- 172.21.15.151:0/4067870669 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70cc00cd70 con 0x7f70d81092d0 2024-04-26T01:49:17.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:17.734+0000 7f70d2a00700 1 -- 172.21.15.151:0/4067870669 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70cc018680 con 0x7f70d81092d0 2024-04-26T01:49:17.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:17.734+0000 7f70e0800700 1 -- 172.21.15.151:0/4067870669 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70d81a4a20 con 0x7f70d81092d0 2024-04-26T01:49:17.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:17.734+0000 7f70e0800700 1 -- 172.21.15.151:0/4067870669 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70d81ac2a0 con 0x7f70d81092d0 2024-04-26T01:49:17.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:17.735+0000 7f70e0800700 1 -- 172.21.15.151:0/4067870669 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70d8069730 con 0x7f70d81092d0 2024-04-26T01:49:17.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:17.737+0000 7f70d2a00700 1 -- 172.21.15.151:0/4067870669 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f70cc010430 con 0x7f70d81092d0 2024-04-26T01:49:17.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:17.738+0000 7f70d2a00700 1 --2- 172.21.15.151:0/4067870669 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f70c00a3cc0 0x7f70c00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:17.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:17.738+0000 7f70d2a00700 1 -- 172.21.15.151:0/4067870669 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f70cc0d01a0 con 0x7f70d81092d0 2024-04-26T01:49:17.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:17.738+0000 7f70dd400700 1 --2- 172.21.15.151:0/4067870669 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f70c00a3cc0 0x7f70c00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:17.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:17.739+0000 7f70dd400700 1 --2- 172.21.15.151:0/4067870669 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f70c00a3cc0 0x7f70c00a6180 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f70c8000c00 tx=0x7f70c8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:17.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:17.743+0000 7f70d2a00700 1 -- 172.21.15.151:0/4067870669 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f70cc09cf80 con 0x7f70d81092d0 2024-04-26T01:49:17.846 INFO:journalctl@ceph.mds.b.smithi165.stdout:Apr 26 01:49:17 smithi165 systemd[1]: Starting Ceph mds.b for c0e49e64-036d-11ef-bc93-c7b262605968... 2024-04-26T01:49:18.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:18.022+0000 7f70e0800700 1 -- 172.21.15.151:0/4067870669 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f70d802df80 con 0x7f70d81092d0 2024-04-26T01:49:18.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:18.023+0000 7f70d2a00700 1 -- 172.21.15.151:0/4067870669 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9671 (secure 0 0 0) 0x7f70cc09c880 con 0x7f70d81092d0 2024-04-26T01:49:18.021 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:49:18.021 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2937864097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2937864097},{"type":"v1","addr":"172.21.15.151:6835","nonce":2937864097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14373,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/2947090802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2947090802},{"type":"v1","addr":"172.21.15.151:6837","nonce":2947090802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/1356563280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1356563280},{"type":"v1","addr":"172.21.15.151:6839","nonce":1356563280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6841/1075240552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1075240552},{"type":"v1","addr":"172.21.15.151:6841","nonce":1075240552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/504675912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":504675912},{"type":"v1","addr":"172.21.15.165:6833","nonce":504675912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1207580015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1207580015},{"type":"v1","addr":"172.21.15.172:6833","nonce":1207580015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/3598138294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3598138294},{"type":"v1","addr":"172.21.15.165:6835","nonce":3598138294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/1847859054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":1847859054},{"type":"v1","addr":"172.21.15.165:6837","nonce":1847859054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24481,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2309751171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2309751171},{"type":"v1","addr":"172.21.15.172:6835","nonce":2309751171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24493,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/55972537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":55972537},{"type":"v1","addr":"172.21.15.172:6837","nonce":55972537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24505,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6839/1242302810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1242302810},{"type":"v1","addr":"172.21.15.172:6839","nonce":1242302810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-26T01:49:18.024 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:18.026+0000 7f70e0800700 1 -- 172.21.15.151:0/4067870669 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f70c00a3cc0 msgr2=0x7f70c00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:18.024 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:18.026+0000 7f70e0800700 1 --2- 172.21.15.151:0/4067870669 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f70c00a3cc0 0x7f70c00a6180 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f70c8000c00 tx=0x7f70c8005fb0 comp rx=0 tx=0).stop 2024-04-26T01:49:18.024 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:18.026+0000 7f70e0800700 1 -- 172.21.15.151:0/4067870669 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f70d81092d0 msgr2=0x7f70d819f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:18.024 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:18.026+0000 7f70e0800700 1 --2- 172.21.15.151:0/4067870669 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f70d81092d0 0x7f70d819f720 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f70cc00e9a0 tx=0x7f70cc00ecb0 comp rx=0 tx=0).stop 2024-04-26T01:49:18.024 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:18.026+0000 7f70e0800700 1 -- 172.21.15.151:0/4067870669 shutdown_connections 2024-04-26T01:49:18.024 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:18.026+0000 7f70e0800700 1 --2- 172.21.15.151:0/4067870669 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f70c00a3cc0 0x7f70c00a6180 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:18.024 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:18.026+0000 7f70e0800700 1 --2- 172.21.15.151:0/4067870669 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f70d810e3b0 0x7f70d81a3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:18.024 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:18.026+0000 7f70e0800700 1 --2- 172.21.15.151:0/4067870669 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f70d81092d0 0x7f70d819f720 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:18.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:18.026+0000 7f70e0800700 1 --2- 172.21.15.151:0/4067870669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70d8109c80 0x7f70d819fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:18.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:18.026+0000 7f70e0800700 1 -- 172.21.15.151:0/4067870669 >> 172.21.15.151:0/4067870669 conn(0x7f70d8076210 msgr2=0x7f70d8076830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:18.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:18.026+0000 7f70e0800700 1 -- 172.21.15.151:0/4067870669 shutdown_connections 2024-04-26T01:49:18.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:18.026+0000 7f70e0800700 1 -- 172.21.15.151:0/4067870669 wait complete. 2024-04-26T01:49:18.025 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 12 2024-04-26T01:49:18.168 INFO:journalctl@ceph.mds.b.smithi165.stdout:Apr 26 01:49:17 smithi165 podman[63310]: 2024-04-26 01:49:17.777346664 +0000 UTC m=+0.017870900 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T01:49:18.168 INFO:journalctl@ceph.mds.b.smithi165.stdout:Apr 26 01:49:17 smithi165 podman[63310]: 2024-04-26 01:49:17.875947576 +0000 UTC m=+0.116471790 container create 749c302b691920f004dc0795d12e06e988f191cacad1881a32c3eb35a8043fab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-b, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-26T01:49:18.168 INFO:journalctl@ceph.mds.b.smithi165.stdout:Apr 26 01:49:18 smithi165 podman[63310]: 2024-04-26 01:49:18.086978221 +0000 UTC m=+0.327502434 container init 749c302b691920f004dc0795d12e06e988f191cacad1881a32c3eb35a8043fab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True) 2024-04-26T01:49:18.168 INFO:journalctl@ceph.mds.b.smithi165.stdout:Apr 26 01:49:18 smithi165 podman[63310]: 2024-04-26 01:49:18.089837581 +0000 UTC m=+0.330361795 container start 749c302b691920f004dc0795d12e06e988f191cacad1881a32c3eb35a8043fab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-b, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-26T01:49:18.168 INFO:journalctl@ceph.mds.b.smithi165.stdout:Apr 26 01:49:18 smithi165 ceph-mds[63329]: -- 172.21.15.165:0/1409106056 <== mon.2 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557f5a417a40 con 0x557f5a3eac00 2024-04-26T01:49:18.168 INFO:journalctl@ceph.mds.b.smithi165.stdout:Apr 26 01:49:18 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-b[63325]: starting mds.b at 2024-04-26T01:49:18.168 INFO:journalctl@ceph.mds.b.smithi165.stdout:Apr 26 01:49:18 smithi165 bash[63310]: 749c302b691920f004dc0795d12e06e988f191cacad1881a32c3eb35a8043fab 2024-04-26T01:49:18.584 INFO:journalctl@ceph.mds.b.smithi165.stdout:Apr 26 01:49:18 smithi165 systemd[1]: Started Ceph mds.b for c0e49e64-036d-11ef-bc93-c7b262605968. 2024-04-26T01:49:18.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:18 smithi165 ceph-mon[32648]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:18.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:18 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/4067870669' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:49:18.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:18 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:18.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:18 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:18.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:18 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:18.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:18 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:18.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:18 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:18.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:18 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:49:18.855 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:18 smithi151 ceph-mon[28057]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:18.855 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:18 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/4067870669' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:49:18.856 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:18 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:18.856 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:18 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:18.856 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:18 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:18.856 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:18 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:18.856 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:18 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:18.856 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:18 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:49:19.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:18 smithi172 ceph-mon[32785]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:19.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:18 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/4067870669' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:49:19.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:18 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:19.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:18 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:19.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:18 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:19.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:18 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:19.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:18 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:19.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:18 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:49:19.649 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:19 smithi165 ceph-mon[32648]: mds.? [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] up:boot 2024-04-26T01:49:19.649 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:19 smithi165 ceph-mon[32648]: fsmap 12 up:standby 2024-04-26T01:49:19.649 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:19 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T01:49:19.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:19 smithi172 ceph-mon[32785]: mds.? [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] up:boot 2024-04-26T01:49:19.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:19 smithi172 ceph-mon[32785]: fsmap 12 up:standby 2024-04-26T01:49:19.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:19 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T01:49:19.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:19 smithi151 ceph-mon[28057]: mds.? [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] up:boot 2024-04-26T01:49:19.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:19 smithi151 ceph-mon[28057]: fsmap 12 up:standby 2024-04-26T01:49:19.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:19 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T01:49:20.917 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:49:21.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:21 smithi151 ceph-mon[28057]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:21.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:21 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:21.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:21 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:21.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:21 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:21.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:21 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:21.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:21 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:21.392 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:49:21.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:21 smithi172 ceph-mon[32785]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:21.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:21 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:21.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:21 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:21.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:21 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:21.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:21 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:21.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:21 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:21.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:21 smithi165 ceph-mon[32648]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:21.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:21 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:21.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:21 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:21.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:21 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:21.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:21 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:21.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:21 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:22.581 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:22 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:22.582 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:22 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:22.582 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:22 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:49:22.582 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:22 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:49:22.582 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:22 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:22.582 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:22 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:49:22.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:22 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:22.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:22 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:22.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:22 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:49:22.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:22 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:49:22.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:22 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:22.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:22 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:49:22.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:22 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:22.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:22 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:22.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:22 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:49:22.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:22 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:49:22.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:22 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:22.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:22 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:49:23.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:23.490+0000 7fecfde00700 1 -- 172.21.15.151:0/3270166822 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fecf806a750 msgr2=0x7fecf806abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:23.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:23.490+0000 7fecfde00700 1 --2- 172.21.15.151:0/3270166822 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fecf806a750 0x7fecf806abb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fece40098f0 tx=0x7fece4009c00 comp rx=0 tx=0).stop 2024-04-26T01:49:23.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:23.490+0000 7fecfde00700 1 -- 172.21.15.151:0/3270166822 shutdown_connections 2024-04-26T01:49:23.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:23.490+0000 7fecfde00700 1 --2- 172.21.15.151:0/3270166822 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fecf806b0f0 0x7fecf810f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:23.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:23.490+0000 7fecfde00700 1 --2- 172.21.15.151:0/3270166822 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fecf8108a30 0x7fecf8108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:23.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:23.490+0000 7fecfde00700 1 --2- 172.21.15.151:0/3270166822 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fecf806a750 0x7fecf806abb0 secure :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fece40098f0 tx=0x7fece4009c00 comp rx=0 tx=0).stop 2024-04-26T01:49:23.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:23.490+0000 7fecfde00700 1 -- 172.21.15.151:0/3270166822 >> 172.21.15.151:0/3270166822 conn(0x7fecf80788f0 msgr2=0x7fecf8078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:23.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:23.491+0000 7fecfde00700 1 -- 172.21.15.151:0/3270166822 shutdown_connections 2024-04-26T01:49:23.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:23.491+0000 7fecfde00700 1 -- 172.21.15.151:0/3270166822 wait complete. 2024-04-26T01:49:23.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:23.492+0000 7fecfde00700 1 Processor -- start 2024-04-26T01:49:23.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:23.492+0000 7fecfde00700 1 -- start start 2024-04-26T01:49:23.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:23.492+0000 7fecfde00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fecf806b0f0 0x7fecf8101a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:23.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:23.492+0000 7fecfde00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fecf8108a30 0x7fecf8101f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:23.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:23.492+0000 7fecfde00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fecf8102490 0x7fecf81a75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:23.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:23.492+0000 7fecfde00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fecf8112bf0 con 0x7fecf8102490 2024-04-26T01:49:23.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:23.492+0000 7fecfde00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fecf8112a70 con 0x7fecf806b0f0 2024-04-26T01:49:23.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:23.492+0000 7fecfde00700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fecf8112d70 con 0x7fecf8108a30 2024-04-26T01:49:23.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:23.493+0000 7fecf6000700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fecf8108a30 0x7fecf8101f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:23.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:23.493+0000 7fecf6000700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fecf8108a30 0x7fecf8101f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:37938/0 (socket says 172.21.15.151:37938) 2024-04-26T01:49:23.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:23.493+0000 7fecf6000700 1 -- 172.21.15.151:0/1942403831 learned_addr learned my addr 172.21.15.151:0/1942403831 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:49:23.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:23.493+0000 7fecf6a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fecf806b0f0 0x7fecf8101a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:23.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:23.493+0000 7fecf7400700 1 --2- 172.21.15.151:0/1942403831 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fecf8102490 0x7fecf81a75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:23.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:23.493+0000 7fecf6000700 1 -- 172.21.15.151:0/1942403831 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fecf806b0f0 msgr2=0x7fecf8101a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:23.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:23.493+0000 7fecf6000700 1 --2- 172.21.15.151:0/1942403831 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fecf806b0f0 0x7fecf8101a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:23.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:23.493+0000 7fecf6000700 1 -- 172.21.15.151:0/1942403831 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fecf8102490 msgr2=0x7fecf81a75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:23.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:23.493+0000 7fecf6000700 1 --2- 172.21.15.151:0/1942403831 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fecf8102490 0x7fecf81a75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:23.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:23.493+0000 7fecf6000700 1 -- 172.21.15.151:0/1942403831 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecf81a7d50 con 0x7fecf8108a30 2024-04-26T01:49:23.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:23.494+0000 7fecf6000700 1 --2- 172.21.15.151:0/1942403831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fecf8108a30 0x7fecf8101f50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fece4009230 tx=0x7fece400fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:23.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:23.494+0000 7fecfca00700 1 -- 172.21.15.151:0/1942403831 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece4005040 con 0x7fecf8108a30 2024-04-26T01:49:23.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:23.494+0000 7fecfca00700 1 -- 172.21.15.151:0/1942403831 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fece4005690 con 0x7fecf8108a30 2024-04-26T01:49:23.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:23.494+0000 7fecfca00700 1 -- 172.21.15.151:0/1942403831 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece40038e0 con 0x7fecf8108a30 2024-04-26T01:49:23.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:23.494+0000 7fecfde00700 1 -- 172.21.15.151:0/1942403831 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecf81a7fe0 con 0x7fecf8108a30 2024-04-26T01:49:23.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:23.494+0000 7fecfde00700 1 -- 172.21.15.151:0/1942403831 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecf81a8380 con 0x7fecf8108a30 2024-04-26T01:49:23.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:23.496+0000 7fecfde00700 1 -- 172.21.15.151:0/1942403831 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecf8069730 con 0x7fecf8108a30 2024-04-26T01:49:23.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:23.498+0000 7fecfca00700 1 -- 172.21.15.151:0/1942403831 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fece400bcd0 con 0x7fecf8108a30 2024-04-26T01:49:23.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:23.500+0000 7fecfca00700 1 --2- 172.21.15.151:0/1942403831 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fecdc0a3cc0 0x7fecdc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:23.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:23.500+0000 7fecfca00700 1 -- 172.21.15.151:0/1942403831 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fece40d0d20 con 0x7fecf8108a30 2024-04-26T01:49:23.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:23.500+0000 7fecf6a00700 1 --2- 172.21.15.151:0/1942403831 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fecdc0a3cc0 0x7fecdc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:23.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:23.501+0000 7fecf6a00700 1 --2- 172.21.15.151:0/1942403831 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fecdc0a3cc0 0x7fecdc0a6180 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fece8009be0 tx=0x7fece8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:23.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:23.504+0000 7fecfca00700 1 -- 172.21.15.151:0/1942403831 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fece4013030 con 0x7fecf8108a30 2024-04-26T01:49:23.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:23 smithi151 ceph-mon[28057]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:23.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:23 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:23.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:23 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:23.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:23 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:23.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:23 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:23.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:23 smithi172 ceph-mon[32785]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:23.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:23 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:23.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:23 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:23.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:23 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:23.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:23 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:23.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:23.789+0000 7fecfde00700 1 -- 172.21.15.151:0/1942403831 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fecf802e140 con 0x7fecf8108a30 2024-04-26T01:49:23.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:23.789+0000 7fecfca00700 1 -- 172.21.15.151:0/1942403831 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10500 (secure 0 0 0) 0x7fece409db90 con 0x7fecf8108a30 2024-04-26T01:49:23.788 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:49:23.788 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2937864097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2937864097},{"type":"v1","addr":"172.21.15.151:6835","nonce":2937864097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14373,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/2947090802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2947090802},{"type":"v1","addr":"172.21.15.151:6837","nonce":2947090802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/1356563280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1356563280},{"type":"v1","addr":"172.21.15.151:6839","nonce":1356563280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6841/1075240552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1075240552},{"type":"v1","addr":"172.21.15.151:6841","nonce":1075240552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/504675912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":504675912},{"type":"v1","addr":"172.21.15.165:6833","nonce":504675912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1207580015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1207580015},{"type":"v1","addr":"172.21.15.172:6833","nonce":1207580015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/3598138294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3598138294},{"type":"v1","addr":"172.21.15.165:6835","nonce":3598138294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/1847859054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":1847859054},{"type":"v1","addr":"172.21.15.165:6837","nonce":1847859054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24481,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2309751171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2309751171},{"type":"v1","addr":"172.21.15.172:6835","nonce":2309751171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24493,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/55972537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":55972537},{"type":"v1","addr":"172.21.15.172:6837","nonce":55972537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24505,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6839/1242302810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1242302810},{"type":"v1","addr":"172.21.15.172:6839","nonce":1242302810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24509,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6839/1969716254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1969716254},{"type":"v1","addr":"172.21.15.165:6839","nonce":1969716254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-26T01:49:23.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:23.794+0000 7fecfde00700 1 -- 172.21.15.151:0/1942403831 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fecdc0a3cc0 msgr2=0x7fecdc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:23.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:23.794+0000 7fecfde00700 1 --2- 172.21.15.151:0/1942403831 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fecdc0a3cc0 0x7fecdc0a6180 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fece8009be0 tx=0x7fece8009260 comp rx=0 tx=0).stop 2024-04-26T01:49:23.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:23.794+0000 7fecfde00700 1 -- 172.21.15.151:0/1942403831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fecf8108a30 msgr2=0x7fecf8101f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:23.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:23.794+0000 7fecfde00700 1 --2- 172.21.15.151:0/1942403831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fecf8108a30 0x7fecf8101f50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fece4009230 tx=0x7fece400fa00 comp rx=0 tx=0).stop 2024-04-26T01:49:23.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:23.794+0000 7fecfde00700 1 -- 172.21.15.151:0/1942403831 shutdown_connections 2024-04-26T01:49:23.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:23.794+0000 7fecfde00700 1 --2- 172.21.15.151:0/1942403831 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fecdc0a3cc0 0x7fecdc0a6180 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:23.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:23.794+0000 7fecfde00700 1 --2- 172.21.15.151:0/1942403831 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fecf8102490 0x7fecf81a75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:23.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:23.794+0000 7fecfde00700 1 --2- 172.21.15.151:0/1942403831 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fecf806b0f0 0x7fecf8101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:23.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:23.794+0000 7fecfde00700 1 --2- 172.21.15.151:0/1942403831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fecf8108a30 0x7fecf8101f50 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:23.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:23.794+0000 7fecfde00700 1 -- 172.21.15.151:0/1942403831 >> 172.21.15.151:0/1942403831 conn(0x7fecf80788f0 msgr2=0x7fecf80ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:23.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:23.794+0000 7fecfde00700 1 -- 172.21.15.151:0/1942403831 shutdown_connections 2024-04-26T01:49:23.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:23.794+0000 7fecfde00700 1 -- 172.21.15.151:0/1942403831 wait complete. 2024-04-26T01:49:23.794 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 13 2024-04-26T01:49:23.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:23 smithi165 ceph-mon[32648]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:23.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:23 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:23.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:23 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:23.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:23 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:23.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:23 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:24.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:24 smithi151 ceph-mon[28057]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:24.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:24 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/1942403831' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:49:24.710 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-04-26T01:49:24.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:24 smithi172 ceph-mon[32785]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:24.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:24 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/1942403831' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:49:24.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:24 smithi165 ceph-mon[32648]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:24.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:24 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/1942403831' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:49:25.139 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:49:26.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:26 smithi151 ceph-mon[28057]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:26.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:26 smithi172 ceph-mon[32785]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:26.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:26 smithi165 ceph-mon[32648]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:27.488 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:27.490+0000 7f91ba000700 1 -- 172.21.15.151:0/169389819 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91b4102a90 msgr2=0x7f91b4102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:27.488 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:27.490+0000 7f91ba000700 1 --2- 172.21.15.151:0/169389819 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91b4102a90 0x7f91b4102f10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f919c0098f0 tx=0x7f919c009c00 comp rx=0 tx=0).stop 2024-04-26T01:49:27.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:27.490+0000 7f91ba000700 1 -- 172.21.15.151:0/169389819 shutdown_connections 2024-04-26T01:49:27.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:27.490+0000 7f91ba000700 1 --2- 172.21.15.151:0/169389819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91b4103450 0x7f91b410f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:27.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:27.490+0000 7f91ba000700 1 --2- 172.21.15.151:0/169389819 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91b4108a50 0x7f91b4108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:27.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:27.490+0000 7f91ba000700 1 --2- 172.21.15.151:0/169389819 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91b4102a90 0x7f91b4102f10 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:27.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:27.490+0000 7f91ba000700 1 -- 172.21.15.151:0/169389819 >> 172.21.15.151:0/169389819 conn(0x7f91b4079ba0 msgr2=0x7f91b4079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:27.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:27.490+0000 7f91ba000700 1 -- 172.21.15.151:0/169389819 shutdown_connections 2024-04-26T01:49:27.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:27.490+0000 7f91ba000700 1 -- 172.21.15.151:0/169389819 wait complete. 2024-04-26T01:49:27.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:27.491+0000 7f91ba000700 1 Processor -- start 2024-04-26T01:49:27.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:27.491+0000 7f91ba000700 1 -- start start 2024-04-26T01:49:27.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:27.492+0000 7f91ba000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91b4102a90 0x7f91b4074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:27.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:27.492+0000 7f91ba000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91b4103450 0x7f91b4075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:27.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:27.492+0000 7f91ba000700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91b4108a50 0x7f91b40757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:27.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:27.492+0000 7f91ba000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f91b4112c50 con 0x7f91b4103450 2024-04-26T01:49:27.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:27.492+0000 7f91ba000700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f91b4112ad0 con 0x7f91b4102a90 2024-04-26T01:49:27.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:27.492+0000 7f91ba000700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f91b4112dd0 con 0x7f91b4108a50 2024-04-26T01:49:27.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:27.492+0000 7f91b3400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91b4102a90 0x7f91b4074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:27.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:27.492+0000 7f91b3e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91b4108a50 0x7f91b40757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:27.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:27.492+0000 7f91b3e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91b4108a50 0x7f91b40757c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:37944/0 (socket says 172.21.15.151:37944) 2024-04-26T01:49:27.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:27.492+0000 7f91b3e00700 1 -- 172.21.15.151:0/4260665013 learned_addr learned my addr 172.21.15.151:0/4260665013 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:49:27.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:27.492+0000 7f91abe00700 1 --2- 172.21.15.151:0/4260665013 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91b4103450 0x7f91b4075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:27.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:27.492+0000 7f91b3400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91b4102a90 0x7f91b4074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:45880/0 (socket says 172.21.15.151:45880) 2024-04-26T01:49:27.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:27.493+0000 7f91b3e00700 1 -- 172.21.15.151:0/4260665013 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91b4102a90 msgr2=0x7f91b4074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:27.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:27.493+0000 7f91b3e00700 1 --2- 172.21.15.151:0/4260665013 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91b4102a90 0x7f91b4074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:27.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:27.493+0000 7f91b3e00700 1 -- 172.21.15.151:0/4260665013 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91b4103450 msgr2=0x7f91b4075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:27.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:27.493+0000 7f91b3e00700 1 --2- 172.21.15.151:0/4260665013 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91b4103450 0x7f91b4075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:27.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:27.493+0000 7f91b3e00700 1 -- 172.21.15.151:0/4260665013 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91b41a76f0 con 0x7f91b4108a50 2024-04-26T01:49:27.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:27.493+0000 7f91b3e00700 1 --2- 172.21.15.151:0/4260665013 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91b4108a50 0x7f91b40757c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f91ac00ef30 tx=0x7f91ac00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:27.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:27.493+0000 7f91b0c00700 1 -- 172.21.15.151:0/4260665013 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91ac0092e0 con 0x7f91b4108a50 2024-04-26T01:49:27.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:27.493+0000 7f91ba000700 1 -- 172.21.15.151:0/4260665013 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91b41a7980 con 0x7f91b4108a50 2024-04-26T01:49:27.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:27.493+0000 7f91b0c00700 1 -- 172.21.15.151:0/4260665013 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91ac004bb0 con 0x7f91b4108a50 2024-04-26T01:49:27.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:27.493+0000 7f91b0c00700 1 -- 172.21.15.151:0/4260665013 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91ac004020 con 0x7f91b4108a50 2024-04-26T01:49:27.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:27.494+0000 7f91ba000700 1 -- 172.21.15.151:0/4260665013 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91b41a7e90 con 0x7f91b4108a50 2024-04-26T01:49:27.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:27.495+0000 7f91ba000700 1 -- 172.21.15.151:0/4260665013 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91b4069730 con 0x7f91b4108a50 2024-04-26T01:49:27.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:27.498+0000 7f91b0c00700 1 -- 172.21.15.151:0/4260665013 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f91ac01b030 con 0x7f91b4108a50 2024-04-26T01:49:27.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:27.499+0000 7f91b0c00700 1 --2- 172.21.15.151:0/4260665013 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f91980a3cc0 0x7f91980a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:27.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:27.500+0000 7f91b0c00700 1 -- 172.21.15.151:0/4260665013 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f91ac0d03c0 con 0x7f91b4108a50 2024-04-26T01:49:27.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:27.500+0000 7f91b3400700 1 --2- 172.21.15.151:0/4260665013 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f91980a3cc0 0x7f91980a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:27.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:27.501+0000 7f91b3400700 1 --2- 172.21.15.151:0/4260665013 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f91980a3cc0 0x7f91980a6180 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f91a4005fd0 tx=0x7f91a4009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:27.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:27.501+0000 7f91b0c00700 1 -- 172.21.15.151:0/4260665013 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f91ac022020 con 0x7f91b4108a50 2024-04-26T01:49:27.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:27.747+0000 7f91ba000700 1 -- 172.21.15.151:0/4260665013 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f91b402e140 con 0x7f91b4108a50 2024-04-26T01:49:27.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:27.748+0000 7f91b0c00700 1 -- 172.21.15.151:0/4260665013 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v83) ==== 84+0+65 (secure 0 0 0) 0x7f91ac010040 con 0x7f91b4108a50 2024-04-26T01:49:27.746 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:49:27.746 INFO:teuthology.orchestra.run.smithi151.stdout:[ 2024-04-26T01:49:27.746 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T01:49:27.746 INFO:teuthology.orchestra.run.smithi151.stdout: "poolnum": 1, 2024-04-26T01:49:27.746 INFO:teuthology.orchestra.run.smithi151.stdout: "poolname": ".mgr" 2024-04-26T01:49:27.746 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T01:49:27.747 INFO:teuthology.orchestra.run.smithi151.stdout:] 2024-04-26T01:49:27.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:27.751+0000 7f91ba000700 1 -- 172.21.15.151:0/4260665013 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f91980a3cc0 msgr2=0x7f91980a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:27.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:27.751+0000 7f91ba000700 1 --2- 172.21.15.151:0/4260665013 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f91980a3cc0 0x7f91980a6180 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f91a4005fd0 tx=0x7f91a4009310 comp rx=0 tx=0).stop 2024-04-26T01:49:27.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:27.751+0000 7f91ba000700 1 -- 172.21.15.151:0/4260665013 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91b4108a50 msgr2=0x7f91b40757c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:27.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:27.751+0000 7f91ba000700 1 --2- 172.21.15.151:0/4260665013 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91b4108a50 0x7f91b40757c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f91ac00ef30 tx=0x7f91ac00c3a0 comp rx=0 tx=0).stop 2024-04-26T01:49:27.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:27.751+0000 7f91ba000700 1 -- 172.21.15.151:0/4260665013 shutdown_connections 2024-04-26T01:49:27.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:27.751+0000 7f91ba000700 1 --2- 172.21.15.151:0/4260665013 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91b4108a50 0x7f91b40757c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:27.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:27.751+0000 7f91ba000700 1 --2- 172.21.15.151:0/4260665013 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91b4102a90 0x7f91b4074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:27.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:27.751+0000 7f91ba000700 1 --2- 172.21.15.151:0/4260665013 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f91980a3cc0 0x7f91980a6180 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:27.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:27.751+0000 7f91ba000700 1 --2- 172.21.15.151:0/4260665013 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91b4103450 0x7f91b4075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:27.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:27.751+0000 7f91ba000700 1 -- 172.21.15.151:0/4260665013 >> 172.21.15.151:0/4260665013 conn(0x7f91b4079ba0 msgr2=0x7f91b4101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:27.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:27.751+0000 7f91ba000700 1 -- 172.21.15.151:0/4260665013 shutdown_connections 2024-04-26T01:49:27.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:27.751+0000 7f91ba000700 1 -- 172.21.15.151:0/4260665013 wait complete. 2024-04-26T01:49:28.654 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T01:49:28.654 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T01:49:28.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:28 smithi165 ceph-mon[32648]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:28.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:28 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:28.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:28 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:28.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:28 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:49:28.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:28 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:49:28.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:28 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:28.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:28 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/4260665013' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-26T01:49:28.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:28 smithi151 ceph-mon[28057]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:28.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:28 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:28.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:28 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:28.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:28 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:49:28.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:28 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:49:28.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:28 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:28.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:28 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/4260665013' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-26T01:49:29.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:28 smithi172 ceph-mon[32785]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:29.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:28 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:29.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:28 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:29.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:28 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:49:29.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:28 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:49:29.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:28 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:49:29.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:28 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/4260665013' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-26T01:49:29.097 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:49:30.648 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:30 smithi165 ceph-mon[32648]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:30.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:30 smithi151 ceph-mon[28057]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:31.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:30 smithi172 ceph-mon[32785]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:31.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:31.495+0000 7fb8e8a00700 1 -- 172.21.15.151:0/3143016160 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8e4109c80 msgr2=0x7fb8e410dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:31.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:31.495+0000 7fb8e8a00700 1 --2- 172.21.15.151:0/3143016160 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8e4109c80 0x7fb8e410dcd0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0009920 tx=0x7fb8d0009c30 comp rx=0 tx=0).stop 2024-04-26T01:49:31.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:31.495+0000 7fb8e8a00700 1 -- 172.21.15.151:0/3143016160 shutdown_connections 2024-04-26T01:49:31.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:31.495+0000 7fb8e8a00700 1 --2- 172.21.15.151:0/3143016160 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb8e410e3b0 0x7fb8e4115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:31.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:31.495+0000 7fb8e8a00700 1 --2- 172.21.15.151:0/3143016160 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb8e41092d0 0x7fb8e41096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:31.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:31.495+0000 7fb8e8a00700 1 --2- 172.21.15.151:0/3143016160 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8e4109c80 0x7fb8e410dcd0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:31.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:31.495+0000 7fb8e8a00700 1 -- 172.21.15.151:0/3143016160 >> 172.21.15.151:0/3143016160 conn(0x7fb8e4076210 msgr2=0x7fb8e4107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:31.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:31.495+0000 7fb8e8a00700 1 -- 172.21.15.151:0/3143016160 shutdown_connections 2024-04-26T01:49:31.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:31.495+0000 7fb8e8a00700 1 -- 172.21.15.151:0/3143016160 wait complete. 2024-04-26T01:49:31.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:31.496+0000 7fb8e8a00700 1 Processor -- start 2024-04-26T01:49:31.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:31.496+0000 7fb8e8a00700 1 -- start start 2024-04-26T01:49:31.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:31.497+0000 7fb8e8a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb8e41092d0 0x7fb8e41a5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:31.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:31.497+0000 7fb8e8a00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8e4109c80 0x7fb8e41a5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:31.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:31.497+0000 7fb8e8a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb8e410e3b0 0x7fb8e419f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:31.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:31.497+0000 7fb8e8a00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e4119270 con 0x7fb8e41092d0 2024-04-26T01:49:31.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:31.497+0000 7fb8e8a00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e41190f0 con 0x7fb8e410e3b0 2024-04-26T01:49:31.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:31.497+0000 7fb8e8a00700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e41193f0 con 0x7fb8e4109c80 2024-04-26T01:49:31.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:31.497+0000 7fb8e1e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb8e41092d0 0x7fb8e41a5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:31.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:31.497+0000 7fb8e1e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb8e41092d0 0x7fb8e41a5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44810/0 (socket says 172.21.15.151:44810) 2024-04-26T01:49:31.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:31.497+0000 7fb8e1e00700 1 -- 172.21.15.151:0/2568720143 learned_addr learned my addr 172.21.15.151:0/2568720143 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:49:31.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:31.497+0000 7fb8e1400700 1 --2- 172.21.15.151:0/2568720143 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8e4109c80 0x7fb8e41a5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:31.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:31.497+0000 7fb8e2800700 1 --2- 172.21.15.151:0/2568720143 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb8e410e3b0 0x7fb8e419f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:31.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:31.498+0000 7fb8e1e00700 1 -- 172.21.15.151:0/2568720143 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8e4109c80 msgr2=0x7fb8e41a5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:31.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:31.498+0000 7fb8e1e00700 1 --2- 172.21.15.151:0/2568720143 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8e4109c80 0x7fb8e41a5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:31.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:31.498+0000 7fb8e1e00700 1 -- 172.21.15.151:0/2568720143 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb8e410e3b0 msgr2=0x7fb8e419f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:31.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:31.498+0000 7fb8e1e00700 1 --2- 172.21.15.151:0/2568720143 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb8e410e3b0 0x7fb8e419f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:31.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:31.498+0000 7fb8e1e00700 1 -- 172.21.15.151:0/2568720143 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8e419ff70 con 0x7fb8e41092d0 2024-04-26T01:49:31.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:31.498+0000 7fb8e1400700 1 --2- 172.21.15.151:0/2568720143 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8e4109c80 0x7fb8e41a5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:49:31.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:31.498+0000 7fb8e1e00700 1 --2- 172.21.15.151:0/2568720143 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb8e41092d0 0x7fb8e41a5570 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb8d400e3c0 tx=0x7fb8d400e6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:31.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:31.498+0000 7fb8daa00700 1 -- 172.21.15.151:0/2568720143 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d400ee40 con 0x7fb8e41092d0 2024-04-26T01:49:31.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:31.498+0000 7fb8daa00700 1 -- 172.21.15.151:0/2568720143 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8d4003a00 con 0x7fb8e41092d0 2024-04-26T01:49:31.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:31.498+0000 7fb8e8a00700 1 -- 172.21.15.151:0/2568720143 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8e41a0200 con 0x7fb8e41092d0 2024-04-26T01:49:31.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:31.498+0000 7fb8daa00700 1 -- 172.21.15.151:0/2568720143 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d4004720 con 0x7fb8e41092d0 2024-04-26T01:49:31.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:31.499+0000 7fb8e8a00700 1 -- 172.21.15.151:0/2568720143 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8e41ac4f0 con 0x7fb8e41092d0 2024-04-26T01:49:31.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:31.500+0000 7fb8e8a00700 1 -- 172.21.15.151:0/2568720143 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8e410a7b0 con 0x7fb8e41092d0 2024-04-26T01:49:31.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:31.502+0000 7fb8daa00700 1 -- 172.21.15.151:0/2568720143 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fb8d401b030 con 0x7fb8e41092d0 2024-04-26T01:49:31.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:31.503+0000 7fb8daa00700 1 --2- 172.21.15.151:0/2568720143 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fb8c80a3cc0 0x7fb8c80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:31.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:31.503+0000 7fb8daa00700 1 -- 172.21.15.151:0/2568720143 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fb8d40cf8a0 con 0x7fb8e41092d0 2024-04-26T01:49:31.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:31.503+0000 7fb8e1400700 1 --2- 172.21.15.151:0/2568720143 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fb8c80a3cc0 0x7fb8c80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:31.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:31.504+0000 7fb8e1400700 1 --2- 172.21.15.151:0/2568720143 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fb8c80a3cc0 0x7fb8c80a6180 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb8d00098f0 tx=0x7fb8d0019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:31.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:31.508+0000 7fb8daa00700 1 -- 172.21.15.151:0/2568720143 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb8d4022060 con 0x7fb8e41092d0 2024-04-26T01:49:31.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:31.738+0000 7fb8e8a00700 1 -- 172.21.15.151:0/2568720143 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fb8e41a0b70 con 0x7fb8e41092d0 2024-04-26T01:49:32.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:32.571+0000 7fb8daa00700 1 -- 172.21.15.151:0/2568720143 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v84) ==== 138+0+0 (secure 0 0 0) 0x7fb8d4010040 con 0x7fb8e41092d0 2024-04-26T01:49:32.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:32.576+0000 7fb8e8a00700 1 -- 172.21.15.151:0/2568720143 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fb8c80a3cc0 msgr2=0x7fb8c80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:32.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:32.576+0000 7fb8e8a00700 1 --2- 172.21.15.151:0/2568720143 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fb8c80a3cc0 0x7fb8c80a6180 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb8d00098f0 tx=0x7fb8d0019040 comp rx=0 tx=0).stop 2024-04-26T01:49:32.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:32.576+0000 7fb8e8a00700 1 -- 172.21.15.151:0/2568720143 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb8e41092d0 msgr2=0x7fb8e41a5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:32.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:32.576+0000 7fb8e8a00700 1 --2- 172.21.15.151:0/2568720143 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb8e41092d0 0x7fb8e41a5570 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb8d400e3c0 tx=0x7fb8d400e6d0 comp rx=0 tx=0).stop 2024-04-26T01:49:32.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:32.576+0000 7fb8e8a00700 1 -- 172.21.15.151:0/2568720143 shutdown_connections 2024-04-26T01:49:32.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:32.576+0000 7fb8e8a00700 1 --2- 172.21.15.151:0/2568720143 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb8e410e3b0 0x7fb8e419f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:32.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:32.576+0000 7fb8e8a00700 1 --2- 172.21.15.151:0/2568720143 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fb8c80a3cc0 0x7fb8c80a6180 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:32.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:32.577+0000 7fb8e8a00700 1 --2- 172.21.15.151:0/2568720143 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb8e41092d0 0x7fb8e41a5570 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:32.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:32.577+0000 7fb8e8a00700 1 --2- 172.21.15.151:0/2568720143 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8e4109c80 0x7fb8e41a5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:32.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:32.577+0000 7fb8e8a00700 1 -- 172.21.15.151:0/2568720143 >> 172.21.15.151:0/2568720143 conn(0x7fb8e4076210 msgr2=0x7fb8e4077e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:32.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:32.577+0000 7fb8e8a00700 1 -- 172.21.15.151:0/2568720143 shutdown_connections 2024-04-26T01:49:32.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:32.577+0000 7fb8e8a00700 1 -- 172.21.15.151:0/2568720143 wait complete. 2024-04-26T01:49:32.581 INFO:teuthology.orchestra.run.smithi151.stderr:pool 'cephfs_metadata' created 2024-04-26T01:49:32.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:32 smithi165 ceph-mon[32648]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:32.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:32 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/2568720143' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-26T01:49:32.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:32 smithi151 ceph-mon[28057]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:32.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:32 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/2568720143' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-26T01:49:33.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:32 smithi172 ceph-mon[32785]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:33.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:32 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/2568720143' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-26T01:49:33.486 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T01:49:33.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:33 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/2568720143' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-26T01:49:33.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:33 smithi165 ceph-mon[32648]: osdmap e84: 12 total, 12 up, 12 in 2024-04-26T01:49:33.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:33 smithi165 ceph-mon[32648]: osdmap e85: 12 total, 12 up, 12 in 2024-04-26T01:49:33.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:33 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/2568720143' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-26T01:49:33.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:33 smithi151 ceph-mon[28057]: osdmap e84: 12 total, 12 up, 12 in 2024-04-26T01:49:33.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:33 smithi151 ceph-mon[28057]: osdmap e85: 12 total, 12 up, 12 in 2024-04-26T01:49:33.937 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:33 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/2568720143' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-26T01:49:33.937 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:33 smithi172 ceph-mon[32785]: osdmap e84: 12 total, 12 up, 12 in 2024-04-26T01:49:33.937 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:33 smithi172 ceph-mon[32785]: osdmap e85: 12 total, 12 up, 12 in 2024-04-26T01:49:33.947 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:49:34.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:34 smithi165 ceph-mon[32648]: pgmap v318: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:34.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:34 smithi165 ceph-mon[32648]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T01:49:34.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:34 smithi165 ceph-mon[32648]: osdmap e86: 12 total, 12 up, 12 in 2024-04-26T01:49:34.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:34 smithi151 ceph-mon[28057]: pgmap v318: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:34.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:34 smithi151 ceph-mon[28057]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T01:49:34.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:34 smithi151 ceph-mon[28057]: osdmap e86: 12 total, 12 up, 12 in 2024-04-26T01:49:34.955 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:34 smithi172 ceph-mon[32785]: pgmap v318: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:34.955 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:34 smithi172 ceph-mon[32785]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T01:49:34.955 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:34 smithi172 ceph-mon[32785]: osdmap e86: 12 total, 12 up, 12 in 2024-04-26T01:49:36.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:36.272+0000 7fde7d000700 1 -- 172.21.15.151:0/1397989906 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde781092d0 msgr2=0x7fde781096b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:36.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:36.272+0000 7fde7d000700 1 --2- 172.21.15.151:0/1397989906 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde781092d0 0x7fde781096b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fde68009920 tx=0x7fde68009c30 comp rx=0 tx=0).stop 2024-04-26T01:49:36.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:36.272+0000 7fde7d000700 1 -- 172.21.15.151:0/1397989906 shutdown_connections 2024-04-26T01:49:36.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:36.272+0000 7fde7d000700 1 --2- 172.21.15.151:0/1397989906 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde7810e3b0 0x7fde78115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:36.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:36.272+0000 7fde7d000700 1 --2- 172.21.15.151:0/1397989906 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde781092d0 0x7fde781096b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:36.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:36.272+0000 7fde7d000700 1 --2- 172.21.15.151:0/1397989906 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fde78109c80 0x7fde7810dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:36.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:36.272+0000 7fde7d000700 1 -- 172.21.15.151:0/1397989906 >> 172.21.15.151:0/1397989906 conn(0x7fde78076210 msgr2=0x7fde78107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:36.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:36.272+0000 7fde7d000700 1 -- 172.21.15.151:0/1397989906 shutdown_connections 2024-04-26T01:49:36.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:36.272+0000 7fde7d000700 1 -- 172.21.15.151:0/1397989906 wait complete. 2024-04-26T01:49:36.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:36.273+0000 7fde7d000700 1 Processor -- start 2024-04-26T01:49:36.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:36.273+0000 7fde7d000700 1 -- start start 2024-04-26T01:49:36.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:36.274+0000 7fde7d000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde781092d0 0x7fde781a5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:36.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:36.274+0000 7fde7d000700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde78109c80 0x7fde781a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:36.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:36.274+0000 7fde7d000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fde7810e3b0 0x7fde7819f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:36.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:36.274+0000 7fde7d000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fde78119290 con 0x7fde781092d0 2024-04-26T01:49:36.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:36.274+0000 7fde7d000700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fde78119110 con 0x7fde7810e3b0 2024-04-26T01:49:36.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:36.274+0000 7fde7d000700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fde78119410 con 0x7fde78109c80 2024-04-26T01:49:36.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:36.274+0000 7fde75e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde781092d0 0x7fde781a5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:36.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:36.274+0000 7fde75400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde78109c80 0x7fde781a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:36.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:36.274+0000 7fde75e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde781092d0 0x7fde781a5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44824/0 (socket says 172.21.15.151:44824) 2024-04-26T01:49:36.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:36.274+0000 7fde75e00700 1 -- 172.21.15.151:0/1000480155 learned_addr learned my addr 172.21.15.151:0/1000480155 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:49:36.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:36.274+0000 7fde76800700 1 --2- 172.21.15.151:0/1000480155 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fde7810e3b0 0x7fde7819f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:36.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:36.275+0000 7fde75400700 1 -- 172.21.15.151:0/1000480155 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fde7810e3b0 msgr2=0x7fde7819f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:36.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:36.275+0000 7fde75400700 1 --2- 172.21.15.151:0/1000480155 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fde7810e3b0 0x7fde7819f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:36.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:36.275+0000 7fde75400700 1 -- 172.21.15.151:0/1000480155 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde781092d0 msgr2=0x7fde781a5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:36.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:36.275+0000 7fde75400700 1 --2- 172.21.15.151:0/1000480155 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde781092d0 0x7fde781a5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:36.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:36.275+0000 7fde75400700 1 -- 172.21.15.151:0/1000480155 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde7819ff10 con 0x7fde78109c80 2024-04-26T01:49:36.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:36.275+0000 7fde75400700 1 --2- 172.21.15.151:0/1000480155 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde78109c80 0x7fde781a5aa0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fde6400e9c0 tx=0x7fde6400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:36.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:36.275+0000 7fde6ea00700 1 -- 172.21.15.151:0/1000480155 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6400c9c0 con 0x7fde78109c80 2024-04-26T01:49:36.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:36.275+0000 7fde6ea00700 1 -- 172.21.15.151:0/1000480155 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde6400cb60 con 0x7fde78109c80 2024-04-26T01:49:36.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:36.275+0000 7fde7d000700 1 -- 172.21.15.151:0/1000480155 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde781a0200 con 0x7fde78109c80 2024-04-26T01:49:36.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:36.275+0000 7fde6ea00700 1 -- 172.21.15.151:0/1000480155 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde64018680 con 0x7fde78109c80 2024-04-26T01:49:36.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:36.275+0000 7fde7d000700 1 -- 172.21.15.151:0/1000480155 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde781ac4e0 con 0x7fde78109c80 2024-04-26T01:49:36.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:36.277+0000 7fde7d000700 1 -- 172.21.15.151:0/1000480155 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde7810a7b0 con 0x7fde78109c80 2024-04-26T01:49:36.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:36.278+0000 7fde6ea00700 1 -- 172.21.15.151:0/1000480155 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fde64010430 con 0x7fde78109c80 2024-04-26T01:49:36.278 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:36.280+0000 7fde6ea00700 1 --2- 172.21.15.151:0/1000480155 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fde5c0a3cc0 0x7fde5c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:36.278 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:36.280+0000 7fde6ea00700 1 -- 172.21.15.151:0/1000480155 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7497+0+0 (secure 0 0 0) 0x7fde640d0240 con 0x7fde78109c80 2024-04-26T01:49:36.278 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:36.280+0000 7fde75e00700 1 --2- 172.21.15.151:0/1000480155 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fde5c0a3cc0 0x7fde5c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:36.278 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:36.281+0000 7fde75e00700 1 --2- 172.21.15.151:0/1000480155 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fde5c0a3cc0 0x7fde5c0a6180 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fde680098f0 tx=0x7fde68019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:36.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:36.284+0000 7fde6ea00700 1 -- 172.21.15.151:0/1000480155 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fde6409ceb0 con 0x7fde78109c80 2024-04-26T01:49:36.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:36.512+0000 7fde7d000700 1 -- 172.21.15.151:0/1000480155 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fde781a0b60 con 0x7fde78109c80 2024-04-26T01:49:36.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:36.621+0000 7fde6ea00700 1 -- 172.21.15.151:0/1000480155 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v87) ==== 170+0+0 (secure 0 0 0) 0x7fde6409c7b0 con 0x7fde78109c80 2024-04-26T01:49:36.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:36.628+0000 7fde7d000700 1 -- 172.21.15.151:0/1000480155 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fde5c0a3cc0 msgr2=0x7fde5c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:36.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:36.628+0000 7fde7d000700 1 --2- 172.21.15.151:0/1000480155 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fde5c0a3cc0 0x7fde5c0a6180 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fde680098f0 tx=0x7fde68019040 comp rx=0 tx=0).stop 2024-04-26T01:49:36.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:36.628+0000 7fde7d000700 1 -- 172.21.15.151:0/1000480155 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde78109c80 msgr2=0x7fde781a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:36.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:36.628+0000 7fde7d000700 1 --2- 172.21.15.151:0/1000480155 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde78109c80 0x7fde781a5aa0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fde6400e9c0 tx=0x7fde6400ecd0 comp rx=0 tx=0).stop 2024-04-26T01:49:36.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:36.629+0000 7fde7d000700 1 -- 172.21.15.151:0/1000480155 shutdown_connections 2024-04-26T01:49:36.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:36.629+0000 7fde7d000700 1 --2- 172.21.15.151:0/1000480155 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fde7810e3b0 0x7fde7819f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:36.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:36.629+0000 7fde7d000700 1 --2- 172.21.15.151:0/1000480155 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fde5c0a3cc0 0x7fde5c0a6180 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:36.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:36.629+0000 7fde7d000700 1 --2- 172.21.15.151:0/1000480155 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde781092d0 0x7fde781a5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:36.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:36.629+0000 7fde7d000700 1 --2- 172.21.15.151:0/1000480155 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde78109c80 0x7fde781a5aa0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:36.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:36.629+0000 7fde7d000700 1 -- 172.21.15.151:0/1000480155 >> 172.21.15.151:0/1000480155 conn(0x7fde78076210 msgr2=0x7fde781086f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:36.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:36.629+0000 7fde7d000700 1 -- 172.21.15.151:0/1000480155 shutdown_connections 2024-04-26T01:49:36.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:36.630+0000 7fde7d000700 1 -- 172.21.15.151:0/1000480155 wait complete. 2024-04-26T01:49:36.631 INFO:teuthology.orchestra.run.smithi151.stderr:pool 'cephfs_data' created 2024-04-26T01:49:36.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:36 smithi151 ceph-mon[28057]: pgmap v321: 33 pgs: 10 active+clean, 3 creating+peering, 20 unknown; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:36.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:36 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/1000480155' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-26T01:49:36.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:36 smithi151 ceph-mon[28057]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-26T01:49:37.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:36 smithi172 ceph-mon[32785]: pgmap v321: 33 pgs: 10 active+clean, 3 creating+peering, 20 unknown; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:37.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:36 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/1000480155' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-26T01:49:37.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:36 smithi172 ceph-mon[32785]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-26T01:49:37.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:36 smithi165 ceph-mon[32648]: pgmap v321: 33 pgs: 10 active+clean, 3 creating+peering, 20 unknown; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:37.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:36 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/1000480155' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-26T01:49:37.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:36 smithi165 ceph-mon[32648]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-26T01:49:37.599 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T01:49:37.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:37 smithi151 ceph-mon[28057]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-26T01:49:37.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:37 smithi151 ceph-mon[28057]: osdmap e87: 12 total, 12 up, 12 in 2024-04-26T01:49:38.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:37 smithi172 ceph-mon[32785]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-26T01:49:38.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:37 smithi172 ceph-mon[32785]: osdmap e87: 12 total, 12 up, 12 in 2024-04-26T01:49:38.041 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:49:38.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:37 smithi165 ceph-mon[32648]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-26T01:49:38.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:37 smithi165 ceph-mon[32648]: osdmap e87: 12 total, 12 up, 12 in 2024-04-26T01:49:38.736 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:38 smithi151 ceph-mon[28057]: pgmap v323: 97 pgs: 30 active+clean, 31 creating+peering, 36 unknown; 577 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:38.736 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:38 smithi151 ceph-mon[28057]: osdmap e88: 12 total, 12 up, 12 in 2024-04-26T01:49:39.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:38 smithi172 ceph-mon[32785]: pgmap v323: 97 pgs: 30 active+clean, 31 creating+peering, 36 unknown; 577 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:39.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:38 smithi172 ceph-mon[32785]: osdmap e88: 12 total, 12 up, 12 in 2024-04-26T01:49:39.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:38 smithi165 ceph-mon[32648]: pgmap v323: 97 pgs: 30 active+clean, 31 creating+peering, 36 unknown; 577 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:39.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:38 smithi165 ceph-mon[32648]: osdmap e88: 12 total, 12 up, 12 in 2024-04-26T01:49:39.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:39 smithi151 ceph-mon[28057]: osdmap e89: 12 total, 12 up, 12 in 2024-04-26T01:49:39.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:39 smithi151 ceph-mon[28057]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T01:49:40.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:39 smithi172 ceph-mon[32785]: osdmap e89: 12 total, 12 up, 12 in 2024-04-26T01:49:40.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:39 smithi172 ceph-mon[32785]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T01:49:40.097 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:39 smithi165 ceph-mon[32648]: osdmap e89: 12 total, 12 up, 12 in 2024-04-26T01:49:40.097 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:39 smithi165 ceph-mon[32648]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T01:49:40.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:40.456+0000 7f84c7e00700 1 -- 172.21.15.151:0/2922621464 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84c0109c80 msgr2=0x7f84c010dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:40.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:40.456+0000 7f84c7e00700 1 --2- 172.21.15.151:0/2922621464 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84c0109c80 0x7f84c010dcd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f84ac0099c0 tx=0x7f84ac009cd0 comp rx=0 tx=0).stop 2024-04-26T01:49:40.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:40.456+0000 7f84c7e00700 1 -- 172.21.15.151:0/2922621464 shutdown_connections 2024-04-26T01:49:40.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:40.456+0000 7f84c7e00700 1 --2- 172.21.15.151:0/2922621464 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84c010e3b0 0x7f84c0115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:40.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:40.456+0000 7f84c7e00700 1 --2- 172.21.15.151:0/2922621464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f84c01092d0 0x7f84c01096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:40.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:40.456+0000 7f84c7e00700 1 --2- 172.21.15.151:0/2922621464 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84c0109c80 0x7f84c010dcd0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:40.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:40.456+0000 7f84c7e00700 1 -- 172.21.15.151:0/2922621464 >> 172.21.15.151:0/2922621464 conn(0x7f84c0076210 msgr2=0x7f84c0107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:40.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:40.456+0000 7f84c7e00700 1 -- 172.21.15.151:0/2922621464 shutdown_connections 2024-04-26T01:49:40.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:40.456+0000 7f84c7e00700 1 -- 172.21.15.151:0/2922621464 wait complete. 2024-04-26T01:49:40.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:40.457+0000 7f84c7e00700 1 Processor -- start 2024-04-26T01:49:40.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:40.458+0000 7f84c7e00700 1 -- start start 2024-04-26T01:49:40.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:40.458+0000 7f84c7e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84c01092d0 0x7f84c019f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:40.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:40.458+0000 7f84c7e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f84c0109c80 0x7f84c019fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:40.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:40.458+0000 7f84c7e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84c010e3b0 0x7f84c01a3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:40.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:40.458+0000 7f84c7e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f84c0119290 con 0x7f84c0109c80 2024-04-26T01:49:40.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:40.458+0000 7f84c7e00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f84c0119110 con 0x7f84c01092d0 2024-04-26T01:49:40.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:40.458+0000 7f84c7e00700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f84c0119410 con 0x7f84c010e3b0 2024-04-26T01:49:40.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:40.459+0000 7f84c5400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84c01092d0 0x7f84c019f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:40.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:40.459+0000 7f84c4a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f84c0109c80 0x7f84c019fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:40.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:40.459+0000 7f84c5400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84c01092d0 0x7f84c019f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:60346/0 (socket says 172.21.15.151:60346) 2024-04-26T01:49:40.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:40.459+0000 7f84c5400700 1 -- 172.21.15.151:0/1440107799 learned_addr learned my addr 172.21.15.151:0/1440107799 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:49:40.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:40.459+0000 7f84c5e00700 1 --2- 172.21.15.151:0/1440107799 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84c010e3b0 0x7f84c01a3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:40.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:40.459+0000 7f84c5400700 1 -- 172.21.15.151:0/1440107799 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84c010e3b0 msgr2=0x7f84c01a3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:40.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:40.459+0000 7f84c5400700 1 --2- 172.21.15.151:0/1440107799 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84c010e3b0 0x7f84c01a3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:40.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:40.459+0000 7f84c5400700 1 -- 172.21.15.151:0/1440107799 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f84c0109c80 msgr2=0x7f84c019fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:40.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:40.459+0000 7f84c5400700 1 --2- 172.21.15.151:0/1440107799 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f84c0109c80 0x7f84c019fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:40.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:40.459+0000 7f84c5400700 1 -- 172.21.15.151:0/1440107799 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84c01a4730 con 0x7f84c01092d0 2024-04-26T01:49:40.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:40.459+0000 7f84c4a00700 1 --2- 172.21.15.151:0/1440107799 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f84c0109c80 0x7f84c019fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:49:40.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:40.459+0000 7f84c5400700 1 --2- 172.21.15.151:0/1440107799 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84c01092d0 0x7f84c019f720 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f84b400eb30 tx=0x7f84b400ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:40.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:40.460+0000 7f84be000700 1 -- 172.21.15.151:0/1440107799 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b400cbd0 con 0x7f84c01092d0 2024-04-26T01:49:40.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:40.460+0000 7f84be000700 1 -- 172.21.15.151:0/1440107799 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84b40044d0 con 0x7f84c01092d0 2024-04-26T01:49:40.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:40.460+0000 7f84c7e00700 1 -- 172.21.15.151:0/1440107799 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84c01a4a20 con 0x7f84c01092d0 2024-04-26T01:49:40.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:40.460+0000 7f84be000700 1 -- 172.21.15.151:0/1440107799 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b4003e30 con 0x7f84c01092d0 2024-04-26T01:49:40.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:40.460+0000 7f84c7e00700 1 -- 172.21.15.151:0/1440107799 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84c01ac2a0 con 0x7f84c01092d0 2024-04-26T01:49:40.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:40.461+0000 7f84c7e00700 1 -- 172.21.15.151:0/1440107799 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84c0069730 con 0x7f84c01092d0 2024-04-26T01:49:40.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:40.463+0000 7f84be000700 1 -- 172.21.15.151:0/1440107799 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f84b4004000 con 0x7f84c01092d0 2024-04-26T01:49:40.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:40.464+0000 7f84be000700 1 --2- 172.21.15.151:0/1440107799 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f84a80a3cc0 0x7f84a80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:40.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:40.465+0000 7f84be000700 1 -- 172.21.15.151:0/1440107799 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7879+0+0 (secure 0 0 0) 0x7f84b40d0640 con 0x7f84c01092d0 2024-04-26T01:49:40.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:40.465+0000 7f84c4a00700 1 --2- 172.21.15.151:0/1440107799 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f84a80a3cc0 0x7f84a80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:40.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:40.465+0000 7f84c4a00700 1 --2- 172.21.15.151:0/1440107799 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f84a80a3cc0 0x7f84a80a6180 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f84ac000c00 tx=0x7f84ac005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:40.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:40.469+0000 7f84be000700 1 -- 172.21.15.151:0/1440107799 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f84b40042f0 con 0x7f84c01092d0 2024-04-26T01:49:40.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:40.744+0000 7f84c7e00700 1 -- 172.21.15.151:0/1440107799 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f84c002df80 con 0x7f84c01092d0 2024-04-26T01:49:40.743 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:40 smithi151 ceph-mon[28057]: pgmap v326: 97 pgs: 35 active+clean, 28 creating+peering, 34 unknown; 577 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:40.945 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:40 smithi165 ceph-mon[32648]: pgmap v326: 97 pgs: 35 active+clean, 28 creating+peering, 34 unknown; 577 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:41.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:40 smithi172 ceph-mon[32785]: pgmap v326: 97 pgs: 35 active+clean, 28 creating+peering, 34 unknown; 577 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:41.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:41.676+0000 7f84be000700 1 -- 172.21.15.151:0/1440107799 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f84b409d130 con 0x7f84c01092d0 2024-04-26T01:49:41.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:41.682+0000 7f84c7e00700 1 -- 172.21.15.151:0/1440107799 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f84a80a3cc0 msgr2=0x7f84a80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:41.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:41.682+0000 7f84c7e00700 1 --2- 172.21.15.151:0/1440107799 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f84a80a3cc0 0x7f84a80a6180 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f84ac000c00 tx=0x7f84ac005fb0 comp rx=0 tx=0).stop 2024-04-26T01:49:41.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:41.682+0000 7f84c7e00700 1 -- 172.21.15.151:0/1440107799 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84c01092d0 msgr2=0x7f84c019f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:41.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:41.682+0000 7f84c7e00700 1 --2- 172.21.15.151:0/1440107799 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84c01092d0 0x7f84c019f720 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f84b400eb30 tx=0x7f84b400ee40 comp rx=0 tx=0).stop 2024-04-26T01:49:41.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:41.682+0000 7f84c7e00700 1 -- 172.21.15.151:0/1440107799 shutdown_connections 2024-04-26T01:49:41.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:41.682+0000 7f84c7e00700 1 --2- 172.21.15.151:0/1440107799 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84c010e3b0 0x7f84c01a3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:41.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:41.682+0000 7f84c7e00700 1 --2- 172.21.15.151:0/1440107799 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84c01092d0 0x7f84c019f720 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:41.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:41.682+0000 7f84c7e00700 1 --2- 172.21.15.151:0/1440107799 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f84a80a3cc0 0x7f84a80a6180 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:41.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:41.682+0000 7f84c7e00700 1 --2- 172.21.15.151:0/1440107799 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f84c0109c80 0x7f84c019fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:41.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:41.682+0000 7f84c7e00700 1 -- 172.21.15.151:0/1440107799 >> 172.21.15.151:0/1440107799 conn(0x7f84c0076210 msgr2=0x7f84c01086f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:41.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:41.684+0000 7f84c7e00700 1 -- 172.21.15.151:0/1440107799 shutdown_connections 2024-04-26T01:49:41.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:41.685+0000 7f84c7e00700 1 -- 172.21.15.151:0/1440107799 wait complete. 2024-04-26T01:49:41.686 INFO:teuthology.orchestra.run.smithi151.stderr:new fs with metadata pool 2 and data pool 3 2024-04-26T01:49:41.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:41 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/1440107799' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T01:49:41.845 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:41 smithi165 ceph-mon[32648]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T01:49:41.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:41 smithi165 ceph-mon[32648]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-26T01:49:42.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:41 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/1440107799' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T01:49:42.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:41 smithi172 ceph-mon[32785]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T01:49:42.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:41 smithi172 ceph-mon[32785]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-26T01:49:42.113 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:41 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/1440107799' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T01:49:42.113 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:41 smithi151 ceph-mon[28057]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T01:49:42.113 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:41 smithi151 ceph-mon[28057]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-26T01:49:42.742 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T01:49:43.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:42 smithi172 ceph-mon[32785]: pgmap v327: 97 pgs: 63 active+clean, 28 creating+peering, 6 unknown; 577 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:43.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:42 smithi172 ceph-mon[32785]: osdmap e90: 12 total, 12 up, 12 in 2024-04-26T01:49:43.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:42 smithi172 ceph-mon[32785]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-26T01:49:43.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:42 smithi172 ceph-mon[32785]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-26T01:49:43.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:42 smithi172 ceph-mon[32785]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-26T01:49:43.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:42 smithi165 ceph-mon[32648]: pgmap v327: 97 pgs: 63 active+clean, 28 creating+peering, 6 unknown; 577 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:43.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:42 smithi165 ceph-mon[32648]: osdmap e90: 12 total, 12 up, 12 in 2024-04-26T01:49:43.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:42 smithi165 ceph-mon[32648]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-26T01:49:43.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:42 smithi165 ceph-mon[32648]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-26T01:49:43.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:42 smithi165 ceph-mon[32648]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-26T01:49:43.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:42 smithi151 ceph-mon[28057]: pgmap v327: 97 pgs: 63 active+clean, 28 creating+peering, 6 unknown; 577 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:49:43.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:42 smithi151 ceph-mon[28057]: osdmap e90: 12 total, 12 up, 12 in 2024-04-26T01:49:43.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:42 smithi151 ceph-mon[28057]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-26T01:49:43.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:42 smithi151 ceph-mon[28057]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-26T01:49:43.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:42 smithi151 ceph-mon[28057]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-26T01:49:43.193 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:49:44.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:43 smithi172 ceph-mon[32785]: mds.? [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] up:active 2024-04-26T01:49:44.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:43 smithi172 ceph-mon[32785]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T01:49:44.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:43 smithi172 ceph-mon[32785]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T01:49:44.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:43 smithi165 ceph-mon[32648]: mds.? [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] up:active 2024-04-26T01:49:44.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:43 smithi165 ceph-mon[32648]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T01:49:44.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:43 smithi165 ceph-mon[32648]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T01:49:44.096 INFO:journalctl@ceph.mds.b.smithi165.stdout:Apr 26 01:49:43 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-b[63325]: 2024-04-26T01:49:43.603+0000 7f39c1229700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T01:49:44.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:43 smithi151 ceph-mon[28057]: mds.? [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] up:active 2024-04-26T01:49:44.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:43 smithi151 ceph-mon[28057]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T01:49:44.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:43 smithi151 ceph-mon[28057]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T01:49:44.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:44 smithi151 ceph-mon[28057]: pgmap v329: 97 pgs: 97 active+clean; 577 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-26T01:49:44.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:44 smithi151 ceph-mon[28057]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-26T01:49:44.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:44 smithi151 ceph-mon[28057]: Cluster is now healthy 2024-04-26T01:49:45.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:44 smithi172 ceph-mon[32785]: pgmap v329: 97 pgs: 97 active+clean; 577 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-26T01:49:45.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:44 smithi172 ceph-mon[32785]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-26T01:49:45.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:44 smithi172 ceph-mon[32785]: Cluster is now healthy 2024-04-26T01:49:45.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:44 smithi165 ceph-mon[32648]: pgmap v329: 97 pgs: 97 active+clean; 577 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-26T01:49:45.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:44 smithi165 ceph-mon[32648]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-26T01:49:45.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:44 smithi165 ceph-mon[32648]: Cluster is now healthy 2024-04-26T01:49:45.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:45.376+0000 7fa09d800700 1 -- 172.21.15.151:0/557485783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa098105870 msgr2=0x7fa0981098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:45.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:45.376+0000 7fa09d800700 1 --2- 172.21.15.151:0/557485783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa098105870 0x7fa0981098c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa0800099c0 tx=0x7fa080009cd0 comp rx=0 tx=0).stop 2024-04-26T01:49:45.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:45.376+0000 7fa09d800700 1 -- 172.21.15.151:0/557485783 shutdown_connections 2024-04-26T01:49:45.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:45.376+0000 7fa09d800700 1 --2- 172.21.15.151:0/557485783 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa098109fa0 0x7fa098111b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:45.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:45.376+0000 7fa09d800700 1 --2- 172.21.15.151:0/557485783 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa098104ec0 0x7fa0981052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:45.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:45.376+0000 7fa09d800700 1 --2- 172.21.15.151:0/557485783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa098105870 0x7fa0981098c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:45.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:45.376+0000 7fa09d800700 1 -- 172.21.15.151:0/557485783 >> 172.21.15.151:0/557485783 conn(0x7fa098100750 msgr2=0x7fa098102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:45.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:45.376+0000 7fa09d800700 1 -- 172.21.15.151:0/557485783 shutdown_connections 2024-04-26T01:49:45.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:45.376+0000 7fa09d800700 1 -- 172.21.15.151:0/557485783 wait complete. 2024-04-26T01:49:45.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:45.377+0000 7fa09d800700 1 Processor -- start 2024-04-26T01:49:45.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:45.377+0000 7fa09d800700 1 -- start start 2024-04-26T01:49:45.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:45.378+0000 7fa09d800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa098104ec0 0x7fa09819b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:45.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:45.378+0000 7fa09d800700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa098105870 0x7fa09819b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:45.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:45.378+0000 7fa09d800700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa098109fa0 0x7fa09819fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:45.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:45.378+0000 7fa09d800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa098114e80 con 0x7fa098104ec0 2024-04-26T01:49:45.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:45.378+0000 7fa09d800700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa098114d00 con 0x7fa098109fa0 2024-04-26T01:49:45.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:45.378+0000 7fa09d800700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa098115000 con 0x7fa098105870 2024-04-26T01:49:45.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:45.378+0000 7fa096a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa098104ec0 0x7fa09819b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:45.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:45.378+0000 7fa08e000700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa098105870 0x7fa09819b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:45.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:45.378+0000 7fa096a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa098104ec0 0x7fa09819b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36074/0 (socket says 172.21.15.151:36074) 2024-04-26T01:49:45.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:45.378+0000 7fa096a00700 1 -- 172.21.15.151:0/2419810692 learned_addr learned my addr 172.21.15.151:0/2419810692 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:49:45.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:45.378+0000 7fa097400700 1 --2- 172.21.15.151:0/2419810692 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa098109fa0 0x7fa09819fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:45.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:45.378+0000 7fa08e000700 1 -- 172.21.15.151:0/2419810692 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa098109fa0 msgr2=0x7fa09819fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:45.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:45.378+0000 7fa08e000700 1 --2- 172.21.15.151:0/2419810692 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa098109fa0 0x7fa09819fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:45.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:45.378+0000 7fa08e000700 1 -- 172.21.15.151:0/2419810692 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa098104ec0 msgr2=0x7fa09819b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:45.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:45.378+0000 7fa08e000700 1 --2- 172.21.15.151:0/2419810692 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa098104ec0 0x7fa09819b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:45.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:45.378+0000 7fa08e000700 1 -- 172.21.15.151:0/2419810692 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0981a03b0 con 0x7fa098105870 2024-04-26T01:49:45.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:45.378+0000 7fa097400700 1 --2- 172.21.15.151:0/2419810692 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa098109fa0 0x7fa09819fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:49:45.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:45.379+0000 7fa08e000700 1 --2- 172.21.15.151:0/2419810692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa098105870 0x7fa09819b930 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa0800098e0 tx=0x7fa080005280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:45.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:45.379+0000 7fa08fe00700 1 -- 172.21.15.151:0/2419810692 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa080026070 con 0x7fa098105870 2024-04-26T01:49:45.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:45.379+0000 7fa08fe00700 1 -- 172.21.15.151:0/2419810692 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa080003c70 con 0x7fa098105870 2024-04-26T01:49:45.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:45.379+0000 7fa09d800700 1 -- 172.21.15.151:0/2419810692 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0981a0640 con 0x7fa098105870 2024-04-26T01:49:45.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:45.379+0000 7fa08fe00700 1 -- 172.21.15.151:0/2419810692 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa08000f9f0 con 0x7fa098105870 2024-04-26T01:49:45.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:45.379+0000 7fa09d800700 1 -- 172.21.15.151:0/2419810692 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0981a7f20 con 0x7fa098105870 2024-04-26T01:49:45.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:45.380+0000 7fa09d800700 1 -- 172.21.15.151:0/2419810692 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0981063a0 con 0x7fa098105870 2024-04-26T01:49:45.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:45.382+0000 7fa08fe00700 1 -- 172.21.15.151:0/2419810692 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fa080029070 con 0x7fa098105870 2024-04-26T01:49:45.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:45.384+0000 7fa08fe00700 1 --2- 172.21.15.151:0/2419810692 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fa0780a3c40 0x7fa0780a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:45.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:45.384+0000 7fa08fe00700 1 -- 172.21.15.151:0/2419810692 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7979+0+0 (secure 0 0 0) 0x7fa0800d12b0 con 0x7fa098105870 2024-04-26T01:49:45.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:45.384+0000 7fa096a00700 1 --2- 172.21.15.151:0/2419810692 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fa0780a3c40 0x7fa0780a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:45.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:45.385+0000 7fa096a00700 1 --2- 172.21.15.151:0/2419810692 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fa0780a3c40 0x7fa0780a6100 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa088009d20 tx=0x7fa088009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:45.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:45.386+0000 7fa08fe00700 1 -- 172.21.15.151:0/2419810692 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa080030020 con 0x7fa098105870 2024-04-26T01:49:45.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:45.611+0000 7fa09d800700 1 -- 172.21.15.151:0/2419810692 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa09819c700 con 0x7fa098105870 2024-04-26T01:49:45.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:45.613+0000 7fa08fe00700 1 -- 172.21.15.151:0/2419810692 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22259 (secure 0 0 0) 0x7fa080013070 con 0x7fa098105870 2024-04-26T01:49:45.611 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:49:45.612 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":90,"fsid":"c0e49e64-036d-11ef-bc93-c7b262605968","created":"2024-04-26T01:40:07.483197+0000","modified":"2024-04-26T01:49:40.746918+0000","last_up_change":"2024-04-26T01:48:21.167691+0000","last_in_change":"2024-04-26T01:48:00.556990+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T01:44:11.411722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T01:49:31.756032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T01:49:36.529819+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ca27e814-a982-482c-ae1a-e596aac5c59a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":669937998},{"type":"v1","addr":"172.21.15.151:6803","nonce":669937998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":669937998},{"type":"v1","addr":"172.21.15.151:6805","nonce":669937998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":669937998},{"type":"v1","addr":"172.21.15.151:6809","nonce":669937998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":669937998},{"type":"v1","addr":"172.21.15.151:6807","nonce":669937998}]},"public_addr":"172.21.15.151:6803/669937998","cluster_addr":"172.21.15.151:6805/669937998","heartbeat_back_addr":"172.21.15.151:6809/669937998","heartbeat_front_addr":"172.21.15.151:6807/669937998","state":["exists","up"]},{"osd":1,"uuid":"e7d192c3-9624-4625-9ccb-a76c114d8aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":1356168171},{"type":"v1","addr":"172.21.15.165:6801","nonce":1356168171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":1356168171},{"type":"v1","addr":"172.21.15.165:6803","nonce":1356168171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":1356168171},{"type":"v1","addr":"172.21.15.165:6807","nonce":1356168171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":1356168171},{"type":"v1","addr":"172.21.15.165:6805","nonce":1356168171}]},"public_addr":"172.21.15.165:6801/1356168171","cluster_addr":"172.21.15.165:6803/1356168171","heartbeat_back_addr":"172.21.15.165:6807/1356168171","heartbeat_front_addr":"172.21.15.165:6805/1356168171","state":["exists","up"]},{"osd":2,"uuid":"0631df59-1ecc-48b4-b00a-48b0118dfc24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":237502635},{"type":"v1","addr":"172.21.15.172:6801","nonce":237502635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":237502635},{"type":"v1","addr":"172.21.15.172:6803","nonce":237502635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":237502635},{"type":"v1","addr":"172.21.15.172:6807","nonce":237502635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":237502635},{"type":"v1","addr":"172.21.15.172:6805","nonce":237502635}]},"public_addr":"172.21.15.172:6801/237502635","cluster_addr":"172.21.15.172:6803/237502635","heartbeat_back_addr":"172.21.15.172:6807/237502635","heartbeat_front_addr":"172.21.15.172:6805/237502635","state":["exists","up"]},{"osd":3,"uuid":"73076e49-2f8a-4a3d-99eb-7b50aff448aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3442279476},{"type":"v1","addr":"172.21.15.151:6811","nonce":3442279476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3442279476},{"type":"v1","addr":"172.21.15.151:6813","nonce":3442279476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":3442279476},{"type":"v1","addr":"172.21.15.151:6817","nonce":3442279476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3442279476},{"type":"v1","addr":"172.21.15.151:6815","nonce":3442279476}]},"public_addr":"172.21.15.151:6811/3442279476","cluster_addr":"172.21.15.151:6813/3442279476","heartbeat_back_addr":"172.21.15.151:6817/3442279476","heartbeat_front_addr":"172.21.15.151:6815/3442279476","state":["exists","up"]},{"osd":4,"uuid":"0230d615-548c-43f0-b8aa-7130f4cbbc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4029980723},{"type":"v1","addr":"172.21.15.165:6809","nonce":4029980723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4029980723},{"type":"v1","addr":"172.21.15.165:6811","nonce":4029980723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4029980723},{"type":"v1","addr":"172.21.15.165:6815","nonce":4029980723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4029980723},{"type":"v1","addr":"172.21.15.165:6813","nonce":4029980723}]},"public_addr":"172.21.15.165:6809/4029980723","cluster_addr":"172.21.15.165:6811/4029980723","heartbeat_back_addr":"172.21.15.165:6815/4029980723","heartbeat_front_addr":"172.21.15.165:6813/4029980723","state":["exists","up"]},{"osd":5,"uuid":"9739f44b-2605-47dd-82fb-d30034742bf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2216578687},{"type":"v1","addr":"172.21.15.172:6809","nonce":2216578687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2216578687},{"type":"v1","addr":"172.21.15.172:6811","nonce":2216578687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2216578687},{"type":"v1","addr":"172.21.15.172:6815","nonce":2216578687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2216578687},{"type":"v1","addr":"172.21.15.172:6813","nonce":2216578687}]},"public_addr":"172.21.15.172:6809/2216578687","cluster_addr":"172.21.15.172:6811/2216578687","heartbeat_back_addr":"172.21.15.172:6815/2216578687","heartbeat_front_addr":"172.21.15.172:6813/2216578687","state":["exists","up"]},{"osd":6,"uuid":"4290baaa-6160-49cb-a4e0-c96f070f2ef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1723252315},{"type":"v1","addr":"172.21.15.151:6819","nonce":1723252315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1723252315},{"type":"v1","addr":"172.21.15.151:6821","nonce":1723252315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":1723252315},{"type":"v1","addr":"172.21.15.151:6825","nonce":1723252315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1723252315},{"type":"v1","addr":"172.21.15.151:6823","nonce":1723252315}]},"public_addr":"172.21.15.151:6819/1723252315","cluster_addr":"172.21.15.151:6821/1723252315","heartbeat_back_addr":"172.21.15.151:6825/1723252315","heartbeat_front_addr":"172.21.15.151:6823/1723252315","state":["exists","up"]},{"osd":7,"uuid":"9f5315f2-036c-4e2f-92ed-3321a9e3c860","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":1977803781},{"type":"v1","addr":"172.21.15.165:6817","nonce":1977803781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":1977803781},{"type":"v1","addr":"172.21.15.165:6819","nonce":1977803781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":1977803781},{"type":"v1","addr":"172.21.15.165:6823","nonce":1977803781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":1977803781},{"type":"v1","addr":"172.21.15.165:6821","nonce":1977803781}]},"public_addr":"172.21.15.165:6817/1977803781","cluster_addr":"172.21.15.165:6819/1977803781","heartbeat_back_addr":"172.21.15.165:6823/1977803781","heartbeat_front_addr":"172.21.15.165:6821/1977803781","state":["exists","up"]},{"osd":8,"uuid":"8ddbe36d-cea7-45ef-a3ca-044a0e64ff85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":491199512},{"type":"v1","addr":"172.21.15.172:6817","nonce":491199512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":491199512},{"type":"v1","addr":"172.21.15.172:6819","nonce":491199512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":491199512},{"type":"v1","addr":"172.21.15.172:6823","nonce":491199512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":491199512},{"type":"v1","addr":"172.21.15.172:6821","nonce":491199512}]},"public_addr":"172.21.15.172:6817/491199512","cluster_addr":"172.21.15.172:6819/491199512","heartbeat_back_addr":"172.21.15.172:6823/491199512","heartbeat_front_addr":"172.21.15.172:6821/491199512","state":["exists","up"]},{"osd":9,"uuid":"40654ba0-1bd9-4b54-97ae-f1ed6c4785d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":1818923828},{"type":"v1","addr":"172.21.15.151:6827","nonce":1818923828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":1818923828},{"type":"v1","addr":"172.21.15.151:6829","nonce":1818923828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1818923828},{"type":"v1","addr":"172.21.15.151:6833","nonce":1818923828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":1818923828},{"type":"v1","addr":"172.21.15.151:6831","nonce":1818923828}]},"public_addr":"172.21.15.151:6827/1818923828","cluster_addr":"172.21.15.151:6829/1818923828","heartbeat_back_addr":"172.21.15.151:6833/1818923828","heartbeat_front_addr":"172.21.15.151:6831/1818923828","state":["exists","up"]},{"osd":10,"uuid":"19e75a1a-7bd0-41d9-b0cc-0fa4d1475dda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":2632621809},{"type":"v1","addr":"172.21.15.165:6825","nonce":2632621809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":2632621809},{"type":"v1","addr":"172.21.15.165:6827","nonce":2632621809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":2632621809},{"type":"v1","addr":"172.21.15.165:6831","nonce":2632621809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":2632621809},{"type":"v1","addr":"172.21.15.165:6829","nonce":2632621809}]},"public_addr":"172.21.15.165:6825/2632621809","cluster_addr":"172.21.15.165:6827/2632621809","heartbeat_back_addr":"172.21.15.165:6831/2632621809","heartbeat_front_addr":"172.21.15.165:6829/2632621809","state":["exists","up"]},{"osd":11,"uuid":"aa84e5f5-93a3-4375-8544-a1175f0eb06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":886604365},{"type":"v1","addr":"172.21.15.172:6825","nonce":886604365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":886604365},{"type":"v1","addr":"172.21.15.172:6827","nonce":886604365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":886604365},{"type":"v1","addr":"172.21.15.172:6831","nonce":886604365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":886604365},{"type":"v1","addr":"172.21.15.172:6829","nonce":886604365}]},"public_addr":"172.21.15.172:6825/886604365","cluster_addr":"172.21.15.172:6827/886604365","heartbeat_back_addr":"172.21.15.172:6831/886604365","heartbeat_front_addr":"172.21.15.172:6829/886604365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:43:18.296278+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:43:42.860620+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:44:07.073601+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:44:39.631760+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:45:04.771214+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:45:29.606674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:46:03.101851+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:46:28.351995+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:46:53.445435+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:47:27.391179+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:47:53.543638+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:48:18.687811+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.151:6801/796314699":"2024-04-27T01:41:25.234322+0000","172.21.15.151:6800/796314699":"2024-04-27T01:41:25.234322+0000","172.21.15.151:0/1173525554":"2024-04-27T01:41:25.234322+0000","172.21.15.151:0/1342496801":"2024-04-27T01:40:50.492554+0000","172.21.15.151:0/11665926":"2024-04-27T01:40:50.492554+0000","172.21.15.151:0/1765448952":"2024-04-27T01:40:50.492554+0000","172.21.15.151:0/46007430":"2024-04-27T01:41:25.234322+0000","172.21.15.151:6800/244725060":"2024-04-27T01:40:50.492554+0000","172.21.15.151:0/3703862543":"2024-04-27T01:41:25.234322+0000","172.21.15.151:6801/244725060":"2024-04-27T01:40:50.492554+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T01:49:45.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:45.616+0000 7fa09d800700 1 -- 172.21.15.151:0/2419810692 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fa0780a3c40 msgr2=0x7fa0780a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:45.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:45.616+0000 7fa09d800700 1 --2- 172.21.15.151:0/2419810692 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fa0780a3c40 0x7fa0780a6100 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa088009d20 tx=0x7fa088009260 comp rx=0 tx=0).stop 2024-04-26T01:49:45.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:45.616+0000 7fa09d800700 1 -- 172.21.15.151:0/2419810692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa098105870 msgr2=0x7fa09819b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:45.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:45.616+0000 7fa09d800700 1 --2- 172.21.15.151:0/2419810692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa098105870 0x7fa09819b930 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa0800098e0 tx=0x7fa080005280 comp rx=0 tx=0).stop 2024-04-26T01:49:45.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:45.616+0000 7fa09d800700 1 -- 172.21.15.151:0/2419810692 shutdown_connections 2024-04-26T01:49:45.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:45.616+0000 7fa09d800700 1 --2- 172.21.15.151:0/2419810692 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa098109fa0 0x7fa09819fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:45.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:45.616+0000 7fa09d800700 1 --2- 172.21.15.151:0/2419810692 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fa0780a3c40 0x7fa0780a6100 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:45.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:45.616+0000 7fa09d800700 1 --2- 172.21.15.151:0/2419810692 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa098104ec0 0x7fa09819b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:45.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:45.616+0000 7fa09d800700 1 --2- 172.21.15.151:0/2419810692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa098105870 0x7fa09819b930 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:45.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:45.616+0000 7fa09d800700 1 -- 172.21.15.151:0/2419810692 >> 172.21.15.151:0/2419810692 conn(0x7fa098100750 msgr2=0x7fa098102a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:45.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:45.617+0000 7fa09d800700 1 -- 172.21.15.151:0/2419810692 shutdown_connections 2024-04-26T01:49:45.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:45.617+0000 7fa09d800700 1 -- 172.21.15.151:0/2419810692 wait complete. 2024-04-26T01:49:45.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:45 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/2419810692' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:49:46.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:45 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/2419810692' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:49:46.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:45 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/2419810692' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:49:46.481 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T01:49:46.928 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:49:47.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:46 smithi172 ceph-mon[32785]: pgmap v330: 97 pgs: 97 active+clean; 577 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 395 B/s wr, 3 op/s 2024-04-26T01:49:47.081 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:46 smithi151 ceph-mon[28057]: pgmap v330: 97 pgs: 97 active+clean; 577 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 395 B/s wr, 3 op/s 2024-04-26T01:49:47.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:46 smithi165 ceph-mon[32648]: pgmap v330: 97 pgs: 97 active+clean; 577 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 395 B/s wr, 3 op/s 2024-04-26T01:49:48.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:48 smithi151 ceph-mon[28057]: pgmap v331: 97 pgs: 97 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-04-26T01:49:49.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:48 smithi172 ceph-mon[32785]: pgmap v331: 97 pgs: 97 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-04-26T01:49:49.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:49.080+0000 7f6318800700 1 -- 172.21.15.151:0/3501788585 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6310109c80 msgr2=0x7f631010dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:49.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:49.080+0000 7f6318800700 1 --2- 172.21.15.151:0/3501788585 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6310109c80 0x7f631010dcd0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f63000098f0 tx=0x7f6300009c00 comp rx=0 tx=0).stop 2024-04-26T01:49:49.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:49.080+0000 7f6318800700 1 -- 172.21.15.151:0/3501788585 shutdown_connections 2024-04-26T01:49:49.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:49.080+0000 7f6318800700 1 --2- 172.21.15.151:0/3501788585 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f631010e3b0 0x7f6310115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:49.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:49.080+0000 7f6318800700 1 --2- 172.21.15.151:0/3501788585 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63101092d0 0x7f63101096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:49.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:49.080+0000 7f6318800700 1 --2- 172.21.15.151:0/3501788585 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6310109c80 0x7f631010dcd0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:49.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:49.080+0000 7f6318800700 1 -- 172.21.15.151:0/3501788585 >> 172.21.15.151:0/3501788585 conn(0x7f6310076210 msgr2=0x7f6310107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:49.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:49.080+0000 7f6318800700 1 -- 172.21.15.151:0/3501788585 shutdown_connections 2024-04-26T01:49:49.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:49.080+0000 7f6318800700 1 -- 172.21.15.151:0/3501788585 wait complete. 2024-04-26T01:49:49.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:49.081+0000 7f6318800700 1 Processor -- start 2024-04-26T01:49:49.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:49.081+0000 7f6318800700 1 -- start start 2024-04-26T01:49:49.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:49.082+0000 7f6318800700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f63101092d0 0x7f63101a5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:49.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:49.082+0000 7f6318800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6310109c80 0x7f63101a5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:49.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:49.082+0000 7f6318800700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f631010e3b0 0x7f631019f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:49.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:49.082+0000 7f6318800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6310119290 con 0x7f6310109c80 2024-04-26T01:49:49.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:49.082+0000 7f6318800700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6310119110 con 0x7f63101092d0 2024-04-26T01:49:49.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:49.082+0000 7f6318800700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6310119410 con 0x7f631010e3b0 2024-04-26T01:49:49.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:49.082+0000 7f6315e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f63101092d0 0x7f63101a5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:49.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:49.082+0000 7f6315e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f63101092d0 0x7f63101a5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:60388/0 (socket says 172.21.15.151:60388) 2024-04-26T01:49:49.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:49.082+0000 7f6315e00700 1 -- 172.21.15.151:0/1013099642 learned_addr learned my addr 172.21.15.151:0/1013099642 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:49:49.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:49.082+0000 7f6315400700 1 --2- 172.21.15.151:0/1013099642 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6310109c80 0x7f63101a5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:49.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:49.082+0000 7f6316800700 1 --2- 172.21.15.151:0/1013099642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f631010e3b0 0x7f631019f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:49.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:49.083+0000 7f6315400700 1 -- 172.21.15.151:0/1013099642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f631010e3b0 msgr2=0x7f631019f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:49.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:49.083+0000 7f6315400700 1 --2- 172.21.15.151:0/1013099642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f631010e3b0 0x7f631019f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:49.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:49.083+0000 7f6315400700 1 -- 172.21.15.151:0/1013099642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f63101092d0 msgr2=0x7f63101a5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:49.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:49.083+0000 7f6315400700 1 --2- 172.21.15.151:0/1013099642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f63101092d0 0x7f63101a5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:49.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:49.083+0000 7f6315400700 1 -- 172.21.15.151:0/1013099642 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f631019fd90 con 0x7f6310109c80 2024-04-26T01:49:49.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:49.083+0000 7f6316800700 1 --2- 172.21.15.151:0/1013099642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f631010e3b0 0x7f631019f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:49:49.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:49.083+0000 7f6315e00700 1 --2- 172.21.15.151:0/1013099642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f63101092d0 0x7f63101a5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:49:49.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:49.083+0000 7f6315400700 1 --2- 172.21.15.151:0/1013099642 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6310109c80 0x7f63101a5960 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6300000c00 tx=0x7f630000fa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:49.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:49.083+0000 7f630aa00700 1 -- 172.21.15.151:0/1013099642 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63000050c0 con 0x7f6310109c80 2024-04-26T01:49:49.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:49.083+0000 7f630aa00700 1 -- 172.21.15.151:0/1013099642 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f630000b560 con 0x7f6310109c80 2024-04-26T01:49:49.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:49.083+0000 7f6318800700 1 -- 172.21.15.151:0/1013099642 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63101a0020 con 0x7f6310109c80 2024-04-26T01:49:49.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:49.083+0000 7f630aa00700 1 -- 172.21.15.151:0/1013099642 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63000039b0 con 0x7f6310109c80 2024-04-26T01:49:49.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:49.084+0000 7f6318800700 1 -- 172.21.15.151:0/1013099642 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63101ac2f0 con 0x7f6310109c80 2024-04-26T01:49:49.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:49.085+0000 7f6318800700 1 -- 172.21.15.151:0/1013099642 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f631010a7b0 con 0x7f6310109c80 2024-04-26T01:49:49.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:49.087+0000 7f630aa00700 1 -- 172.21.15.151:0/1013099642 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f630000b9e0 con 0x7f6310109c80 2024-04-26T01:49:49.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:49.088+0000 7f630aa00700 1 --2- 172.21.15.151:0/1013099642 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f62f80a3cc0 0x7f62f80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:49.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:49.089+0000 7f630aa00700 1 -- 172.21.15.151:0/1013099642 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7979+0+0 (secure 0 0 0) 0x7f63000d1390 con 0x7f6310109c80 2024-04-26T01:49:49.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:49.089+0000 7f6315e00700 1 --2- 172.21.15.151:0/1013099642 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f62f80a3cc0 0x7f62f80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:49.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:49.090+0000 7f6315e00700 1 --2- 172.21.15.151:0/1013099642 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f62f80a3cc0 0x7f62f80a6180 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6304005950 tx=0x7f6304009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:49.090 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:49.092+0000 7f630aa00700 1 -- 172.21.15.151:0/1013099642 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6300013030 con 0x7f6310109c80 2024-04-26T01:49:49.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:48 smithi165 ceph-mon[32648]: pgmap v331: 97 pgs: 97 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-04-26T01:49:49.317 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:49.319+0000 7f6318800700 1 -- 172.21.15.151:0/1013099642 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f631002df80 con 0x7f6310109c80 2024-04-26T01:49:49.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:49.321+0000 7f630aa00700 1 -- 172.21.15.151:0/1013099642 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22259 (secure 0 0 0) 0x7f630000e3e0 con 0x7f6310109c80 2024-04-26T01:49:49.320 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:49:49.320 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":90,"fsid":"c0e49e64-036d-11ef-bc93-c7b262605968","created":"2024-04-26T01:40:07.483197+0000","modified":"2024-04-26T01:49:40.746918+0000","last_up_change":"2024-04-26T01:48:21.167691+0000","last_in_change":"2024-04-26T01:48:00.556990+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T01:44:11.411722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T01:49:31.756032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T01:49:36.529819+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ca27e814-a982-482c-ae1a-e596aac5c59a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":669937998},{"type":"v1","addr":"172.21.15.151:6803","nonce":669937998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":669937998},{"type":"v1","addr":"172.21.15.151:6805","nonce":669937998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":669937998},{"type":"v1","addr":"172.21.15.151:6809","nonce":669937998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":669937998},{"type":"v1","addr":"172.21.15.151:6807","nonce":669937998}]},"public_addr":"172.21.15.151:6803/669937998","cluster_addr":"172.21.15.151:6805/669937998","heartbeat_back_addr":"172.21.15.151:6809/669937998","heartbeat_front_addr":"172.21.15.151:6807/669937998","state":["exists","up"]},{"osd":1,"uuid":"e7d192c3-9624-4625-9ccb-a76c114d8aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":1356168171},{"type":"v1","addr":"172.21.15.165:6801","nonce":1356168171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":1356168171},{"type":"v1","addr":"172.21.15.165:6803","nonce":1356168171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":1356168171},{"type":"v1","addr":"172.21.15.165:6807","nonce":1356168171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":1356168171},{"type":"v1","addr":"172.21.15.165:6805","nonce":1356168171}]},"public_addr":"172.21.15.165:6801/1356168171","cluster_addr":"172.21.15.165:6803/1356168171","heartbeat_back_addr":"172.21.15.165:6807/1356168171","heartbeat_front_addr":"172.21.15.165:6805/1356168171","state":["exists","up"]},{"osd":2,"uuid":"0631df59-1ecc-48b4-b00a-48b0118dfc24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":237502635},{"type":"v1","addr":"172.21.15.172:6801","nonce":237502635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":237502635},{"type":"v1","addr":"172.21.15.172:6803","nonce":237502635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":237502635},{"type":"v1","addr":"172.21.15.172:6807","nonce":237502635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":237502635},{"type":"v1","addr":"172.21.15.172:6805","nonce":237502635}]},"public_addr":"172.21.15.172:6801/237502635","cluster_addr":"172.21.15.172:6803/237502635","heartbeat_back_addr":"172.21.15.172:6807/237502635","heartbeat_front_addr":"172.21.15.172:6805/237502635","state":["exists","up"]},{"osd":3,"uuid":"73076e49-2f8a-4a3d-99eb-7b50aff448aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3442279476},{"type":"v1","addr":"172.21.15.151:6811","nonce":3442279476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3442279476},{"type":"v1","addr":"172.21.15.151:6813","nonce":3442279476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":3442279476},{"type":"v1","addr":"172.21.15.151:6817","nonce":3442279476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3442279476},{"type":"v1","addr":"172.21.15.151:6815","nonce":3442279476}]},"public_addr":"172.21.15.151:6811/3442279476","cluster_addr":"172.21.15.151:6813/3442279476","heartbeat_back_addr":"172.21.15.151:6817/3442279476","heartbeat_front_addr":"172.21.15.151:6815/3442279476","state":["exists","up"]},{"osd":4,"uuid":"0230d615-548c-43f0-b8aa-7130f4cbbc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4029980723},{"type":"v1","addr":"172.21.15.165:6809","nonce":4029980723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4029980723},{"type":"v1","addr":"172.21.15.165:6811","nonce":4029980723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4029980723},{"type":"v1","addr":"172.21.15.165:6815","nonce":4029980723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4029980723},{"type":"v1","addr":"172.21.15.165:6813","nonce":4029980723}]},"public_addr":"172.21.15.165:6809/4029980723","cluster_addr":"172.21.15.165:6811/4029980723","heartbeat_back_addr":"172.21.15.165:6815/4029980723","heartbeat_front_addr":"172.21.15.165:6813/4029980723","state":["exists","up"]},{"osd":5,"uuid":"9739f44b-2605-47dd-82fb-d30034742bf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2216578687},{"type":"v1","addr":"172.21.15.172:6809","nonce":2216578687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2216578687},{"type":"v1","addr":"172.21.15.172:6811","nonce":2216578687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2216578687},{"type":"v1","addr":"172.21.15.172:6815","nonce":2216578687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2216578687},{"type":"v1","addr":"172.21.15.172:6813","nonce":2216578687}]},"public_addr":"172.21.15.172:6809/2216578687","cluster_addr":"172.21.15.172:6811/2216578687","heartbeat_back_addr":"172.21.15.172:6815/2216578687","heartbeat_front_addr":"172.21.15.172:6813/2216578687","state":["exists","up"]},{"osd":6,"uuid":"4290baaa-6160-49cb-a4e0-c96f070f2ef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1723252315},{"type":"v1","addr":"172.21.15.151:6819","nonce":1723252315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1723252315},{"type":"v1","addr":"172.21.15.151:6821","nonce":1723252315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":1723252315},{"type":"v1","addr":"172.21.15.151:6825","nonce":1723252315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1723252315},{"type":"v1","addr":"172.21.15.151:6823","nonce":1723252315}]},"public_addr":"172.21.15.151:6819/1723252315","cluster_addr":"172.21.15.151:6821/1723252315","heartbeat_back_addr":"172.21.15.151:6825/1723252315","heartbeat_front_addr":"172.21.15.151:6823/1723252315","state":["exists","up"]},{"osd":7,"uuid":"9f5315f2-036c-4e2f-92ed-3321a9e3c860","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":1977803781},{"type":"v1","addr":"172.21.15.165:6817","nonce":1977803781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":1977803781},{"type":"v1","addr":"172.21.15.165:6819","nonce":1977803781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":1977803781},{"type":"v1","addr":"172.21.15.165:6823","nonce":1977803781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":1977803781},{"type":"v1","addr":"172.21.15.165:6821","nonce":1977803781}]},"public_addr":"172.21.15.165:6817/1977803781","cluster_addr":"172.21.15.165:6819/1977803781","heartbeat_back_addr":"172.21.15.165:6823/1977803781","heartbeat_front_addr":"172.21.15.165:6821/1977803781","state":["exists","up"]},{"osd":8,"uuid":"8ddbe36d-cea7-45ef-a3ca-044a0e64ff85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":491199512},{"type":"v1","addr":"172.21.15.172:6817","nonce":491199512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":491199512},{"type":"v1","addr":"172.21.15.172:6819","nonce":491199512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":491199512},{"type":"v1","addr":"172.21.15.172:6823","nonce":491199512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":491199512},{"type":"v1","addr":"172.21.15.172:6821","nonce":491199512}]},"public_addr":"172.21.15.172:6817/491199512","cluster_addr":"172.21.15.172:6819/491199512","heartbeat_back_addr":"172.21.15.172:6823/491199512","heartbeat_front_addr":"172.21.15.172:6821/491199512","state":["exists","up"]},{"osd":9,"uuid":"40654ba0-1bd9-4b54-97ae-f1ed6c4785d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":1818923828},{"type":"v1","addr":"172.21.15.151:6827","nonce":1818923828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":1818923828},{"type":"v1","addr":"172.21.15.151:6829","nonce":1818923828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1818923828},{"type":"v1","addr":"172.21.15.151:6833","nonce":1818923828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":1818923828},{"type":"v1","addr":"172.21.15.151:6831","nonce":1818923828}]},"public_addr":"172.21.15.151:6827/1818923828","cluster_addr":"172.21.15.151:6829/1818923828","heartbeat_back_addr":"172.21.15.151:6833/1818923828","heartbeat_front_addr":"172.21.15.151:6831/1818923828","state":["exists","up"]},{"osd":10,"uuid":"19e75a1a-7bd0-41d9-b0cc-0fa4d1475dda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":2632621809},{"type":"v1","addr":"172.21.15.165:6825","nonce":2632621809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":2632621809},{"type":"v1","addr":"172.21.15.165:6827","nonce":2632621809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":2632621809},{"type":"v1","addr":"172.21.15.165:6831","nonce":2632621809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":2632621809},{"type":"v1","addr":"172.21.15.165:6829","nonce":2632621809}]},"public_addr":"172.21.15.165:6825/2632621809","cluster_addr":"172.21.15.165:6827/2632621809","heartbeat_back_addr":"172.21.15.165:6831/2632621809","heartbeat_front_addr":"172.21.15.165:6829/2632621809","state":["exists","up"]},{"osd":11,"uuid":"aa84e5f5-93a3-4375-8544-a1175f0eb06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":886604365},{"type":"v1","addr":"172.21.15.172:6825","nonce":886604365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":886604365},{"type":"v1","addr":"172.21.15.172:6827","nonce":886604365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":886604365},{"type":"v1","addr":"172.21.15.172:6831","nonce":886604365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":886604365},{"type":"v1","addr":"172.21.15.172:6829","nonce":886604365}]},"public_addr":"172.21.15.172:6825/886604365","cluster_addr":"172.21.15.172:6827/886604365","heartbeat_back_addr":"172.21.15.172:6831/886604365","heartbeat_front_addr":"172.21.15.172:6829/886604365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:43:18.296278+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:43:42.860620+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:44:07.073601+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:44:39.631760+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:45:04.771214+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:45:29.606674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:46:03.101851+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:46:28.351995+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:46:53.445435+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:47:27.391179+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:47:53.543638+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:48:18.687811+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.151:6801/796314699":"2024-04-27T01:41:25.234322+0000","172.21.15.151:6800/796314699":"2024-04-27T01:41:25.234322+0000","172.21.15.151:0/1173525554":"2024-04-27T01:41:25.234322+0000","172.21.15.151:0/1342496801":"2024-04-27T01:40:50.492554+0000","172.21.15.151:0/11665926":"2024-04-27T01:40:50.492554+0000","172.21.15.151:0/1765448952":"2024-04-27T01:40:50.492554+0000","172.21.15.151:0/46007430":"2024-04-27T01:41:25.234322+0000","172.21.15.151:6800/244725060":"2024-04-27T01:40:50.492554+0000","172.21.15.151:0/3703862543":"2024-04-27T01:41:25.234322+0000","172.21.15.151:6801/244725060":"2024-04-27T01:40:50.492554+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T01:49:49.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:49.324+0000 7f6318800700 1 -- 172.21.15.151:0/1013099642 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f62f80a3cc0 msgr2=0x7f62f80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:49.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:49.324+0000 7f6318800700 1 --2- 172.21.15.151:0/1013099642 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f62f80a3cc0 0x7f62f80a6180 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6304005950 tx=0x7f6304009310 comp rx=0 tx=0).stop 2024-04-26T01:49:49.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:49.324+0000 7f6318800700 1 -- 172.21.15.151:0/1013099642 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6310109c80 msgr2=0x7f63101a5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:49.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:49.324+0000 7f6318800700 1 --2- 172.21.15.151:0/1013099642 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6310109c80 0x7f63101a5960 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6300000c00 tx=0x7f630000fa20 comp rx=0 tx=0).stop 2024-04-26T01:49:49.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:49.324+0000 7f6318800700 1 -- 172.21.15.151:0/1013099642 shutdown_connections 2024-04-26T01:49:49.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:49.324+0000 7f6318800700 1 --2- 172.21.15.151:0/1013099642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f631010e3b0 0x7f631019f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:49.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:49.324+0000 7f6318800700 1 --2- 172.21.15.151:0/1013099642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f63101092d0 0x7f63101a5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:49.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:49.324+0000 7f6318800700 1 --2- 172.21.15.151:0/1013099642 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f62f80a3cc0 0x7f62f80a6180 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:49.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:49.324+0000 7f6318800700 1 --2- 172.21.15.151:0/1013099642 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6310109c80 0x7f63101a5960 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:49.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:49.324+0000 7f6318800700 1 -- 172.21.15.151:0/1013099642 >> 172.21.15.151:0/1013099642 conn(0x7f6310076210 msgr2=0x7f63101086f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:49.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:49.324+0000 7f6318800700 1 -- 172.21.15.151:0/1013099642 shutdown_connections 2024-04-26T01:49:49.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:49.324+0000 7f6318800700 1 -- 172.21.15.151:0/1013099642 wait complete. 2024-04-26T01:49:50.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:49 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/1013099642' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:49:50.072 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:49 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/1013099642' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:49:50.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:49 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/1013099642' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:49:50.133 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-04-26T01:49:50.581 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:49:50.977 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:50 smithi151 ceph-mon[28057]: pgmap v332: 97 pgs: 97 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-26T01:49:51.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:50 smithi172 ceph-mon[32785]: pgmap v332: 97 pgs: 97 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-26T01:49:51.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:50 smithi165 ceph-mon[32648]: pgmap v332: 97 pgs: 97 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-26T01:49:53.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:53.005+0000 7fc972000700 1 -- 172.21.15.151:0/112643849 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc96c101960 msgr2=0x7fc96c10c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:53.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:53.005+0000 7fc972000700 1 --2- 172.21.15.151:0/112643849 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc96c101960 0x7fc96c10c9f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc958009990 tx=0x7fc958009ca0 comp rx=0 tx=0).stop 2024-04-26T01:49:53.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:53.005+0000 7fc972000700 1 -- 172.21.15.151:0/112643849 shutdown_connections 2024-04-26T01:49:53.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:53.005+0000 7fc972000700 1 --2- 172.21.15.151:0/112643849 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc96c10cfc0 0x7fc96c10f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:53.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:53.005+0000 7fc972000700 1 --2- 172.21.15.151:0/112643849 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc96c101040 0x7fc96c101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:53.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:53.005+0000 7fc972000700 1 --2- 172.21.15.151:0/112643849 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc96c101960 0x7fc96c10c9f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:53.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:53.005+0000 7fc972000700 1 -- 172.21.15.151:0/112643849 >> 172.21.15.151:0/112643849 conn(0x7fc96c0766e0 msgr2=0x7fc96c076af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:53.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:53.005+0000 7fc972000700 1 -- 172.21.15.151:0/112643849 shutdown_connections 2024-04-26T01:49:53.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:53.005+0000 7fc972000700 1 -- 172.21.15.151:0/112643849 wait complete. 2024-04-26T01:49:53.004 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:52 smithi151 ceph-mon[28057]: pgmap v333: 97 pgs: 97 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-26T01:49:53.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:53.006+0000 7fc972000700 1 Processor -- start 2024-04-26T01:49:53.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:53.007+0000 7fc972000700 1 -- start start 2024-04-26T01:49:53.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:53.007+0000 7fc972000700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc96c101040 0x7fc96c102850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:53.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:53.007+0000 7fc972000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc96c101960 0x7fc96c102d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:53.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:53.007+0000 7fc972000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc96c10cfc0 0x7fc96c1032d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:53.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:53.007+0000 7fc972000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc96c112bc0 con 0x7fc96c101960 2024-04-26T01:49:53.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:53.007+0000 7fc972000700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc96c112a40 con 0x7fc96c10cfc0 2024-04-26T01:49:53.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:53.007+0000 7fc972000700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc96c112d40 con 0x7fc96c101040 2024-04-26T01:49:53.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:53.008+0000 7fc96be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc96c101960 0x7fc96c102d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:53.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:53.008+0000 7fc96be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc96c101960 0x7fc96c102d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:51822/0 (socket says 172.21.15.151:51822) 2024-04-26T01:49:53.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:53.008+0000 7fc96be00700 1 -- 172.21.15.151:0/608385851 learned_addr learned my addr 172.21.15.151:0/608385851 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:49:53.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:53.008+0000 7fc970c00700 1 --2- 172.21.15.151:0/608385851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc96c101040 0x7fc96c102850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:53.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:53.008+0000 7fc971600700 1 --2- 172.21.15.151:0/608385851 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc96c10cfc0 0x7fc96c1032d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:53.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:53.008+0000 7fc96be00700 1 -- 172.21.15.151:0/608385851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc96c101040 msgr2=0x7fc96c102850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:53.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:53.008+0000 7fc96be00700 1 --2- 172.21.15.151:0/608385851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc96c101040 0x7fc96c102850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:53.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:53.008+0000 7fc96be00700 1 -- 172.21.15.151:0/608385851 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc96c10cfc0 msgr2=0x7fc96c1032d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:53.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:53.008+0000 7fc96be00700 1 --2- 172.21.15.151:0/608385851 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc96c10cfc0 0x7fc96c1032d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:53.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:53.008+0000 7fc96be00700 1 -- 172.21.15.151:0/608385851 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc96c1a7590 con 0x7fc96c101960 2024-04-26T01:49:53.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:53.008+0000 7fc970c00700 1 --2- 172.21.15.151:0/608385851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc96c101040 0x7fc96c102850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:49:53.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:53.009+0000 7fc96be00700 1 --2- 172.21.15.151:0/608385851 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc96c101960 0x7fc96c102d90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc958000c00 tx=0x7fc9580057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:53.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:53.009+0000 7fc969600700 1 -- 172.21.15.151:0/608385851 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc958027070 con 0x7fc96c101960 2024-04-26T01:49:53.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:53.009+0000 7fc969600700 1 -- 172.21.15.151:0/608385851 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc958022470 con 0x7fc96c101960 2024-04-26T01:49:53.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:53.009+0000 7fc972000700 1 -- 172.21.15.151:0/608385851 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc96c1a7820 con 0x7fc96c101960 2024-04-26T01:49:53.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:53.009+0000 7fc969600700 1 -- 172.21.15.151:0/608385851 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc958017650 con 0x7fc96c101960 2024-04-26T01:49:53.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:53.009+0000 7fc972000700 1 -- 172.21.15.151:0/608385851 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc96c1a7c70 con 0x7fc96c101960 2024-04-26T01:49:53.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:53.010+0000 7fc972000700 1 -- 172.21.15.151:0/608385851 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc96c069730 con 0x7fc96c101960 2024-04-26T01:49:53.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:53.013+0000 7fc969600700 1 -- 172.21.15.151:0/608385851 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fc95802a070 con 0x7fc96c101960 2024-04-26T01:49:53.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:53.014+0000 7fc969600700 1 --2- 172.21.15.151:0/608385851 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fc9540a3c40 0x7fc9540a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:53.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:53.014+0000 7fc969600700 1 -- 172.21.15.151:0/608385851 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7979+0+0 (secure 0 0 0) 0x7fc9580d0f60 con 0x7fc96c101960 2024-04-26T01:49:53.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:53.014+0000 7fc970c00700 1 --2- 172.21.15.151:0/608385851 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fc9540a3c40 0x7fc9540a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:53.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:53.015+0000 7fc970c00700 1 --2- 172.21.15.151:0/608385851 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fc9540a3c40 0x7fc9540a6100 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc960011490 tx=0x7fc960009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:53.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:53.017+0000 7fc969600700 1 -- 172.21.15.151:0/608385851 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc958013070 con 0x7fc96c101960 2024-04-26T01:49:53.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:52 smithi172 ceph-mon[32785]: pgmap v333: 97 pgs: 97 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-26T01:49:53.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:52 smithi165 ceph-mon[32648]: pgmap v333: 97 pgs: 97 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-26T01:49:53.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:53.289+0000 7fc972000700 1 -- 172.21.15.151:0/608385851 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fc96c02df80 con 0x7fc96c101960 2024-04-26T01:49:53.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:53.726+0000 7fc969600700 1 -- 172.21.15.151:0/608385851 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7fc95809da80 con 0x7fc96c101960 2024-04-26T01:49:53.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:53.732+0000 7fc972000700 1 -- 172.21.15.151:0/608385851 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fc9540a3c40 msgr2=0x7fc9540a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:53.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:53.732+0000 7fc972000700 1 --2- 172.21.15.151:0/608385851 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fc9540a3c40 0x7fc9540a6100 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc960011490 tx=0x7fc960009260 comp rx=0 tx=0).stop 2024-04-26T01:49:53.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:53.732+0000 7fc972000700 1 -- 172.21.15.151:0/608385851 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc96c101960 msgr2=0x7fc96c102d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:53.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:53.732+0000 7fc972000700 1 --2- 172.21.15.151:0/608385851 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc96c101960 0x7fc96c102d90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc958000c00 tx=0x7fc9580057c0 comp rx=0 tx=0).stop 2024-04-26T01:49:53.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:53.732+0000 7fc972000700 1 -- 172.21.15.151:0/608385851 shutdown_connections 2024-04-26T01:49:53.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:53.732+0000 7fc972000700 1 --2- 172.21.15.151:0/608385851 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc96c10cfc0 0x7fc96c1032d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:53.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:53.732+0000 7fc972000700 1 --2- 172.21.15.151:0/608385851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc96c101040 0x7fc96c102850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:53.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:53.732+0000 7fc972000700 1 --2- 172.21.15.151:0/608385851 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fc9540a3c40 0x7fc9540a6100 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:53.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:53.732+0000 7fc972000700 1 --2- 172.21.15.151:0/608385851 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc96c101960 0x7fc96c102d90 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:53.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:53.732+0000 7fc972000700 1 -- 172.21.15.151:0/608385851 >> 172.21.15.151:0/608385851 conn(0x7fc96c0766e0 msgr2=0x7fc96c0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:53.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:53.732+0000 7fc972000700 1 -- 172.21.15.151:0/608385851 shutdown_connections 2024-04-26T01:49:53.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:53.733+0000 7fc972000700 1 -- 172.21.15.151:0/608385851 wait complete. 2024-04-26T01:49:54.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:53 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/608385851' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T01:49:54.083 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:53 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/608385851' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T01:49:54.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:53 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/608385851' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T01:49:54.532 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-04-26T01:49:54.532 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-04-26T01:49:54.963 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:49:55.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:54 smithi172 ceph-mon[32785]: pgmap v334: 97 pgs: 97 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-04-26T01:49:55.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:54 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/608385851' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-26T01:49:55.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:54 smithi172 ceph-mon[32785]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T01:49:55.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:54 smithi165 ceph-mon[32648]: pgmap v334: 97 pgs: 97 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-04-26T01:49:55.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:54 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/608385851' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-26T01:49:55.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:54 smithi165 ceph-mon[32648]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T01:49:55.112 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:54 smithi151 ceph-mon[28057]: pgmap v334: 97 pgs: 97 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-04-26T01:49:55.112 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:54 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/608385851' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-26T01:49:55.112 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:54 smithi151 ceph-mon[28057]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T01:49:56.992 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:56 smithi151 ceph-mon[28057]: pgmap v335: 97 pgs: 97 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 3 op/s 2024-04-26T01:49:57.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:56 smithi172 ceph-mon[32785]: pgmap v335: 97 pgs: 97 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 3 op/s 2024-04-26T01:49:57.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:56 smithi165 ceph-mon[32648]: pgmap v335: 97 pgs: 97 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 3 op/s 2024-04-26T01:49:57.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:57.302+0000 7ffa1d600700 1 -- 172.21.15.151:0/2665301949 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa18103320 msgr2=0x7ffa1810f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:57.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:57.302+0000 7ffa1d600700 1 --2- 172.21.15.151:0/2665301949 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa18103320 0x7ffa1810f830 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ffa100098f0 tx=0x7ffa10009c00 comp rx=0 tx=0).stop 2024-04-26T01:49:57.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:57.303+0000 7ffa1d600700 1 -- 172.21.15.151:0/2665301949 shutdown_connections 2024-04-26T01:49:57.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:57.303+0000 7ffa1d600700 1 --2- 172.21.15.151:0/2665301949 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa18103320 0x7ffa1810f830 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:57.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:57.303+0000 7ffa1d600700 1 --2- 172.21.15.151:0/2665301949 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa18108960 0x7ffa18108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:57.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:57.303+0000 7ffa1d600700 1 --2- 172.21.15.151:0/2665301949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa18102960 0x7ffa18102de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:57.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:57.303+0000 7ffa1d600700 1 -- 172.21.15.151:0/2665301949 >> 172.21.15.151:0/2665301949 conn(0x7ffa180fe450 msgr2=0x7ffa18100870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:57.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:57.303+0000 7ffa1d600700 1 -- 172.21.15.151:0/2665301949 shutdown_connections 2024-04-26T01:49:57.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:57.303+0000 7ffa1d600700 1 -- 172.21.15.151:0/2665301949 wait complete. 2024-04-26T01:49:57.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:57.304+0000 7ffa1d600700 1 Processor -- start 2024-04-26T01:49:57.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:57.305+0000 7ffa1d600700 1 -- start start 2024-04-26T01:49:57.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:57.305+0000 7ffa1d600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa18102960 0x7ffa18076930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:57.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:57.305+0000 7ffa1d600700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa18103320 0x7ffa18074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:57.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:57.305+0000 7ffa1d600700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa18108960 0x7ffa180753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:57.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:57.305+0000 7ffa1d600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffa18112ba0 con 0x7ffa18102960 2024-04-26T01:49:57.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:57.305+0000 7ffa1d600700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ffa18112a20 con 0x7ffa18108960 2024-04-26T01:49:57.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:57.305+0000 7ffa1d600700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ffa18112d20 con 0x7ffa18103320 2024-04-26T01:49:57.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:57.305+0000 7ffa17e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa18102960 0x7ffa18076930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:57.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:57.306+0000 7ffa17e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa18102960 0x7ffa18076930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:51828/0 (socket says 172.21.15.151:51828) 2024-04-26T01:49:57.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:57.306+0000 7ffa0f400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa18103320 0x7ffa18074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:57.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:57.306+0000 7ffa17e00700 1 -- 172.21.15.151:0/1357118977 learned_addr learned my addr 172.21.15.151:0/1357118977 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:49:57.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:57.306+0000 7ffa1cc00700 1 --2- 172.21.15.151:0/1357118977 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa18108960 0x7ffa180753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:57.304 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:57.306+0000 7ffa0f400700 1 -- 172.21.15.151:0/1357118977 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa18108960 msgr2=0x7ffa180753a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:57.304 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:57.306+0000 7ffa0f400700 1 --2- 172.21.15.151:0/1357118977 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa18108960 0x7ffa180753a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:57.304 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:57.306+0000 7ffa0f400700 1 -- 172.21.15.151:0/1357118977 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa18102960 msgr2=0x7ffa18076930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:57.304 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:57.306+0000 7ffa0f400700 1 --2- 172.21.15.151:0/1357118977 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa18102960 0x7ffa18076930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:57.304 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:57.306+0000 7ffa0f400700 1 -- 172.21.15.151:0/1357118977 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa181aba00 con 0x7ffa18103320 2024-04-26T01:49:57.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:57.306+0000 7ffa1cc00700 1 --2- 172.21.15.151:0/1357118977 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa18108960 0x7ffa180753a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:49:57.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:57.306+0000 7ffa0f400700 1 --2- 172.21.15.151:0/1357118977 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa18103320 0x7ffa18074d40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ffa0000e9f0 tx=0x7ffa0000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:57.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:57.307+0000 7ffa15600700 1 -- 172.21.15.151:0/1357118977 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa00004220 con 0x7ffa18103320 2024-04-26T01:49:57.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:57.307+0000 7ffa1d600700 1 -- 172.21.15.151:0/1357118977 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa181abc90 con 0x7ffa18103320 2024-04-26T01:49:57.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:57.307+0000 7ffa15600700 1 -- 172.21.15.151:0/1357118977 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa000043c0 con 0x7ffa18103320 2024-04-26T01:49:57.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:57.307+0000 7ffa15600700 1 -- 172.21.15.151:0/1357118977 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa00005020 con 0x7ffa18103320 2024-04-26T01:49:57.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:57.307+0000 7ffa1d600700 1 -- 172.21.15.151:0/1357118977 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa181ac1a0 con 0x7ffa18103320 2024-04-26T01:49:57.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:57.308+0000 7ffa1d600700 1 -- 172.21.15.151:0/1357118977 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa181040a0 con 0x7ffa18103320 2024-04-26T01:49:57.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:57.310+0000 7ffa15600700 1 -- 172.21.15.151:0/1357118977 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7ffa00004750 con 0x7ffa18103320 2024-04-26T01:49:57.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:57.311+0000 7ffa15600700 1 --2- 172.21.15.151:0/1357118977 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7ff9fc0a3c40 0x7ff9fc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:57.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:57.312+0000 7ffa15600700 1 -- 172.21.15.151:0/1357118977 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7979+0+0 (secure 0 0 0) 0x7ffa000d0440 con 0x7ffa18103320 2024-04-26T01:49:57.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:57.312+0000 7ffa17e00700 1 --2- 172.21.15.151:0/1357118977 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7ff9fc0a3c40 0x7ff9fc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:57.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:57.312+0000 7ffa17e00700 1 --2- 172.21.15.151:0/1357118977 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7ff9fc0a3c40 0x7ff9fc0a6100 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ffa08009fd0 tx=0x7ffa0800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:57.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:57.314+0000 7ffa15600700 1 -- 172.21.15.151:0/1357118977 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffa0009ced0 con 0x7ffa18103320 2024-04-26T01:49:57.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:57.589+0000 7ffa1d600700 1 -- 172.21.15.151:0/1357118977 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7ffa1802df80 con 0x7ffa18103320 2024-04-26T01:49:57.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:57.749+0000 7ffa15600700 1 -- 172.21.15.151:0/1357118977 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7ffa0009c7d0 con 0x7ffa18103320 2024-04-26T01:49:57.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:57.753+0000 7ffa1d600700 1 -- 172.21.15.151:0/1357118977 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7ff9fc0a3c40 msgr2=0x7ff9fc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:57.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:57.753+0000 7ffa1d600700 1 --2- 172.21.15.151:0/1357118977 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7ff9fc0a3c40 0x7ff9fc0a6100 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ffa08009fd0 tx=0x7ffa0800b040 comp rx=0 tx=0).stop 2024-04-26T01:49:57.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:57.753+0000 7ffa1d600700 1 -- 172.21.15.151:0/1357118977 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa18103320 msgr2=0x7ffa18074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:57.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:57.753+0000 7ffa1d600700 1 --2- 172.21.15.151:0/1357118977 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa18103320 0x7ffa18074d40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ffa0000e9f0 tx=0x7ffa0000ed00 comp rx=0 tx=0).stop 2024-04-26T01:49:57.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:57.753+0000 7ffa1d600700 1 -- 172.21.15.151:0/1357118977 shutdown_connections 2024-04-26T01:49:57.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:57.753+0000 7ffa1d600700 1 --2- 172.21.15.151:0/1357118977 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa18108960 0x7ffa180753a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:57.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:57.753+0000 7ffa1d600700 1 --2- 172.21.15.151:0/1357118977 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7ff9fc0a3c40 0x7ff9fc0a6100 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:57.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:57.753+0000 7ffa1d600700 1 --2- 172.21.15.151:0/1357118977 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa18102960 0x7ffa18076930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:57.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:57.753+0000 7ffa1d600700 1 --2- 172.21.15.151:0/1357118977 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa18103320 0x7ffa18074d40 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:57.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:57.753+0000 7ffa1d600700 1 -- 172.21.15.151:0/1357118977 >> 172.21.15.151:0/1357118977 conn(0x7ffa180fe450 msgr2=0x7ffa180fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:57.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:57.754+0000 7ffa1d600700 1 -- 172.21.15.151:0/1357118977 shutdown_connections 2024-04-26T01:49:57.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:49:57.754+0000 7ffa1d600700 1 -- 172.21.15.151:0/1357118977 wait complete. 2024-04-26T01:49:57.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:57 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/1357118977' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-26T01:49:57.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:57 smithi151 ceph-mon[28057]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-26T01:49:58.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:57 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/1357118977' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-26T01:49:58.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:57 smithi172 ceph-mon[32785]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-26T01:49:58.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:57 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/1357118977' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-26T01:49:58.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:57 smithi165 ceph-mon[32648]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-26T01:49:58.646 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-04-26T01:49:58.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:58 smithi151 ceph-mon[28057]: pgmap v336: 97 pgs: 97 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s wr, 1 op/s 2024-04-26T01:49:58.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:58 smithi151 ceph-mon[28057]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-26T01:49:58.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:58 smithi151 ceph-mon[28057]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-26T01:49:58.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:58 smithi151 ceph-mon[28057]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-26T01:49:58.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:58 smithi151 ceph-mon[28057]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T01:49:58.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:58 smithi151 ceph-mon[28057]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-26T01:49:58.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:58 smithi151 ceph-mon[28057]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-26T01:49:59.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:58 smithi172 ceph-mon[32785]: pgmap v336: 97 pgs: 97 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s wr, 1 op/s 2024-04-26T01:49:59.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:58 smithi172 ceph-mon[32785]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-26T01:49:59.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:58 smithi172 ceph-mon[32785]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-26T01:49:59.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:58 smithi172 ceph-mon[32785]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-26T01:49:59.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:58 smithi172 ceph-mon[32785]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T01:49:59.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:58 smithi172 ceph-mon[32785]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-26T01:49:59.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:58 smithi172 ceph-mon[32785]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-26T01:49:59.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:58 smithi165 ceph-mon[32648]: pgmap v336: 97 pgs: 97 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s wr, 1 op/s 2024-04-26T01:49:59.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:58 smithi165 ceph-mon[32648]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-26T01:49:59.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:58 smithi165 ceph-mon[32648]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-26T01:49:59.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:58 smithi165 ceph-mon[32648]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-26T01:49:59.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:58 smithi165 ceph-mon[32648]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T01:49:59.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:58 smithi165 ceph-mon[32648]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-26T01:49:59.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:58 smithi165 ceph-mon[32648]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-26T01:49:59.104 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:50:00.010 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:59 smithi165 ceph-mon[32648]: mds.? [v2:172.21.15.172:6838/1242302810,v1:172.21.15.172:6839/1242302810] up:active 2024-04-26T01:50:00.010 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:59 smithi165 ceph-mon[32648]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-26T01:50:00.010 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:59 smithi165 ceph-mon[32648]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-26T01:50:00.010 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:59 smithi165 ceph-mon[32648]: Cluster is now healthy 2024-04-26T01:50:00.010 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:59 smithi165 ceph-mon[32648]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-26T01:50:00.010 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:59 smithi165 ceph-mon[32648]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-04-26T01:50:00.010 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:49:59 smithi165 ceph-mon[32648]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-26T01:50:00.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:59 smithi172 ceph-mon[32785]: mds.? [v2:172.21.15.172:6838/1242302810,v1:172.21.15.172:6839/1242302810] up:active 2024-04-26T01:50:00.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:59 smithi172 ceph-mon[32785]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-26T01:50:00.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:59 smithi172 ceph-mon[32785]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-26T01:50:00.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:59 smithi172 ceph-mon[32785]: Cluster is now healthy 2024-04-26T01:50:00.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:59 smithi172 ceph-mon[32785]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-26T01:50:00.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:59 smithi172 ceph-mon[32785]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-04-26T01:50:00.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:49:59 smithi172 ceph-mon[32785]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-26T01:50:00.106 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:59 smithi151 ceph-mon[28057]: mds.? [v2:172.21.15.172:6838/1242302810,v1:172.21.15.172:6839/1242302810] up:active 2024-04-26T01:50:00.106 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:59 smithi151 ceph-mon[28057]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-26T01:50:00.106 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:59 smithi151 ceph-mon[28057]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-26T01:50:00.106 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:59 smithi151 ceph-mon[28057]: Cluster is now healthy 2024-04-26T01:50:00.106 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:59 smithi151 ceph-mon[28057]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-26T01:50:00.106 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:59 smithi151 ceph-mon[28057]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-04-26T01:50:00.107 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:49:59 smithi151 ceph-mon[28057]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-26T01:50:00.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:00 smithi151 ceph-mon[28057]: pgmap v337: 97 pgs: 97 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-04-26T01:50:00.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:00 smithi151 ceph-mon[28057]: mds.? [v2:172.21.15.172:6836/55972537,v1:172.21.15.172:6837/55972537] up:active 2024-04-26T01:50:00.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:00 smithi151 ceph-mon[28057]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-04-26T01:50:00.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:00 smithi151 ceph-mon[28057]: overall HEALTH_OK 2024-04-26T01:50:01.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:00 smithi172 ceph-mon[32785]: pgmap v337: 97 pgs: 97 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-04-26T01:50:01.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:00 smithi172 ceph-mon[32785]: mds.? [v2:172.21.15.172:6836/55972537,v1:172.21.15.172:6837/55972537] up:active 2024-04-26T01:50:01.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:00 smithi172 ceph-mon[32785]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-04-26T01:50:01.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:00 smithi172 ceph-mon[32785]: overall HEALTH_OK 2024-04-26T01:50:01.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:00 smithi165 ceph-mon[32648]: pgmap v337: 97 pgs: 97 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-04-26T01:50:01.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:00 smithi165 ceph-mon[32648]: mds.? [v2:172.21.15.172:6836/55972537,v1:172.21.15.172:6837/55972537] up:active 2024-04-26T01:50:01.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:00 smithi165 ceph-mon[32648]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-04-26T01:50:01.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:00 smithi165 ceph-mon[32648]: overall HEALTH_OK 2024-04-26T01:50:01.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:01.433+0000 7f5ee7200700 1 -- 172.21.15.151:0/1884647143 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ee810f030 msgr2=0x7f5ee810f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:01.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:01.433+0000 7f5ee7200700 1 --2- 172.21.15.151:0/1884647143 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ee810f030 0x7f5ee810f410 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8009990 tx=0x7f5ed8009ca0 comp rx=0 tx=0).stop 2024-04-26T01:50:01.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:01.433+0000 7f5ee7200700 1 -- 172.21.15.151:0/1884647143 shutdown_connections 2024-04-26T01:50:01.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:01.433+0000 7f5ee7200700 1 --2- 172.21.15.151:0/1884647143 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ee8109940 0x7f5ee8115de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:01.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:01.433+0000 7f5ee7200700 1 --2- 172.21.15.151:0/1884647143 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ee810f030 0x7f5ee810f410 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:01.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:01.433+0000 7f5ee7200700 1 --2- 172.21.15.151:0/1884647143 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ee8108f80 0x7f5ee8109400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:01.433 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:01.433+0000 7f5ee7200700 1 -- 172.21.15.151:0/1884647143 >> 172.21.15.151:0/1884647143 conn(0x7f5ee80787d0 msgr2=0x7f5ee8107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:01.433 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:01.433+0000 7f5ee7200700 1 -- 172.21.15.151:0/1884647143 shutdown_connections 2024-04-26T01:50:01.433 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:01.433+0000 7f5ee7200700 1 -- 172.21.15.151:0/1884647143 wait complete. 2024-04-26T01:50:01.434 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:01.434+0000 7f5ee7200700 1 Processor -- start 2024-04-26T01:50:01.434 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:01.435+0000 7f5ee7200700 1 -- start start 2024-04-26T01:50:01.434 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:01.435+0000 7f5ee7200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ee8108f80 0x7f5ee819f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:01.434 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:01.435+0000 7f5ee7200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ee8109940 0x7f5ee819faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:01.434 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:01.435+0000 7f5ee7200700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ee810f030 0x7f5ee81a3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:01.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:01.435+0000 7f5ee7200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee81190f0 con 0x7f5ee8109940 2024-04-26T01:50:01.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:01.435+0000 7f5ee7200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee8118f70 con 0x7f5ee8108f80 2024-04-26T01:50:01.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:01.435+0000 7f5ee7200700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee8119270 con 0x7f5ee810f030 2024-04-26T01:50:01.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:01.435+0000 7f5ee5400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ee8109940 0x7f5ee819faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:01.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:01.435+0000 7f5ee5400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ee8109940 0x7f5ee819faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:57858/0 (socket says 172.21.15.151:57858) 2024-04-26T01:50:01.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:01.435+0000 7f5ee5400700 1 -- 172.21.15.151:0/3599706616 learned_addr learned my addr 172.21.15.151:0/3599706616 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:50:01.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:01.435+0000 7f5ee5e00700 1 --2- 172.21.15.151:0/3599706616 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ee8108f80 0x7f5ee819f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:01.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:01.435+0000 7f5ee6800700 1 --2- 172.21.15.151:0/3599706616 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ee810f030 0x7f5ee81a3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:01.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:01.436+0000 7f5ee5400700 1 -- 172.21.15.151:0/3599706616 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ee810f030 msgr2=0x7f5ee81a3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:01.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:01.436+0000 7f5ee5400700 1 --2- 172.21.15.151:0/3599706616 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ee810f030 0x7f5ee81a3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:01.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:01.436+0000 7f5ee5400700 1 -- 172.21.15.151:0/3599706616 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ee8108f80 msgr2=0x7f5ee819f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:01.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:01.436+0000 7f5ee5400700 1 --2- 172.21.15.151:0/3599706616 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ee8108f80 0x7f5ee819f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:01.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:01.436+0000 7f5ee5400700 1 -- 172.21.15.151:0/3599706616 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ee81a4570 con 0x7f5ee8109940 2024-04-26T01:50:01.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:01.436+0000 7f5ee5e00700 1 --2- 172.21.15.151:0/3599706616 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ee8108f80 0x7f5ee819f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:50:01.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:01.436+0000 7f5ee5400700 1 --2- 172.21.15.151:0/3599706616 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ee8109940 0x7f5ee819faa0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5ed400eb30 tx=0x7f5ed400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:01.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:01.436+0000 7f5edea00700 1 -- 172.21.15.151:0/3599706616 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed400cbd0 con 0x7f5ee8109940 2024-04-26T01:50:01.437 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:01.437+0000 7f5ee7200700 1 -- 172.21.15.151:0/3599706616 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ee81a4860 con 0x7f5ee8109940 2024-04-26T01:50:01.437 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:01.437+0000 7f5edea00700 1 -- 172.21.15.151:0/3599706616 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ed40044d0 con 0x7f5ee8109940 2024-04-26T01:50:01.437 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:01.437+0000 7f5edea00700 1 -- 172.21.15.151:0/3599706616 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed4003df0 con 0x7f5ee8109940 2024-04-26T01:50:01.437 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:01.437+0000 7f5ee7200700 1 -- 172.21.15.151:0/3599706616 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ee81ac0e0 con 0x7f5ee8109940 2024-04-26T01:50:01.437 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:01.438+0000 7f5ee7200700 1 -- 172.21.15.151:0/3599706616 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ee8111380 con 0x7f5ee8109940 2024-04-26T01:50:01.438 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:01.440+0000 7f5edea00700 1 -- 172.21.15.151:0/3599706616 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f5ed4004000 con 0x7f5ee8109940 2024-04-26T01:50:01.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:01.441+0000 7f5edea00700 1 --2- 172.21.15.151:0/3599706616 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f5ecc0a3cc0 0x7f5ecc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:01.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:01.441+0000 7f5edea00700 1 -- 172.21.15.151:0/3599706616 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7979+0+0 (secure 0 0 0) 0x7f5ed40d0640 con 0x7f5ee8109940 2024-04-26T01:50:01.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:01.442+0000 7f5ee5e00700 1 --2- 172.21.15.151:0/3599706616 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f5ecc0a3cc0 0x7f5ecc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:01.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:01.442+0000 7f5ee5e00700 1 --2- 172.21.15.151:0/3599706616 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f5ecc0a3cc0 0x7f5ecc0a6180 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5ed80098e0 tx=0x7f5ed8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:01.442 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:01.444+0000 7f5edea00700 1 -- 172.21.15.151:0/3599706616 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5ed40042f0 con 0x7f5ee8109940 2024-04-26T01:50:01.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:01.724+0000 7f5ee7200700 1 -- 172.21.15.151:0/3599706616 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f5ee8069730 con 0x7f5ee8109940 2024-04-26T01:50:01.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:01.783+0000 7f5edea00700 1 -- 172.21.15.151:0/3599706616 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7f5ed409d160 con 0x7f5ee8109940 2024-04-26T01:50:01.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:01.786+0000 7f5ee7200700 1 -- 172.21.15.151:0/3599706616 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f5ecc0a3cc0 msgr2=0x7f5ecc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:01.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:01.786+0000 7f5ee7200700 1 --2- 172.21.15.151:0/3599706616 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f5ecc0a3cc0 0x7f5ecc0a6180 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5ed80098e0 tx=0x7f5ed8005960 comp rx=0 tx=0).stop 2024-04-26T01:50:01.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:01.786+0000 7f5ee7200700 1 -- 172.21.15.151:0/3599706616 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ee8109940 msgr2=0x7f5ee819faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:01.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:01.786+0000 7f5ee7200700 1 --2- 172.21.15.151:0/3599706616 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ee8109940 0x7f5ee819faa0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5ed400eb30 tx=0x7f5ed400ee40 comp rx=0 tx=0).stop 2024-04-26T01:50:01.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:01.786+0000 7f5ee7200700 1 -- 172.21.15.151:0/3599706616 shutdown_connections 2024-04-26T01:50:01.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:01.786+0000 7f5ee7200700 1 --2- 172.21.15.151:0/3599706616 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ee810f030 0x7f5ee81a3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:01.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:01.786+0000 7f5ee7200700 1 --2- 172.21.15.151:0/3599706616 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ee8108f80 0x7f5ee819f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:01.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:01.786+0000 7f5ee7200700 1 --2- 172.21.15.151:0/3599706616 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f5ecc0a3cc0 0x7f5ecc0a6180 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:01.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:01.786+0000 7f5ee7200700 1 --2- 172.21.15.151:0/3599706616 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ee8109940 0x7f5ee819faa0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:01.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:01.786+0000 7f5ee7200700 1 -- 172.21.15.151:0/3599706616 >> 172.21.15.151:0/3599706616 conn(0x7f5ee80787d0 msgr2=0x7f5ee8079b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:01.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:01.786+0000 7f5ee7200700 1 -- 172.21.15.151:0/3599706616 shutdown_connections 2024-04-26T01:50:01.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:01.787+0000 7f5ee7200700 1 -- 172.21.15.151:0/3599706616 wait complete. 2024-04-26T01:50:02.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:01 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/3599706616' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-26T01:50:02.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:01 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/3599706616' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-26T01:50:02.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:01 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/3599706616' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-26T01:50:02.666 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-04-26T01:50:02.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:02 smithi151 ceph-mon[28057]: pgmap v338: 97 pgs: 97 active+clean; 580 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 5 op/s 2024-04-26T01:50:02.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:02 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/3599706616' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-26T01:50:02.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:02 smithi151 ceph-mon[28057]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-04-26T01:50:02.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:02 smithi151 ceph-mon[28057]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T01:50:03.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:02 smithi172 ceph-mon[32785]: pgmap v338: 97 pgs: 97 active+clean; 580 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 5 op/s 2024-04-26T01:50:03.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:02 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/3599706616' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-26T01:50:03.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:02 smithi172 ceph-mon[32785]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-04-26T01:50:03.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:02 smithi172 ceph-mon[32785]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T01:50:03.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:02 smithi165 ceph-mon[32648]: pgmap v338: 97 pgs: 97 active+clean; 580 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 5 op/s 2024-04-26T01:50:03.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:02 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/3599706616' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-26T01:50:03.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:02 smithi165 ceph-mon[32648]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-04-26T01:50:03.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:02 smithi165 ceph-mon[32648]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T01:50:03.153 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:50:05.042 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:04 smithi151 ceph-mon[28057]: pgmap v339: 97 pgs: 97 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s rd, 1.8 KiB/s wr, 6 op/s 2024-04-26T01:50:05.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:04 smithi165 ceph-mon[32648]: pgmap v339: 97 pgs: 97 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s rd, 1.8 KiB/s wr, 6 op/s 2024-04-26T01:50:05.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:04 smithi172 ceph-mon[32785]: pgmap v339: 97 pgs: 97 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s rd, 1.8 KiB/s wr, 6 op/s 2024-04-26T01:50:05.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:05.323+0000 7f72ab200700 1 -- 172.21.15.151:0/136595013 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72a4103450 msgr2=0x7f72a410f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:05.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:05.323+0000 7f72ab200700 1 --2- 172.21.15.151:0/136595013 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72a4103450 0x7f72a410f8b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f729c00b0a0 tx=0x7f729c00b3b0 comp rx=0 tx=0).stop 2024-04-26T01:50:05.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:05.324+0000 7f72ab200700 1 -- 172.21.15.151:0/136595013 shutdown_connections 2024-04-26T01:50:05.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:05.324+0000 7f72ab200700 1 --2- 172.21.15.151:0/136595013 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72a4103450 0x7f72a410f8b0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:05.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:05.324+0000 7f72ab200700 1 --2- 172.21.15.151:0/136595013 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72a4108a50 0x7f72a4108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:05.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:05.324+0000 7f72ab200700 1 --2- 172.21.15.151:0/136595013 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f72a4102a90 0x7f72a4102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:05.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:05.324+0000 7f72ab200700 1 -- 172.21.15.151:0/136595013 >> 172.21.15.151:0/136595013 conn(0x7f72a4079ba0 msgr2=0x7f72a4079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:05.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:05.324+0000 7f72ab200700 1 -- 172.21.15.151:0/136595013 shutdown_connections 2024-04-26T01:50:05.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:05.324+0000 7f72ab200700 1 -- 172.21.15.151:0/136595013 wait complete. 2024-04-26T01:50:05.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:05.325+0000 7f72ab200700 1 Processor -- start 2024-04-26T01:50:05.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:05.325+0000 7f72ab200700 1 -- start start 2024-04-26T01:50:05.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:05.326+0000 7f72ab200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72a4102a90 0x7f72a419b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:05.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:05.326+0000 7f72ab200700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f72a4103450 0x7f72a419b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:05.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:05.326+0000 7f72ab200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72a4108a50 0x7f72a419fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:05.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:05.326+0000 7f72ab200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f72a4112c30 con 0x7f72a4102a90 2024-04-26T01:50:05.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:05.326+0000 7f72ab200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f72a4112ab0 con 0x7f72a4108a50 2024-04-26T01:50:05.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:05.326+0000 7f72ab200700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f72a4112db0 con 0x7f72a4103450 2024-04-26T01:50:05.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:05.326+0000 7f72a3400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f72a4103450 0x7f72a419b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:05.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:05.326+0000 7f72a3400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f72a4103450 0x7f72a419b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:37458/0 (socket says 172.21.15.151:37458) 2024-04-26T01:50:05.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:05.326+0000 7f72a3400700 1 -- 172.21.15.151:0/1705779034 learned_addr learned my addr 172.21.15.151:0/1705779034 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:50:05.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:05.327+0000 7f72a3e00700 1 --2- 172.21.15.151:0/1705779034 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72a4102a90 0x7f72a419b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:05.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:05.327+0000 7f72a9200700 1 --2- 172.21.15.151:0/1705779034 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72a4108a50 0x7f72a419fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:05.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:05.327+0000 7f72a3400700 1 -- 172.21.15.151:0/1705779034 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72a4108a50 msgr2=0x7f72a419fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:05.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:05.327+0000 7f72a3400700 1 --2- 172.21.15.151:0/1705779034 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72a4108a50 0x7f72a419fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:05.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:05.327+0000 7f72a3400700 1 -- 172.21.15.151:0/1705779034 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72a4102a90 msgr2=0x7f72a419b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:05.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:05.327+0000 7f72a3400700 1 --2- 172.21.15.151:0/1705779034 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72a4102a90 0x7f72a419b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:05.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:05.327+0000 7f72a3400700 1 -- 172.21.15.151:0/1705779034 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72a41a03b0 con 0x7f72a4103450 2024-04-26T01:50:05.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:05.327+0000 7f72a3400700 1 --2- 172.21.15.151:0/1705779034 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f72a4103450 0x7f72a419b8e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f729000e9a0 tx=0x7f729000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:05.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:05.328+0000 7f72a0c00700 1 -- 172.21.15.151:0/1705779034 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729000ca10 con 0x7f72a4103450 2024-04-26T01:50:05.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:05.328+0000 7f72a0c00700 1 -- 172.21.15.151:0/1705779034 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f729000cbb0 con 0x7f72a4103450 2024-04-26T01:50:05.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:05.328+0000 7f72ab200700 1 -- 172.21.15.151:0/1705779034 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72a41a06a0 con 0x7f72a4103450 2024-04-26T01:50:05.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:05.328+0000 7f72ab200700 1 -- 172.21.15.151:0/1705779034 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72a41a7f20 con 0x7f72a4103450 2024-04-26T01:50:05.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:05.328+0000 7f72a0c00700 1 -- 172.21.15.151:0/1705779034 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7290018680 con 0x7f72a4103450 2024-04-26T01:50:05.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:05.329+0000 7f72ab200700 1 -- 172.21.15.151:0/1705779034 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72a4069730 con 0x7f72a4103450 2024-04-26T01:50:05.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:05.331+0000 7f72a0c00700 1 -- 172.21.15.151:0/1705779034 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f7290010430 con 0x7f72a4103450 2024-04-26T01:50:05.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:05.332+0000 7f72a0c00700 1 --2- 172.21.15.151:0/1705779034 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f72880a3cc0 0x7f72880a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:05.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:05.332+0000 7f72a0c00700 1 -- 172.21.15.151:0/1705779034 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7979+0+0 (secure 0 0 0) 0x7f72900d0760 con 0x7f72a4103450 2024-04-26T01:50:05.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:05.333+0000 7f72a3e00700 1 --2- 172.21.15.151:0/1705779034 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f72880a3cc0 0x7f72880a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:05.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:05.333+0000 7f72a3e00700 1 --2- 172.21.15.151:0/1705779034 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f72880a3cc0 0x7f72880a6180 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7294009be0 tx=0x7f7294009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:05.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:05.335+0000 7f72a0c00700 1 -- 172.21.15.151:0/1705779034 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f729009d1f0 con 0x7f72a4103450 2024-04-26T01:50:05.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:05.605+0000 7f72ab200700 1 -- 172.21.15.151:0/1705779034 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f72a41a8150 con 0x7f72a4103450 2024-04-26T01:50:05.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:05.807+0000 7f72a0c00700 1 -- 172.21.15.151:0/1705779034 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v25) ==== 115+0+0 (secure 0 0 0) 0x7f729009caf0 con 0x7f72a4103450 2024-04-26T01:50:05.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:05.811+0000 7f72ab200700 1 -- 172.21.15.151:0/1705779034 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f72880a3cc0 msgr2=0x7f72880a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:05.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:05.811+0000 7f72ab200700 1 --2- 172.21.15.151:0/1705779034 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f72880a3cc0 0x7f72880a6180 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7294009be0 tx=0x7f7294009260 comp rx=0 tx=0).stop 2024-04-26T01:50:05.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:05.811+0000 7f72ab200700 1 -- 172.21.15.151:0/1705779034 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f72a4103450 msgr2=0x7f72a419b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:05.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:05.811+0000 7f72ab200700 1 --2- 172.21.15.151:0/1705779034 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f72a4103450 0x7f72a419b8e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f729000e9a0 tx=0x7f729000ecb0 comp rx=0 tx=0).stop 2024-04-26T01:50:05.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:05.811+0000 7f72ab200700 1 -- 172.21.15.151:0/1705779034 shutdown_connections 2024-04-26T01:50:05.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:05.811+0000 7f72ab200700 1 --2- 172.21.15.151:0/1705779034 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72a4108a50 0x7f72a419fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:05.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:05.811+0000 7f72ab200700 1 --2- 172.21.15.151:0/1705779034 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f72880a3cc0 0x7f72880a6180 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:05.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:05.811+0000 7f72ab200700 1 --2- 172.21.15.151:0/1705779034 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72a4102a90 0x7f72a419b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:05.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:05.811+0000 7f72ab200700 1 --2- 172.21.15.151:0/1705779034 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f72a4103450 0x7f72a419b8e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:05.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:05.811+0000 7f72ab200700 1 -- 172.21.15.151:0/1705779034 >> 172.21.15.151:0/1705779034 conn(0x7f72a4079ba0 msgr2=0x7f72a410df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:05.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:05.811+0000 7f72ab200700 1 -- 172.21.15.151:0/1705779034 shutdown_connections 2024-04-26T01:50:05.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:05.811+0000 7f72ab200700 1 -- 172.21.15.151:0/1705779034 wait complete. 2024-04-26T01:50:05.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:05 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/1705779034' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-26T01:50:05.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:05 smithi151 ceph-mon[28057]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-26T01:50:06.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:05 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/1705779034' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-26T01:50:06.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:05 smithi165 ceph-mon[32648]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-26T01:50:06.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:05 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/1705779034' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-26T01:50:06.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:05 smithi172 ceph-mon[32785]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-26T01:50:06.598 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-04-26T01:50:06.599 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-04-26T01:50:07.058 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:50:07.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:06 smithi165 ceph-mon[32648]: pgmap v340: 97 pgs: 97 active+clean; 582 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-04-26T01:50:07.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:06 smithi165 ceph-mon[32648]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-26T01:50:07.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:06 smithi165 ceph-mon[32648]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T01:50:07.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:06 smithi151 ceph-mon[28057]: pgmap v340: 97 pgs: 97 active+clean; 582 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-04-26T01:50:07.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:06 smithi151 ceph-mon[28057]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-26T01:50:07.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:06 smithi151 ceph-mon[28057]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T01:50:07.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:06 smithi172 ceph-mon[32785]: pgmap v340: 97 pgs: 97 active+clean; 582 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-04-26T01:50:07.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:06 smithi172 ceph-mon[32785]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-26T01:50:07.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:06 smithi172 ceph-mon[32785]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T01:50:09.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:08 smithi165 ceph-mon[32648]: pgmap v341: 97 pgs: 97 active+clean; 582 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-04-26T01:50:09.108 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:08 smithi151 ceph-mon[28057]: pgmap v341: 97 pgs: 97 active+clean; 582 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-04-26T01:50:09.281 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:09.282+0000 7fabc6200700 1 -- 172.21.15.151:0/4122478214 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fabc006bab0 msgr2=0x7fabc010f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:09.281 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:09.282+0000 7fabc6200700 1 --2- 172.21.15.151:0/4122478214 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fabc006bab0 0x7fabc010f920 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fabb800b0a0 tx=0x7fabb800b3b0 comp rx=0 tx=0).stop 2024-04-26T01:50:09.281 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:09.282+0000 7fabc6200700 1 -- 172.21.15.151:0/4122478214 shutdown_connections 2024-04-26T01:50:09.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:09.282+0000 7fabc6200700 1 --2- 172.21.15.151:0/4122478214 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fabc006bab0 0x7fabc010f920 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:09.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:09.282+0000 7fabc6200700 1 --2- 172.21.15.151:0/4122478214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fabc0108a50 0x7fabc0108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:09.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:09.282+0000 7fabc6200700 1 --2- 172.21.15.151:0/4122478214 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fabc006b110 0x7fabc006b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:09.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:09.282+0000 7fabc6200700 1 -- 172.21.15.151:0/4122478214 >> 172.21.15.151:0/4122478214 conn(0x7fabc00788f0 msgr2=0x7fabc0078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:09.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:09.283+0000 7fabc6200700 1 -- 172.21.15.151:0/4122478214 shutdown_connections 2024-04-26T01:50:09.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:09.283+0000 7fabc6200700 1 -- 172.21.15.151:0/4122478214 wait complete. 2024-04-26T01:50:09.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:09.284+0000 7fabc6200700 1 Processor -- start 2024-04-26T01:50:09.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:09.284+0000 7fabc6200700 1 -- start start 2024-04-26T01:50:09.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:09.284+0000 7fabc6200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fabc006b110 0x7fabc019b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:09.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:09.284+0000 7fabc6200700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fabc006bab0 0x7fabc019b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:09.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:09.284+0000 7fabc6200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fabc0108a50 0x7fabc019fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:09.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:09.284+0000 7fabc6200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fabc0112c50 con 0x7fabc0108a50 2024-04-26T01:50:09.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:09.284+0000 7fabc6200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fabc0112ad0 con 0x7fabc006b110 2024-04-26T01:50:09.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:09.284+0000 7fabc6200700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fabc0112dd0 con 0x7fabc006bab0 2024-04-26T01:50:09.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:09.285+0000 7fabbf400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fabc006b110 0x7fabc019b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:09.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:09.285+0000 7fabb7e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fabc006bab0 0x7fabc019b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:09.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:09.285+0000 7fabbfe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fabc0108a50 0x7fabc019fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:09.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:09.285+0000 7fabbf400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fabc006b110 0x7fabc019b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:50056/0 (socket says 172.21.15.151:50056) 2024-04-26T01:50:09.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:09.285+0000 7fabbf400700 1 -- 172.21.15.151:0/89743318 learned_addr learned my addr 172.21.15.151:0/89743318 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:50:09.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:09.285+0000 7fabb7e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fabc006bab0 0x7fabc019b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:37476/0 (socket says 172.21.15.151:37476) 2024-04-26T01:50:09.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:09.285+0000 7fabbf400700 1 -- 172.21.15.151:0/89743318 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fabc006bab0 msgr2=0x7fabc019b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:09.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:09.285+0000 7fabbf400700 1 --2- 172.21.15.151:0/89743318 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fabc006bab0 0x7fabc019b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:09.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:09.285+0000 7fabbf400700 1 -- 172.21.15.151:0/89743318 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fabc0108a50 msgr2=0x7fabc019fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:09.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:09.285+0000 7fabbf400700 1 --2- 172.21.15.151:0/89743318 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fabc0108a50 0x7fabc019fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:09.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:09.285+0000 7fabbf400700 1 -- 172.21.15.151:0/89743318 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabc01a0300 con 0x7fabc006b110 2024-04-26T01:50:09.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:09.285+0000 7fabbf400700 1 --2- 172.21.15.151:0/89743318 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fabc006b110 0x7fabc019b2f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fabb000e9c0 tx=0x7fabb000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:09.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:09.286+0000 7fabbcc00700 1 -- 172.21.15.151:0/89743318 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb000cbd0 con 0x7fabc006b110 2024-04-26T01:50:09.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:09.286+0000 7fabbcc00700 1 -- 172.21.15.151:0/89743318 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabb000ee60 con 0x7fabc006b110 2024-04-26T01:50:09.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:09.286+0000 7fabc6200700 1 -- 172.21.15.151:0/89743318 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabc01a05f0 con 0x7fabc006b110 2024-04-26T01:50:09.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:09.286+0000 7fabbcc00700 1 -- 172.21.15.151:0/89743318 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb00055c0 con 0x7fabc006b110 2024-04-26T01:50:09.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:09.286+0000 7fabc6200700 1 -- 172.21.15.151:0/89743318 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabc01a7e70 con 0x7fabc006b110 2024-04-26T01:50:09.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:08 smithi172 ceph-mon[32785]: pgmap v341: 97 pgs: 97 active+clean; 582 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-04-26T01:50:09.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:09.287+0000 7fabc6200700 1 -- 172.21.15.151:0/89743318 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabc0104190 con 0x7fabc006b110 2024-04-26T01:50:09.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:09.289+0000 7fabbcc00700 1 -- 172.21.15.151:0/89743318 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fabb0004750 con 0x7fabc006b110 2024-04-26T01:50:09.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:09.290+0000 7fabbcc00700 1 --2- 172.21.15.151:0/89743318 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7faba40a3cc0 0x7faba40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:09.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:09.290+0000 7fabbcc00700 1 -- 172.21.15.151:0/89743318 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7979+0+0 (secure 0 0 0) 0x7fabb00d0490 con 0x7fabc006b110 2024-04-26T01:50:09.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:09.291+0000 7fabb7e00700 1 --2- 172.21.15.151:0/89743318 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7faba40a3cc0 0x7faba40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:09.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:09.291+0000 7fabb7e00700 1 --2- 172.21.15.151:0/89743318 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7faba40a3cc0 0x7faba40a6180 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fabc0078e60 tx=0x7faba8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:09.293 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:09.295+0000 7fabbcc00700 1 -- 172.21.15.151:0/89743318 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fabb009cf20 con 0x7fabc006b110 2024-04-26T01:50:09.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:09.557+0000 7fabc6200700 1 -- 172.21.15.151:0/89743318 --> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fabc0062e80 con 0x7faba40a3cc0 2024-04-26T01:50:09.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:09.592+0000 7fabbcc00700 1 -- 172.21.15.151:0/89743318 <== mgr.14152 v2:172.21.15.151:6800/2251256505 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fabc0062e80 con 0x7faba40a3cc0 2024-04-26T01:50:09.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:09.594+0000 7fabc6200700 1 -- 172.21.15.151:0/89743318 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7faba40a3cc0 msgr2=0x7faba40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:09.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:09.594+0000 7fabc6200700 1 --2- 172.21.15.151:0/89743318 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7faba40a3cc0 0x7faba40a6180 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fabc0078e60 tx=0x7faba8009260 comp rx=0 tx=0).stop 2024-04-26T01:50:09.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:09.594+0000 7fabc6200700 1 -- 172.21.15.151:0/89743318 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fabc006b110 msgr2=0x7fabc019b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:09.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:09.594+0000 7fabc6200700 1 --2- 172.21.15.151:0/89743318 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fabc006b110 0x7fabc019b2f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fabb000e9c0 tx=0x7fabb000ecd0 comp rx=0 tx=0).stop 2024-04-26T01:50:09.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:09.594+0000 7fabc6200700 1 -- 172.21.15.151:0/89743318 shutdown_connections 2024-04-26T01:50:09.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:09.594+0000 7fabc6200700 1 --2- 172.21.15.151:0/89743318 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7faba40a3cc0 0x7faba40a6180 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:09.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:09.594+0000 7fabc6200700 1 --2- 172.21.15.151:0/89743318 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fabc0108a50 0x7fabc019fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:09.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:09.594+0000 7fabc6200700 1 --2- 172.21.15.151:0/89743318 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fabc006b110 0x7fabc019b2f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:09.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:09.594+0000 7fabc6200700 1 --2- 172.21.15.151:0/89743318 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fabc006bab0 0x7fabc019b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:09.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:09.594+0000 7fabc6200700 1 -- 172.21.15.151:0/89743318 >> 172.21.15.151:0/89743318 conn(0x7fabc00788f0 msgr2=0x7fabc010de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:09.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:09.595+0000 7fabc6200700 1 -- 172.21.15.151:0/89743318 shutdown_connections 2024-04-26T01:50:09.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:09.595+0000 7fabc6200700 1 -- 172.21.15.151:0/89743318 wait complete. 2024-04-26T01:50:09.872 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:50:09 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:50:09.565+0000 7f6792000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:50:09.872 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:50:09 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:50:09.565+0000 7f6792000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:50:09.872 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:50:09 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:50:09.565+0000 7f6792000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:50:09.872 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:50:09 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:50:09.565+0000 7f6792000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:50:09.872 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:50:09 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:50:09.565+0000 7f6792000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:50:10.441 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-26T01:50:10.896 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:50:11.078 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:10 smithi151 ceph-mon[28057]: pgmap v342: 97 pgs: 97 active+clean; 582 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-04-26T01:50:11.079 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:10 smithi151 ceph-mon[28057]: from='client.24581 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:50:11.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:10 smithi165 ceph-mon[32648]: pgmap v342: 97 pgs: 97 active+clean; 582 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-04-26T01:50:11.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:10 smithi165 ceph-mon[32648]: from='client.24581 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:50:11.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:10 smithi172 ceph-mon[32785]: pgmap v342: 97 pgs: 97 active+clean; 582 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-04-26T01:50:11.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:10 smithi172 ceph-mon[32785]: from='client.24581 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:50:13.039 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:12 smithi151 ceph-mon[28057]: pgmap v343: 97 pgs: 97 active+clean; 588 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.3 KiB/s wr, 12 op/s 2024-04-26T01:50:13.039 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:12 smithi151 ceph-mon[28057]: mgrmap e17: x(active, since 8m), standbys: y, z 2024-04-26T01:50:13.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:12 smithi165 ceph-mon[32648]: pgmap v343: 97 pgs: 97 active+clean; 588 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.3 KiB/s wr, 12 op/s 2024-04-26T01:50:13.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:12 smithi165 ceph-mon[32648]: mgrmap e17: x(active, since 8m), standbys: y, z 2024-04-26T01:50:13.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:13.214+0000 7feb84e00700 1 -- 172.21.15.151:0/689597819 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb80103320 msgr2=0x7feb8010f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:13.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:13.214+0000 7feb84e00700 1 --2- 172.21.15.151:0/689597819 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb80103320 0x7feb8010f830 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7feb7800b0a0 tx=0x7feb7800b3b0 comp rx=0 tx=0).stop 2024-04-26T01:50:13.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:13.214+0000 7feb84e00700 1 -- 172.21.15.151:0/689597819 shutdown_connections 2024-04-26T01:50:13.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:13.214+0000 7feb84e00700 1 --2- 172.21.15.151:0/689597819 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb80103320 0x7feb8010f830 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:13.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:13.214+0000 7feb84e00700 1 --2- 172.21.15.151:0/689597819 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feb80108960 0x7feb80108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:13.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:13.214+0000 7feb84e00700 1 --2- 172.21.15.151:0/689597819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb80102960 0x7feb80102de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:13.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:13.214+0000 7feb84e00700 1 -- 172.21.15.151:0/689597819 >> 172.21.15.151:0/689597819 conn(0x7feb800fe450 msgr2=0x7feb80100870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:13.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:13.215+0000 7feb84e00700 1 -- 172.21.15.151:0/689597819 shutdown_connections 2024-04-26T01:50:13.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:13.215+0000 7feb84e00700 1 -- 172.21.15.151:0/689597819 wait complete. 2024-04-26T01:50:13.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:13.216+0000 7feb84e00700 1 Processor -- start 2024-04-26T01:50:13.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:13.216+0000 7feb84e00700 1 -- start start 2024-04-26T01:50:13.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:13.216+0000 7feb84e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feb80102960 0x7feb801098b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:13.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:13.216+0000 7feb84e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb80103320 0x7feb80109df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:13.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:13.216+0000 7feb84e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb80108960 0x7feb8010a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:13.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:13.216+0000 7feb84e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7feb80112b40 con 0x7feb80103320 2024-04-26T01:50:13.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:13.216+0000 7feb84e00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7feb801129c0 con 0x7feb80102960 2024-04-26T01:50:13.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:13.216+0000 7feb84e00700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7feb80112cc0 con 0x7feb80108960 2024-04-26T01:50:13.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:13.217+0000 7feb7ea00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb80103320 0x7feb80109df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:13.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:13.217+0000 7feb7f400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feb80102960 0x7feb801098b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:13.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:13.217+0000 7feb7ea00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb80103320 0x7feb80109df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:50750/0 (socket says 172.21.15.151:50750) 2024-04-26T01:50:13.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:13.217+0000 7feb7ea00700 1 -- 172.21.15.151:0/3552869942 learned_addr learned my addr 172.21.15.151:0/3552869942 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:50:13.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:13.217+0000 7feb7fe00700 1 --2- 172.21.15.151:0/3552869942 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb80108960 0x7feb8010a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:13.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:13.217+0000 7feb7f400700 1 -- 172.21.15.151:0/3552869942 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb80108960 msgr2=0x7feb8010a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:13.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:13.217+0000 7feb7f400700 1 --2- 172.21.15.151:0/3552869942 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb80108960 0x7feb8010a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:13.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:13.217+0000 7feb7f400700 1 -- 172.21.15.151:0/3552869942 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb80103320 msgr2=0x7feb80109df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:13.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:13.217+0000 7feb7f400700 1 --2- 172.21.15.151:0/3552869942 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb80103320 0x7feb80109df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:13.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:13.217+0000 7feb7f400700 1 -- 172.21.15.151:0/3552869942 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb801ab850 con 0x7feb80102960 2024-04-26T01:50:13.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:13.217+0000 7feb7f400700 1 --2- 172.21.15.151:0/3552869942 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feb80102960 0x7feb801098b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7feb7000ea20 tx=0x7feb7000ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:13.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:13.218+0000 7feb77e00700 1 -- 172.21.15.151:0/3552869942 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb70004220 con 0x7feb80102960 2024-04-26T01:50:13.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:13.218+0000 7feb77e00700 1 -- 172.21.15.151:0/3552869942 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb7000ce20 con 0x7feb80102960 2024-04-26T01:50:13.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:13.218+0000 7feb84e00700 1 -- 172.21.15.151:0/3552869942 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb801abb40 con 0x7feb80102960 2024-04-26T01:50:13.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:13.218+0000 7feb77e00700 1 -- 172.21.15.151:0/3552869942 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb70003cf0 con 0x7feb80102960 2024-04-26T01:50:13.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:13.218+0000 7feb84e00700 1 -- 172.21.15.151:0/3552869942 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb801ac080 con 0x7feb80102960 2024-04-26T01:50:13.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:13.219+0000 7feb84e00700 1 -- 172.21.15.151:0/3552869942 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb801040a0 con 0x7feb80102960 2024-04-26T01:50:13.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:13.221+0000 7feb77e00700 1 -- 172.21.15.151:0/3552869942 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7feb7000c970 con 0x7feb80102960 2024-04-26T01:50:13.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:13.222+0000 7feb77e00700 1 --2- 172.21.15.151:0/3552869942 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7feb640a3cd0 0x7feb640a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:13.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:13.222+0000 7feb77e00700 1 -- 172.21.15.151:0/3552869942 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7979+0+0 (secure 0 0 0) 0x7feb700d0630 con 0x7feb80102960 2024-04-26T01:50:13.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:13.222+0000 7feb7ea00700 1 --2- 172.21.15.151:0/3552869942 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7feb640a3cd0 0x7feb640a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:13.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:13.223+0000 7feb7ea00700 1 --2- 172.21.15.151:0/3552869942 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7feb640a3cd0 0x7feb640a6190 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7feb8010b070 tx=0x7feb6c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:13.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:13.227+0000 7feb77e00700 1 -- 172.21.15.151:0/3552869942 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feb7009cf60 con 0x7feb80102960 2024-04-26T01:50:13.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:12 smithi172 ceph-mon[32785]: pgmap v343: 97 pgs: 97 active+clean; 588 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.3 KiB/s wr, 12 op/s 2024-04-26T01:50:13.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:12 smithi172 ceph-mon[32785]: mgrmap e17: x(active, since 8m), standbys: y, z 2024-04-26T01:50:13.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:13.490+0000 7feb84e00700 1 -- 172.21.15.151:0/3552869942 --> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7feb80107c20 con 0x7feb640a3cd0 2024-04-26T01:50:13.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:13.534+0000 7feb77e00700 1 -- 172.21.15.151:0/3552869942 <== mgr.14152 v2:172.21.15.151:6800/2251256505 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7feb80107c20 con 0x7feb640a3cd0 2024-04-26T01:50:13.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:13.537+0000 7feb84e00700 1 -- 172.21.15.151:0/3552869942 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7feb640a3cd0 msgr2=0x7feb640a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:13.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:13.537+0000 7feb84e00700 1 --2- 172.21.15.151:0/3552869942 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7feb640a3cd0 0x7feb640a6190 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7feb8010b070 tx=0x7feb6c009260 comp rx=0 tx=0).stop 2024-04-26T01:50:13.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:13.537+0000 7feb84e00700 1 -- 172.21.15.151:0/3552869942 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feb80102960 msgr2=0x7feb801098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:13.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:13.537+0000 7feb84e00700 1 --2- 172.21.15.151:0/3552869942 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feb80102960 0x7feb801098b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7feb7000ea20 tx=0x7feb7000ed30 comp rx=0 tx=0).stop 2024-04-26T01:50:13.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:13.538+0000 7feb84e00700 1 -- 172.21.15.151:0/3552869942 shutdown_connections 2024-04-26T01:50:13.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:13.538+0000 7feb84e00700 1 --2- 172.21.15.151:0/3552869942 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb80108960 0x7feb8010a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:13.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:13.538+0000 7feb84e00700 1 --2- 172.21.15.151:0/3552869942 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feb80102960 0x7feb801098b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:13.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:13.538+0000 7feb84e00700 1 --2- 172.21.15.151:0/3552869942 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7feb640a3cd0 0x7feb640a6190 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:13.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:13.538+0000 7feb84e00700 1 --2- 172.21.15.151:0/3552869942 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb80103320 0x7feb80109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:13.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:13.538+0000 7feb84e00700 1 -- 172.21.15.151:0/3552869942 >> 172.21.15.151:0/3552869942 conn(0x7feb800fe450 msgr2=0x7feb8010f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:13.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:13.538+0000 7feb84e00700 1 -- 172.21.15.151:0/3552869942 shutdown_connections 2024-04-26T01:50:13.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:13.538+0000 7feb84e00700 1 -- 172.21.15.151:0/3552869942 wait complete. 2024-04-26T01:50:14.431 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-26T01:50:14.880 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:50:15.205 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:14 smithi151 ceph-mon[28057]: pgmap v344: 97 pgs: 97 active+clean; 588 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-04-26T01:50:15.205 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:14 smithi151 ceph-mon[28057]: from='client.24593 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:50:15.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:14 smithi172 ceph-mon[32785]: pgmap v344: 97 pgs: 97 active+clean; 588 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-04-26T01:50:15.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:14 smithi172 ceph-mon[32785]: from='client.24593 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:50:15.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:14 smithi165 ceph-mon[32648]: pgmap v344: 97 pgs: 97 active+clean; 588 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-04-26T01:50:15.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:14 smithi165 ceph-mon[32648]: from='client.24593 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:50:17.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:17.177+0000 7fa97e200700 1 -- 172.21.15.151:0/2312457805 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa978104db0 msgr2=0x7fa978105190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:17.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:17.177+0000 7fa97e200700 1 --2- 172.21.15.151:0/2312457805 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa978104db0 0x7fa978105190 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa96c0099c0 tx=0x7fa96c009cd0 comp rx=0 tx=0).stop 2024-04-26T01:50:17.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:17.177+0000 7fa97e200700 1 -- 172.21.15.151:0/2312457805 shutdown_connections 2024-04-26T01:50:17.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:17.177+0000 7fa97e200700 1 --2- 172.21.15.151:0/2312457805 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa978109e90 0x7fa978111a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:17.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:17.177+0000 7fa97e200700 1 --2- 172.21.15.151:0/2312457805 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa978104db0 0x7fa978105190 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:17.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:17.177+0000 7fa97e200700 1 --2- 172.21.15.151:0/2312457805 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa978105760 0x7fa9781097b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:17.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:17.177+0000 7fa97e200700 1 -- 172.21.15.151:0/2312457805 >> 172.21.15.151:0/2312457805 conn(0x7fa978100620 msgr2=0x7fa978102a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:17.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:17.177+0000 7fa97e200700 1 -- 172.21.15.151:0/2312457805 shutdown_connections 2024-04-26T01:50:17.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:17.177+0000 7fa97e200700 1 -- 172.21.15.151:0/2312457805 wait complete. 2024-04-26T01:50:17.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:17.178+0000 7fa97e200700 1 Processor -- start 2024-04-26T01:50:17.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:17.179+0000 7fa97e200700 1 -- start start 2024-04-26T01:50:17.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:17.179+0000 7fa97e200700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa978104db0 0x7fa97819b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:17.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:17.179+0000 7fa97e200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa978105760 0x7fa97819b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:17.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:17.179+0000 7fa97e200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa978109e90 0x7fa97819fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:17.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:17.179+0000 7fa97e200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa978114d30 con 0x7fa978105760 2024-04-26T01:50:17.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:17.179+0000 7fa97e200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa978114bb0 con 0x7fa978109e90 2024-04-26T01:50:17.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:17.179+0000 7fa97e200700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa978114eb0 con 0x7fa978104db0 2024-04-26T01:50:17.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:17.179+0000 7fa977e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa978105760 0x7fa97819b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:17.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:17.179+0000 7fa977e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa978105760 0x7fa97819b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:50776/0 (socket says 172.21.15.151:50776) 2024-04-26T01:50:17.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:17.179+0000 7fa977e00700 1 -- 172.21.15.151:0/2787935591 learned_addr learned my addr 172.21.15.151:0/2787935591 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:50:17.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:17.179+0000 7fa97d800700 1 --2- 172.21.15.151:0/2787935591 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa978109e90 0x7fa97819fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:17.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:17.180+0000 7fa97ce00700 1 --2- 172.21.15.151:0/2787935591 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa978104db0 0x7fa97819b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:17.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:17.180+0000 7fa977e00700 1 -- 172.21.15.151:0/2787935591 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa978104db0 msgr2=0x7fa97819b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:17.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:17.180+0000 7fa977e00700 1 --2- 172.21.15.151:0/2787935591 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa978104db0 0x7fa97819b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:17.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:17.180+0000 7fa977e00700 1 -- 172.21.15.151:0/2787935591 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa978109e90 msgr2=0x7fa97819fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:17.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:17.180+0000 7fa977e00700 1 --2- 172.21.15.151:0/2787935591 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa978109e90 0x7fa97819fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:17.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:17.180+0000 7fa977e00700 1 -- 172.21.15.151:0/2787935591 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9781a0250 con 0x7fa978105760 2024-04-26T01:50:17.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:17.180+0000 7fa977e00700 1 --2- 172.21.15.151:0/2787935591 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa978105760 0x7fa97819b780 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa96400e970 tx=0x7fa96400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:17.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:17.181+0000 7fa975600700 1 -- 172.21.15.151:0/2787935591 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa964004a30 con 0x7fa978105760 2024-04-26T01:50:17.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:17.181+0000 7fa975600700 1 -- 172.21.15.151:0/2787935591 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa96400cc70 con 0x7fa978105760 2024-04-26T01:50:17.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:17.181+0000 7fa97e200700 1 -- 172.21.15.151:0/2787935591 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9781a0540 con 0x7fa978105760 2024-04-26T01:50:17.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:17.181+0000 7fa975600700 1 -- 172.21.15.151:0/2787935591 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa964005580 con 0x7fa978105760 2024-04-26T01:50:17.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:17.181+0000 7fa97e200700 1 -- 172.21.15.151:0/2787935591 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9781a7dc0 con 0x7fa978105760 2024-04-26T01:50:17.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:17.183+0000 7fa97e200700 1 -- 172.21.15.151:0/2787935591 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa978069730 con 0x7fa978105760 2024-04-26T01:50:17.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:17.184+0000 7fa975600700 1 -- 172.21.15.151:0/2787935591 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fa96400c7c0 con 0x7fa978105760 2024-04-26T01:50:17.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:17.186+0000 7fa975600700 1 --2- 172.21.15.151:0/2787935591 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fa9600a3d50 0x7fa9600a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:17.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:17.186+0000 7fa975600700 1 -- 172.21.15.151:0/2787935591 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7979+0+0 (secure 0 0 0) 0x7fa9640d0600 con 0x7fa978105760 2024-04-26T01:50:17.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:17.186+0000 7fa97ce00700 1 --2- 172.21.15.151:0/2787935591 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fa9600a3d50 0x7fa9600a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:17.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:17.187+0000 7fa97ce00700 1 --2- 172.21.15.151:0/2787935591 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fa9600a3d50 0x7fa9600a6210 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa96c003760 tx=0x7fa96c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:17.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:17.190+0000 7fa975600700 1 -- 172.21.15.151:0/2787935591 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa96409cf30 con 0x7fa978105760 2024-04-26T01:50:17.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:16 smithi172 ceph-mon[32785]: pgmap v345: 97 pgs: 97 active+clean; 589 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-26T01:50:17.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:16 smithi165 ceph-mon[32648]: pgmap v345: 97 pgs: 97 active+clean; 589 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-26T01:50:17.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:16 smithi151 ceph-mon[28057]: pgmap v345: 97 pgs: 97 active+clean; 589 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-26T01:50:17.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:17.453+0000 7fa97e200700 1 -- 172.21.15.151:0/2787935591 --> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fa97819c5a0 con 0x7fa9600a3d50 2024-04-26T01:50:17.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:17.469+0000 7fa975600700 1 -- 172.21.15.151:0/2787935591 <== mgr.14152 v2:172.21.15.151:6800/2251256505 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fa97819c5a0 con 0x7fa9600a3d50 2024-04-26T01:50:17.468 INFO:teuthology.orchestra.run.smithi151.stdout:/volumes/qa/sv_0/e048b885-5839-4072-8eaa-d8009c6dd2a7 2024-04-26T01:50:17.471 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:17.472+0000 7fa97e200700 1 -- 172.21.15.151:0/2787935591 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fa9600a3d50 msgr2=0x7fa9600a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:17.471 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:17.472+0000 7fa97e200700 1 --2- 172.21.15.151:0/2787935591 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fa9600a3d50 0x7fa9600a6210 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa96c003760 tx=0x7fa96c005fb0 comp rx=0 tx=0).stop 2024-04-26T01:50:17.471 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:17.472+0000 7fa97e200700 1 -- 172.21.15.151:0/2787935591 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa978105760 msgr2=0x7fa97819b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:17.471 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:17.472+0000 7fa97e200700 1 --2- 172.21.15.151:0/2787935591 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa978105760 0x7fa97819b780 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa96400e970 tx=0x7fa96400ec80 comp rx=0 tx=0).stop 2024-04-26T01:50:17.471 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:17.472+0000 7fa97e200700 1 -- 172.21.15.151:0/2787935591 shutdown_connections 2024-04-26T01:50:17.471 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:17.472+0000 7fa97e200700 1 --2- 172.21.15.151:0/2787935591 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa978109e90 0x7fa97819fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:17.471 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:17.472+0000 7fa97e200700 1 --2- 172.21.15.151:0/2787935591 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa978104db0 0x7fa97819b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:17.471 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:17.472+0000 7fa97e200700 1 --2- 172.21.15.151:0/2787935591 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fa9600a3d50 0x7fa9600a6210 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:17.471 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:17.473+0000 7fa97e200700 1 --2- 172.21.15.151:0/2787935591 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa978105760 0x7fa97819b780 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:17.471 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:17.473+0000 7fa97e200700 1 -- 172.21.15.151:0/2787935591 >> 172.21.15.151:0/2787935591 conn(0x7fa978100620 msgr2=0x7fa978109020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:17.471 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:17.473+0000 7fa97e200700 1 -- 172.21.15.151:0/2787935591 shutdown_connections 2024-04-26T01:50:17.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:17.473+0000 7fa97e200700 1 -- 172.21.15.151:0/2787935591 wait complete. 2024-04-26T01:50:18.384 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-26T01:50:18.833 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:50:19.211 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:18 smithi151 ceph-mon[28057]: pgmap v346: 97 pgs: 97 active+clean; 603 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.3 KiB/s wr, 8 op/s 2024-04-26T01:50:19.211 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:18 smithi151 ceph-mon[28057]: from='client.14502 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:50:19.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:18 smithi172 ceph-mon[32785]: pgmap v346: 97 pgs: 97 active+clean; 603 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.3 KiB/s wr, 8 op/s 2024-04-26T01:50:19.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:18 smithi172 ceph-mon[32785]: from='client.14502 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:50:19.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:18 smithi165 ceph-mon[32648]: pgmap v346: 97 pgs: 97 active+clean; 603 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.3 KiB/s wr, 8 op/s 2024-04-26T01:50:19.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:18 smithi165 ceph-mon[32648]: from='client.14502 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:50:21.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:21.028+0000 7f8a36800700 1 -- 172.21.15.151:0/3660131074 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a30102a90 msgr2=0x7f8a30102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:21.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:21.028+0000 7f8a36800700 1 --2- 172.21.15.151:0/3660131074 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a30102a90 0x7f8a30102f10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8a18009990 tx=0x7f8a18009ca0 comp rx=0 tx=0).stop 2024-04-26T01:50:21.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:21.028+0000 7f8a36800700 1 -- 172.21.15.151:0/3660131074 shutdown_connections 2024-04-26T01:50:21.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:21.028+0000 7f8a36800700 1 --2- 172.21.15.151:0/3660131074 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a30103450 0x7f8a3010f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:21.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:21.028+0000 7f8a36800700 1 --2- 172.21.15.151:0/3660131074 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a30108a50 0x7f8a30108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:21.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:21.028+0000 7f8a36800700 1 --2- 172.21.15.151:0/3660131074 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a30102a90 0x7f8a30102f10 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:21.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:21.028+0000 7f8a36800700 1 -- 172.21.15.151:0/3660131074 >> 172.21.15.151:0/3660131074 conn(0x7f8a30079ba0 msgr2=0x7f8a30079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:21.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:21.029+0000 7f8a36800700 1 -- 172.21.15.151:0/3660131074 shutdown_connections 2024-04-26T01:50:21.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:21.029+0000 7f8a36800700 1 -- 172.21.15.151:0/3660131074 wait complete. 2024-04-26T01:50:21.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:21.030+0000 7f8a36800700 1 Processor -- start 2024-04-26T01:50:21.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:21.030+0000 7f8a36800700 1 -- start start 2024-04-26T01:50:21.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:21.030+0000 7f8a36800700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a30102a90 0x7f8a30199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:21.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:21.030+0000 7f8a36800700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a30103450 0x7f8a301996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:21.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:21.030+0000 7f8a36800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a30108a50 0x7f8a3019da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:21.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:21.030+0000 7f8a36800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8a30112c50 con 0x7f8a30108a50 2024-04-26T01:50:21.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:21.030+0000 7f8a36800700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8a30112ad0 con 0x7f8a30102a90 2024-04-26T01:50:21.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:21.030+0000 7f8a36800700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8a30112dd0 con 0x7f8a30103450 2024-04-26T01:50:21.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:21.031+0000 7f8a2f400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a30102a90 0x7f8a30199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:21.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:21.031+0000 7f8a2fe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a30108a50 0x7f8a3019da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:21.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:21.031+0000 7f8a27e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a30103450 0x7f8a301996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:21.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:21.031+0000 7f8a2f400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a30102a90 0x7f8a30199160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:39060/0 (socket says 172.21.15.151:39060) 2024-04-26T01:50:21.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:21.031+0000 7f8a2f400700 1 -- 172.21.15.151:0/2960433218 learned_addr learned my addr 172.21.15.151:0/2960433218 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:50:21.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:21.031+0000 7f8a27e00700 1 -- 172.21.15.151:0/2960433218 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a30102a90 msgr2=0x7f8a30199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:21.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:21.031+0000 7f8a27e00700 1 --2- 172.21.15.151:0/2960433218 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a30102a90 0x7f8a30199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:21.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:21.031+0000 7f8a27e00700 1 -- 172.21.15.151:0/2960433218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a30108a50 msgr2=0x7f8a3019da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:21.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:21.031+0000 7f8a27e00700 1 --2- 172.21.15.151:0/2960433218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a30108a50 0x7f8a3019da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:21.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:21.031+0000 7f8a27e00700 1 -- 172.21.15.151:0/2960433218 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a3019e170 con 0x7f8a30103450 2024-04-26T01:50:21.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:21.031+0000 7f8a2f400700 1 --2- 172.21.15.151:0/2960433218 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a30102a90 0x7f8a30199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:50:21.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:21.031+0000 7f8a27e00700 1 --2- 172.21.15.151:0/2960433218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a30103450 0x7f8a301996a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8a180098e0 tx=0x7f8a180057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:21.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:21.032+0000 7f8a2cc00700 1 -- 172.21.15.151:0/2960433218 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a18027070 con 0x7f8a30103450 2024-04-26T01:50:21.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:21.032+0000 7f8a36800700 1 -- 172.21.15.151:0/2960433218 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a3019e400 con 0x7f8a30103450 2024-04-26T01:50:21.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:21.032+0000 7f8a2cc00700 1 -- 172.21.15.151:0/2960433218 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a18022470 con 0x7f8a30103450 2024-04-26T01:50:21.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:21.032+0000 7f8a2cc00700 1 -- 172.21.15.151:0/2960433218 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a18017650 con 0x7f8a30103450 2024-04-26T01:50:21.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:21.032+0000 7f8a36800700 1 -- 172.21.15.151:0/2960433218 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a301a5ce0 con 0x7f8a30103450 2024-04-26T01:50:21.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:21.033+0000 7f8a36800700 1 -- 172.21.15.151:0/2960433218 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a30069730 con 0x7f8a30103450 2024-04-26T01:50:21.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:21.035+0000 7f8a2cc00700 1 -- 172.21.15.151:0/2960433218 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f8a1802a070 con 0x7f8a30103450 2024-04-26T01:50:21.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:21.036+0000 7f8a2cc00700 1 --2- 172.21.15.151:0/2960433218 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f8a140a3cd0 0x7f8a140a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:21.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:21.036+0000 7f8a2cc00700 1 -- 172.21.15.151:0/2960433218 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7979+0+0 (secure 0 0 0) 0x7f8a180d1180 con 0x7f8a30103450 2024-04-26T01:50:21.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:21.036+0000 7f8a2f400700 1 --2- 172.21.15.151:0/2960433218 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f8a140a3cd0 0x7f8a140a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:21.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:21.037+0000 7f8a2f400700 1 --2- 172.21.15.151:0/2960433218 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f8a140a3cd0 0x7f8a140a6190 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8a20011550 tx=0x7f8a20009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:21.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:21.041+0000 7f8a2cc00700 1 -- 172.21.15.151:0/2960433218 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8a1805eff0 con 0x7f8a30103450 2024-04-26T01:50:21.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:20 smithi151 ceph-mon[28057]: pgmap v347: 97 pgs: 97 active+clean; 603 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.3 KiB/s wr, 6 op/s 2024-04-26T01:50:21.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:20 smithi172 ceph-mon[32785]: pgmap v347: 97 pgs: 97 active+clean; 603 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.3 KiB/s wr, 6 op/s 2024-04-26T01:50:21.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:21.304+0000 7f8a36800700 1 -- 172.21.15.151:0/2960433218 --> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f8a30062e80 con 0x7f8a140a3cd0 2024-04-26T01:50:21.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:21.346+0000 7f8a2cc00700 1 -- 172.21.15.151:0/2960433218 <== mgr.14152 v2:172.21.15.151:6800/2251256505 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8a30062e80 con 0x7f8a140a3cd0 2024-04-26T01:50:21.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:20 smithi165 ceph-mon[32648]: pgmap v347: 97 pgs: 97 active+clean; 603 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.3 KiB/s wr, 6 op/s 2024-04-26T01:50:21.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:21.349+0000 7f8a36800700 1 -- 172.21.15.151:0/2960433218 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f8a140a3cd0 msgr2=0x7f8a140a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:21.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:21.349+0000 7f8a36800700 1 --2- 172.21.15.151:0/2960433218 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f8a140a3cd0 0x7f8a140a6190 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8a20011550 tx=0x7f8a20009260 comp rx=0 tx=0).stop 2024-04-26T01:50:21.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:21.349+0000 7f8a36800700 1 -- 172.21.15.151:0/2960433218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a30103450 msgr2=0x7f8a301996a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:21.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:21.349+0000 7f8a36800700 1 --2- 172.21.15.151:0/2960433218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a30103450 0x7f8a301996a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8a180098e0 tx=0x7f8a180057c0 comp rx=0 tx=0).stop 2024-04-26T01:50:21.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:21.349+0000 7f8a36800700 1 -- 172.21.15.151:0/2960433218 shutdown_connections 2024-04-26T01:50:21.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:21.349+0000 7f8a36800700 1 --2- 172.21.15.151:0/2960433218 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f8a140a3cd0 0x7f8a140a6190 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:21.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:21.349+0000 7f8a36800700 1 --2- 172.21.15.151:0/2960433218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a30108a50 0x7f8a3019da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:21.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:21.349+0000 7f8a36800700 1 --2- 172.21.15.151:0/2960433218 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a30102a90 0x7f8a30199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:21.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:21.349+0000 7f8a36800700 1 --2- 172.21.15.151:0/2960433218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a30103450 0x7f8a301996a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:21.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:21.349+0000 7f8a36800700 1 -- 172.21.15.151:0/2960433218 >> 172.21.15.151:0/2960433218 conn(0x7f8a30079ba0 msgr2=0x7f8a30101930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:21.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:21.349+0000 7f8a36800700 1 -- 172.21.15.151:0/2960433218 shutdown_connections 2024-04-26T01:50:21.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:21.349+0000 7f8a36800700 1 -- 172.21.15.151:0/2960433218 wait complete. 2024-04-26T01:50:22.095 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:21 smithi151 ceph-mon[28057]: from='client.24622 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:50:22.169 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-26T01:50:22.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:21 smithi172 ceph-mon[32785]: from='client.24622 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:50:22.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:21 smithi165 ceph-mon[32648]: from='client.24622 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:50:22.632 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:50:23.053 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:22 smithi151 ceph-mon[28057]: pgmap v348: 97 pgs: 97 active+clean; 608 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.0 KiB/s wr, 8 op/s 2024-04-26T01:50:23.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:22 smithi172 ceph-mon[32785]: pgmap v348: 97 pgs: 97 active+clean; 608 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.0 KiB/s wr, 8 op/s 2024-04-26T01:50:23.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:22 smithi165 ceph-mon[32648]: pgmap v348: 97 pgs: 97 active+clean; 608 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.0 KiB/s wr, 8 op/s 2024-04-26T01:50:24.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:24.878+0000 7fb72ee00700 1 -- 172.21.15.151:0/2597639330 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb728108a50 msgr2=0x7fb728108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:24.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:24.878+0000 7fb72ee00700 1 --2- 172.21.15.151:0/2597639330 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb728108a50 0x7fb728108e30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb718009990 tx=0x7fb718009ca0 comp rx=0 tx=0).stop 2024-04-26T01:50:24.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:24.878+0000 7fb72ee00700 1 -- 172.21.15.151:0/2597639330 shutdown_connections 2024-04-26T01:50:24.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:24.878+0000 7fb72ee00700 1 --2- 172.21.15.151:0/2597639330 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb728103450 0x7fb72810f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:24.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:24.878+0000 7fb72ee00700 1 --2- 172.21.15.151:0/2597639330 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb728108a50 0x7fb728108e30 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:24.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:24.878+0000 7fb72ee00700 1 --2- 172.21.15.151:0/2597639330 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb728102a90 0x7fb728102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:24.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:24.878+0000 7fb72ee00700 1 -- 172.21.15.151:0/2597639330 >> 172.21.15.151:0/2597639330 conn(0x7fb728079ba0 msgr2=0x7fb728079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:24.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:24.878+0000 7fb72ee00700 1 -- 172.21.15.151:0/2597639330 shutdown_connections 2024-04-26T01:50:24.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:24.878+0000 7fb72ee00700 1 -- 172.21.15.151:0/2597639330 wait complete. 2024-04-26T01:50:24.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:24.879+0000 7fb72ee00700 1 Processor -- start 2024-04-26T01:50:24.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:24.879+0000 7fb72ee00700 1 -- start start 2024-04-26T01:50:24.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:24.879+0000 7fb72ee00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb728102a90 0x7fb72819b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:24.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:24.879+0000 7fb72ee00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb728103450 0x7fb72819b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:24.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:24.880+0000 7fb72ee00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb728108a50 0x7fb72819fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:24.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:24.880+0000 7fb72ee00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb728112c30 con 0x7fb728102a90 2024-04-26T01:50:24.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:24.880+0000 7fb72ee00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb728112ab0 con 0x7fb728103450 2024-04-26T01:50:24.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:24.880+0000 7fb72ee00700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb728112db0 con 0x7fb728108a50 2024-04-26T01:50:24.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:24.880+0000 7fb71f400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb728103450 0x7fb72819b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:24.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:24.880+0000 7fb727e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb728102a90 0x7fb72819b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:24.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:24.880+0000 7fb72ce00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb728108a50 0x7fb72819fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:24.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:24.880+0000 7fb71f400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb728103450 0x7fb72819b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:39074/0 (socket says 172.21.15.151:39074) 2024-04-26T01:50:24.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:24.880+0000 7fb71f400700 1 -- 172.21.15.151:0/1386822665 learned_addr learned my addr 172.21.15.151:0/1386822665 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:50:24.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:24.880+0000 7fb71f400700 1 -- 172.21.15.151:0/1386822665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb728108a50 msgr2=0x7fb72819fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:24.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:24.880+0000 7fb71f400700 1 --2- 172.21.15.151:0/1386822665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb728108a50 0x7fb72819fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:24.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:24.880+0000 7fb71f400700 1 -- 172.21.15.151:0/1386822665 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb728102a90 msgr2=0x7fb72819b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:24.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:24.880+0000 7fb71f400700 1 --2- 172.21.15.151:0/1386822665 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb728102a90 0x7fb72819b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:24.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:24.880+0000 7fb71f400700 1 -- 172.21.15.151:0/1386822665 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7281a03c0 con 0x7fb728103450 2024-04-26T01:50:24.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:24.880+0000 7fb727e00700 1 --2- 172.21.15.151:0/1386822665 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb728102a90 0x7fb72819b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:50:24.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:24.880+0000 7fb72ce00700 1 --2- 172.21.15.151:0/1386822665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb728108a50 0x7fb72819fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:50:24.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:24.881+0000 7fb71f400700 1 --2- 172.21.15.151:0/1386822665 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb728103450 0x7fb72819b8f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb71000e9a0 tx=0x7fb71000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:24.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:24.881+0000 7fb725600700 1 -- 172.21.15.151:0/1386822665 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71000ca10 con 0x7fb728103450 2024-04-26T01:50:24.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:24.881+0000 7fb72ee00700 1 -- 172.21.15.151:0/1386822665 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7281a06b0 con 0x7fb728103450 2024-04-26T01:50:24.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:24.881+0000 7fb725600700 1 -- 172.21.15.151:0/1386822665 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb71000cbb0 con 0x7fb728103450 2024-04-26T01:50:24.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:24.881+0000 7fb725600700 1 -- 172.21.15.151:0/1386822665 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb710018680 con 0x7fb728103450 2024-04-26T01:50:24.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:24.881+0000 7fb72ee00700 1 -- 172.21.15.151:0/1386822665 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7281a7f30 con 0x7fb728103450 2024-04-26T01:50:24.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:24.882+0000 7fb72ee00700 1 -- 172.21.15.151:0/1386822665 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb728069730 con 0x7fb728103450 2024-04-26T01:50:24.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:24.884+0000 7fb725600700 1 -- 172.21.15.151:0/1386822665 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fb710010430 con 0x7fb728103450 2024-04-26T01:50:24.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:24.885+0000 7fb725600700 1 --2- 172.21.15.151:0/1386822665 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fb70c0a3cd0 0x7fb70c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:24.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:24.886+0000 7fb725600700 1 -- 172.21.15.151:0/1386822665 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7979+0+0 (secure 0 0 0) 0x7fb7100d0710 con 0x7fb728103450 2024-04-26T01:50:24.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:24.886+0000 7fb727e00700 1 --2- 172.21.15.151:0/1386822665 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fb70c0a3cd0 0x7fb70c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:24.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:24.887+0000 7fb727e00700 1 --2- 172.21.15.151:0/1386822665 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fb70c0a3cd0 0x7fb70c0a6190 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb718006010 tx=0x7fb71800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:24.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:24.888+0000 7fb725600700 1 -- 172.21.15.151:0/1386822665 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb71009d040 con 0x7fb728103450 2024-04-26T01:50:25.131 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:25.133+0000 7fb72ee00700 1 -- 172.21.15.151:0/1386822665 --> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fb728062e80 con 0x7fb70c0a3cd0 2024-04-26T01:50:25.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:25.146+0000 7fb725600700 1 -- 172.21.15.151:0/1386822665 <== mgr.14152 v2:172.21.15.151:6800/2251256505 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fb728062e80 con 0x7fb70c0a3cd0 2024-04-26T01:50:25.145 INFO:teuthology.orchestra.run.smithi151.stdout:/volumes/qa/sv_1/87d7cc94-2a5d-4cec-9bbf-5cb1e42ad242 2024-04-26T01:50:25.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:25.149+0000 7fb72ee00700 1 -- 172.21.15.151:0/1386822665 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fb70c0a3cd0 msgr2=0x7fb70c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:25.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:25.149+0000 7fb72ee00700 1 --2- 172.21.15.151:0/1386822665 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fb70c0a3cd0 0x7fb70c0a6190 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb718006010 tx=0x7fb71800b260 comp rx=0 tx=0).stop 2024-04-26T01:50:25.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:25.149+0000 7fb72ee00700 1 -- 172.21.15.151:0/1386822665 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb728103450 msgr2=0x7fb72819b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:25.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:25.149+0000 7fb72ee00700 1 --2- 172.21.15.151:0/1386822665 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb728103450 0x7fb72819b8f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb71000e9a0 tx=0x7fb71000ecb0 comp rx=0 tx=0).stop 2024-04-26T01:50:25.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:25.149+0000 7fb72ee00700 1 -- 172.21.15.151:0/1386822665 shutdown_connections 2024-04-26T01:50:25.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:25.149+0000 7fb72ee00700 1 --2- 172.21.15.151:0/1386822665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb728108a50 0x7fb72819fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:25.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:25.149+0000 7fb72ee00700 1 --2- 172.21.15.151:0/1386822665 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fb70c0a3cd0 0x7fb70c0a6190 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:25.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:25.149+0000 7fb72ee00700 1 --2- 172.21.15.151:0/1386822665 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb728102a90 0x7fb72819b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:25.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:25.149+0000 7fb72ee00700 1 --2- 172.21.15.151:0/1386822665 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb728103450 0x7fb72819b8f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:25.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:25.149+0000 7fb72ee00700 1 -- 172.21.15.151:0/1386822665 >> 172.21.15.151:0/1386822665 conn(0x7fb728079ba0 msgr2=0x7fb72810df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:25.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:25.149+0000 7fb72ee00700 1 -- 172.21.15.151:0/1386822665 shutdown_connections 2024-04-26T01:50:25.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:25.150+0000 7fb72ee00700 1 -- 172.21.15.151:0/1386822665 wait complete. 2024-04-26T01:50:25.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:24 smithi172 ceph-mon[32785]: pgmap v349: 97 pgs: 97 active+clean; 608 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-04-26T01:50:25.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:24 smithi165 ceph-mon[32648]: pgmap v349: 97 pgs: 97 active+clean; 608 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-04-26T01:50:25.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:24 smithi151 ceph-mon[28057]: pgmap v349: 97 pgs: 97 active+clean; 608 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-04-26T01:50:25.971 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:50:26.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:25 smithi172 ceph-mon[32785]: from='client.24611 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:50:26.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:25 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-26T01:50:26.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:25 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-04-26T01:50:26.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:25 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-04-26T01:50:26.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:25 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-04-26T01:50:26.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:25 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-04-26T01:50:26.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:25 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-04-26T01:50:26.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:25 smithi165 ceph-mon[32648]: from='client.24611 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:50:26.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:25 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-26T01:50:26.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:25 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-04-26T01:50:26.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:25 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-04-26T01:50:26.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:25 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-04-26T01:50:26.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:25 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-04-26T01:50:26.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:25 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-04-26T01:50:26.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:25 smithi151 ceph-mon[28057]: from='client.24611 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:50:26.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:25 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-26T01:50:26.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:25 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-04-26T01:50:26.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:25 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-04-26T01:50:26.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:25 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-04-26T01:50:26.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:25 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-04-26T01:50:26.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:25 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-04-26T01:50:26.407 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:50:27.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:27 smithi172 ceph-mon[32785]: pgmap v350: 97 pgs: 97 active+clean; 608 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-04-26T01:50:27.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:27 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-26T01:50:27.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:27 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-04-26T01:50:27.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:27 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-04-26T01:50:27.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:27 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-04-26T01:50:27.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:27 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-04-26T01:50:27.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:27 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-04-26T01:50:27.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:27 smithi172 ceph-mon[32785]: osdmap e91: 12 total, 12 up, 12 in 2024-04-26T01:50:27.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:27 smithi165 ceph-mon[32648]: pgmap v350: 97 pgs: 97 active+clean; 608 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-04-26T01:50:27.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:27 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-26T01:50:27.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:27 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-04-26T01:50:27.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:27 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-04-26T01:50:27.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:27 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-04-26T01:50:27.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:27 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-04-26T01:50:27.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:27 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-04-26T01:50:27.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:27 smithi165 ceph-mon[32648]: osdmap e91: 12 total, 12 up, 12 in 2024-04-26T01:50:27.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:27 smithi151 ceph-mon[28057]: pgmap v350: 97 pgs: 97 active+clean; 608 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-04-26T01:50:27.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:27 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-26T01:50:27.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:27 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-04-26T01:50:27.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:27 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-04-26T01:50:27.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:27 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-04-26T01:50:27.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:27 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-04-26T01:50:27.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:27 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-04-26T01:50:27.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:27 smithi151 ceph-mon[28057]: osdmap e91: 12 total, 12 up, 12 in 2024-04-26T01:50:28.091 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:28 smithi151 ceph-mon[28057]: osdmap e92: 12 total, 12 up, 12 in 2024-04-26T01:50:28.091 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:28 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:50:28.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:28 smithi172 ceph-mon[32785]: osdmap e92: 12 total, 12 up, 12 in 2024-04-26T01:50:28.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:28 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:50:28.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:28 smithi165 ceph-mon[32648]: osdmap e92: 12 total, 12 up, 12 in 2024-04-26T01:50:28.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:28 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:50:28.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:28.628+0000 7f681c200700 1 -- 172.21.15.151:0/3748299302 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6814102a90 msgr2=0x7f6814102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:28.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:28.628+0000 7f681c200700 1 --2- 172.21.15.151:0/3748299302 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6814102a90 0x7f6814102f10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6800009990 tx=0x7f6800009ca0 comp rx=0 tx=0).stop 2024-04-26T01:50:28.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:28.628+0000 7f681c200700 1 -- 172.21.15.151:0/3748299302 shutdown_connections 2024-04-26T01:50:28.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:28.629+0000 7f681c200700 1 --2- 172.21.15.151:0/3748299302 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6814103450 0x7f681410f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:28.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:28.629+0000 7f681c200700 1 --2- 172.21.15.151:0/3748299302 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6814108a50 0x7f6814108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:28.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:28.629+0000 7f681c200700 1 --2- 172.21.15.151:0/3748299302 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6814102a90 0x7f6814102f10 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:28.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:28.629+0000 7f681c200700 1 -- 172.21.15.151:0/3748299302 >> 172.21.15.151:0/3748299302 conn(0x7f6814079ba0 msgr2=0x7f6814079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:28.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:28.629+0000 7f681c200700 1 -- 172.21.15.151:0/3748299302 shutdown_connections 2024-04-26T01:50:28.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:28.629+0000 7f681c200700 1 -- 172.21.15.151:0/3748299302 wait complete. 2024-04-26T01:50:28.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:28.630+0000 7f681c200700 1 Processor -- start 2024-04-26T01:50:28.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:28.630+0000 7f681c200700 1 -- start start 2024-04-26T01:50:28.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:28.630+0000 7f681c200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6814102a90 0x7f681419b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:28.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:28.630+0000 7f681c200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6814103450 0x7f681419b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:28.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:28.630+0000 7f681c200700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6814108a50 0x7f681419fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:28.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:28.630+0000 7f681c200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6814112c00 con 0x7f6814102a90 2024-04-26T01:50:28.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:28.630+0000 7f681c200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6814112a80 con 0x7f6814103450 2024-04-26T01:50:28.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:28.630+0000 7f681c200700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6814112d80 con 0x7f6814108a50 2024-04-26T01:50:28.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:28.631+0000 7f6818e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6814103450 0x7f681419b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:28.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:28.631+0000 7f6818e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6814103450 0x7f681419b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:39100/0 (socket says 172.21.15.151:39100) 2024-04-26T01:50:28.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:28.631+0000 7f6818e00700 1 -- 172.21.15.151:0/1034852826 learned_addr learned my addr 172.21.15.151:0/1034852826 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:50:28.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:28.631+0000 7f681a200700 1 --2- 172.21.15.151:0/1034852826 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6814108a50 0x7f681419fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:28.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:28.631+0000 7f6819800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6814102a90 0x7f681419b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:28.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:28.631+0000 7f6818e00700 1 -- 172.21.15.151:0/1034852826 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6814108a50 msgr2=0x7f681419fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:28.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:28.631+0000 7f6818e00700 1 --2- 172.21.15.151:0/1034852826 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6814108a50 0x7f681419fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:28.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:28.631+0000 7f6818e00700 1 -- 172.21.15.151:0/1034852826 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6814102a90 msgr2=0x7f681419b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:28.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:28.631+0000 7f6818e00700 1 --2- 172.21.15.151:0/1034852826 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6814102a90 0x7f681419b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:28.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:28.631+0000 7f6818e00700 1 -- 172.21.15.151:0/1034852826 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68141a02d0 con 0x7f6814103450 2024-04-26T01:50:28.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:28.631+0000 7f6818e00700 1 --2- 172.21.15.151:0/1034852826 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6814103450 0x7f681419b850 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f68000057f0 tx=0x7f680000bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:28.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:28.632+0000 7f6812000700 1 -- 172.21.15.151:0/1034852826 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6800026070 con 0x7f6814103450 2024-04-26T01:50:28.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:28.632+0000 7f6812000700 1 -- 172.21.15.151:0/1034852826 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6800021410 con 0x7f6814103450 2024-04-26T01:50:28.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:28.632+0000 7f6812000700 1 -- 172.21.15.151:0/1034852826 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6800017650 con 0x7f6814103450 2024-04-26T01:50:28.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:28.632+0000 7f681c200700 1 -- 172.21.15.151:0/1034852826 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68141a0560 con 0x7f6814103450 2024-04-26T01:50:28.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:28.632+0000 7f681c200700 1 -- 172.21.15.151:0/1034852826 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68141a7e40 con 0x7f6814103450 2024-04-26T01:50:28.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:28.633+0000 7f681c200700 1 -- 172.21.15.151:0/1034852826 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6814069730 con 0x7f6814103450 2024-04-26T01:50:28.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:28.635+0000 7f6812000700 1 -- 172.21.15.151:0/1034852826 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f6800029070 con 0x7f6814103450 2024-04-26T01:50:28.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:28.636+0000 7f6812000700 1 --2- 172.21.15.151:0/1034852826 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f67fc0a3cd0 0x7f67fc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:28.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:28.637+0000 7f6812000700 1 -- 172.21.15.151:0/1034852826 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f68000d1090 con 0x7f6814103450 2024-04-26T01:50:28.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:28.637+0000 7f6819800700 1 --2- 172.21.15.151:0/1034852826 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f67fc0a3cd0 0x7f67fc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:28.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:28.637+0000 7f6819800700 1 --2- 172.21.15.151:0/1034852826 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f67fc0a3cd0 0x7f67fc0a6190 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6808009680 tx=0x7f6808009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:28.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:28.641+0000 7f6812000700 1 -- 172.21.15.151:0/1034852826 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6800013070 con 0x7f6814103450 2024-04-26T01:50:28.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:28.949+0000 7f681c200700 1 -- 172.21.15.151:0/1034852826 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f681402df80 con 0x7f6814103450 2024-04-26T01:50:28.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:28.950+0000 7f6812000700 1 -- 172.21.15.151:0/1034852826 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11889 (secure 0 0 0) 0x7f680009d9a0 con 0x7f6814103450 2024-04-26T01:50:28.948 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:50:28.949 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2937864097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2937864097},{"type":"v1","addr":"172.21.15.151:6835","nonce":2937864097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14373,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/2947090802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2947090802},{"type":"v1","addr":"172.21.15.151:6837","nonce":2947090802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/1356563280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1356563280},{"type":"v1","addr":"172.21.15.151:6839","nonce":1356563280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6841/1075240552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1075240552},{"type":"v1","addr":"172.21.15.151:6841","nonce":1075240552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/504675912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":504675912},{"type":"v1","addr":"172.21.15.165:6833","nonce":504675912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1207580015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1207580015},{"type":"v1","addr":"172.21.15.172:6833","nonce":1207580015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:49:40.746876+0000","modified":"2024-04-26T01:50:05.608353+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24505,"mds_2":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/3598138294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3598138294},{"type":"v1","addr":"172.21.15.165:6835","nonce":3598138294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/1847859054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":1847859054},{"type":"v1","addr":"172.21.15.165:6837","nonce":1847859054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/2309751171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2309751171},{"type":"v1","addr":"172.21.15.172:6835","nonce":2309751171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.172:6837/55972537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":55972537},{"type":"v1","addr":"172.21.15.172:6837","nonce":55972537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.172:6839/1242302810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1242302810},{"type":"v1","addr":"172.21.15.172:6839","nonce":1242302810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/1969716254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1969716254},{"type":"v1","addr":"172.21.15.165:6839","nonce":1969716254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24505,24509,24493]},"id":1}]} 2024-04-26T01:50:28.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:28.953+0000 7f681c200700 1 -- 172.21.15.151:0/1034852826 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f67fc0a3cd0 msgr2=0x7f67fc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:28.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:28.953+0000 7f681c200700 1 --2- 172.21.15.151:0/1034852826 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f67fc0a3cd0 0x7f67fc0a6190 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6808009680 tx=0x7f6808009260 comp rx=0 tx=0).stop 2024-04-26T01:50:28.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:28.953+0000 7f681c200700 1 -- 172.21.15.151:0/1034852826 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6814103450 msgr2=0x7f681419b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:28.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:28.953+0000 7f681c200700 1 --2- 172.21.15.151:0/1034852826 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6814103450 0x7f681419b850 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f68000057f0 tx=0x7f680000bfd0 comp rx=0 tx=0).stop 2024-04-26T01:50:28.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:28.953+0000 7f681c200700 1 -- 172.21.15.151:0/1034852826 shutdown_connections 2024-04-26T01:50:28.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:28.953+0000 7f681c200700 1 --2- 172.21.15.151:0/1034852826 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6814108a50 0x7f681419fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:28.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:28.953+0000 7f681c200700 1 --2- 172.21.15.151:0/1034852826 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f67fc0a3cd0 0x7f67fc0a6190 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:28.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:28.953+0000 7f681c200700 1 --2- 172.21.15.151:0/1034852826 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6814102a90 0x7f681419b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:28.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:28.953+0000 7f681c200700 1 --2- 172.21.15.151:0/1034852826 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6814103450 0x7f681419b850 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:28.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:28.953+0000 7f681c200700 1 -- 172.21.15.151:0/1034852826 >> 172.21.15.151:0/1034852826 conn(0x7f6814079ba0 msgr2=0x7f681410dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:28.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:28.953+0000 7f681c200700 1 -- 172.21.15.151:0/1034852826 shutdown_connections 2024-04-26T01:50:28.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:28.953+0000 7f681c200700 1 -- 172.21.15.151:0/1034852826 wait complete. 2024-04-26T01:50:28.952 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 25 2024-04-26T01:50:29.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:29 smithi172 ceph-mon[32785]: pgmap v353: 97 pgs: 1 peering, 96 active+clean; 635 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.9 KiB/s wr, 9 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-04-26T01:50:29.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:29 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/1034852826' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:50:29.200 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:29 smithi151 ceph-mon[28057]: pgmap v353: 97 pgs: 1 peering, 96 active+clean; 635 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.9 KiB/s wr, 9 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-04-26T01:50:29.200 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:29 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/1034852826' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:50:29.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:29 smithi165 ceph-mon[32648]: pgmap v353: 97 pgs: 1 peering, 96 active+clean; 635 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.9 KiB/s wr, 9 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-04-26T01:50:29.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:29 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/1034852826' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:50:30.475 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T01:50:30.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:30 smithi151 ceph-mon[28057]: pgmap v354: 97 pgs: 1 peering, 96 active+clean; 635 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.9 KiB/s wr, 6 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-04-26T01:50:30.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:30 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:50:30.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:30 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:50:30.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:30 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:50:30.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:30 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:50:30.938 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:50:31.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:30 smithi172 ceph-mon[32785]: pgmap v354: 97 pgs: 1 peering, 96 active+clean; 635 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.9 KiB/s wr, 6 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-04-26T01:50:31.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:30 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:50:31.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:30 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:50:31.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:30 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:50:31.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:30 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:50:31.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:30 smithi165 ceph-mon[32648]: pgmap v354: 97 pgs: 1 peering, 96 active+clean; 635 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.9 KiB/s wr, 6 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-04-26T01:50:31.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:30 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:50:31.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:30 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:50:31.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:30 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:50:31.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:30 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:50:32.847 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:32 smithi151 ceph-mon[28057]: pgmap v355: 97 pgs: 97 active+clean; 636 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 4.2 KiB/s wr, 9 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-04-26T01:50:33.038 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:32 smithi172 ceph-mon[32785]: pgmap v355: 97 pgs: 97 active+clean; 636 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 4.2 KiB/s wr, 9 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-04-26T01:50:33.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:32 smithi165 ceph-mon[32648]: pgmap v355: 97 pgs: 97 active+clean; 636 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 4.2 KiB/s wr, 9 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-04-26T01:50:33.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:33.383+0000 7f1338400700 1 -- 172.21.15.151:0/2288806309 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f133006a750 msgr2=0x7f133006abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:33.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:33.383+0000 7f1338400700 1 --2- 172.21.15.151:0/2288806309 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f133006a750 0x7f133006abb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f131c009990 tx=0x7f131c009ca0 comp rx=0 tx=0).stop 2024-04-26T01:50:33.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:33.383+0000 7f1338400700 1 -- 172.21.15.151:0/2288806309 shutdown_connections 2024-04-26T01:50:33.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:33.383+0000 7f1338400700 1 --2- 172.21.15.151:0/2288806309 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f133006b0f0 0x7f133010f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:33.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:33.383+0000 7f1338400700 1 --2- 172.21.15.151:0/2288806309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1330108a30 0x7f1330108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:33.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:33.383+0000 7f1338400700 1 --2- 172.21.15.151:0/2288806309 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f133006a750 0x7f133006abb0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:33.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:33.383+0000 7f1338400700 1 -- 172.21.15.151:0/2288806309 >> 172.21.15.151:0/2288806309 conn(0x7f13300788f0 msgr2=0x7f1330078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:33.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:33.383+0000 7f1338400700 1 -- 172.21.15.151:0/2288806309 shutdown_connections 2024-04-26T01:50:33.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:33.383+0000 7f1338400700 1 -- 172.21.15.151:0/2288806309 wait complete. 2024-04-26T01:50:33.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:33.384+0000 7f1338400700 1 Processor -- start 2024-04-26T01:50:33.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:33.385+0000 7f1338400700 1 -- start start 2024-04-26T01:50:33.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:33.385+0000 7f1338400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f133006a750 0x7f133019b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:33.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:33.385+0000 7f1338400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f133006b0f0 0x7f133019b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:33.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:33.385+0000 7f1338400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1330108a30 0x7f133019fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:33.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:33.385+0000 7f1338400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1330112c20 con 0x7f133006a750 2024-04-26T01:50:33.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:33.385+0000 7f1338400700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1330112aa0 con 0x7f133006b0f0 2024-04-26T01:50:33.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:33.385+0000 7f1338400700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1330112da0 con 0x7f1330108a30 2024-04-26T01:50:33.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:33.385+0000 7f1335a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f133006a750 0x7f133019b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:33.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:33.385+0000 7f1336400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1330108a30 0x7f133019fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:33.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:33.385+0000 7f1335a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f133006a750 0x7f133019b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53044/0 (socket says 172.21.15.151:53044) 2024-04-26T01:50:33.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:33.385+0000 7f1335a00700 1 -- 172.21.15.151:0/3477145474 learned_addr learned my addr 172.21.15.151:0/3477145474 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:50:33.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:33.386+0000 7f1335000700 1 --2- 172.21.15.151:0/3477145474 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f133006b0f0 0x7f133019b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:33.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:33.386+0000 7f1335000700 1 -- 172.21.15.151:0/3477145474 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1330108a30 msgr2=0x7f133019fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:33.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:33.386+0000 7f1335000700 1 --2- 172.21.15.151:0/3477145474 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1330108a30 0x7f133019fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:33.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:33.386+0000 7f1335000700 1 -- 172.21.15.151:0/3477145474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f133006a750 msgr2=0x7f133019b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:33.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:33.386+0000 7f1335000700 1 --2- 172.21.15.151:0/3477145474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f133006a750 0x7f133019b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:33.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:33.386+0000 7f1335000700 1 -- 172.21.15.151:0/3477145474 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13301a0370 con 0x7f133006b0f0 2024-04-26T01:50:33.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:33.386+0000 7f1335000700 1 --2- 172.21.15.151:0/3477145474 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f133006b0f0 0x7f133019b8f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f131c000c00 tx=0x7f131c0047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:33.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:33.386+0000 7f132e000700 1 -- 172.21.15.151:0/3477145474 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131c026070 con 0x7f133006b0f0 2024-04-26T01:50:33.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:33.387+0000 7f1338400700 1 -- 172.21.15.151:0/3477145474 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13301a0600 con 0x7f133006b0f0 2024-04-26T01:50:33.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:33.387+0000 7f132e000700 1 -- 172.21.15.151:0/3477145474 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f131c021410 con 0x7f133006b0f0 2024-04-26T01:50:33.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:33.387+0000 7f132e000700 1 -- 172.21.15.151:0/3477145474 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131c017650 con 0x7f133006b0f0 2024-04-26T01:50:33.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:33.387+0000 7f1338400700 1 -- 172.21.15.151:0/3477145474 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13301a7ee0 con 0x7f133006b0f0 2024-04-26T01:50:33.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:33.388+0000 7f1338400700 1 -- 172.21.15.151:0/3477145474 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1330069730 con 0x7f133006b0f0 2024-04-26T01:50:33.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:33.390+0000 7f132e000700 1 -- 172.21.15.151:0/3477145474 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f131c029070 con 0x7f133006b0f0 2024-04-26T01:50:33.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:33.391+0000 7f132e000700 1 --2- 172.21.15.151:0/3477145474 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f13180a3d50 0x7f13180a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:33.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:33.391+0000 7f132e000700 1 -- 172.21.15.151:0/3477145474 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f131c0d1880 con 0x7f133006b0f0 2024-04-26T01:50:33.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:33.392+0000 7f1335a00700 1 --2- 172.21.15.151:0/3477145474 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f13180a3d50 0x7f13180a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:33.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:33.392+0000 7f1335a00700 1 --2- 172.21.15.151:0/3477145474 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f13180a3d50 0x7f13180a6210 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1324009ca0 tx=0x7f1324009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:33.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:33.394+0000 7f132e000700 1 -- 172.21.15.151:0/3477145474 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f131c030070 con 0x7f133006b0f0 2024-04-26T01:50:33.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:33.621+0000 7f1338400700 1 -- 172.21.15.151:0/3477145474 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f133002df80 con 0x7f133006b0f0 2024-04-26T01:50:33.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:33.623+0000 7f132e000700 1 -- 172.21.15.151:0/3477145474 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+22599 (secure 0 0 0) 0x7f131c013070 con 0x7f133006b0f0 2024-04-26T01:50:33.622 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:50:33.622 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":92,"fsid":"c0e49e64-036d-11ef-bc93-c7b262605968","created":"2024-04-26T01:40:07.483197+0000","modified":"2024-04-26T01:50:26.985192+0000","last_up_change":"2024-04-26T01:48:21.167691+0000","last_in_change":"2024-04-26T01:48:00.556990+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T01:44:11.411722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T01:49:31.756032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T01:49:36.529819+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ca27e814-a982-482c-ae1a-e596aac5c59a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":669937998},{"type":"v1","addr":"172.21.15.151:6803","nonce":669937998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":669937998},{"type":"v1","addr":"172.21.15.151:6805","nonce":669937998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":669937998},{"type":"v1","addr":"172.21.15.151:6809","nonce":669937998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":669937998},{"type":"v1","addr":"172.21.15.151:6807","nonce":669937998}]},"public_addr":"172.21.15.151:6803/669937998","cluster_addr":"172.21.15.151:6805/669937998","heartbeat_back_addr":"172.21.15.151:6809/669937998","heartbeat_front_addr":"172.21.15.151:6807/669937998","state":["exists","up"]},{"osd":1,"uuid":"e7d192c3-9624-4625-9ccb-a76c114d8aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":1356168171},{"type":"v1","addr":"172.21.15.165:6801","nonce":1356168171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":1356168171},{"type":"v1","addr":"172.21.15.165:6803","nonce":1356168171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":1356168171},{"type":"v1","addr":"172.21.15.165:6807","nonce":1356168171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":1356168171},{"type":"v1","addr":"172.21.15.165:6805","nonce":1356168171}]},"public_addr":"172.21.15.165:6801/1356168171","cluster_addr":"172.21.15.165:6803/1356168171","heartbeat_back_addr":"172.21.15.165:6807/1356168171","heartbeat_front_addr":"172.21.15.165:6805/1356168171","state":["exists","up"]},{"osd":2,"uuid":"0631df59-1ecc-48b4-b00a-48b0118dfc24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":237502635},{"type":"v1","addr":"172.21.15.172:6801","nonce":237502635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":237502635},{"type":"v1","addr":"172.21.15.172:6803","nonce":237502635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":237502635},{"type":"v1","addr":"172.21.15.172:6807","nonce":237502635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":237502635},{"type":"v1","addr":"172.21.15.172:6805","nonce":237502635}]},"public_addr":"172.21.15.172:6801/237502635","cluster_addr":"172.21.15.172:6803/237502635","heartbeat_back_addr":"172.21.15.172:6807/237502635","heartbeat_front_addr":"172.21.15.172:6805/237502635","state":["exists","up"]},{"osd":3,"uuid":"73076e49-2f8a-4a3d-99eb-7b50aff448aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3442279476},{"type":"v1","addr":"172.21.15.151:6811","nonce":3442279476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3442279476},{"type":"v1","addr":"172.21.15.151:6813","nonce":3442279476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":3442279476},{"type":"v1","addr":"172.21.15.151:6817","nonce":3442279476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3442279476},{"type":"v1","addr":"172.21.15.151:6815","nonce":3442279476}]},"public_addr":"172.21.15.151:6811/3442279476","cluster_addr":"172.21.15.151:6813/3442279476","heartbeat_back_addr":"172.21.15.151:6817/3442279476","heartbeat_front_addr":"172.21.15.151:6815/3442279476","state":["exists","up"]},{"osd":4,"uuid":"0230d615-548c-43f0-b8aa-7130f4cbbc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4029980723},{"type":"v1","addr":"172.21.15.165:6809","nonce":4029980723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4029980723},{"type":"v1","addr":"172.21.15.165:6811","nonce":4029980723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4029980723},{"type":"v1","addr":"172.21.15.165:6815","nonce":4029980723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4029980723},{"type":"v1","addr":"172.21.15.165:6813","nonce":4029980723}]},"public_addr":"172.21.15.165:6809/4029980723","cluster_addr":"172.21.15.165:6811/4029980723","heartbeat_back_addr":"172.21.15.165:6815/4029980723","heartbeat_front_addr":"172.21.15.165:6813/4029980723","state":["exists","up"]},{"osd":5,"uuid":"9739f44b-2605-47dd-82fb-d30034742bf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2216578687},{"type":"v1","addr":"172.21.15.172:6809","nonce":2216578687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2216578687},{"type":"v1","addr":"172.21.15.172:6811","nonce":2216578687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2216578687},{"type":"v1","addr":"172.21.15.172:6815","nonce":2216578687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2216578687},{"type":"v1","addr":"172.21.15.172:6813","nonce":2216578687}]},"public_addr":"172.21.15.172:6809/2216578687","cluster_addr":"172.21.15.172:6811/2216578687","heartbeat_back_addr":"172.21.15.172:6815/2216578687","heartbeat_front_addr":"172.21.15.172:6813/2216578687","state":["exists","up"]},{"osd":6,"uuid":"4290baaa-6160-49cb-a4e0-c96f070f2ef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1723252315},{"type":"v1","addr":"172.21.15.151:6819","nonce":1723252315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1723252315},{"type":"v1","addr":"172.21.15.151:6821","nonce":1723252315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":1723252315},{"type":"v1","addr":"172.21.15.151:6825","nonce":1723252315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1723252315},{"type":"v1","addr":"172.21.15.151:6823","nonce":1723252315}]},"public_addr":"172.21.15.151:6819/1723252315","cluster_addr":"172.21.15.151:6821/1723252315","heartbeat_back_addr":"172.21.15.151:6825/1723252315","heartbeat_front_addr":"172.21.15.151:6823/1723252315","state":["exists","up"]},{"osd":7,"uuid":"9f5315f2-036c-4e2f-92ed-3321a9e3c860","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":1977803781},{"type":"v1","addr":"172.21.15.165:6817","nonce":1977803781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":1977803781},{"type":"v1","addr":"172.21.15.165:6819","nonce":1977803781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":1977803781},{"type":"v1","addr":"172.21.15.165:6823","nonce":1977803781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":1977803781},{"type":"v1","addr":"172.21.15.165:6821","nonce":1977803781}]},"public_addr":"172.21.15.165:6817/1977803781","cluster_addr":"172.21.15.165:6819/1977803781","heartbeat_back_addr":"172.21.15.165:6823/1977803781","heartbeat_front_addr":"172.21.15.165:6821/1977803781","state":["exists","up"]},{"osd":8,"uuid":"8ddbe36d-cea7-45ef-a3ca-044a0e64ff85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":491199512},{"type":"v1","addr":"172.21.15.172:6817","nonce":491199512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":491199512},{"type":"v1","addr":"172.21.15.172:6819","nonce":491199512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":491199512},{"type":"v1","addr":"172.21.15.172:6823","nonce":491199512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":491199512},{"type":"v1","addr":"172.21.15.172:6821","nonce":491199512}]},"public_addr":"172.21.15.172:6817/491199512","cluster_addr":"172.21.15.172:6819/491199512","heartbeat_back_addr":"172.21.15.172:6823/491199512","heartbeat_front_addr":"172.21.15.172:6821/491199512","state":["exists","up"]},{"osd":9,"uuid":"40654ba0-1bd9-4b54-97ae-f1ed6c4785d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":1818923828},{"type":"v1","addr":"172.21.15.151:6827","nonce":1818923828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":1818923828},{"type":"v1","addr":"172.21.15.151:6829","nonce":1818923828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1818923828},{"type":"v1","addr":"172.21.15.151:6833","nonce":1818923828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":1818923828},{"type":"v1","addr":"172.21.15.151:6831","nonce":1818923828}]},"public_addr":"172.21.15.151:6827/1818923828","cluster_addr":"172.21.15.151:6829/1818923828","heartbeat_back_addr":"172.21.15.151:6833/1818923828","heartbeat_front_addr":"172.21.15.151:6831/1818923828","state":["exists","up"]},{"osd":10,"uuid":"19e75a1a-7bd0-41d9-b0cc-0fa4d1475dda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":2632621809},{"type":"v1","addr":"172.21.15.165:6825","nonce":2632621809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":2632621809},{"type":"v1","addr":"172.21.15.165:6827","nonce":2632621809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":2632621809},{"type":"v1","addr":"172.21.15.165:6831","nonce":2632621809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":2632621809},{"type":"v1","addr":"172.21.15.165:6829","nonce":2632621809}]},"public_addr":"172.21.15.165:6825/2632621809","cluster_addr":"172.21.15.165:6827/2632621809","heartbeat_back_addr":"172.21.15.165:6831/2632621809","heartbeat_front_addr":"172.21.15.165:6829/2632621809","state":["exists","up"]},{"osd":11,"uuid":"aa84e5f5-93a3-4375-8544-a1175f0eb06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":886604365},{"type":"v1","addr":"172.21.15.172:6825","nonce":886604365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":886604365},{"type":"v1","addr":"172.21.15.172:6827","nonce":886604365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":886604365},{"type":"v1","addr":"172.21.15.172:6831","nonce":886604365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":886604365},{"type":"v1","addr":"172.21.15.172:6829","nonce":886604365}]},"public_addr":"172.21.15.172:6825/886604365","cluster_addr":"172.21.15.172:6827/886604365","heartbeat_back_addr":"172.21.15.172:6831/886604365","heartbeat_front_addr":"172.21.15.172:6829/886604365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:43:18.296278+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:43:42.860620+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:44:07.073601+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:44:39.631760+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:45:04.771214+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:45:29.606674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:46:03.101851+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:46:28.351995+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:46:53.445435+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:47:27.391179+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:47:53.543638+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:48:18.687811+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.151:6801/796314699":"2024-04-27T01:41:25.234322+0000","172.21.15.151:6800/796314699":"2024-04-27T01:41:25.234322+0000","172.21.15.151:0/1173525554":"2024-04-27T01:41:25.234322+0000","172.21.15.151:0/1342496801":"2024-04-27T01:40:50.492554+0000","172.21.15.151:0/11665926":"2024-04-27T01:40:50.492554+0000","172.21.15.151:0/1765448952":"2024-04-27T01:40:50.492554+0000","172.21.15.151:0/46007430":"2024-04-27T01:41:25.234322+0000","172.21.15.151:6800/244725060":"2024-04-27T01:40:50.492554+0000","172.21.15.151:0/3703862543":"2024-04-27T01:41:25.234322+0000","172.21.15.151:6801/244725060":"2024-04-27T01:40:50.492554+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T01:50:33.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:33.626+0000 7f1338400700 1 -- 172.21.15.151:0/3477145474 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f13180a3d50 msgr2=0x7f13180a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:33.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:33.626+0000 7f1338400700 1 --2- 172.21.15.151:0/3477145474 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f13180a3d50 0x7f13180a6210 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1324009ca0 tx=0x7f1324009260 comp rx=0 tx=0).stop 2024-04-26T01:50:33.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:33.626+0000 7f1338400700 1 -- 172.21.15.151:0/3477145474 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f133006b0f0 msgr2=0x7f133019b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:33.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:33.626+0000 7f1338400700 1 --2- 172.21.15.151:0/3477145474 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f133006b0f0 0x7f133019b8f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f131c000c00 tx=0x7f131c0047b0 comp rx=0 tx=0).stop 2024-04-26T01:50:33.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:33.626+0000 7f1338400700 1 -- 172.21.15.151:0/3477145474 shutdown_connections 2024-04-26T01:50:33.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:33.626+0000 7f1338400700 1 --2- 172.21.15.151:0/3477145474 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1330108a30 0x7f133019fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:33.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:33.626+0000 7f1338400700 1 --2- 172.21.15.151:0/3477145474 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f13180a3d50 0x7f13180a6210 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:33.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:33.626+0000 7f1338400700 1 --2- 172.21.15.151:0/3477145474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f133006a750 0x7f133019b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:33.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:33.626+0000 7f1338400700 1 --2- 172.21.15.151:0/3477145474 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f133006b0f0 0x7f133019b8f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:33.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:33.626+0000 7f1338400700 1 -- 172.21.15.151:0/3477145474 >> 172.21.15.151:0/3477145474 conn(0x7f13300788f0 msgr2=0x7f133010df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:33.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:33.627+0000 7f1338400700 1 -- 172.21.15.151:0/3477145474 shutdown_connections 2024-04-26T01:50:33.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:33.627+0000 7f1338400700 1 -- 172.21.15.151:0/3477145474 wait complete. 2024-04-26T01:50:33.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:33 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:50:33.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:33 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:50:33.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:33 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/3477145474' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:50:34.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:33 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:50:34.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:33 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:50:34.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:33 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/3477145474' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:50:34.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:33 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:50:34.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:33 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:50:34.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:33 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/3477145474' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:50:34.593 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-26T01:50:34.593 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T01:50:34.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:34 smithi151 ceph-mon[28057]: pgmap v356: 97 pgs: 97 active+clean; 636 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 4.1 KiB/s wr, 9 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-04-26T01:50:34.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:34 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:50:34.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:34 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:50:34.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:34 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:50:34.938 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:34 smithi172 ceph-mon[32785]: pgmap v356: 97 pgs: 97 active+clean; 636 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 4.1 KiB/s wr, 9 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-04-26T01:50:34.938 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:34 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:50:34.938 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:34 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:50:34.938 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:34 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:50:35.047 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:50:35.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:34 smithi165 ceph-mon[32648]: pgmap v356: 97 pgs: 97 active+clean; 636 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 4.1 KiB/s wr, 9 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-04-26T01:50:35.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:34 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:50:35.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:34 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:50:35.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:34 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:50:36.772 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:36 smithi151 ceph-mon[28057]: pgmap v357: 97 pgs: 97 active+clean; 636 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 543 B/s wr, 5 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-04-26T01:50:37.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:36 smithi172 ceph-mon[32785]: pgmap v357: 97 pgs: 97 active+clean; 636 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 543 B/s wr, 5 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-04-26T01:50:37.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:36 smithi165 ceph-mon[32648]: pgmap v357: 97 pgs: 97 active+clean; 636 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 543 B/s wr, 5 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-04-26T01:50:37.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:37.263+0000 7fd638200700 1 -- 172.21.15.151:0/3375934523 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd63010e3b0 msgr2=0x7fd630115f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:37.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:37.263+0000 7fd638200700 1 --2- 172.21.15.151:0/3375934523 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd63010e3b0 0x7fd630115f80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd62800b0a0 tx=0x7fd62800b3b0 comp rx=0 tx=0).stop 2024-04-26T01:50:37.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:37.263+0000 7fd638200700 1 -- 172.21.15.151:0/3375934523 shutdown_connections 2024-04-26T01:50:37.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:37.263+0000 7fd638200700 1 --2- 172.21.15.151:0/3375934523 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd63010e3b0 0x7fd630115f80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:37.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:37.263+0000 7fd638200700 1 --2- 172.21.15.151:0/3375934523 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6301092d0 0x7fd6301096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:37.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:37.263+0000 7fd638200700 1 --2- 172.21.15.151:0/3375934523 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd630109c80 0x7fd63010dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:37.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:37.263+0000 7fd638200700 1 -- 172.21.15.151:0/3375934523 >> 172.21.15.151:0/3375934523 conn(0x7fd630076210 msgr2=0x7fd630107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:37.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:37.263+0000 7fd638200700 1 -- 172.21.15.151:0/3375934523 shutdown_connections 2024-04-26T01:50:37.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:37.263+0000 7fd638200700 1 -- 172.21.15.151:0/3375934523 wait complete. 2024-04-26T01:50:37.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:37.264+0000 7fd638200700 1 Processor -- start 2024-04-26T01:50:37.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:37.264+0000 7fd638200700 1 -- start start 2024-04-26T01:50:37.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:37.265+0000 7fd638200700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6301092d0 0x7fd63019f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:37.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:37.265+0000 7fd638200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd630109c80 0x7fd63019fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:37.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:37.265+0000 7fd638200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd63010e3b0 0x7fd6301a4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:37.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:37.265+0000 7fd638200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd6301192c0 con 0x7fd63010e3b0 2024-04-26T01:50:37.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:37.265+0000 7fd638200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd630119140 con 0x7fd630109c80 2024-04-26T01:50:37.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:37.265+0000 7fd638200700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd630119440 con 0x7fd6301092d0 2024-04-26T01:50:37.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:37.265+0000 7fd635800700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6301092d0 0x7fd63019f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:37.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:37.265+0000 7fd635800700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6301092d0 0x7fd63019f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:43928/0 (socket says 172.21.15.151:43928) 2024-04-26T01:50:37.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:37.265+0000 7fd635800700 1 -- 172.21.15.151:0/1136712753 learned_addr learned my addr 172.21.15.151:0/1136712753 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:50:37.278 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:37.265+0000 7fd636200700 1 --2- 172.21.15.151:0/1136712753 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd63010e3b0 0x7fd6301a4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:37.278 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:37.265+0000 7fd634e00700 1 --2- 172.21.15.151:0/1136712753 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd630109c80 0x7fd63019fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:37.278 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:37.265+0000 7fd635800700 1 -- 172.21.15.151:0/1136712753 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd630109c80 msgr2=0x7fd63019fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:37.278 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:37.265+0000 7fd635800700 1 --2- 172.21.15.151:0/1136712753 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd630109c80 0x7fd63019fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:37.278 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:37.265+0000 7fd635800700 1 -- 172.21.15.151:0/1136712753 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd63010e3b0 msgr2=0x7fd6301a4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:37.278 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:37.265+0000 7fd635800700 1 --2- 172.21.15.151:0/1136712753 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd63010e3b0 0x7fd6301a4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:37.278 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:37.265+0000 7fd635800700 1 -- 172.21.15.151:0/1136712753 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6301a4760 con 0x7fd6301092d0 2024-04-26T01:50:37.278 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:37.266+0000 7fd636200700 1 --2- 172.21.15.151:0/1136712753 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd63010e3b0 0x7fd6301a4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:50:37.278 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:37.266+0000 7fd634e00700 1 --2- 172.21.15.151:0/1136712753 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd630109c80 0x7fd63019fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:50:37.278 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:37.266+0000 7fd635800700 1 --2- 172.21.15.151:0/1136712753 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6301092d0 0x7fd63019f750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd62400e9c0 tx=0x7fd62400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:37.278 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:37.266+0000 7fd62e000700 1 -- 172.21.15.151:0/1136712753 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62400cbd0 con 0x7fd6301092d0 2024-04-26T01:50:37.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:37.266+0000 7fd62e000700 1 -- 172.21.15.151:0/1136712753 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd62400ee60 con 0x7fd6301092d0 2024-04-26T01:50:37.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:37.266+0000 7fd638200700 1 -- 172.21.15.151:0/1136712753 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6301a4a50 con 0x7fd6301092d0 2024-04-26T01:50:37.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:37.266+0000 7fd62e000700 1 -- 172.21.15.151:0/1136712753 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6240055c0 con 0x7fd6301092d0 2024-04-26T01:50:37.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:37.266+0000 7fd638200700 1 -- 172.21.15.151:0/1136712753 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6301ac2d0 con 0x7fd6301092d0 2024-04-26T01:50:37.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:37.267+0000 7fd638200700 1 -- 172.21.15.151:0/1136712753 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd630069730 con 0x7fd6301092d0 2024-04-26T01:50:37.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:37.269+0000 7fd62e000700 1 -- 172.21.15.151:0/1136712753 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fd624004750 con 0x7fd6301092d0 2024-04-26T01:50:37.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:37.270+0000 7fd62e000700 1 --2- 172.21.15.151:0/1136712753 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fd6180a3cd0 0x7fd6180a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:37.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:37.271+0000 7fd62e000700 1 -- 172.21.15.151:0/1136712753 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7fd6240d0840 con 0x7fd6301092d0 2024-04-26T01:50:37.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:37.271+0000 7fd634e00700 1 --2- 172.21.15.151:0/1136712753 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fd6180a3cd0 0x7fd6180a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:37.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:37.271+0000 7fd634e00700 1 --2- 172.21.15.151:0/1136712753 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fd6180a3cd0 0x7fd6180a6190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd61c009680 tx=0x7fd61c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:37.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:37.275+0000 7fd62e000700 1 -- 172.21.15.151:0/1136712753 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd62409d0c0 con 0x7fd6301092d0 2024-04-26T01:50:37.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:37.499+0000 7fd638200700 1 -- 172.21.15.151:0/1136712753 --> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd630062e80 con 0x7fd6180a3cd0 2024-04-26T01:50:37.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:37.510+0000 7fd62e000700 1 -- 172.21.15.151:0/1136712753 <== mgr.14152 v2:172.21.15.151:6800/2251256505 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258727 (secure 0 0 0) 0x7fd630062e80 con 0x7fd6180a3cd0 2024-04-26T01:50:37.513 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:50:37.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:37.515+0000 7fd638200700 1 -- 172.21.15.151:0/1136712753 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fd6180a3cd0 msgr2=0x7fd6180a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:37.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:37.515+0000 7fd638200700 1 --2- 172.21.15.151:0/1136712753 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fd6180a3cd0 0x7fd6180a6190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd61c009680 tx=0x7fd61c009260 comp rx=0 tx=0).stop 2024-04-26T01:50:37.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:37.515+0000 7fd638200700 1 -- 172.21.15.151:0/1136712753 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6301092d0 msgr2=0x7fd63019f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:37.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:37.515+0000 7fd638200700 1 --2- 172.21.15.151:0/1136712753 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6301092d0 0x7fd63019f750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd62400e9c0 tx=0x7fd62400ecd0 comp rx=0 tx=0).stop 2024-04-26T01:50:37.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:37.515+0000 7fd638200700 1 -- 172.21.15.151:0/1136712753 shutdown_connections 2024-04-26T01:50:37.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:37.515+0000 7fd638200700 1 --2- 172.21.15.151:0/1136712753 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fd6180a3cd0 0x7fd6180a6190 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:37.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:37.515+0000 7fd638200700 1 --2- 172.21.15.151:0/1136712753 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd63010e3b0 0x7fd6301a4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:37.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:37.515+0000 7fd638200700 1 --2- 172.21.15.151:0/1136712753 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6301092d0 0x7fd63019f750 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:37.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:37.515+0000 7fd638200700 1 --2- 172.21.15.151:0/1136712753 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd630109c80 0x7fd63019fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:37.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:37.515+0000 7fd638200700 1 -- 172.21.15.151:0/1136712753 >> 172.21.15.151:0/1136712753 conn(0x7fd630076210 msgr2=0x7fd630076790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:37.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:37.515+0000 7fd638200700 1 -- 172.21.15.151:0/1136712753 shutdown_connections 2024-04-26T01:50:37.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:37.515+0000 7fd638200700 1 -- 172.21.15.151:0/1136712753 wait complete. 2024-04-26T01:50:37.517 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-04-26T01:50:38.521 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":358,"stamp":"2024-04-26T01:50:37.403851+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655341,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":370,"num_read_kb":305,"num_write":347,"num_write_kb":3051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":278,"ondisk_log_size":278,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":387956,"kb_used_data":13788,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124553868,"statfs":{"total":1151940427776,"available":1151543160832,"internally_reserved":0,"allocated":14118912,"data_stored":7795599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5395,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":67,"num_read_kb":37,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.414929"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.996096+0000","last_change":"2024-04-26T01:49:37.641666+0000","last_active":"2024-04-26T01:50:26.996096+0000","last_peered":"2024-04-26T01:50:26.996096+0000","last_clean":"2024-04-26T01:50:26.996096+0000","last_became_active":"2024-04-26T01:49:37.641310+0000","last_became_peered":"2024-04-26T01:49:37.641310+0000","last_unstale":"2024-04-26T01:50:26.996096+0000","last_undegraded":"2024-04-26T01:50:26.996096+0000","last_fullsized":"2024-04-26T01:50:26.996096+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:28:02.952851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.993389+0000","last_change":"2024-04-26T01:49:37.637607+0000","last_active":"2024-04-26T01:50:26.993389+0000","last_peered":"2024-04-26T01:50:26.993389+0000","last_clean":"2024-04-26T01:50:26.993389+0000","last_became_active":"2024-04-26T01:49:37.637283+0000","last_became_peered":"2024-04-26T01:49:37.637283+0000","last_unstale":"2024-04-26T01:50:26.993389+0000","last_undegraded":"2024-04-26T01:50:26.993389+0000","last_fullsized":"2024-04-26T01:50:26.993389+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:16:46.519381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.992971+0000","last_change":"2024-04-26T01:49:37.647769+0000","last_active":"2024-04-26T01:50:26.992971+0000","last_peered":"2024-04-26T01:50:26.992971+0000","last_clean":"2024-04-26T01:50:26.992971+0000","last_became_active":"2024-04-26T01:49:37.643750+0000","last_became_peered":"2024-04-26T01:49:37.643750+0000","last_unstale":"2024-04-26T01:50:26.992971+0000","last_undegraded":"2024-04-26T01:50:26.992971+0000","last_fullsized":"2024-04-26T01:50:26.992971+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:34:18.801267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.485851+0000","last_change":"2024-04-26T01:49:37.643291+0000","last_active":"2024-04-26T01:50:27.485851+0000","last_peered":"2024-04-26T01:50:27.485851+0000","last_clean":"2024-04-26T01:50:27.485851+0000","last_became_active":"2024-04-26T01:49:37.642947+0000","last_became_peered":"2024-04-26T01:49:37.642947+0000","last_unstale":"2024-04-26T01:50:27.485851+0000","last_undegraded":"2024-04-26T01:50:27.485851+0000","last_fullsized":"2024-04-26T01:50:27.485851+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:31:21.600533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998208+0000","last_change":"2024-04-26T01:49:37.665311+0000","last_active":"2024-04-26T01:50:26.998208+0000","last_peered":"2024-04-26T01:50:26.998208+0000","last_clean":"2024-04-26T01:50:26.998208+0000","last_became_active":"2024-04-26T01:49:37.664944+0000","last_became_peered":"2024-04-26T01:49:37.664944+0000","last_unstale":"2024-04-26T01:50:26.998208+0000","last_undegraded":"2024-04-26T01:50:26.998208+0000","last_fullsized":"2024-04-26T01:50:26.998208+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:49:50.744630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.992822+0000","last_change":"2024-04-26T01:49:37.638667+0000","last_active":"2024-04-26T01:50:26.992822+0000","last_peered":"2024-04-26T01:50:26.992822+0000","last_clean":"2024-04-26T01:50:26.992822+0000","last_became_active":"2024-04-26T01:49:37.638226+0000","last_became_peered":"2024-04-26T01:49:37.638226+0000","last_unstale":"2024-04-26T01:50:26.992822+0000","last_undegraded":"2024-04-26T01:50:26.992822+0000","last_fullsized":"2024-04-26T01:50:26.992822+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:46:17.786883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998214+0000","last_change":"2024-04-26T01:49:37.650082+0000","last_active":"2024-04-26T01:50:26.998214+0000","last_peered":"2024-04-26T01:50:26.998214+0000","last_clean":"2024-04-26T01:50:26.998214+0000","last_became_active":"2024-04-26T01:49:37.649637+0000","last_became_peered":"2024-04-26T01:49:37.649637+0000","last_unstale":"2024-04-26T01:50:26.998214+0000","last_undegraded":"2024-04-26T01:50:26.998214+0000","last_fullsized":"2024-04-26T01:50:26.998214+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:21:04.686398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.991647+0000","last_change":"2024-04-26T01:49:37.661255+0000","last_active":"2024-04-26T01:50:26.991647+0000","last_peered":"2024-04-26T01:50:26.991647+0000","last_clean":"2024-04-26T01:50:26.991647+0000","last_became_active":"2024-04-26T01:49:37.658882+0000","last_became_peered":"2024-04-26T01:49:37.658882+0000","last_unstale":"2024-04-26T01:50:26.991647+0000","last_undegraded":"2024-04-26T01:50:26.991647+0000","last_fullsized":"2024-04-26T01:50:26.991647+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:36:27.874191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.837626+0000","last_change":"2024-04-26T01:49:37.648874+0000","last_active":"2024-04-26T01:50:27.837626+0000","last_peered":"2024-04-26T01:50:27.837626+0000","last_clean":"2024-04-26T01:50:27.837626+0000","last_became_active":"2024-04-26T01:49:37.648490+0000","last_became_peered":"2024-04-26T01:49:37.648490+0000","last_unstale":"2024-04-26T01:50:27.837626+0000","last_undegraded":"2024-04-26T01:50:27.837626+0000","last_fullsized":"2024-04-26T01:50:27.837626+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:46:12.552861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.993321+0000","last_change":"2024-04-26T01:49:37.643842+0000","last_active":"2024-04-26T01:50:26.993321+0000","last_peered":"2024-04-26T01:50:26.993321+0000","last_clean":"2024-04-26T01:50:26.993321+0000","last_became_active":"2024-04-26T01:49:37.643084+0000","last_became_peered":"2024-04-26T01:49:37.643084+0000","last_unstale":"2024-04-26T01:50:26.993321+0000","last_undegraded":"2024-04-26T01:50:26.993321+0000","last_fullsized":"2024-04-26T01:50:26.993321+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:16:53.102318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998567+0000","last_change":"2024-04-26T01:49:37.644209+0000","last_active":"2024-04-26T01:50:26.998567+0000","last_peered":"2024-04-26T01:50:26.998567+0000","last_clean":"2024-04-26T01:50:26.998567+0000","last_became_active":"2024-04-26T01:49:37.643659+0000","last_became_peered":"2024-04-26T01:49:37.643659+0000","last_unstale":"2024-04-26T01:50:26.998567+0000","last_undegraded":"2024-04-26T01:50:26.998567+0000","last_fullsized":"2024-04-26T01:50:26.998567+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:24:46.363873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.001615+0000","last_change":"2024-04-26T01:50:27.001615+0000","last_active":"2024-04-26T01:50:27.001615+0000","last_peered":"2024-04-26T01:50:27.001615+0000","last_clean":"2024-04-26T01:50:27.001615+0000","last_became_active":"2024-04-26T01:50:27.001039+0000","last_became_peered":"2024-04-26T01:50:27.001039+0000","last_unstale":"2024-04-26T01:50:27.001615+0000","last_undegraded":"2024-04-26T01:50:27.001615+0000","last_fullsized":"2024-04-26T01:50:27.001615+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:42:46.148500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.006036+0000","last_change":"2024-04-26T01:49:37.651085+0000","last_active":"2024-04-26T01:50:27.006036+0000","last_peered":"2024-04-26T01:50:27.006036+0000","last_clean":"2024-04-26T01:50:27.006036+0000","last_became_active":"2024-04-26T01:49:37.649662+0000","last_became_peered":"2024-04-26T01:49:37.649662+0000","last_unstale":"2024-04-26T01:50:27.006036+0000","last_undegraded":"2024-04-26T01:50:27.006036+0000","last_fullsized":"2024-04-26T01:50:27.006036+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:41:34.537533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.992971+0000","last_change":"2024-04-26T01:49:37.648432+0000","last_active":"2024-04-26T01:50:26.992971+0000","last_peered":"2024-04-26T01:50:26.992971+0000","last_clean":"2024-04-26T01:50:26.992971+0000","last_became_active":"2024-04-26T01:49:37.648047+0000","last_became_peered":"2024-04-26T01:49:37.648047+0000","last_unstale":"2024-04-26T01:50:26.992971+0000","last_undegraded":"2024-04-26T01:50:26.992971+0000","last_fullsized":"2024-04-26T01:50:26.992971+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:00:06.843384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.992100+0000","last_change":"2024-04-26T01:49:37.644203+0000","last_active":"2024-04-26T01:50:26.992100+0000","last_peered":"2024-04-26T01:50:26.992100+0000","last_clean":"2024-04-26T01:50:26.992100+0000","last_became_active":"2024-04-26T01:49:37.643772+0000","last_became_peered":"2024-04-26T01:49:37.643772+0000","last_unstale":"2024-04-26T01:50:26.992100+0000","last_undegraded":"2024-04-26T01:50:26.992100+0000","last_fullsized":"2024-04-26T01:50:26.992100+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:52:20.707845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.837131+0000","last_change":"2024-04-26T01:49:37.647364+0000","last_active":"2024-04-26T01:50:27.837131+0000","last_peered":"2024-04-26T01:50:27.837131+0000","last_clean":"2024-04-26T01:50:27.837131+0000","last_became_active":"2024-04-26T01:49:37.647054+0000","last_became_peered":"2024-04-26T01:49:37.647054+0000","last_unstale":"2024-04-26T01:50:27.837131+0000","last_undegraded":"2024-04-26T01:50:27.837131+0000","last_fullsized":"2024-04-26T01:50:27.837131+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:53:09.631463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.996185+0000","last_change":"2024-04-26T01:49:37.641843+0000","last_active":"2024-04-26T01:50:26.996185+0000","last_peered":"2024-04-26T01:50:26.996185+0000","last_clean":"2024-04-26T01:50:26.996185+0000","last_became_active":"2024-04-26T01:49:37.641514+0000","last_became_peered":"2024-04-26T01:49:37.641514+0000","last_unstale":"2024-04-26T01:50:26.996185+0000","last_undegraded":"2024-04-26T01:50:26.996185+0000","last_fullsized":"2024-04-26T01:50:26.996185+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:24:32.261834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998685+0000","last_change":"2024-04-26T01:49:37.644750+0000","last_active":"2024-04-26T01:50:26.998685+0000","last_peered":"2024-04-26T01:50:26.998685+0000","last_clean":"2024-04-26T01:50:26.998685+0000","last_became_active":"2024-04-26T01:49:37.644086+0000","last_became_peered":"2024-04-26T01:49:37.644086+0000","last_unstale":"2024-04-26T01:50:26.998685+0000","last_undegraded":"2024-04-26T01:50:26.998685+0000","last_fullsized":"2024-04-26T01:50:26.998685+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:59:42.614846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998814+0000","last_change":"2024-04-26T01:49:37.664204+0000","last_active":"2024-04-26T01:50:26.998814+0000","last_peered":"2024-04-26T01:50:26.998814+0000","last_clean":"2024-04-26T01:50:26.998814+0000","last_became_active":"2024-04-26T01:49:37.663878+0000","last_became_peered":"2024-04-26T01:49:37.663878+0000","last_unstale":"2024-04-26T01:50:26.998814+0000","last_undegraded":"2024-04-26T01:50:26.998814+0000","last_fullsized":"2024-04-26T01:50:26.998814+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:43:49.272308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.996978+0000","last_change":"2024-04-26T01:49:33.592017+0000","last_active":"2024-04-26T01:50:26.996978+0000","last_peered":"2024-04-26T01:50:26.996978+0000","last_clean":"2024-04-26T01:50:26.996978+0000","last_became_active":"2024-04-26T01:49:33.587928+0000","last_became_peered":"2024-04-26T01:49:33.587928+0000","last_unstale":"2024-04-26T01:50:26.996978+0000","last_undegraded":"2024-04-26T01:50:26.996978+0000","last_fullsized":"2024-04-26T01:50:26.996978+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:02:04.313785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.993721+0000","last_change":"2024-04-26T01:49:37.662010+0000","last_active":"2024-04-26T01:50:26.993721+0000","last_peered":"2024-04-26T01:50:26.993721+0000","last_clean":"2024-04-26T01:50:26.993721+0000","last_became_active":"2024-04-26T01:49:37.661673+0000","last_became_peered":"2024-04-26T01:49:37.661673+0000","last_unstale":"2024-04-26T01:50:26.993721+0000","last_undegraded":"2024-04-26T01:50:26.993721+0000","last_fullsized":"2024-04-26T01:50:26.993721+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:37:34.155998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.994164+0000","last_change":"2024-04-26T01:49:34.185758+0000","last_active":"2024-04-26T01:50:26.994164+0000","last_peered":"2024-04-26T01:50:26.994164+0000","last_clean":"2024-04-26T01:50:26.994164+0000","last_became_active":"2024-04-26T01:49:34.184669+0000","last_became_peered":"2024-04-26T01:49:34.184669+0000","last_unstale":"2024-04-26T01:50:26.994164+0000","last_undegraded":"2024-04-26T01:50:26.994164+0000","last_fullsized":"2024-04-26T01:50:26.994164+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:23:23.742032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.993378+0000","last_change":"2024-04-26T01:49:37.641307+0000","last_active":"2024-04-26T01:50:26.993378+0000","last_peered":"2024-04-26T01:50:26.993378+0000","last_clean":"2024-04-26T01:50:26.993378+0000","last_became_active":"2024-04-26T01:49:37.640677+0000","last_became_peered":"2024-04-26T01:49:37.640677+0000","last_unstale":"2024-04-26T01:50:26.993378+0000","last_undegraded":"2024-04-26T01:50:26.993378+0000","last_fullsized":"2024-04-26T01:50:26.993378+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:35:56.859045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.007585+0000","last_change":"2024-04-26T01:50:27.007034+0000","last_active":"2024-04-26T01:50:27.007585+0000","last_peered":"2024-04-26T01:50:27.007585+0000","last_clean":"2024-04-26T01:50:27.007585+0000","last_became_active":"2024-04-26T01:50:27.000945+0000","last_became_peered":"2024-04-26T01:50:27.000945+0000","last_unstale":"2024-04-26T01:50:27.007585+0000","last_undegraded":"2024-04-26T01:50:27.007585+0000","last_fullsized":"2024-04-26T01:50:27.007585+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:39:02.484719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.837252+0000","last_change":"2024-04-26T01:49:37.645936+0000","last_active":"2024-04-26T01:50:27.837252+0000","last_peered":"2024-04-26T01:50:27.837252+0000","last_clean":"2024-04-26T01:50:27.837252+0000","last_became_active":"2024-04-26T01:49:37.645535+0000","last_became_peered":"2024-04-26T01:49:37.645535+0000","last_unstale":"2024-04-26T01:50:27.837252+0000","last_undegraded":"2024-04-26T01:50:27.837252+0000","last_fullsized":"2024-04-26T01:50:27.837252+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:45:04.434415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"90'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.999301+0000","last_change":"2024-04-26T01:49:33.589904+0000","last_active":"2024-04-26T01:50:26.999301+0000","last_peered":"2024-04-26T01:50:26.999301+0000","last_clean":"2024-04-26T01:50:26.999301+0000","last_became_active":"2024-04-26T01:49:33.589332+0000","last_became_peered":"2024-04-26T01:49:33.589332+0000","last_unstale":"2024-04-26T01:50:26.999301+0000","last_undegraded":"2024-04-26T01:50:26.999301+0000","last_fullsized":"2024-04-26T01:50:26.999301+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:28:07.972583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.068950+0000","last_change":"2024-04-26T01:49:37.652093+0000","last_active":"2024-04-26T01:50:27.068950+0000","last_peered":"2024-04-26T01:50:27.068950+0000","last_clean":"2024-04-26T01:50:27.068950+0000","last_became_active":"2024-04-26T01:49:37.651506+0000","last_became_peered":"2024-04-26T01:49:37.651506+0000","last_unstale":"2024-04-26T01:50:27.068950+0000","last_undegraded":"2024-04-26T01:50:27.068950+0000","last_fullsized":"2024-04-26T01:50:27.068950+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:01:31.421498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":71,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:35.846423+0000","last_change":"2024-04-26T01:49:33.587652+0000","last_active":"2024-04-26T01:50:35.846423+0000","last_peered":"2024-04-26T01:50:35.846423+0000","last_clean":"2024-04-26T01:50:35.846423+0000","last_became_active":"2024-04-26T01:49:33.587294+0000","last_became_peered":"2024-04-26T01:49:33.587294+0000","last_unstale":"2024-04-26T01:50:35.846423+0000","last_undegraded":"2024-04-26T01:50:35.846423+0000","last_fullsized":"2024-04-26T01:50:35.846423+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:32:46.803415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":39,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998426+0000","last_change":"2024-04-26T01:49:37.647899+0000","last_active":"2024-04-26T01:50:26.998426+0000","last_peered":"2024-04-26T01:50:26.998426+0000","last_clean":"2024-04-26T01:50:26.998426+0000","last_became_active":"2024-04-26T01:49:37.647420+0000","last_became_peered":"2024-04-26T01:49:37.647420+0000","last_unstale":"2024-04-26T01:50:26.998426+0000","last_undegraded":"2024-04-26T01:50:26.998426+0000","last_fullsized":"2024-04-26T01:50:26.998426+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:16:39.262842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.992761+0000","last_change":"2024-04-26T01:49:33.596096+0000","last_active":"2024-04-26T01:50:26.992761+0000","last_peered":"2024-04-26T01:50:26.992761+0000","last_clean":"2024-04-26T01:50:26.992761+0000","last_became_active":"2024-04-26T01:49:33.595604+0000","last_became_peered":"2024-04-26T01:49:33.595604+0000","last_unstale":"2024-04-26T01:50:26.992761+0000","last_undegraded":"2024-04-26T01:50:26.992761+0000","last_fullsized":"2024-04-26T01:50:26.992761+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:44:26.497448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.068784+0000","last_change":"2024-04-26T01:49:37.665001+0000","last_active":"2024-04-26T01:50:27.068784+0000","last_peered":"2024-04-26T01:50:27.068784+0000","last_clean":"2024-04-26T01:50:27.068784+0000","last_became_active":"2024-04-26T01:49:37.664598+0000","last_became_peered":"2024-04-26T01:49:37.664598+0000","last_unstale":"2024-04-26T01:50:27.068784+0000","last_undegraded":"2024-04-26T01:50:27.068784+0000","last_fullsized":"2024-04-26T01:50:27.068784+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:54:28.299606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.005914+0000","last_change":"2024-04-26T01:49:33.602593+0000","last_active":"2024-04-26T01:50:27.005914+0000","last_peered":"2024-04-26T01:50:27.005914+0000","last_clean":"2024-04-26T01:50:27.005914+0000","last_became_active":"2024-04-26T01:49:33.602223+0000","last_became_peered":"2024-04-26T01:49:33.602223+0000","last_unstale":"2024-04-26T01:50:27.005914+0000","last_undegraded":"2024-04-26T01:50:27.005914+0000","last_fullsized":"2024-04-26T01:50:27.005914+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:43:29.404719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.991756+0000","last_change":"2024-04-26T01:49:37.662351+0000","last_active":"2024-04-26T01:50:26.991756+0000","last_peered":"2024-04-26T01:50:26.991756+0000","last_clean":"2024-04-26T01:50:26.991756+0000","last_became_active":"2024-04-26T01:49:37.661900+0000","last_became_peered":"2024-04-26T01:49:37.661900+0000","last_unstale":"2024-04-26T01:50:26.991756+0000","last_undegraded":"2024-04-26T01:50:26.991756+0000","last_fullsized":"2024-04-26T01:50:26.991756+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:49:10.878580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.068414+0000","last_change":"2024-04-26T01:49:33.586538+0000","last_active":"2024-04-26T01:50:27.068414+0000","last_peered":"2024-04-26T01:50:27.068414+0000","last_clean":"2024-04-26T01:50:27.068414+0000","last_became_active":"2024-04-26T01:49:33.586127+0000","last_became_peered":"2024-04-26T01:49:33.586127+0000","last_unstale":"2024-04-26T01:50:27.068414+0000","last_undegraded":"2024-04-26T01:50:27.068414+0000","last_fullsized":"2024-04-26T01:50:27.068414+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:05.599479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.997197+0000","last_change":"2024-04-26T01:49:37.657154+0000","last_active":"2024-04-26T01:50:26.997197+0000","last_peered":"2024-04-26T01:50:26.997197+0000","last_clean":"2024-04-26T01:50:26.997197+0000","last_became_active":"2024-04-26T01:49:37.656814+0000","last_became_peered":"2024-04-26T01:49:37.656814+0000","last_unstale":"2024-04-26T01:50:26.997197+0000","last_undegraded":"2024-04-26T01:50:26.997197+0000","last_fullsized":"2024-04-26T01:50:26.997197+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:26:54.538267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":63,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:33.842974+0000","last_change":"2024-04-26T01:49:33.602441+0000","last_active":"2024-04-26T01:50:33.842974+0000","last_peered":"2024-04-26T01:50:33.842974+0000","last_clean":"2024-04-26T01:50:33.842974+0000","last_became_active":"2024-04-26T01:49:33.602116+0000","last_became_peered":"2024-04-26T01:49:33.602116+0000","last_unstale":"2024-04-26T01:50:33.842974+0000","last_undegraded":"2024-04-26T01:50:33.842974+0000","last_fullsized":"2024-04-26T01:50:33.842974+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:45:52.541043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":34,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.837562+0000","last_change":"2024-04-26T01:49:37.645921+0000","last_active":"2024-04-26T01:50:27.837562+0000","last_peered":"2024-04-26T01:50:27.837562+0000","last_clean":"2024-04-26T01:50:27.837562+0000","last_became_active":"2024-04-26T01:49:37.645590+0000","last_became_peered":"2024-04-26T01:49:37.645590+0000","last_unstale":"2024-04-26T01:50:27.837562+0000","last_undegraded":"2024-04-26T01:50:27.837562+0000","last_fullsized":"2024-04-26T01:50:27.837562+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:00:08.931051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998582+0000","last_change":"2024-04-26T01:49:33.589350+0000","last_active":"2024-04-26T01:50:26.998582+0000","last_peered":"2024-04-26T01:50:26.998582+0000","last_clean":"2024-04-26T01:50:26.998582+0000","last_became_active":"2024-04-26T01:49:33.588943+0000","last_became_peered":"2024-04-26T01:49:33.588943+0000","last_unstale":"2024-04-26T01:50:26.998582+0000","last_undegraded":"2024-04-26T01:50:26.998582+0000","last_fullsized":"2024-04-26T01:50:26.998582+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:36:36.104012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.993335+0000","last_change":"2024-04-26T01:49:37.648458+0000","last_active":"2024-04-26T01:50:26.993335+0000","last_peered":"2024-04-26T01:50:26.993335+0000","last_clean":"2024-04-26T01:50:26.993335+0000","last_became_active":"2024-04-26T01:49:37.648037+0000","last_became_peered":"2024-04-26T01:49:37.648037+0000","last_unstale":"2024-04-26T01:50:26.993335+0000","last_undegraded":"2024-04-26T01:50:26.993335+0000","last_fullsized":"2024-04-26T01:50:26.993335+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:28:05.711795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.069074+0000","last_change":"2024-04-26T01:49:34.189193+0000","last_active":"2024-04-26T01:50:27.069074+0000","last_peered":"2024-04-26T01:50:27.069074+0000","last_clean":"2024-04-26T01:50:27.069074+0000","last_became_active":"2024-04-26T01:49:34.188827+0000","last_became_peered":"2024-04-26T01:49:34.188827+0000","last_unstale":"2024-04-26T01:50:27.069074+0000","last_undegraded":"2024-04-26T01:50:27.069074+0000","last_fullsized":"2024-04-26T01:50:27.069074+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:02:04.583215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.837295+0000","last_change":"2024-04-26T01:49:37.644208+0000","last_active":"2024-04-26T01:50:27.837295+0000","last_peered":"2024-04-26T01:50:27.837295+0000","last_clean":"2024-04-26T01:50:27.837295+0000","last_became_active":"2024-04-26T01:49:37.643787+0000","last_became_peered":"2024-04-26T01:49:37.643787+0000","last_unstale":"2024-04-26T01:50:27.837295+0000","last_undegraded":"2024-04-26T01:50:27.837295+0000","last_fullsized":"2024-04-26T01:50:27.837295+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:42:21.243628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"92'28","reported_seq":131,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:34.845501+0000","last_change":"2024-04-26T01:49:33.590965+0000","last_active":"2024-04-26T01:50:34.845501+0000","last_peered":"2024-04-26T01:50:34.845501+0000","last_clean":"2024-04-26T01:50:34.845501+0000","last_became_active":"2024-04-26T01:49:33.590543+0000","last_became_peered":"2024-04-26T01:49:33.590543+0000","last_unstale":"2024-04-26T01:50:34.845501+0000","last_undegraded":"2024-04-26T01:50:34.845501+0000","last_fullsized":"2024-04-26T01:50:34.845501+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:26.627549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60347,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":77,"num_read_kb":97,"num_write":29,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998608+0000","last_change":"2024-04-26T01:49:37.641172+0000","last_active":"2024-04-26T01:50:26.998608+0000","last_peered":"2024-04-26T01:50:26.998608+0000","last_clean":"2024-04-26T01:50:26.998608+0000","last_became_active":"2024-04-26T01:49:37.640866+0000","last_became_peered":"2024-04-26T01:49:37.640866+0000","last_unstale":"2024-04-26T01:50:26.998608+0000","last_undegraded":"2024-04-26T01:50:26.998608+0000","last_fullsized":"2024-04-26T01:50:26.998608+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:29:12.461005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.999009+0000","last_change":"2024-04-26T01:49:33.598466+0000","last_active":"2024-04-26T01:50:26.999009+0000","last_peered":"2024-04-26T01:50:26.999009+0000","last_clean":"2024-04-26T01:50:26.999009+0000","last_became_active":"2024-04-26T01:49:33.598042+0000","last_became_peered":"2024-04-26T01:49:33.598042+0000","last_unstale":"2024-04-26T01:50:26.999009+0000","last_undegraded":"2024-04-26T01:50:26.999009+0000","last_fullsized":"2024-04-26T01:50:26.999009+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:55:54.901016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.004884+0000","last_change":"2024-04-26T01:49:37.665772+0000","last_active":"2024-04-26T01:50:27.004884+0000","last_peered":"2024-04-26T01:50:27.004884+0000","last_clean":"2024-04-26T01:50:27.004884+0000","last_became_active":"2024-04-26T01:49:37.665524+0000","last_became_peered":"2024-04-26T01:49:37.665524+0000","last_unstale":"2024-04-26T01:50:27.004884+0000","last_undegraded":"2024-04-26T01:50:27.004884+0000","last_fullsized":"2024-04-26T01:50:27.004884+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:00:19.729098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.068588+0000","last_change":"2024-04-26T01:49:33.599456+0000","last_active":"2024-04-26T01:50:27.068588+0000","last_peered":"2024-04-26T01:50:27.068588+0000","last_clean":"2024-04-26T01:50:27.068588+0000","last_became_active":"2024-04-26T01:49:33.599101+0000","last_became_peered":"2024-04-26T01:49:33.599101+0000","last_unstale":"2024-04-26T01:50:27.068588+0000","last_undegraded":"2024-04-26T01:50:27.068588+0000","last_fullsized":"2024-04-26T01:50:27.068588+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:14:35.923011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998636+0000","last_change":"2024-04-26T01:49:37.647193+0000","last_active":"2024-04-26T01:50:26.998636+0000","last_peered":"2024-04-26T01:50:26.998636+0000","last_clean":"2024-04-26T01:50:26.998636+0000","last_became_active":"2024-04-26T01:49:37.646881+0000","last_became_peered":"2024-04-26T01:49:37.646881+0000","last_unstale":"2024-04-26T01:50:26.998636+0000","last_undegraded":"2024-04-26T01:50:26.998636+0000","last_fullsized":"2024-04-26T01:50:26.998636+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:50:10.006986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'4","reported_seq":65,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:31.854857+0000","last_change":"2024-04-26T01:49:33.590398+0000","last_active":"2024-04-26T01:50:31.854857+0000","last_peered":"2024-04-26T01:50:31.854857+0000","last_clean":"2024-04-26T01:50:31.854857+0000","last_became_active":"2024-04-26T01:49:33.590033+0000","last_became_peered":"2024-04-26T01:49:33.590033+0000","last_unstale":"2024-04-26T01:50:31.854857+0000","last_undegraded":"2024-04-26T01:50:31.854857+0000","last_fullsized":"2024-04-26T01:50:31.854857+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:27:43.203781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998130+0000","last_change":"2024-04-26T01:49:37.649784+0000","last_active":"2024-04-26T01:50:26.998130+0000","last_peered":"2024-04-26T01:50:26.998130+0000","last_clean":"2024-04-26T01:50:26.998130+0000","last_became_active":"2024-04-26T01:49:37.649431+0000","last_became_peered":"2024-04-26T01:49:37.649431+0000","last_unstale":"2024-04-26T01:50:26.998130+0000","last_undegraded":"2024-04-26T01:50:26.998130+0000","last_fullsized":"2024-04-26T01:50:26.998130+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:45:19.045983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998164+0000","last_change":"2024-04-26T01:49:34.182987+0000","last_active":"2024-04-26T01:50:26.998164+0000","last_peered":"2024-04-26T01:50:26.998164+0000","last_clean":"2024-04-26T01:50:26.998164+0000","last_became_active":"2024-04-26T01:49:34.182600+0000","last_became_peered":"2024-04-26T01:49:34.182600+0000","last_unstale":"2024-04-26T01:50:26.998164+0000","last_undegraded":"2024-04-26T01:50:26.998164+0000","last_fullsized":"2024-04-26T01:50:26.998164+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:05:52.774170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.068490+0000","last_change":"2024-04-26T01:49:37.646097+0000","last_active":"2024-04-26T01:50:27.068490+0000","last_peered":"2024-04-26T01:50:27.068490+0000","last_clean":"2024-04-26T01:50:27.068490+0000","last_became_active":"2024-04-26T01:49:37.645763+0000","last_became_peered":"2024-04-26T01:49:37.645763+0000","last_unstale":"2024-04-26T01:50:27.068490+0000","last_undegraded":"2024-04-26T01:50:27.068490+0000","last_fullsized":"2024-04-26T01:50:27.068490+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:42:37.159950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998991+0000","last_change":"2024-04-26T01:49:33.591457+0000","last_active":"2024-04-26T01:50:26.998991+0000","last_peered":"2024-04-26T01:50:26.998991+0000","last_clean":"2024-04-26T01:50:26.998991+0000","last_became_active":"2024-04-26T01:49:33.590655+0000","last_became_peered":"2024-04-26T01:49:33.590655+0000","last_unstale":"2024-04-26T01:50:26.998991+0000","last_undegraded":"2024-04-26T01:50:26.998991+0000","last_fullsized":"2024-04-26T01:50:26.998991+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:33:18.803796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.995971+0000","last_change":"2024-04-26T01:49:37.646423+0000","last_active":"2024-04-26T01:50:26.995971+0000","last_peered":"2024-04-26T01:50:26.995971+0000","last_clean":"2024-04-26T01:50:26.995971+0000","last_became_active":"2024-04-26T01:49:37.646046+0000","last_became_peered":"2024-04-26T01:49:37.646046+0000","last_unstale":"2024-04-26T01:50:26.995971+0000","last_undegraded":"2024-04-26T01:50:26.995971+0000","last_fullsized":"2024-04-26T01:50:26.995971+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:26:03.570479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.997279+0000","last_change":"2024-04-26T01:49:34.192848+0000","last_active":"2024-04-26T01:50:26.997279+0000","last_peered":"2024-04-26T01:50:26.997279+0000","last_clean":"2024-04-26T01:50:26.997279+0000","last_became_active":"2024-04-26T01:49:34.190449+0000","last_became_peered":"2024-04-26T01:49:34.190449+0000","last_unstale":"2024-04-26T01:50:26.997279+0000","last_undegraded":"2024-04-26T01:50:26.997279+0000","last_fullsized":"2024-04-26T01:50:26.997279+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:05:22.094991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'1","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.993052+0000","last_change":"2024-04-26T01:49:37.667147+0000","last_active":"2024-04-26T01:50:26.993052+0000","last_peered":"2024-04-26T01:50:26.993052+0000","last_clean":"2024-04-26T01:50:26.993052+0000","last_became_active":"2024-04-26T01:49:37.666752+0000","last_became_peered":"2024-04-26T01:49:37.666752+0000","last_unstale":"2024-04-26T01:50:26.993052+0000","last_undegraded":"2024-04-26T01:50:26.993052+0000","last_fullsized":"2024-04-26T01:50:26.993052+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:18:31.778916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998424+0000","last_change":"2024-04-26T01:49:33.596155+0000","last_active":"2024-04-26T01:50:26.998424+0000","last_peered":"2024-04-26T01:50:26.998424+0000","last_clean":"2024-04-26T01:50:26.998424+0000","last_became_active":"2024-04-26T01:49:33.595760+0000","last_became_peered":"2024-04-26T01:49:33.595760+0000","last_unstale":"2024-04-26T01:50:26.998424+0000","last_undegraded":"2024-04-26T01:50:26.998424+0000","last_fullsized":"2024-04-26T01:50:26.998424+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:07:57.208116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.992926+0000","last_change":"2024-04-26T01:49:37.641660+0000","last_active":"2024-04-26T01:50:26.992926+0000","last_peered":"2024-04-26T01:50:26.992926+0000","last_clean":"2024-04-26T01:50:26.992926+0000","last_became_active":"2024-04-26T01:49:37.641302+0000","last_became_peered":"2024-04-26T01:49:37.641302+0000","last_unstale":"2024-04-26T01:50:26.992926+0000","last_undegraded":"2024-04-26T01:50:26.992926+0000","last_fullsized":"2024-04-26T01:50:26.992926+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:32:59.889189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.068616+0000","last_change":"2024-04-26T01:49:37.643083+0000","last_active":"2024-04-26T01:50:27.068616+0000","last_peered":"2024-04-26T01:50:27.068616+0000","last_clean":"2024-04-26T01:50:27.068616+0000","last_became_active":"2024-04-26T01:49:37.642548+0000","last_became_peered":"2024-04-26T01:49:37.642548+0000","last_unstale":"2024-04-26T01:50:27.068616+0000","last_undegraded":"2024-04-26T01:50:27.068616+0000","last_fullsized":"2024-04-26T01:50:27.068616+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:09:06.136510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":65,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:34.859754+0000","last_change":"2024-04-26T01:49:34.185069+0000","last_active":"2024-04-26T01:50:34.859754+0000","last_peered":"2024-04-26T01:50:34.859754+0000","last_clean":"2024-04-26T01:50:34.859754+0000","last_became_active":"2024-04-26T01:49:34.183989+0000","last_became_peered":"2024-04-26T01:49:34.183989+0000","last_unstale":"2024-04-26T01:50:34.859754+0000","last_undegraded":"2024-04-26T01:50:34.859754+0000","last_fullsized":"2024-04-26T01:50:34.859754+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:04:26.327960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.992763+0000","last_change":"2024-04-26T01:49:37.655512+0000","last_active":"2024-04-26T01:50:26.992763+0000","last_peered":"2024-04-26T01:50:26.992763+0000","last_clean":"2024-04-26T01:50:26.992763+0000","last_became_active":"2024-04-26T01:49:37.655142+0000","last_became_peered":"2024-04-26T01:49:37.655142+0000","last_unstale":"2024-04-26T01:50:26.992763+0000","last_undegraded":"2024-04-26T01:50:26.992763+0000","last_fullsized":"2024-04-26T01:50:26.992763+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:02:58.090885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.485748+0000","last_change":"2024-04-26T01:49:33.599578+0000","last_active":"2024-04-26T01:50:27.485748+0000","last_peered":"2024-04-26T01:50:27.485748+0000","last_clean":"2024-04-26T01:50:27.485748+0000","last_became_active":"2024-04-26T01:49:33.599010+0000","last_became_peered":"2024-04-26T01:49:33.599010+0000","last_unstale":"2024-04-26T01:50:27.485748+0000","last_undegraded":"2024-04-26T01:50:27.485748+0000","last_fullsized":"2024-04-26T01:50:27.485748+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:16:09.925304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":360,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998346+0000","last_change":"2024-04-26T01:47:59.584819+0000","last_active":"2024-04-26T01:50:26.998346+0000","last_peered":"2024-04-26T01:50:26.998346+0000","last_clean":"2024-04-26T01:50:26.998346+0000","last_became_active":"2024-04-26T01:47:59.584356+0000","last_became_peered":"2024-04-26T01:47:59.584356+0000","last_unstale":"2024-04-26T01:50:26.998346+0000","last_undegraded":"2024-04-26T01:50:26.998346+0000","last_fullsized":"2024-04-26T01:50:26.998346+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:44:11.653568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:44:11.653568+0000","last_clean_scrub_stamp":"2024-04-26T01:44:11.653568+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:25:11.168583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.008911+0000","last_change":"2024-04-26T01:50:27.008911+0000","last_active":"2024-04-26T01:50:27.008911+0000","last_peered":"2024-04-26T01:50:27.008911+0000","last_clean":"2024-04-26T01:50:27.008911+0000","last_became_active":"2024-04-26T01:50:27.008507+0000","last_became_peered":"2024-04-26T01:50:27.008507+0000","last_unstale":"2024-04-26T01:50:27.008911+0000","last_undegraded":"2024-04-26T01:50:27.008911+0000","last_fullsized":"2024-04-26T01:50:27.008911+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:21:14.985935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998428+0000","last_change":"2024-04-26T01:49:33.590171+0000","last_active":"2024-04-26T01:50:26.998428+0000","last_peered":"2024-04-26T01:50:26.998428+0000","last_clean":"2024-04-26T01:50:26.998428+0000","last_became_active":"2024-04-26T01:49:33.589312+0000","last_became_peered":"2024-04-26T01:49:33.589312+0000","last_unstale":"2024-04-26T01:50:26.998428+0000","last_undegraded":"2024-04-26T01:50:26.998428+0000","last_fullsized":"2024-04-26T01:50:26.998428+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:01:25.441527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998743+0000","last_change":"2024-04-26T01:49:34.190338+0000","last_active":"2024-04-26T01:50:26.998743+0000","last_peered":"2024-04-26T01:50:26.998743+0000","last_clean":"2024-04-26T01:50:26.998743+0000","last_became_active":"2024-04-26T01:49:34.189984+0000","last_became_peered":"2024-04-26T01:49:34.189984+0000","last_unstale":"2024-04-26T01:50:26.998743+0000","last_undegraded":"2024-04-26T01:50:26.998743+0000","last_fullsized":"2024-04-26T01:50:26.998743+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:58.588012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.006025+0000","last_change":"2024-04-26T01:49:37.649368+0000","last_active":"2024-04-26T01:50:27.006025+0000","last_peered":"2024-04-26T01:50:27.006025+0000","last_clean":"2024-04-26T01:50:27.006025+0000","last_became_active":"2024-04-26T01:49:37.648987+0000","last_became_peered":"2024-04-26T01:49:37.648987+0000","last_unstale":"2024-04-26T01:50:27.006025+0000","last_undegraded":"2024-04-26T01:50:27.006025+0000","last_fullsized":"2024-04-26T01:50:27.006025+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:04:59.080703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"92'11","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998931+0000","last_change":"2024-04-26T01:49:33.600813+0000","last_active":"2024-04-26T01:50:26.998931+0000","last_peered":"2024-04-26T01:50:26.998931+0000","last_clean":"2024-04-26T01:50:26.998931+0000","last_became_active":"2024-04-26T01:49:33.600361+0000","last_became_peered":"2024-04-26T01:49:33.600361+0000","last_unstale":"2024-04-26T01:50:26.998931+0000","last_undegraded":"2024-04-26T01:50:26.998931+0000","last_fullsized":"2024-04-26T01:50:26.998931+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:02:58.638449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.993970+0000","last_change":"2024-04-26T01:49:37.659381+0000","last_active":"2024-04-26T01:50:26.993970+0000","last_peered":"2024-04-26T01:50:26.993970+0000","last_clean":"2024-04-26T01:50:26.993970+0000","last_became_active":"2024-04-26T01:49:37.656205+0000","last_became_peered":"2024-04-26T01:49:37.656205+0000","last_unstale":"2024-04-26T01:50:26.993970+0000","last_undegraded":"2024-04-26T01:50:26.993970+0000","last_fullsized":"2024-04-26T01:50:26.993970+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:02:10.103330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.486009+0000","last_change":"2024-04-26T01:49:33.854732+0000","last_active":"2024-04-26T01:50:27.486009+0000","last_peered":"2024-04-26T01:50:27.486009+0000","last_clean":"2024-04-26T01:50:27.486009+0000","last_became_active":"2024-04-26T01:49:33.854396+0000","last_became_peered":"2024-04-26T01:49:33.854396+0000","last_unstale":"2024-04-26T01:50:27.486009+0000","last_undegraded":"2024-04-26T01:50:27.486009+0000","last_fullsized":"2024-04-26T01:50:27.486009+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:12:37.183725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.993580+0000","last_change":"2024-04-26T01:49:37.656366+0000","last_active":"2024-04-26T01:50:26.993580+0000","last_peered":"2024-04-26T01:50:26.993580+0000","last_clean":"2024-04-26T01:50:26.993580+0000","last_became_active":"2024-04-26T01:49:37.655869+0000","last_became_peered":"2024-04-26T01:49:37.655869+0000","last_unstale":"2024-04-26T01:50:26.993580+0000","last_undegraded":"2024-04-26T01:50:26.993580+0000","last_fullsized":"2024-04-26T01:50:26.993580+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:01:57.083315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.837142+0000","last_change":"2024-04-26T01:49:34.186070+0000","last_active":"2024-04-26T01:50:27.837142+0000","last_peered":"2024-04-26T01:50:27.837142+0000","last_clean":"2024-04-26T01:50:27.837142+0000","last_became_active":"2024-04-26T01:49:34.185742+0000","last_became_peered":"2024-04-26T01:49:34.185742+0000","last_unstale":"2024-04-26T01:50:27.837142+0000","last_undegraded":"2024-04-26T01:50:27.837142+0000","last_fullsized":"2024-04-26T01:50:27.837142+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:05.870056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998112+0000","last_change":"2024-04-26T01:49:37.640938+0000","last_active":"2024-04-26T01:50:26.998112+0000","last_peered":"2024-04-26T01:50:26.998112+0000","last_clean":"2024-04-26T01:50:26.998112+0000","last_became_active":"2024-04-26T01:49:37.640608+0000","last_became_peered":"2024-04-26T01:49:37.640608+0000","last_unstale":"2024-04-26T01:50:26.998112+0000","last_undegraded":"2024-04-26T01:50:26.998112+0000","last_fullsized":"2024-04-26T01:50:26.998112+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:30:30.460641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.991892+0000","last_change":"2024-04-26T01:49:33.604152+0000","last_active":"2024-04-26T01:50:26.991892+0000","last_peered":"2024-04-26T01:50:26.991892+0000","last_clean":"2024-04-26T01:50:26.991892+0000","last_became_active":"2024-04-26T01:49:33.603479+0000","last_became_peered":"2024-04-26T01:49:33.603479+0000","last_unstale":"2024-04-26T01:50:26.991892+0000","last_undegraded":"2024-04-26T01:50:26.991892+0000","last_fullsized":"2024-04-26T01:50:26.991892+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:25:13.685392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.995729+0000","last_change":"2024-04-26T01:49:37.645152+0000","last_active":"2024-04-26T01:50:26.995729+0000","last_peered":"2024-04-26T01:50:26.995729+0000","last_clean":"2024-04-26T01:50:26.995729+0000","last_became_active":"2024-04-26T01:49:37.644789+0000","last_became_peered":"2024-04-26T01:49:37.644789+0000","last_unstale":"2024-04-26T01:50:26.995729+0000","last_undegraded":"2024-04-26T01:50:26.995729+0000","last_fullsized":"2024-04-26T01:50:26.995729+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:20:01.494222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.010485+0000","last_change":"2024-04-26T01:50:27.009973+0000","last_active":"2024-04-26T01:50:27.010485+0000","last_peered":"2024-04-26T01:50:27.010485+0000","last_clean":"2024-04-26T01:50:27.010485+0000","last_became_active":"2024-04-26T01:50:27.001678+0000","last_became_peered":"2024-04-26T01:50:27.001678+0000","last_unstale":"2024-04-26T01:50:27.010485+0000","last_undegraded":"2024-04-26T01:50:27.010485+0000","last_fullsized":"2024-04-26T01:50:27.010485+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:59:54.807792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.992365+0000","last_change":"2024-04-26T01:49:37.662665+0000","last_active":"2024-04-26T01:50:26.992365+0000","last_peered":"2024-04-26T01:50:26.992365+0000","last_clean":"2024-04-26T01:50:26.992365+0000","last_became_active":"2024-04-26T01:49:37.662216+0000","last_became_peered":"2024-04-26T01:49:37.662216+0000","last_unstale":"2024-04-26T01:50:26.992365+0000","last_undegraded":"2024-04-26T01:50:26.992365+0000","last_fullsized":"2024-04-26T01:50:26.992365+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:10:54.724858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998326+0000","last_change":"2024-04-26T01:49:33.592078+0000","last_active":"2024-04-26T01:50:26.998326+0000","last_peered":"2024-04-26T01:50:26.998326+0000","last_clean":"2024-04-26T01:50:26.998326+0000","last_became_active":"2024-04-26T01:49:33.590265+0000","last_became_peered":"2024-04-26T01:49:33.590265+0000","last_unstale":"2024-04-26T01:50:26.998326+0000","last_undegraded":"2024-04-26T01:50:26.998326+0000","last_fullsized":"2024-04-26T01:50:26.998326+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:44:27.163705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998574+0000","last_change":"2024-04-26T01:49:37.645943+0000","last_active":"2024-04-26T01:50:26.998574+0000","last_peered":"2024-04-26T01:50:26.998574+0000","last_clean":"2024-04-26T01:50:26.998574+0000","last_became_active":"2024-04-26T01:49:37.645586+0000","last_became_peered":"2024-04-26T01:49:37.645586+0000","last_unstale":"2024-04-26T01:50:26.998574+0000","last_undegraded":"2024-04-26T01:50:26.998574+0000","last_fullsized":"2024-04-26T01:50:26.998574+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:40.682344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.000204+0000","last_change":"2024-04-26T01:49:34.187115+0000","last_active":"2024-04-26T01:50:27.000204+0000","last_peered":"2024-04-26T01:50:27.000204+0000","last_clean":"2024-04-26T01:50:27.000204+0000","last_became_active":"2024-04-26T01:49:34.186660+0000","last_became_peered":"2024-04-26T01:49:34.186660+0000","last_unstale":"2024-04-26T01:50:27.000204+0000","last_undegraded":"2024-04-26T01:50:27.000204+0000","last_fullsized":"2024-04-26T01:50:27.000204+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:56:32.274752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998810+0000","last_change":"2024-04-26T01:49:37.650382+0000","last_active":"2024-04-26T01:50:26.998810+0000","last_peered":"2024-04-26T01:50:26.998810+0000","last_clean":"2024-04-26T01:50:26.998810+0000","last_became_active":"2024-04-26T01:49:37.649545+0000","last_became_peered":"2024-04-26T01:49:37.649545+0000","last_unstale":"2024-04-26T01:50:26.998810+0000","last_undegraded":"2024-04-26T01:50:26.998810+0000","last_fullsized":"2024-04-26T01:50:26.998810+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:43:16.291311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.068936+0000","last_change":"2024-04-26T01:49:33.595881+0000","last_active":"2024-04-26T01:50:27.068936+0000","last_peered":"2024-04-26T01:50:27.068936+0000","last_clean":"2024-04-26T01:50:27.068936+0000","last_became_active":"2024-04-26T01:49:33.595480+0000","last_became_peered":"2024-04-26T01:49:33.595480+0000","last_unstale":"2024-04-26T01:50:27.068936+0000","last_undegraded":"2024-04-26T01:50:27.068936+0000","last_fullsized":"2024-04-26T01:50:27.068936+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:30:58.028230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'1","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998549+0000","last_change":"2024-04-26T01:49:37.660765+0000","last_active":"2024-04-26T01:50:26.998549+0000","last_peered":"2024-04-26T01:50:26.998549+0000","last_clean":"2024-04-26T01:50:26.998549+0000","last_became_active":"2024-04-26T01:49:37.660475+0000","last_became_peered":"2024-04-26T01:49:37.660475+0000","last_unstale":"2024-04-26T01:50:26.998549+0000","last_undegraded":"2024-04-26T01:50:26.998549+0000","last_fullsized":"2024-04-26T01:50:26.998549+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:36:00.483005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"90'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.006174+0000","last_change":"2024-04-26T01:49:33.598247+0000","last_active":"2024-04-26T01:50:27.006174+0000","last_peered":"2024-04-26T01:50:27.006174+0000","last_clean":"2024-04-26T01:50:27.006174+0000","last_became_active":"2024-04-26T01:49:33.597796+0000","last_became_peered":"2024-04-26T01:49:33.597796+0000","last_unstale":"2024-04-26T01:50:27.006174+0000","last_undegraded":"2024-04-26T01:50:27.006174+0000","last_fullsized":"2024-04-26T01:50:27.006174+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:53:38.994798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998820+0000","last_change":"2024-04-26T01:49:37.648903+0000","last_active":"2024-04-26T01:50:26.998820+0000","last_peered":"2024-04-26T01:50:26.998820+0000","last_clean":"2024-04-26T01:50:26.998820+0000","last_became_active":"2024-04-26T01:49:37.648570+0000","last_became_peered":"2024-04-26T01:49:37.648570+0000","last_unstale":"2024-04-26T01:50:26.998820+0000","last_undegraded":"2024-04-26T01:50:26.998820+0000","last_fullsized":"2024-04-26T01:50:26.998820+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:33:12.678513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.995800+0000","last_change":"2024-04-26T01:49:37.648485+0000","last_active":"2024-04-26T01:50:26.995800+0000","last_peered":"2024-04-26T01:50:26.995800+0000","last_clean":"2024-04-26T01:50:26.995800+0000","last_became_active":"2024-04-26T01:49:37.648129+0000","last_became_peered":"2024-04-26T01:49:37.648129+0000","last_unstale":"2024-04-26T01:50:26.995800+0000","last_undegraded":"2024-04-26T01:50:26.995800+0000","last_fullsized":"2024-04-26T01:50:26.995800+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:56:12.047821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.999299+0000","last_change":"2024-04-26T01:49:37.645929+0000","last_active":"2024-04-26T01:50:26.999299+0000","last_peered":"2024-04-26T01:50:26.999299+0000","last_clean":"2024-04-26T01:50:26.999299+0000","last_became_active":"2024-04-26T01:49:37.645473+0000","last_became_peered":"2024-04-26T01:49:37.645473+0000","last_unstale":"2024-04-26T01:50:26.999299+0000","last_undegraded":"2024-04-26T01:50:26.999299+0000","last_fullsized":"2024-04-26T01:50:26.999299+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:39:56.687973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998457+0000","last_change":"2024-04-26T01:49:37.657494+0000","last_active":"2024-04-26T01:50:26.998457+0000","last_peered":"2024-04-26T01:50:26.998457+0000","last_clean":"2024-04-26T01:50:26.998457+0000","last_became_active":"2024-04-26T01:49:37.657121+0000","last_became_peered":"2024-04-26T01:49:37.657121+0000","last_unstale":"2024-04-26T01:50:26.998457+0000","last_undegraded":"2024-04-26T01:50:26.998457+0000","last_fullsized":"2024-04-26T01:50:26.998457+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:50:44.691179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.992636+0000","last_change":"2024-04-26T01:49:37.646012+0000","last_active":"2024-04-26T01:50:26.992636+0000","last_peered":"2024-04-26T01:50:26.992636+0000","last_clean":"2024-04-26T01:50:26.992636+0000","last_became_active":"2024-04-26T01:49:37.645695+0000","last_became_peered":"2024-04-26T01:49:37.645695+0000","last_unstale":"2024-04-26T01:50:26.992636+0000","last_undegraded":"2024-04-26T01:50:26.992636+0000","last_fullsized":"2024-04-26T01:50:26.992636+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:40:51.117523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.005878+0000","last_change":"2024-04-26T01:49:37.646503+0000","last_active":"2024-04-26T01:50:27.005878+0000","last_peered":"2024-04-26T01:50:27.005878+0000","last_clean":"2024-04-26T01:50:27.005878+0000","last_became_active":"2024-04-26T01:49:37.645200+0000","last_became_peered":"2024-04-26T01:49:37.645200+0000","last_unstale":"2024-04-26T01:50:27.005878+0000","last_undegraded":"2024-04-26T01:50:27.005878+0000","last_fullsized":"2024-04-26T01:50:27.005878+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:31:10.957199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.002908+0000","last_change":"2024-04-26T01:50:27.002908+0000","last_active":"2024-04-26T01:50:27.002908+0000","last_peered":"2024-04-26T01:50:27.002908+0000","last_clean":"2024-04-26T01:50:27.002908+0000","last_became_active":"2024-04-26T01:50:27.002454+0000","last_became_peered":"2024-04-26T01:50:27.002454+0000","last_unstale":"2024-04-26T01:50:27.002908+0000","last_undegraded":"2024-04-26T01:50:27.002908+0000","last_fullsized":"2024-04-26T01:50:27.002908+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:18:01.848427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998823+0000","last_change":"2024-04-26T01:49:37.642676+0000","last_active":"2024-04-26T01:50:26.998823+0000","last_peered":"2024-04-26T01:50:26.998823+0000","last_clean":"2024-04-26T01:50:26.998823+0000","last_became_active":"2024-04-26T01:49:37.642302+0000","last_became_peered":"2024-04-26T01:49:37.642302+0000","last_unstale":"2024-04-26T01:50:26.998823+0000","last_undegraded":"2024-04-26T01:50:26.998823+0000","last_fullsized":"2024-04-26T01:50:26.998823+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:51:43.454649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.003914+0000","last_change":"2024-04-26T01:50:27.003914+0000","last_active":"2024-04-26T01:50:27.003914+0000","last_peered":"2024-04-26T01:50:27.003914+0000","last_clean":"2024-04-26T01:50:27.003914+0000","last_became_active":"2024-04-26T01:50:27.002344+0000","last_became_peered":"2024-04-26T01:50:27.002344+0000","last_unstale":"2024-04-26T01:50:27.003914+0000","last_undegraded":"2024-04-26T01:50:27.003914+0000","last_fullsized":"2024-04-26T01:50:27.003914+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:46:02.101508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.068475+0000","last_change":"2024-04-26T01:49:37.663306+0000","last_active":"2024-04-26T01:50:27.068475+0000","last_peered":"2024-04-26T01:50:27.068475+0000","last_clean":"2024-04-26T01:50:27.068475+0000","last_became_active":"2024-04-26T01:49:37.662999+0000","last_became_peered":"2024-04-26T01:49:37.662999+0000","last_unstale":"2024-04-26T01:50:27.068475+0000","last_undegraded":"2024-04-26T01:50:27.068475+0000","last_fullsized":"2024-04-26T01:50:27.068475+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:55.155096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.991831+0000","last_change":"2024-04-26T01:49:37.655157+0000","last_active":"2024-04-26T01:50:26.991831+0000","last_peered":"2024-04-26T01:50:26.991831+0000","last_clean":"2024-04-26T01:50:26.991831+0000","last_became_active":"2024-04-26T01:49:37.654810+0000","last_became_peered":"2024-04-26T01:49:37.654810+0000","last_unstale":"2024-04-26T01:50:26.991831+0000","last_undegraded":"2024-04-26T01:50:26.991831+0000","last_fullsized":"2024-04-26T01:50:26.991831+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:28:04.012382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.837305+0000","last_change":"2024-04-26T01:49:37.667228+0000","last_active":"2024-04-26T01:50:27.837305+0000","last_peered":"2024-04-26T01:50:27.837305+0000","last_clean":"2024-04-26T01:50:27.837305+0000","last_became_active":"2024-04-26T01:49:37.666835+0000","last_became_peered":"2024-04-26T01:49:37.666835+0000","last_unstale":"2024-04-26T01:50:27.837305+0000","last_undegraded":"2024-04-26T01:50:27.837305+0000","last_fullsized":"2024-04-26T01:50:27.837305+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:54:16.902836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.005906+0000","last_change":"2024-04-26T01:49:37.658231+0000","last_active":"2024-04-26T01:50:27.005906+0000","last_peered":"2024-04-26T01:50:27.005906+0000","last_clean":"2024-04-26T01:50:27.005906+0000","last_became_active":"2024-04-26T01:49:37.657842+0000","last_became_peered":"2024-04-26T01:49:37.657842+0000","last_unstale":"2024-04-26T01:50:27.005906+0000","last_undegraded":"2024-04-26T01:50:27.005906+0000","last_fullsized":"2024-04-26T01:50:27.005906+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:02:01.445262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.995790+0000","last_change":"2024-04-26T01:49:37.645221+0000","last_active":"2024-04-26T01:50:26.995790+0000","last_peered":"2024-04-26T01:50:26.995790+0000","last_clean":"2024-04-26T01:50:26.995790+0000","last_became_active":"2024-04-26T01:49:37.644814+0000","last_became_peered":"2024-04-26T01:49:37.644814+0000","last_unstale":"2024-04-26T01:50:26.995790+0000","last_undegraded":"2024-04-26T01:50:26.995790+0000","last_fullsized":"2024-04-26T01:50:26.995790+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:03:17.850633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64739,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":234,"num_read_kb":186,"num_write":164,"num_write_kb":111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":194217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318303,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":995328,"data_stored":486380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579937,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28668,"kb_used_data":1652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716484,"statfs":{"total":95995035648,"available":95965679616,"internally_reserved":0,"allocated":1691648,"data_stored":1138599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743463,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1011712,"data_stored":486616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005102,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":486536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266738,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28592,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716560,"statfs":{"total":95995035648,"available":95965757440,"internally_reserved":0,"allocated":1605632,"data_stored":1078857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430265,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":999424,"data_stored":486005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1060864,"data_stored":546145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1019904,"data_stored":487276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411408,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1085440,"data_stored":547886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117065,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1015808,"data_stored":486316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855429,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1003520,"data_stored":486833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691903,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28600,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716552,"statfs":{"total":95995035648,"available":95965749248,"internally_reserved":0,"allocated":1617920,"data_stored":1078150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":60347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T01:50:38.526 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T01:50:38.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:38 smithi151 ceph-mon[28057]: pgmap v358: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 786 B/s wr, 7 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-26T01:50:38.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:38 smithi151 ceph-mon[28057]: from='client.24646 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:50:39.011 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:50:39.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:38 smithi172 ceph-mon[32785]: pgmap v358: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 786 B/s wr, 7 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-26T01:50:39.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:38 smithi172 ceph-mon[32785]: from='client.24646 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:50:39.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:38 smithi165 ceph-mon[32648]: pgmap v358: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 786 B/s wr, 7 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-26T01:50:39.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:38 smithi165 ceph-mon[32648]: from='client.24646 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:50:40.770 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:40 smithi151 ceph-mon[28057]: pgmap v359: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-26T01:50:41.213 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:40 smithi165 ceph-mon[32648]: pgmap v359: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-26T01:50:41.213 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:40 smithi172 ceph-mon[32785]: pgmap v359: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-26T01:50:41.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:41.299+0000 7f6265e00700 1 -- 172.21.15.151:0/2263829847 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6260101940 msgr2=0x7f6260101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:41.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:41.299+0000 7f6265e00700 1 --2- 172.21.15.151:0/2263829847 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6260101940 0x7f6260101dc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f624c009990 tx=0x7f624c009ca0 comp rx=0 tx=0).stop 2024-04-26T01:50:41.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:41.300+0000 7f6265e00700 1 -- 172.21.15.151:0/2263829847 shutdown_connections 2024-04-26T01:50:41.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:41.300+0000 7f6265e00700 1 --2- 172.21.15.151:0/2263829847 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6260102300 0x7f626010e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:41.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:41.300+0000 7f6265e00700 1 --2- 172.21.15.151:0/2263829847 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6260107990 0x7f6260107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:41.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:41.300+0000 7f6265e00700 1 --2- 172.21.15.151:0/2263829847 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6260101940 0x7f6260101dc0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:41.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:41.300+0000 7f6265e00700 1 -- 172.21.15.151:0/2263829847 >> 172.21.15.151:0/2263829847 conn(0x7f62600788f0 msgr2=0x7f6260078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:41.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:41.300+0000 7f6265e00700 1 -- 172.21.15.151:0/2263829847 shutdown_connections 2024-04-26T01:50:41.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:41.300+0000 7f6265e00700 1 -- 172.21.15.151:0/2263829847 wait complete. 2024-04-26T01:50:41.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:41.301+0000 7f6265e00700 1 Processor -- start 2024-04-26T01:50:41.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:41.302+0000 7f6265e00700 1 -- start start 2024-04-26T01:50:41.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:41.302+0000 7f6265e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6260101940 0x7f62601991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:41.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:41.302+0000 7f6265e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6260102300 0x7f6260199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:41.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:41.302+0000 7f6265e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6260107990 0x7f626019daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:41.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:41.302+0000 7f6265e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6260111b30 con 0x7f6260102300 2024-04-26T01:50:41.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:41.302+0000 7f6265e00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f62601119b0 con 0x7f6260101940 2024-04-26T01:50:41.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:41.302+0000 7f6265e00700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6260111cb0 con 0x7f6260107990 2024-04-26T01:50:41.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:41.302+0000 7f625e000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6260102300 0x7f6260199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:41.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:41.302+0000 7f625f400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6260107990 0x7f626019daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:41.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:41.302+0000 7f625ea00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6260101940 0x7f62601991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:41.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:41.302+0000 7f625f400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6260107990 0x7f626019daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:54582/0 (socket says 172.21.15.151:54582) 2024-04-26T01:50:41.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:41.302+0000 7f625f400700 1 -- 172.21.15.151:0/1439569551 learned_addr learned my addr 172.21.15.151:0/1439569551 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:50:41.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:41.303+0000 7f625ea00700 1 -- 172.21.15.151:0/1439569551 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6260107990 msgr2=0x7f626019daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:41.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:41.303+0000 7f625ea00700 1 --2- 172.21.15.151:0/1439569551 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6260107990 0x7f626019daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:41.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:41.303+0000 7f625ea00700 1 -- 172.21.15.151:0/1439569551 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6260102300 msgr2=0x7f6260199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:41.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:41.303+0000 7f625ea00700 1 --2- 172.21.15.151:0/1439569551 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6260102300 0x7f6260199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:41.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:41.303+0000 7f625ea00700 1 -- 172.21.15.151:0/1439569551 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f626019e1e0 con 0x7f6260101940 2024-04-26T01:50:41.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:41.303+0000 7f625f400700 1 --2- 172.21.15.151:0/1439569551 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6260107990 0x7f626019daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:50:41.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:41.303+0000 7f625ea00700 1 --2- 172.21.15.151:0/1439569551 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6260101940 0x7f62601991d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f625000e9f0 tx=0x7f625000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:41.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:41.303+0000 7f6264a00700 1 -- 172.21.15.151:0/1439569551 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625000c9c0 con 0x7f6260101940 2024-04-26T01:50:41.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:41.304+0000 7f6264a00700 1 -- 172.21.15.151:0/1439569551 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f625000cb60 con 0x7f6260101940 2024-04-26T01:50:41.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:41.304+0000 7f6264a00700 1 -- 172.21.15.151:0/1439569551 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6250018680 con 0x7f6260101940 2024-04-26T01:50:41.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:41.304+0000 7f6265e00700 1 -- 172.21.15.151:0/1439569551 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f626019e4d0 con 0x7f6260101940 2024-04-26T01:50:41.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:41.304+0000 7f6265e00700 1 -- 172.21.15.151:0/1439569551 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62601a5d50 con 0x7f6260101940 2024-04-26T01:50:41.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:41.305+0000 7f6265e00700 1 -- 172.21.15.151:0/1439569551 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6260069730 con 0x7f6260101940 2024-04-26T01:50:41.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:41.307+0000 7f6264a00700 1 -- 172.21.15.151:0/1439569551 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f6250010430 con 0x7f6260101940 2024-04-26T01:50:41.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:41.308+0000 7f6264a00700 1 --2- 172.21.15.151:0/1439569551 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f62440a3cd0 0x7f62440a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:41.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:41.308+0000 7f6264a00700 1 -- 172.21.15.151:0/1439569551 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f62500d09a0 con 0x7f6260101940 2024-04-26T01:50:41.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:41.308+0000 7f625e000700 1 --2- 172.21.15.151:0/1439569551 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f62440a3cd0 0x7f62440a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:41.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:41.309+0000 7f625e000700 1 --2- 172.21.15.151:0/1439569551 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f62440a3cd0 0x7f62440a6190 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6260079180 tx=0x7f624c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:41.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:41.311+0000 7f6264a00700 1 -- 172.21.15.151:0/1439569551 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f625009d220 con 0x7f6260101940 2024-04-26T01:50:41.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:41.527+0000 7f6265e00700 1 -- 172.21.15.151:0/1439569551 --> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f626019a530 con 0x7f62440a3cd0 2024-04-26T01:50:41.549 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:41.551+0000 7f6264a00700 1 -- 172.21.15.151:0/1439569551 <== mgr.14152 v2:172.21.15.151:6800/2251256505 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258727 (secure 0 0 0) 0x7f624c0672c0 con 0x7f62440a3cd0 2024-04-26T01:50:41.555 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:50:41.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:41.559+0000 7f6265e00700 1 -- 172.21.15.151:0/1439569551 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f62440a3cd0 msgr2=0x7f62440a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:41.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:41.559+0000 7f6265e00700 1 --2- 172.21.15.151:0/1439569551 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f62440a3cd0 0x7f62440a6190 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6260079180 tx=0x7f624c005fd0 comp rx=0 tx=0).stop 2024-04-26T01:50:41.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:41.559+0000 7f6265e00700 1 -- 172.21.15.151:0/1439569551 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6260101940 msgr2=0x7f62601991d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:41.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:41.559+0000 7f6265e00700 1 --2- 172.21.15.151:0/1439569551 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6260101940 0x7f62601991d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f625000e9f0 tx=0x7f625000ed00 comp rx=0 tx=0).stop 2024-04-26T01:50:41.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:41.559+0000 7f6265e00700 1 -- 172.21.15.151:0/1439569551 shutdown_connections 2024-04-26T01:50:41.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:41.559+0000 7f6265e00700 1 --2- 172.21.15.151:0/1439569551 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6260107990 0x7f626019daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:41.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:41.559+0000 7f6265e00700 1 --2- 172.21.15.151:0/1439569551 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6260101940 0x7f62601991d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:41.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:41.559+0000 7f6265e00700 1 --2- 172.21.15.151:0/1439569551 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f62440a3cd0 0x7f62440a6190 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:41.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:41.559+0000 7f6265e00700 1 --2- 172.21.15.151:0/1439569551 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6260102300 0x7f6260199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:41.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:41.559+0000 7f6265e00700 1 -- 172.21.15.151:0/1439569551 >> 172.21.15.151:0/1439569551 conn(0x7f62600788f0 msgr2=0x7f626006ac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:41.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:41.559+0000 7f6265e00700 1 -- 172.21.15.151:0/1439569551 shutdown_connections 2024-04-26T01:50:41.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:41.559+0000 7f6265e00700 1 -- 172.21.15.151:0/1439569551 wait complete. 2024-04-26T01:50:41.559 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-04-26T01:50:42.503 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":360,"stamp":"2024-04-26T01:50:41.406173+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655341,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":395,"num_read_kb":320,"num_write":347,"num_write_kb":3051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":278,"ondisk_log_size":278,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":387956,"kb_used_data":13788,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124553868,"statfs":{"total":1151940427776,"available":1151543160832,"internally_reserved":0,"allocated":14118912,"data_stored":7795599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5395,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":82,"num_read_kb":52,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004885"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.996096+0000","last_change":"2024-04-26T01:49:37.641666+0000","last_active":"2024-04-26T01:50:26.996096+0000","last_peered":"2024-04-26T01:50:26.996096+0000","last_clean":"2024-04-26T01:50:26.996096+0000","last_became_active":"2024-04-26T01:49:37.641310+0000","last_became_peered":"2024-04-26T01:49:37.641310+0000","last_unstale":"2024-04-26T01:50:26.996096+0000","last_undegraded":"2024-04-26T01:50:26.996096+0000","last_fullsized":"2024-04-26T01:50:26.996096+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:28:02.952851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.993389+0000","last_change":"2024-04-26T01:49:37.637607+0000","last_active":"2024-04-26T01:50:26.993389+0000","last_peered":"2024-04-26T01:50:26.993389+0000","last_clean":"2024-04-26T01:50:26.993389+0000","last_became_active":"2024-04-26T01:49:37.637283+0000","last_became_peered":"2024-04-26T01:49:37.637283+0000","last_unstale":"2024-04-26T01:50:26.993389+0000","last_undegraded":"2024-04-26T01:50:26.993389+0000","last_fullsized":"2024-04-26T01:50:26.993389+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:16:46.519381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.992971+0000","last_change":"2024-04-26T01:49:37.647769+0000","last_active":"2024-04-26T01:50:26.992971+0000","last_peered":"2024-04-26T01:50:26.992971+0000","last_clean":"2024-04-26T01:50:26.992971+0000","last_became_active":"2024-04-26T01:49:37.643750+0000","last_became_peered":"2024-04-26T01:49:37.643750+0000","last_unstale":"2024-04-26T01:50:26.992971+0000","last_undegraded":"2024-04-26T01:50:26.992971+0000","last_fullsized":"2024-04-26T01:50:26.992971+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:34:18.801267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.485851+0000","last_change":"2024-04-26T01:49:37.643291+0000","last_active":"2024-04-26T01:50:27.485851+0000","last_peered":"2024-04-26T01:50:27.485851+0000","last_clean":"2024-04-26T01:50:27.485851+0000","last_became_active":"2024-04-26T01:49:37.642947+0000","last_became_peered":"2024-04-26T01:49:37.642947+0000","last_unstale":"2024-04-26T01:50:27.485851+0000","last_undegraded":"2024-04-26T01:50:27.485851+0000","last_fullsized":"2024-04-26T01:50:27.485851+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:31:21.600533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998208+0000","last_change":"2024-04-26T01:49:37.665311+0000","last_active":"2024-04-26T01:50:26.998208+0000","last_peered":"2024-04-26T01:50:26.998208+0000","last_clean":"2024-04-26T01:50:26.998208+0000","last_became_active":"2024-04-26T01:49:37.664944+0000","last_became_peered":"2024-04-26T01:49:37.664944+0000","last_unstale":"2024-04-26T01:50:26.998208+0000","last_undegraded":"2024-04-26T01:50:26.998208+0000","last_fullsized":"2024-04-26T01:50:26.998208+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:49:50.744630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.992822+0000","last_change":"2024-04-26T01:49:37.638667+0000","last_active":"2024-04-26T01:50:26.992822+0000","last_peered":"2024-04-26T01:50:26.992822+0000","last_clean":"2024-04-26T01:50:26.992822+0000","last_became_active":"2024-04-26T01:49:37.638226+0000","last_became_peered":"2024-04-26T01:49:37.638226+0000","last_unstale":"2024-04-26T01:50:26.992822+0000","last_undegraded":"2024-04-26T01:50:26.992822+0000","last_fullsized":"2024-04-26T01:50:26.992822+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:46:17.786883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998214+0000","last_change":"2024-04-26T01:49:37.650082+0000","last_active":"2024-04-26T01:50:26.998214+0000","last_peered":"2024-04-26T01:50:26.998214+0000","last_clean":"2024-04-26T01:50:26.998214+0000","last_became_active":"2024-04-26T01:49:37.649637+0000","last_became_peered":"2024-04-26T01:49:37.649637+0000","last_unstale":"2024-04-26T01:50:26.998214+0000","last_undegraded":"2024-04-26T01:50:26.998214+0000","last_fullsized":"2024-04-26T01:50:26.998214+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:21:04.686398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.991647+0000","last_change":"2024-04-26T01:49:37.661255+0000","last_active":"2024-04-26T01:50:26.991647+0000","last_peered":"2024-04-26T01:50:26.991647+0000","last_clean":"2024-04-26T01:50:26.991647+0000","last_became_active":"2024-04-26T01:49:37.658882+0000","last_became_peered":"2024-04-26T01:49:37.658882+0000","last_unstale":"2024-04-26T01:50:26.991647+0000","last_undegraded":"2024-04-26T01:50:26.991647+0000","last_fullsized":"2024-04-26T01:50:26.991647+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:36:27.874191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.837626+0000","last_change":"2024-04-26T01:49:37.648874+0000","last_active":"2024-04-26T01:50:27.837626+0000","last_peered":"2024-04-26T01:50:27.837626+0000","last_clean":"2024-04-26T01:50:27.837626+0000","last_became_active":"2024-04-26T01:49:37.648490+0000","last_became_peered":"2024-04-26T01:49:37.648490+0000","last_unstale":"2024-04-26T01:50:27.837626+0000","last_undegraded":"2024-04-26T01:50:27.837626+0000","last_fullsized":"2024-04-26T01:50:27.837626+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:46:12.552861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.993321+0000","last_change":"2024-04-26T01:49:37.643842+0000","last_active":"2024-04-26T01:50:26.993321+0000","last_peered":"2024-04-26T01:50:26.993321+0000","last_clean":"2024-04-26T01:50:26.993321+0000","last_became_active":"2024-04-26T01:49:37.643084+0000","last_became_peered":"2024-04-26T01:49:37.643084+0000","last_unstale":"2024-04-26T01:50:26.993321+0000","last_undegraded":"2024-04-26T01:50:26.993321+0000","last_fullsized":"2024-04-26T01:50:26.993321+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:16:53.102318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998567+0000","last_change":"2024-04-26T01:49:37.644209+0000","last_active":"2024-04-26T01:50:26.998567+0000","last_peered":"2024-04-26T01:50:26.998567+0000","last_clean":"2024-04-26T01:50:26.998567+0000","last_became_active":"2024-04-26T01:49:37.643659+0000","last_became_peered":"2024-04-26T01:49:37.643659+0000","last_unstale":"2024-04-26T01:50:26.998567+0000","last_undegraded":"2024-04-26T01:50:26.998567+0000","last_fullsized":"2024-04-26T01:50:26.998567+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:24:46.363873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.001615+0000","last_change":"2024-04-26T01:50:27.001615+0000","last_active":"2024-04-26T01:50:27.001615+0000","last_peered":"2024-04-26T01:50:27.001615+0000","last_clean":"2024-04-26T01:50:27.001615+0000","last_became_active":"2024-04-26T01:50:27.001039+0000","last_became_peered":"2024-04-26T01:50:27.001039+0000","last_unstale":"2024-04-26T01:50:27.001615+0000","last_undegraded":"2024-04-26T01:50:27.001615+0000","last_fullsized":"2024-04-26T01:50:27.001615+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:42:46.148500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.006036+0000","last_change":"2024-04-26T01:49:37.651085+0000","last_active":"2024-04-26T01:50:27.006036+0000","last_peered":"2024-04-26T01:50:27.006036+0000","last_clean":"2024-04-26T01:50:27.006036+0000","last_became_active":"2024-04-26T01:49:37.649662+0000","last_became_peered":"2024-04-26T01:49:37.649662+0000","last_unstale":"2024-04-26T01:50:27.006036+0000","last_undegraded":"2024-04-26T01:50:27.006036+0000","last_fullsized":"2024-04-26T01:50:27.006036+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:41:34.537533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.992971+0000","last_change":"2024-04-26T01:49:37.648432+0000","last_active":"2024-04-26T01:50:26.992971+0000","last_peered":"2024-04-26T01:50:26.992971+0000","last_clean":"2024-04-26T01:50:26.992971+0000","last_became_active":"2024-04-26T01:49:37.648047+0000","last_became_peered":"2024-04-26T01:49:37.648047+0000","last_unstale":"2024-04-26T01:50:26.992971+0000","last_undegraded":"2024-04-26T01:50:26.992971+0000","last_fullsized":"2024-04-26T01:50:26.992971+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:00:06.843384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.992100+0000","last_change":"2024-04-26T01:49:37.644203+0000","last_active":"2024-04-26T01:50:26.992100+0000","last_peered":"2024-04-26T01:50:26.992100+0000","last_clean":"2024-04-26T01:50:26.992100+0000","last_became_active":"2024-04-26T01:49:37.643772+0000","last_became_peered":"2024-04-26T01:49:37.643772+0000","last_unstale":"2024-04-26T01:50:26.992100+0000","last_undegraded":"2024-04-26T01:50:26.992100+0000","last_fullsized":"2024-04-26T01:50:26.992100+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:52:20.707845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.837131+0000","last_change":"2024-04-26T01:49:37.647364+0000","last_active":"2024-04-26T01:50:27.837131+0000","last_peered":"2024-04-26T01:50:27.837131+0000","last_clean":"2024-04-26T01:50:27.837131+0000","last_became_active":"2024-04-26T01:49:37.647054+0000","last_became_peered":"2024-04-26T01:49:37.647054+0000","last_unstale":"2024-04-26T01:50:27.837131+0000","last_undegraded":"2024-04-26T01:50:27.837131+0000","last_fullsized":"2024-04-26T01:50:27.837131+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:53:09.631463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.996185+0000","last_change":"2024-04-26T01:49:37.641843+0000","last_active":"2024-04-26T01:50:26.996185+0000","last_peered":"2024-04-26T01:50:26.996185+0000","last_clean":"2024-04-26T01:50:26.996185+0000","last_became_active":"2024-04-26T01:49:37.641514+0000","last_became_peered":"2024-04-26T01:49:37.641514+0000","last_unstale":"2024-04-26T01:50:26.996185+0000","last_undegraded":"2024-04-26T01:50:26.996185+0000","last_fullsized":"2024-04-26T01:50:26.996185+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:24:32.261834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998685+0000","last_change":"2024-04-26T01:49:37.644750+0000","last_active":"2024-04-26T01:50:26.998685+0000","last_peered":"2024-04-26T01:50:26.998685+0000","last_clean":"2024-04-26T01:50:26.998685+0000","last_became_active":"2024-04-26T01:49:37.644086+0000","last_became_peered":"2024-04-26T01:49:37.644086+0000","last_unstale":"2024-04-26T01:50:26.998685+0000","last_undegraded":"2024-04-26T01:50:26.998685+0000","last_fullsized":"2024-04-26T01:50:26.998685+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:59:42.614846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998814+0000","last_change":"2024-04-26T01:49:37.664204+0000","last_active":"2024-04-26T01:50:26.998814+0000","last_peered":"2024-04-26T01:50:26.998814+0000","last_clean":"2024-04-26T01:50:26.998814+0000","last_became_active":"2024-04-26T01:49:37.663878+0000","last_became_peered":"2024-04-26T01:49:37.663878+0000","last_unstale":"2024-04-26T01:50:26.998814+0000","last_undegraded":"2024-04-26T01:50:26.998814+0000","last_fullsized":"2024-04-26T01:50:26.998814+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:43:49.272308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.996978+0000","last_change":"2024-04-26T01:49:33.592017+0000","last_active":"2024-04-26T01:50:26.996978+0000","last_peered":"2024-04-26T01:50:26.996978+0000","last_clean":"2024-04-26T01:50:26.996978+0000","last_became_active":"2024-04-26T01:49:33.587928+0000","last_became_peered":"2024-04-26T01:49:33.587928+0000","last_unstale":"2024-04-26T01:50:26.996978+0000","last_undegraded":"2024-04-26T01:50:26.996978+0000","last_fullsized":"2024-04-26T01:50:26.996978+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:02:04.313785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.993721+0000","last_change":"2024-04-26T01:49:37.662010+0000","last_active":"2024-04-26T01:50:26.993721+0000","last_peered":"2024-04-26T01:50:26.993721+0000","last_clean":"2024-04-26T01:50:26.993721+0000","last_became_active":"2024-04-26T01:49:37.661673+0000","last_became_peered":"2024-04-26T01:49:37.661673+0000","last_unstale":"2024-04-26T01:50:26.993721+0000","last_undegraded":"2024-04-26T01:50:26.993721+0000","last_fullsized":"2024-04-26T01:50:26.993721+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:37:34.155998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.994164+0000","last_change":"2024-04-26T01:49:34.185758+0000","last_active":"2024-04-26T01:50:26.994164+0000","last_peered":"2024-04-26T01:50:26.994164+0000","last_clean":"2024-04-26T01:50:26.994164+0000","last_became_active":"2024-04-26T01:49:34.184669+0000","last_became_peered":"2024-04-26T01:49:34.184669+0000","last_unstale":"2024-04-26T01:50:26.994164+0000","last_undegraded":"2024-04-26T01:50:26.994164+0000","last_fullsized":"2024-04-26T01:50:26.994164+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:23:23.742032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.993378+0000","last_change":"2024-04-26T01:49:37.641307+0000","last_active":"2024-04-26T01:50:26.993378+0000","last_peered":"2024-04-26T01:50:26.993378+0000","last_clean":"2024-04-26T01:50:26.993378+0000","last_became_active":"2024-04-26T01:49:37.640677+0000","last_became_peered":"2024-04-26T01:49:37.640677+0000","last_unstale":"2024-04-26T01:50:26.993378+0000","last_undegraded":"2024-04-26T01:50:26.993378+0000","last_fullsized":"2024-04-26T01:50:26.993378+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:35:56.859045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.007585+0000","last_change":"2024-04-26T01:50:27.007034+0000","last_active":"2024-04-26T01:50:27.007585+0000","last_peered":"2024-04-26T01:50:27.007585+0000","last_clean":"2024-04-26T01:50:27.007585+0000","last_became_active":"2024-04-26T01:50:27.000945+0000","last_became_peered":"2024-04-26T01:50:27.000945+0000","last_unstale":"2024-04-26T01:50:27.007585+0000","last_undegraded":"2024-04-26T01:50:27.007585+0000","last_fullsized":"2024-04-26T01:50:27.007585+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:39:02.484719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.837252+0000","last_change":"2024-04-26T01:49:37.645936+0000","last_active":"2024-04-26T01:50:27.837252+0000","last_peered":"2024-04-26T01:50:27.837252+0000","last_clean":"2024-04-26T01:50:27.837252+0000","last_became_active":"2024-04-26T01:49:37.645535+0000","last_became_peered":"2024-04-26T01:49:37.645535+0000","last_unstale":"2024-04-26T01:50:27.837252+0000","last_undegraded":"2024-04-26T01:50:27.837252+0000","last_fullsized":"2024-04-26T01:50:27.837252+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:45:04.434415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"90'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.999301+0000","last_change":"2024-04-26T01:49:33.589904+0000","last_active":"2024-04-26T01:50:26.999301+0000","last_peered":"2024-04-26T01:50:26.999301+0000","last_clean":"2024-04-26T01:50:26.999301+0000","last_became_active":"2024-04-26T01:49:33.589332+0000","last_became_peered":"2024-04-26T01:49:33.589332+0000","last_unstale":"2024-04-26T01:50:26.999301+0000","last_undegraded":"2024-04-26T01:50:26.999301+0000","last_fullsized":"2024-04-26T01:50:26.999301+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:28:07.972583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.068950+0000","last_change":"2024-04-26T01:49:37.652093+0000","last_active":"2024-04-26T01:50:27.068950+0000","last_peered":"2024-04-26T01:50:27.068950+0000","last_clean":"2024-04-26T01:50:27.068950+0000","last_became_active":"2024-04-26T01:49:37.651506+0000","last_became_peered":"2024-04-26T01:49:37.651506+0000","last_unstale":"2024-04-26T01:50:27.068950+0000","last_undegraded":"2024-04-26T01:50:27.068950+0000","last_fullsized":"2024-04-26T01:50:27.068950+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:01:31.421498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":71,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:35.846423+0000","last_change":"2024-04-26T01:49:33.587652+0000","last_active":"2024-04-26T01:50:35.846423+0000","last_peered":"2024-04-26T01:50:35.846423+0000","last_clean":"2024-04-26T01:50:35.846423+0000","last_became_active":"2024-04-26T01:49:33.587294+0000","last_became_peered":"2024-04-26T01:49:33.587294+0000","last_unstale":"2024-04-26T01:50:35.846423+0000","last_undegraded":"2024-04-26T01:50:35.846423+0000","last_fullsized":"2024-04-26T01:50:35.846423+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:32:46.803415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":39,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998426+0000","last_change":"2024-04-26T01:49:37.647899+0000","last_active":"2024-04-26T01:50:26.998426+0000","last_peered":"2024-04-26T01:50:26.998426+0000","last_clean":"2024-04-26T01:50:26.998426+0000","last_became_active":"2024-04-26T01:49:37.647420+0000","last_became_peered":"2024-04-26T01:49:37.647420+0000","last_unstale":"2024-04-26T01:50:26.998426+0000","last_undegraded":"2024-04-26T01:50:26.998426+0000","last_fullsized":"2024-04-26T01:50:26.998426+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:16:39.262842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.992761+0000","last_change":"2024-04-26T01:49:33.596096+0000","last_active":"2024-04-26T01:50:26.992761+0000","last_peered":"2024-04-26T01:50:26.992761+0000","last_clean":"2024-04-26T01:50:26.992761+0000","last_became_active":"2024-04-26T01:49:33.595604+0000","last_became_peered":"2024-04-26T01:49:33.595604+0000","last_unstale":"2024-04-26T01:50:26.992761+0000","last_undegraded":"2024-04-26T01:50:26.992761+0000","last_fullsized":"2024-04-26T01:50:26.992761+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:44:26.497448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.068784+0000","last_change":"2024-04-26T01:49:37.665001+0000","last_active":"2024-04-26T01:50:27.068784+0000","last_peered":"2024-04-26T01:50:27.068784+0000","last_clean":"2024-04-26T01:50:27.068784+0000","last_became_active":"2024-04-26T01:49:37.664598+0000","last_became_peered":"2024-04-26T01:49:37.664598+0000","last_unstale":"2024-04-26T01:50:27.068784+0000","last_undegraded":"2024-04-26T01:50:27.068784+0000","last_fullsized":"2024-04-26T01:50:27.068784+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:54:28.299606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.005914+0000","last_change":"2024-04-26T01:49:33.602593+0000","last_active":"2024-04-26T01:50:27.005914+0000","last_peered":"2024-04-26T01:50:27.005914+0000","last_clean":"2024-04-26T01:50:27.005914+0000","last_became_active":"2024-04-26T01:49:33.602223+0000","last_became_peered":"2024-04-26T01:49:33.602223+0000","last_unstale":"2024-04-26T01:50:27.005914+0000","last_undegraded":"2024-04-26T01:50:27.005914+0000","last_fullsized":"2024-04-26T01:50:27.005914+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:43:29.404719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.991756+0000","last_change":"2024-04-26T01:49:37.662351+0000","last_active":"2024-04-26T01:50:26.991756+0000","last_peered":"2024-04-26T01:50:26.991756+0000","last_clean":"2024-04-26T01:50:26.991756+0000","last_became_active":"2024-04-26T01:49:37.661900+0000","last_became_peered":"2024-04-26T01:49:37.661900+0000","last_unstale":"2024-04-26T01:50:26.991756+0000","last_undegraded":"2024-04-26T01:50:26.991756+0000","last_fullsized":"2024-04-26T01:50:26.991756+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:49:10.878580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.068414+0000","last_change":"2024-04-26T01:49:33.586538+0000","last_active":"2024-04-26T01:50:27.068414+0000","last_peered":"2024-04-26T01:50:27.068414+0000","last_clean":"2024-04-26T01:50:27.068414+0000","last_became_active":"2024-04-26T01:49:33.586127+0000","last_became_peered":"2024-04-26T01:49:33.586127+0000","last_unstale":"2024-04-26T01:50:27.068414+0000","last_undegraded":"2024-04-26T01:50:27.068414+0000","last_fullsized":"2024-04-26T01:50:27.068414+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:05.599479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.997197+0000","last_change":"2024-04-26T01:49:37.657154+0000","last_active":"2024-04-26T01:50:26.997197+0000","last_peered":"2024-04-26T01:50:26.997197+0000","last_clean":"2024-04-26T01:50:26.997197+0000","last_became_active":"2024-04-26T01:49:37.656814+0000","last_became_peered":"2024-04-26T01:49:37.656814+0000","last_unstale":"2024-04-26T01:50:26.997197+0000","last_undegraded":"2024-04-26T01:50:26.997197+0000","last_fullsized":"2024-04-26T01:50:26.997197+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:26:54.538267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":68,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:38.849567+0000","last_change":"2024-04-26T01:49:33.602441+0000","last_active":"2024-04-26T01:50:38.849567+0000","last_peered":"2024-04-26T01:50:38.849567+0000","last_clean":"2024-04-26T01:50:38.849567+0000","last_became_active":"2024-04-26T01:49:33.602116+0000","last_became_peered":"2024-04-26T01:49:33.602116+0000","last_unstale":"2024-04-26T01:50:38.849567+0000","last_undegraded":"2024-04-26T01:50:38.849567+0000","last_fullsized":"2024-04-26T01:50:38.849567+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:45:52.541043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":39,"num_read_kb":39,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.837562+0000","last_change":"2024-04-26T01:49:37.645921+0000","last_active":"2024-04-26T01:50:27.837562+0000","last_peered":"2024-04-26T01:50:27.837562+0000","last_clean":"2024-04-26T01:50:27.837562+0000","last_became_active":"2024-04-26T01:49:37.645590+0000","last_became_peered":"2024-04-26T01:49:37.645590+0000","last_unstale":"2024-04-26T01:50:27.837562+0000","last_undegraded":"2024-04-26T01:50:27.837562+0000","last_fullsized":"2024-04-26T01:50:27.837562+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:00:08.931051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998582+0000","last_change":"2024-04-26T01:49:33.589350+0000","last_active":"2024-04-26T01:50:26.998582+0000","last_peered":"2024-04-26T01:50:26.998582+0000","last_clean":"2024-04-26T01:50:26.998582+0000","last_became_active":"2024-04-26T01:49:33.588943+0000","last_became_peered":"2024-04-26T01:49:33.588943+0000","last_unstale":"2024-04-26T01:50:26.998582+0000","last_undegraded":"2024-04-26T01:50:26.998582+0000","last_fullsized":"2024-04-26T01:50:26.998582+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:36:36.104012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.993335+0000","last_change":"2024-04-26T01:49:37.648458+0000","last_active":"2024-04-26T01:50:26.993335+0000","last_peered":"2024-04-26T01:50:26.993335+0000","last_clean":"2024-04-26T01:50:26.993335+0000","last_became_active":"2024-04-26T01:49:37.648037+0000","last_became_peered":"2024-04-26T01:49:37.648037+0000","last_unstale":"2024-04-26T01:50:26.993335+0000","last_undegraded":"2024-04-26T01:50:26.993335+0000","last_fullsized":"2024-04-26T01:50:26.993335+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:28:05.711795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.069074+0000","last_change":"2024-04-26T01:49:34.189193+0000","last_active":"2024-04-26T01:50:27.069074+0000","last_peered":"2024-04-26T01:50:27.069074+0000","last_clean":"2024-04-26T01:50:27.069074+0000","last_became_active":"2024-04-26T01:49:34.188827+0000","last_became_peered":"2024-04-26T01:49:34.188827+0000","last_unstale":"2024-04-26T01:50:27.069074+0000","last_undegraded":"2024-04-26T01:50:27.069074+0000","last_fullsized":"2024-04-26T01:50:27.069074+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:02:04.583215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.837295+0000","last_change":"2024-04-26T01:49:37.644208+0000","last_active":"2024-04-26T01:50:27.837295+0000","last_peered":"2024-04-26T01:50:27.837295+0000","last_clean":"2024-04-26T01:50:27.837295+0000","last_became_active":"2024-04-26T01:49:37.643787+0000","last_became_peered":"2024-04-26T01:49:37.643787+0000","last_unstale":"2024-04-26T01:50:27.837295+0000","last_undegraded":"2024-04-26T01:50:27.837295+0000","last_fullsized":"2024-04-26T01:50:27.837295+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:42:21.243628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"92'28","reported_seq":141,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:39.851222+0000","last_change":"2024-04-26T01:49:33.590965+0000","last_active":"2024-04-26T01:50:39.851222+0000","last_peered":"2024-04-26T01:50:39.851222+0000","last_clean":"2024-04-26T01:50:39.851222+0000","last_became_active":"2024-04-26T01:49:33.590543+0000","last_became_peered":"2024-04-26T01:49:33.590543+0000","last_unstale":"2024-04-26T01:50:39.851222+0000","last_undegraded":"2024-04-26T01:50:39.851222+0000","last_fullsized":"2024-04-26T01:50:39.851222+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:26.627549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60347,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":87,"num_read_kb":102,"num_write":29,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998608+0000","last_change":"2024-04-26T01:49:37.641172+0000","last_active":"2024-04-26T01:50:26.998608+0000","last_peered":"2024-04-26T01:50:26.998608+0000","last_clean":"2024-04-26T01:50:26.998608+0000","last_became_active":"2024-04-26T01:49:37.640866+0000","last_became_peered":"2024-04-26T01:49:37.640866+0000","last_unstale":"2024-04-26T01:50:26.998608+0000","last_undegraded":"2024-04-26T01:50:26.998608+0000","last_fullsized":"2024-04-26T01:50:26.998608+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:29:12.461005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.999009+0000","last_change":"2024-04-26T01:49:33.598466+0000","last_active":"2024-04-26T01:50:26.999009+0000","last_peered":"2024-04-26T01:50:26.999009+0000","last_clean":"2024-04-26T01:50:26.999009+0000","last_became_active":"2024-04-26T01:49:33.598042+0000","last_became_peered":"2024-04-26T01:49:33.598042+0000","last_unstale":"2024-04-26T01:50:26.999009+0000","last_undegraded":"2024-04-26T01:50:26.999009+0000","last_fullsized":"2024-04-26T01:50:26.999009+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:55:54.901016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.004884+0000","last_change":"2024-04-26T01:49:37.665772+0000","last_active":"2024-04-26T01:50:27.004884+0000","last_peered":"2024-04-26T01:50:27.004884+0000","last_clean":"2024-04-26T01:50:27.004884+0000","last_became_active":"2024-04-26T01:49:37.665524+0000","last_became_peered":"2024-04-26T01:49:37.665524+0000","last_unstale":"2024-04-26T01:50:27.004884+0000","last_undegraded":"2024-04-26T01:50:27.004884+0000","last_fullsized":"2024-04-26T01:50:27.004884+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:00:19.729098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.068588+0000","last_change":"2024-04-26T01:49:33.599456+0000","last_active":"2024-04-26T01:50:27.068588+0000","last_peered":"2024-04-26T01:50:27.068588+0000","last_clean":"2024-04-26T01:50:27.068588+0000","last_became_active":"2024-04-26T01:49:33.599101+0000","last_became_peered":"2024-04-26T01:49:33.599101+0000","last_unstale":"2024-04-26T01:50:27.068588+0000","last_undegraded":"2024-04-26T01:50:27.068588+0000","last_fullsized":"2024-04-26T01:50:27.068588+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:14:35.923011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998636+0000","last_change":"2024-04-26T01:49:37.647193+0000","last_active":"2024-04-26T01:50:26.998636+0000","last_peered":"2024-04-26T01:50:26.998636+0000","last_clean":"2024-04-26T01:50:26.998636+0000","last_became_active":"2024-04-26T01:49:37.646881+0000","last_became_peered":"2024-04-26T01:49:37.646881+0000","last_unstale":"2024-04-26T01:50:26.998636+0000","last_undegraded":"2024-04-26T01:50:26.998636+0000","last_fullsized":"2024-04-26T01:50:26.998636+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:50:10.006986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'4","reported_seq":70,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:36.864307+0000","last_change":"2024-04-26T01:49:33.590398+0000","last_active":"2024-04-26T01:50:36.864307+0000","last_peered":"2024-04-26T01:50:36.864307+0000","last_clean":"2024-04-26T01:50:36.864307+0000","last_became_active":"2024-04-26T01:49:33.590033+0000","last_became_peered":"2024-04-26T01:49:33.590033+0000","last_unstale":"2024-04-26T01:50:36.864307+0000","last_undegraded":"2024-04-26T01:50:36.864307+0000","last_fullsized":"2024-04-26T01:50:36.864307+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:27:43.203781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":41,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998130+0000","last_change":"2024-04-26T01:49:37.649784+0000","last_active":"2024-04-26T01:50:26.998130+0000","last_peered":"2024-04-26T01:50:26.998130+0000","last_clean":"2024-04-26T01:50:26.998130+0000","last_became_active":"2024-04-26T01:49:37.649431+0000","last_became_peered":"2024-04-26T01:49:37.649431+0000","last_unstale":"2024-04-26T01:50:26.998130+0000","last_undegraded":"2024-04-26T01:50:26.998130+0000","last_fullsized":"2024-04-26T01:50:26.998130+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:45:19.045983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998164+0000","last_change":"2024-04-26T01:49:34.182987+0000","last_active":"2024-04-26T01:50:26.998164+0000","last_peered":"2024-04-26T01:50:26.998164+0000","last_clean":"2024-04-26T01:50:26.998164+0000","last_became_active":"2024-04-26T01:49:34.182600+0000","last_became_peered":"2024-04-26T01:49:34.182600+0000","last_unstale":"2024-04-26T01:50:26.998164+0000","last_undegraded":"2024-04-26T01:50:26.998164+0000","last_fullsized":"2024-04-26T01:50:26.998164+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:05:52.774170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.068490+0000","last_change":"2024-04-26T01:49:37.646097+0000","last_active":"2024-04-26T01:50:27.068490+0000","last_peered":"2024-04-26T01:50:27.068490+0000","last_clean":"2024-04-26T01:50:27.068490+0000","last_became_active":"2024-04-26T01:49:37.645763+0000","last_became_peered":"2024-04-26T01:49:37.645763+0000","last_unstale":"2024-04-26T01:50:27.068490+0000","last_undegraded":"2024-04-26T01:50:27.068490+0000","last_fullsized":"2024-04-26T01:50:27.068490+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:42:37.159950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998991+0000","last_change":"2024-04-26T01:49:33.591457+0000","last_active":"2024-04-26T01:50:26.998991+0000","last_peered":"2024-04-26T01:50:26.998991+0000","last_clean":"2024-04-26T01:50:26.998991+0000","last_became_active":"2024-04-26T01:49:33.590655+0000","last_became_peered":"2024-04-26T01:49:33.590655+0000","last_unstale":"2024-04-26T01:50:26.998991+0000","last_undegraded":"2024-04-26T01:50:26.998991+0000","last_fullsized":"2024-04-26T01:50:26.998991+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:33:18.803796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.995971+0000","last_change":"2024-04-26T01:49:37.646423+0000","last_active":"2024-04-26T01:50:26.995971+0000","last_peered":"2024-04-26T01:50:26.995971+0000","last_clean":"2024-04-26T01:50:26.995971+0000","last_became_active":"2024-04-26T01:49:37.646046+0000","last_became_peered":"2024-04-26T01:49:37.646046+0000","last_unstale":"2024-04-26T01:50:26.995971+0000","last_undegraded":"2024-04-26T01:50:26.995971+0000","last_fullsized":"2024-04-26T01:50:26.995971+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:26:03.570479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.997279+0000","last_change":"2024-04-26T01:49:34.192848+0000","last_active":"2024-04-26T01:50:26.997279+0000","last_peered":"2024-04-26T01:50:26.997279+0000","last_clean":"2024-04-26T01:50:26.997279+0000","last_became_active":"2024-04-26T01:49:34.190449+0000","last_became_peered":"2024-04-26T01:49:34.190449+0000","last_unstale":"2024-04-26T01:50:26.997279+0000","last_undegraded":"2024-04-26T01:50:26.997279+0000","last_fullsized":"2024-04-26T01:50:26.997279+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:05:22.094991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'1","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.993052+0000","last_change":"2024-04-26T01:49:37.667147+0000","last_active":"2024-04-26T01:50:26.993052+0000","last_peered":"2024-04-26T01:50:26.993052+0000","last_clean":"2024-04-26T01:50:26.993052+0000","last_became_active":"2024-04-26T01:49:37.666752+0000","last_became_peered":"2024-04-26T01:49:37.666752+0000","last_unstale":"2024-04-26T01:50:26.993052+0000","last_undegraded":"2024-04-26T01:50:26.993052+0000","last_fullsized":"2024-04-26T01:50:26.993052+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:18:31.778916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998424+0000","last_change":"2024-04-26T01:49:33.596155+0000","last_active":"2024-04-26T01:50:26.998424+0000","last_peered":"2024-04-26T01:50:26.998424+0000","last_clean":"2024-04-26T01:50:26.998424+0000","last_became_active":"2024-04-26T01:49:33.595760+0000","last_became_peered":"2024-04-26T01:49:33.595760+0000","last_unstale":"2024-04-26T01:50:26.998424+0000","last_undegraded":"2024-04-26T01:50:26.998424+0000","last_fullsized":"2024-04-26T01:50:26.998424+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:07:57.208116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.992926+0000","last_change":"2024-04-26T01:49:37.641660+0000","last_active":"2024-04-26T01:50:26.992926+0000","last_peered":"2024-04-26T01:50:26.992926+0000","last_clean":"2024-04-26T01:50:26.992926+0000","last_became_active":"2024-04-26T01:49:37.641302+0000","last_became_peered":"2024-04-26T01:49:37.641302+0000","last_unstale":"2024-04-26T01:50:26.992926+0000","last_undegraded":"2024-04-26T01:50:26.992926+0000","last_fullsized":"2024-04-26T01:50:26.992926+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:32:59.889189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.068616+0000","last_change":"2024-04-26T01:49:37.643083+0000","last_active":"2024-04-26T01:50:27.068616+0000","last_peered":"2024-04-26T01:50:27.068616+0000","last_clean":"2024-04-26T01:50:27.068616+0000","last_became_active":"2024-04-26T01:49:37.642548+0000","last_became_peered":"2024-04-26T01:49:37.642548+0000","last_unstale":"2024-04-26T01:50:27.068616+0000","last_undegraded":"2024-04-26T01:50:27.068616+0000","last_fullsized":"2024-04-26T01:50:27.068616+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:09:06.136510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":70,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:39.868712+0000","last_change":"2024-04-26T01:49:34.185069+0000","last_active":"2024-04-26T01:50:39.868712+0000","last_peered":"2024-04-26T01:50:39.868712+0000","last_clean":"2024-04-26T01:50:39.868712+0000","last_became_active":"2024-04-26T01:49:34.183989+0000","last_became_peered":"2024-04-26T01:49:34.183989+0000","last_unstale":"2024-04-26T01:50:39.868712+0000","last_undegraded":"2024-04-26T01:50:39.868712+0000","last_fullsized":"2024-04-26T01:50:39.868712+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:04:26.327960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":40,"num_read_kb":40,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.992763+0000","last_change":"2024-04-26T01:49:37.655512+0000","last_active":"2024-04-26T01:50:26.992763+0000","last_peered":"2024-04-26T01:50:26.992763+0000","last_clean":"2024-04-26T01:50:26.992763+0000","last_became_active":"2024-04-26T01:49:37.655142+0000","last_became_peered":"2024-04-26T01:49:37.655142+0000","last_unstale":"2024-04-26T01:50:26.992763+0000","last_undegraded":"2024-04-26T01:50:26.992763+0000","last_fullsized":"2024-04-26T01:50:26.992763+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:02:58.090885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.485748+0000","last_change":"2024-04-26T01:49:33.599578+0000","last_active":"2024-04-26T01:50:27.485748+0000","last_peered":"2024-04-26T01:50:27.485748+0000","last_clean":"2024-04-26T01:50:27.485748+0000","last_became_active":"2024-04-26T01:49:33.599010+0000","last_became_peered":"2024-04-26T01:49:33.599010+0000","last_unstale":"2024-04-26T01:50:27.485748+0000","last_undegraded":"2024-04-26T01:50:27.485748+0000","last_fullsized":"2024-04-26T01:50:27.485748+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:16:09.925304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":360,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998346+0000","last_change":"2024-04-26T01:47:59.584819+0000","last_active":"2024-04-26T01:50:26.998346+0000","last_peered":"2024-04-26T01:50:26.998346+0000","last_clean":"2024-04-26T01:50:26.998346+0000","last_became_active":"2024-04-26T01:47:59.584356+0000","last_became_peered":"2024-04-26T01:47:59.584356+0000","last_unstale":"2024-04-26T01:50:26.998346+0000","last_undegraded":"2024-04-26T01:50:26.998346+0000","last_fullsized":"2024-04-26T01:50:26.998346+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:44:11.653568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:44:11.653568+0000","last_clean_scrub_stamp":"2024-04-26T01:44:11.653568+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:25:11.168583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.008911+0000","last_change":"2024-04-26T01:50:27.008911+0000","last_active":"2024-04-26T01:50:27.008911+0000","last_peered":"2024-04-26T01:50:27.008911+0000","last_clean":"2024-04-26T01:50:27.008911+0000","last_became_active":"2024-04-26T01:50:27.008507+0000","last_became_peered":"2024-04-26T01:50:27.008507+0000","last_unstale":"2024-04-26T01:50:27.008911+0000","last_undegraded":"2024-04-26T01:50:27.008911+0000","last_fullsized":"2024-04-26T01:50:27.008911+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:21:14.985935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998428+0000","last_change":"2024-04-26T01:49:33.590171+0000","last_active":"2024-04-26T01:50:26.998428+0000","last_peered":"2024-04-26T01:50:26.998428+0000","last_clean":"2024-04-26T01:50:26.998428+0000","last_became_active":"2024-04-26T01:49:33.589312+0000","last_became_peered":"2024-04-26T01:49:33.589312+0000","last_unstale":"2024-04-26T01:50:26.998428+0000","last_undegraded":"2024-04-26T01:50:26.998428+0000","last_fullsized":"2024-04-26T01:50:26.998428+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:01:25.441527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998743+0000","last_change":"2024-04-26T01:49:34.190338+0000","last_active":"2024-04-26T01:50:26.998743+0000","last_peered":"2024-04-26T01:50:26.998743+0000","last_clean":"2024-04-26T01:50:26.998743+0000","last_became_active":"2024-04-26T01:49:34.189984+0000","last_became_peered":"2024-04-26T01:49:34.189984+0000","last_unstale":"2024-04-26T01:50:26.998743+0000","last_undegraded":"2024-04-26T01:50:26.998743+0000","last_fullsized":"2024-04-26T01:50:26.998743+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:58.588012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.006025+0000","last_change":"2024-04-26T01:49:37.649368+0000","last_active":"2024-04-26T01:50:27.006025+0000","last_peered":"2024-04-26T01:50:27.006025+0000","last_clean":"2024-04-26T01:50:27.006025+0000","last_became_active":"2024-04-26T01:49:37.648987+0000","last_became_peered":"2024-04-26T01:49:37.648987+0000","last_unstale":"2024-04-26T01:50:27.006025+0000","last_undegraded":"2024-04-26T01:50:27.006025+0000","last_fullsized":"2024-04-26T01:50:27.006025+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:04:59.080703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"92'11","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998931+0000","last_change":"2024-04-26T01:49:33.600813+0000","last_active":"2024-04-26T01:50:26.998931+0000","last_peered":"2024-04-26T01:50:26.998931+0000","last_clean":"2024-04-26T01:50:26.998931+0000","last_became_active":"2024-04-26T01:49:33.600361+0000","last_became_peered":"2024-04-26T01:49:33.600361+0000","last_unstale":"2024-04-26T01:50:26.998931+0000","last_undegraded":"2024-04-26T01:50:26.998931+0000","last_fullsized":"2024-04-26T01:50:26.998931+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:02:58.638449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.993970+0000","last_change":"2024-04-26T01:49:37.659381+0000","last_active":"2024-04-26T01:50:26.993970+0000","last_peered":"2024-04-26T01:50:26.993970+0000","last_clean":"2024-04-26T01:50:26.993970+0000","last_became_active":"2024-04-26T01:49:37.656205+0000","last_became_peered":"2024-04-26T01:49:37.656205+0000","last_unstale":"2024-04-26T01:50:26.993970+0000","last_undegraded":"2024-04-26T01:50:26.993970+0000","last_fullsized":"2024-04-26T01:50:26.993970+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:02:10.103330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.486009+0000","last_change":"2024-04-26T01:49:33.854732+0000","last_active":"2024-04-26T01:50:27.486009+0000","last_peered":"2024-04-26T01:50:27.486009+0000","last_clean":"2024-04-26T01:50:27.486009+0000","last_became_active":"2024-04-26T01:49:33.854396+0000","last_became_peered":"2024-04-26T01:49:33.854396+0000","last_unstale":"2024-04-26T01:50:27.486009+0000","last_undegraded":"2024-04-26T01:50:27.486009+0000","last_fullsized":"2024-04-26T01:50:27.486009+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:12:37.183725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.993580+0000","last_change":"2024-04-26T01:49:37.656366+0000","last_active":"2024-04-26T01:50:26.993580+0000","last_peered":"2024-04-26T01:50:26.993580+0000","last_clean":"2024-04-26T01:50:26.993580+0000","last_became_active":"2024-04-26T01:49:37.655869+0000","last_became_peered":"2024-04-26T01:49:37.655869+0000","last_unstale":"2024-04-26T01:50:26.993580+0000","last_undegraded":"2024-04-26T01:50:26.993580+0000","last_fullsized":"2024-04-26T01:50:26.993580+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:01:57.083315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.837142+0000","last_change":"2024-04-26T01:49:34.186070+0000","last_active":"2024-04-26T01:50:27.837142+0000","last_peered":"2024-04-26T01:50:27.837142+0000","last_clean":"2024-04-26T01:50:27.837142+0000","last_became_active":"2024-04-26T01:49:34.185742+0000","last_became_peered":"2024-04-26T01:49:34.185742+0000","last_unstale":"2024-04-26T01:50:27.837142+0000","last_undegraded":"2024-04-26T01:50:27.837142+0000","last_fullsized":"2024-04-26T01:50:27.837142+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:05.870056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998112+0000","last_change":"2024-04-26T01:49:37.640938+0000","last_active":"2024-04-26T01:50:26.998112+0000","last_peered":"2024-04-26T01:50:26.998112+0000","last_clean":"2024-04-26T01:50:26.998112+0000","last_became_active":"2024-04-26T01:49:37.640608+0000","last_became_peered":"2024-04-26T01:49:37.640608+0000","last_unstale":"2024-04-26T01:50:26.998112+0000","last_undegraded":"2024-04-26T01:50:26.998112+0000","last_fullsized":"2024-04-26T01:50:26.998112+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:30:30.460641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.991892+0000","last_change":"2024-04-26T01:49:33.604152+0000","last_active":"2024-04-26T01:50:26.991892+0000","last_peered":"2024-04-26T01:50:26.991892+0000","last_clean":"2024-04-26T01:50:26.991892+0000","last_became_active":"2024-04-26T01:49:33.603479+0000","last_became_peered":"2024-04-26T01:49:33.603479+0000","last_unstale":"2024-04-26T01:50:26.991892+0000","last_undegraded":"2024-04-26T01:50:26.991892+0000","last_fullsized":"2024-04-26T01:50:26.991892+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:25:13.685392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.995729+0000","last_change":"2024-04-26T01:49:37.645152+0000","last_active":"2024-04-26T01:50:26.995729+0000","last_peered":"2024-04-26T01:50:26.995729+0000","last_clean":"2024-04-26T01:50:26.995729+0000","last_became_active":"2024-04-26T01:49:37.644789+0000","last_became_peered":"2024-04-26T01:49:37.644789+0000","last_unstale":"2024-04-26T01:50:26.995729+0000","last_undegraded":"2024-04-26T01:50:26.995729+0000","last_fullsized":"2024-04-26T01:50:26.995729+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:20:01.494222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.010485+0000","last_change":"2024-04-26T01:50:27.009973+0000","last_active":"2024-04-26T01:50:27.010485+0000","last_peered":"2024-04-26T01:50:27.010485+0000","last_clean":"2024-04-26T01:50:27.010485+0000","last_became_active":"2024-04-26T01:50:27.001678+0000","last_became_peered":"2024-04-26T01:50:27.001678+0000","last_unstale":"2024-04-26T01:50:27.010485+0000","last_undegraded":"2024-04-26T01:50:27.010485+0000","last_fullsized":"2024-04-26T01:50:27.010485+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:59:54.807792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.992365+0000","last_change":"2024-04-26T01:49:37.662665+0000","last_active":"2024-04-26T01:50:26.992365+0000","last_peered":"2024-04-26T01:50:26.992365+0000","last_clean":"2024-04-26T01:50:26.992365+0000","last_became_active":"2024-04-26T01:49:37.662216+0000","last_became_peered":"2024-04-26T01:49:37.662216+0000","last_unstale":"2024-04-26T01:50:26.992365+0000","last_undegraded":"2024-04-26T01:50:26.992365+0000","last_fullsized":"2024-04-26T01:50:26.992365+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:10:54.724858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998326+0000","last_change":"2024-04-26T01:49:33.592078+0000","last_active":"2024-04-26T01:50:26.998326+0000","last_peered":"2024-04-26T01:50:26.998326+0000","last_clean":"2024-04-26T01:50:26.998326+0000","last_became_active":"2024-04-26T01:49:33.590265+0000","last_became_peered":"2024-04-26T01:49:33.590265+0000","last_unstale":"2024-04-26T01:50:26.998326+0000","last_undegraded":"2024-04-26T01:50:26.998326+0000","last_fullsized":"2024-04-26T01:50:26.998326+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:44:27.163705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998574+0000","last_change":"2024-04-26T01:49:37.645943+0000","last_active":"2024-04-26T01:50:26.998574+0000","last_peered":"2024-04-26T01:50:26.998574+0000","last_clean":"2024-04-26T01:50:26.998574+0000","last_became_active":"2024-04-26T01:49:37.645586+0000","last_became_peered":"2024-04-26T01:49:37.645586+0000","last_unstale":"2024-04-26T01:50:26.998574+0000","last_undegraded":"2024-04-26T01:50:26.998574+0000","last_fullsized":"2024-04-26T01:50:26.998574+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:40.682344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.000204+0000","last_change":"2024-04-26T01:49:34.187115+0000","last_active":"2024-04-26T01:50:27.000204+0000","last_peered":"2024-04-26T01:50:27.000204+0000","last_clean":"2024-04-26T01:50:27.000204+0000","last_became_active":"2024-04-26T01:49:34.186660+0000","last_became_peered":"2024-04-26T01:49:34.186660+0000","last_unstale":"2024-04-26T01:50:27.000204+0000","last_undegraded":"2024-04-26T01:50:27.000204+0000","last_fullsized":"2024-04-26T01:50:27.000204+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:56:32.274752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998810+0000","last_change":"2024-04-26T01:49:37.650382+0000","last_active":"2024-04-26T01:50:26.998810+0000","last_peered":"2024-04-26T01:50:26.998810+0000","last_clean":"2024-04-26T01:50:26.998810+0000","last_became_active":"2024-04-26T01:49:37.649545+0000","last_became_peered":"2024-04-26T01:49:37.649545+0000","last_unstale":"2024-04-26T01:50:26.998810+0000","last_undegraded":"2024-04-26T01:50:26.998810+0000","last_fullsized":"2024-04-26T01:50:26.998810+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:43:16.291311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.068936+0000","last_change":"2024-04-26T01:49:33.595881+0000","last_active":"2024-04-26T01:50:27.068936+0000","last_peered":"2024-04-26T01:50:27.068936+0000","last_clean":"2024-04-26T01:50:27.068936+0000","last_became_active":"2024-04-26T01:49:33.595480+0000","last_became_peered":"2024-04-26T01:49:33.595480+0000","last_unstale":"2024-04-26T01:50:27.068936+0000","last_undegraded":"2024-04-26T01:50:27.068936+0000","last_fullsized":"2024-04-26T01:50:27.068936+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:30:58.028230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'1","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998549+0000","last_change":"2024-04-26T01:49:37.660765+0000","last_active":"2024-04-26T01:50:26.998549+0000","last_peered":"2024-04-26T01:50:26.998549+0000","last_clean":"2024-04-26T01:50:26.998549+0000","last_became_active":"2024-04-26T01:49:37.660475+0000","last_became_peered":"2024-04-26T01:49:37.660475+0000","last_unstale":"2024-04-26T01:50:26.998549+0000","last_undegraded":"2024-04-26T01:50:26.998549+0000","last_fullsized":"2024-04-26T01:50:26.998549+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:36:00.483005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"90'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.006174+0000","last_change":"2024-04-26T01:49:33.598247+0000","last_active":"2024-04-26T01:50:27.006174+0000","last_peered":"2024-04-26T01:50:27.006174+0000","last_clean":"2024-04-26T01:50:27.006174+0000","last_became_active":"2024-04-26T01:49:33.597796+0000","last_became_peered":"2024-04-26T01:49:33.597796+0000","last_unstale":"2024-04-26T01:50:27.006174+0000","last_undegraded":"2024-04-26T01:50:27.006174+0000","last_fullsized":"2024-04-26T01:50:27.006174+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:53:38.994798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998820+0000","last_change":"2024-04-26T01:49:37.648903+0000","last_active":"2024-04-26T01:50:26.998820+0000","last_peered":"2024-04-26T01:50:26.998820+0000","last_clean":"2024-04-26T01:50:26.998820+0000","last_became_active":"2024-04-26T01:49:37.648570+0000","last_became_peered":"2024-04-26T01:49:37.648570+0000","last_unstale":"2024-04-26T01:50:26.998820+0000","last_undegraded":"2024-04-26T01:50:26.998820+0000","last_fullsized":"2024-04-26T01:50:26.998820+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:33:12.678513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.995800+0000","last_change":"2024-04-26T01:49:37.648485+0000","last_active":"2024-04-26T01:50:26.995800+0000","last_peered":"2024-04-26T01:50:26.995800+0000","last_clean":"2024-04-26T01:50:26.995800+0000","last_became_active":"2024-04-26T01:49:37.648129+0000","last_became_peered":"2024-04-26T01:49:37.648129+0000","last_unstale":"2024-04-26T01:50:26.995800+0000","last_undegraded":"2024-04-26T01:50:26.995800+0000","last_fullsized":"2024-04-26T01:50:26.995800+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:56:12.047821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.999299+0000","last_change":"2024-04-26T01:49:37.645929+0000","last_active":"2024-04-26T01:50:26.999299+0000","last_peered":"2024-04-26T01:50:26.999299+0000","last_clean":"2024-04-26T01:50:26.999299+0000","last_became_active":"2024-04-26T01:49:37.645473+0000","last_became_peered":"2024-04-26T01:49:37.645473+0000","last_unstale":"2024-04-26T01:50:26.999299+0000","last_undegraded":"2024-04-26T01:50:26.999299+0000","last_fullsized":"2024-04-26T01:50:26.999299+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:39:56.687973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998457+0000","last_change":"2024-04-26T01:49:37.657494+0000","last_active":"2024-04-26T01:50:26.998457+0000","last_peered":"2024-04-26T01:50:26.998457+0000","last_clean":"2024-04-26T01:50:26.998457+0000","last_became_active":"2024-04-26T01:49:37.657121+0000","last_became_peered":"2024-04-26T01:49:37.657121+0000","last_unstale":"2024-04-26T01:50:26.998457+0000","last_undegraded":"2024-04-26T01:50:26.998457+0000","last_fullsized":"2024-04-26T01:50:26.998457+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:50:44.691179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.992636+0000","last_change":"2024-04-26T01:49:37.646012+0000","last_active":"2024-04-26T01:50:26.992636+0000","last_peered":"2024-04-26T01:50:26.992636+0000","last_clean":"2024-04-26T01:50:26.992636+0000","last_became_active":"2024-04-26T01:49:37.645695+0000","last_became_peered":"2024-04-26T01:49:37.645695+0000","last_unstale":"2024-04-26T01:50:26.992636+0000","last_undegraded":"2024-04-26T01:50:26.992636+0000","last_fullsized":"2024-04-26T01:50:26.992636+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:40:51.117523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.005878+0000","last_change":"2024-04-26T01:49:37.646503+0000","last_active":"2024-04-26T01:50:27.005878+0000","last_peered":"2024-04-26T01:50:27.005878+0000","last_clean":"2024-04-26T01:50:27.005878+0000","last_became_active":"2024-04-26T01:49:37.645200+0000","last_became_peered":"2024-04-26T01:49:37.645200+0000","last_unstale":"2024-04-26T01:50:27.005878+0000","last_undegraded":"2024-04-26T01:50:27.005878+0000","last_fullsized":"2024-04-26T01:50:27.005878+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:31:10.957199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.002908+0000","last_change":"2024-04-26T01:50:27.002908+0000","last_active":"2024-04-26T01:50:27.002908+0000","last_peered":"2024-04-26T01:50:27.002908+0000","last_clean":"2024-04-26T01:50:27.002908+0000","last_became_active":"2024-04-26T01:50:27.002454+0000","last_became_peered":"2024-04-26T01:50:27.002454+0000","last_unstale":"2024-04-26T01:50:27.002908+0000","last_undegraded":"2024-04-26T01:50:27.002908+0000","last_fullsized":"2024-04-26T01:50:27.002908+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:18:01.848427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998823+0000","last_change":"2024-04-26T01:49:37.642676+0000","last_active":"2024-04-26T01:50:26.998823+0000","last_peered":"2024-04-26T01:50:26.998823+0000","last_clean":"2024-04-26T01:50:26.998823+0000","last_became_active":"2024-04-26T01:49:37.642302+0000","last_became_peered":"2024-04-26T01:49:37.642302+0000","last_unstale":"2024-04-26T01:50:26.998823+0000","last_undegraded":"2024-04-26T01:50:26.998823+0000","last_fullsized":"2024-04-26T01:50:26.998823+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:51:43.454649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.003914+0000","last_change":"2024-04-26T01:50:27.003914+0000","last_active":"2024-04-26T01:50:27.003914+0000","last_peered":"2024-04-26T01:50:27.003914+0000","last_clean":"2024-04-26T01:50:27.003914+0000","last_became_active":"2024-04-26T01:50:27.002344+0000","last_became_peered":"2024-04-26T01:50:27.002344+0000","last_unstale":"2024-04-26T01:50:27.003914+0000","last_undegraded":"2024-04-26T01:50:27.003914+0000","last_fullsized":"2024-04-26T01:50:27.003914+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:46:02.101508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.068475+0000","last_change":"2024-04-26T01:49:37.663306+0000","last_active":"2024-04-26T01:50:27.068475+0000","last_peered":"2024-04-26T01:50:27.068475+0000","last_clean":"2024-04-26T01:50:27.068475+0000","last_became_active":"2024-04-26T01:49:37.662999+0000","last_became_peered":"2024-04-26T01:49:37.662999+0000","last_unstale":"2024-04-26T01:50:27.068475+0000","last_undegraded":"2024-04-26T01:50:27.068475+0000","last_fullsized":"2024-04-26T01:50:27.068475+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:55.155096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.991831+0000","last_change":"2024-04-26T01:49:37.655157+0000","last_active":"2024-04-26T01:50:26.991831+0000","last_peered":"2024-04-26T01:50:26.991831+0000","last_clean":"2024-04-26T01:50:26.991831+0000","last_became_active":"2024-04-26T01:49:37.654810+0000","last_became_peered":"2024-04-26T01:49:37.654810+0000","last_unstale":"2024-04-26T01:50:26.991831+0000","last_undegraded":"2024-04-26T01:50:26.991831+0000","last_fullsized":"2024-04-26T01:50:26.991831+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:28:04.012382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.837305+0000","last_change":"2024-04-26T01:49:37.667228+0000","last_active":"2024-04-26T01:50:27.837305+0000","last_peered":"2024-04-26T01:50:27.837305+0000","last_clean":"2024-04-26T01:50:27.837305+0000","last_became_active":"2024-04-26T01:49:37.666835+0000","last_became_peered":"2024-04-26T01:49:37.666835+0000","last_unstale":"2024-04-26T01:50:27.837305+0000","last_undegraded":"2024-04-26T01:50:27.837305+0000","last_fullsized":"2024-04-26T01:50:27.837305+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:54:16.902836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.005906+0000","last_change":"2024-04-26T01:49:37.658231+0000","last_active":"2024-04-26T01:50:27.005906+0000","last_peered":"2024-04-26T01:50:27.005906+0000","last_clean":"2024-04-26T01:50:27.005906+0000","last_became_active":"2024-04-26T01:49:37.657842+0000","last_became_peered":"2024-04-26T01:49:37.657842+0000","last_unstale":"2024-04-26T01:50:27.005906+0000","last_undegraded":"2024-04-26T01:50:27.005906+0000","last_fullsized":"2024-04-26T01:50:27.005906+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:02:01.445262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.995790+0000","last_change":"2024-04-26T01:49:37.645221+0000","last_active":"2024-04-26T01:50:26.995790+0000","last_peered":"2024-04-26T01:50:26.995790+0000","last_clean":"2024-04-26T01:50:26.995790+0000","last_became_active":"2024-04-26T01:49:37.644814+0000","last_became_peered":"2024-04-26T01:49:37.644814+0000","last_unstale":"2024-04-26T01:50:26.995790+0000","last_undegraded":"2024-04-26T01:50:26.995790+0000","last_fullsized":"2024-04-26T01:50:26.995790+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:03:17.850633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64739,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":259,"num_read_kb":201,"num_write":164,"num_write_kb":111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":194217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318303,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":995328,"data_stored":486380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579938,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28668,"kb_used_data":1652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716484,"statfs":{"total":95995035648,"available":95965679616,"internally_reserved":0,"allocated":1691648,"data_stored":1138599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743464,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1011712,"data_stored":486616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005103,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":486536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266739,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28592,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716560,"statfs":{"total":95995035648,"available":95965757440,"internally_reserved":0,"allocated":1605632,"data_stored":1078857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430266,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":999424,"data_stored":486005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1060864,"data_stored":546145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1019904,"data_stored":487276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1085440,"data_stored":547886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117065,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1015808,"data_stored":486316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855430,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1003520,"data_stored":486833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691903,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28600,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716552,"statfs":{"total":95995035648,"available":95965749248,"internally_reserved":0,"allocated":1617920,"data_stored":1078150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":60347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T01:50:42.509 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-26T01:50:42.509 INFO:tasks.cephadm:Setting up client nodes... 2024-04-26T01:50:42.510 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-26T01:50:42.996 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:50:43.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:42 smithi172 ceph-mon[32785]: pgmap v360: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 7 op/s 2024-04-26T01:50:43.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:42 smithi172 ceph-mon[32785]: from='client.24635 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:50:43.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:42 smithi165 ceph-mon[32648]: pgmap v360: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 7 op/s 2024-04-26T01:50:43.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:42 smithi165 ceph-mon[32648]: from='client.24635 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:50:43.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:42 smithi151 ceph-mon[28057]: pgmap v360: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 7 op/s 2024-04-26T01:50:43.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:42 smithi151 ceph-mon[28057]: from='client.24635 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:50:44.957 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:44 smithi151 ceph-mon[28057]: pgmap v361: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-26T01:50:45.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:44 smithi172 ceph-mon[32785]: pgmap v361: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-26T01:50:45.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:44 smithi165 ceph-mon[32648]: pgmap v361: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-26T01:50:45.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:45.259+0000 7f3a61800700 1 -- 172.21.15.151:0/1642669882 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3a5c0754a0 msgr2=0x7f3a5c075920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:45.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:45.259+0000 7f3a61800700 1 --2- 172.21.15.151:0/1642669882 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3a5c0754a0 0x7f3a5c075920 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3a48009990 tx=0x7f3a48009ca0 comp rx=0 tx=0).stop 2024-04-26T01:50:45.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:45.259+0000 7f3a61800700 1 -- 172.21.15.151:0/1642669882 shutdown_connections 2024-04-26T01:50:45.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:45.259+0000 7f3a61800700 1 --2- 172.21.15.151:0/1642669882 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a5c10eee0 0x7f3a5c1112d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:45.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:45.259+0000 7f3a61800700 1 --2- 172.21.15.151:0/1642669882 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3a5c076a70 0x7f3a5c074ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:45.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:45.259+0000 7f3a61800700 1 --2- 172.21.15.151:0/1642669882 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3a5c0754a0 0x7f3a5c075920 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:45.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:45.259+0000 7f3a61800700 1 -- 172.21.15.151:0/1642669882 >> 172.21.15.151:0/1642669882 conn(0x7f3a5c0fde80 msgr2=0x7f3a5c1002e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:45.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:45.259+0000 7f3a61800700 1 -- 172.21.15.151:0/1642669882 shutdown_connections 2024-04-26T01:50:45.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:45.260+0000 7f3a61800700 1 -- 172.21.15.151:0/1642669882 wait complete. 2024-04-26T01:50:45.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:45.261+0000 7f3a61800700 1 Processor -- start 2024-04-26T01:50:45.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:45.261+0000 7f3a61800700 1 -- start start 2024-04-26T01:50:45.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:45.261+0000 7f3a61800700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3a5c0754a0 0x7f3a5c19efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:45.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:45.261+0000 7f3a61800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3a5c076a70 0x7f3a5c19f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:45.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:45.261+0000 7f3a61800700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a5c10eee0 0x7f3a5c1a6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:45.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:45.261+0000 7f3a61800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3a5c1149e0 con 0x7f3a5c076a70 2024-04-26T01:50:45.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:45.261+0000 7f3a61800700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3a5c114860 con 0x7f3a5c0754a0 2024-04-26T01:50:45.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:45.261+0000 7f3a61800700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3a5c114b60 con 0x7f3a5c10eee0 2024-04-26T01:50:45.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:45.261+0000 7f3a5b400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3a5c076a70 0x7f3a5c19f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:45.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:45.262+0000 7f3a5be00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3a5c0754a0 0x7f3a5c19efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:45.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:45.262+0000 7f3a5b400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3a5c076a70 0x7f3a5c19f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:50284/0 (socket says 172.21.15.151:50284) 2024-04-26T01:50:45.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:45.262+0000 7f3a5b400700 1 -- 172.21.15.151:0/1916916900 learned_addr learned my addr 172.21.15.151:0/1916916900 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:50:45.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:45.262+0000 7f3a60e00700 1 --2- 172.21.15.151:0/1916916900 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a5c10eee0 0x7f3a5c1a6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:45.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:45.262+0000 7f3a5be00700 1 -- 172.21.15.151:0/1916916900 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a5c10eee0 msgr2=0x7f3a5c1a6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:45.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:45.262+0000 7f3a5be00700 1 --2- 172.21.15.151:0/1916916900 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a5c10eee0 0x7f3a5c1a6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:45.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:45.262+0000 7f3a5be00700 1 -- 172.21.15.151:0/1916916900 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3a5c076a70 msgr2=0x7f3a5c19f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:45.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:45.262+0000 7f3a5be00700 1 --2- 172.21.15.151:0/1916916900 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3a5c076a70 0x7f3a5c19f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:45.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:45.262+0000 7f3a5be00700 1 -- 172.21.15.151:0/1916916900 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a5c1a6c20 con 0x7f3a5c0754a0 2024-04-26T01:50:45.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:45.262+0000 7f3a5b400700 1 --2- 172.21.15.151:0/1916916900 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3a5c076a70 0x7f3a5c19f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:50:45.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:45.262+0000 7f3a5be00700 1 --2- 172.21.15.151:0/1916916900 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3a5c0754a0 0x7f3a5c19efc0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c00e9c0 tx=0x7f3a4c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:45.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:45.263+0000 7f3a58c00700 1 -- 172.21.15.151:0/1916916900 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a4c00c9c0 con 0x7f3a5c0754a0 2024-04-26T01:50:45.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:45.263+0000 7f3a61800700 1 -- 172.21.15.151:0/1916916900 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a5c1a6f10 con 0x7f3a5c0754a0 2024-04-26T01:50:45.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:45.263+0000 7f3a58c00700 1 -- 172.21.15.151:0/1916916900 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a4c00cb60 con 0x7f3a5c0754a0 2024-04-26T01:50:45.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:45.263+0000 7f3a58c00700 1 -- 172.21.15.151:0/1916916900 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a4c018680 con 0x7f3a5c0754a0 2024-04-26T01:50:45.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:45.263+0000 7f3a61800700 1 -- 172.21.15.151:0/1916916900 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a5c1a7370 con 0x7f3a5c0754a0 2024-04-26T01:50:45.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:45.264+0000 7f3a61800700 1 -- 172.21.15.151:0/1916916900 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a5c0640a0 con 0x7f3a5c0754a0 2024-04-26T01:50:45.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:45.266+0000 7f3a58c00700 1 -- 172.21.15.151:0/1916916900 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f3a4c010430 con 0x7f3a5c0754a0 2024-04-26T01:50:45.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:45.267+0000 7f3a58c00700 1 --2- 172.21.15.151:0/1916916900 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f3a400a3e00 0x7f3a400a62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:45.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:45.267+0000 7f3a58c00700 1 -- 172.21.15.151:0/1916916900 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f3a4c0d06f0 con 0x7f3a5c0754a0 2024-04-26T01:50:45.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:45.268+0000 7f3a5b400700 1 --2- 172.21.15.151:0/1916916900 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f3a400a3e00 0x7f3a400a62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:45.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:45.268+0000 7f3a5b400700 1 --2- 172.21.15.151:0/1916916900 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f3a400a3e00 0x7f3a400a62c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3a48006010 tx=0x7f3a4800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:45.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:45.270+0000 7f3a58c00700 1 -- 172.21.15.151:0/1916916900 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3a4c09cf70 con 0x7f3a5c0754a0 2024-04-26T01:50:45.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:45.585+0000 7f3a61800700 1 -- 172.21.15.151:0/1916916900 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f3a5c02df80 con 0x7f3a5c0754a0 2024-04-26T01:50:45.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:45.594+0000 7f3a58c00700 1 -- 172.21.15.151:0/1916916900 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f3a4c09c870 con 0x7f3a5c0754a0 2024-04-26T01:50:45.592 INFO:teuthology.orchestra.run.smithi151.stdout:[client.0] 2024-04-26T01:50:45.592 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQB1CCtmjfoLIxAAGiecU0TZemMjvJOc2MfT3w== 2024-04-26T01:50:45.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:45.597+0000 7f3a61800700 1 -- 172.21.15.151:0/1916916900 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f3a400a3e00 msgr2=0x7f3a400a62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:45.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:45.597+0000 7f3a61800700 1 --2- 172.21.15.151:0/1916916900 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f3a400a3e00 0x7f3a400a62c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3a48006010 tx=0x7f3a4800b260 comp rx=0 tx=0).stop 2024-04-26T01:50:45.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:45.597+0000 7f3a61800700 1 -- 172.21.15.151:0/1916916900 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3a5c0754a0 msgr2=0x7f3a5c19efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:45.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:45.597+0000 7f3a61800700 1 --2- 172.21.15.151:0/1916916900 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3a5c0754a0 0x7f3a5c19efc0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c00e9c0 tx=0x7f3a4c00ecd0 comp rx=0 tx=0).stop 2024-04-26T01:50:45.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:45.598+0000 7f3a61800700 1 -- 172.21.15.151:0/1916916900 shutdown_connections 2024-04-26T01:50:45.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:45.598+0000 7f3a61800700 1 --2- 172.21.15.151:0/1916916900 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a5c10eee0 0x7f3a5c1a6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:45.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:45.598+0000 7f3a61800700 1 --2- 172.21.15.151:0/1916916900 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3a5c0754a0 0x7f3a5c19efc0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:45.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:45.598+0000 7f3a61800700 1 --2- 172.21.15.151:0/1916916900 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f3a400a3e00 0x7f3a400a62c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:45.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:45.598+0000 7f3a61800700 1 --2- 172.21.15.151:0/1916916900 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3a5c076a70 0x7f3a5c19f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:45.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:45.598+0000 7f3a61800700 1 -- 172.21.15.151:0/1916916900 >> 172.21.15.151:0/1916916900 conn(0x7f3a5c0fde80 msgr2=0x7f3a5c102280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:45.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:45.598+0000 7f3a61800700 1 -- 172.21.15.151:0/1916916900 shutdown_connections 2024-04-26T01:50:45.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:45.598+0000 7f3a61800700 1 -- 172.21.15.151:0/1916916900 wait complete. 2024-04-26T01:50:45.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:45 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/1916916900' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T01:50:45.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:45 smithi151 ceph-mon[28057]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T01:50:45.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:45 smithi151 ceph-mon[28057]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-26T01:50:46.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:45 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/1916916900' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T01:50:46.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:45 smithi172 ceph-mon[32785]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T01:50:46.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:45 smithi172 ceph-mon[32785]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-26T01:50:46.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:45 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/1916916900' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T01:50:46.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:45 smithi165 ceph-mon[32648]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T01:50:46.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:45 smithi165 ceph-mon[32648]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-26T01:50:46.543 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-26T01:50:46.543 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-26T01:50:46.543 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-26T01:50:46.676 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-26T01:50:46.677 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-26T01:50:46.677 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-26T01:50:46.793 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:46 smithi151 ceph-mon[28057]: pgmap v362: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-26T01:50:47.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:46 smithi172 ceph-mon[32785]: pgmap v362: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-26T01:50:47.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:46 smithi165 ceph-mon[32648]: pgmap v362: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-26T01:50:47.170 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:50:49.003 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:48 smithi151 ceph-mon[28057]: pgmap v363: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-26T01:50:49.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:48 smithi172 ceph-mon[32785]: pgmap v363: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-26T01:50:49.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:48 smithi165 ceph-mon[32648]: pgmap v363: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-26T01:50:49.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:49.560+0000 7f4129200700 1 -- 172.21.15.151:0/1077512770 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f412406a750 msgr2=0x7f412406ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:49.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:49.560+0000 7f4129200700 1 --2- 172.21.15.151:0/1077512770 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f412406a750 0x7f412406ab30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4114009990 tx=0x7f4114009ca0 comp rx=0 tx=0).stop 2024-04-26T01:50:49.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:49.560+0000 7f4129200700 1 -- 172.21.15.151:0/1077512770 shutdown_connections 2024-04-26T01:50:49.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:49.560+0000 7f4129200700 1 --2- 172.21.15.151:0/1077512770 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f412410f4f0 0x7f41241118e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:49.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:49.560+0000 7f4129200700 1 --2- 172.21.15.151:0/1077512770 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f412406a750 0x7f412406ab30 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:49.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:49.560+0000 7f4129200700 1 --2- 172.21.15.151:0/1077512770 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f412406b070 0x7f412410efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:49.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:49.560+0000 7f4129200700 1 -- 172.21.15.151:0/1077512770 >> 172.21.15.151:0/1077512770 conn(0x7f41240788f0 msgr2=0x7f4124078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:49.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:49.560+0000 7f4129200700 1 -- 172.21.15.151:0/1077512770 shutdown_connections 2024-04-26T01:50:49.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:49.560+0000 7f4129200700 1 -- 172.21.15.151:0/1077512770 wait complete. 2024-04-26T01:50:49.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:49.561+0000 7f4129200700 1 Processor -- start 2024-04-26T01:50:49.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:49.561+0000 7f4129200700 1 -- start start 2024-04-26T01:50:49.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:49.562+0000 7f4129200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f412406a750 0x7f412419b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:49.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:49.562+0000 7f4129200700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f412406b070 0x7f412419b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:49.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:49.562+0000 7f4129200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f412410f4f0 0x7f412419fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:49.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:49.562+0000 7f4129200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4124114e70 con 0x7f412406a750 2024-04-26T01:50:49.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:49.562+0000 7f4129200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4124114cf0 con 0x7f412410f4f0 2024-04-26T01:50:49.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:49.562+0000 7f4129200700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4124114ff0 con 0x7f412406b070 2024-04-26T01:50:49.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:49.562+0000 7f4121e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f412406a750 0x7f412419b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:49.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:49.562+0000 7f4119400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f412406b070 0x7f412419b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:49.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:49.562+0000 7f4119400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f412406b070 0x7f412419b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:54632/0 (socket says 172.21.15.151:54632) 2024-04-26T01:50:49.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:49.562+0000 7f4122800700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f412410f4f0 0x7f412419fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:49.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:49.562+0000 7f4119400700 1 -- 172.21.15.151:0/2028652296 learned_addr learned my addr 172.21.15.151:0/2028652296 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:50:49.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:49.563+0000 7f4119400700 1 -- 172.21.15.151:0/2028652296 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f412410f4f0 msgr2=0x7f412419fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:49.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:49.563+0000 7f4119400700 1 --2- 172.21.15.151:0/2028652296 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f412410f4f0 0x7f412419fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:49.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:49.563+0000 7f4119400700 1 -- 172.21.15.151:0/2028652296 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f412406a750 msgr2=0x7f412419b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:49.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:49.563+0000 7f4119400700 1 --2- 172.21.15.151:0/2028652296 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f412406a750 0x7f412419b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:49.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:49.563+0000 7f4119400700 1 -- 172.21.15.151:0/2028652296 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41241a0380 con 0x7f412406b070 2024-04-26T01:50:49.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:49.563+0000 7f4122800700 1 --2- 172.21.15.151:0/2028652296 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f412410f4f0 0x7f412419fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:50:49.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:49.563+0000 7f4119400700 1 --2- 172.21.15.151:0/2028652296 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f412406b070 0x7f412419b900 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f410c00ef10 tx=0x7f410c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:49.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:49.563+0000 7f411b400700 1 -- 172.21.15.151:0/2028652296 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f410c00c9c0 con 0x7f412406b070 2024-04-26T01:50:49.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:49.563+0000 7f4129200700 1 -- 172.21.15.151:0/2028652296 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41241a0610 con 0x7f412406b070 2024-04-26T01:50:49.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:49.563+0000 7f411b400700 1 -- 172.21.15.151:0/2028652296 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f410c019070 con 0x7f412406b070 2024-04-26T01:50:49.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:49.563+0000 7f411b400700 1 -- 172.21.15.151:0/2028652296 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f410c014480 con 0x7f412406b070 2024-04-26T01:50:49.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:49.564+0000 7f4129200700 1 -- 172.21.15.151:0/2028652296 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41241a7ef0 con 0x7f412406b070 2024-04-26T01:50:49.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:49.565+0000 7f4129200700 1 -- 172.21.15.151:0/2028652296 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4124069730 con 0x7f412406b070 2024-04-26T01:50:49.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:49.567+0000 7f411b400700 1 -- 172.21.15.151:0/2028652296 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f410c012050 con 0x7f412406b070 2024-04-26T01:50:49.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:49.568+0000 7f411b400700 1 --2- 172.21.15.151:0/2028652296 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f41080a3cd0 0x7f41080a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:49.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:49.568+0000 7f411b400700 1 -- 172.21.15.151:0/2028652296 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f410c0d07b0 con 0x7f412406b070 2024-04-26T01:50:49.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:49.568+0000 7f4121e00700 1 --2- 172.21.15.151:0/2028652296 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f41080a3cd0 0x7f41080a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:49.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:49.569+0000 7f4121e00700 1 --2- 172.21.15.151:0/2028652296 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f41080a3cd0 0x7f41080a6190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4114006010 tx=0x7f411400b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:49.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:49.570+0000 7f411b400700 1 -- 172.21.15.151:0/2028652296 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f410c010040 con 0x7f412406b070 2024-04-26T01:50:49.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:49.866+0000 7f4129200700 1 -- 172.21.15.151:0/2028652296 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f412402df80 con 0x7f412406b070 2024-04-26T01:50:49.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:49.871+0000 7f411b400700 1 -- 172.21.15.151:0/2028652296 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+265006 (secure 0 0 0) 0x7f410c09d030 con 0x7f412406b070 2024-04-26T01:50:49.874 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:50:49.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:49.877+0000 7f4129200700 1 -- 172.21.15.151:0/2028652296 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f41080a3cd0 msgr2=0x7f41080a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:49.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:49.877+0000 7f4129200700 1 --2- 172.21.15.151:0/2028652296 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f41080a3cd0 0x7f41080a6190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4114006010 tx=0x7f411400b260 comp rx=0 tx=0).stop 2024-04-26T01:50:49.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:49.877+0000 7f4129200700 1 -- 172.21.15.151:0/2028652296 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f412406b070 msgr2=0x7f412419b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:49.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:49.877+0000 7f4129200700 1 --2- 172.21.15.151:0/2028652296 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f412406b070 0x7f412419b900 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f410c00ef10 tx=0x7f410c00c3a0 comp rx=0 tx=0).stop 2024-04-26T01:50:49.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:49.877+0000 7f4129200700 1 -- 172.21.15.151:0/2028652296 shutdown_connections 2024-04-26T01:50:49.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:49.877+0000 7f4129200700 1 --2- 172.21.15.151:0/2028652296 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f412410f4f0 0x7f412419fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:49.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:49.877+0000 7f4129200700 1 --2- 172.21.15.151:0/2028652296 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f41080a3cd0 0x7f41080a6190 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:49.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:49.877+0000 7f4129200700 1 --2- 172.21.15.151:0/2028652296 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f412406a750 0x7f412419b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:49.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:49.877+0000 7f4129200700 1 --2- 172.21.15.151:0/2028652296 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f412406b070 0x7f412419b900 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:49.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:49.877+0000 7f4129200700 1 -- 172.21.15.151:0/2028652296 >> 172.21.15.151:0/2028652296 conn(0x7f41240788f0 msgr2=0x7f41241109e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:49.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:49.877+0000 7f4129200700 1 -- 172.21.15.151:0/2028652296 shutdown_connections 2024-04-26T01:50:49.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:49.877+0000 7f4129200700 1 -- 172.21.15.151:0/2028652296 wait complete. 2024-04-26T01:50:50.823 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":2251256505},{"type":"v1","addr":"172.21.15.151:6801","nonce":2251256505}]},"active_addr":"172.21.15.151:6801/2251256505","active_change":"2024-04-26T01:41:25.234427+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24107,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.151:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.151:0","nonce":755039608}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.151:0","nonce":4066781930}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.151:0","nonce":3463483606}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.151:0","nonce":2357384001}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.151:0","nonce":2433013311}]}]} 2024-04-26T01:50:50.828 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-26T01:50:50.828 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-26T01:50:50.828 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T01:50:51.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:50 smithi172 ceph-mon[32785]: pgmap v364: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:50:51.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:50 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/2028652296' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-26T01:50:51.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:50 smithi165 ceph-mon[32648]: pgmap v364: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:50:51.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:50 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/2028652296' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-26T01:50:51.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:50 smithi151 ceph-mon[28057]: pgmap v364: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:50:51.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:50 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/2028652296' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-26T01:50:51.281 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:50:53.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:52 smithi172 ceph-mon[32785]: pgmap v365: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:50:53.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:52 smithi165 ceph-mon[32648]: pgmap v365: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:50:53.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:52 smithi151 ceph-mon[28057]: pgmap v365: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:50:53.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:53.707+0000 7f1eb6000700 1 -- 172.21.15.151:0/3125027078 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1eb0103450 msgr2=0x7f1eb010f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:53.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:53.707+0000 7f1eb6000700 1 --2- 172.21.15.151:0/3125027078 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1eb0103450 0x7f1eb010f8b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8009990 tx=0x7f1ea8009ca0 comp rx=0 tx=0).stop 2024-04-26T01:50:53.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:53.707+0000 7f1eb6000700 1 -- 172.21.15.151:0/3125027078 shutdown_connections 2024-04-26T01:50:53.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:53.707+0000 7f1eb6000700 1 --2- 172.21.15.151:0/3125027078 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1eb0103450 0x7f1eb010f8b0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:53.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:53.707+0000 7f1eb6000700 1 --2- 172.21.15.151:0/3125027078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1eb0108a50 0x7f1eb0108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:53.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:53.707+0000 7f1eb6000700 1 --2- 172.21.15.151:0/3125027078 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1eb0102a90 0x7f1eb0102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:53.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:53.707+0000 7f1eb6000700 1 -- 172.21.15.151:0/3125027078 >> 172.21.15.151:0/3125027078 conn(0x7f1eb0079ba0 msgr2=0x7f1eb0079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:53.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:53.707+0000 7f1eb6000700 1 -- 172.21.15.151:0/3125027078 shutdown_connections 2024-04-26T01:50:53.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:53.707+0000 7f1eb6000700 1 -- 172.21.15.151:0/3125027078 wait complete. 2024-04-26T01:50:53.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:53.708+0000 7f1eb6000700 1 Processor -- start 2024-04-26T01:50:53.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:53.709+0000 7f1eb6000700 1 -- start start 2024-04-26T01:50:53.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:53.709+0000 7f1eb6000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1eb0102a90 0x7f1eb019b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:53.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:53.709+0000 7f1eb6000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1eb0103450 0x7f1eb019b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:53.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:53.709+0000 7f1eb6000700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1eb0108a50 0x7f1eb019fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:53.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:53.709+0000 7f1eb6000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb0112c50 con 0x7f1eb0103450 2024-04-26T01:50:53.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:53.709+0000 7f1eb6000700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb0112ad0 con 0x7f1eb0102a90 2024-04-26T01:50:53.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:53.709+0000 7f1eb6000700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb0112dd0 con 0x7f1eb0108a50 2024-04-26T01:50:53.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:53.710+0000 7f1ea7e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1eb0103450 0x7f1eb019b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:53.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:53.710+0000 7f1eaf400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1eb0102a90 0x7f1eb019b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:53.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:53.710+0000 7f1ea7e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1eb0103450 0x7f1eb019b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41048/0 (socket says 172.21.15.151:41048) 2024-04-26T01:50:53.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:53.710+0000 7f1ea7e00700 1 -- 172.21.15.151:0/1206711271 learned_addr learned my addr 172.21.15.151:0/1206711271 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:50:53.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:53.710+0000 7f1eafe00700 1 --2- 172.21.15.151:0/1206711271 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1eb0108a50 0x7f1eb019fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:53.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:53.710+0000 7f1eaf400700 1 -- 172.21.15.151:0/1206711271 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1eb0108a50 msgr2=0x7f1eb019fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:53.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:53.710+0000 7f1eaf400700 1 --2- 172.21.15.151:0/1206711271 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1eb0108a50 0x7f1eb019fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:53.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:53.710+0000 7f1eaf400700 1 -- 172.21.15.151:0/1206711271 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1eb0103450 msgr2=0x7f1eb019b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:53.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:53.710+0000 7f1eaf400700 1 --2- 172.21.15.151:0/1206711271 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1eb0103450 0x7f1eb019b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:53.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:53.710+0000 7f1eaf400700 1 -- 172.21.15.151:0/1206711271 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eb01a0310 con 0x7f1eb0102a90 2024-04-26T01:50:53.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:53.710+0000 7f1eaf400700 1 --2- 172.21.15.151:0/1206711271 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1eb0102a90 0x7f1eb019b300 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1ea000efc0 tx=0x7f1ea000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:53.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:53.711+0000 7f1eacc00700 1 -- 172.21.15.151:0/1206711271 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea00092e0 con 0x7f1eb0102a90 2024-04-26T01:50:53.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:53.711+0000 7f1eacc00700 1 -- 172.21.15.151:0/1206711271 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ea0004bb0 con 0x7f1eb0102a90 2024-04-26T01:50:53.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:53.711+0000 7f1eb6000700 1 -- 172.21.15.151:0/1206711271 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1eb01a05a0 con 0x7f1eb0102a90 2024-04-26T01:50:53.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:53.711+0000 7f1eacc00700 1 -- 172.21.15.151:0/1206711271 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea0004280 con 0x7f1eb0102a90 2024-04-26T01:50:53.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:53.711+0000 7f1eb6000700 1 -- 172.21.15.151:0/1206711271 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1eb01a7e80 con 0x7f1eb0102a90 2024-04-26T01:50:53.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:53.712+0000 7f1eb6000700 1 -- 172.21.15.151:0/1206711271 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1eb0104180 con 0x7f1eb0102a90 2024-04-26T01:50:53.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:53.714+0000 7f1eacc00700 1 -- 172.21.15.151:0/1206711271 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f1ea001b030 con 0x7f1eb0102a90 2024-04-26T01:50:53.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:53.716+0000 7f1eacc00700 1 --2- 172.21.15.151:0/1206711271 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f1e940a3cd0 0x7f1e940a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:53.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:53.717+0000 7f1eacc00700 1 -- 172.21.15.151:0/1206711271 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f1ea00d00f0 con 0x7f1eb0102a90 2024-04-26T01:50:53.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:53.717+0000 7f1ea7e00700 1 --2- 172.21.15.151:0/1206711271 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f1e940a3cd0 0x7f1e940a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:53.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:53.717+0000 7f1ea7e00700 1 --2- 172.21.15.151:0/1206711271 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f1e940a3cd0 0x7f1e940a6190 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1e98009680 tx=0x7f1e98009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:53.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:53.722+0000 7f1eacc00700 1 -- 172.21.15.151:0/1206711271 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1ea0022020 con 0x7f1eb0102a90 2024-04-26T01:50:53.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:53.952+0000 7f1eb6000700 1 -- 172.21.15.151:0/1206711271 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1eb01a8050 con 0x7f1eb0102a90 2024-04-26T01:50:53.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:53.954+0000 7f1eacc00700 1 -- 172.21.15.151:0/1206711271 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+22599 (secure 0 0 0) 0x7f1ea0010040 con 0x7f1eb0102a90 2024-04-26T01:50:53.952 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:50:53.952 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":92,"fsid":"c0e49e64-036d-11ef-bc93-c7b262605968","created":"2024-04-26T01:40:07.483197+0000","modified":"2024-04-26T01:50:26.985192+0000","last_up_change":"2024-04-26T01:48:21.167691+0000","last_in_change":"2024-04-26T01:48:00.556990+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T01:44:11.411722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T01:49:31.756032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T01:49:36.529819+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ca27e814-a982-482c-ae1a-e596aac5c59a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":669937998},{"type":"v1","addr":"172.21.15.151:6803","nonce":669937998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":669937998},{"type":"v1","addr":"172.21.15.151:6805","nonce":669937998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":669937998},{"type":"v1","addr":"172.21.15.151:6809","nonce":669937998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":669937998},{"type":"v1","addr":"172.21.15.151:6807","nonce":669937998}]},"public_addr":"172.21.15.151:6803/669937998","cluster_addr":"172.21.15.151:6805/669937998","heartbeat_back_addr":"172.21.15.151:6809/669937998","heartbeat_front_addr":"172.21.15.151:6807/669937998","state":["exists","up"]},{"osd":1,"uuid":"e7d192c3-9624-4625-9ccb-a76c114d8aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":1356168171},{"type":"v1","addr":"172.21.15.165:6801","nonce":1356168171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":1356168171},{"type":"v1","addr":"172.21.15.165:6803","nonce":1356168171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":1356168171},{"type":"v1","addr":"172.21.15.165:6807","nonce":1356168171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":1356168171},{"type":"v1","addr":"172.21.15.165:6805","nonce":1356168171}]},"public_addr":"172.21.15.165:6801/1356168171","cluster_addr":"172.21.15.165:6803/1356168171","heartbeat_back_addr":"172.21.15.165:6807/1356168171","heartbeat_front_addr":"172.21.15.165:6805/1356168171","state":["exists","up"]},{"osd":2,"uuid":"0631df59-1ecc-48b4-b00a-48b0118dfc24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":237502635},{"type":"v1","addr":"172.21.15.172:6801","nonce":237502635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":237502635},{"type":"v1","addr":"172.21.15.172:6803","nonce":237502635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":237502635},{"type":"v1","addr":"172.21.15.172:6807","nonce":237502635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":237502635},{"type":"v1","addr":"172.21.15.172:6805","nonce":237502635}]},"public_addr":"172.21.15.172:6801/237502635","cluster_addr":"172.21.15.172:6803/237502635","heartbeat_back_addr":"172.21.15.172:6807/237502635","heartbeat_front_addr":"172.21.15.172:6805/237502635","state":["exists","up"]},{"osd":3,"uuid":"73076e49-2f8a-4a3d-99eb-7b50aff448aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3442279476},{"type":"v1","addr":"172.21.15.151:6811","nonce":3442279476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3442279476},{"type":"v1","addr":"172.21.15.151:6813","nonce":3442279476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":3442279476},{"type":"v1","addr":"172.21.15.151:6817","nonce":3442279476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3442279476},{"type":"v1","addr":"172.21.15.151:6815","nonce":3442279476}]},"public_addr":"172.21.15.151:6811/3442279476","cluster_addr":"172.21.15.151:6813/3442279476","heartbeat_back_addr":"172.21.15.151:6817/3442279476","heartbeat_front_addr":"172.21.15.151:6815/3442279476","state":["exists","up"]},{"osd":4,"uuid":"0230d615-548c-43f0-b8aa-7130f4cbbc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4029980723},{"type":"v1","addr":"172.21.15.165:6809","nonce":4029980723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4029980723},{"type":"v1","addr":"172.21.15.165:6811","nonce":4029980723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4029980723},{"type":"v1","addr":"172.21.15.165:6815","nonce":4029980723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4029980723},{"type":"v1","addr":"172.21.15.165:6813","nonce":4029980723}]},"public_addr":"172.21.15.165:6809/4029980723","cluster_addr":"172.21.15.165:6811/4029980723","heartbeat_back_addr":"172.21.15.165:6815/4029980723","heartbeat_front_addr":"172.21.15.165:6813/4029980723","state":["exists","up"]},{"osd":5,"uuid":"9739f44b-2605-47dd-82fb-d30034742bf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2216578687},{"type":"v1","addr":"172.21.15.172:6809","nonce":2216578687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2216578687},{"type":"v1","addr":"172.21.15.172:6811","nonce":2216578687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2216578687},{"type":"v1","addr":"172.21.15.172:6815","nonce":2216578687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2216578687},{"type":"v1","addr":"172.21.15.172:6813","nonce":2216578687}]},"public_addr":"172.21.15.172:6809/2216578687","cluster_addr":"172.21.15.172:6811/2216578687","heartbeat_back_addr":"172.21.15.172:6815/2216578687","heartbeat_front_addr":"172.21.15.172:6813/2216578687","state":["exists","up"]},{"osd":6,"uuid":"4290baaa-6160-49cb-a4e0-c96f070f2ef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1723252315},{"type":"v1","addr":"172.21.15.151:6819","nonce":1723252315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1723252315},{"type":"v1","addr":"172.21.15.151:6821","nonce":1723252315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":1723252315},{"type":"v1","addr":"172.21.15.151:6825","nonce":1723252315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1723252315},{"type":"v1","addr":"172.21.15.151:6823","nonce":1723252315}]},"public_addr":"172.21.15.151:6819/1723252315","cluster_addr":"172.21.15.151:6821/1723252315","heartbeat_back_addr":"172.21.15.151:6825/1723252315","heartbeat_front_addr":"172.21.15.151:6823/1723252315","state":["exists","up"]},{"osd":7,"uuid":"9f5315f2-036c-4e2f-92ed-3321a9e3c860","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":1977803781},{"type":"v1","addr":"172.21.15.165:6817","nonce":1977803781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":1977803781},{"type":"v1","addr":"172.21.15.165:6819","nonce":1977803781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":1977803781},{"type":"v1","addr":"172.21.15.165:6823","nonce":1977803781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":1977803781},{"type":"v1","addr":"172.21.15.165:6821","nonce":1977803781}]},"public_addr":"172.21.15.165:6817/1977803781","cluster_addr":"172.21.15.165:6819/1977803781","heartbeat_back_addr":"172.21.15.165:6823/1977803781","heartbeat_front_addr":"172.21.15.165:6821/1977803781","state":["exists","up"]},{"osd":8,"uuid":"8ddbe36d-cea7-45ef-a3ca-044a0e64ff85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":491199512},{"type":"v1","addr":"172.21.15.172:6817","nonce":491199512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":491199512},{"type":"v1","addr":"172.21.15.172:6819","nonce":491199512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":491199512},{"type":"v1","addr":"172.21.15.172:6823","nonce":491199512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":491199512},{"type":"v1","addr":"172.21.15.172:6821","nonce":491199512}]},"public_addr":"172.21.15.172:6817/491199512","cluster_addr":"172.21.15.172:6819/491199512","heartbeat_back_addr":"172.21.15.172:6823/491199512","heartbeat_front_addr":"172.21.15.172:6821/491199512","state":["exists","up"]},{"osd":9,"uuid":"40654ba0-1bd9-4b54-97ae-f1ed6c4785d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":1818923828},{"type":"v1","addr":"172.21.15.151:6827","nonce":1818923828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":1818923828},{"type":"v1","addr":"172.21.15.151:6829","nonce":1818923828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1818923828},{"type":"v1","addr":"172.21.15.151:6833","nonce":1818923828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":1818923828},{"type":"v1","addr":"172.21.15.151:6831","nonce":1818923828}]},"public_addr":"172.21.15.151:6827/1818923828","cluster_addr":"172.21.15.151:6829/1818923828","heartbeat_back_addr":"172.21.15.151:6833/1818923828","heartbeat_front_addr":"172.21.15.151:6831/1818923828","state":["exists","up"]},{"osd":10,"uuid":"19e75a1a-7bd0-41d9-b0cc-0fa4d1475dda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":2632621809},{"type":"v1","addr":"172.21.15.165:6825","nonce":2632621809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":2632621809},{"type":"v1","addr":"172.21.15.165:6827","nonce":2632621809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":2632621809},{"type":"v1","addr":"172.21.15.165:6831","nonce":2632621809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":2632621809},{"type":"v1","addr":"172.21.15.165:6829","nonce":2632621809}]},"public_addr":"172.21.15.165:6825/2632621809","cluster_addr":"172.21.15.165:6827/2632621809","heartbeat_back_addr":"172.21.15.165:6831/2632621809","heartbeat_front_addr":"172.21.15.165:6829/2632621809","state":["exists","up"]},{"osd":11,"uuid":"aa84e5f5-93a3-4375-8544-a1175f0eb06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":886604365},{"type":"v1","addr":"172.21.15.172:6825","nonce":886604365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":886604365},{"type":"v1","addr":"172.21.15.172:6827","nonce":886604365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":886604365},{"type":"v1","addr":"172.21.15.172:6831","nonce":886604365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":886604365},{"type":"v1","addr":"172.21.15.172:6829","nonce":886604365}]},"public_addr":"172.21.15.172:6825/886604365","cluster_addr":"172.21.15.172:6827/886604365","heartbeat_back_addr":"172.21.15.172:6831/886604365","heartbeat_front_addr":"172.21.15.172:6829/886604365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:43:18.296278+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:43:42.860620+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:44:07.073601+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:44:39.631760+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:45:04.771214+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:45:29.606674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:46:03.101851+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:46:28.351995+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:46:53.445435+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:47:27.391179+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:47:53.543638+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:48:18.687811+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.151:6801/796314699":"2024-04-27T01:41:25.234322+0000","172.21.15.151:6800/796314699":"2024-04-27T01:41:25.234322+0000","172.21.15.151:0/1173525554":"2024-04-27T01:41:25.234322+0000","172.21.15.151:0/1342496801":"2024-04-27T01:40:50.492554+0000","172.21.15.151:0/11665926":"2024-04-27T01:40:50.492554+0000","172.21.15.151:0/1765448952":"2024-04-27T01:40:50.492554+0000","172.21.15.151:0/46007430":"2024-04-27T01:41:25.234322+0000","172.21.15.151:6800/244725060":"2024-04-27T01:40:50.492554+0000","172.21.15.151:0/3703862543":"2024-04-27T01:41:25.234322+0000","172.21.15.151:6801/244725060":"2024-04-27T01:40:50.492554+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T01:50:53.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:53.956+0000 7f1eb6000700 1 -- 172.21.15.151:0/1206711271 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f1e940a3cd0 msgr2=0x7f1e940a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:53.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:53.956+0000 7f1eb6000700 1 --2- 172.21.15.151:0/1206711271 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f1e940a3cd0 0x7f1e940a6190 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1e98009680 tx=0x7f1e98009260 comp rx=0 tx=0).stop 2024-04-26T01:50:53.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:53.957+0000 7f1eb6000700 1 -- 172.21.15.151:0/1206711271 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1eb0102a90 msgr2=0x7f1eb019b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:53.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:53.957+0000 7f1eb6000700 1 --2- 172.21.15.151:0/1206711271 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1eb0102a90 0x7f1eb019b300 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1ea000efc0 tx=0x7f1ea000c3a0 comp rx=0 tx=0).stop 2024-04-26T01:50:53.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:53.957+0000 7f1eb6000700 1 -- 172.21.15.151:0/1206711271 shutdown_connections 2024-04-26T01:50:53.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:53.957+0000 7f1eb6000700 1 --2- 172.21.15.151:0/1206711271 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1eb0108a50 0x7f1eb019fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:53.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:53.957+0000 7f1eb6000700 1 --2- 172.21.15.151:0/1206711271 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1eb0102a90 0x7f1eb019b300 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:53.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:53.957+0000 7f1eb6000700 1 --2- 172.21.15.151:0/1206711271 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f1e940a3cd0 0x7f1e940a6190 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:53.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:53.957+0000 7f1eb6000700 1 --2- 172.21.15.151:0/1206711271 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1eb0103450 0x7f1eb019b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:53.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:53.957+0000 7f1eb6000700 1 -- 172.21.15.151:0/1206711271 >> 172.21.15.151:0/1206711271 conn(0x7f1eb0079ba0 msgr2=0x7f1eb010df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:53.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:53.957+0000 7f1eb6000700 1 -- 172.21.15.151:0/1206711271 shutdown_connections 2024-04-26T01:50:53.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:53.957+0000 7f1eb6000700 1 -- 172.21.15.151:0/1206711271 wait complete. 2024-04-26T01:50:54.815 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-26T01:50:54.816 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T01:50:55.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:54 smithi172 ceph-mon[32785]: pgmap v366: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:50:55.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:54 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/1206711271' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:50:55.056 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:54 smithi151 ceph-mon[28057]: pgmap v366: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:50:55.056 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:54 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/1206711271' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:50:55.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:54 smithi165 ceph-mon[32648]: pgmap v366: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:50:55.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:54 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/1206711271' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:50:55.235 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:50:57.029 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:56 smithi151 ceph-mon[28057]: pgmap v367: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:50:57.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:56 smithi172 ceph-mon[32785]: pgmap v367: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:50:57.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:56 smithi165 ceph-mon[32648]: pgmap v367: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:50:57.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:57.672+0000 7f10efe00700 1 -- 172.21.15.151:0/634367448 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10e8102a90 msgr2=0x7f10e8102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:57.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:57.672+0000 7f10efe00700 1 --2- 172.21.15.151:0/634367448 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10e8102a90 0x7f10e8102f10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f10d4009990 tx=0x7f10d4009ca0 comp rx=0 tx=0).stop 2024-04-26T01:50:57.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:57.672+0000 7f10efe00700 1 -- 172.21.15.151:0/634367448 shutdown_connections 2024-04-26T01:50:57.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:57.672+0000 7f10efe00700 1 --2- 172.21.15.151:0/634367448 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10e8103450 0x7f10e810f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:57.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:57.672+0000 7f10efe00700 1 --2- 172.21.15.151:0/634367448 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10e8108a50 0x7f10e8108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:57.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:57.672+0000 7f10efe00700 1 --2- 172.21.15.151:0/634367448 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10e8102a90 0x7f10e8102f10 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:57.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:57.672+0000 7f10efe00700 1 -- 172.21.15.151:0/634367448 >> 172.21.15.151:0/634367448 conn(0x7f10e8079ba0 msgr2=0x7f10e8079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:57.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:57.672+0000 7f10efe00700 1 -- 172.21.15.151:0/634367448 shutdown_connections 2024-04-26T01:50:57.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:57.672+0000 7f10efe00700 1 -- 172.21.15.151:0/634367448 wait complete. 2024-04-26T01:50:57.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:57.673+0000 7f10efe00700 1 Processor -- start 2024-04-26T01:50:57.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:57.673+0000 7f10efe00700 1 -- start start 2024-04-26T01:50:57.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:57.673+0000 7f10efe00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10e8102a90 0x7f10e8074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:57.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:57.674+0000 7f10efe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10e8103450 0x7f10e8075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:57.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:57.674+0000 7f10efe00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10e8108a50 0x7f10e80757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:57.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:57.674+0000 7f10efe00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f10e8112c50 con 0x7f10e8103450 2024-04-26T01:50:57.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:57.674+0000 7f10efe00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f10e8112ad0 con 0x7f10e8102a90 2024-04-26T01:50:57.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:57.674+0000 7f10efe00700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f10e8112dd0 con 0x7f10e8108a50 2024-04-26T01:50:57.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:57.674+0000 7f10ed400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10e8102a90 0x7f10e8074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:57.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:57.674+0000 7f10ed400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10e8102a90 0x7f10e8074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:41056/0 (socket says 172.21.15.151:41056) 2024-04-26T01:50:57.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:57.674+0000 7f10ed400700 1 -- 172.21.15.151:0/767954066 learned_addr learned my addr 172.21.15.151:0/767954066 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:50:57.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:57.674+0000 7f10eca00700 1 --2- 172.21.15.151:0/767954066 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10e8103450 0x7f10e8075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:57.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:57.674+0000 7f10ede00700 1 --2- 172.21.15.151:0/767954066 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10e8108a50 0x7f10e80757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:57.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:57.674+0000 7f10ed400700 1 -- 172.21.15.151:0/767954066 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10e8108a50 msgr2=0x7f10e80757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:57.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:57.674+0000 7f10ed400700 1 --2- 172.21.15.151:0/767954066 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10e8108a50 0x7f10e80757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:57.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:57.674+0000 7f10ed400700 1 -- 172.21.15.151:0/767954066 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10e8103450 msgr2=0x7f10e8075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:57.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:57.674+0000 7f10ed400700 1 --2- 172.21.15.151:0/767954066 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10e8103450 0x7f10e8075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:57.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:57.674+0000 7f10ed400700 1 -- 172.21.15.151:0/767954066 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10e81a7710 con 0x7f10e8102a90 2024-04-26T01:50:57.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:57.675+0000 7f10ed400700 1 --2- 172.21.15.151:0/767954066 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10e8102a90 0x7f10e8074d40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f10dc00e9c0 tx=0x7f10dc00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:57.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:57.675+0000 7f10e6000700 1 -- 172.21.15.151:0/767954066 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10dc00cbd0 con 0x7f10e8102a90 2024-04-26T01:50:57.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:57.675+0000 7f10e6000700 1 -- 172.21.15.151:0/767954066 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10dc00ee60 con 0x7f10e8102a90 2024-04-26T01:50:57.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:57.675+0000 7f10efe00700 1 -- 172.21.15.151:0/767954066 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10e81a7a00 con 0x7f10e8102a90 2024-04-26T01:50:57.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:57.675+0000 7f10e6000700 1 -- 172.21.15.151:0/767954066 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10dc0055c0 con 0x7f10e8102a90 2024-04-26T01:50:57.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:57.675+0000 7f10efe00700 1 -- 172.21.15.151:0/767954066 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10e81a7f40 con 0x7f10e8102a90 2024-04-26T01:50:57.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:57.676+0000 7f10efe00700 1 -- 172.21.15.151:0/767954066 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10e8069730 con 0x7f10e8102a90 2024-04-26T01:50:57.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:57.678+0000 7f10e6000700 1 -- 172.21.15.151:0/767954066 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f10dc004750 con 0x7f10e8102a90 2024-04-26T01:50:57.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:57.680+0000 7f10e6000700 1 --2- 172.21.15.151:0/767954066 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f10d00a3d50 0x7f10d00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:57.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:57.680+0000 7f10e6000700 1 -- 172.21.15.151:0/767954066 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f10dc0d06d0 con 0x7f10e8102a90 2024-04-26T01:50:57.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:57.680+0000 7f10eca00700 1 --2- 172.21.15.151:0/767954066 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f10d00a3d50 0x7f10d00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:57.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:57.680+0000 7f10eca00700 1 --2- 172.21.15.151:0/767954066 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f10d00a3d50 0x7f10d00a6210 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f10e807a2e0 tx=0x7f10d400b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:57.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:57.684+0000 7f10e6000700 1 -- 172.21.15.151:0/767954066 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f10dc09cf50 con 0x7f10e8102a90 2024-04-26T01:50:57.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:57.915+0000 7f10efe00700 1 -- 172.21.15.151:0/767954066 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f10e802df80 con 0x7f10e8102a90 2024-04-26T01:50:57.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:57.917+0000 7f10e6000700 1 -- 172.21.15.151:0/767954066 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+22599 (secure 0 0 0) 0x7f10dc09c850 con 0x7f10e8102a90 2024-04-26T01:50:57.915 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:50:57.916 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":92,"fsid":"c0e49e64-036d-11ef-bc93-c7b262605968","created":"2024-04-26T01:40:07.483197+0000","modified":"2024-04-26T01:50:26.985192+0000","last_up_change":"2024-04-26T01:48:21.167691+0000","last_in_change":"2024-04-26T01:48:00.556990+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T01:44:11.411722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T01:49:31.756032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T01:49:36.529819+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ca27e814-a982-482c-ae1a-e596aac5c59a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":669937998},{"type":"v1","addr":"172.21.15.151:6803","nonce":669937998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":669937998},{"type":"v1","addr":"172.21.15.151:6805","nonce":669937998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":669937998},{"type":"v1","addr":"172.21.15.151:6809","nonce":669937998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":669937998},{"type":"v1","addr":"172.21.15.151:6807","nonce":669937998}]},"public_addr":"172.21.15.151:6803/669937998","cluster_addr":"172.21.15.151:6805/669937998","heartbeat_back_addr":"172.21.15.151:6809/669937998","heartbeat_front_addr":"172.21.15.151:6807/669937998","state":["exists","up"]},{"osd":1,"uuid":"e7d192c3-9624-4625-9ccb-a76c114d8aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":1356168171},{"type":"v1","addr":"172.21.15.165:6801","nonce":1356168171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":1356168171},{"type":"v1","addr":"172.21.15.165:6803","nonce":1356168171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":1356168171},{"type":"v1","addr":"172.21.15.165:6807","nonce":1356168171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":1356168171},{"type":"v1","addr":"172.21.15.165:6805","nonce":1356168171}]},"public_addr":"172.21.15.165:6801/1356168171","cluster_addr":"172.21.15.165:6803/1356168171","heartbeat_back_addr":"172.21.15.165:6807/1356168171","heartbeat_front_addr":"172.21.15.165:6805/1356168171","state":["exists","up"]},{"osd":2,"uuid":"0631df59-1ecc-48b4-b00a-48b0118dfc24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":237502635},{"type":"v1","addr":"172.21.15.172:6801","nonce":237502635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":237502635},{"type":"v1","addr":"172.21.15.172:6803","nonce":237502635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":237502635},{"type":"v1","addr":"172.21.15.172:6807","nonce":237502635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":237502635},{"type":"v1","addr":"172.21.15.172:6805","nonce":237502635}]},"public_addr":"172.21.15.172:6801/237502635","cluster_addr":"172.21.15.172:6803/237502635","heartbeat_back_addr":"172.21.15.172:6807/237502635","heartbeat_front_addr":"172.21.15.172:6805/237502635","state":["exists","up"]},{"osd":3,"uuid":"73076e49-2f8a-4a3d-99eb-7b50aff448aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3442279476},{"type":"v1","addr":"172.21.15.151:6811","nonce":3442279476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3442279476},{"type":"v1","addr":"172.21.15.151:6813","nonce":3442279476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":3442279476},{"type":"v1","addr":"172.21.15.151:6817","nonce":3442279476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3442279476},{"type":"v1","addr":"172.21.15.151:6815","nonce":3442279476}]},"public_addr":"172.21.15.151:6811/3442279476","cluster_addr":"172.21.15.151:6813/3442279476","heartbeat_back_addr":"172.21.15.151:6817/3442279476","heartbeat_front_addr":"172.21.15.151:6815/3442279476","state":["exists","up"]},{"osd":4,"uuid":"0230d615-548c-43f0-b8aa-7130f4cbbc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4029980723},{"type":"v1","addr":"172.21.15.165:6809","nonce":4029980723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4029980723},{"type":"v1","addr":"172.21.15.165:6811","nonce":4029980723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4029980723},{"type":"v1","addr":"172.21.15.165:6815","nonce":4029980723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4029980723},{"type":"v1","addr":"172.21.15.165:6813","nonce":4029980723}]},"public_addr":"172.21.15.165:6809/4029980723","cluster_addr":"172.21.15.165:6811/4029980723","heartbeat_back_addr":"172.21.15.165:6815/4029980723","heartbeat_front_addr":"172.21.15.165:6813/4029980723","state":["exists","up"]},{"osd":5,"uuid":"9739f44b-2605-47dd-82fb-d30034742bf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2216578687},{"type":"v1","addr":"172.21.15.172:6809","nonce":2216578687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2216578687},{"type":"v1","addr":"172.21.15.172:6811","nonce":2216578687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2216578687},{"type":"v1","addr":"172.21.15.172:6815","nonce":2216578687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2216578687},{"type":"v1","addr":"172.21.15.172:6813","nonce":2216578687}]},"public_addr":"172.21.15.172:6809/2216578687","cluster_addr":"172.21.15.172:6811/2216578687","heartbeat_back_addr":"172.21.15.172:6815/2216578687","heartbeat_front_addr":"172.21.15.172:6813/2216578687","state":["exists","up"]},{"osd":6,"uuid":"4290baaa-6160-49cb-a4e0-c96f070f2ef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1723252315},{"type":"v1","addr":"172.21.15.151:6819","nonce":1723252315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1723252315},{"type":"v1","addr":"172.21.15.151:6821","nonce":1723252315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":1723252315},{"type":"v1","addr":"172.21.15.151:6825","nonce":1723252315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1723252315},{"type":"v1","addr":"172.21.15.151:6823","nonce":1723252315}]},"public_addr":"172.21.15.151:6819/1723252315","cluster_addr":"172.21.15.151:6821/1723252315","heartbeat_back_addr":"172.21.15.151:6825/1723252315","heartbeat_front_addr":"172.21.15.151:6823/1723252315","state":["exists","up"]},{"osd":7,"uuid":"9f5315f2-036c-4e2f-92ed-3321a9e3c860","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":1977803781},{"type":"v1","addr":"172.21.15.165:6817","nonce":1977803781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":1977803781},{"type":"v1","addr":"172.21.15.165:6819","nonce":1977803781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":1977803781},{"type":"v1","addr":"172.21.15.165:6823","nonce":1977803781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":1977803781},{"type":"v1","addr":"172.21.15.165:6821","nonce":1977803781}]},"public_addr":"172.21.15.165:6817/1977803781","cluster_addr":"172.21.15.165:6819/1977803781","heartbeat_back_addr":"172.21.15.165:6823/1977803781","heartbeat_front_addr":"172.21.15.165:6821/1977803781","state":["exists","up"]},{"osd":8,"uuid":"8ddbe36d-cea7-45ef-a3ca-044a0e64ff85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":491199512},{"type":"v1","addr":"172.21.15.172:6817","nonce":491199512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":491199512},{"type":"v1","addr":"172.21.15.172:6819","nonce":491199512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":491199512},{"type":"v1","addr":"172.21.15.172:6823","nonce":491199512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":491199512},{"type":"v1","addr":"172.21.15.172:6821","nonce":491199512}]},"public_addr":"172.21.15.172:6817/491199512","cluster_addr":"172.21.15.172:6819/491199512","heartbeat_back_addr":"172.21.15.172:6823/491199512","heartbeat_front_addr":"172.21.15.172:6821/491199512","state":["exists","up"]},{"osd":9,"uuid":"40654ba0-1bd9-4b54-97ae-f1ed6c4785d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":1818923828},{"type":"v1","addr":"172.21.15.151:6827","nonce":1818923828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":1818923828},{"type":"v1","addr":"172.21.15.151:6829","nonce":1818923828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1818923828},{"type":"v1","addr":"172.21.15.151:6833","nonce":1818923828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":1818923828},{"type":"v1","addr":"172.21.15.151:6831","nonce":1818923828}]},"public_addr":"172.21.15.151:6827/1818923828","cluster_addr":"172.21.15.151:6829/1818923828","heartbeat_back_addr":"172.21.15.151:6833/1818923828","heartbeat_front_addr":"172.21.15.151:6831/1818923828","state":["exists","up"]},{"osd":10,"uuid":"19e75a1a-7bd0-41d9-b0cc-0fa4d1475dda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":2632621809},{"type":"v1","addr":"172.21.15.165:6825","nonce":2632621809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":2632621809},{"type":"v1","addr":"172.21.15.165:6827","nonce":2632621809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":2632621809},{"type":"v1","addr":"172.21.15.165:6831","nonce":2632621809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":2632621809},{"type":"v1","addr":"172.21.15.165:6829","nonce":2632621809}]},"public_addr":"172.21.15.165:6825/2632621809","cluster_addr":"172.21.15.165:6827/2632621809","heartbeat_back_addr":"172.21.15.165:6831/2632621809","heartbeat_front_addr":"172.21.15.165:6829/2632621809","state":["exists","up"]},{"osd":11,"uuid":"aa84e5f5-93a3-4375-8544-a1175f0eb06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":886604365},{"type":"v1","addr":"172.21.15.172:6825","nonce":886604365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":886604365},{"type":"v1","addr":"172.21.15.172:6827","nonce":886604365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":886604365},{"type":"v1","addr":"172.21.15.172:6831","nonce":886604365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":886604365},{"type":"v1","addr":"172.21.15.172:6829","nonce":886604365}]},"public_addr":"172.21.15.172:6825/886604365","cluster_addr":"172.21.15.172:6827/886604365","heartbeat_back_addr":"172.21.15.172:6831/886604365","heartbeat_front_addr":"172.21.15.172:6829/886604365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:43:18.296278+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:43:42.860620+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:44:07.073601+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:44:39.631760+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:45:04.771214+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:45:29.606674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:46:03.101851+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:46:28.351995+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:46:53.445435+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:47:27.391179+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:47:53.543638+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:48:18.687811+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.151:6801/796314699":"2024-04-27T01:41:25.234322+0000","172.21.15.151:6800/796314699":"2024-04-27T01:41:25.234322+0000","172.21.15.151:0/1173525554":"2024-04-27T01:41:25.234322+0000","172.21.15.151:0/1342496801":"2024-04-27T01:40:50.492554+0000","172.21.15.151:0/11665926":"2024-04-27T01:40:50.492554+0000","172.21.15.151:0/1765448952":"2024-04-27T01:40:50.492554+0000","172.21.15.151:0/46007430":"2024-04-27T01:41:25.234322+0000","172.21.15.151:6800/244725060":"2024-04-27T01:40:50.492554+0000","172.21.15.151:0/3703862543":"2024-04-27T01:41:25.234322+0000","172.21.15.151:6801/244725060":"2024-04-27T01:40:50.492554+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T01:50:57.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:57.920+0000 7f10efe00700 1 -- 172.21.15.151:0/767954066 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f10d00a3d50 msgr2=0x7f10d00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:57.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:57.920+0000 7f10efe00700 1 --2- 172.21.15.151:0/767954066 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f10d00a3d50 0x7f10d00a6210 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f10e807a2e0 tx=0x7f10d400b260 comp rx=0 tx=0).stop 2024-04-26T01:50:57.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:57.920+0000 7f10efe00700 1 -- 172.21.15.151:0/767954066 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10e8102a90 msgr2=0x7f10e8074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:57.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:57.920+0000 7f10efe00700 1 --2- 172.21.15.151:0/767954066 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10e8102a90 0x7f10e8074d40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f10dc00e9c0 tx=0x7f10dc00ecd0 comp rx=0 tx=0).stop 2024-04-26T01:50:57.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:57.920+0000 7f10efe00700 1 -- 172.21.15.151:0/767954066 shutdown_connections 2024-04-26T01:50:57.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:57.920+0000 7f10efe00700 1 --2- 172.21.15.151:0/767954066 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10e8108a50 0x7f10e80757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:57.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:57.920+0000 7f10efe00700 1 --2- 172.21.15.151:0/767954066 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10e8102a90 0x7f10e8074d40 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:57.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:57.920+0000 7f10efe00700 1 --2- 172.21.15.151:0/767954066 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f10d00a3d50 0x7f10d00a6210 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:57.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:57.920+0000 7f10efe00700 1 --2- 172.21.15.151:0/767954066 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10e8103450 0x7f10e8075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:57.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:57.920+0000 7f10efe00700 1 -- 172.21.15.151:0/767954066 >> 172.21.15.151:0/767954066 conn(0x7f10e8079ba0 msgr2=0x7f10e8101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:57.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:57.920+0000 7f10efe00700 1 -- 172.21.15.151:0/767954066 shutdown_connections 2024-04-26T01:50:57.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:50:57.920+0000 7f10efe00700 1 -- 172.21.15.151:0/767954066 wait complete. 2024-04-26T01:50:58.734 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-26T01:50:58.735 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-26T01:50:58.735 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-26T01:50:58.736 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-26T01:50:58.736 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-26T01:50:58.737 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-26T01:50:58.737 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-26T01:50:58.738 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-26T01:50:58.738 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-26T01:50:58.739 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-26T01:50:58.739 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-26T01:50:58.740 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-26T01:50:58.906 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:58 smithi151 ceph-mon[28057]: pgmap v368: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T01:50:58.906 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:50:58 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/767954066' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:50:59.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:58 smithi172 ceph-mon[32785]: pgmap v368: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T01:50:59.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:50:58 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/767954066' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:50:59.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:58 smithi165 ceph-mon[32648]: pgmap v368: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T01:50:59.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:50:58 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/767954066' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:50:59.789 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:50:59.853 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:51:00.179 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:51:00.180 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:51:00.180 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:51:01.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:51:00 smithi172 ceph-mon[32785]: pgmap v369: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:51:01.061 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:51:00 smithi151 ceph-mon[28057]: pgmap v369: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:51:01.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:51:00 smithi165 ceph-mon[32648]: pgmap v369: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:51:02.820 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:51:02.829 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:51:02.833 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:51:02.838 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:51:02.839 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:51:02.840 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:51:02.841 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:51:03.007 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:51:02 smithi151 ceph-mon[28057]: pgmap v370: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:51:03.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:51:02 smithi172 ceph-mon[32785]: pgmap v370: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:51:03.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:51:02 smithi165 ceph-mon[32648]: pgmap v370: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:51:05.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:51:04 smithi172 ceph-mon[32785]: pgmap v371: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-04-26T01:51:05.073 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:51:04 smithi151 ceph-mon[28057]: pgmap v371: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-04-26T01:51:05.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:51:04 smithi165 ceph-mon[32648]: pgmap v371: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-04-26T01:51:06.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:51:06 smithi151 ceph-mon[28057]: pgmap v372: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-04-26T01:51:07.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:51:06 smithi172 ceph-mon[32785]: pgmap v372: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-04-26T01:51:07.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:51:06 smithi165 ceph-mon[32648]: pgmap v372: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-04-26T01:51:07.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:07.342+0000 7f1266600700 1 -- 172.21.15.151:0/1439256152 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1260105870 msgr2=0x7f12601098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:07.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:07.342+0000 7f1266600700 1 --2- 172.21.15.151:0/1439256152 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1260105870 0x7f12601098c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f124c009990 tx=0x7f124c009ca0 comp rx=0 tx=0).stop 2024-04-26T01:51:07.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:07.342+0000 7f1266600700 1 -- 172.21.15.151:0/1439256152 shutdown_connections 2024-04-26T01:51:07.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:07.342+0000 7f1266600700 1 --2- 172.21.15.151:0/1439256152 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1260109fa0 0x7f1260111b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:07.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:07.342+0000 7f1266600700 1 --2- 172.21.15.151:0/1439256152 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1260104ec0 0x7f12601052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:07.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:07.342+0000 7f1266600700 1 --2- 172.21.15.151:0/1439256152 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1260105870 0x7f12601098c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:07.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:07.342+0000 7f1266600700 1 -- 172.21.15.151:0/1439256152 >> 172.21.15.151:0/1439256152 conn(0x7f1260100750 msgr2=0x7f1260102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:07.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:07.342+0000 7f1266600700 1 -- 172.21.15.151:0/1439256152 shutdown_connections 2024-04-26T01:51:07.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:07.342+0000 7f1266600700 1 -- 172.21.15.151:0/1439256152 wait complete. 2024-04-26T01:51:07.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:07.343+0000 7f1266600700 1 Processor -- start 2024-04-26T01:51:07.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:07.343+0000 7f1266600700 1 -- start start 2024-04-26T01:51:07.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:07.344+0000 7f1266600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1260104ec0 0x7f126019b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:07.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:07.344+0000 7f1266600700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1260105870 0x7f126019b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:07.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:07.344+0000 7f1266600700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1260109fa0 0x7f126019fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:07.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:07.344+0000 7f1266600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1260114e80 con 0x7f1260104ec0 2024-04-26T01:51:07.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:07.344+0000 7f1266600700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1260114d00 con 0x7f1260109fa0 2024-04-26T01:51:07.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:07.344+0000 7f1266600700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1260115000 con 0x7f1260105870 2024-04-26T01:51:07.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:07.344+0000 7f125f400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1260104ec0 0x7f126019b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:07.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:07.344+0000 7f125fe00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1260109fa0 0x7f126019fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:07.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:07.344+0000 7f125ea00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1260105870 0x7f126019b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:07.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:07.344+0000 7f125ea00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1260105870 0x7f126019b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:50450/0 (socket says 172.21.15.151:50450) 2024-04-26T01:51:07.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:07.344+0000 7f125ea00700 1 -- 172.21.15.151:0/2403171391 learned_addr learned my addr 172.21.15.151:0/2403171391 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:51:07.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:07.344+0000 7f125fe00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1260109fa0 0x7f126019fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:35046/0 (socket says 172.21.15.151:35046) 2024-04-26T01:51:07.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:07.345+0000 7f125fe00700 1 -- 172.21.15.151:0/2403171391 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1260105870 msgr2=0x7f126019b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:07.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:07.345+0000 7f125fe00700 1 --2- 172.21.15.151:0/2403171391 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1260105870 0x7f126019b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:07.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:07.345+0000 7f125fe00700 1 -- 172.21.15.151:0/2403171391 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1260104ec0 msgr2=0x7f126019b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:07.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:07.345+0000 7f125fe00700 1 --2- 172.21.15.151:0/2403171391 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1260104ec0 0x7f126019b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:07.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:07.345+0000 7f125fe00700 1 -- 172.21.15.151:0/2403171391 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12601a0360 con 0x7f1260109fa0 2024-04-26T01:51:07.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:07.345+0000 7f125fe00700 1 --2- 172.21.15.151:0/2403171391 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1260109fa0 0x7f126019fc20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f125800efc0 tx=0x7f125800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:07.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:07.345+0000 7f1257e00700 1 -- 172.21.15.151:0/2403171391 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12580092e0 con 0x7f1260109fa0 2024-04-26T01:51:07.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:07.345+0000 7f1257e00700 1 -- 172.21.15.151:0/2403171391 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1258004bb0 con 0x7f1260109fa0 2024-04-26T01:51:07.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:07.345+0000 7f1266600700 1 -- 172.21.15.151:0/2403171391 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12601a05f0 con 0x7f1260109fa0 2024-04-26T01:51:07.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:07.345+0000 7f1257e00700 1 -- 172.21.15.151:0/2403171391 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1258004020 con 0x7f1260109fa0 2024-04-26T01:51:07.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:07.345+0000 7f1266600700 1 -- 172.21.15.151:0/2403171391 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12601a7ed0 con 0x7f1260109fa0 2024-04-26T01:51:07.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:07.346+0000 7f1266600700 1 -- 172.21.15.151:0/2403171391 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1260063190 con 0x7f1260109fa0 2024-04-26T01:51:07.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:07.348+0000 7f1257e00700 1 -- 172.21.15.151:0/2403171391 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f125801b030 con 0x7f1260109fa0 2024-04-26T01:51:07.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:07.350+0000 7f1257e00700 1 --2- 172.21.15.151:0/2403171391 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f12440a3d50 0x7f12440a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:07.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:07.350+0000 7f1257e00700 1 -- 172.21.15.151:0/2403171391 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f12580d08b0 con 0x7f1260109fa0 2024-04-26T01:51:07.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:07.350+0000 7f125f400700 1 --2- 172.21.15.151:0/2403171391 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f12440a3d50 0x7f12440a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:07.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:07.350+0000 7f1257e00700 1 --2- 172.21.15.151:0/2403171391 >> [v2:172.21.15.151:6826/1818923828,v1:172.21.15.151:6827/1818923828] conn(0x7f12440ad790 0x7f12440afc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:07.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:07.350+0000 7f1257e00700 1 -- 172.21.15.151:0/2403171391 --> [v2:172.21.15.151:6826/1818923828,v1:172.21.15.151:6827/1818923828] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f12440b0300 con 0x7f12440ad790 2024-04-26T01:51:07.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:07.350+0000 7f1257e00700 1 -- 172.21.15.151:0/2403171391 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f12580d0c60 con 0x7f1260109fa0 2024-04-26T01:51:07.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:07.351+0000 7f125ea00700 1 --2- 172.21.15.151:0/2403171391 >> [v2:172.21.15.151:6826/1818923828,v1:172.21.15.151:6827/1818923828] conn(0x7f12440ad790 0x7f12440afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:07.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:07.351+0000 7f125f400700 1 --2- 172.21.15.151:0/2403171391 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f12440a3d50 0x7f12440a6210 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1250009c60 tx=0x7f1250009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:07.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:07.351+0000 7f125ea00700 1 --2- 172.21.15.151:0/2403171391 >> [v2:172.21.15.151:6826/1818923828,v1:172.21.15.151:6827/1818923828] conn(0x7f12440ad790 0x7f12440afc30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:07.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:07.352+0000 7f1257e00700 1 -- 172.21.15.151:0/2403171391 <== osd.9 v2:172.21.15.151:6826/1818923828 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f12440b0300 con 0x7f12440ad790 2024-04-26T01:51:07.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:07.370+0000 7f1266600700 1 -- 172.21.15.151:0/2403171391 --> [v2:172.21.15.151:6826/1818923828,v1:172.21.15.151:6827/1818923828] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1260063590 con 0x7f12440ad790 2024-04-26T01:51:07.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:07.371+0000 7f1257e00700 1 -- 172.21.15.151:0/2403171391 <== osd.9 v2:172.21.15.151:6826/1818923828 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1260063590 con 0x7f12440ad790 2024-04-26T01:51:07.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:07.371+0000 7f1266600700 1 -- 172.21.15.151:0/2403171391 >> [v2:172.21.15.151:6826/1818923828,v1:172.21.15.151:6827/1818923828] conn(0x7f12440ad790 msgr2=0x7f12440afc30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:07.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:07.371+0000 7f1266600700 1 --2- 172.21.15.151:0/2403171391 >> [v2:172.21.15.151:6826/1818923828,v1:172.21.15.151:6827/1818923828] conn(0x7f12440ad790 0x7f12440afc30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:07.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:07.372+0000 7f1266600700 1 -- 172.21.15.151:0/2403171391 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f12440a3d50 msgr2=0x7f12440a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:07.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:07.372+0000 7f1266600700 1 --2- 172.21.15.151:0/2403171391 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f12440a3d50 0x7f12440a6210 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1250009c60 tx=0x7f1250009260 comp rx=0 tx=0).stop 2024-04-26T01:51:07.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:07.372+0000 7f1266600700 1 -- 172.21.15.151:0/2403171391 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1260109fa0 msgr2=0x7f126019fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:07.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:07.372+0000 7f1266600700 1 --2- 172.21.15.151:0/2403171391 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1260109fa0 0x7f126019fc20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f125800efc0 tx=0x7f125800c3a0 comp rx=0 tx=0).stop 2024-04-26T01:51:07.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:07.372+0000 7f125fe00700 1 -- 172.21.15.151:0/2403171391 reap_dead start 2024-04-26T01:51:07.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:07.372+0000 7f1266600700 1 -- 172.21.15.151:0/2403171391 shutdown_connections 2024-04-26T01:51:07.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:07.372+0000 7f1266600700 1 -- 172.21.15.151:0/2403171391 >> 172.21.15.151:0/2403171391 conn(0x7f1260100750 msgr2=0x7f1260102a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:07.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:07.372+0000 7f1266600700 1 -- 172.21.15.151:0/2403171391 shutdown_connections 2024-04-26T01:51:07.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:07.372+0000 7f1266600700 1 -- 172.21.15.151:0/2403171391 wait complete. 2024-04-26T01:51:08.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:51:08 smithi151 ceph-mon[28057]: pgmap v373: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:51:09.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:51:08 smithi172 ceph-mon[32785]: pgmap v373: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:51:09.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:51:08 smithi165 ceph-mon[32648]: pgmap v373: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:51:10.294 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:05.392+0000 7f8120000700 1 -- 172.21.15.151:0/664058757 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8118102a90 msgr2=0x7f8118102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:10.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:05.392+0000 7f8120000700 1 --2- 172.21.15.151:0/664058757 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8118102a90 0x7f8118102f10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8104009990 tx=0x7f8104009ca0 comp rx=0 tx=0).stop 2024-04-26T01:51:10.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:05.392+0000 7f8120000700 1 -- 172.21.15.151:0/664058757 shutdown_connections 2024-04-26T01:51:10.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:05.392+0000 7f8120000700 1 --2- 172.21.15.151:0/664058757 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8118103450 0x7f811810f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:10.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:05.392+0000 7f8120000700 1 --2- 172.21.15.151:0/664058757 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8118108a50 0x7f8118108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:10.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:05.392+0000 7f8120000700 1 --2- 172.21.15.151:0/664058757 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8118102a90 0x7f8118102f10 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:10.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:05.392+0000 7f8120000700 1 -- 172.21.15.151:0/664058757 >> 172.21.15.151:0/664058757 conn(0x7f8118079ba0 msgr2=0x7f8118079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:10.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:05.392+0000 7f8120000700 1 -- 172.21.15.151:0/664058757 shutdown_connections 2024-04-26T01:51:10.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:05.392+0000 7f8120000700 1 -- 172.21.15.151:0/664058757 wait complete. 2024-04-26T01:51:10.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:05.393+0000 7f8120000700 1 Processor -- start 2024-04-26T01:51:10.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:05.393+0000 7f8120000700 1 -- start start 2024-04-26T01:51:10.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:05.394+0000 7f8120000700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8118102a90 0x7f811819b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:10.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:05.394+0000 7f8120000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8118103450 0x7f811819b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:10.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:05.394+0000 7f8120000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8118108a50 0x7f811819fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:10.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:05.394+0000 7f8120000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8118112c50 con 0x7f8118108a50 2024-04-26T01:51:10.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:05.394+0000 7f8120000700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8118112ad0 con 0x7f8118103450 2024-04-26T01:51:10.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:05.394+0000 7f8120000700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8118112dd0 con 0x7f8118102a90 2024-04-26T01:51:10.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:05.394+0000 7f811d600700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8118102a90 0x7f811819b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:10.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:05.394+0000 7f811d600700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8118102a90 0x7f811819b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:50444/0 (socket says 172.21.15.151:50444) 2024-04-26T01:51:10.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:05.394+0000 7f811d600700 1 -- 172.21.15.151:0/1086167910 learned_addr learned my addr 172.21.15.151:0/1086167910 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:51:10.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:05.394+0000 7f811cc00700 1 --2- 172.21.15.151:0/1086167910 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8118103450 0x7f811819b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:10.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:05.394+0000 7f811e000700 1 --2- 172.21.15.151:0/1086167910 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8118108a50 0x7f811819fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:10.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:05.395+0000 7f811d600700 1 -- 172.21.15.151:0/1086167910 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8118103450 msgr2=0x7f811819b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:10.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:05.395+0000 7f811d600700 1 --2- 172.21.15.151:0/1086167910 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8118103450 0x7f811819b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:10.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:05.395+0000 7f811d600700 1 -- 172.21.15.151:0/1086167910 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8118108a50 msgr2=0x7f811819fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:10.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:05.395+0000 7f811d600700 1 --2- 172.21.15.151:0/1086167910 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8118108a50 0x7f811819fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:10.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:05.395+0000 7f811d600700 1 -- 172.21.15.151:0/1086167910 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81181a03b0 con 0x7f8118102a90 2024-04-26T01:51:10.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:05.395+0000 7f811d600700 1 --2- 172.21.15.151:0/1086167910 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8118102a90 0x7f811819b3a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f810c00e9a0 tx=0x7f810c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:10.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:05.395+0000 7f8116000700 1 -- 172.21.15.151:0/1086167910 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810c00c9c0 con 0x7f8118102a90 2024-04-26T01:51:10.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:05.395+0000 7f8116000700 1 -- 172.21.15.151:0/1086167910 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f810c00cb60 con 0x7f8118102a90 2024-04-26T01:51:10.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:05.395+0000 7f8120000700 1 -- 172.21.15.151:0/1086167910 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81181a06a0 con 0x7f8118102a90 2024-04-26T01:51:10.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:05.395+0000 7f8116000700 1 -- 172.21.15.151:0/1086167910 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810c018680 con 0x7f8118102a90 2024-04-26T01:51:10.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:05.395+0000 7f8120000700 1 -- 172.21.15.151:0/1086167910 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81181a7f20 con 0x7f8118102a90 2024-04-26T01:51:10.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:05.397+0000 7f8120000700 1 -- 172.21.15.151:0/1086167910 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8118063150 con 0x7f8118102a90 2024-04-26T01:51:10.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:05.399+0000 7f8116000700 1 -- 172.21.15.151:0/1086167910 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f810c010430 con 0x7f8118102a90 2024-04-26T01:51:10.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:05.400+0000 7f8116000700 1 --2- 172.21.15.151:0/1086167910 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f81000a3cd0 0x7f81000a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:10.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:05.400+0000 7f8116000700 1 -- 172.21.15.151:0/1086167910 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f810c0d0da0 con 0x7f8118102a90 2024-04-26T01:51:10.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:05.400+0000 7f811cc00700 1 --2- 172.21.15.151:0/1086167910 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f81000a3cd0 0x7f81000a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:10.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:05.401+0000 7f8116000700 1 --2- 172.21.15.151:0/1086167910 >> [v2:172.21.15.172:6808/2216578687,v1:172.21.15.172:6809/2216578687] conn(0x7f81000ad710 0x7f81000afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:10.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:05.401+0000 7f8116000700 1 -- 172.21.15.151:0/1086167910 --> [v2:172.21.15.172:6808/2216578687,v1:172.21.15.172:6809/2216578687] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f810c019ec0 con 0x7f81000ad710 2024-04-26T01:51:10.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:05.401+0000 7f8116000700 1 -- 172.21.15.151:0/1086167910 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f810c0d1150 con 0x7f8118102a90 2024-04-26T01:51:10.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:05.401+0000 7f811e000700 1 --2- 172.21.15.151:0/1086167910 >> [v2:172.21.15.172:6808/2216578687,v1:172.21.15.172:6809/2216578687] conn(0x7f81000ad710 0x7f81000afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:10.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:05.401+0000 7f811cc00700 1 --2- 172.21.15.151:0/1086167910 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f81000a3cd0 0x7f81000a6190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f811807a4a0 tx=0x7f8104005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:10.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:05.402+0000 7f811e000700 1 --2- 172.21.15.151:0/1086167910 >> [v2:172.21.15.172:6808/2216578687,v1:172.21.15.172:6809/2216578687] conn(0x7f81000ad710 0x7f81000afbb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:10.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:05.403+0000 7f8116000700 1 -- 172.21.15.151:0/1086167910 <== osd.5 v2:172.21.15.172:6808/2216578687 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f810c019ec0 con 0x7f81000ad710 2024-04-26T01:51:10.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:05.421+0000 7f8120000700 1 -- 172.21.15.151:0/1086167910 --> [v2:172.21.15.172:6808/2216578687,v1:172.21.15.172:6809/2216578687] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f81180635d0 con 0x7f81000ad710 2024-04-26T01:51:10.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:05.422+0000 7f8116000700 1 -- 172.21.15.151:0/1086167910 <== osd.5 v2:172.21.15.172:6808/2216578687 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f81180635d0 con 0x7f81000ad710 2024-04-26T01:51:10.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:05.422+0000 7f8120000700 1 -- 172.21.15.151:0/1086167910 >> [v2:172.21.15.172:6808/2216578687,v1:172.21.15.172:6809/2216578687] conn(0x7f81000ad710 msgr2=0x7f81000afbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:10.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:05.422+0000 7f8120000700 1 --2- 172.21.15.151:0/1086167910 >> [v2:172.21.15.172:6808/2216578687,v1:172.21.15.172:6809/2216578687] conn(0x7f81000ad710 0x7f81000afbb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:10.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:05.422+0000 7f8120000700 1 -- 172.21.15.151:0/1086167910 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f81000a3cd0 msgr2=0x7f81000a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:10.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:05.422+0000 7f8120000700 1 --2- 172.21.15.151:0/1086167910 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f81000a3cd0 0x7f81000a6190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f811807a4a0 tx=0x7f8104005fd0 comp rx=0 tx=0).stop 2024-04-26T01:51:10.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:05.422+0000 7f8120000700 1 -- 172.21.15.151:0/1086167910 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8118102a90 msgr2=0x7f811819b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:10.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:05.422+0000 7f8120000700 1 --2- 172.21.15.151:0/1086167910 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8118102a90 0x7f811819b3a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f810c00e9a0 tx=0x7f810c00ecb0 comp rx=0 tx=0).stop 2024-04-26T01:51:10.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:05.423+0000 7f811e000700 1 -- 172.21.15.151:0/1086167910 reap_dead start 2024-04-26T01:51:10.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:05.423+0000 7f8120000700 1 -- 172.21.15.151:0/1086167910 shutdown_connections 2024-04-26T01:51:10.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:05.423+0000 7f8120000700 1 -- 172.21.15.151:0/1086167910 >> 172.21.15.151:0/1086167910 conn(0x7f8118079ba0 msgr2=0x7f811810df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:10.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:05.423+0000 7f8120000700 1 -- 172.21.15.151:0/1086167910 shutdown_connections 2024-04-26T01:51:10.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:05.423+0000 7f8120000700 1 -- 172.21.15.151:0/1086167910 wait complete. 2024-04-26T01:51:11.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:51:10 smithi172 ceph-mon[32785]: pgmap v374: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-04-26T01:51:11.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:51:10 smithi165 ceph-mon[32648]: pgmap v374: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-04-26T01:51:11.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:51:10 smithi151 ceph-mon[28057]: pgmap v374: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-04-26T01:51:13.020 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:51:12 smithi151 ceph-mon[28057]: pgmap v375: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:51:13.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:51:12 smithi172 ceph-mon[32785]: pgmap v375: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:51:13.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:51:12 smithi165 ceph-mon[32648]: pgmap v375: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:51:13.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:13.338+0000 7f81fc800700 1 -- 172.21.15.151:0/955560580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81f4105870 msgr2=0x7f81f41098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:13.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:13.338+0000 7f81fc800700 1 --2- 172.21.15.151:0/955560580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81f4105870 0x7f81f41098c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f81e4009920 tx=0x7f81e4009c30 comp rx=0 tx=0).stop 2024-04-26T01:51:13.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:13.338+0000 7f81fc800700 1 -- 172.21.15.151:0/955560580 shutdown_connections 2024-04-26T01:51:13.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:13.338+0000 7f81fc800700 1 --2- 172.21.15.151:0/955560580 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f81f4109fa0 0x7f81f4111b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:13.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:13.338+0000 7f81fc800700 1 --2- 172.21.15.151:0/955560580 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f81f4104ec0 0x7f81f41052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:13.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:13.338+0000 7f81fc800700 1 --2- 172.21.15.151:0/955560580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81f4105870 0x7f81f41098c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:13.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:13.338+0000 7f81fc800700 1 -- 172.21.15.151:0/955560580 >> 172.21.15.151:0/955560580 conn(0x7f81f4100750 msgr2=0x7f81f4102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:13.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:13.338+0000 7f81fc800700 1 -- 172.21.15.151:0/955560580 shutdown_connections 2024-04-26T01:51:13.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:13.338+0000 7f81fc800700 1 -- 172.21.15.151:0/955560580 wait complete. 2024-04-26T01:51:13.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:13.339+0000 7f81fc800700 1 Processor -- start 2024-04-26T01:51:13.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:13.339+0000 7f81fc800700 1 -- start start 2024-04-26T01:51:13.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:13.340+0000 7f81fc800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f81f4104ec0 0x7f81f419b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:13.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:13.340+0000 7f81fc800700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f81f4105870 0x7f81f419b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:13.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:13.340+0000 7f81fc800700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81f4109fa0 0x7f81f419fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:13.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:13.340+0000 7f81fc800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f81f4114e60 con 0x7f81f4104ec0 2024-04-26T01:51:13.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:13.340+0000 7f81fc800700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f81f4114ce0 con 0x7f81f4105870 2024-04-26T01:51:13.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:13.340+0000 7f81fc800700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f81f4114fe0 con 0x7f81f4109fa0 2024-04-26T01:51:13.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:13.340+0000 7f81f9e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f81f4104ec0 0x7f81f419b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:13.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:13.340+0000 7f81f9e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f81f4104ec0 0x7f81f419b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42078/0 (socket says 172.21.15.151:42078) 2024-04-26T01:51:13.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:13.340+0000 7f81f9e00700 1 -- 172.21.15.151:0/2993894772 learned_addr learned my addr 172.21.15.151:0/2993894772 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:51:13.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:13.340+0000 7f81f9400700 1 --2- 172.21.15.151:0/2993894772 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f81f4105870 0x7f81f419b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:13.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:13.340+0000 7f81fa800700 1 --2- 172.21.15.151:0/2993894772 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81f4109fa0 0x7f81f419fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:13.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:13.341+0000 7f81f9e00700 1 -- 172.21.15.151:0/2993894772 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81f4109fa0 msgr2=0x7f81f419fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:13.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:13.341+0000 7f81f9e00700 1 --2- 172.21.15.151:0/2993894772 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81f4109fa0 0x7f81f419fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:13.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:13.341+0000 7f81f9e00700 1 -- 172.21.15.151:0/2993894772 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f81f4105870 msgr2=0x7f81f419b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:13.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:13.341+0000 7f81f9e00700 1 --2- 172.21.15.151:0/2993894772 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f81f4105870 0x7f81f419b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:13.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:13.341+0000 7f81f9e00700 1 -- 172.21.15.151:0/2993894772 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81f41a0470 con 0x7f81f4104ec0 2024-04-26T01:51:13.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:13.341+0000 7f81f9400700 1 --2- 172.21.15.151:0/2993894772 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f81f4105870 0x7f81f419b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:51:13.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:13.341+0000 7f81f9e00700 1 --2- 172.21.15.151:0/2993894772 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f81f4104ec0 0x7f81f419b460 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f81e800e9c0 tx=0x7f81e800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:13.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:13.341+0000 7f81eea00700 1 -- 172.21.15.151:0/2993894772 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e8004a30 con 0x7f81f4104ec0 2024-04-26T01:51:13.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:13.341+0000 7f81fc800700 1 -- 172.21.15.151:0/2993894772 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81f41a0760 con 0x7f81f4104ec0 2024-04-26T01:51:13.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:13.341+0000 7f81eea00700 1 -- 172.21.15.151:0/2993894772 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81e800cc70 con 0x7f81f4104ec0 2024-04-26T01:51:13.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:13.341+0000 7f81eea00700 1 -- 172.21.15.151:0/2993894772 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e8005580 con 0x7f81f4104ec0 2024-04-26T01:51:13.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:13.342+0000 7f81fc800700 1 -- 172.21.15.151:0/2993894772 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81f41a7fe0 con 0x7f81f4104ec0 2024-04-26T01:51:13.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:13.342+0000 7f81fc800700 1 -- 172.21.15.151:0/2993894772 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f81f40630d0 con 0x7f81f4104ec0 2024-04-26T01:51:13.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:13.345+0000 7f81eea00700 1 -- 172.21.15.151:0/2993894772 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f81e800c7c0 con 0x7f81f4104ec0 2024-04-26T01:51:13.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:13.346+0000 7f81eea00700 1 --2- 172.21.15.151:0/2993894772 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f81dc0a3d50 0x7f81dc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:13.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:13.346+0000 7f81eea00700 1 -- 172.21.15.151:0/2993894772 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f81e80d09f0 con 0x7f81f4104ec0 2024-04-26T01:51:13.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:13.346+0000 7f81f9400700 1 --2- 172.21.15.151:0/2993894772 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f81dc0a3d50 0x7f81dc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:13.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:13.347+0000 7f81eea00700 1 --2- 172.21.15.151:0/2993894772 >> [v2:172.21.15.165:6808/4029980723,v1:172.21.15.165:6809/4029980723] conn(0x7f81dc0ad790 0x7f81dc0afc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:13.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:13.347+0000 7f81eea00700 1 -- 172.21.15.151:0/2993894772 --> [v2:172.21.15.165:6808/4029980723,v1:172.21.15.165:6809/4029980723] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f81dc0b0300 con 0x7f81dc0ad790 2024-04-26T01:51:13.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:13.347+0000 7f81eea00700 1 -- 172.21.15.151:0/2993894772 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f81e80d0da0 con 0x7f81f4104ec0 2024-04-26T01:51:13.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:13.347+0000 7f81fa800700 1 --2- 172.21.15.151:0/2993894772 >> [v2:172.21.15.165:6808/4029980723,v1:172.21.15.165:6809/4029980723] conn(0x7f81dc0ad790 0x7f81dc0afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:13.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:13.347+0000 7f81f9400700 1 --2- 172.21.15.151:0/2993894772 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f81dc0a3d50 0x7f81dc0a6210 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f81e40098f0 tx=0x7f81e4019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:13.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:13.348+0000 7f81fa800700 1 --2- 172.21.15.151:0/2993894772 >> [v2:172.21.15.165:6808/4029980723,v1:172.21.15.165:6809/4029980723] conn(0x7f81dc0ad790 0x7f81dc0afc30 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:13.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:13.349+0000 7f81eea00700 1 -- 172.21.15.151:0/2993894772 <== osd.4 v2:172.21.15.165:6808/4029980723 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f81dc0b0300 con 0x7f81dc0ad790 2024-04-26T01:51:13.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:13.367+0000 7f81fc800700 1 -- 172.21.15.151:0/2993894772 --> [v2:172.21.15.165:6808/4029980723,v1:172.21.15.165:6809/4029980723] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f81f4063550 con 0x7f81dc0ad790 2024-04-26T01:51:13.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:13.367+0000 7f81eea00700 1 -- 172.21.15.151:0/2993894772 <== osd.4 v2:172.21.15.165:6808/4029980723 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f81f4063550 con 0x7f81dc0ad790 2024-04-26T01:51:13.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:13.368+0000 7f81fc800700 1 -- 172.21.15.151:0/2993894772 >> [v2:172.21.15.165:6808/4029980723,v1:172.21.15.165:6809/4029980723] conn(0x7f81dc0ad790 msgr2=0x7f81dc0afc30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:13.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:13.368+0000 7f81fc800700 1 --2- 172.21.15.151:0/2993894772 >> [v2:172.21.15.165:6808/4029980723,v1:172.21.15.165:6809/4029980723] conn(0x7f81dc0ad790 0x7f81dc0afc30 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:13.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:13.368+0000 7f81fc800700 1 -- 172.21.15.151:0/2993894772 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f81dc0a3d50 msgr2=0x7f81dc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:13.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:13.368+0000 7f81fc800700 1 --2- 172.21.15.151:0/2993894772 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f81dc0a3d50 0x7f81dc0a6210 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f81e40098f0 tx=0x7f81e4019040 comp rx=0 tx=0).stop 2024-04-26T01:51:13.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:13.368+0000 7f81fc800700 1 -- 172.21.15.151:0/2993894772 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f81f4104ec0 msgr2=0x7f81f419b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:13.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:13.368+0000 7f81fc800700 1 --2- 172.21.15.151:0/2993894772 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f81f4104ec0 0x7f81f419b460 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f81e800e9c0 tx=0x7f81e800ecd0 comp rx=0 tx=0).stop 2024-04-26T01:51:13.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:13.368+0000 7f81fa800700 1 -- 172.21.15.151:0/2993894772 reap_dead start 2024-04-26T01:51:13.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:13.368+0000 7f81fc800700 1 -- 172.21.15.151:0/2993894772 shutdown_connections 2024-04-26T01:51:13.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:13.368+0000 7f81fc800700 1 -- 172.21.15.151:0/2993894772 >> 172.21.15.151:0/2993894772 conn(0x7f81f4100750 msgr2=0x7f81f4102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:13.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:13.369+0000 7f81fc800700 1 -- 172.21.15.151:0/2993894772 shutdown_connections 2024-04-26T01:51:13.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:13.369+0000 7f81fc800700 1 -- 172.21.15.151:0/2993894772 wait complete. 2024-04-26T01:51:13.686 INFO:teuthology.orchestra.run.smithi151.stdout:296352743470 2024-04-26T01:51:13.686 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-26T01:51:14.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:12.624+0000 7fb991000700 1 -- 172.21.15.151:0/3635474066 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb98c102970 msgr2=0x7fb98c102df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:14.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:12.624+0000 7fb991000700 1 --2- 172.21.15.151:0/3635474066 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb98c102970 0x7fb98c102df0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb978009990 tx=0x7fb978009ca0 comp rx=0 tx=0).stop 2024-04-26T01:51:14.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:12.624+0000 7fb991000700 1 -- 172.21.15.151:0/3635474066 shutdown_connections 2024-04-26T01:51:14.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:12.624+0000 7fb991000700 1 --2- 172.21.15.151:0/3635474066 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb98c103330 0x7fb98c10f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:14.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:12.624+0000 7fb991000700 1 --2- 172.21.15.151:0/3635474066 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb98c108970 0x7fb98c108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:14.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:12.624+0000 7fb991000700 1 --2- 172.21.15.151:0/3635474066 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb98c102970 0x7fb98c102df0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:14.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:12.624+0000 7fb991000700 1 -- 172.21.15.151:0/3635474066 >> 172.21.15.151:0/3635474066 conn(0x7fb98c0fe4c0 msgr2=0x7fb98c1008e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:14.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:12.624+0000 7fb991000700 1 -- 172.21.15.151:0/3635474066 shutdown_connections 2024-04-26T01:51:14.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:12.624+0000 7fb991000700 1 -- 172.21.15.151:0/3635474066 wait complete. 2024-04-26T01:51:14.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:12.625+0000 7fb991000700 1 Processor -- start 2024-04-26T01:51:14.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:12.625+0000 7fb991000700 1 -- start start 2024-04-26T01:51:14.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:12.626+0000 7fb991000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb98c102970 0x7fb98c19b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:14.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:12.626+0000 7fb991000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb98c103330 0x7fb98c19b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:14.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:12.626+0000 7fb991000700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb98c108970 0x7fb98c19fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:14.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:12.626+0000 7fb991000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb98c112b60 con 0x7fb98c102970 2024-04-26T01:51:14.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:12.626+0000 7fb991000700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb98c1129e0 con 0x7fb98c103330 2024-04-26T01:51:14.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:12.626+0000 7fb991000700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb98c112ce0 con 0x7fb98c108970 2024-04-26T01:51:14.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:12.626+0000 7fb98b400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb98c102970 0x7fb98c19b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:14.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:12.626+0000 7fb98be00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb98c108970 0x7fb98c19fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:14.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:12.626+0000 7fb98b400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb98c102970 0x7fb98c19b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42060/0 (socket says 172.21.15.151:42060) 2024-04-26T01:51:14.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:12.626+0000 7fb98b400700 1 -- 172.21.15.151:0/2913208696 learned_addr learned my addr 172.21.15.151:0/2913208696 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:51:14.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:12.626+0000 7fb98aa00700 1 --2- 172.21.15.151:0/2913208696 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb98c103330 0x7fb98c19b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:14.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:12.626+0000 7fb98be00700 1 -- 172.21.15.151:0/2913208696 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb98c103330 msgr2=0x7fb98c19b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:14.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:12.626+0000 7fb98be00700 1 --2- 172.21.15.151:0/2913208696 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb98c103330 0x7fb98c19b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:14.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:12.626+0000 7fb98be00700 1 -- 172.21.15.151:0/2913208696 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb98c102970 msgr2=0x7fb98c19b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:14.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:12.626+0000 7fb98be00700 1 --2- 172.21.15.151:0/2913208696 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb98c102970 0x7fb98c19b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:14.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:12.626+0000 7fb98be00700 1 -- 172.21.15.151:0/2913208696 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb98c1a0200 con 0x7fb98c108970 2024-04-26T01:51:14.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:12.626+0000 7fb98b400700 1 --2- 172.21.15.151:0/2913208696 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb98c102970 0x7fb98c19b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:51:14.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:12.627+0000 7fb98be00700 1 --2- 172.21.15.151:0/2913208696 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb98c108970 0x7fb98c19fac0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb98400efc0 tx=0x7fb98400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:14.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:12.627+0000 7fb983e00700 1 -- 172.21.15.151:0/2913208696 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9840092e0 con 0x7fb98c108970 2024-04-26T01:51:14.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:12.627+0000 7fb991000700 1 -- 172.21.15.151:0/2913208696 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb98c1a0490 con 0x7fb98c108970 2024-04-26T01:51:14.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:12.627+0000 7fb983e00700 1 -- 172.21.15.151:0/2913208696 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb984004bb0 con 0x7fb98c108970 2024-04-26T01:51:14.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:12.627+0000 7fb983e00700 1 -- 172.21.15.151:0/2913208696 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb984004020 con 0x7fb98c108970 2024-04-26T01:51:14.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:12.627+0000 7fb991000700 1 -- 172.21.15.151:0/2913208696 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb98c1a7d70 con 0x7fb98c108970 2024-04-26T01:51:14.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:12.628+0000 7fb991000700 1 -- 172.21.15.151:0/2913208696 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb98c0ffc90 con 0x7fb98c108970 2024-04-26T01:51:14.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:12.630+0000 7fb983e00700 1 -- 172.21.15.151:0/2913208696 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fb98401b030 con 0x7fb98c108970 2024-04-26T01:51:14.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:12.631+0000 7fb983e00700 1 --2- 172.21.15.151:0/2913208696 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fb9700a3cd0 0x7fb9700a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:14.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:12.632+0000 7fb983e00700 1 -- 172.21.15.151:0/2913208696 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7fb9840d09d0 con 0x7fb98c108970 2024-04-26T01:51:14.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:12.632+0000 7fb98b400700 1 --2- 172.21.15.151:0/2913208696 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fb9700a3cd0 0x7fb9700a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:14.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:12.632+0000 7fb983e00700 1 --2- 172.21.15.151:0/2913208696 >> [v2:172.21.15.172:6816/491199512,v1:172.21.15.172:6817/491199512] conn(0x7fb9700ad710 0x7fb9700afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:14.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:12.632+0000 7fb983e00700 1 -- 172.21.15.151:0/2913208696 --> [v2:172.21.15.172:6816/491199512,v1:172.21.15.172:6817/491199512] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb98401de50 con 0x7fb9700ad710 2024-04-26T01:51:14.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:12.632+0000 7fb983e00700 1 -- 172.21.15.151:0/2913208696 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fb9840d0d80 con 0x7fb98c108970 2024-04-26T01:51:14.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:12.632+0000 7fb98b400700 1 --2- 172.21.15.151:0/2913208696 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fb9700a3cd0 0x7fb9700a6190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb97c011550 tx=0x7fb97c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:14.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:12.632+0000 7fb98aa00700 1 --2- 172.21.15.151:0/2913208696 >> [v2:172.21.15.172:6816/491199512,v1:172.21.15.172:6817/491199512] conn(0x7fb9700ad710 0x7fb9700afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:14.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:12.633+0000 7fb98aa00700 1 --2- 172.21.15.151:0/2913208696 >> [v2:172.21.15.172:6816/491199512,v1:172.21.15.172:6817/491199512] conn(0x7fb9700ad710 0x7fb9700afbb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:14.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:12.634+0000 7fb983e00700 1 -- 172.21.15.151:0/2913208696 <== osd.8 v2:172.21.15.172:6816/491199512 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb98401de50 con 0x7fb9700ad710 2024-04-26T01:51:14.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:12.652+0000 7fb991000700 1 -- 172.21.15.151:0/2913208696 --> [v2:172.21.15.172:6816/491199512,v1:172.21.15.172:6817/491199512] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb98c1040b0 con 0x7fb9700ad710 2024-04-26T01:51:14.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:12.653+0000 7fb983e00700 1 -- 172.21.15.151:0/2913208696 <== osd.8 v2:172.21.15.172:6816/491199512 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb98c1040b0 con 0x7fb9700ad710 2024-04-26T01:51:14.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:12.653+0000 7fb991000700 1 -- 172.21.15.151:0/2913208696 >> [v2:172.21.15.172:6816/491199512,v1:172.21.15.172:6817/491199512] conn(0x7fb9700ad710 msgr2=0x7fb9700afbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:14.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:12.653+0000 7fb991000700 1 --2- 172.21.15.151:0/2913208696 >> [v2:172.21.15.172:6816/491199512,v1:172.21.15.172:6817/491199512] conn(0x7fb9700ad710 0x7fb9700afbb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:14.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:12.653+0000 7fb991000700 1 -- 172.21.15.151:0/2913208696 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fb9700a3cd0 msgr2=0x7fb9700a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:14.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:12.653+0000 7fb991000700 1 --2- 172.21.15.151:0/2913208696 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fb9700a3cd0 0x7fb9700a6190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb97c011550 tx=0x7fb97c009260 comp rx=0 tx=0).stop 2024-04-26T01:51:14.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:12.653+0000 7fb991000700 1 -- 172.21.15.151:0/2913208696 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb98c108970 msgr2=0x7fb98c19fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:14.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:12.653+0000 7fb991000700 1 --2- 172.21.15.151:0/2913208696 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb98c108970 0x7fb98c19fac0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb98400efc0 tx=0x7fb98400c3a0 comp rx=0 tx=0).stop 2024-04-26T01:51:14.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:12.654+0000 7fb98be00700 1 -- 172.21.15.151:0/2913208696 reap_dead start 2024-04-26T01:51:14.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:12.654+0000 7fb991000700 1 -- 172.21.15.151:0/2913208696 shutdown_connections 2024-04-26T01:51:14.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:12.654+0000 7fb991000700 1 -- 172.21.15.151:0/2913208696 >> 172.21.15.151:0/2913208696 conn(0x7fb98c0fe4c0 msgr2=0x7fb98c10e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:14.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:12.654+0000 7fb991000700 1 -- 172.21.15.151:0/2913208696 shutdown_connections 2024-04-26T01:51:14.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:12.654+0000 7fb991000700 1 -- 172.21.15.151:0/2913208696 wait complete. 2024-04-26T01:51:14.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:14.464+0000 7fc743e00700 1 -- 172.21.15.151:0/1444241116 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc744108f80 msgr2=0x7fc744109400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:14.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:14.464+0000 7fc743e00700 1 --2- 172.21.15.151:0/1444241116 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc744108f80 0x7fc744109400 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc730009990 tx=0x7fc730009ca0 comp rx=0 tx=0).stop 2024-04-26T01:51:14.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:14.464+0000 7fc743e00700 1 -- 172.21.15.151:0/1444241116 shutdown_connections 2024-04-26T01:51:14.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:14.464+0000 7fc743e00700 1 --2- 172.21.15.151:0/1444241116 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc744109940 0x7fc744115de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:14.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:14.464+0000 7fc743e00700 1 --2- 172.21.15.151:0/1444241116 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc74410f030 0x7fc74410f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:14.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:14.464+0000 7fc743e00700 1 --2- 172.21.15.151:0/1444241116 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc744108f80 0x7fc744109400 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:14.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:14.464+0000 7fc743e00700 1 -- 172.21.15.151:0/1444241116 >> 172.21.15.151:0/1444241116 conn(0x7fc7440787d0 msgr2=0x7fc744107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:14.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:14.464+0000 7fc743e00700 1 -- 172.21.15.151:0/1444241116 shutdown_connections 2024-04-26T01:51:14.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:14.464+0000 7fc743e00700 1 -- 172.21.15.151:0/1444241116 wait complete. 2024-04-26T01:51:14.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:14.466+0000 7fc743e00700 1 Processor -- start 2024-04-26T01:51:14.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:14.466+0000 7fc743e00700 1 -- start start 2024-04-26T01:51:14.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:14.466+0000 7fc743e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc744108f80 0x7fc74419f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:14.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:14.466+0000 7fc743e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc744109940 0x7fc74419fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:14.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:14.466+0000 7fc743e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc74410f030 0x7fc7441a3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:14.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:14.466+0000 7fc743e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc744119170 con 0x7fc74410f030 2024-04-26T01:51:14.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:14.466+0000 7fc743e00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc744118ff0 con 0x7fc744108f80 2024-04-26T01:51:14.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:14.466+0000 7fc743e00700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc7441192f0 con 0x7fc744109940 2024-04-26T01:51:14.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:14.467+0000 7fc742000700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc744109940 0x7fc74419fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:14.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:14.467+0000 7fc742000700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc744109940 0x7fc74419fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:52020/0 (socket says 172.21.15.151:52020) 2024-04-26T01:51:14.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:14.467+0000 7fc742000700 1 -- 172.21.15.151:0/3254894570 learned_addr learned my addr 172.21.15.151:0/3254894570 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:51:14.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:14.467+0000 7fc742a00700 1 --2- 172.21.15.151:0/3254894570 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc744108f80 0x7fc74419f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:14.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:14.467+0000 7fc743400700 1 --2- 172.21.15.151:0/3254894570 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc74410f030 0x7fc7441a3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:14.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:14.467+0000 7fc742000700 1 -- 172.21.15.151:0/3254894570 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc744108f80 msgr2=0x7fc74419f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:14.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:14.467+0000 7fc742000700 1 --2- 172.21.15.151:0/3254894570 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc744108f80 0x7fc74419f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:14.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:14.467+0000 7fc742000700 1 -- 172.21.15.151:0/3254894570 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc74410f030 msgr2=0x7fc7441a3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:14.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:14.467+0000 7fc742000700 1 --2- 172.21.15.151:0/3254894570 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc74410f030 0x7fc7441a3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:14.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:14.467+0000 7fc742000700 1 -- 172.21.15.151:0/3254894570 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7441a4690 con 0x7fc744109940 2024-04-26T01:51:14.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:14.468+0000 7fc742a00700 1 --2- 172.21.15.151:0/3254894570 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc744108f80 0x7fc74419f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:51:14.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:14.468+0000 7fc742000700 1 --2- 172.21.15.151:0/3254894570 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc744109940 0x7fc74419fbc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc730000c00 tx=0x7fc7300057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:14.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:14.468+0000 7fc748a00700 1 -- 172.21.15.151:0/3254894570 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc730027070 con 0x7fc744109940 2024-04-26T01:51:14.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:14.468+0000 7fc748a00700 1 -- 172.21.15.151:0/3254894570 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc730022470 con 0x7fc744109940 2024-04-26T01:51:14.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:14.468+0000 7fc748a00700 1 -- 172.21.15.151:0/3254894570 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc730017650 con 0x7fc744109940 2024-04-26T01:51:14.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:14.468+0000 7fc743e00700 1 -- 172.21.15.151:0/3254894570 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7441a4920 con 0x7fc744109940 2024-04-26T01:51:14.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:14.468+0000 7fc743e00700 1 -- 172.21.15.151:0/3254894570 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7441ac200 con 0x7fc744109940 2024-04-26T01:51:14.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:14.469+0000 7fc743e00700 1 -- 172.21.15.151:0/3254894570 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc744063190 con 0x7fc744109940 2024-04-26T01:51:14.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:14.472+0000 7fc748a00700 1 -- 172.21.15.151:0/3254894570 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fc73002a070 con 0x7fc744109940 2024-04-26T01:51:14.471 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:14.473+0000 7fc748a00700 1 --2- 172.21.15.151:0/3254894570 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fc7280a3cd0 0x7fc7280a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:14.471 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:14.473+0000 7fc748a00700 1 -- 172.21.15.151:0/3254894570 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7fc7300d13f0 con 0x7fc744109940 2024-04-26T01:51:14.471 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:14.473+0000 7fc742a00700 1 --2- 172.21.15.151:0/3254894570 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fc7280a3cd0 0x7fc7280a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:14.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:14.473+0000 7fc748a00700 1 --2- 172.21.15.151:0/3254894570 >> [v2:172.21.15.151:6818/1723252315,v1:172.21.15.151:6819/1723252315] conn(0x7fc7280ad710 0x7fc7280afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:14.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:14.474+0000 7fc748a00700 1 -- 172.21.15.151:0/3254894570 --> [v2:172.21.15.151:6818/1723252315,v1:172.21.15.151:6819/1723252315] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc7280b0280 con 0x7fc7280ad710 2024-04-26T01:51:14.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:14.474+0000 7fc748a00700 1 -- 172.21.15.151:0/3254894570 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fc7300d17a0 con 0x7fc744109940 2024-04-26T01:51:14.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:14.474+0000 7fc743400700 1 --2- 172.21.15.151:0/3254894570 >> [v2:172.21.15.151:6818/1723252315,v1:172.21.15.151:6819/1723252315] conn(0x7fc7280ad710 0x7fc7280afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:14.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:14.474+0000 7fc742a00700 1 --2- 172.21.15.151:0/3254894570 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fc7280a3cd0 0x7fc7280a6190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc734009fd0 tx=0x7fc734009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:14.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:14.474+0000 7fc743400700 1 --2- 172.21.15.151:0/3254894570 >> [v2:172.21.15.151:6818/1723252315,v1:172.21.15.151:6819/1723252315] conn(0x7fc7280ad710 0x7fc7280afbb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:14.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:14.476+0000 7fc748a00700 1 -- 172.21.15.151:0/3254894570 <== osd.6 v2:172.21.15.151:6818/1723252315 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc7280b0280 con 0x7fc7280ad710 2024-04-26T01:51:14.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:14.494+0000 7fc743e00700 1 -- 172.21.15.151:0/3254894570 --> [v2:172.21.15.151:6818/1723252315,v1:172.21.15.151:6819/1723252315] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc744063610 con 0x7fc7280ad710 2024-04-26T01:51:14.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:14.494+0000 7fc748a00700 1 -- 172.21.15.151:0/3254894570 <== osd.6 v2:172.21.15.151:6818/1723252315 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc744063610 con 0x7fc7280ad710 2024-04-26T01:51:14.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:14.495+0000 7fc743e00700 1 -- 172.21.15.151:0/3254894570 >> [v2:172.21.15.151:6818/1723252315,v1:172.21.15.151:6819/1723252315] conn(0x7fc7280ad710 msgr2=0x7fc7280afbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:14.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:14.495+0000 7fc743e00700 1 --2- 172.21.15.151:0/3254894570 >> [v2:172.21.15.151:6818/1723252315,v1:172.21.15.151:6819/1723252315] conn(0x7fc7280ad710 0x7fc7280afbb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:14.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:14.495+0000 7fc743e00700 1 -- 172.21.15.151:0/3254894570 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fc7280a3cd0 msgr2=0x7fc7280a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:14.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:14.495+0000 7fc743e00700 1 --2- 172.21.15.151:0/3254894570 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fc7280a3cd0 0x7fc7280a6190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc734009fd0 tx=0x7fc734009260 comp rx=0 tx=0).stop 2024-04-26T01:51:14.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:14.495+0000 7fc743e00700 1 -- 172.21.15.151:0/3254894570 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc744109940 msgr2=0x7fc74419fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:14.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:14.495+0000 7fc743e00700 1 --2- 172.21.15.151:0/3254894570 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc744109940 0x7fc74419fbc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc730000c00 tx=0x7fc7300057c0 comp rx=0 tx=0).stop 2024-04-26T01:51:14.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:14.495+0000 7fc743400700 1 -- 172.21.15.151:0/3254894570 reap_dead start 2024-04-26T01:51:14.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:14.495+0000 7fc743e00700 1 -- 172.21.15.151:0/3254894570 shutdown_connections 2024-04-26T01:51:14.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:14.495+0000 7fc743e00700 1 -- 172.21.15.151:0/3254894570 >> 172.21.15.151:0/3254894570 conn(0x7fc7440787d0 msgr2=0x7fc744079920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:14.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:14.495+0000 7fc743e00700 1 -- 172.21.15.151:0/3254894570 shutdown_connections 2024-04-26T01:51:14.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:14.495+0000 7fc743e00700 1 -- 172.21.15.151:0/3254894570 wait complete. 2024-04-26T01:51:15.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:51:14 smithi172 ceph-mon[32785]: pgmap v376: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:51:15.041 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:51:14 smithi151 ceph-mon[28057]: pgmap v376: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:51:15.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:51:14 smithi165 ceph-mon[32648]: pgmap v376: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:51:15.645 INFO:teuthology.orchestra.run.smithi151.stdout:150323855437 2024-04-26T01:51:15.645 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-26T01:51:16.239 INFO:teuthology.orchestra.run.smithi151.stdout:171798691909 2024-04-26T01:51:16.240 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-26T01:51:16.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:51:16 smithi151 ceph-mon[28057]: pgmap v377: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-26T01:51:17.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:51:16 smithi172 ceph-mon[32785]: pgmap v377: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-26T01:51:17.092 INFO:teuthology.orchestra.run.smithi151.stdout:261993005110 2024-04-26T01:51:17.093 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-26T01:51:17.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:51:16 smithi165 ceph-mon[32648]: pgmap v377: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-26T01:51:19.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:51:18 smithi172 ceph-mon[32785]: pgmap v378: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T01:51:19.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:51:18 smithi165 ceph-mon[32648]: pgmap v378: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T01:51:19.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:51:18 smithi151 ceph-mon[28057]: pgmap v378: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T01:51:19.420 INFO:teuthology.orchestra.run.smithi151.stdout:206158430273 2024-04-26T01:51:19.421 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-26T01:51:20.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:51:20 smithi151 ceph-mon[28057]: pgmap v379: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:51:21.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:51:20 smithi172 ceph-mon[32785]: pgmap v379: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:51:21.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:51:20 smithi165 ceph-mon[32648]: pgmap v379: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:51:21.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:21.925+0000 7fbe19600700 1 -- 172.21.15.151:0/2194529615 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbe141092d0 msgr2=0x7fbe141096b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:21.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:21.925+0000 7fbe19600700 1 --2- 172.21.15.151:0/2194529615 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbe141092d0 0x7fbe141096b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbe040099c0 tx=0x7fbe04009cd0 comp rx=0 tx=0).stop 2024-04-26T01:51:21.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:21.925+0000 7fbe19600700 1 -- 172.21.15.151:0/2194529615 shutdown_connections 2024-04-26T01:51:21.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:21.925+0000 7fbe19600700 1 --2- 172.21.15.151:0/2194529615 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe1410e3b0 0x7fbe14115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:21.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:21.925+0000 7fbe19600700 1 --2- 172.21.15.151:0/2194529615 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbe141092d0 0x7fbe141096b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:21.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:21.925+0000 7fbe19600700 1 --2- 172.21.15.151:0/2194529615 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe14109c80 0x7fbe1410dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:21.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:21.925+0000 7fbe19600700 1 -- 172.21.15.151:0/2194529615 >> 172.21.15.151:0/2194529615 conn(0x7fbe14076210 msgr2=0x7fbe14107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:21.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:21.925+0000 7fbe19600700 1 -- 172.21.15.151:0/2194529615 shutdown_connections 2024-04-26T01:51:21.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:21.926+0000 7fbe19600700 1 -- 172.21.15.151:0/2194529615 wait complete. 2024-04-26T01:51:21.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:21.927+0000 7fbe19600700 1 Processor -- start 2024-04-26T01:51:21.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:21.927+0000 7fbe19600700 1 -- start start 2024-04-26T01:51:21.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:21.927+0000 7fbe19600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbe141092d0 0x7fbe141a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:21.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:21.927+0000 7fbe19600700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe14109c80 0x7fbe141a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:21.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:21.927+0000 7fbe19600700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe1410e3b0 0x7fbe1419f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:21.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:21.927+0000 7fbe19600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbe141192b0 con 0x7fbe141092d0 2024-04-26T01:51:21.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:21.927+0000 7fbe19600700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbe14119130 con 0x7fbe1410e3b0 2024-04-26T01:51:21.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:21.927+0000 7fbe19600700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbe14119430 con 0x7fbe14109c80 2024-04-26T01:51:21.926 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:21.928+0000 7fbe12a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbe141092d0 0x7fbe141a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:21.926 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:21.928+0000 7fbe12a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbe141092d0 0x7fbe141a53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:40362/0 (socket says 172.21.15.151:40362) 2024-04-26T01:51:21.926 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:21.928+0000 7fbe13400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe1410e3b0 0x7fbe1419f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:21.926 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:21.928+0000 7fbe12a00700 1 -- 172.21.15.151:0/494227712 learned_addr learned my addr 172.21.15.151:0/494227712 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:51:21.926 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:21.928+0000 7fbe12000700 1 --2- 172.21.15.151:0/494227712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe14109c80 0x7fbe141a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:21.926 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:21.928+0000 7fbe13400700 1 -- 172.21.15.151:0/494227712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe14109c80 msgr2=0x7fbe141a5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:21.926 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:21.928+0000 7fbe13400700 1 --2- 172.21.15.151:0/494227712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe14109c80 0x7fbe141a5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:21.926 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:21.928+0000 7fbe13400700 1 -- 172.21.15.151:0/494227712 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbe141092d0 msgr2=0x7fbe141a53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:21.926 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:21.928+0000 7fbe13400700 1 --2- 172.21.15.151:0/494227712 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbe141092d0 0x7fbe141a53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:21.926 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:21.928+0000 7fbe13400700 1 -- 172.21.15.151:0/494227712 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe1419fd50 con 0x7fbe1410e3b0 2024-04-26T01:51:21.926 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:21.928+0000 7fbe12a00700 1 --2- 172.21.15.151:0/494227712 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbe141092d0 0x7fbe141a53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:51:21.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:21.928+0000 7fbe13400700 1 --2- 172.21.15.151:0/494227712 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe1410e3b0 0x7fbe1419f5f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c00efb0 tx=0x7fbe0c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:21.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:21.929+0000 7fbe0b400700 1 -- 172.21.15.151:0/494227712 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c003a00 con 0x7fbe1410e3b0 2024-04-26T01:51:21.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:21.929+0000 7fbe0b400700 1 -- 172.21.15.151:0/494227712 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe0c00ce50 con 0x7fbe1410e3b0 2024-04-26T01:51:21.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:21.929+0000 7fbe19600700 1 -- 172.21.15.151:0/494227712 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe141a0040 con 0x7fbe1410e3b0 2024-04-26T01:51:21.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:21.929+0000 7fbe0b400700 1 -- 172.21.15.151:0/494227712 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c0047c0 con 0x7fbe1410e3b0 2024-04-26T01:51:21.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:21.929+0000 7fbe19600700 1 -- 172.21.15.151:0/494227712 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe141ac2b0 con 0x7fbe1410e3b0 2024-04-26T01:51:21.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:21.930+0000 7fbe19600700 1 -- 172.21.15.151:0/494227712 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbe14063190 con 0x7fbe1410e3b0 2024-04-26T01:51:21.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:21.932+0000 7fbe0b400700 1 -- 172.21.15.151:0/494227712 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fbe0c012050 con 0x7fbe1410e3b0 2024-04-26T01:51:21.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:21.933+0000 7fbe0b400700 1 --2- 172.21.15.151:0/494227712 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fbdf80a3cd0 0x7fbdf80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:21.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:21.933+0000 7fbe0b400700 1 -- 172.21.15.151:0/494227712 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7fbe0c0d02e0 con 0x7fbe1410e3b0 2024-04-26T01:51:21.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:21.933+0000 7fbe12a00700 1 --2- 172.21.15.151:0/494227712 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fbdf80a3cd0 0x7fbdf80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:21.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:21.934+0000 7fbe0b400700 1 --2- 172.21.15.151:0/494227712 >> [v2:172.21.15.151:6810/3442279476,v1:172.21.15.151:6811/3442279476] conn(0x7fbdf80ad710 0x7fbdf80afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:21.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:21.934+0000 7fbe0b400700 1 -- 172.21.15.151:0/494227712 --> [v2:172.21.15.151:6810/3442279476,v1:172.21.15.151:6811/3442279476] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbe0c01ee50 con 0x7fbdf80ad710 2024-04-26T01:51:21.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:21.934+0000 7fbe0b400700 1 -- 172.21.15.151:0/494227712 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fbe0c0d0690 con 0x7fbe1410e3b0 2024-04-26T01:51:21.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:21.934+0000 7fbe12000700 1 --2- 172.21.15.151:0/494227712 >> [v2:172.21.15.151:6810/3442279476,v1:172.21.15.151:6811/3442279476] conn(0x7fbdf80ad710 0x7fbdf80afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:21.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:21.934+0000 7fbe12a00700 1 --2- 172.21.15.151:0/494227712 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fbdf80a3cd0 0x7fbdf80a6190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbe04009990 tx=0x7fbe04005ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:21.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:21.935+0000 7fbe12000700 1 --2- 172.21.15.151:0/494227712 >> [v2:172.21.15.151:6810/3442279476,v1:172.21.15.151:6811/3442279476] conn(0x7fbdf80ad710 0x7fbdf80afbb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:21.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:21.936+0000 7fbe0b400700 1 -- 172.21.15.151:0/494227712 <== osd.3 v2:172.21.15.151:6810/3442279476 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbe0c01ee50 con 0x7fbdf80ad710 2024-04-26T01:51:21.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:21.954+0000 7fbe19600700 1 -- 172.21.15.151:0/494227712 --> [v2:172.21.15.151:6810/3442279476,v1:172.21.15.151:6811/3442279476] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbe14063610 con 0x7fbdf80ad710 2024-04-26T01:51:21.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:21.954+0000 7fbe0b400700 1 -- 172.21.15.151:0/494227712 <== osd.3 v2:172.21.15.151:6810/3442279476 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fbe14063610 con 0x7fbdf80ad710 2024-04-26T01:51:21.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:21.954+0000 7fbe19600700 1 -- 172.21.15.151:0/494227712 >> [v2:172.21.15.151:6810/3442279476,v1:172.21.15.151:6811/3442279476] conn(0x7fbdf80ad710 msgr2=0x7fbdf80afbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:21.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:21.954+0000 7fbe19600700 1 --2- 172.21.15.151:0/494227712 >> [v2:172.21.15.151:6810/3442279476,v1:172.21.15.151:6811/3442279476] conn(0x7fbdf80ad710 0x7fbdf80afbb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:21.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:21.955+0000 7fbe19600700 1 -- 172.21.15.151:0/494227712 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fbdf80a3cd0 msgr2=0x7fbdf80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:21.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:21.955+0000 7fbe19600700 1 --2- 172.21.15.151:0/494227712 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fbdf80a3cd0 0x7fbdf80a6190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbe04009990 tx=0x7fbe04005ab0 comp rx=0 tx=0).stop 2024-04-26T01:51:21.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:21.955+0000 7fbe19600700 1 -- 172.21.15.151:0/494227712 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe1410e3b0 msgr2=0x7fbe1419f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:21.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:21.955+0000 7fbe19600700 1 --2- 172.21.15.151:0/494227712 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe1410e3b0 0x7fbe1419f5f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c00efb0 tx=0x7fbe0c00c3a0 comp rx=0 tx=0).stop 2024-04-26T01:51:21.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:21.955+0000 7fbe13400700 1 -- 172.21.15.151:0/494227712 reap_dead start 2024-04-26T01:51:21.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:21.955+0000 7fbe19600700 1 -- 172.21.15.151:0/494227712 shutdown_connections 2024-04-26T01:51:21.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:21.955+0000 7fbe19600700 1 -- 172.21.15.151:0/494227712 >> 172.21.15.151:0/494227712 conn(0x7fbe14076210 msgr2=0x7fbe14077e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:21.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:21.955+0000 7fbe19600700 1 -- 172.21.15.151:0/494227712 shutdown_connections 2024-04-26T01:51:21.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:21.955+0000 7fbe19600700 1 -- 172.21.15.151:0/494227712 wait complete. 2024-04-26T01:51:22.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:20.608+0000 7f1956000700 1 -- 172.21.15.151:0/800710621 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1950075790 msgr2=0x7f195010ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:22.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:20.608+0000 7f1956000700 1 --2- 172.21.15.151:0/800710621 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1950075790 0x7f195010ec70 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1938009990 tx=0x7f1938009ca0 comp rx=0 tx=0).stop 2024-04-26T01:51:22.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:20.608+0000 7f1956000700 1 -- 172.21.15.151:0/800710621 shutdown_connections 2024-04-26T01:51:22.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:20.608+0000 7f1956000700 1 --2- 172.21.15.151:0/800710621 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f195010f1b0 0x7f1950111620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:22.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:20.608+0000 7f1956000700 1 --2- 172.21.15.151:0/800710621 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1950074e70 0x7f1950075250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:22.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:20.608+0000 7f1956000700 1 --2- 172.21.15.151:0/800710621 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1950075790 0x7f195010ec70 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:22.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:20.608+0000 7f1956000700 1 -- 172.21.15.151:0/800710621 >> 172.21.15.151:0/800710621 conn(0x7f19500fe4c0 msgr2=0x7f19501008e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:22.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:20.608+0000 7f1956000700 1 -- 172.21.15.151:0/800710621 shutdown_connections 2024-04-26T01:51:22.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:20.608+0000 7f1956000700 1 -- 172.21.15.151:0/800710621 wait complete. 2024-04-26T01:51:22.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:20.609+0000 7f1956000700 1 Processor -- start 2024-04-26T01:51:22.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:20.610+0000 7f1956000700 1 -- start start 2024-04-26T01:51:22.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:20.610+0000 7f1956000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1950074e70 0x7f195019b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:22.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:20.610+0000 7f1956000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1950075790 0x7f195019b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:22.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:20.610+0000 7f1956000700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f195010f1b0 0x7f195019fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:22.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:20.610+0000 7f1956000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1950114d30 con 0x7f1950074e70 2024-04-26T01:51:22.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:20.610+0000 7f1956000700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1950114bb0 con 0x7f1950075790 2024-04-26T01:51:22.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:20.610+0000 7f1956000700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1950114eb0 con 0x7f195010f1b0 2024-04-26T01:51:22.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:20.610+0000 7f1947e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1950075790 0x7f195019b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:22.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:20.610+0000 7f1954c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1950074e70 0x7f195019b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:22.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:20.610+0000 7f1947e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1950075790 0x7f195019b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:48250/0 (socket says 172.21.15.151:48250) 2024-04-26T01:51:22.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:20.610+0000 7f1947e00700 1 -- 172.21.15.151:0/2645824856 learned_addr learned my addr 172.21.15.151:0/2645824856 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:51:22.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:20.610+0000 7f1955600700 1 --2- 172.21.15.151:0/2645824856 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f195010f1b0 0x7f195019fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:22.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:20.611+0000 7f1947e00700 1 -- 172.21.15.151:0/2645824856 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f195010f1b0 msgr2=0x7f195019fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:22.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:20.611+0000 7f1947e00700 1 --2- 172.21.15.151:0/2645824856 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f195010f1b0 0x7f195019fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:22.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:20.611+0000 7f1947e00700 1 -- 172.21.15.151:0/2645824856 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1950074e70 msgr2=0x7f195019b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:22.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:20.611+0000 7f1947e00700 1 --2- 172.21.15.151:0/2645824856 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1950074e70 0x7f195019b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:22.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:20.611+0000 7f1947e00700 1 -- 172.21.15.151:0/2645824856 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19501a01f0 con 0x7f1950075790 2024-04-26T01:51:22.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:20.611+0000 7f1947e00700 1 --2- 172.21.15.151:0/2645824856 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1950075790 0x7f195019b720 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f19380057f0 tx=0x7f193800bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:22.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:20.611+0000 7f194e000700 1 -- 172.21.15.151:0/2645824856 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1938026070 con 0x7f1950075790 2024-04-26T01:51:22.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:20.611+0000 7f194e000700 1 -- 172.21.15.151:0/2645824856 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1938021410 con 0x7f1950075790 2024-04-26T01:51:22.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:20.611+0000 7f194e000700 1 -- 172.21.15.151:0/2645824856 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1938017650 con 0x7f1950075790 2024-04-26T01:51:22.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:20.611+0000 7f1956000700 1 -- 172.21.15.151:0/2645824856 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19501a0480 con 0x7f1950075790 2024-04-26T01:51:22.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:20.612+0000 7f1956000700 1 -- 172.21.15.151:0/2645824856 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19501a7d60 con 0x7f1950075790 2024-04-26T01:51:22.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:20.612+0000 7f1956000700 1 -- 172.21.15.151:0/2645824856 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1950063150 con 0x7f1950075790 2024-04-26T01:51:22.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:20.615+0000 7f194e000700 1 -- 172.21.15.151:0/2645824856 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f1938029070 con 0x7f1950075790 2024-04-26T01:51:22.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:20.616+0000 7f194e000700 1 --2- 172.21.15.151:0/2645824856 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f19340a3d50 0x7f19340a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:22.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:20.616+0000 7f194e000700 1 -- 172.21.15.151:0/2645824856 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f19380d1600 con 0x7f1950075790 2024-04-26T01:51:22.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:20.616+0000 7f1954c00700 1 --2- 172.21.15.151:0/2645824856 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f19340a3d50 0x7f19340a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:22.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:20.617+0000 7f194e000700 1 --2- 172.21.15.151:0/2645824856 >> [v2:172.21.15.165:6816/1977803781,v1:172.21.15.165:6817/1977803781] conn(0x7f19340ad790 0x7f19340afc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:22.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:20.617+0000 7f194e000700 1 -- 172.21.15.151:0/2645824856 --> [v2:172.21.15.165:6816/1977803781,v1:172.21.15.165:6817/1977803781] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f19340b0300 con 0x7f19340ad790 2024-04-26T01:51:22.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:20.617+0000 7f194e000700 1 -- 172.21.15.151:0/2645824856 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f19380d19b0 con 0x7f1950075790 2024-04-26T01:51:22.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:20.617+0000 7f1955600700 1 --2- 172.21.15.151:0/2645824856 >> [v2:172.21.15.165:6816/1977803781,v1:172.21.15.165:6817/1977803781] conn(0x7f19340ad790 0x7f19340afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:22.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:20.617+0000 7f1954c00700 1 --2- 172.21.15.151:0/2645824856 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f19340a3d50 0x7f19340a6210 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1940009c60 tx=0x7f1940009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:22.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:20.618+0000 7f1955600700 1 --2- 172.21.15.151:0/2645824856 >> [v2:172.21.15.165:6816/1977803781,v1:172.21.15.165:6817/1977803781] conn(0x7f19340ad790 0x7f19340afc30 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:22.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:20.618+0000 7f194e000700 1 -- 172.21.15.151:0/2645824856 <== osd.7 v2:172.21.15.165:6816/1977803781 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f19340b0300 con 0x7f19340ad790 2024-04-26T01:51:22.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:20.637+0000 7f1956000700 1 -- 172.21.15.151:0/2645824856 --> [v2:172.21.15.165:6816/1977803781,v1:172.21.15.165:6817/1977803781] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1950063550 con 0x7f19340ad790 2024-04-26T01:51:22.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:20.637+0000 7f194e000700 1 -- 172.21.15.151:0/2645824856 <== osd.7 v2:172.21.15.165:6816/1977803781 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1950063550 con 0x7f19340ad790 2024-04-26T01:51:22.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:20.638+0000 7f1956000700 1 -- 172.21.15.151:0/2645824856 >> [v2:172.21.15.165:6816/1977803781,v1:172.21.15.165:6817/1977803781] conn(0x7f19340ad790 msgr2=0x7f19340afc30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:22.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:20.638+0000 7f1956000700 1 --2- 172.21.15.151:0/2645824856 >> [v2:172.21.15.165:6816/1977803781,v1:172.21.15.165:6817/1977803781] conn(0x7f19340ad790 0x7f19340afc30 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:22.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:20.638+0000 7f1956000700 1 -- 172.21.15.151:0/2645824856 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f19340a3d50 msgr2=0x7f19340a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:22.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:20.638+0000 7f1956000700 1 --2- 172.21.15.151:0/2645824856 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f19340a3d50 0x7f19340a6210 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1940009c60 tx=0x7f1940009260 comp rx=0 tx=0).stop 2024-04-26T01:51:22.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:20.638+0000 7f1956000700 1 -- 172.21.15.151:0/2645824856 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1950075790 msgr2=0x7f195019b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:22.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:20.638+0000 7f1956000700 1 --2- 172.21.15.151:0/2645824856 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1950075790 0x7f195019b720 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f19380057f0 tx=0x7f193800bfd0 comp rx=0 tx=0).stop 2024-04-26T01:51:22.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:20.638+0000 7f1955600700 1 -- 172.21.15.151:0/2645824856 reap_dead start 2024-04-26T01:51:22.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:20.638+0000 7f1956000700 1 -- 172.21.15.151:0/2645824856 shutdown_connections 2024-04-26T01:51:22.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:20.638+0000 7f1956000700 1 -- 172.21.15.151:0/2645824856 >> 172.21.15.151:0/2645824856 conn(0x7f19500fe4c0 msgr2=0x7f19500ff190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:22.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:20.638+0000 7f1956000700 1 -- 172.21.15.151:0/2645824856 shutdown_connections 2024-04-26T01:51:22.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:20.638+0000 7f1956000700 1 -- 172.21.15.151:0/2645824856 wait complete. 2024-04-26T01:51:22.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:22.450+0000 7f943e600700 1 -- 172.21.15.151:0/598703625 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f94381092d0 msgr2=0x7f94381096b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:22.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:22.450+0000 7f943e600700 1 --2- 172.21.15.151:0/598703625 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f94381092d0 0x7f94381096b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f942800b0a0 tx=0x7f942800b3b0 comp rx=0 tx=0).stop 2024-04-26T01:51:22.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:22.450+0000 7f943e600700 1 -- 172.21.15.151:0/598703625 shutdown_connections 2024-04-26T01:51:22.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:22.450+0000 7f943e600700 1 --2- 172.21.15.151:0/598703625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f943810e3b0 0x7f9438115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:22.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:22.450+0000 7f943e600700 1 --2- 172.21.15.151:0/598703625 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f94381092d0 0x7f94381096b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:22.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:22.450+0000 7f943e600700 1 --2- 172.21.15.151:0/598703625 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9438109c80 0x7f943810dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:22.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:22.450+0000 7f943e600700 1 -- 172.21.15.151:0/598703625 >> 172.21.15.151:0/598703625 conn(0x7f9438076210 msgr2=0x7f9438107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:22.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:22.450+0000 7f943e600700 1 -- 172.21.15.151:0/598703625 shutdown_connections 2024-04-26T01:51:22.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:22.450+0000 7f943e600700 1 -- 172.21.15.151:0/598703625 wait complete. 2024-04-26T01:51:22.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:22.452+0000 7f943e600700 1 Processor -- start 2024-04-26T01:51:22.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:22.452+0000 7f943e600700 1 -- start start 2024-04-26T01:51:22.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:22.453+0000 7f943e600700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94381092d0 0x7f94381a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:22.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:22.453+0000 7f943e600700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9438109c80 0x7f94381a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:22.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:22.453+0000 7f943e600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f943810e3b0 0x7f943819f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:22.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:22.453+0000 7f943e600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9438119290 con 0x7f943810e3b0 2024-04-26T01:51:22.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:22.453+0000 7f943e600700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9438119110 con 0x7f9438109c80 2024-04-26T01:51:22.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:22.453+0000 7f943e600700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9438119410 con 0x7f94381092d0 2024-04-26T01:51:22.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:22.453+0000 7f9437400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94381092d0 0x7f94381a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:22.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:22.453+0000 7f9437400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94381092d0 0x7f94381a53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:35158/0 (socket says 172.21.15.151:35158) 2024-04-26T01:51:22.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:22.453+0000 7f9437400700 1 -- 172.21.15.151:0/264597427 learned_addr learned my addr 172.21.15.151:0/264597427 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:51:22.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:22.453+0000 7f942fe00700 1 --2- 172.21.15.151:0/264597427 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9438109c80 0x7f94381a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:22.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:22.453+0000 7f9437e00700 1 --2- 172.21.15.151:0/264597427 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f943810e3b0 0x7f943819f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:22.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:22.453+0000 7f9437400700 1 -- 172.21.15.151:0/264597427 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9438109c80 msgr2=0x7f94381a5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:22.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:22.453+0000 7f9437400700 1 --2- 172.21.15.151:0/264597427 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9438109c80 0x7f94381a5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:22.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:22.453+0000 7f9437400700 1 -- 172.21.15.151:0/264597427 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f943810e3b0 msgr2=0x7f943819f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:22.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:22.453+0000 7f9437400700 1 --2- 172.21.15.151:0/264597427 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f943810e3b0 0x7f943819f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:22.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:22.453+0000 7f9437400700 1 -- 172.21.15.151:0/264597427 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f943819fcf0 con 0x7f94381092d0 2024-04-26T01:51:22.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:22.453+0000 7f9437e00700 1 --2- 172.21.15.151:0/264597427 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f943810e3b0 0x7f943819f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T01:51:22.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:22.454+0000 7f9437400700 1 --2- 172.21.15.151:0/264597427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94381092d0 0x7f94381a53d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f942800b070 tx=0x7f9428004d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:22.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:22.454+0000 7f9434c00700 1 -- 172.21.15.151:0/264597427 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9428021070 con 0x7f94381092d0 2024-04-26T01:51:22.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:22.454+0000 7f9434c00700 1 -- 172.21.15.151:0/264597427 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9428003c00 con 0x7f94381092d0 2024-04-26T01:51:22.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:22.454+0000 7f943e600700 1 -- 172.21.15.151:0/264597427 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f943819ff80 con 0x7f94381092d0 2024-04-26T01:51:22.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:22.454+0000 7f9434c00700 1 -- 172.21.15.151:0/264597427 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9428004790 con 0x7f94381092d0 2024-04-26T01:51:22.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:22.454+0000 7f943e600700 1 -- 172.21.15.151:0/264597427 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94381ac2a0 con 0x7f94381092d0 2024-04-26T01:51:22.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:22.455+0000 7f943e600700 1 -- 172.21.15.151:0/264597427 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f94380631d0 con 0x7f94381092d0 2024-04-26T01:51:22.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:22.458+0000 7f9434c00700 1 -- 172.21.15.151:0/264597427 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f9428003db0 con 0x7f94381092d0 2024-04-26T01:51:22.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:22.459+0000 7f9434c00700 1 --2- 172.21.15.151:0/264597427 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f941c0a3cd0 0x7f941c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:22.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:22.459+0000 7f9434c00700 1 -- 172.21.15.151:0/264597427 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f942800f020 con 0x7f94381092d0 2024-04-26T01:51:22.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:22.459+0000 7f942fe00700 1 --2- 172.21.15.151:0/264597427 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f941c0a3cd0 0x7f941c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:22.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:22.459+0000 7f9434c00700 1 --2- 172.21.15.151:0/264597427 >> [v2:172.21.15.151:6802/669937998,v1:172.21.15.151:6803/669937998] conn(0x7f941c0ad710 0x7f941c0afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:22.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:22.459+0000 7f9434c00700 1 -- 172.21.15.151:0/264597427 --> [v2:172.21.15.151:6802/669937998,v1:172.21.15.151:6803/669937998] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f94280040a0 con 0x7f941c0ad710 2024-04-26T01:51:22.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:22.459+0000 7f9434c00700 1 -- 172.21.15.151:0/264597427 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f94280d2020 con 0x7f94381092d0 2024-04-26T01:51:22.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:22.460+0000 7f9437e00700 1 --2- 172.21.15.151:0/264597427 >> [v2:172.21.15.151:6802/669937998,v1:172.21.15.151:6803/669937998] conn(0x7f941c0ad710 0x7f941c0afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:22.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:22.460+0000 7f942fe00700 1 --2- 172.21.15.151:0/264597427 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f941c0a3cd0 0x7f941c0a6190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9438077080 tx=0x7f9420009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:22.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:22.460+0000 7f9437e00700 1 --2- 172.21.15.151:0/264597427 >> [v2:172.21.15.151:6802/669937998,v1:172.21.15.151:6803/669937998] conn(0x7f941c0ad710 0x7f941c0afbb0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:22.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:22.461+0000 7f9434c00700 1 -- 172.21.15.151:0/264597427 <== osd.0 v2:172.21.15.151:6802/669937998 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f94280040a0 con 0x7f941c0ad710 2024-04-26T01:51:22.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:22.479+0000 7f943e600700 1 -- 172.21.15.151:0/264597427 --> [v2:172.21.15.151:6802/669937998,v1:172.21.15.151:6803/669937998] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9438063650 con 0x7f941c0ad710 2024-04-26T01:51:22.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:22.480+0000 7f9434c00700 1 -- 172.21.15.151:0/264597427 <== osd.0 v2:172.21.15.151:6802/669937998 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9438063650 con 0x7f941c0ad710 2024-04-26T01:51:22.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:22.480+0000 7f943e600700 1 -- 172.21.15.151:0/264597427 >> [v2:172.21.15.151:6802/669937998,v1:172.21.15.151:6803/669937998] conn(0x7f941c0ad710 msgr2=0x7f941c0afbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:22.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:22.480+0000 7f943e600700 1 --2- 172.21.15.151:0/264597427 >> [v2:172.21.15.151:6802/669937998,v1:172.21.15.151:6803/669937998] conn(0x7f941c0ad710 0x7f941c0afbb0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:22.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:22.480+0000 7f943e600700 1 -- 172.21.15.151:0/264597427 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f941c0a3cd0 msgr2=0x7f941c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:22.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:22.480+0000 7f943e600700 1 --2- 172.21.15.151:0/264597427 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f941c0a3cd0 0x7f941c0a6190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9438077080 tx=0x7f9420009260 comp rx=0 tx=0).stop 2024-04-26T01:51:22.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:22.480+0000 7f943e600700 1 -- 172.21.15.151:0/264597427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94381092d0 msgr2=0x7f94381a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:22.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:22.480+0000 7f943e600700 1 --2- 172.21.15.151:0/264597427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94381092d0 0x7f94381a53d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f942800b070 tx=0x7f9428004d30 comp rx=0 tx=0).stop 2024-04-26T01:51:22.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:22.480+0000 7f9437e00700 1 -- 172.21.15.151:0/264597427 reap_dead start 2024-04-26T01:51:22.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:22.481+0000 7f943e600700 1 -- 172.21.15.151:0/264597427 shutdown_connections 2024-04-26T01:51:22.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:22.481+0000 7f943e600700 1 -- 172.21.15.151:0/264597427 >> 172.21.15.151:0/264597427 conn(0x7f9438076210 msgr2=0x7f94380778b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:22.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:22.481+0000 7f943e600700 1 -- 172.21.15.151:0/264597427 shutdown_connections 2024-04-26T01:51:22.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:22.481+0000 7f943e600700 1 -- 172.21.15.151:0/264597427 wait complete. 2024-04-26T01:51:22.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:51:22 smithi151 ceph-mon[28057]: pgmap v380: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:51:23.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:51:22 smithi172 ceph-mon[32785]: pgmap v380: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:51:23.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:51:22 smithi165 ceph-mon[32648]: pgmap v380: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:51:23.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.170+0000 7f6ff1e00700 1 -- 172.21.15.151:0/2491207480 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6fec076f70 msgr2=0x7f6fec077350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:23.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.170+0000 7f6ff1e00700 1 --2- 172.21.15.151:0/2491207480 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6fec076f70 0x7f6fec077350 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc00b0a0 tx=0x7f6fdc00b3b0 comp rx=0 tx=0).stop 2024-04-26T01:51:23.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.171+0000 7f6ff1e00700 1 -- 172.21.15.151:0/2491207480 shutdown_connections 2024-04-26T01:51:23.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.171+0000 7f6ff1e00700 1 --2- 172.21.15.151:0/2491207480 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6fec1138d0 0x7f6fec115cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:23.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.171+0000 7f6ff1e00700 1 --2- 172.21.15.151:0/2491207480 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6fec076f70 0x7f6fec077350 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:23.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.171+0000 7f6ff1e00700 1 --2- 172.21.15.151:0/2491207480 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fec077890 0x7f6fec113390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:23.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.171+0000 7f6ff1e00700 1 -- 172.21.15.151:0/2491207480 >> 172.21.15.151:0/2491207480 conn(0x7f6fec100720 msgr2=0x7f6fec102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:23.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.171+0000 7f6ff1e00700 1 -- 172.21.15.151:0/2491207480 shutdown_connections 2024-04-26T01:51:23.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.171+0000 7f6ff1e00700 1 -- 172.21.15.151:0/2491207480 wait complete. 2024-04-26T01:51:23.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.172+0000 7f6ff1e00700 1 Processor -- start 2024-04-26T01:51:23.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.172+0000 7f6ff1e00700 1 -- start start 2024-04-26T01:51:23.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.172+0000 7f6ff1e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6fec076f70 0x7f6fec19f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:23.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.172+0000 7f6ff1e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fec077890 0x7f6fec19fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:23.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.173+0000 7f6ff1e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6fec1138d0 0x7f6fec1a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:23.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.173+0000 7f6ff1e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6fec1192b0 con 0x7f6fec076f70 2024-04-26T01:51:23.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.173+0000 7f6ff1e00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6fec119130 con 0x7f6fec077890 2024-04-26T01:51:23.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.173+0000 7f6ff1e00700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6fec119430 con 0x7f6fec1138d0 2024-04-26T01:51:23.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.173+0000 7f6feaa00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6fec076f70 0x7f6fec19f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:23.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.173+0000 7f6feaa00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6fec076f70 0x7f6fec19f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:40402/0 (socket says 172.21.15.151:40402) 2024-04-26T01:51:23.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.173+0000 7f6feaa00700 1 -- 172.21.15.151:0/528274453 learned_addr learned my addr 172.21.15.151:0/528274453 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:51:23.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.173+0000 7f6fe2000700 1 --2- 172.21.15.151:0/528274453 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fec077890 0x7f6fec19fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:23.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.173+0000 7f6feb400700 1 --2- 172.21.15.151:0/528274453 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6fec1138d0 0x7f6fec1a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:23.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.173+0000 7f6feaa00700 1 -- 172.21.15.151:0/528274453 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6fec1138d0 msgr2=0x7f6fec1a4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:23.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.173+0000 7f6feaa00700 1 --2- 172.21.15.151:0/528274453 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6fec1138d0 0x7f6fec1a4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:23.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.173+0000 7f6feaa00700 1 -- 172.21.15.151:0/528274453 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fec077890 msgr2=0x7f6fec19fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:23.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.173+0000 7f6feaa00700 1 --2- 172.21.15.151:0/528274453 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fec077890 0x7f6fec19fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:23.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.173+0000 7f6feaa00700 1 -- 172.21.15.151:0/528274453 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fec1a4750 con 0x7f6fec076f70 2024-04-26T01:51:23.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.174+0000 7f6feaa00700 1 --2- 172.21.15.151:0/528274453 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6fec076f70 0x7f6fec19f740 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc00d010 tx=0x7f6fdc004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:23.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.174+0000 7f6ff0a00700 1 -- 172.21.15.151:0/528274453 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc021070 con 0x7f6fec076f70 2024-04-26T01:51:23.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.174+0000 7f6ff0a00700 1 -- 172.21.15.151:0/528274453 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fdc007c40 con 0x7f6fec076f70 2024-04-26T01:51:23.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.174+0000 7f6ff1e00700 1 -- 172.21.15.151:0/528274453 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fec1a49e0 con 0x7f6fec076f70 2024-04-26T01:51:23.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.174+0000 7f6ff0a00700 1 -- 172.21.15.151:0/528274453 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc004760 con 0x7f6fec076f70 2024-04-26T01:51:23.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.174+0000 7f6ff1e00700 1 -- 172.21.15.151:0/528274453 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fec1ac2c0 con 0x7f6fec076f70 2024-04-26T01:51:23.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.175+0000 7f6fe1600700 1 -- 172.21.15.151:0/528274453 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6fc4001030 con 0x7f6fec076f70 2024-04-26T01:51:23.174 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.177+0000 7f6ff0a00700 1 -- 172.21.15.151:0/528274453 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f6fdc003a00 con 0x7f6fec076f70 2024-04-26T01:51:23.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.179+0000 7f6ff0a00700 1 --2- 172.21.15.151:0/528274453 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f6fd00a3cd0 0x7f6fd00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:23.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.179+0000 7f6ff0a00700 1 -- 172.21.15.151:0/528274453 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f6fdc00f020 con 0x7f6fec076f70 2024-04-26T01:51:23.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.179+0000 7f6fe2000700 1 --2- 172.21.15.151:0/528274453 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f6fd00a3cd0 0x7f6fd00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:23.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.179+0000 7f6ff0a00700 1 --2- 172.21.15.151:0/528274453 >> [v2:172.21.15.172:6824/886604365,v1:172.21.15.172:6825/886604365] conn(0x7f6fd00ad710 0x7f6fd00afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:23.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.179+0000 7f6ff0a00700 1 -- 172.21.15.151:0/528274453 --> [v2:172.21.15.172:6824/886604365,v1:172.21.15.172:6825/886604365] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6fdc00ce60 con 0x7f6fd00ad710 2024-04-26T01:51:23.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.179+0000 7f6ff0a00700 1 -- 172.21.15.151:0/528274453 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f6fdc0d2080 con 0x7f6fec076f70 2024-04-26T01:51:23.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.180+0000 7f6fe2000700 1 --2- 172.21.15.151:0/528274453 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f6fd00a3cd0 0x7f6fd00a6190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6fec103de0 tx=0x7f6fd40073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:23.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.180+0000 7f6feb400700 1 --2- 172.21.15.151:0/528274453 >> [v2:172.21.15.172:6824/886604365,v1:172.21.15.172:6825/886604365] conn(0x7f6fd00ad710 0x7f6fd00afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:23.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.180+0000 7f6feb400700 1 --2- 172.21.15.151:0/528274453 >> [v2:172.21.15.172:6824/886604365,v1:172.21.15.172:6825/886604365] conn(0x7f6fd00ad710 0x7f6fd00afbb0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:23.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.181+0000 7f6ff0a00700 1 -- 172.21.15.151:0/528274453 <== osd.11 v2:172.21.15.172:6824/886604365 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6fdc00ce60 con 0x7f6fd00ad710 2024-04-26T01:51:23.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.199+0000 7f6fe1600700 1 -- 172.21.15.151:0/528274453 --> [v2:172.21.15.172:6824/886604365,v1:172.21.15.172:6825/886604365] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6fc4002e60 con 0x7f6fd00ad710 2024-04-26T01:51:23.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.200+0000 7f6ff0a00700 1 -- 172.21.15.151:0/528274453 <== osd.11 v2:172.21.15.172:6824/886604365 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6fc4002e60 con 0x7f6fd00ad710 2024-04-26T01:51:23.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.200+0000 7f6fe1600700 1 -- 172.21.15.151:0/528274453 >> [v2:172.21.15.172:6824/886604365,v1:172.21.15.172:6825/886604365] conn(0x7f6fd00ad710 msgr2=0x7f6fd00afbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:23.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.200+0000 7f6fe1600700 1 --2- 172.21.15.151:0/528274453 >> [v2:172.21.15.172:6824/886604365,v1:172.21.15.172:6825/886604365] conn(0x7f6fd00ad710 0x7f6fd00afbb0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:23.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.200+0000 7f6fe1600700 1 -- 172.21.15.151:0/528274453 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f6fd00a3cd0 msgr2=0x7f6fd00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:23.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.200+0000 7f6fe1600700 1 --2- 172.21.15.151:0/528274453 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f6fd00a3cd0 0x7f6fd00a6190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6fec103de0 tx=0x7f6fd40073d0 comp rx=0 tx=0).stop 2024-04-26T01:51:23.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.200+0000 7f6fe1600700 1 -- 172.21.15.151:0/528274453 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6fec076f70 msgr2=0x7f6fec19f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:23.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.200+0000 7f6fe1600700 1 --2- 172.21.15.151:0/528274453 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6fec076f70 0x7f6fec19f740 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc00d010 tx=0x7f6fdc004d00 comp rx=0 tx=0).stop 2024-04-26T01:51:23.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.200+0000 7f6feb400700 1 -- 172.21.15.151:0/528274453 reap_dead start 2024-04-26T01:51:23.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.201+0000 7f6fe1600700 1 -- 172.21.15.151:0/528274453 shutdown_connections 2024-04-26T01:51:23.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.201+0000 7f6fe1600700 1 -- 172.21.15.151:0/528274453 >> 172.21.15.151:0/528274453 conn(0x7f6fec100720 msgr2=0x7f6fec102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:23.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.201+0000 7f6fe1600700 1 -- 172.21.15.151:0/528274453 shutdown_connections 2024-04-26T01:51:23.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.201+0000 7f6fe1600700 1 -- 172.21.15.151:0/528274453 wait complete. 2024-04-26T01:51:23.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.935+0000 7fb5ebe00700 1 -- 172.21.15.151:0/1638534707 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5ec109940 msgr2=0x7fb5ec115de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:23.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.935+0000 7fb5ebe00700 1 --2- 172.21.15.151:0/1638534707 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5ec109940 0x7fb5ec115de0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4009f90 tx=0x7fb5e400b330 comp rx=0 tx=0).stop 2024-04-26T01:51:23.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.935+0000 7fb5ebe00700 1 -- 172.21.15.151:0/1638534707 shutdown_connections 2024-04-26T01:51:23.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.935+0000 7fb5ebe00700 1 --2- 172.21.15.151:0/1638534707 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5ec109940 0x7fb5ec115de0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:23.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.935+0000 7fb5ebe00700 1 --2- 172.21.15.151:0/1638534707 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5ec10f030 0x7fb5ec10f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:23.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.935+0000 7fb5ebe00700 1 --2- 172.21.15.151:0/1638534707 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5ec108f80 0x7fb5ec109400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:23.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.935+0000 7fb5ebe00700 1 -- 172.21.15.151:0/1638534707 >> 172.21.15.151:0/1638534707 conn(0x7fb5ec0787d0 msgr2=0x7fb5ec107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:23.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.936+0000 7fb5ebe00700 1 -- 172.21.15.151:0/1638534707 shutdown_connections 2024-04-26T01:51:23.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.936+0000 7fb5ebe00700 1 -- 172.21.15.151:0/1638534707 wait complete. 2024-04-26T01:51:23.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.937+0000 7fb5ebe00700 1 Processor -- start 2024-04-26T01:51:23.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.937+0000 7fb5ebe00700 1 -- start start 2024-04-26T01:51:23.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.938+0000 7fb5ebe00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5ec108f80 0x7fb5ec19f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:23.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.938+0000 7fb5ebe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5ec109940 0x7fb5ec19fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:23.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.938+0000 7fb5ebe00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5ec10f030 0x7fb5ec1a3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:23.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.938+0000 7fb5ebe00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb5ec119160 con 0x7fb5ec109940 2024-04-26T01:51:23.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.938+0000 7fb5ebe00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb5ec118fe0 con 0x7fb5ec108f80 2024-04-26T01:51:23.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.938+0000 7fb5ebe00700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb5ec1192e0 con 0x7fb5ec10f030 2024-04-26T01:51:23.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.938+0000 7fb5ea000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5ec109940 0x7fb5ec19fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:23.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.938+0000 7fb5ea000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5ec109940 0x7fb5ec19fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:40424/0 (socket says 172.21.15.151:40424) 2024-04-26T01:51:23.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.938+0000 7fb5ea000700 1 -- 172.21.15.151:0/3649151803 learned_addr learned my addr 172.21.15.151:0/3649151803 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:51:23.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.938+0000 7fb5eb400700 1 --2- 172.21.15.151:0/3649151803 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5ec10f030 0x7fb5ec1a3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:23.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.938+0000 7fb5eaa00700 1 --2- 172.21.15.151:0/3649151803 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5ec108f80 0x7fb5ec19f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:23.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.939+0000 7fb5ea000700 1 -- 172.21.15.151:0/3649151803 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5ec10f030 msgr2=0x7fb5ec1a3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:23.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.939+0000 7fb5ea000700 1 --2- 172.21.15.151:0/3649151803 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5ec10f030 0x7fb5ec1a3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:23.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.939+0000 7fb5ea000700 1 -- 172.21.15.151:0/3649151803 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5ec108f80 msgr2=0x7fb5ec19f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:23.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.939+0000 7fb5ea000700 1 --2- 172.21.15.151:0/3649151803 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5ec108f80 0x7fb5ec19f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:23.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.939+0000 7fb5ea000700 1 -- 172.21.15.151:0/3649151803 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5ec1a4560 con 0x7fb5ec109940 2024-04-26T01:51:23.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.939+0000 7fb5eb400700 1 --2- 172.21.15.151:0/3649151803 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5ec10f030 0x7fb5ec1a3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:51:23.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.939+0000 7fb5ea000700 1 --2- 172.21.15.151:0/3649151803 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5ec109940 0x7fb5ec19fa90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb5d800e9c0 tx=0x7fb5d800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:23.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.939+0000 7fb5e3400700 1 -- 172.21.15.151:0/3649151803 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d8004a30 con 0x7fb5ec109940 2024-04-26T01:51:23.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.939+0000 7fb5ebe00700 1 -- 172.21.15.151:0/3649151803 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5ec1a4850 con 0x7fb5ec109940 2024-04-26T01:51:23.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.939+0000 7fb5e3400700 1 -- 172.21.15.151:0/3649151803 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5d800cc70 con 0x7fb5ec109940 2024-04-26T01:51:23.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.939+0000 7fb5e3400700 1 -- 172.21.15.151:0/3649151803 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d80054a0 con 0x7fb5ec109940 2024-04-26T01:51:23.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.939+0000 7fb5ebe00700 1 -- 172.21.15.151:0/3649151803 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5ec1ac0d0 con 0x7fb5ec109940 2024-04-26T01:51:23.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.940+0000 7fb5ebe00700 1 -- 172.21.15.151:0/3649151803 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb5ec0631d0 con 0x7fb5ec109940 2024-04-26T01:51:23.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.942+0000 7fb5e3400700 1 -- 172.21.15.151:0/3649151803 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fb5d8004bd0 con 0x7fb5ec109940 2024-04-26T01:51:23.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.944+0000 7fb5e3400700 1 --2- 172.21.15.151:0/3649151803 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fb5d00a3d50 0x7fb5d00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:23.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.944+0000 7fb5e3400700 1 -- 172.21.15.151:0/3649151803 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7fb5d80d0a00 con 0x7fb5ec109940 2024-04-26T01:51:23.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.944+0000 7fb5eaa00700 1 --2- 172.21.15.151:0/3649151803 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fb5d00a3d50 0x7fb5d00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:23.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.944+0000 7fb5e3400700 1 --2- 172.21.15.151:0/3649151803 >> [v2:172.21.15.165:6800/1356168171,v1:172.21.15.165:6801/1356168171] conn(0x7fb5d00ad790 0x7fb5d00afc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:23.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.944+0000 7fb5e3400700 1 -- 172.21.15.151:0/3649151803 --> [v2:172.21.15.165:6800/1356168171,v1:172.21.15.165:6801/1356168171] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb5d00b0300 con 0x7fb5d00ad790 2024-04-26T01:51:23.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.944+0000 7fb5e3400700 1 -- 172.21.15.151:0/3649151803 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fb5d80d0db0 con 0x7fb5ec109940 2024-04-26T01:51:23.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.944+0000 7fb5eb400700 1 --2- 172.21.15.151:0/3649151803 >> [v2:172.21.15.165:6800/1356168171,v1:172.21.15.165:6801/1356168171] conn(0x7fb5d00ad790 0x7fb5d00afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:23.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.945+0000 7fb5eaa00700 1 --2- 172.21.15.151:0/3649151803 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fb5d00a3d50 0x7fb5d00a6210 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc009c60 tx=0x7fb5dc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:23.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.945+0000 7fb5eb400700 1 --2- 172.21.15.151:0/3649151803 >> [v2:172.21.15.165:6800/1356168171,v1:172.21.15.165:6801/1356168171] conn(0x7fb5d00ad790 0x7fb5d00afc30 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:23.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.946+0000 7fb5e3400700 1 -- 172.21.15.151:0/3649151803 <== osd.1 v2:172.21.15.165:6800/1356168171 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb5d00b0300 con 0x7fb5d00ad790 2024-04-26T01:51:23.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.966+0000 7fb5ebe00700 1 -- 172.21.15.151:0/3649151803 --> [v2:172.21.15.165:6800/1356168171,v1:172.21.15.165:6801/1356168171] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb5ec0635d0 con 0x7fb5d00ad790 2024-04-26T01:51:23.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.966+0000 7fb5e3400700 1 -- 172.21.15.151:0/3649151803 <== osd.1 v2:172.21.15.165:6800/1356168171 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb5ec0635d0 con 0x7fb5d00ad790 2024-04-26T01:51:23.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.967+0000 7fb5ebe00700 1 -- 172.21.15.151:0/3649151803 >> [v2:172.21.15.165:6800/1356168171,v1:172.21.15.165:6801/1356168171] conn(0x7fb5d00ad790 msgr2=0x7fb5d00afc30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:23.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.967+0000 7fb5ebe00700 1 --2- 172.21.15.151:0/3649151803 >> [v2:172.21.15.165:6800/1356168171,v1:172.21.15.165:6801/1356168171] conn(0x7fb5d00ad790 0x7fb5d00afc30 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:23.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.967+0000 7fb5ebe00700 1 -- 172.21.15.151:0/3649151803 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fb5d00a3d50 msgr2=0x7fb5d00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:23.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.967+0000 7fb5ebe00700 1 --2- 172.21.15.151:0/3649151803 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fb5d00a3d50 0x7fb5d00a6210 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc009c60 tx=0x7fb5dc009260 comp rx=0 tx=0).stop 2024-04-26T01:51:23.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.967+0000 7fb5ebe00700 1 -- 172.21.15.151:0/3649151803 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5ec109940 msgr2=0x7fb5ec19fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:23.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.967+0000 7fb5ebe00700 1 --2- 172.21.15.151:0/3649151803 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5ec109940 0x7fb5ec19fa90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb5d800e9c0 tx=0x7fb5d800ecd0 comp rx=0 tx=0).stop 2024-04-26T01:51:23.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.967+0000 7fb5eb400700 1 -- 172.21.15.151:0/3649151803 reap_dead start 2024-04-26T01:51:23.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.967+0000 7fb5ebe00700 1 -- 172.21.15.151:0/3649151803 shutdown_connections 2024-04-26T01:51:23.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.967+0000 7fb5ebe00700 1 -- 172.21.15.151:0/3649151803 >> 172.21.15.151:0/3649151803 conn(0x7fb5ec0787d0 msgr2=0x7fb5ec079a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:23.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.968+0000 7fb5ebe00700 1 -- 172.21.15.151:0/3649151803 shutdown_connections 2024-04-26T01:51:23.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:23.968+0000 7fb5ebe00700 1 -- 172.21.15.151:0/3649151803 wait complete. 2024-04-26T01:51:25.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:51:24 smithi165 ceph-mon[32648]: pgmap v381: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:51:25.120 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:51:24 smithi151 ceph-mon[28057]: pgmap v381: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:51:25.163 INFO:teuthology.orchestra.run.smithi151.stdout:115964117074 2024-04-26T01:51:25.163 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-26T01:51:25.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:25.159+0000 7f764cc00700 1 -- 172.21.15.151:0/2791052052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7648107980 msgr2=0x7f7648107d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:25.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:25.159+0000 7f764cc00700 1 --2- 172.21.15.151:0/2791052052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7648107980 0x7f7648107d60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f76380098f0 tx=0x7f7638009c00 comp rx=0 tx=0).stop 2024-04-26T01:51:25.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:25.160+0000 7f764cc00700 1 -- 172.21.15.151:0/2791052052 shutdown_connections 2024-04-26T01:51:25.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:25.160+0000 7f764cc00700 1 --2- 172.21.15.151:0/2791052052 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76481022f0 0x7f764810e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:25.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:25.160+0000 7f764cc00700 1 --2- 172.21.15.151:0/2791052052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7648107980 0x7f7648107d60 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:25.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:25.160+0000 7f764cc00700 1 --2- 172.21.15.151:0/2791052052 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7648101930 0x7f7648101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:25.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:25.160+0000 7f764cc00700 1 -- 172.21.15.151:0/2791052052 >> 172.21.15.151:0/2791052052 conn(0x7f76480788b0 msgr2=0x7f7648078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:25.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:25.160+0000 7f764cc00700 1 -- 172.21.15.151:0/2791052052 shutdown_connections 2024-04-26T01:51:25.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:25.160+0000 7f764cc00700 1 -- 172.21.15.151:0/2791052052 wait complete. 2024-04-26T01:51:25.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:25.161+0000 7f764cc00700 1 Processor -- start 2024-04-26T01:51:25.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:25.161+0000 7f764cc00700 1 -- start start 2024-04-26T01:51:25.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:25.161+0000 7f764cc00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7648101930 0x7f76481034e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:25.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:25.161+0000 7f764cc00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76481022f0 0x7f7648103a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:25.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:25.161+0000 7f764cc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7648107980 0x7f7648103f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:25.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:25.161+0000 7f764cc00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7648111b40 con 0x7f7648107980 2024-04-26T01:51:25.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:25.161+0000 7f764cc00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f76481119c0 con 0x7f76481022f0 2024-04-26T01:51:25.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:25.161+0000 7f764cc00700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7648111cc0 con 0x7f7648101930 2024-04-26T01:51:25.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:25.162+0000 7f7646800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7648107980 0x7f7648103f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:25.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:25.162+0000 7f7646800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7648107980 0x7f7648103f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:40440/0 (socket says 172.21.15.151:40440) 2024-04-26T01:51:25.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:25.162+0000 7f7646800700 1 -- 172.21.15.151:0/1461338585 learned_addr learned my addr 172.21.15.151:0/1461338585 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:51:25.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:25.162+0000 7f7645e00700 1 --2- 172.21.15.151:0/1461338585 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7648101930 0x7f76481034e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:25.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:25.162+0000 7f7645400700 1 --2- 172.21.15.151:0/1461338585 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76481022f0 0x7f7648103a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:25.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:25.162+0000 7f7646800700 1 -- 172.21.15.151:0/1461338585 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7648101930 msgr2=0x7f76481034e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:25.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:25.162+0000 7f7646800700 1 --2- 172.21.15.151:0/1461338585 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7648101930 0x7f76481034e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:25.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:25.162+0000 7f7646800700 1 -- 172.21.15.151:0/1461338585 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76481022f0 msgr2=0x7f7648103a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:25.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:25.162+0000 7f7646800700 1 --2- 172.21.15.151:0/1461338585 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76481022f0 0x7f7648103a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:25.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:25.162+0000 7f7646800700 1 -- 172.21.15.151:0/1461338585 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76481a7740 con 0x7f7648107980 2024-04-26T01:51:25.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:25.162+0000 7f7645e00700 1 --2- 172.21.15.151:0/1461338585 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7648101930 0x7f76481034e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:51:25.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:25.163+0000 7f7646800700 1 --2- 172.21.15.151:0/1461338585 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7648107980 0x7f7648103f60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f764000c110 tx=0x7f764000c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:25.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:25.163+0000 7f763ea00700 1 -- 172.21.15.151:0/1461338585 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76400092e0 con 0x7f7648107980 2024-04-26T01:51:25.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:25.163+0000 7f763ea00700 1 -- 172.21.15.151:0/1461338585 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7640004bb0 con 0x7f7648107980 2024-04-26T01:51:25.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:25.163+0000 7f763ea00700 1 -- 172.21.15.151:0/1461338585 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7640007be0 con 0x7f7648107980 2024-04-26T01:51:25.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:25.163+0000 7f764cc00700 1 -- 172.21.15.151:0/1461338585 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76481a79d0 con 0x7f7648107980 2024-04-26T01:51:25.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:25.163+0000 7f764cc00700 1 -- 172.21.15.151:0/1461338585 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76481a7f10 con 0x7f7648107980 2024-04-26T01:51:25.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:25.164+0000 7f764cc00700 1 -- 172.21.15.151:0/1461338585 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f76481068c0 con 0x7f7648107980 2024-04-26T01:51:25.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:25.168+0000 7f763ea00700 1 -- 172.21.15.151:0/1461338585 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f764001a020 con 0x7f7648107980 2024-04-26T01:51:25.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:25.170+0000 7f763ea00700 1 --2- 172.21.15.151:0/1461338585 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f762c0a3d50 0x7f762c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:25.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:25.170+0000 7f763ea00700 1 -- 172.21.15.151:0/1461338585 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f76400d0990 con 0x7f7648107980 2024-04-26T01:51:25.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:25.170+0000 7f7645e00700 1 --2- 172.21.15.151:0/1461338585 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f762c0a3d50 0x7f762c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:25.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:25.170+0000 7f763ea00700 1 --2- 172.21.15.151:0/1461338585 >> [v2:172.21.15.172:6800/237502635,v1:172.21.15.172:6801/237502635] conn(0x7f762c0ad790 0x7f762c0afc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:25.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:25.170+0000 7f763ea00700 1 -- 172.21.15.151:0/1461338585 --> [v2:172.21.15.172:6800/237502635,v1:172.21.15.172:6801/237502635] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f762c0b0300 con 0x7f762c0ad790 2024-04-26T01:51:25.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:25.170+0000 7f763ea00700 1 -- 172.21.15.151:0/1461338585 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f76400d0d40 con 0x7f7648107980 2024-04-26T01:51:25.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:25.171+0000 7f7645400700 1 --2- 172.21.15.151:0/1461338585 >> [v2:172.21.15.172:6800/237502635,v1:172.21.15.172:6801/237502635] conn(0x7f762c0ad790 0x7f762c0afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:25.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:25.171+0000 7f7645e00700 1 --2- 172.21.15.151:0/1461338585 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f762c0a3d50 0x7f762c0a6210 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f76380098c0 tx=0x7f7638019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:25.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:25.171+0000 7f7645400700 1 --2- 172.21.15.151:0/1461338585 >> [v2:172.21.15.172:6800/237502635,v1:172.21.15.172:6801/237502635] conn(0x7f762c0ad790 0x7f762c0afc30 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:25.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:25.172+0000 7f763ea00700 1 -- 172.21.15.151:0/1461338585 <== osd.2 v2:172.21.15.172:6800/237502635 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f762c0b0300 con 0x7f762c0ad790 2024-04-26T01:51:25.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:25.191+0000 7f764cc00700 1 -- 172.21.15.151:0/1461338585 --> [v2:172.21.15.172:6800/237502635,v1:172.21.15.172:6801/237502635] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7648103050 con 0x7f762c0ad790 2024-04-26T01:51:25.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:25.192+0000 7f763ea00700 1 -- 172.21.15.151:0/1461338585 <== osd.2 v2:172.21.15.172:6800/237502635 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7648103050 con 0x7f762c0ad790 2024-04-26T01:51:25.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:25.192+0000 7f764cc00700 1 -- 172.21.15.151:0/1461338585 >> [v2:172.21.15.172:6800/237502635,v1:172.21.15.172:6801/237502635] conn(0x7f762c0ad790 msgr2=0x7f762c0afc30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:25.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:25.192+0000 7f764cc00700 1 --2- 172.21.15.151:0/1461338585 >> [v2:172.21.15.172:6800/237502635,v1:172.21.15.172:6801/237502635] conn(0x7f762c0ad790 0x7f762c0afc30 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:25.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:25.193+0000 7f764cc00700 1 -- 172.21.15.151:0/1461338585 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f762c0a3d50 msgr2=0x7f762c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:25.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:25.193+0000 7f764cc00700 1 --2- 172.21.15.151:0/1461338585 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f762c0a3d50 0x7f762c0a6210 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f76380098c0 tx=0x7f7638019040 comp rx=0 tx=0).stop 2024-04-26T01:51:25.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:25.193+0000 7f764cc00700 1 -- 172.21.15.151:0/1461338585 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7648107980 msgr2=0x7f7648103f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:25.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:25.193+0000 7f764cc00700 1 --2- 172.21.15.151:0/1461338585 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7648107980 0x7f7648103f60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f764000c110 tx=0x7f764000c420 comp rx=0 tx=0).stop 2024-04-26T01:51:25.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:25.193+0000 7f7646800700 1 -- 172.21.15.151:0/1461338585 reap_dead start 2024-04-26T01:51:25.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:25.193+0000 7f764cc00700 1 -- 172.21.15.151:0/1461338585 shutdown_connections 2024-04-26T01:51:25.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:25.193+0000 7f764cc00700 1 -- 172.21.15.151:0/1461338585 >> 172.21.15.151:0/1461338585 conn(0x7f76480788b0 msgr2=0x7f764806b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:25.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:25.193+0000 7f764cc00700 1 -- 172.21.15.151:0/1461338585 shutdown_connections 2024-04-26T01:51:25.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:25.193+0000 7f764cc00700 1 -- 172.21.15.151:0/1461338585 wait complete. 2024-04-26T01:51:25.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:51:24 smithi172 ceph-mon[32785]: pgmap v381: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:51:26.789 INFO:teuthology.orchestra.run.smithi151.stdout:55834574943 2024-04-26T01:51:26.790 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-26T01:51:27.005 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:51:26 smithi151 ceph-mon[28057]: pgmap v382: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T01:51:27.005 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:51:26 smithi151 ceph-mon[28057]: mgrmap e18: x(active, since 10m), standbys: y, z 2024-04-26T01:51:27.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:51:26 smithi165 ceph-mon[32648]: pgmap v382: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T01:51:27.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:51:26 smithi165 ceph-mon[32648]: mgrmap e18: x(active, since 10m), standbys: y, z 2024-04-26T01:51:27.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:51:26 smithi172 ceph-mon[32785]: pgmap v382: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T01:51:27.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:51:26 smithi172 ceph-mon[32785]: mgrmap e18: x(active, since 10m), standbys: y, z 2024-04-26T01:51:27.587 INFO:teuthology.orchestra.run.smithi151.stdout:227633266748 2024-04-26T01:51:27.587 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-26T01:51:27.781 INFO:teuthology.orchestra.run.smithi151.stdout:352187318313 2024-04-26T01:51:27.781 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-26T01:51:28.319 INFO:teuthology.orchestra.run.smithi151.stdout:34359738467 2024-04-26T01:51:28.320 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-26T01:51:28.985 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:51:28 smithi151 ceph-mon[28057]: pgmap v383: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 7 op/s 2024-04-26T01:51:29.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:29.036+0000 7fc7e9200700 1 -- 172.21.15.151:0/1981362356 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7e406b980 msgr2=0x7fc7e410f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:29.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:29.037+0000 7fc7e9200700 1 --2- 172.21.15.151:0/1981362356 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7e406b980 0x7fc7e410f8f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc00b0a0 tx=0x7fc7dc00b3b0 comp rx=0 tx=0).stop 2024-04-26T01:51:29.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:29.037+0000 7fc7e9200700 1 -- 172.21.15.151:0/1981362356 shutdown_connections 2024-04-26T01:51:29.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:29.037+0000 7fc7e9200700 1 --2- 172.21.15.151:0/1981362356 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7e406b980 0x7fc7e410f8f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:29.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:29.037+0000 7fc7e9200700 1 --2- 172.21.15.151:0/1981362356 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7e4108a20 0x7fc7e4108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:29.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:29.037+0000 7fc7e9200700 1 --2- 172.21.15.151:0/1981362356 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc7e406afe0 0x7fc7e406b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:29.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:29.037+0000 7fc7e9200700 1 -- 172.21.15.151:0/1981362356 >> 172.21.15.151:0/1981362356 conn(0x7fc7e40788b0 msgr2=0x7fc7e4078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:29.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:29.037+0000 7fc7e9200700 1 -- 172.21.15.151:0/1981362356 shutdown_connections 2024-04-26T01:51:29.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:29.037+0000 7fc7e9200700 1 -- 172.21.15.151:0/1981362356 wait complete. 2024-04-26T01:51:29.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:29.038+0000 7fc7e9200700 1 Processor -- start 2024-04-26T01:51:29.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:29.038+0000 7fc7e9200700 1 -- start start 2024-04-26T01:51:29.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:29.038+0000 7fc7e9200700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc7e406afe0 0x7fc7e4101a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:29.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:29.038+0000 7fc7e9200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7e406b980 0x7fc7e4101f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:29.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:29.038+0000 7fc7e9200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7e4108a20 0x7fc7e4102480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:29.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:29.038+0000 7fc7e9200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e4112ba0 con 0x7fc7e406b980 2024-04-26T01:51:29.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:29.038+0000 7fc7e9200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e4112a20 con 0x7fc7e4108a20 2024-04-26T01:51:29.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:29.038+0000 7fc7e9200700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e4112d20 con 0x7fc7e406afe0 2024-04-26T01:51:29.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:29.039+0000 7fc7e1400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7e406b980 0x7fc7e4101f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:29.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:29.039+0000 7fc7e1400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7e406b980 0x7fc7e4101f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:40468/0 (socket says 172.21.15.151:40468) 2024-04-26T01:51:29.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:29.039+0000 7fc7e1400700 1 -- 172.21.15.151:0/2165243397 learned_addr learned my addr 172.21.15.151:0/2165243397 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:51:29.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:29.039+0000 7fc7e1e00700 1 --2- 172.21.15.151:0/2165243397 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc7e406afe0 0x7fc7e4101a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:29.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:29.039+0000 7fc7e2800700 1 --2- 172.21.15.151:0/2165243397 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7e4108a20 0x7fc7e4102480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:29.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:29.039+0000 7fc7e1400700 1 -- 172.21.15.151:0/2165243397 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc7e406afe0 msgr2=0x7fc7e4101a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:29.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:29.039+0000 7fc7e1400700 1 --2- 172.21.15.151:0/2165243397 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc7e406afe0 0x7fc7e4101a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:29.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:29.039+0000 7fc7e1400700 1 -- 172.21.15.151:0/2165243397 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7e4108a20 msgr2=0x7fc7e4102480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:29.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:29.039+0000 7fc7e1400700 1 --2- 172.21.15.151:0/2165243397 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7e4108a20 0x7fc7e4102480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:29.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:29.039+0000 7fc7e1400700 1 -- 172.21.15.151:0/2165243397 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7e41a7610 con 0x7fc7e406b980 2024-04-26T01:51:29.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:29.040+0000 7fc7e1400700 1 --2- 172.21.15.151:0/2165243397 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7e406b980 0x7fc7e4101f40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc7d000d950 tx=0x7fc7d000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:29.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:29.040+0000 7fc7daa00700 1 -- 172.21.15.151:0/2165243397 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d0016070 con 0x7fc7e406b980 2024-04-26T01:51:29.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:29.040+0000 7fc7daa00700 1 -- 172.21.15.151:0/2165243397 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7d0009df0 con 0x7fc7e406b980 2024-04-26T01:51:29.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:29.040+0000 7fc7daa00700 1 -- 172.21.15.151:0/2165243397 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d000f3c0 con 0x7fc7e406b980 2024-04-26T01:51:29.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:29.040+0000 7fc7e9200700 1 -- 172.21.15.151:0/2165243397 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7e41a7900 con 0x7fc7e406b980 2024-04-26T01:51:29.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:29.040+0000 7fc7e9200700 1 -- 172.21.15.151:0/2165243397 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7e41a7df0 con 0x7fc7e406b980 2024-04-26T01:51:29.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:29.041+0000 7fc7e9200700 1 -- 172.21.15.151:0/2165243397 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc7e40630d0 con 0x7fc7e406b980 2024-04-26T01:51:29.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:29.043+0000 7fc7daa00700 1 -- 172.21.15.151:0/2165243397 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7fc7d0019070 con 0x7fc7e406b980 2024-04-26T01:51:29.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:29.045+0000 7fc7daa00700 1 --2- 172.21.15.151:0/2165243397 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fc7c80a3cc0 0x7fc7c80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:29.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:29.045+0000 7fc7daa00700 1 -- 172.21.15.151:0/2165243397 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7fc7d00cff50 con 0x7fc7e406b980 2024-04-26T01:51:29.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:29.045+0000 7fc7e1e00700 1 --2- 172.21.15.151:0/2165243397 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fc7c80a3cc0 0x7fc7c80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:29.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:29.045+0000 7fc7daa00700 1 --2- 172.21.15.151:0/2165243397 >> [v2:172.21.15.165:6824/2632621809,v1:172.21.15.165:6825/2632621809] conn(0x7fc7c80ad700 0x7fc7c80afba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:29.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:29.045+0000 7fc7daa00700 1 -- 172.21.15.151:0/2165243397 --> [v2:172.21.15.165:6824/2632621809,v1:172.21.15.165:6825/2632621809] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc7c80b0270 con 0x7fc7c80ad700 2024-04-26T01:51:29.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:29.045+0000 7fc7daa00700 1 -- 172.21.15.151:0/2165243397 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fc7d00d2570 con 0x7fc7e406b980 2024-04-26T01:51:29.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:29.045+0000 7fc7e2800700 1 --2- 172.21.15.151:0/2165243397 >> [v2:172.21.15.165:6824/2632621809,v1:172.21.15.165:6825/2632621809] conn(0x7fc7c80ad700 0x7fc7c80afba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:29.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:29.046+0000 7fc7e1e00700 1 --2- 172.21.15.151:0/2165243397 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fc7c80a3cc0 0x7fc7c80a6180 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4009680 tx=0x7fc7d4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:29.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:29.046+0000 7fc7e2800700 1 --2- 172.21.15.151:0/2165243397 >> [v2:172.21.15.165:6824/2632621809,v1:172.21.15.165:6825/2632621809] conn(0x7fc7c80ad700 0x7fc7c80afba0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:29.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:29.047+0000 7fc7daa00700 1 -- 172.21.15.151:0/2165243397 <== osd.10 v2:172.21.15.165:6824/2632621809 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc7c80b0270 con 0x7fc7c80ad700 2024-04-26T01:51:29.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:29.065+0000 7fc7e9200700 1 -- 172.21.15.151:0/2165243397 --> [v2:172.21.15.165:6824/2632621809,v1:172.21.15.165:6825/2632621809] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc7e40635a0 con 0x7fc7c80ad700 2024-04-26T01:51:29.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:29.066+0000 7fc7daa00700 1 -- 172.21.15.151:0/2165243397 <== osd.10 v2:172.21.15.165:6824/2632621809 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc7e40635a0 con 0x7fc7c80ad700 2024-04-26T01:51:29.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:29.066+0000 7fc7e9200700 1 -- 172.21.15.151:0/2165243397 >> [v2:172.21.15.165:6824/2632621809,v1:172.21.15.165:6825/2632621809] conn(0x7fc7c80ad700 msgr2=0x7fc7c80afba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:29.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:29.066+0000 7fc7e9200700 1 --2- 172.21.15.151:0/2165243397 >> [v2:172.21.15.165:6824/2632621809,v1:172.21.15.165:6825/2632621809] conn(0x7fc7c80ad700 0x7fc7c80afba0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:29.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:29.066+0000 7fc7e9200700 1 -- 172.21.15.151:0/2165243397 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fc7c80a3cc0 msgr2=0x7fc7c80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:29.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:29.066+0000 7fc7e9200700 1 --2- 172.21.15.151:0/2165243397 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fc7c80a3cc0 0x7fc7c80a6180 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4009680 tx=0x7fc7d4009260 comp rx=0 tx=0).stop 2024-04-26T01:51:29.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:29.067+0000 7fc7e9200700 1 -- 172.21.15.151:0/2165243397 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7e406b980 msgr2=0x7fc7e4101f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:29.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:29.067+0000 7fc7e9200700 1 --2- 172.21.15.151:0/2165243397 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7e406b980 0x7fc7e4101f40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc7d000d950 tx=0x7fc7d000dc60 comp rx=0 tx=0).stop 2024-04-26T01:51:29.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:29.067+0000 7fc7e2800700 1 -- 172.21.15.151:0/2165243397 reap_dead start 2024-04-26T01:51:29.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:29.067+0000 7fc7e9200700 1 -- 172.21.15.151:0/2165243397 shutdown_connections 2024-04-26T01:51:29.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:29.067+0000 7fc7e9200700 1 -- 172.21.15.151:0/2165243397 >> 172.21.15.151:0/2165243397 conn(0x7fc7e40788b0 msgr2=0x7fc7e40ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:29.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:29.067+0000 7fc7e9200700 1 -- 172.21.15.151:0/2165243397 shutdown_connections 2024-04-26T01:51:29.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:29.067+0000 7fc7e9200700 1 -- 172.21.15.151:0/2165243397 wait complete. 2024-04-26T01:51:29.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:51:28 smithi165 ceph-mon[32648]: pgmap v383: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 7 op/s 2024-04-26T01:51:29.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:51:28 smithi172 ceph-mon[32785]: pgmap v383: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 7 op/s 2024-04-26T01:51:29.416 INFO:teuthology.orchestra.run.smithi151.stdout:77309411418 2024-04-26T01:51:29.417 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-26T01:51:29.747 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:51:30.085 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:51:30.087 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:51:30.094 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:51:30.161 INFO:teuthology.orchestra.run.smithi151.stdout:317827579948 2024-04-26T01:51:30.162 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-26T01:51:30.219 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:51:31.004 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:51:30 smithi151 ceph-mon[28057]: pgmap v384: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-26T01:51:31.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:51:30 smithi165 ceph-mon[32648]: pgmap v384: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-26T01:51:31.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:51:30 smithi172 ceph-mon[32785]: pgmap v384: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-26T01:51:31.419 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:51:31.425 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:51:33.070 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:51:32 smithi151 ceph-mon[28057]: pgmap v385: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-26T01:51:33.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:51:32 smithi165 ceph-mon[32648]: pgmap v385: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-26T01:51:33.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:51:32 smithi172 ceph-mon[32785]: pgmap v385: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-26T01:51:35.064 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:51:34 smithi151 ceph-mon[28057]: pgmap v386: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-26T01:51:35.064 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:51:34 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:51:35.088 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:51:34 smithi172 ceph-mon[32785]: pgmap v386: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-26T01:51:35.088 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:51:34 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:51:35.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:51:34 smithi165 ceph-mon[32648]: pgmap v386: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-26T01:51:35.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:51:34 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:51:37.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:51:36 smithi165 ceph-mon[32648]: pgmap v387: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-26T01:51:37.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:51:36 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:51:37.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:51:36 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:51:37.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:51:36 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:51:37.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:51:36 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:51:37.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:51:36 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:51:37.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:51:36 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:51:37.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:51:36 smithi151 ceph-mon[28057]: pgmap v387: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-26T01:51:37.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:51:36 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:51:37.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:51:36 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:51:37.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:51:36 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:51:37.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:51:36 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:51:37.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:51:36 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:51:37.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:51:36 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:51:37.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:51:36 smithi172 ceph-mon[32785]: pgmap v387: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-26T01:51:37.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:51:36 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:51:37.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:51:36 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:51:37.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:51:36 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:51:37.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:51:36 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:51:37.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:51:36 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:51:37.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:51:36 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:51:37.814 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:51:37.815 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:51:37.815 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:51:39.057 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:51:38 smithi151 ceph-mon[28057]: pgmap v388: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-26T01:51:39.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:51:38 smithi165 ceph-mon[32648]: pgmap v388: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-26T01:51:39.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:51:38 smithi172 ceph-mon[32785]: pgmap v388: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-26T01:51:39.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:51:39 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/858205801' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-26T01:51:40.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:51:39 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/858205801' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-26T01:51:40.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.973+0000 7fa9fd600700 1 -- 172.21.15.151:0/3446354232 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa9f8109c80 msgr2=0x7fa9f810dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:40.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.973+0000 7fa9fd600700 1 --2- 172.21.15.151:0/3446354232 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa9f8109c80 0x7fa9f810dcd0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0009990 tx=0x7fa9e0009ca0 comp rx=0 tx=0).stop 2024-04-26T01:51:40.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.973+0000 7fa9fd600700 1 -- 172.21.15.151:0/3446354232 shutdown_connections 2024-04-26T01:51:40.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.973+0000 7fa9fd600700 1 --2- 172.21.15.151:0/3446354232 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9f810e3b0 0x7fa9f8115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:40.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.973+0000 7fa9fd600700 1 --2- 172.21.15.151:0/3446354232 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9f81092d0 0x7fa9f81096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:40.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.973+0000 7fa9fd600700 1 --2- 172.21.15.151:0/3446354232 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa9f8109c80 0x7fa9f810dcd0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:40.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.973+0000 7fa9fd600700 1 -- 172.21.15.151:0/3446354232 >> 172.21.15.151:0/3446354232 conn(0x7fa9f8076210 msgr2=0x7fa9f8107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:40.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.973+0000 7fa9fd600700 1 -- 172.21.15.151:0/3446354232 shutdown_connections 2024-04-26T01:51:40.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.973+0000 7fa9fd600700 1 -- 172.21.15.151:0/3446354232 wait complete. 2024-04-26T01:51:40.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.975+0000 7fa9fd600700 1 Processor -- start 2024-04-26T01:51:40.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.975+0000 7fa9fd600700 1 -- start start 2024-04-26T01:51:40.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.975+0000 7fa9fd600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9f81092d0 0x7fa9f81a5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:40.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.975+0000 7fa9fd600700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9f8109c80 0x7fa9f81a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:40.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.975+0000 7fa9fd600700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa9f810e3b0 0x7fa9f819f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:40.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.975+0000 7fa9fd600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f8119270 con 0x7fa9f81092d0 2024-04-26T01:51:40.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.975+0000 7fa9fd600700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f81190f0 con 0x7fa9f810e3b0 2024-04-26T01:51:40.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.975+0000 7fa9fd600700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f81193f0 con 0x7fa9f8109c80 2024-04-26T01:51:40.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.975+0000 7fa9f6a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9f81092d0 0x7fa9f81a5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:40.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.976+0000 7fa9efe00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9f8109c80 0x7fa9f81a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:40.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.976+0000 7fa9f6a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9f81092d0 0x7fa9f81a5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:57886/0 (socket says 172.21.15.151:57886) 2024-04-26T01:51:40.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.976+0000 7fa9f6a00700 1 -- 172.21.15.151:0/1244121108 learned_addr learned my addr 172.21.15.151:0/1244121108 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:51:40.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.976+0000 7fa9f7400700 1 --2- 172.21.15.151:0/1244121108 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa9f810e3b0 0x7fa9f819f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:40.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.976+0000 7fa9f6a00700 1 -- 172.21.15.151:0/1244121108 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9f8109c80 msgr2=0x7fa9f81a5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:40.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.976+0000 7fa9f6a00700 1 --2- 172.21.15.151:0/1244121108 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9f8109c80 0x7fa9f81a5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:40.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.976+0000 7fa9f6a00700 1 -- 172.21.15.151:0/1244121108 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa9f810e3b0 msgr2=0x7fa9f819f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:40.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.976+0000 7fa9f6a00700 1 --2- 172.21.15.151:0/1244121108 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa9f810e3b0 0x7fa9f819f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:40.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.976+0000 7fa9f6a00700 1 -- 172.21.15.151:0/1244121108 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9f819fd50 con 0x7fa9f81092d0 2024-04-26T01:51:40.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.976+0000 7fa9f7400700 1 --2- 172.21.15.151:0/1244121108 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa9f810e3b0 0x7fa9f819f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:51:40.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.976+0000 7fa9f6a00700 1 --2- 172.21.15.151:0/1244121108 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9f81092d0 0x7fa9f81a5430 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa9e800e9a0 tx=0x7fa9e800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:40.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.977+0000 7fa9ef400700 1 -- 172.21.15.151:0/1244121108 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e800ca10 con 0x7fa9f81092d0 2024-04-26T01:51:40.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.977+0000 7fa9ef400700 1 -- 172.21.15.151:0/1244121108 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9e800cbb0 con 0x7fa9f81092d0 2024-04-26T01:51:40.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.977+0000 7fa9fd600700 1 -- 172.21.15.151:0/1244121108 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9f81a0040 con 0x7fa9f81092d0 2024-04-26T01:51:40.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.977+0000 7fa9ef400700 1 -- 172.21.15.151:0/1244121108 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e8018680 con 0x7fa9f81092d0 2024-04-26T01:51:40.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.977+0000 7fa9fd600700 1 -- 172.21.15.151:0/1244121108 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9f81ac300 con 0x7fa9f81092d0 2024-04-26T01:51:40.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.978+0000 7fa9fd600700 1 -- 172.21.15.151:0/1244121108 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9f8069730 con 0x7fa9f81092d0 2024-04-26T01:51:40.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.980+0000 7fa9ef400700 1 -- 172.21.15.151:0/1244121108 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7fa9e8010430 con 0x7fa9f81092d0 2024-04-26T01:51:40.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.981+0000 7fa9ef400700 1 --2- 172.21.15.151:0/1244121108 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fa9dc0a3c40 0x7fa9dc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:40.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.982+0000 7fa9ef400700 1 -- 172.21.15.151:0/1244121108 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7fa9e80d0560 con 0x7fa9f81092d0 2024-04-26T01:51:40.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.982+0000 7fa9efe00700 1 --2- 172.21.15.151:0/1244121108 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fa9dc0a3c40 0x7fa9dc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:40.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.983+0000 7fa9efe00700 1 --2- 172.21.15.151:0/1244121108 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fa9dc0a3c40 0x7fa9dc0a6100 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8077110 tx=0x7fa9e000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:40.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.987+0000 7fa9ef400700 1 -- 172.21.15.151:0/1244121108 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa9e809cf40 con 0x7fa9f81092d0 2024-04-26T01:51:40.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:40.225+0000 7fa9fd600700 1 -- 172.21.15.151:0/1244121108 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fa9f802df80 con 0x7fa9f81092d0 2024-04-26T01:51:40.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:40.226+0000 7fa9ef400700 1 -- 172.21.15.151:0/1244121108 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa9e809c840 con 0x7fa9f81092d0 2024-04-26T01:51:40.225 INFO:teuthology.orchestra.run.smithi151.stdout:150323855443 2024-04-26T01:51:40.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:40.231+0000 7fa9fd600700 1 -- 172.21.15.151:0/1244121108 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fa9dc0a3c40 msgr2=0x7fa9dc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:40.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:40.231+0000 7fa9fd600700 1 --2- 172.21.15.151:0/1244121108 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fa9dc0a3c40 0x7fa9dc0a6100 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8077110 tx=0x7fa9e000b260 comp rx=0 tx=0).stop 2024-04-26T01:51:40.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:40.231+0000 7fa9fd600700 1 -- 172.21.15.151:0/1244121108 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9f81092d0 msgr2=0x7fa9f81a5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:40.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:40.231+0000 7fa9fd600700 1 --2- 172.21.15.151:0/1244121108 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9f81092d0 0x7fa9f81a5430 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa9e800e9a0 tx=0x7fa9e800ecb0 comp rx=0 tx=0).stop 2024-04-26T01:51:40.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:40.231+0000 7fa9fd600700 1 -- 172.21.15.151:0/1244121108 shutdown_connections 2024-04-26T01:51:40.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:40.231+0000 7fa9fd600700 1 --2- 172.21.15.151:0/1244121108 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa9f810e3b0 0x7fa9f819f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:40.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:40.231+0000 7fa9fd600700 1 --2- 172.21.15.151:0/1244121108 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fa9dc0a3c40 0x7fa9dc0a6100 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:40.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:40.231+0000 7fa9fd600700 1 --2- 172.21.15.151:0/1244121108 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9f81092d0 0x7fa9f81a5430 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:40.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:40.231+0000 7fa9fd600700 1 --2- 172.21.15.151:0/1244121108 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9f8109c80 0x7fa9f81a5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:40.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:40.231+0000 7fa9fd600700 1 -- 172.21.15.151:0/1244121108 >> 172.21.15.151:0/1244121108 conn(0x7fa9f8076210 msgr2=0x7fa9f8077dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:40.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:40.231+0000 7fa9fd600700 1 -- 172.21.15.151:0/1244121108 shutdown_connections 2024-04-26T01:51:40.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:40.231+0000 7fa9fd600700 1 -- 172.21.15.151:0/1244121108 wait complete. 2024-04-26T01:51:40.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:51:39 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/858205801' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-26T01:51:40.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.361+0000 7f56fc000700 1 -- 172.21.15.151:0/1237851990 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56f4074c70 msgr2=0x7f56f4075050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:40.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.361+0000 7f56fc000700 1 --2- 172.21.15.151:0/1237851990 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56f4074c70 0x7f56f4075050 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f56e8009990 tx=0x7f56e8009ca0 comp rx=0 tx=0).stop 2024-04-26T01:51:40.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.361+0000 7f56fc000700 1 -- 172.21.15.151:0/1237851990 shutdown_connections 2024-04-26T01:51:40.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.361+0000 7f56fc000700 1 --2- 172.21.15.151:0/1237851990 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f56f410f440 0x7f56f4111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:40.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.361+0000 7f56fc000700 1 --2- 172.21.15.151:0/1237851990 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56f4074c70 0x7f56f4075050 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:40.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.361+0000 7f56fc000700 1 --2- 172.21.15.151:0/1237851990 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56f4075590 0x7f56f410ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:40.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.361+0000 7f56fc000700 1 -- 172.21.15.151:0/1237851990 >> 172.21.15.151:0/1237851990 conn(0x7f56f4079ba0 msgr2=0x7f56f4079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:40.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.361+0000 7f56fc000700 1 -- 172.21.15.151:0/1237851990 shutdown_connections 2024-04-26T01:51:40.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.361+0000 7f56fc000700 1 -- 172.21.15.151:0/1237851990 wait complete. 2024-04-26T01:51:40.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.363+0000 7f56fc000700 1 Processor -- start 2024-04-26T01:51:40.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.363+0000 7f56fc000700 1 -- start start 2024-04-26T01:51:40.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.363+0000 7f56fc000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56f4074c70 0x7f56f419b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:40.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.363+0000 7f56fc000700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56f4075590 0x7f56f419b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:40.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.363+0000 7f56fc000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f56f410f440 0x7f56f419fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:40.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.363+0000 7f56fc000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f56f4114e20 con 0x7f56f4074c70 2024-04-26T01:51:40.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.363+0000 7f56fc000700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f56f4114ca0 con 0x7f56f410f440 2024-04-26T01:51:40.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.363+0000 7f56fc000700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f56f4114fa0 con 0x7f56f4075590 2024-04-26T01:51:40.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.363+0000 7f56f9600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56f4074c70 0x7f56f419b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:40.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.364+0000 7f56f9600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56f4074c70 0x7f56f419b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:34876/0 (socket says 172.21.15.151:34876) 2024-04-26T01:51:40.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.364+0000 7f56f9600700 1 -- 172.21.15.151:0/858205801 learned_addr learned my addr 172.21.15.151:0/858205801 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:51:40.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.364+0000 7f56f8c00700 1 --2- 172.21.15.151:0/858205801 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56f4075590 0x7f56f419b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:40.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.364+0000 7f56fa000700 1 --2- 172.21.15.151:0/858205801 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f56f410f440 0x7f56f419fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:40.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.364+0000 7f56f8c00700 1 -- 172.21.15.151:0/858205801 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f56f410f440 msgr2=0x7f56f419fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:40.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.364+0000 7f56f8c00700 1 --2- 172.21.15.151:0/858205801 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f56f410f440 0x7f56f419fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:40.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.364+0000 7f56f8c00700 1 -- 172.21.15.151:0/858205801 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56f4074c70 msgr2=0x7f56f419b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:40.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.364+0000 7f56f8c00700 1 --2- 172.21.15.151:0/858205801 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56f4074c70 0x7f56f419b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:40.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.364+0000 7f56f8c00700 1 -- 172.21.15.151:0/858205801 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56f41a0340 con 0x7f56f4075590 2024-04-26T01:51:40.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.364+0000 7f56f8c00700 1 --2- 172.21.15.151:0/858205801 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56f4075590 0x7f56f419b870 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f56e000e9c0 tx=0x7f56e000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:40.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.365+0000 7f56f2000700 1 -- 172.21.15.151:0/858205801 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e0004230 con 0x7f56f4075590 2024-04-26T01:51:40.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.365+0000 7f56f2000700 1 -- 172.21.15.151:0/858205801 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56e00043d0 con 0x7f56f4075590 2024-04-26T01:51:40.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.365+0000 7f56fc000700 1 -- 172.21.15.151:0/858205801 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56f41a0630 con 0x7f56f4075590 2024-04-26T01:51:40.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.365+0000 7f56f2000700 1 -- 172.21.15.151:0/858205801 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e0003d90 con 0x7f56f4075590 2024-04-26T01:51:40.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.365+0000 7f56fc000700 1 -- 172.21.15.151:0/858205801 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56f41a7eb0 con 0x7f56f4075590 2024-04-26T01:51:40.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.366+0000 7f56fc000700 1 -- 172.21.15.151:0/858205801 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56f4069730 con 0x7f56f4075590 2024-04-26T01:51:40.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.368+0000 7f56f2000700 1 -- 172.21.15.151:0/858205801 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f56e000c7c0 con 0x7f56f4075590 2024-04-26T01:51:40.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.369+0000 7f56f2000700 1 --2- 172.21.15.151:0/858205801 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f56dc0a3cc0 0x7f56dc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:40.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.369+0000 7f56f2000700 1 -- 172.21.15.151:0/858205801 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f56e00d0540 con 0x7f56f4075590 2024-04-26T01:51:40.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.370+0000 7f56f9600700 1 --2- 172.21.15.151:0/858205801 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f56dc0a3cc0 0x7f56dc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:40.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.370+0000 7f56f9600700 1 --2- 172.21.15.151:0/858205801 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f56dc0a3cc0 0x7f56dc0a6180 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f56e8005cd0 tx=0x7f56e8005c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:40.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.372+0000 7f56f2000700 1 -- 172.21.15.151:0/858205801 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f56e009cfc0 con 0x7f56f4075590 2024-04-26T01:51:40.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.609+0000 7f56fc000700 1 -- 172.21.15.151:0/858205801 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f56f402df80 con 0x7f56f4075590 2024-04-26T01:51:40.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.609+0000 7f56f2000700 1 -- 172.21.15.151:0/858205801 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f56e009c8c0 con 0x7f56f4075590 2024-04-26T01:51:40.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.612+0000 7f56fc000700 1 -- 172.21.15.151:0/858205801 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f56dc0a3cc0 msgr2=0x7f56dc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:40.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.612+0000 7f56fc000700 1 --2- 172.21.15.151:0/858205801 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f56dc0a3cc0 0x7f56dc0a6180 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f56e8005cd0 tx=0x7f56e8005c60 comp rx=0 tx=0).stop 2024-04-26T01:51:40.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.613+0000 7f56fc000700 1 -- 172.21.15.151:0/858205801 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56f4075590 msgr2=0x7f56f419b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:40.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.613+0000 7f56fc000700 1 --2- 172.21.15.151:0/858205801 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56f4075590 0x7f56f419b870 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f56e000e9c0 tx=0x7f56e000ecd0 comp rx=0 tx=0).stop 2024-04-26T01:51:40.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.613+0000 7f56fc000700 1 -- 172.21.15.151:0/858205801 shutdown_connections 2024-04-26T01:51:40.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.613+0000 7f56fc000700 1 --2- 172.21.15.151:0/858205801 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f56f410f440 0x7f56f419fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:40.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.613+0000 7f56fc000700 1 --2- 172.21.15.151:0/858205801 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f56dc0a3cc0 0x7f56dc0a6180 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:40.328 INFO:teuthology.orchestra.run.smithi151.stdout:55834574946 2024-04-26T01:51:40.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.613+0000 7f56fc000700 1 --2- 172.21.15.151:0/858205801 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56f4074c70 0x7f56f419b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:40.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.613+0000 7f56fc000700 1 --2- 172.21.15.151:0/858205801 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56f4075590 0x7f56f419b870 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:40.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.613+0000 7f56fc000700 1 -- 172.21.15.151:0/858205801 >> 172.21.15.151:0/858205801 conn(0x7f56f4079ba0 msgr2=0x7f56f40ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:40.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.613+0000 7f56fc000700 1 -- 172.21.15.151:0/858205801 shutdown_connections 2024-04-26T01:51:40.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:39.613+0000 7f56fc000700 1 -- 172.21.15.151:0/858205801 wait complete. 2024-04-26T01:51:40.920 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:51:40 smithi151 ceph-mon[28057]: pgmap v389: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T01:51:40.920 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:51:40 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/1244121108' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-26T01:51:41.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:51:40 smithi165 ceph-mon[32648]: pgmap v389: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T01:51:41.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:51:40 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/1244121108' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-26T01:51:41.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:51:40 smithi172 ceph-mon[32785]: pgmap v389: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T01:51:41.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:51:40 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/1244121108' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-26T01:51:41.475 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574943 got 55834574946 for osd.1 2024-04-26T01:51:41.475 DEBUG:teuthology.parallel:result is None 2024-04-26T01:51:41.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:41.853+0000 7f9608e00700 1 -- 172.21.15.151:0/3161659991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9604109e80 msgr2=0x7f9604111a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:41.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:41.853+0000 7f9608e00700 1 --2- 172.21.15.151:0/3161659991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9604109e80 0x7f9604111a50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f95fc009fc0 tx=0x7f95fc00b330 comp rx=0 tx=0).stop 2024-04-26T01:51:41.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:41.853+0000 7f9608e00700 1 -- 172.21.15.151:0/3161659991 shutdown_connections 2024-04-26T01:51:41.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:41.853+0000 7f9608e00700 1 --2- 172.21.15.151:0/3161659991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9604109e80 0x7f9604111a50 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:41.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:41.853+0000 7f9608e00700 1 --2- 172.21.15.151:0/3161659991 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9604104da0 0x7f9604105180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:41.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:41.853+0000 7f9608e00700 1 --2- 172.21.15.151:0/3161659991 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9604105750 0x7f96041097a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:41.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:41.853+0000 7f9608e00700 1 -- 172.21.15.151:0/3161659991 >> 172.21.15.151:0/3161659991 conn(0x7f9604100670 msgr2=0x7f9604102a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:41.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:41.853+0000 7f9608e00700 1 -- 172.21.15.151:0/3161659991 shutdown_connections 2024-04-26T01:51:41.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:41.853+0000 7f9608e00700 1 -- 172.21.15.151:0/3161659991 wait complete. 2024-04-26T01:51:41.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:41.854+0000 7f9608e00700 1 Processor -- start 2024-04-26T01:51:41.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:41.854+0000 7f9608e00700 1 -- start start 2024-04-26T01:51:41.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:41.855+0000 7f9608e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9604104da0 0x7f960419b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:41.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:41.855+0000 7f9608e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9604105750 0x7f960419b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:41.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:41.855+0000 7f9608e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9604109e80 0x7f960419fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:41.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:41.855+0000 7f9608e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9604114d80 con 0x7f9604104da0 2024-04-26T01:51:41.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:41.855+0000 7f9608e00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9604114c00 con 0x7f9604105750 2024-04-26T01:51:41.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:41.855+0000 7f9608e00700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9604114f00 con 0x7f9604109e80 2024-04-26T01:51:41.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:41.855+0000 7f9602a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9604105750 0x7f960419b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:41.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:41.855+0000 7f9602a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9604105750 0x7f960419b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:45772/0 (socket says 172.21.15.151:45772) 2024-04-26T01:51:41.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:41.855+0000 7f9602a00700 1 -- 172.21.15.151:0/1892928669 learned_addr learned my addr 172.21.15.151:0/1892928669 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:51:41.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:41.855+0000 7f9603e00700 1 --2- 172.21.15.151:0/1892928669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9604109e80 0x7f960419fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:41.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:41.855+0000 7f9603400700 1 --2- 172.21.15.151:0/1892928669 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9604104da0 0x7f960419b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:41.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:41.856+0000 7f9602a00700 1 -- 172.21.15.151:0/1892928669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9604109e80 msgr2=0x7f960419fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:41.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:41.856+0000 7f9602a00700 1 --2- 172.21.15.151:0/1892928669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9604109e80 0x7f960419fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:41.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:41.856+0000 7f9602a00700 1 -- 172.21.15.151:0/1892928669 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9604104da0 msgr2=0x7f960419b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:41.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:41.856+0000 7f9602a00700 1 --2- 172.21.15.151:0/1892928669 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9604104da0 0x7f960419b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:41.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:41.856+0000 7f9602a00700 1 -- 172.21.15.151:0/1892928669 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96041a0280 con 0x7f9604105750 2024-04-26T01:51:41.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:41.856+0000 7f9603e00700 1 --2- 172.21.15.151:0/1892928669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9604109e80 0x7f960419fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:51:41.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:41.856+0000 7f9602a00700 1 --2- 172.21.15.151:0/1892928669 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9604105750 0x7f960419b7b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f95ec00e9f0 tx=0x7f95ec00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:41.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:41.856+0000 7f95fbe00700 1 -- 172.21.15.151:0/1892928669 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ec00c9c0 con 0x7f9604105750 2024-04-26T01:51:41.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:41.856+0000 7f9608e00700 1 -- 172.21.15.151:0/1892928669 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96041a0570 con 0x7f9604105750 2024-04-26T01:51:41.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:41.856+0000 7f95fbe00700 1 -- 172.21.15.151:0/1892928669 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95ec00cb60 con 0x7f9604105750 2024-04-26T01:51:41.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:41.856+0000 7f95fbe00700 1 -- 172.21.15.151:0/1892928669 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ec018680 con 0x7f9604105750 2024-04-26T01:51:41.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:41.856+0000 7f9608e00700 1 -- 172.21.15.151:0/1892928669 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96041a7df0 con 0x7f9604105750 2024-04-26T01:51:41.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:41.857+0000 7f9608e00700 1 -- 172.21.15.151:0/1892928669 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9604069730 con 0x7f9604105750 2024-04-26T01:51:41.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:41.859+0000 7f95fbe00700 1 -- 172.21.15.151:0/1892928669 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f95ec010430 con 0x7f9604105750 2024-04-26T01:51:41.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:41.861+0000 7f95fbe00700 1 --2- 172.21.15.151:0/1892928669 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f95e80a3c40 0x7f95e80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:41.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:41.861+0000 7f95fbe00700 1 -- 172.21.15.151:0/1892928669 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f95ec0d0560 con 0x7f9604105750 2024-04-26T01:51:41.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:41.861+0000 7f9603400700 1 --2- 172.21.15.151:0/1892928669 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f95e80a3c40 0x7f95e80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:41.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:41.862+0000 7f9603400700 1 --2- 172.21.15.151:0/1892928669 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f95e80a3c40 0x7f95e80a6100 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f95f4009c60 tx=0x7f95f4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:41.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:41.865+0000 7f95fbe00700 1 -- 172.21.15.151:0/1892928669 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f95ec09cf40 con 0x7f9604105750 2024-04-26T01:51:42.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:42.094+0000 7f9608e00700 1 -- 172.21.15.151:0/1892928669 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f960402df80 con 0x7f9604105750 2024-04-26T01:51:42.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:42.095+0000 7f95fbe00700 1 -- 172.21.15.151:0/1892928669 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f95ec09c840 con 0x7f9604105750 2024-04-26T01:51:42.093 INFO:teuthology.orchestra.run.smithi151.stdout:261993005116 2024-04-26T01:51:42.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:42.098+0000 7f9608e00700 1 -- 172.21.15.151:0/1892928669 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f95e80a3c40 msgr2=0x7f95e80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:42.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:42.098+0000 7f9608e00700 1 --2- 172.21.15.151:0/1892928669 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f95e80a3c40 0x7f95e80a6100 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f95f4009c60 tx=0x7f95f4009260 comp rx=0 tx=0).stop 2024-04-26T01:51:42.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:42.098+0000 7f9608e00700 1 -- 172.21.15.151:0/1892928669 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9604105750 msgr2=0x7f960419b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:42.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:42.098+0000 7f9608e00700 1 --2- 172.21.15.151:0/1892928669 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9604105750 0x7f960419b7b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f95ec00e9f0 tx=0x7f95ec00ed00 comp rx=0 tx=0).stop 2024-04-26T01:51:42.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:42.098+0000 7f9608e00700 1 -- 172.21.15.151:0/1892928669 shutdown_connections 2024-04-26T01:51:42.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:42.098+0000 7f9608e00700 1 --2- 172.21.15.151:0/1892928669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9604109e80 0x7f960419fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:42.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:42.098+0000 7f9608e00700 1 --2- 172.21.15.151:0/1892928669 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f95e80a3c40 0x7f95e80a6100 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:42.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:42.098+0000 7f9608e00700 1 --2- 172.21.15.151:0/1892928669 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9604104da0 0x7f960419b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:42.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:42.098+0000 7f9608e00700 1 --2- 172.21.15.151:0/1892928669 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9604105750 0x7f960419b7b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:42.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:42.098+0000 7f9608e00700 1 -- 172.21.15.151:0/1892928669 >> 172.21.15.151:0/1892928669 conn(0x7f9604100670 msgr2=0x7f9604101240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:42.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:42.098+0000 7f9608e00700 1 -- 172.21.15.151:0/1892928669 shutdown_connections 2024-04-26T01:51:42.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:42.098+0000 7f9608e00700 1 -- 172.21.15.151:0/1892928669 wait complete. 2024-04-26T01:51:42.689 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:51:42.946 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:51:42 smithi151 ceph-mon[28057]: pgmap v390: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-04-26T01:51:42.946 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:51:42 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/1892928669' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-26T01:51:43.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:42.998+0000 7f8c69e00700 1 -- 172.21.15.151:0/3324538286 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c64102a90 msgr2=0x7f8c64102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:43.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:42.998+0000 7f8c69e00700 1 --2- 172.21.15.151:0/3324538286 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c64102a90 0x7f8c64102f10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8c50009990 tx=0x7f8c50009ca0 comp rx=0 tx=0).stop 2024-04-26T01:51:43.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:42.998+0000 7f8c69e00700 1 -- 172.21.15.151:0/3324538286 shutdown_connections 2024-04-26T01:51:43.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:42.998+0000 7f8c69e00700 1 --2- 172.21.15.151:0/3324538286 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c64103450 0x7f8c6410f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:43.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:42.998+0000 7f8c69e00700 1 --2- 172.21.15.151:0/3324538286 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c64108a50 0x7f8c64108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:43.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:42.998+0000 7f8c69e00700 1 --2- 172.21.15.151:0/3324538286 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c64102a90 0x7f8c64102f10 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:43.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:42.998+0000 7f8c69e00700 1 -- 172.21.15.151:0/3324538286 >> 172.21.15.151:0/3324538286 conn(0x7f8c64079ba0 msgr2=0x7f8c64079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:43.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:42.998+0000 7f8c69e00700 1 -- 172.21.15.151:0/3324538286 shutdown_connections 2024-04-26T01:51:43.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:42.998+0000 7f8c69e00700 1 -- 172.21.15.151:0/3324538286 wait complete. 2024-04-26T01:51:43.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:42.999+0000 7f8c69e00700 1 Processor -- start 2024-04-26T01:51:43.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:43.000+0000 7f8c69e00700 1 -- start start 2024-04-26T01:51:43.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:43.000+0000 7f8c69e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c64102a90 0x7f8c6419b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:43.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:43.000+0000 7f8c69e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c64103450 0x7f8c6419b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:43.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:43.000+0000 7f8c69e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c64108a50 0x7f8c6419fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:43.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:43.000+0000 7f8c69e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8c64112c50 con 0x7f8c64102a90 2024-04-26T01:51:43.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:43.000+0000 7f8c69e00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8c64112ad0 con 0x7f8c64103450 2024-04-26T01:51:43.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:43.000+0000 7f8c69e00700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8c64112dd0 con 0x7f8c64108a50 2024-04-26T01:51:43.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:43.000+0000 7f8c62a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c64102a90 0x7f8c6419b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:43.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:43.000+0000 7f8c62a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c64102a90 0x7f8c6419b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:57920/0 (socket says 172.21.15.151:57920) 2024-04-26T01:51:43.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:43.000+0000 7f8c62a00700 1 -- 172.21.15.151:0/3694385355 learned_addr learned my addr 172.21.15.151:0/3694385355 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:51:43.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:43.001+0000 7f8c62000700 1 --2- 172.21.15.151:0/3694385355 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c64103450 0x7f8c6419b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:43.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:43.001+0000 7f8c63400700 1 --2- 172.21.15.151:0/3694385355 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c64108a50 0x7f8c6419fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:43.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:43.001+0000 7f8c62a00700 1 -- 172.21.15.151:0/3694385355 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c64108a50 msgr2=0x7f8c6419fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:43.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:43.001+0000 7f8c62a00700 1 --2- 172.21.15.151:0/3694385355 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c64108a50 0x7f8c6419fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:43.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:43.001+0000 7f8c62a00700 1 -- 172.21.15.151:0/3694385355 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c64103450 msgr2=0x7f8c6419b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:43.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:43.001+0000 7f8c62a00700 1 --2- 172.21.15.151:0/3694385355 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c64103450 0x7f8c6419b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:43.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:43.001+0000 7f8c62a00700 1 -- 172.21.15.151:0/3694385355 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c641a0390 con 0x7f8c64102a90 2024-04-26T01:51:43.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:43.001+0000 7f8c62000700 1 --2- 172.21.15.151:0/3694385355 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c64103450 0x7f8c6419b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:51:43.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:43.001+0000 7f8c62a00700 1 --2- 172.21.15.151:0/3694385355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c64102a90 0x7f8c6419b380 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8c5400e9c0 tx=0x7f8c5400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:43.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:43.001+0000 7f8c68a00700 1 -- 172.21.15.151:0/3694385355 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5400cbd0 con 0x7f8c64102a90 2024-04-26T01:51:43.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:43.002+0000 7f8c68a00700 1 -- 172.21.15.151:0/3694385355 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c5400ee50 con 0x7f8c64102a90 2024-04-26T01:51:43.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:43.002+0000 7f8c69e00700 1 -- 172.21.15.151:0/3694385355 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c641a0680 con 0x7f8c64102a90 2024-04-26T01:51:43.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:43.002+0000 7f8c68a00700 1 -- 172.21.15.151:0/3694385355 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c54005550 con 0x7f8c64102a90 2024-04-26T01:51:43.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:43.002+0000 7f8c69e00700 1 -- 172.21.15.151:0/3694385355 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c641a7f00 con 0x7f8c64102a90 2024-04-26T01:51:43.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:43.003+0000 7f8c69e00700 1 -- 172.21.15.151:0/3694385355 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c64069730 con 0x7f8c64102a90 2024-04-26T01:51:43.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:43.005+0000 7f8c68a00700 1 -- 172.21.15.151:0/3694385355 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f8c54004750 con 0x7f8c64102a90 2024-04-26T01:51:43.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:43.006+0000 7f8c68a00700 1 --2- 172.21.15.151:0/3694385355 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f8c480a3c40 0x7f8c480a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:43.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:43.006+0000 7f8c68a00700 1 -- 172.21.15.151:0/3694385355 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f8c540d05c0 con 0x7f8c64102a90 2024-04-26T01:51:43.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:43.006+0000 7f8c62000700 1 --2- 172.21.15.151:0/3694385355 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f8c480a3c40 0x7f8c480a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:43.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:43.007+0000 7f8c62000700 1 --2- 172.21.15.151:0/3694385355 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f8c480a3c40 0x7f8c480a6100 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8c6407a430 tx=0x7f8c5000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:43.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:43.011+0000 7f8c68a00700 1 -- 172.21.15.151:0/3694385355 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8c5409cfa0 con 0x7f8c64102a90 2024-04-26T01:51:43.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:51:42 smithi165 ceph-mon[32648]: pgmap v390: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-04-26T01:51:43.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:51:42 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/1892928669' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-26T01:51:43.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:43.237+0000 7f8c69e00700 1 -- 172.21.15.151:0/3694385355 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f8c6419c6e0 con 0x7f8c64102a90 2024-04-26T01:51:43.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:43.238+0000 7f8c68a00700 1 -- 172.21.15.151:0/3694385355 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f8c5409c8a0 con 0x7f8c64102a90 2024-04-26T01:51:43.237 INFO:teuthology.orchestra.run.smithi151.stdout:296352743477 2024-04-26T01:51:43.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:43.241+0000 7f8c69e00700 1 -- 172.21.15.151:0/3694385355 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f8c480a3c40 msgr2=0x7f8c480a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:43.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:43.241+0000 7f8c69e00700 1 --2- 172.21.15.151:0/3694385355 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f8c480a3c40 0x7f8c480a6100 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8c6407a430 tx=0x7f8c5000b260 comp rx=0 tx=0).stop 2024-04-26T01:51:43.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:43.241+0000 7f8c69e00700 1 -- 172.21.15.151:0/3694385355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c64102a90 msgr2=0x7f8c6419b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:43.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:43.241+0000 7f8c69e00700 1 --2- 172.21.15.151:0/3694385355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c64102a90 0x7f8c6419b380 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8c5400e9c0 tx=0x7f8c5400ecd0 comp rx=0 tx=0).stop 2024-04-26T01:51:43.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:43.242+0000 7f8c69e00700 1 -- 172.21.15.151:0/3694385355 shutdown_connections 2024-04-26T01:51:43.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:43.242+0000 7f8c69e00700 1 --2- 172.21.15.151:0/3694385355 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c64108a50 0x7f8c6419fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:43.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:43.242+0000 7f8c69e00700 1 --2- 172.21.15.151:0/3694385355 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f8c480a3c40 0x7f8c480a6100 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:43.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:43.242+0000 7f8c69e00700 1 --2- 172.21.15.151:0/3694385355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c64102a90 0x7f8c6419b380 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:43.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:43.242+0000 7f8c69e00700 1 --2- 172.21.15.151:0/3694385355 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c64103450 0x7f8c6419b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:43.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:43.242+0000 7f8c69e00700 1 -- 172.21.15.151:0/3694385355 >> 172.21.15.151:0/3694385355 conn(0x7f8c64079ba0 msgr2=0x7f8c6410dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:43.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:43.242+0000 7f8c69e00700 1 -- 172.21.15.151:0/3694385355 shutdown_connections 2024-04-26T01:51:43.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:43.242+0000 7f8c69e00700 1 -- 172.21.15.151:0/3694385355 wait complete. 2024-04-26T01:51:43.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:51:42 smithi172 ceph-mon[32785]: pgmap v390: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-04-26T01:51:43.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:51:42 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/1892928669' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-26T01:51:44.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.074+0000 7face3200700 1 -- 172.21.15.151:0/1877334531 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7facdc109c80 msgr2=0x7facdc10dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:44.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.074+0000 7face3200700 1 --2- 172.21.15.151:0/1877334531 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7facdc109c80 0x7facdc10dcd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7facc4009990 tx=0x7facc4009ca0 comp rx=0 tx=0).stop 2024-04-26T01:51:44.074 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:51:43 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/3694385355' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-26T01:51:44.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.074+0000 7face3200700 1 -- 172.21.15.151:0/1877334531 shutdown_connections 2024-04-26T01:51:44.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.074+0000 7face3200700 1 --2- 172.21.15.151:0/1877334531 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7facdc10e3b0 0x7facdc115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:44.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.074+0000 7face3200700 1 --2- 172.21.15.151:0/1877334531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7facdc1092d0 0x7facdc1096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:44.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.074+0000 7face3200700 1 --2- 172.21.15.151:0/1877334531 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7facdc109c80 0x7facdc10dcd0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:44.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.075+0000 7face3200700 1 -- 172.21.15.151:0/1877334531 >> 172.21.15.151:0/1877334531 conn(0x7facdc076210 msgr2=0x7facdc107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:44.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.075+0000 7face3200700 1 -- 172.21.15.151:0/1877334531 shutdown_connections 2024-04-26T01:51:44.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.075+0000 7face3200700 1 -- 172.21.15.151:0/1877334531 wait complete. 2024-04-26T01:51:44.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.076+0000 7face3200700 1 Processor -- start 2024-04-26T01:51:44.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.076+0000 7face3200700 1 -- start start 2024-04-26T01:51:44.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.076+0000 7face3200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7facdc1092d0 0x7facdc19f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:44.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.076+0000 7face3200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7facdc109c80 0x7facdc19fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:44.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.076+0000 7face3200700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7facdc10e3b0 0x7facdc1a4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:44.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.076+0000 7face3200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7facdc119270 con 0x7facdc1092d0 2024-04-26T01:51:44.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.076+0000 7face3200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7facdc1190f0 con 0x7facdc109c80 2024-04-26T01:51:44.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.076+0000 7face3200700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7facdc1193f0 con 0x7facdc10e3b0 2024-04-26T01:51:44.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.077+0000 7facdbe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7facdc1092d0 0x7facdc19f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:44.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.077+0000 7facdbe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7facdc1092d0 0x7facdc19f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:57934/0 (socket says 172.21.15.151:57934) 2024-04-26T01:51:44.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.077+0000 7facdbe00700 1 -- 172.21.15.151:0/1926938427 learned_addr learned my addr 172.21.15.151:0/1926938427 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:51:44.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.077+0000 7facd3400700 1 --2- 172.21.15.151:0/1926938427 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7facdc109c80 0x7facdc19fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:44.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.077+0000 7face1200700 1 --2- 172.21.15.151:0/1926938427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7facdc10e3b0 0x7facdc1a4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:44.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.077+0000 7facd3400700 1 -- 172.21.15.151:0/1926938427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7facdc10e3b0 msgr2=0x7facdc1a4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:44.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.077+0000 7facd3400700 1 --2- 172.21.15.151:0/1926938427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7facdc10e3b0 0x7facdc1a4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:44.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.077+0000 7facd3400700 1 -- 172.21.15.151:0/1926938427 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7facdc1092d0 msgr2=0x7facdc19f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:44.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.077+0000 7facd3400700 1 --2- 172.21.15.151:0/1926938427 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7facdc1092d0 0x7facdc19f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:44.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.077+0000 7facd3400700 1 -- 172.21.15.151:0/1926938427 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facdc1a4760 con 0x7facdc109c80 2024-04-26T01:51:44.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.078+0000 7facd3400700 1 --2- 172.21.15.151:0/1926938427 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7facdc109c80 0x7facdc19fce0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7facc40098e0 tx=0x7facc40057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:44.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.078+0000 7facd9600700 1 -- 172.21.15.151:0/1926938427 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc4027070 con 0x7facdc109c80 2024-04-26T01:51:44.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.078+0000 7facd9600700 1 -- 172.21.15.151:0/1926938427 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facc400b610 con 0x7facdc109c80 2024-04-26T01:51:44.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.078+0000 7face3200700 1 -- 172.21.15.151:0/1926938427 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facdc1a49f0 con 0x7facdc109c80 2024-04-26T01:51:44.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.078+0000 7facd9600700 1 -- 172.21.15.151:0/1926938427 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc40173f0 con 0x7facdc109c80 2024-04-26T01:51:44.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.078+0000 7face3200700 1 -- 172.21.15.151:0/1926938427 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facdc1ac2d0 con 0x7facdc109c80 2024-04-26T01:51:44.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.079+0000 7face3200700 1 -- 172.21.15.151:0/1926938427 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facdc069730 con 0x7facdc109c80 2024-04-26T01:51:44.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.081+0000 7facd9600700 1 -- 172.21.15.151:0/1926938427 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7facc402a070 con 0x7facdc109c80 2024-04-26T01:51:44.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.082+0000 7facd9600700 1 --2- 172.21.15.151:0/1926938427 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7facc00a3cc0 0x7facc00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:44.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.083+0000 7facd9600700 1 -- 172.21.15.151:0/1926938427 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7facc40d13b0 con 0x7facdc109c80 2024-04-26T01:51:44.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.083+0000 7facdbe00700 1 --2- 172.21.15.151:0/1926938427 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7facc00a3cc0 0x7facc00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:44.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.084+0000 7facdbe00700 1 --2- 172.21.15.151:0/1926938427 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7facc00a3cc0 0x7facc00a6180 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7faccc009c10 tx=0x7faccc00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:44.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.085+0000 7facd9600700 1 -- 172.21.15.151:0/1926938427 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7facc4031080 con 0x7facdc109c80 2024-04-26T01:51:44.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:51:43 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/3694385355' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-26T01:51:44.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:51:43 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/3694385355' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-26T01:51:44.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.324+0000 7face3200700 1 -- 172.21.15.151:0/1926938427 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7facdc02df80 con 0x7facdc109c80 2024-04-26T01:51:44.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.325+0000 7facd9600700 1 -- 172.21.15.151:0/1926938427 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7facc4013070 con 0x7facdc109c80 2024-04-26T01:51:44.323 INFO:teuthology.orchestra.run.smithi151.stdout:206158430279 2024-04-26T01:51:44.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.328+0000 7face3200700 1 -- 172.21.15.151:0/1926938427 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7facc00a3cc0 msgr2=0x7facc00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:44.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.328+0000 7face3200700 1 --2- 172.21.15.151:0/1926938427 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7facc00a3cc0 0x7facc00a6180 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7faccc009c10 tx=0x7faccc00b040 comp rx=0 tx=0).stop 2024-04-26T01:51:44.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.328+0000 7face3200700 1 -- 172.21.15.151:0/1926938427 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7facdc109c80 msgr2=0x7facdc19fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:44.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.328+0000 7face3200700 1 --2- 172.21.15.151:0/1926938427 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7facdc109c80 0x7facdc19fce0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7facc40098e0 tx=0x7facc40057c0 comp rx=0 tx=0).stop 2024-04-26T01:51:44.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.328+0000 7face3200700 1 -- 172.21.15.151:0/1926938427 shutdown_connections 2024-04-26T01:51:44.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.328+0000 7face3200700 1 --2- 172.21.15.151:0/1926938427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7facdc10e3b0 0x7facdc1a4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:44.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.329+0000 7face3200700 1 --2- 172.21.15.151:0/1926938427 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7facc00a3cc0 0x7facc00a6180 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:44.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.329+0000 7face3200700 1 --2- 172.21.15.151:0/1926938427 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7facdc1092d0 0x7facdc19f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:44.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.329+0000 7face3200700 1 --2- 172.21.15.151:0/1926938427 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7facdc109c80 0x7facdc19fce0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:44.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.329+0000 7face3200700 1 -- 172.21.15.151:0/1926938427 >> 172.21.15.151:0/1926938427 conn(0x7facdc076210 msgr2=0x7facdc077dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:44.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.329+0000 7face3200700 1 -- 172.21.15.151:0/1926938427 shutdown_connections 2024-04-26T01:51:44.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.329+0000 7face3200700 1 -- 172.21.15.151:0/1926938427 wait complete. 2024-04-26T01:51:44.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.675+0000 7f45da800700 1 -- 172.21.15.151:0/496472199 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f45d4108a50 msgr2=0x7f45d4108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:44.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.675+0000 7f45da800700 1 --2- 172.21.15.151:0/496472199 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f45d4108a50 0x7f45d4108e30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f45c400b0a0 tx=0x7f45c400b3b0 comp rx=0 tx=0).stop 2024-04-26T01:51:44.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.675+0000 7f45da800700 1 -- 172.21.15.151:0/496472199 shutdown_connections 2024-04-26T01:51:44.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.675+0000 7f45da800700 1 --2- 172.21.15.151:0/496472199 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45d4103450 0x7f45d410f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:44.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.675+0000 7f45da800700 1 --2- 172.21.15.151:0/496472199 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f45d4108a50 0x7f45d4108e30 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:44.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.675+0000 7f45da800700 1 --2- 172.21.15.151:0/496472199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f45d4102a90 0x7f45d4102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:44.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.675+0000 7f45da800700 1 -- 172.21.15.151:0/496472199 >> 172.21.15.151:0/496472199 conn(0x7f45d4079ba0 msgr2=0x7f45d4079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:44.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.675+0000 7f45da800700 1 -- 172.21.15.151:0/496472199 shutdown_connections 2024-04-26T01:51:44.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.675+0000 7f45da800700 1 -- 172.21.15.151:0/496472199 wait complete. 2024-04-26T01:51:44.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.677+0000 7f45da800700 1 Processor -- start 2024-04-26T01:51:44.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.677+0000 7f45da800700 1 -- start start 2024-04-26T01:51:44.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.677+0000 7f45da800700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f45d4102a90 0x7f45d419b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:44.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.677+0000 7f45da800700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f45d4103450 0x7f45d419b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:44.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.677+0000 7f45da800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45d4108a50 0x7f45d419fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:44.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.677+0000 7f45da800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f45d4112c30 con 0x7f45d4108a50 2024-04-26T01:51:44.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.677+0000 7f45da800700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f45d4112ab0 con 0x7f45d4103450 2024-04-26T01:51:44.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.677+0000 7f45da800700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f45d4112db0 con 0x7f45d4102a90 2024-04-26T01:51:44.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.678+0000 7f45d3400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f45d4102a90 0x7f45d419b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:44.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.678+0000 7f45d3400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f45d4102a90 0x7f45d419b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:55782/0 (socket says 172.21.15.151:55782) 2024-04-26T01:51:44.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.678+0000 7f45d3400700 1 -- 172.21.15.151:0/757288046 learned_addr learned my addr 172.21.15.151:0/757288046 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:51:44.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.678+0000 7f45caa00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f45d4103450 0x7f45d419b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:44.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.678+0000 7f45d3e00700 1 --2- 172.21.15.151:0/757288046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45d4108a50 0x7f45d419fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:44.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.678+0000 7f45d3400700 1 -- 172.21.15.151:0/757288046 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f45d4103450 msgr2=0x7f45d419b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:44.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.678+0000 7f45d3400700 1 --2- 172.21.15.151:0/757288046 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f45d4103450 0x7f45d419b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:44.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.678+0000 7f45d3400700 1 -- 172.21.15.151:0/757288046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45d4108a50 msgr2=0x7f45d419fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:44.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.678+0000 7f45d3400700 1 --2- 172.21.15.151:0/757288046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45d4108a50 0x7f45d419fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:44.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.678+0000 7f45d3400700 1 -- 172.21.15.151:0/757288046 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45d41a0370 con 0x7f45d4102a90 2024-04-26T01:51:44.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.679+0000 7f45caa00700 1 --2- 172.21.15.151:0/757288046 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f45d4103450 0x7f45d419b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:51:44.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.679+0000 7f45d3400700 1 --2- 172.21.15.151:0/757288046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f45d4102a90 0x7f45d419b360 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f45c400d010 tx=0x7f45c4004d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:44.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.679+0000 7f45d0c00700 1 -- 172.21.15.151:0/757288046 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c4021070 con 0x7f45d4102a90 2024-04-26T01:51:44.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.679+0000 7f45da800700 1 -- 172.21.15.151:0/757288046 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45d41a0600 con 0x7f45d4102a90 2024-04-26T01:51:44.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.679+0000 7f45d0c00700 1 -- 172.21.15.151:0/757288046 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45c4007e30 con 0x7f45d4102a90 2024-04-26T01:51:44.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.679+0000 7f45d0c00700 1 -- 172.21.15.151:0/757288046 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c4004640 con 0x7f45d4102a90 2024-04-26T01:51:44.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.679+0000 7f45da800700 1 -- 172.21.15.151:0/757288046 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45d41a7ee0 con 0x7f45d4102a90 2024-04-26T01:51:44.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.681+0000 7f45da800700 1 -- 172.21.15.151:0/757288046 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45d4069730 con 0x7f45d4102a90 2024-04-26T01:51:44.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.682+0000 7f45d0c00700 1 -- 172.21.15.151:0/757288046 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f45c4007980 con 0x7f45d4102a90 2024-04-26T01:51:44.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.685+0000 7f45d0c00700 1 --2- 172.21.15.151:0/757288046 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f45b80a3cc0 0x7f45b80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:44.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.685+0000 7f45d0c00700 1 -- 172.21.15.151:0/757288046 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f45c400f020 con 0x7f45d4102a90 2024-04-26T01:51:44.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.685+0000 7f45caa00700 1 --2- 172.21.15.151:0/757288046 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f45b80a3cc0 0x7f45b80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:44.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.686+0000 7f45caa00700 1 --2- 172.21.15.151:0/757288046 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f45b80a3cc0 0x7f45b80a6180 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f45d407a4a0 tx=0x7f45bc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:44.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.688+0000 7f45d0c00700 1 -- 172.21.15.151:0/757288046 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f45c402a020 con 0x7f45d4102a90 2024-04-26T01:51:44.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.921+0000 7f45da800700 1 -- 172.21.15.151:0/757288046 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f45d402df80 con 0x7f45d4102a90 2024-04-26T01:51:44.920 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:51:44 smithi151 ceph-mon[28057]: pgmap v391: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:51:44.920 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:51:44 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/1926938427' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-26T01:51:44.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.922+0000 7f45d0c00700 1 -- 172.21.15.151:0/757288046 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f45c4004000 con 0x7f45d4102a90 2024-04-26T01:51:44.921 INFO:teuthology.orchestra.run.smithi151.stdout:227633266752 2024-04-26T01:51:44.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.925+0000 7f45da800700 1 -- 172.21.15.151:0/757288046 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f45b80a3cc0 msgr2=0x7f45b80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:44.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.925+0000 7f45da800700 1 --2- 172.21.15.151:0/757288046 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f45b80a3cc0 0x7f45b80a6180 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f45d407a4a0 tx=0x7f45bc009260 comp rx=0 tx=0).stop 2024-04-26T01:51:44.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.925+0000 7f45da800700 1 -- 172.21.15.151:0/757288046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f45d4102a90 msgr2=0x7f45d419b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:44.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.925+0000 7f45da800700 1 --2- 172.21.15.151:0/757288046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f45d4102a90 0x7f45d419b360 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f45c400d010 tx=0x7f45c4004d00 comp rx=0 tx=0).stop 2024-04-26T01:51:44.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.925+0000 7f45da800700 1 -- 172.21.15.151:0/757288046 shutdown_connections 2024-04-26T01:51:44.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.925+0000 7f45da800700 1 --2- 172.21.15.151:0/757288046 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f45b80a3cc0 0x7f45b80a6180 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:44.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.925+0000 7f45da800700 1 --2- 172.21.15.151:0/757288046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45d4108a50 0x7f45d419fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:44.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.925+0000 7f45da800700 1 --2- 172.21.15.151:0/757288046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f45d4102a90 0x7f45d419b360 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:44.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.925+0000 7f45da800700 1 --2- 172.21.15.151:0/757288046 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f45d4103450 0x7f45d419b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:44.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.925+0000 7f45da800700 1 -- 172.21.15.151:0/757288046 >> 172.21.15.151:0/757288046 conn(0x7f45d4079ba0 msgr2=0x7f45d410df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:44.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.926+0000 7f45da800700 1 -- 172.21.15.151:0/757288046 shutdown_connections 2024-04-26T01:51:44.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:44.926+0000 7f45da800700 1 -- 172.21.15.151:0/757288046 wait complete. 2024-04-26T01:51:45.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:51:44 smithi165 ceph-mon[32648]: pgmap v391: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:51:45.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:51:44 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/1926938427' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-26T01:51:45.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:51:44 smithi172 ceph-mon[32785]: pgmap v391: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:51:45.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:51:44 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/1926938427' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-26T01:51:45.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:51:45 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/757288046' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-26T01:51:46.015 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:51:46.064 INFO:tasks.cephadm.ceph_manager.ceph:need seq 150323855437 got 150323855443 for osd.4 2024-04-26T01:51:46.065 DEBUG:teuthology.parallel:result is None 2024-04-26T01:51:46.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:51:45 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/757288046' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-26T01:51:46.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:51:45 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/757288046' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-26T01:51:46.936 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:51:46 smithi151 ceph-mon[28057]: pgmap v392: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:51:47.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:51:46 smithi172 ceph-mon[32785]: pgmap v392: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:51:47.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:51:46 smithi165 ceph-mon[32648]: pgmap v392: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:51:47.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:47.107+0000 7f6056400700 1 -- 172.21.15.151:0/458209971 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6050108960 msgr2=0x7f6050108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:47.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:47.107+0000 7f6056400700 1 --2- 172.21.15.151:0/458209971 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6050108960 0x7f6050108d40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6044009990 tx=0x7f6044009ca0 comp rx=0 tx=0).stop 2024-04-26T01:51:47.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:47.107+0000 7f6056400700 1 -- 172.21.15.151:0/458209971 shutdown_connections 2024-04-26T01:51:47.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:47.107+0000 7f6056400700 1 --2- 172.21.15.151:0/458209971 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6050103320 0x7f605010f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:47.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:47.107+0000 7f6056400700 1 --2- 172.21.15.151:0/458209971 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6050108960 0x7f6050108d40 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:47.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:47.107+0000 7f6056400700 1 --2- 172.21.15.151:0/458209971 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6050102960 0x7f6050102de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:47.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:47.107+0000 7f6056400700 1 -- 172.21.15.151:0/458209971 >> 172.21.15.151:0/458209971 conn(0x7f60500fe490 msgr2=0x7f60501008b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:47.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:47.107+0000 7f6056400700 1 -- 172.21.15.151:0/458209971 shutdown_connections 2024-04-26T01:51:47.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:47.107+0000 7f6056400700 1 -- 172.21.15.151:0/458209971 wait complete. 2024-04-26T01:51:47.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:47.108+0000 7f6056400700 1 Processor -- start 2024-04-26T01:51:47.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:47.108+0000 7f6056400700 1 -- start start 2024-04-26T01:51:47.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:47.109+0000 7f6056400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6050102960 0x7f6050198ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:47.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:47.109+0000 7f6056400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6050103320 0x7f6050199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:47.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:47.109+0000 7f6056400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6050108960 0x7f605019d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:47.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:47.109+0000 7f6056400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6050112b40 con 0x7f6050108960 2024-04-26T01:51:47.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:47.109+0000 7f6056400700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f60501129c0 con 0x7f6050103320 2024-04-26T01:51:47.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:47.109+0000 7f6056400700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6050112cc0 con 0x7f6050102960 2024-04-26T01:51:47.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:47.109+0000 7f6055a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6050108960 0x7f605019d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:47.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:47.109+0000 7f6055000700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6050102960 0x7f6050198ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:47.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:47.109+0000 7f6055a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6050108960 0x7f605019d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:57972/0 (socket says 172.21.15.151:57972) 2024-04-26T01:51:47.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:47.109+0000 7f604fe00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6050103320 0x7f6050199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:47.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:47.109+0000 7f6055a00700 1 -- 172.21.15.151:0/1081506840 learned_addr learned my addr 172.21.15.151:0/1081506840 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:51:47.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:47.110+0000 7f6055000700 1 -- 172.21.15.151:0/1081506840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6050103320 msgr2=0x7f6050199530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:47.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:47.110+0000 7f6055000700 1 --2- 172.21.15.151:0/1081506840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6050103320 0x7f6050199530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:47.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:47.110+0000 7f6055000700 1 -- 172.21.15.151:0/1081506840 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6050108960 msgr2=0x7f605019d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:47.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:47.110+0000 7f6055000700 1 --2- 172.21.15.151:0/1081506840 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6050108960 0x7f605019d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:47.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:47.110+0000 7f6055000700 1 -- 172.21.15.151:0/1081506840 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f605019e000 con 0x7f6050102960 2024-04-26T01:51:47.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:47.110+0000 7f604fe00700 1 --2- 172.21.15.151:0/1081506840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6050103320 0x7f6050199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:51:47.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:47.110+0000 7f6055000700 1 --2- 172.21.15.151:0/1081506840 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6050102960 0x7f6050198ff0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f60440059d0 tx=0x7f60440047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:47.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:47.110+0000 7f604d600700 1 -- 172.21.15.151:0/1081506840 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6044027070 con 0x7f6050102960 2024-04-26T01:51:47.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:47.111+0000 7f6056400700 1 -- 172.21.15.151:0/1081506840 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f605019e290 con 0x7f6050102960 2024-04-26T01:51:47.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:47.111+0000 7f604d600700 1 -- 172.21.15.151:0/1081506840 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6044022470 con 0x7f6050102960 2024-04-26T01:51:47.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:47.111+0000 7f604d600700 1 -- 172.21.15.151:0/1081506840 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6044017650 con 0x7f6050102960 2024-04-26T01:51:47.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:47.111+0000 7f6056400700 1 -- 172.21.15.151:0/1081506840 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60501a5b70 con 0x7f6050102960 2024-04-26T01:51:47.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:47.112+0000 7f6056400700 1 -- 172.21.15.151:0/1081506840 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f605010aef0 con 0x7f6050102960 2024-04-26T01:51:47.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:47.114+0000 7f604d600700 1 -- 172.21.15.151:0/1081506840 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f604402a070 con 0x7f6050102960 2024-04-26T01:51:47.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:47.115+0000 7f604d600700 1 --2- 172.21.15.151:0/1081506840 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f60380a3cc0 0x7f60380a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:47.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:47.115+0000 7f604d600700 1 -- 172.21.15.151:0/1081506840 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f60440d11a0 con 0x7f6050102960 2024-04-26T01:51:47.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:47.116+0000 7f604fe00700 1 --2- 172.21.15.151:0/1081506840 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f60380a3cc0 0x7f60380a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:47.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:47.116+0000 7f604fe00700 1 --2- 172.21.15.151:0/1081506840 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f60380a3cc0 0x7f60380a6180 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6050100af0 tx=0x7f603c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:47.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:47.118+0000 7f604d600700 1 -- 172.21.15.151:0/1081506840 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6044013070 con 0x7f6050102960 2024-04-26T01:51:47.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:47.353+0000 7f6056400700 1 -- 172.21.15.151:0/1081506840 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f605019a350 con 0x7f6050102960 2024-04-26T01:51:47.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:47.353+0000 7f604d600700 1 -- 172.21.15.151:0/1081506840 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f604409dc10 con 0x7f6050102960 2024-04-26T01:51:47.659 INFO:teuthology.orchestra.run.smithi151.stdout:115964117079 2024-04-26T01:51:47.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:47.358+0000 7f6056400700 1 -- 172.21.15.151:0/1081506840 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f60380a3cc0 msgr2=0x7f60380a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:47.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:47.358+0000 7f6056400700 1 --2- 172.21.15.151:0/1081506840 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f60380a3cc0 0x7f60380a6180 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6050100af0 tx=0x7f603c009260 comp rx=0 tx=0).stop 2024-04-26T01:51:47.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:47.358+0000 7f6056400700 1 -- 172.21.15.151:0/1081506840 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6050102960 msgr2=0x7f6050198ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:47.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:47.358+0000 7f6056400700 1 --2- 172.21.15.151:0/1081506840 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6050102960 0x7f6050198ff0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f60440059d0 tx=0x7f60440047b0 comp rx=0 tx=0).stop 2024-04-26T01:51:47.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:47.358+0000 7f6056400700 1 -- 172.21.15.151:0/1081506840 shutdown_connections 2024-04-26T01:51:47.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:47.358+0000 7f6056400700 1 --2- 172.21.15.151:0/1081506840 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f60380a3cc0 0x7f60380a6180 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:47.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:47.358+0000 7f6056400700 1 --2- 172.21.15.151:0/1081506840 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6050108960 0x7f605019d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:47.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:47.358+0000 7f6056400700 1 --2- 172.21.15.151:0/1081506840 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6050102960 0x7f6050198ff0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:47.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:47.358+0000 7f6056400700 1 --2- 172.21.15.151:0/1081506840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6050103320 0x7f6050199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:47.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:47.358+0000 7f6056400700 1 -- 172.21.15.151:0/1081506840 >> 172.21.15.151:0/1081506840 conn(0x7f60500fe490 msgr2=0x7f60500fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:47.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:47.358+0000 7f6056400700 1 -- 172.21.15.151:0/1081506840 shutdown_connections 2024-04-26T01:51:47.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:47.358+0000 7f6056400700 1 -- 172.21.15.151:0/1081506840 wait complete. 2024-04-26T01:51:47.865 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:51:47 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/1081506840' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-26T01:51:48.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:51:47 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/1081506840' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-26T01:51:48.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:51:47 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/1081506840' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-26T01:51:49.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:51:48 smithi151 ceph-mon[28057]: pgmap v393: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T01:51:49.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:51:48 smithi172 ceph-mon[32785]: pgmap v393: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T01:51:49.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:51:48 smithi165 ceph-mon[32648]: pgmap v393: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T01:51:49.628 INFO:tasks.cephadm.ceph_manager.ceph:need seq 115964117074 got 115964117079 for osd.3 2024-04-26T01:51:49.628 DEBUG:teuthology.parallel:result is None 2024-04-26T01:51:51.062 INFO:tasks.cephadm.ceph_manager.ceph:need seq 261993005110 got 261993005116 for osd.8 2024-04-26T01:51:51.062 DEBUG:teuthology.parallel:result is None 2024-04-26T01:51:51.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:51:50 smithi151 ceph-mon[28057]: pgmap v394: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:51:51.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:51:50 smithi172 ceph-mon[32785]: pgmap v394: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:51:51.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:51:50 smithi165 ceph-mon[32648]: pgmap v394: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:51:51.547 INFO:tasks.cephadm.ceph_manager.ceph:need seq 227633266748 got 227633266752 for osd.7 2024-04-26T01:51:51.547 DEBUG:teuthology.parallel:result is None 2024-04-26T01:51:51.548 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430273 got 206158430279 for osd.6 2024-04-26T01:51:51.548 DEBUG:teuthology.parallel:result is None 2024-04-26T01:51:52.087 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:51:51 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/2171979175' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-26T01:51:52.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:51:51 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/2171979175' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-26T01:51:52.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:51:51 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/2171979175' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-26T01:51:52.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:52.423+0000 7f3776000700 1 -- 172.21.15.151:0/3443672521 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3770109c80 msgr2=0x7f377010dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:52.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:52.423+0000 7f3776000700 1 --2- 172.21.15.151:0/3443672521 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3770109c80 0x7f377010dcd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f375c0099c0 tx=0x7f375c009cd0 comp rx=0 tx=0).stop 2024-04-26T01:51:52.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:52.423+0000 7f3776000700 1 -- 172.21.15.151:0/3443672521 shutdown_connections 2024-04-26T01:51:52.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:52.423+0000 7f3776000700 1 --2- 172.21.15.151:0/3443672521 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f377010e3b0 0x7f3770115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:52.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:52.423+0000 7f3776000700 1 --2- 172.21.15.151:0/3443672521 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37701092d0 0x7f37701096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:52.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:52.423+0000 7f3776000700 1 --2- 172.21.15.151:0/3443672521 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3770109c80 0x7f377010dcd0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:52.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:52.423+0000 7f3776000700 1 -- 172.21.15.151:0/3443672521 >> 172.21.15.151:0/3443672521 conn(0x7f3770076210 msgr2=0x7f3770107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:52.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:52.423+0000 7f3776000700 1 -- 172.21.15.151:0/3443672521 shutdown_connections 2024-04-26T01:51:52.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:52.423+0000 7f3776000700 1 -- 172.21.15.151:0/3443672521 wait complete. 2024-04-26T01:51:52.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:52.425+0000 7f3776000700 1 Processor -- start 2024-04-26T01:51:52.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:52.425+0000 7f3776000700 1 -- start start 2024-04-26T01:51:52.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:52.425+0000 7f3776000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f37701092d0 0x7f377019f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:52.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:52.425+0000 7f3776000700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3770109c80 0x7f377019fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:52.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:52.425+0000 7f3776000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f377010e3b0 0x7f37701a4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:52.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:52.425+0000 7f3776000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3770119220 con 0x7f377010e3b0 2024-04-26T01:51:52.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:52.425+0000 7f3776000700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f37701190a0 con 0x7f37701092d0 2024-04-26T01:51:52.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:52.425+0000 7f3776000700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f37701193a0 con 0x7f3770109c80 2024-04-26T01:51:52.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:52.426+0000 7f376fe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f377010e3b0 0x7f37701a4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:52.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:52.426+0000 7f376fe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f377010e3b0 0x7f37701a4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:39210/0 (socket says 172.21.15.151:39210) 2024-04-26T01:51:52.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:52.426+0000 7f376fe00700 1 -- 172.21.15.151:0/399244736 learned_addr learned my addr 172.21.15.151:0/399244736 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:51:52.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:52.426+0000 7f376ea00700 1 --2- 172.21.15.151:0/399244736 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3770109c80 0x7f377019fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:52.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:52.426+0000 7f376f400700 1 --2- 172.21.15.151:0/399244736 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f37701092d0 0x7f377019f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:52.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:52.426+0000 7f376fe00700 1 -- 172.21.15.151:0/399244736 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3770109c80 msgr2=0x7f377019fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:52.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:52.426+0000 7f376fe00700 1 --2- 172.21.15.151:0/399244736 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3770109c80 0x7f377019fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:52.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:52.426+0000 7f376fe00700 1 -- 172.21.15.151:0/399244736 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f37701092d0 msgr2=0x7f377019f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:52.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:52.426+0000 7f376fe00700 1 --2- 172.21.15.151:0/399244736 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f37701092d0 0x7f377019f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:52.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:52.426+0000 7f376fe00700 1 -- 172.21.15.151:0/399244736 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37701a4760 con 0x7f377010e3b0 2024-04-26T01:51:52.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:52.426+0000 7f376ea00700 1 --2- 172.21.15.151:0/399244736 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3770109c80 0x7f377019fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:51:52.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:52.426+0000 7f376f400700 1 --2- 172.21.15.151:0/399244736 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f37701092d0 0x7f377019f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:51:52.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:52.426+0000 7f376fe00700 1 --2- 172.21.15.151:0/399244736 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f377010e3b0 0x7f37701a4020 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f376800ef30 tx=0x7f376800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:52.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:52.427+0000 7f3767e00700 1 -- 172.21.15.151:0/399244736 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37680092e0 con 0x7f377010e3b0 2024-04-26T01:51:52.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:52.427+0000 7f3776000700 1 -- 172.21.15.151:0/399244736 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37701a49f0 con 0x7f377010e3b0 2024-04-26T01:51:52.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:52.427+0000 7f3767e00700 1 -- 172.21.15.151:0/399244736 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3768004bb0 con 0x7f377010e3b0 2024-04-26T01:51:52.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:52.427+0000 7f3767e00700 1 -- 172.21.15.151:0/399244736 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3768004020 con 0x7f377010e3b0 2024-04-26T01:51:52.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:52.427+0000 7f3776000700 1 -- 172.21.15.151:0/399244736 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37701ac2d0 con 0x7f377010e3b0 2024-04-26T01:51:52.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:52.428+0000 7f3776000700 1 -- 172.21.15.151:0/399244736 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3770069730 con 0x7f377010e3b0 2024-04-26T01:51:52.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:52.430+0000 7f3767e00700 1 -- 172.21.15.151:0/399244736 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f376801b030 con 0x7f377010e3b0 2024-04-26T01:51:52.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:52.431+0000 7f3767e00700 1 --2- 172.21.15.151:0/399244736 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f37540a3c40 0x7f37540a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:52.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:52.431+0000 7f3767e00700 1 -- 172.21.15.151:0/399244736 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f37680cfe40 con 0x7f377010e3b0 2024-04-26T01:51:52.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:52.432+0000 7f376f400700 1 --2- 172.21.15.151:0/399244736 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f37540a3c40 0x7f37540a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:52.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:52.432+0000 7f376f400700 1 --2- 172.21.15.151:0/399244736 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f37540a3c40 0x7f37540a6100 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3760007fd0 tx=0x7f37600073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:52.434 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:52.436+0000 7f3767e00700 1 -- 172.21.15.151:0/399244736 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3768022020 con 0x7f377010e3b0 2024-04-26T01:51:52.668 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:52.669+0000 7f3776000700 1 -- 172.21.15.151:0/399244736 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f377002df80 con 0x7f377010e3b0 2024-04-26T01:51:52.668 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:52.670+0000 7f3767e00700 1 -- 172.21.15.151:0/399244736 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f3768010040 con 0x7f377010e3b0 2024-04-26T01:51:52.669 INFO:teuthology.orchestra.run.smithi151.stdout:352187318318 2024-04-26T01:51:52.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:52.673+0000 7f3776000700 1 -- 172.21.15.151:0/399244736 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f37540a3c40 msgr2=0x7f37540a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:52.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:52.673+0000 7f3776000700 1 --2- 172.21.15.151:0/399244736 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f37540a3c40 0x7f37540a6100 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3760007fd0 tx=0x7f37600073d0 comp rx=0 tx=0).stop 2024-04-26T01:51:52.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:52.673+0000 7f3776000700 1 -- 172.21.15.151:0/399244736 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f377010e3b0 msgr2=0x7f37701a4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:52.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:52.673+0000 7f3776000700 1 --2- 172.21.15.151:0/399244736 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f377010e3b0 0x7f37701a4020 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f376800ef30 tx=0x7f376800c3a0 comp rx=0 tx=0).stop 2024-04-26T01:51:52.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:52.673+0000 7f3776000700 1 -- 172.21.15.151:0/399244736 shutdown_connections 2024-04-26T01:51:52.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:52.673+0000 7f3776000700 1 --2- 172.21.15.151:0/399244736 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f37540a3c40 0x7f37540a6100 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:52.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:52.673+0000 7f3776000700 1 --2- 172.21.15.151:0/399244736 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f377010e3b0 0x7f37701a4020 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:52.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:52.673+0000 7f3776000700 1 --2- 172.21.15.151:0/399244736 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f37701092d0 0x7f377019f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:52.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:52.673+0000 7f3776000700 1 --2- 172.21.15.151:0/399244736 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3770109c80 0x7f377019fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:52.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:52.673+0000 7f3776000700 1 -- 172.21.15.151:0/399244736 >> 172.21.15.151:0/399244736 conn(0x7f3770076210 msgr2=0x7f37700768d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:52.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:52.674+0000 7f3776000700 1 -- 172.21.15.151:0/399244736 shutdown_connections 2024-04-26T01:51:52.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:52.674+0000 7f3776000700 1 -- 172.21.15.151:0/399244736 wait complete. 2024-04-26T01:51:52.956 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:51:52 smithi151 ceph-mon[28057]: pgmap v395: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:51:52.956 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:51:52 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/399244736' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-26T01:51:53.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:50.917+0000 7f6337a00700 1 -- 172.21.15.151:0/4138978645 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6330102a90 msgr2=0x7f6330102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:53.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:50.917+0000 7f6337a00700 1 --2- 172.21.15.151:0/4138978645 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6330102a90 0x7f6330102f10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f631c009990 tx=0x7f631c009ca0 comp rx=0 tx=0).stop 2024-04-26T01:51:53.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:50.917+0000 7f6337a00700 1 -- 172.21.15.151:0/4138978645 shutdown_connections 2024-04-26T01:51:53.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:50.917+0000 7f6337a00700 1 --2- 172.21.15.151:0/4138978645 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6330103450 0x7f633010f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:53.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:50.917+0000 7f6337a00700 1 --2- 172.21.15.151:0/4138978645 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6330108a50 0x7f6330108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:53.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:50.917+0000 7f6337a00700 1 --2- 172.21.15.151:0/4138978645 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6330102a90 0x7f6330102f10 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:53.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:50.917+0000 7f6337a00700 1 -- 172.21.15.151:0/4138978645 >> 172.21.15.151:0/4138978645 conn(0x7f6330079ba0 msgr2=0x7f6330079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:53.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:50.917+0000 7f6337a00700 1 -- 172.21.15.151:0/4138978645 shutdown_connections 2024-04-26T01:51:53.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:50.917+0000 7f6337a00700 1 -- 172.21.15.151:0/4138978645 wait complete. 2024-04-26T01:51:53.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:50.919+0000 7f6337a00700 1 Processor -- start 2024-04-26T01:51:53.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:50.919+0000 7f6337a00700 1 -- start start 2024-04-26T01:51:53.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:50.919+0000 7f6337a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6330102a90 0x7f6330199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:53.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:50.920+0000 7f6337a00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6330103450 0x7f6330199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:53.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:50.920+0000 7f6337a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6330108a50 0x7f633019db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:53.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:50.920+0000 7f6337a00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6330112c50 con 0x7f6330108a50 2024-04-26T01:51:53.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:50.920+0000 7f6337a00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6330112ad0 con 0x7f6330102a90 2024-04-26T01:51:53.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:50.920+0000 7f6337a00700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6330112dd0 con 0x7f6330103450 2024-04-26T01:51:53.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:50.920+0000 7f6335000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6330102a90 0x7f6330199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:53.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:50.920+0000 7f6335a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6330108a50 0x7f633019db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:53.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:50.920+0000 7f6335000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6330102a90 0x7f6330199250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:42162/0 (socket says 172.21.15.151:42162) 2024-04-26T01:51:53.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:50.920+0000 7f6335000700 1 -- 172.21.15.151:0/2171979175 learned_addr learned my addr 172.21.15.151:0/2171979175 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:51:53.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:50.920+0000 7f632fe00700 1 --2- 172.21.15.151:0/2171979175 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6330103450 0x7f6330199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:53.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:50.920+0000 7f6335000700 1 -- 172.21.15.151:0/2171979175 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6330103450 msgr2=0x7f6330199790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:53.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:50.920+0000 7f6335000700 1 --2- 172.21.15.151:0/2171979175 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6330103450 0x7f6330199790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:53.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:50.920+0000 7f6335000700 1 -- 172.21.15.151:0/2171979175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6330108a50 msgr2=0x7f633019db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:53.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:50.920+0000 7f6335000700 1 --2- 172.21.15.151:0/2171979175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6330108a50 0x7f633019db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:53.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:50.920+0000 7f6335000700 1 -- 172.21.15.151:0/2171979175 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f633019e260 con 0x7f6330102a90 2024-04-26T01:51:53.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:50.920+0000 7f6335a00700 1 --2- 172.21.15.151:0/2171979175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6330108a50 0x7f633019db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:51:53.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:50.920+0000 7f632fe00700 1 --2- 172.21.15.151:0/2171979175 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6330103450 0x7f6330199790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:51:53.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:50.921+0000 7f6335000700 1 --2- 172.21.15.151:0/2171979175 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6330102a90 0x7f6330199250 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f632400d980 tx=0x7f632400dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:53.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:50.921+0000 7f632d600700 1 -- 172.21.15.151:0/2171979175 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6324016070 con 0x7f6330102a90 2024-04-26T01:51:53.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:50.921+0000 7f6337a00700 1 -- 172.21.15.151:0/2171979175 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f633019e550 con 0x7f6330102a90 2024-04-26T01:51:53.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:50.921+0000 7f632d600700 1 -- 172.21.15.151:0/2171979175 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6324004830 con 0x7f6330102a90 2024-04-26T01:51:53.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:50.921+0000 7f632d600700 1 -- 172.21.15.151:0/2171979175 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6324005540 con 0x7f6330102a90 2024-04-26T01:51:53.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:50.921+0000 7f6337a00700 1 -- 172.21.15.151:0/2171979175 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63301a5dd0 con 0x7f6330102a90 2024-04-26T01:51:53.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:50.922+0000 7f6337a00700 1 -- 172.21.15.151:0/2171979175 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6330069730 con 0x7f6330102a90 2024-04-26T01:51:53.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:50.924+0000 7f632d600700 1 -- 172.21.15.151:0/2171979175 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f6324004220 con 0x7f6330102a90 2024-04-26T01:51:53.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:50.925+0000 7f632d600700 1 --2- 172.21.15.151:0/2171979175 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f63180a3c40 0x7f63180a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:53.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:50.926+0000 7f632fe00700 1 --2- 172.21.15.151:0/2171979175 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f63180a3c40 0x7f63180a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:53.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:50.926+0000 7f632d600700 1 -- 172.21.15.151:0/2171979175 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f63240d0100 con 0x7f6330102a90 2024-04-26T01:51:53.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:50.926+0000 7f632fe00700 1 --2- 172.21.15.151:0/2171979175 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f63180a3c40 0x7f63180a6100 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f633007a350 tx=0x7f631c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:53.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:50.930+0000 7f632d600700 1 -- 172.21.15.151:0/2171979175 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f632401f020 con 0x7f6330102a90 2024-04-26T01:51:53.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:51.162+0000 7f6337a00700 1 -- 172.21.15.151:0/2171979175 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f633002df80 con 0x7f6330102a90 2024-04-26T01:51:53.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:51.165+0000 7f632d600700 1 -- 172.21.15.151:0/2171979175 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f6324022350 con 0x7f6330102a90 2024-04-26T01:51:53.270 INFO:teuthology.orchestra.run.smithi151.stdout:171798691918 2024-04-26T01:51:53.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:51.170+0000 7f6337a00700 1 -- 172.21.15.151:0/2171979175 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f63180a3c40 msgr2=0x7f63180a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:53.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:51.170+0000 7f6337a00700 1 --2- 172.21.15.151:0/2171979175 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f63180a3c40 0x7f63180a6100 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f633007a350 tx=0x7f631c01a040 comp rx=0 tx=0).stop 2024-04-26T01:51:53.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:51.170+0000 7f6337a00700 1 -- 172.21.15.151:0/2171979175 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6330102a90 msgr2=0x7f6330199250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:53.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:51.170+0000 7f6337a00700 1 --2- 172.21.15.151:0/2171979175 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6330102a90 0x7f6330199250 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f632400d980 tx=0x7f632400dc90 comp rx=0 tx=0).stop 2024-04-26T01:51:53.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:51.170+0000 7f6337a00700 1 -- 172.21.15.151:0/2171979175 shutdown_connections 2024-04-26T01:51:53.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:51.170+0000 7f6337a00700 1 --2- 172.21.15.151:0/2171979175 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f63180a3c40 0x7f63180a6100 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:53.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:51.170+0000 7f6337a00700 1 --2- 172.21.15.151:0/2171979175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6330108a50 0x7f633019db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:53.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:51.170+0000 7f6337a00700 1 --2- 172.21.15.151:0/2171979175 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6330102a90 0x7f6330199250 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:53.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:51.170+0000 7f6337a00700 1 --2- 172.21.15.151:0/2171979175 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6330103450 0x7f6330199790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:53.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:51.170+0000 7f6337a00700 1 -- 172.21.15.151:0/2171979175 >> 172.21.15.151:0/2171979175 conn(0x7f6330079ba0 msgr2=0x7f63301019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:53.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:51.170+0000 7f6337a00700 1 -- 172.21.15.151:0/2171979175 shutdown_connections 2024-04-26T01:51:53.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:51.170+0000 7f6337a00700 1 -- 172.21.15.151:0/2171979175 wait complete. 2024-04-26T01:51:53.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:51:52 smithi172 ceph-mon[32785]: pgmap v395: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:51:53.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:51:52 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/399244736' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-26T01:51:53.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:51:52 smithi165 ceph-mon[32648]: pgmap v395: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:51:53.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:51:52 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/399244736' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-26T01:51:54.091 INFO:tasks.cephadm.ceph_manager.ceph:need seq 171798691909 got 171798691918 for osd.5 2024-04-26T01:51:54.091 DEBUG:teuthology.parallel:result is None 2024-04-26T01:51:54.709 INFO:tasks.cephadm.ceph_manager.ceph:need seq 296352743470 got 296352743477 for osd.9 2024-04-26T01:51:54.709 DEBUG:teuthology.parallel:result is None 2024-04-26T01:51:54.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:54.606+0000 7f0cfe200700 1 -- 172.21.15.151:0/469797725 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cf8109f70 msgr2=0x7f0cf8111b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:54.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:54.606+0000 7f0cfe200700 1 --2- 172.21.15.151:0/469797725 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cf8109f70 0x7f0cf8111b40 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0cf000b0d0 tx=0x7f0cf000b3e0 comp rx=0 tx=0).stop 2024-04-26T01:51:54.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:54.606+0000 7f0cfe200700 1 -- 172.21.15.151:0/469797725 shutdown_connections 2024-04-26T01:51:54.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:54.606+0000 7f0cfe200700 1 --2- 172.21.15.151:0/469797725 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cf8109f70 0x7f0cf8111b40 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:54.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:54.606+0000 7f0cfe200700 1 --2- 172.21.15.151:0/469797725 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0cf8104e90 0x7f0cf8105270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:54.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:54.606+0000 7f0cfe200700 1 --2- 172.21.15.151:0/469797725 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0cf8105840 0x7f0cf8109890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:54.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:54.606+0000 7f0cfe200700 1 -- 172.21.15.151:0/469797725 >> 172.21.15.151:0/469797725 conn(0x7f0cf8100720 msgr2=0x7f0cf8102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:54.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:54.606+0000 7f0cfe200700 1 -- 172.21.15.151:0/469797725 shutdown_connections 2024-04-26T01:51:54.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:54.606+0000 7f0cfe200700 1 -- 172.21.15.151:0/469797725 wait complete. 2024-04-26T01:51:54.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:54.607+0000 7f0cfe200700 1 Processor -- start 2024-04-26T01:51:54.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:54.607+0000 7f0cfe200700 1 -- start start 2024-04-26T01:51:54.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:54.608+0000 7f0cfe200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cf8104e90 0x7f0cf819b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:54.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:54.608+0000 7f0cfe200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0cf8105840 0x7f0cf819b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:54.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:54.608+0000 7f0cfe200700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0cf8109f70 0x7f0cf819fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:54.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:54.608+0000 7f0cfe200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf8114e80 con 0x7f0cf8104e90 2024-04-26T01:51:54.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:54.608+0000 7f0cfe200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf8114d00 con 0x7f0cf8105840 2024-04-26T01:51:54.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:54.608+0000 7f0cfe200700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf8115000 con 0x7f0cf8109f70 2024-04-26T01:51:54.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:54.608+0000 7f0cf7400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cf8104e90 0x7f0cf819b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:54.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:54.608+0000 7f0cf6a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0cf8105840 0x7f0cf819b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:54.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:54.608+0000 7f0cf7e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0cf8109f70 0x7f0cf819fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:54.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:54.608+0000 7f0cf7400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cf8104e90 0x7f0cf819b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:39226/0 (socket says 172.21.15.151:39226) 2024-04-26T01:51:54.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:54.608+0000 7f0cf7400700 1 -- 172.21.15.151:0/2275223898 learned_addr learned my addr 172.21.15.151:0/2275223898 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:51:54.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:54.608+0000 7f0cf7e00700 1 -- 172.21.15.151:0/2275223898 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0cf8105840 msgr2=0x7f0cf819b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:54.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:54.608+0000 7f0cf7e00700 1 --2- 172.21.15.151:0/2275223898 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0cf8105840 0x7f0cf819b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:54.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:54.608+0000 7f0cf7e00700 1 -- 172.21.15.151:0/2275223898 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cf8104e90 msgr2=0x7f0cf819b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:54.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:54.608+0000 7f0cf7e00700 1 --2- 172.21.15.151:0/2275223898 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cf8104e90 0x7f0cf819b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:54.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:54.608+0000 7f0cf7e00700 1 -- 172.21.15.151:0/2275223898 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cf81a02e0 con 0x7f0cf8109f70 2024-04-26T01:51:54.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:54.608+0000 7f0cf7400700 1 --2- 172.21.15.151:0/2275223898 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cf8104e90 0x7f0cf819b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:51:54.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:54.609+0000 7f0cf7e00700 1 --2- 172.21.15.151:0/2275223898 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0cf8109f70 0x7f0cf819fba0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0cf000b770 tx=0x7f0cf0007980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:54.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:54.609+0000 7f0cefe00700 1 -- 172.21.15.151:0/2275223898 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf001f070 con 0x7f0cf8109f70 2024-04-26T01:51:54.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:54.609+0000 7f0cfe200700 1 -- 172.21.15.151:0/2275223898 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cf81a0570 con 0x7f0cf8109f70 2024-04-26T01:51:54.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:54.609+0000 7f0cefe00700 1 -- 172.21.15.151:0/2275223898 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cf0007dc0 con 0x7f0cf8109f70 2024-04-26T01:51:54.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:54.609+0000 7f0cefe00700 1 -- 172.21.15.151:0/2275223898 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf001b430 con 0x7f0cf8109f70 2024-04-26T01:51:54.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:54.609+0000 7f0cfe200700 1 -- 172.21.15.151:0/2275223898 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cf81a7e50 con 0x7f0cf8109f70 2024-04-26T01:51:54.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:54.610+0000 7f0cfe200700 1 -- 172.21.15.151:0/2275223898 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cf8106370 con 0x7f0cf8109f70 2024-04-26T01:51:54.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:54.612+0000 7f0cefe00700 1 -- 172.21.15.151:0/2275223898 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f0cf0022070 con 0x7f0cf8109f70 2024-04-26T01:51:54.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:54.614+0000 7f0cefe00700 1 --2- 172.21.15.151:0/2275223898 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f0cdc0a3cc0 0x7f0cdc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:54.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:54.614+0000 7f0cefe00700 1 -- 172.21.15.151:0/2275223898 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f0cf00d0db0 con 0x7f0cf8109f70 2024-04-26T01:51:54.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:54.614+0000 7f0cf7400700 1 --2- 172.21.15.151:0/2275223898 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f0cdc0a3cc0 0x7f0cdc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:54.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:54.615+0000 7f0cf7400700 1 --2- 172.21.15.151:0/2275223898 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f0cdc0a3cc0 0x7f0cdc0a6180 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8011590 tx=0x7f0ce8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:54.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:54.618+0000 7f0cefe00700 1 -- 172.21.15.151:0/2275223898 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0cf00d5070 con 0x7f0cf8109f70 2024-04-26T01:51:54.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:54.853+0000 7f0cfe200700 1 -- 172.21.15.151:0/2275223898 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f0cf802df80 con 0x7f0cf8109f70 2024-04-26T01:51:54.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:54.854+0000 7f0cefe00700 1 -- 172.21.15.151:0/2275223898 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0cf009d820 con 0x7f0cf8109f70 2024-04-26T01:51:54.852 INFO:teuthology.orchestra.run.smithi151.stdout:34359738473 2024-04-26T01:51:54.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:54.857+0000 7f0cfe200700 1 -- 172.21.15.151:0/2275223898 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f0cdc0a3cc0 msgr2=0x7f0cdc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:54.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:54.857+0000 7f0cfe200700 1 --2- 172.21.15.151:0/2275223898 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f0cdc0a3cc0 0x7f0cdc0a6180 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8011590 tx=0x7f0ce8009260 comp rx=0 tx=0).stop 2024-04-26T01:51:54.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:54.857+0000 7f0cfe200700 1 -- 172.21.15.151:0/2275223898 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0cf8109f70 msgr2=0x7f0cf819fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:54.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:54.857+0000 7f0cfe200700 1 --2- 172.21.15.151:0/2275223898 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0cf8109f70 0x7f0cf819fba0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0cf000b770 tx=0x7f0cf0007980 comp rx=0 tx=0).stop 2024-04-26T01:51:54.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:54.857+0000 7f0cfe200700 1 -- 172.21.15.151:0/2275223898 shutdown_connections 2024-04-26T01:51:54.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:54.857+0000 7f0cfe200700 1 --2- 172.21.15.151:0/2275223898 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0cf8109f70 0x7f0cf819fba0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:54.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:54.857+0000 7f0cfe200700 1 --2- 172.21.15.151:0/2275223898 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f0cdc0a3cc0 0x7f0cdc0a6180 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:54.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:54.857+0000 7f0cfe200700 1 --2- 172.21.15.151:0/2275223898 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cf8104e90 0x7f0cf819b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:54.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:54.857+0000 7f0cfe200700 1 --2- 172.21.15.151:0/2275223898 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0cf8105840 0x7f0cf819b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:54.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:54.857+0000 7f0cfe200700 1 -- 172.21.15.151:0/2275223898 >> 172.21.15.151:0/2275223898 conn(0x7f0cf8100720 msgr2=0x7f0cf8101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:54.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:54.857+0000 7f0cfe200700 1 -- 172.21.15.151:0/2275223898 shutdown_connections 2024-04-26T01:51:54.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:54.857+0000 7f0cfe200700 1 -- 172.21.15.151:0/2275223898 wait complete. 2024-04-26T01:51:55.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:51:54 smithi151 ceph-mon[28057]: pgmap v396: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:51:55.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:51:54 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/2275223898' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-26T01:51:55.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:51:54 smithi172 ceph-mon[32785]: pgmap v396: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:51:55.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:51:54 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/2275223898' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-26T01:51:55.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:51:54 smithi165 ceph-mon[32648]: pgmap v396: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:51:55.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:51:54 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/2275223898' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-26T01:51:56.015 INFO:tasks.cephadm.ceph_manager.ceph:need seq 352187318313 got 352187318318 for osd.11 2024-04-26T01:51:56.016 DEBUG:teuthology.parallel:result is None 2024-04-26T01:51:56.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:56.688+0000 7fcfa3c00700 1 -- 172.21.15.151:0/4160419317 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf9c06afe0 msgr2=0x7fcf9c06b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:56.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:56.688+0000 7fcfa3c00700 1 --2- 172.21.15.151:0/4160419317 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf9c06afe0 0x7fcf9c06b440 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcf88009990 tx=0x7fcf88009ca0 comp rx=0 tx=0).stop 2024-04-26T01:51:56.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:56.689+0000 7fcfa3c00700 1 -- 172.21.15.151:0/4160419317 shutdown_connections 2024-04-26T01:51:56.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:56.689+0000 7fcfa3c00700 1 --2- 172.21.15.151:0/4160419317 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcf9c06b980 0x7fcf9c10f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:56.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:56.689+0000 7fcfa3c00700 1 --2- 172.21.15.151:0/4160419317 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf9c108a20 0x7fcf9c108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:56.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:56.689+0000 7fcfa3c00700 1 --2- 172.21.15.151:0/4160419317 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf9c06afe0 0x7fcf9c06b440 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:56.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:56.689+0000 7fcfa3c00700 1 -- 172.21.15.151:0/4160419317 >> 172.21.15.151:0/4160419317 conn(0x7fcf9c0788b0 msgr2=0x7fcf9c078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:56.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:56.689+0000 7fcfa3c00700 1 -- 172.21.15.151:0/4160419317 shutdown_connections 2024-04-26T01:51:56.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:56.689+0000 7fcfa3c00700 1 -- 172.21.15.151:0/4160419317 wait complete. 2024-04-26T01:51:56.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:56.691+0000 7fcfa3c00700 1 Processor -- start 2024-04-26T01:51:56.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:56.691+0000 7fcfa3c00700 1 -- start start 2024-04-26T01:51:56.690 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:56.692+0000 7fcfa3c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf9c06afe0 0x7fcf9c109970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:56.690 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:56.692+0000 7fcfa3c00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcf9c06b980 0x7fcf9c109eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:56.690 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:56.692+0000 7fcfa3c00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf9c108a20 0x7fcf9c10a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:56.690 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:56.692+0000 7fcfa3c00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcf9c112c10 con 0x7fcf9c06afe0 2024-04-26T01:51:56.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:56.692+0000 7fcfa3c00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcf9c112a90 con 0x7fcf9c06b980 2024-04-26T01:51:56.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:56.692+0000 7fcfa3c00700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcf9c112d90 con 0x7fcf9c108a20 2024-04-26T01:51:56.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:56.692+0000 7fcfa1200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf9c06afe0 0x7fcf9c109970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:56.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:56.692+0000 7fcfa1c00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf9c108a20 0x7fcf9c10a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:56.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:56.692+0000 7fcfa1c00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf9c108a20 0x7fcf9c10a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:35306/0 (socket says 172.21.15.151:35306) 2024-04-26T01:51:56.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:56.692+0000 7fcfa1c00700 1 -- 172.21.15.151:0/475472456 learned_addr learned my addr 172.21.15.151:0/475472456 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:51:56.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:56.692+0000 7fcf9be00700 1 --2- 172.21.15.151:0/475472456 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcf9c06b980 0x7fcf9c109eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:56.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:56.693+0000 7fcfa1c00700 1 -- 172.21.15.151:0/475472456 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcf9c06b980 msgr2=0x7fcf9c109eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:56.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:56.693+0000 7fcfa1c00700 1 --2- 172.21.15.151:0/475472456 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcf9c06b980 0x7fcf9c109eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:56.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:56.693+0000 7fcfa1c00700 1 -- 172.21.15.151:0/475472456 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf9c06afe0 msgr2=0x7fcf9c109970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:56.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:56.693+0000 7fcfa1c00700 1 --2- 172.21.15.151:0/475472456 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf9c06afe0 0x7fcf9c109970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:56.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:56.693+0000 7fcfa1c00700 1 -- 172.21.15.151:0/475472456 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf9c1aba90 con 0x7fcf9c108a20 2024-04-26T01:51:56.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:56.693+0000 7fcfa1c00700 1 --2- 172.21.15.151:0/475472456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf9c108a20 0x7fcf9c10a3f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcf9400ef60 tx=0x7fcf9400ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:56.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:56.693+0000 7fcf99600700 1 -- 172.21.15.151:0/475472456 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf940092e0 con 0x7fcf9c108a20 2024-04-26T01:51:56.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:56.694+0000 7fcf99600700 1 -- 172.21.15.151:0/475472456 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf94004bb0 con 0x7fcf9c108a20 2024-04-26T01:51:56.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:56.694+0000 7fcfa3c00700 1 -- 172.21.15.151:0/475472456 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf9c1abd20 con 0x7fcf9c108a20 2024-04-26T01:51:56.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:56.694+0000 7fcf99600700 1 -- 172.21.15.151:0/475472456 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf94004020 con 0x7fcf9c108a20 2024-04-26T01:51:56.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:56.694+0000 7fcfa3c00700 1 -- 172.21.15.151:0/475472456 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf9c1ac210 con 0x7fcf9c108a20 2024-04-26T01:51:56.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:56.696+0000 7fcfa3c00700 1 -- 172.21.15.151:0/475472456 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf9c104160 con 0x7fcf9c108a20 2024-04-26T01:51:56.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:56.697+0000 7fcf99600700 1 -- 172.21.15.151:0/475472456 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7fcf9401b030 con 0x7fcf9c108a20 2024-04-26T01:51:56.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:56.699+0000 7fcf99600700 1 --2- 172.21.15.151:0/475472456 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fcf840a3c40 0x7fcf840a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:56.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:56.700+0000 7fcf99600700 1 -- 172.21.15.151:0/475472456 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7fcf940cfdc0 con 0x7fcf9c108a20 2024-04-26T01:51:56.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:56.700+0000 7fcfa1200700 1 --2- 172.21.15.151:0/475472456 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fcf840a3c40 0x7fcf840a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:56.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:56.700+0000 7fcfa1200700 1 --2- 172.21.15.151:0/475472456 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fcf840a3c40 0x7fcf840a6100 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcf90009be0 tx=0x7fcf90009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:56.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:56.703+0000 7fcf99600700 1 -- 172.21.15.151:0/475472456 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcf94022020 con 0x7fcf9c108a20 2024-04-26T01:51:56.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:56.950+0000 7fcfa3c00700 1 -- 172.21.15.151:0/475472456 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fcf9c02df80 con 0x7fcf9c108a20 2024-04-26T01:51:56.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:56.952+0000 7fcf99600700 1 -- 172.21.15.151:0/475472456 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fcf94010040 con 0x7fcf9c108a20 2024-04-26T01:51:56.951 INFO:teuthology.orchestra.run.smithi151.stdout:317827579953 2024-04-26T01:51:56.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:56.956+0000 7fcfa3c00700 1 -- 172.21.15.151:0/475472456 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fcf840a3c40 msgr2=0x7fcf840a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:56.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:56.956+0000 7fcfa3c00700 1 --2- 172.21.15.151:0/475472456 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fcf840a3c40 0x7fcf840a6100 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcf90009be0 tx=0x7fcf90009260 comp rx=0 tx=0).stop 2024-04-26T01:51:56.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:56.956+0000 7fcfa3c00700 1 -- 172.21.15.151:0/475472456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf9c108a20 msgr2=0x7fcf9c10a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:56.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:56.956+0000 7fcfa3c00700 1 --2- 172.21.15.151:0/475472456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf9c108a20 0x7fcf9c10a3f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcf9400ef60 tx=0x7fcf9400ef90 comp rx=0 tx=0).stop 2024-04-26T01:51:56.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:56.957+0000 7fcfa3c00700 1 -- 172.21.15.151:0/475472456 shutdown_connections 2024-04-26T01:51:56.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:56.957+0000 7fcfa3c00700 1 --2- 172.21.15.151:0/475472456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf9c108a20 0x7fcf9c10a3f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:56.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:56.957+0000 7fcfa3c00700 1 --2- 172.21.15.151:0/475472456 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fcf840a3c40 0x7fcf840a6100 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:56.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:56.957+0000 7fcfa3c00700 1 --2- 172.21.15.151:0/475472456 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf9c06afe0 0x7fcf9c109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:56.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:56.957+0000 7fcfa3c00700 1 --2- 172.21.15.151:0/475472456 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcf9c06b980 0x7fcf9c109eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:56.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:56.957+0000 7fcfa3c00700 1 -- 172.21.15.151:0/475472456 >> 172.21.15.151:0/475472456 conn(0x7fcf9c0788b0 msgr2=0x7fcf9c10df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:56.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:56.957+0000 7fcfa3c00700 1 -- 172.21.15.151:0/475472456 shutdown_connections 2024-04-26T01:51:56.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:56.958+0000 7fcfa3c00700 1 -- 172.21.15.151:0/475472456 wait complete. 2024-04-26T01:51:57.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:51:56 smithi151 ceph-mon[28057]: pgmap v397: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:51:57.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:51:56 smithi172 ceph-mon[32785]: pgmap v397: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:51:57.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:51:56 smithi165 ceph-mon[32648]: pgmap v397: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:51:58.080 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738467 got 34359738473 for osd.0 2024-04-26T01:51:58.080 DEBUG:teuthology.parallel:result is None 2024-04-26T01:51:58.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:57.754+0000 7f16bb200700 1 -- 172.21.15.151:0/3847717764 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f16b41092d0 msgr2=0x7f16b41096b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:58.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:57.754+0000 7f16bb200700 1 --2- 172.21.15.151:0/3847717764 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f16b41092d0 0x7f16b41096b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f16a40099c0 tx=0x7f16a4009cd0 comp rx=0 tx=0).stop 2024-04-26T01:51:58.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:57.754+0000 7f16bb200700 1 -- 172.21.15.151:0/3847717764 shutdown_connections 2024-04-26T01:51:58.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:57.754+0000 7f16bb200700 1 --2- 172.21.15.151:0/3847717764 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16b410e3b0 0x7f16b4115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:58.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:57.754+0000 7f16bb200700 1 --2- 172.21.15.151:0/3847717764 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f16b41092d0 0x7f16b41096b0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:58.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:57.754+0000 7f16bb200700 1 --2- 172.21.15.151:0/3847717764 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16b4109c80 0x7f16b410dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:58.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:57.754+0000 7f16bb200700 1 -- 172.21.15.151:0/3847717764 >> 172.21.15.151:0/3847717764 conn(0x7f16b4076210 msgr2=0x7f16b4107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:58.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:57.754+0000 7f16bb200700 1 -- 172.21.15.151:0/3847717764 shutdown_connections 2024-04-26T01:51:58.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:57.754+0000 7f16bb200700 1 -- 172.21.15.151:0/3847717764 wait complete. 2024-04-26T01:51:58.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:57.755+0000 7f16bb200700 1 Processor -- start 2024-04-26T01:51:58.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:57.755+0000 7f16bb200700 1 -- start start 2024-04-26T01:51:58.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:57.756+0000 7f16bb200700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16b41092d0 0x7f16b419f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:58.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:57.756+0000 7f16bb200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f16b4109c80 0x7f16b419fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:58.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:57.756+0000 7f16bb200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16b410e3b0 0x7f16b41a3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:58.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:57.756+0000 7f16bb200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f16b4119260 con 0x7f16b4109c80 2024-04-26T01:51:58.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:57.756+0000 7f16bb200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f16b41190e0 con 0x7f16b410e3b0 2024-04-26T01:51:58.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:57.756+0000 7f16bb200700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f16b41193e0 con 0x7f16b41092d0 2024-04-26T01:51:58.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:57.756+0000 7f16b3400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f16b4109c80 0x7f16b419fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:58.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:57.756+0000 7f16b3400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f16b4109c80 0x7f16b419fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:39276/0 (socket says 172.21.15.151:39276) 2024-04-26T01:51:58.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:57.756+0000 7f16b3400700 1 -- 172.21.15.151:0/583175343 learned_addr learned my addr 172.21.15.151:0/583175343 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:51:58.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:57.756+0000 7f16b3e00700 1 --2- 172.21.15.151:0/583175343 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16b41092d0 0x7f16b419f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:58.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:57.756+0000 7f16b9200700 1 --2- 172.21.15.151:0/583175343 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16b410e3b0 0x7f16b41a3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:58.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:57.757+0000 7f16b3400700 1 -- 172.21.15.151:0/583175343 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16b41092d0 msgr2=0x7f16b419f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:58.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:57.757+0000 7f16b3400700 1 --2- 172.21.15.151:0/583175343 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16b41092d0 0x7f16b419f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:58.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:57.757+0000 7f16b3400700 1 -- 172.21.15.151:0/583175343 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16b410e3b0 msgr2=0x7f16b41a3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:58.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:57.757+0000 7f16b3400700 1 --2- 172.21.15.151:0/583175343 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16b410e3b0 0x7f16b41a3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:58.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:57.757+0000 7f16b3400700 1 -- 172.21.15.151:0/583175343 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16b41a4730 con 0x7f16b4109c80 2024-04-26T01:51:58.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:57.757+0000 7f16b3e00700 1 --2- 172.21.15.151:0/583175343 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16b41092d0 0x7f16b419f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:51:58.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:57.757+0000 7f16b3400700 1 --2- 172.21.15.151:0/583175343 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f16b4109c80 0x7f16b419fcb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f16a000e9f0 tx=0x7f16a000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:58.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:57.757+0000 7f16b0c00700 1 -- 172.21.15.151:0/583175343 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a0004a30 con 0x7f16b4109c80 2024-04-26T01:51:58.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:57.757+0000 7f16b0c00700 1 -- 172.21.15.151:0/583175343 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16a000cc70 con 0x7f16b4109c80 2024-04-26T01:51:58.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:57.757+0000 7f16bb200700 1 -- 172.21.15.151:0/583175343 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16b41a4a20 con 0x7f16b4109c80 2024-04-26T01:51:58.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:57.757+0000 7f16b0c00700 1 -- 172.21.15.151:0/583175343 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a0005530 con 0x7f16b4109c80 2024-04-26T01:51:58.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:57.757+0000 7f16bb200700 1 -- 172.21.15.151:0/583175343 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16b41ac2a0 con 0x7f16b4109c80 2024-04-26T01:51:58.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:57.758+0000 7f16bb200700 1 -- 172.21.15.151:0/583175343 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16b4069730 con 0x7f16b4109c80 2024-04-26T01:51:58.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:57.760+0000 7f16b0c00700 1 -- 172.21.15.151:0/583175343 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f16a0004bd0 con 0x7f16b4109c80 2024-04-26T01:51:58.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:57.762+0000 7f16b0c00700 1 --2- 172.21.15.151:0/583175343 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f16980a3cc0 0x7f16980a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:58.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:57.762+0000 7f16b0c00700 1 -- 172.21.15.151:0/583175343 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f16a00d0750 con 0x7f16b4109c80 2024-04-26T01:51:58.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:57.762+0000 7f16b3e00700 1 --2- 172.21.15.151:0/583175343 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f16980a3cc0 0x7f16980a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:58.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:57.763+0000 7f16b3e00700 1 --2- 172.21.15.151:0/583175343 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f16980a3cc0 0x7f16980a6180 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f16a4003760 tx=0x7f16a4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:58.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:57.766+0000 7f16b0c00700 1 -- 172.21.15.151:0/583175343 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f16a000ce20 con 0x7f16b4109c80 2024-04-26T01:51:58.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:57.995+0000 7f16bb200700 1 -- 172.21.15.151:0/583175343 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f16b402df80 con 0x7f16b4109c80 2024-04-26T01:51:58.108 INFO:teuthology.orchestra.run.smithi151.stdout:77309411425 2024-04-26T01:51:58.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:57.996+0000 7f16b0c00700 1 -- 172.21.15.151:0/583175343 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f16a009d130 con 0x7f16b4109c80 2024-04-26T01:51:58.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:57.999+0000 7f16bb200700 1 -- 172.21.15.151:0/583175343 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f16980a3cc0 msgr2=0x7f16980a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:58.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:57.999+0000 7f16bb200700 1 --2- 172.21.15.151:0/583175343 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f16980a3cc0 0x7f16980a6180 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f16a4003760 tx=0x7f16a4005fb0 comp rx=0 tx=0).stop 2024-04-26T01:51:58.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:57.999+0000 7f16bb200700 1 -- 172.21.15.151:0/583175343 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f16b4109c80 msgr2=0x7f16b419fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:58.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:57.999+0000 7f16bb200700 1 --2- 172.21.15.151:0/583175343 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f16b4109c80 0x7f16b419fcb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f16a000e9f0 tx=0x7f16a000ed00 comp rx=0 tx=0).stop 2024-04-26T01:51:58.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:57.999+0000 7f16bb200700 1 -- 172.21.15.151:0/583175343 shutdown_connections 2024-04-26T01:51:58.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:57.999+0000 7f16bb200700 1 --2- 172.21.15.151:0/583175343 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16b410e3b0 0x7f16b41a3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:58.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:57.999+0000 7f16bb200700 1 --2- 172.21.15.151:0/583175343 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16b41092d0 0x7f16b419f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:58.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:57.999+0000 7f16bb200700 1 --2- 172.21.15.151:0/583175343 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f16980a3cc0 0x7f16980a6180 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:58.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:57.999+0000 7f16bb200700 1 --2- 172.21.15.151:0/583175343 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f16b4109c80 0x7f16b419fcb0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:58.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:57.999+0000 7f16bb200700 1 -- 172.21.15.151:0/583175343 >> 172.21.15.151:0/583175343 conn(0x7f16b4076210 msgr2=0x7f16b4077e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:58.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:58.000+0000 7f16bb200700 1 -- 172.21.15.151:0/583175343 shutdown_connections 2024-04-26T01:51:58.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:51:58.000+0000 7f16bb200700 1 -- 172.21.15.151:0/583175343 wait complete. 2024-04-26T01:51:58.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:51:57 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/475472456' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-26T01:51:58.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:51:57 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/475472456' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-26T01:51:58.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:51:57 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/475472456' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-26T01:51:59.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:51:58 smithi151 ceph-mon[28057]: pgmap v398: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T01:51:59.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:51:58 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/583175343' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-26T01:51:59.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:51:58 smithi172 ceph-mon[32785]: pgmap v398: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T01:51:59.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:51:58 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/583175343' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-26T01:51:59.341 INFO:tasks.cephadm.ceph_manager.ceph:need seq 317827579948 got 317827579953 for osd.10 2024-04-26T01:51:59.341 DEBUG:teuthology.parallel:result is None 2024-04-26T01:51:59.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:51:58 smithi165 ceph-mon[32648]: pgmap v398: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T01:51:59.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:51:58 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/583175343' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-26T01:52:00.221 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411418 got 77309411425 for osd.2 2024-04-26T01:52:00.221 DEBUG:teuthology.parallel:result is None 2024-04-26T01:52:00.221 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-26T01:52:00.222 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T01:52:00.675 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:52:01.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:52:00 smithi151 ceph-mon[28057]: pgmap v399: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:52:01.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:52:00 smithi172 ceph-mon[32785]: pgmap v399: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:52:01.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:52:00 smithi165 ceph-mon[32648]: pgmap v399: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:52:02.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:02.862+0000 7f21f7a00700 1 -- 172.21.15.151:0/1924588672 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21f010f440 msgr2=0x7f21f0111830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:02.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:02.862+0000 7f21f7a00700 1 --2- 172.21.15.151:0/1924588672 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21f010f440 0x7f21f0111830 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f21e800b0a0 tx=0x7f21e800b3b0 comp rx=0 tx=0).stop 2024-04-26T01:52:02.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:02.862+0000 7f21f7a00700 1 -- 172.21.15.151:0/1924588672 shutdown_connections 2024-04-26T01:52:02.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:02.862+0000 7f21f7a00700 1 --2- 172.21.15.151:0/1924588672 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21f010f440 0x7f21f0111830 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:02.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:02.862+0000 7f21f7a00700 1 --2- 172.21.15.151:0/1924588672 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f21f0074c70 0x7f21f0075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:02.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:02.862+0000 7f21f7a00700 1 --2- 172.21.15.151:0/1924588672 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f21f0075590 0x7f21f010ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:02.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:02.862+0000 7f21f7a00700 1 -- 172.21.15.151:0/1924588672 >> 172.21.15.151:0/1924588672 conn(0x7f21f0079ba0 msgr2=0x7f21f0079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:02.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:02.862+0000 7f21f7a00700 1 -- 172.21.15.151:0/1924588672 shutdown_connections 2024-04-26T01:52:02.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:02.862+0000 7f21f7a00700 1 -- 172.21.15.151:0/1924588672 wait complete. 2024-04-26T01:52:02.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:02.864+0000 7f21f7a00700 1 Processor -- start 2024-04-26T01:52:02.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:02.864+0000 7f21f7a00700 1 -- start start 2024-04-26T01:52:02.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:02.864+0000 7f21f7a00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21f0074c70 0x7f21f019b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:02.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:02.864+0000 7f21f7a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f21f0075590 0x7f21f019b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:02.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:02.864+0000 7f21f7a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f21f010f440 0x7f21f019fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:02.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:02.865+0000 7f21f7a00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f21f0114df0 con 0x7f21f010f440 2024-04-26T01:52:02.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:02.865+0000 7f21f7a00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f21f0114c70 con 0x7f21f0075590 2024-04-26T01:52:02.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:02.865+0000 7f21f7a00700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f21f0114f70 con 0x7f21f0074c70 2024-04-26T01:52:02.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:02.865+0000 7f21efe00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f21f0075590 0x7f21f019b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:02.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:02.865+0000 7f21f5000700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21f0074c70 0x7f21f019b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:02.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:02.865+0000 7f21f5000700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21f0074c70 0x7f21f019b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:59654/0 (socket says 172.21.15.151:59654) 2024-04-26T01:52:02.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:02.865+0000 7f21f5a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f21f010f440 0x7f21f019fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:02.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:02.865+0000 7f21efe00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f21f0075590 0x7f21f019b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:50886/0 (socket says 172.21.15.151:50886) 2024-04-26T01:52:02.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:02.865+0000 7f21f5000700 1 -- 172.21.15.151:0/666222173 learned_addr learned my addr 172.21.15.151:0/666222173 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:52:02.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:02.865+0000 7f21f5000700 1 -- 172.21.15.151:0/666222173 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f21f0075590 msgr2=0x7f21f019b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:02.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:02.865+0000 7f21f5000700 1 --2- 172.21.15.151:0/666222173 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f21f0075590 0x7f21f019b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:02.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:02.865+0000 7f21f5000700 1 -- 172.21.15.151:0/666222173 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f21f010f440 msgr2=0x7f21f019fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:02.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:02.865+0000 7f21f5000700 1 --2- 172.21.15.151:0/666222173 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f21f010f440 0x7f21f019fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:02.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:02.865+0000 7f21f5000700 1 -- 172.21.15.151:0/666222173 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21f01a0270 con 0x7f21f0074c70 2024-04-26T01:52:02.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:02.865+0000 7f21efe00700 1 --2- 172.21.15.151:0/666222173 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f21f0075590 0x7f21f019b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:52:02.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:02.865+0000 7f21f5a00700 1 --2- 172.21.15.151:0/666222173 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f21f010f440 0x7f21f019fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:52:02.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:02.866+0000 7f21f5000700 1 --2- 172.21.15.151:0/666222173 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21f0074c70 0x7f21f019b260 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f21e400e9c0 tx=0x7f21e400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:02.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:02.866+0000 7f21ed600700 1 -- 172.21.15.151:0/666222173 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e400cbd0 con 0x7f21f0074c70 2024-04-26T01:52:02.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:02.866+0000 7f21ed600700 1 -- 172.21.15.151:0/666222173 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21e400ee60 con 0x7f21f0074c70 2024-04-26T01:52:02.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:02.866+0000 7f21ed600700 1 -- 172.21.15.151:0/666222173 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e40055c0 con 0x7f21f0074c70 2024-04-26T01:52:02.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:02.866+0000 7f21f7a00700 1 -- 172.21.15.151:0/666222173 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21f01a0560 con 0x7f21f0074c70 2024-04-26T01:52:02.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:02.866+0000 7f21f7a00700 1 -- 172.21.15.151:0/666222173 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21f01a7de0 con 0x7f21f0074c70 2024-04-26T01:52:02.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:02.868+0000 7f21f7a00700 1 -- 172.21.15.151:0/666222173 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21f0069730 con 0x7f21f0074c70 2024-04-26T01:52:02.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:02.870+0000 7f21ed600700 1 -- 172.21.15.151:0/666222173 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f21e4004750 con 0x7f21f0074c70 2024-04-26T01:52:02.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:02.871+0000 7f21ed600700 1 --2- 172.21.15.151:0/666222173 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f21d80a3cc0 0x7f21d80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:02.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:02.872+0000 7f21ed600700 1 -- 172.21.15.151:0/666222173 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f21e40d05b0 con 0x7f21f0074c70 2024-04-26T01:52:02.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:02.872+0000 7f21efe00700 1 --2- 172.21.15.151:0/666222173 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f21d80a3cc0 0x7f21d80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:02.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:02.873+0000 7f21efe00700 1 --2- 172.21.15.151:0/666222173 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f21d80a3cc0 0x7f21d80a6180 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f21f0111100 tx=0x7f21dc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:02.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:02.876+0000 7f21ed600700 1 -- 172.21.15.151:0/666222173 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f21e409d010 con 0x7f21f0074c70 2024-04-26T01:52:03.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:52:02 smithi172 ceph-mon[32785]: pgmap v400: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:52:03.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:52:02 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:52:03.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:52:02 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:52:03.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:52:02 smithi165 ceph-mon[32648]: pgmap v400: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:52:03.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:52:02 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:52:03.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:52:02 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:52:03.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:03.103+0000 7f21f7a00700 1 -- 172.21.15.151:0/666222173 --> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f21f0062e80 con 0x7f21d80a3cc0 2024-04-26T01:52:03.102 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:52:02 smithi151 ceph-mon[28057]: pgmap v400: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:52:03.102 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:52:02 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:52:03.102 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:52:02 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:52:03.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:03.119+0000 7f21ed600700 1 -- 172.21.15.151:0/666222173 <== mgr.14152 v2:172.21.15.151:6800/2251256505 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258841 (secure 0 0 0) 0x7f21f0062e80 con 0x7f21d80a3cc0 2024-04-26T01:52:03.123 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:52:03.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:03.125+0000 7f21f7a00700 1 -- 172.21.15.151:0/666222173 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f21d80a3cc0 msgr2=0x7f21d80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:03.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:03.125+0000 7f21f7a00700 1 --2- 172.21.15.151:0/666222173 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f21d80a3cc0 0x7f21d80a6180 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f21f0111100 tx=0x7f21dc009260 comp rx=0 tx=0).stop 2024-04-26T01:52:03.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:03.125+0000 7f21f7a00700 1 -- 172.21.15.151:0/666222173 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21f0074c70 msgr2=0x7f21f019b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:03.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:03.125+0000 7f21f7a00700 1 --2- 172.21.15.151:0/666222173 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21f0074c70 0x7f21f019b260 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f21e400e9c0 tx=0x7f21e400ecd0 comp rx=0 tx=0).stop 2024-04-26T01:52:03.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:03.125+0000 7f21f7a00700 1 -- 172.21.15.151:0/666222173 shutdown_connections 2024-04-26T01:52:03.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:03.125+0000 7f21f7a00700 1 --2- 172.21.15.151:0/666222173 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f21d80a3cc0 0x7f21d80a6180 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:03.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:03.125+0000 7f21f7a00700 1 --2- 172.21.15.151:0/666222173 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f21f010f440 0x7f21f019fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:03.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:03.125+0000 7f21f7a00700 1 --2- 172.21.15.151:0/666222173 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21f0074c70 0x7f21f019b260 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:03.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:03.125+0000 7f21f7a00700 1 --2- 172.21.15.151:0/666222173 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f21f0075590 0x7f21f019b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:03.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:03.125+0000 7f21f7a00700 1 -- 172.21.15.151:0/666222173 >> 172.21.15.151:0/666222173 conn(0x7f21f0079ba0 msgr2=0x7f21f00ffbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:03.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:03.125+0000 7f21f7a00700 1 -- 172.21.15.151:0/666222173 shutdown_connections 2024-04-26T01:52:03.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:03.125+0000 7f21f7a00700 1 -- 172.21.15.151:0/666222173 wait complete. 2024-04-26T01:52:03.126 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-04-26T01:52:03.928 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":400,"stamp":"2024-04-26T01:52:01.440634+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655872,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":877,"num_read_kb":562,"num_write":356,"num_write_kb":3058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":786,"ondisk_log_size":786,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":387900,"kb_used_data":13788,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124553924,"statfs":{"total":1151940427776,"available":1151543218176,"internally_reserved":0,"allocated":14118912,"data_stored":7797192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":80,"num_read_kb":45,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005585"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.381128+0000","last_change":"2024-04-26T01:49:37.641666+0000","last_active":"2024-04-26T01:51:25.381128+0000","last_peered":"2024-04-26T01:51:25.381128+0000","last_clean":"2024-04-26T01:51:25.381128+0000","last_became_active":"2024-04-26T01:49:37.641310+0000","last_became_peered":"2024-04-26T01:49:37.641310+0000","last_unstale":"2024-04-26T01:51:25.381128+0000","last_undegraded":"2024-04-26T01:51:25.381128+0000","last_fullsized":"2024-04-26T01:51:25.381128+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:28:02.952851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.374761+0000","last_change":"2024-04-26T01:49:37.637607+0000","last_active":"2024-04-26T01:51:25.374761+0000","last_peered":"2024-04-26T01:51:25.374761+0000","last_clean":"2024-04-26T01:51:25.374761+0000","last_became_active":"2024-04-26T01:49:37.637283+0000","last_became_peered":"2024-04-26T01:49:37.637283+0000","last_unstale":"2024-04-26T01:51:25.374761+0000","last_undegraded":"2024-04-26T01:51:25.374761+0000","last_fullsized":"2024-04-26T01:51:25.374761+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:16:46.519381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.387398+0000","last_change":"2024-04-26T01:49:37.647769+0000","last_active":"2024-04-26T01:51:25.387398+0000","last_peered":"2024-04-26T01:51:25.387398+0000","last_clean":"2024-04-26T01:51:25.387398+0000","last_became_active":"2024-04-26T01:49:37.643750+0000","last_became_peered":"2024-04-26T01:49:37.643750+0000","last_unstale":"2024-04-26T01:51:25.387398+0000","last_undegraded":"2024-04-26T01:51:25.387398+0000","last_fullsized":"2024-04-26T01:51:25.387398+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:34:18.801267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.372634+0000","last_change":"2024-04-26T01:49:37.643291+0000","last_active":"2024-04-26T01:51:25.372634+0000","last_peered":"2024-04-26T01:51:25.372634+0000","last_clean":"2024-04-26T01:51:25.372634+0000","last_became_active":"2024-04-26T01:49:37.642947+0000","last_became_peered":"2024-04-26T01:49:37.642947+0000","last_unstale":"2024-04-26T01:51:25.372634+0000","last_undegraded":"2024-04-26T01:51:25.372634+0000","last_fullsized":"2024-04-26T01:51:25.372634+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:31:21.600533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.411232+0000","last_change":"2024-04-26T01:49:37.665311+0000","last_active":"2024-04-26T01:51:25.411232+0000","last_peered":"2024-04-26T01:51:25.411232+0000","last_clean":"2024-04-26T01:51:25.411232+0000","last_became_active":"2024-04-26T01:49:37.664944+0000","last_became_peered":"2024-04-26T01:49:37.664944+0000","last_unstale":"2024-04-26T01:51:25.411232+0000","last_undegraded":"2024-04-26T01:51:25.411232+0000","last_fullsized":"2024-04-26T01:51:25.411232+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:49:50.744630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.388269+0000","last_change":"2024-04-26T01:49:37.638667+0000","last_active":"2024-04-26T01:51:25.388269+0000","last_peered":"2024-04-26T01:51:25.388269+0000","last_clean":"2024-04-26T01:51:25.388269+0000","last_became_active":"2024-04-26T01:49:37.638226+0000","last_became_peered":"2024-04-26T01:49:37.638226+0000","last_unstale":"2024-04-26T01:51:25.388269+0000","last_undegraded":"2024-04-26T01:51:25.388269+0000","last_fullsized":"2024-04-26T01:51:25.388269+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:46:17.786883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.392209+0000","last_change":"2024-04-26T01:49:37.650082+0000","last_active":"2024-04-26T01:51:25.392209+0000","last_peered":"2024-04-26T01:51:25.392209+0000","last_clean":"2024-04-26T01:51:25.392209+0000","last_became_active":"2024-04-26T01:49:37.649637+0000","last_became_peered":"2024-04-26T01:49:37.649637+0000","last_unstale":"2024-04-26T01:51:25.392209+0000","last_undegraded":"2024-04-26T01:51:25.392209+0000","last_fullsized":"2024-04-26T01:51:25.392209+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:21:04.686398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.416839+0000","last_change":"2024-04-26T01:49:37.661255+0000","last_active":"2024-04-26T01:51:25.416839+0000","last_peered":"2024-04-26T01:51:25.416839+0000","last_clean":"2024-04-26T01:51:25.416839+0000","last_became_active":"2024-04-26T01:49:37.658882+0000","last_became_peered":"2024-04-26T01:49:37.658882+0000","last_unstale":"2024-04-26T01:51:25.416839+0000","last_undegraded":"2024-04-26T01:51:25.416839+0000","last_fullsized":"2024-04-26T01:51:25.416839+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:36:27.874191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.384359+0000","last_change":"2024-04-26T01:49:37.648874+0000","last_active":"2024-04-26T01:51:25.384359+0000","last_peered":"2024-04-26T01:51:25.384359+0000","last_clean":"2024-04-26T01:51:25.384359+0000","last_became_active":"2024-04-26T01:49:37.648490+0000","last_became_peered":"2024-04-26T01:49:37.648490+0000","last_unstale":"2024-04-26T01:51:25.384359+0000","last_undegraded":"2024-04-26T01:51:25.384359+0000","last_fullsized":"2024-04-26T01:51:25.384359+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:46:12.552861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.373500+0000","last_change":"2024-04-26T01:49:37.643842+0000","last_active":"2024-04-26T01:51:25.373500+0000","last_peered":"2024-04-26T01:51:25.373500+0000","last_clean":"2024-04-26T01:51:25.373500+0000","last_became_active":"2024-04-26T01:49:37.643084+0000","last_became_peered":"2024-04-26T01:49:37.643084+0000","last_unstale":"2024-04-26T01:51:25.373500+0000","last_undegraded":"2024-04-26T01:51:25.373500+0000","last_fullsized":"2024-04-26T01:51:25.373500+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:16:53.102318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.384294+0000","last_change":"2024-04-26T01:49:37.644209+0000","last_active":"2024-04-26T01:51:25.384294+0000","last_peered":"2024-04-26T01:51:25.384294+0000","last_clean":"2024-04-26T01:51:25.384294+0000","last_became_active":"2024-04-26T01:49:37.643659+0000","last_became_peered":"2024-04-26T01:49:37.643659+0000","last_unstale":"2024-04-26T01:51:25.384294+0000","last_undegraded":"2024-04-26T01:51:25.384294+0000","last_fullsized":"2024-04-26T01:51:25.384294+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:24:46.363873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.398749+0000","last_change":"2024-04-26T01:50:27.001615+0000","last_active":"2024-04-26T01:51:25.398749+0000","last_peered":"2024-04-26T01:51:25.398749+0000","last_clean":"2024-04-26T01:51:25.398749+0000","last_became_active":"2024-04-26T01:50:27.001039+0000","last_became_peered":"2024-04-26T01:50:27.001039+0000","last_unstale":"2024-04-26T01:51:25.398749+0000","last_undegraded":"2024-04-26T01:51:25.398749+0000","last_fullsized":"2024-04-26T01:51:25.398749+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:42:46.148500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.402097+0000","last_change":"2024-04-26T01:49:37.651085+0000","last_active":"2024-04-26T01:51:25.402097+0000","last_peered":"2024-04-26T01:51:25.402097+0000","last_clean":"2024-04-26T01:51:25.402097+0000","last_became_active":"2024-04-26T01:49:37.649662+0000","last_became_peered":"2024-04-26T01:49:37.649662+0000","last_unstale":"2024-04-26T01:51:25.402097+0000","last_undegraded":"2024-04-26T01:51:25.402097+0000","last_fullsized":"2024-04-26T01:51:25.402097+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:41:34.537533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.373700+0000","last_change":"2024-04-26T01:49:37.648432+0000","last_active":"2024-04-26T01:51:25.373700+0000","last_peered":"2024-04-26T01:51:25.373700+0000","last_clean":"2024-04-26T01:51:25.373700+0000","last_became_active":"2024-04-26T01:49:37.648047+0000","last_became_peered":"2024-04-26T01:49:37.648047+0000","last_unstale":"2024-04-26T01:51:25.373700+0000","last_undegraded":"2024-04-26T01:51:25.373700+0000","last_fullsized":"2024-04-26T01:51:25.373700+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:00:06.843384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.382303+0000","last_change":"2024-04-26T01:49:37.644203+0000","last_active":"2024-04-26T01:51:25.382303+0000","last_peered":"2024-04-26T01:51:25.382303+0000","last_clean":"2024-04-26T01:51:25.382303+0000","last_became_active":"2024-04-26T01:49:37.643772+0000","last_became_peered":"2024-04-26T01:49:37.643772+0000","last_unstale":"2024-04-26T01:51:25.382303+0000","last_undegraded":"2024-04-26T01:51:25.382303+0000","last_fullsized":"2024-04-26T01:51:25.382303+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:52:20.707845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.384590+0000","last_change":"2024-04-26T01:49:37.647364+0000","last_active":"2024-04-26T01:51:25.384590+0000","last_peered":"2024-04-26T01:51:25.384590+0000","last_clean":"2024-04-26T01:51:25.384590+0000","last_became_active":"2024-04-26T01:49:37.647054+0000","last_became_peered":"2024-04-26T01:49:37.647054+0000","last_unstale":"2024-04-26T01:51:25.384590+0000","last_undegraded":"2024-04-26T01:51:25.384590+0000","last_fullsized":"2024-04-26T01:51:25.384590+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:53:09.631463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.384289+0000","last_change":"2024-04-26T01:49:37.641843+0000","last_active":"2024-04-26T01:51:25.384289+0000","last_peered":"2024-04-26T01:51:25.384289+0000","last_clean":"2024-04-26T01:51:25.384289+0000","last_became_active":"2024-04-26T01:49:37.641514+0000","last_became_peered":"2024-04-26T01:49:37.641514+0000","last_unstale":"2024-04-26T01:51:25.384289+0000","last_undegraded":"2024-04-26T01:51:25.384289+0000","last_fullsized":"2024-04-26T01:51:25.384289+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:24:32.261834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.412843+0000","last_change":"2024-04-26T01:49:37.644750+0000","last_active":"2024-04-26T01:51:25.412843+0000","last_peered":"2024-04-26T01:51:25.412843+0000","last_clean":"2024-04-26T01:51:25.412843+0000","last_became_active":"2024-04-26T01:49:37.644086+0000","last_became_peered":"2024-04-26T01:49:37.644086+0000","last_unstale":"2024-04-26T01:51:25.412843+0000","last_undegraded":"2024-04-26T01:51:25.412843+0000","last_fullsized":"2024-04-26T01:51:25.412843+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:59:42.614846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.403738+0000","last_change":"2024-04-26T01:49:37.664204+0000","last_active":"2024-04-26T01:51:25.403738+0000","last_peered":"2024-04-26T01:51:25.403738+0000","last_clean":"2024-04-26T01:51:25.403738+0000","last_became_active":"2024-04-26T01:49:37.663878+0000","last_became_peered":"2024-04-26T01:49:37.663878+0000","last_unstale":"2024-04-26T01:51:25.403738+0000","last_undegraded":"2024-04-26T01:51:25.403738+0000","last_fullsized":"2024-04-26T01:51:25.403738+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:43:49.272308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.997103+0000","last_change":"2024-04-26T01:49:33.592017+0000","last_active":"2024-04-26T01:50:26.997103+0000","last_peered":"2024-04-26T01:50:26.997103+0000","last_clean":"2024-04-26T01:50:26.997103+0000","last_became_active":"2024-04-26T01:49:33.587928+0000","last_became_peered":"2024-04-26T01:49:33.587928+0000","last_unstale":"2024-04-26T01:50:26.997103+0000","last_undegraded":"2024-04-26T01:50:26.997103+0000","last_fullsized":"2024-04-26T01:50:26.997103+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:02:04.313785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.411761+0000","last_change":"2024-04-26T01:49:37.662010+0000","last_active":"2024-04-26T01:51:25.411761+0000","last_peered":"2024-04-26T01:51:25.411761+0000","last_clean":"2024-04-26T01:51:25.411761+0000","last_became_active":"2024-04-26T01:49:37.661673+0000","last_became_peered":"2024-04-26T01:49:37.661673+0000","last_unstale":"2024-04-26T01:51:25.411761+0000","last_undegraded":"2024-04-26T01:51:25.411761+0000","last_fullsized":"2024-04-26T01:51:25.411761+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:37:34.155998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.994246+0000","last_change":"2024-04-26T01:49:34.185758+0000","last_active":"2024-04-26T01:50:26.994246+0000","last_peered":"2024-04-26T01:50:26.994246+0000","last_clean":"2024-04-26T01:50:26.994246+0000","last_became_active":"2024-04-26T01:49:34.184669+0000","last_became_peered":"2024-04-26T01:49:34.184669+0000","last_unstale":"2024-04-26T01:50:26.994246+0000","last_undegraded":"2024-04-26T01:50:26.994246+0000","last_fullsized":"2024-04-26T01:50:26.994246+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:23:23.742032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.398961+0000","last_change":"2024-04-26T01:49:37.641307+0000","last_active":"2024-04-26T01:51:25.398961+0000","last_peered":"2024-04-26T01:51:25.398961+0000","last_clean":"2024-04-26T01:51:25.398961+0000","last_became_active":"2024-04-26T01:49:37.640677+0000","last_became_peered":"2024-04-26T01:49:37.640677+0000","last_unstale":"2024-04-26T01:51:25.398961+0000","last_undegraded":"2024-04-26T01:51:25.398961+0000","last_fullsized":"2024-04-26T01:51:25.398961+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:35:56.859045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.007585+0000","last_change":"2024-04-26T01:50:27.007034+0000","last_active":"2024-04-26T01:50:27.007585+0000","last_peered":"2024-04-26T01:50:27.007585+0000","last_clean":"2024-04-26T01:50:27.007585+0000","last_became_active":"2024-04-26T01:50:27.000945+0000","last_became_peered":"2024-04-26T01:50:27.000945+0000","last_unstale":"2024-04-26T01:50:27.007585+0000","last_undegraded":"2024-04-26T01:50:27.007585+0000","last_fullsized":"2024-04-26T01:50:27.007585+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:39:02.484719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.406751+0000","last_change":"2024-04-26T01:49:37.645936+0000","last_active":"2024-04-26T01:51:25.406751+0000","last_peered":"2024-04-26T01:51:25.406751+0000","last_clean":"2024-04-26T01:51:25.406751+0000","last_became_active":"2024-04-26T01:49:37.645535+0000","last_became_peered":"2024-04-26T01:49:37.645535+0000","last_unstale":"2024-04-26T01:51:25.406751+0000","last_undegraded":"2024-04-26T01:51:25.406751+0000","last_fullsized":"2024-04-26T01:51:25.406751+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:45:04.434415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"90'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.999301+0000","last_change":"2024-04-26T01:49:33.589904+0000","last_active":"2024-04-26T01:50:26.999301+0000","last_peered":"2024-04-26T01:50:26.999301+0000","last_clean":"2024-04-26T01:50:26.999301+0000","last_became_active":"2024-04-26T01:49:33.589332+0000","last_became_peered":"2024-04-26T01:49:33.589332+0000","last_unstale":"2024-04-26T01:50:26.999301+0000","last_undegraded":"2024-04-26T01:50:26.999301+0000","last_fullsized":"2024-04-26T01:50:26.999301+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:28:07.972583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.385634+0000","last_change":"2024-04-26T01:49:37.652093+0000","last_active":"2024-04-26T01:51:25.385634+0000","last_peered":"2024-04-26T01:51:25.385634+0000","last_clean":"2024-04-26T01:51:25.385634+0000","last_became_active":"2024-04-26T01:49:37.651506+0000","last_became_peered":"2024-04-26T01:49:37.651506+0000","last_unstale":"2024-04-26T01:51:25.385634+0000","last_undegraded":"2024-04-26T01:51:25.385634+0000","last_fullsized":"2024-04-26T01:51:25.385634+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:01:31.421498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"92'7","reported_seq":153,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:55.964614+0000","last_change":"2024-04-26T01:49:33.587652+0000","last_active":"2024-04-26T01:51:55.964614+0000","last_peered":"2024-04-26T01:51:55.964614+0000","last_clean":"2024-04-26T01:51:55.964614+0000","last_became_active":"2024-04-26T01:49:33.587294+0000","last_became_peered":"2024-04-26T01:49:33.587294+0000","last_unstale":"2024-04-26T01:51:55.964614+0000","last_undegraded":"2024-04-26T01:51:55.964614+0000","last_fullsized":"2024-04-26T01:51:55.964614+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:32:46.803415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.398412+0000","last_change":"2024-04-26T01:49:37.647899+0000","last_active":"2024-04-26T01:51:25.398412+0000","last_peered":"2024-04-26T01:51:25.398412+0000","last_clean":"2024-04-26T01:51:25.398412+0000","last_became_active":"2024-04-26T01:49:37.647420+0000","last_became_peered":"2024-04-26T01:49:37.647420+0000","last_unstale":"2024-04-26T01:51:25.398412+0000","last_undegraded":"2024-04-26T01:51:25.398412+0000","last_fullsized":"2024-04-26T01:51:25.398412+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:16:39.262842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.992761+0000","last_change":"2024-04-26T01:49:33.596096+0000","last_active":"2024-04-26T01:50:26.992761+0000","last_peered":"2024-04-26T01:50:26.992761+0000","last_clean":"2024-04-26T01:50:26.992761+0000","last_became_active":"2024-04-26T01:49:33.595604+0000","last_became_peered":"2024-04-26T01:49:33.595604+0000","last_unstale":"2024-04-26T01:50:26.992761+0000","last_undegraded":"2024-04-26T01:50:26.992761+0000","last_fullsized":"2024-04-26T01:50:26.992761+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:44:26.497448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.412127+0000","last_change":"2024-04-26T01:49:37.665001+0000","last_active":"2024-04-26T01:51:25.412127+0000","last_peered":"2024-04-26T01:51:25.412127+0000","last_clean":"2024-04-26T01:51:25.412127+0000","last_became_active":"2024-04-26T01:49:37.664598+0000","last_became_peered":"2024-04-26T01:49:37.664598+0000","last_unstale":"2024-04-26T01:51:25.412127+0000","last_undegraded":"2024-04-26T01:51:25.412127+0000","last_fullsized":"2024-04-26T01:51:25.412127+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:54:28.299606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.006003+0000","last_change":"2024-04-26T01:49:33.602593+0000","last_active":"2024-04-26T01:50:27.006003+0000","last_peered":"2024-04-26T01:50:27.006003+0000","last_clean":"2024-04-26T01:50:27.006003+0000","last_became_active":"2024-04-26T01:49:33.602223+0000","last_became_peered":"2024-04-26T01:49:33.602223+0000","last_unstale":"2024-04-26T01:50:27.006003+0000","last_undegraded":"2024-04-26T01:50:27.006003+0000","last_fullsized":"2024-04-26T01:50:27.006003+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:43:29.404719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.415197+0000","last_change":"2024-04-26T01:49:37.662351+0000","last_active":"2024-04-26T01:51:25.415197+0000","last_peered":"2024-04-26T01:51:25.415197+0000","last_clean":"2024-04-26T01:51:25.415197+0000","last_became_active":"2024-04-26T01:49:37.661900+0000","last_became_peered":"2024-04-26T01:49:37.661900+0000","last_unstale":"2024-04-26T01:51:25.415197+0000","last_undegraded":"2024-04-26T01:51:25.415197+0000","last_fullsized":"2024-04-26T01:51:25.415197+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:49:10.878580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.068414+0000","last_change":"2024-04-26T01:49:33.586538+0000","last_active":"2024-04-26T01:50:27.068414+0000","last_peered":"2024-04-26T01:50:27.068414+0000","last_clean":"2024-04-26T01:50:27.068414+0000","last_became_active":"2024-04-26T01:49:33.586127+0000","last_became_peered":"2024-04-26T01:49:33.586127+0000","last_unstale":"2024-04-26T01:50:27.068414+0000","last_undegraded":"2024-04-26T01:50:27.068414+0000","last_fullsized":"2024-04-26T01:50:27.068414+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:05.599479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.412509+0000","last_change":"2024-04-26T01:49:37.657154+0000","last_active":"2024-04-26T01:51:25.412509+0000","last_peered":"2024-04-26T01:51:25.412509+0000","last_clean":"2024-04-26T01:51:25.412509+0000","last_became_active":"2024-04-26T01:49:37.656814+0000","last_became_peered":"2024-04-26T01:49:37.656814+0000","last_unstale":"2024-04-26T01:51:25.412509+0000","last_undegraded":"2024-04-26T01:51:25.412509+0000","last_fullsized":"2024-04-26T01:51:25.412509+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:26:54.538267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"92'4","reported_seq":150,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:59.970024+0000","last_change":"2024-04-26T01:49:33.602441+0000","last_active":"2024-04-26T01:51:59.970024+0000","last_peered":"2024-04-26T01:51:59.970024+0000","last_clean":"2024-04-26T01:51:59.970024+0000","last_became_active":"2024-04-26T01:49:33.602116+0000","last_became_peered":"2024-04-26T01:49:33.602116+0000","last_unstale":"2024-04-26T01:51:59.970024+0000","last_undegraded":"2024-04-26T01:51:59.970024+0000","last_fullsized":"2024-04-26T01:51:59.970024+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:45:52.541043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":119,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.412565+0000","last_change":"2024-04-26T01:49:37.645921+0000","last_active":"2024-04-26T01:51:25.412565+0000","last_peered":"2024-04-26T01:51:25.412565+0000","last_clean":"2024-04-26T01:51:25.412565+0000","last_became_active":"2024-04-26T01:49:37.645590+0000","last_became_peered":"2024-04-26T01:49:37.645590+0000","last_unstale":"2024-04-26T01:51:25.412565+0000","last_undegraded":"2024-04-26T01:51:25.412565+0000","last_fullsized":"2024-04-26T01:51:25.412565+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:00:08.931051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998659+0000","last_change":"2024-04-26T01:49:33.589350+0000","last_active":"2024-04-26T01:50:26.998659+0000","last_peered":"2024-04-26T01:50:26.998659+0000","last_clean":"2024-04-26T01:50:26.998659+0000","last_became_active":"2024-04-26T01:49:33.588943+0000","last_became_peered":"2024-04-26T01:49:33.588943+0000","last_unstale":"2024-04-26T01:50:26.998659+0000","last_undegraded":"2024-04-26T01:50:26.998659+0000","last_fullsized":"2024-04-26T01:50:26.998659+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:36:36.104012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.383760+0000","last_change":"2024-04-26T01:49:37.648458+0000","last_active":"2024-04-26T01:51:25.383760+0000","last_peered":"2024-04-26T01:51:25.383760+0000","last_clean":"2024-04-26T01:51:25.383760+0000","last_became_active":"2024-04-26T01:49:37.648037+0000","last_became_peered":"2024-04-26T01:49:37.648037+0000","last_unstale":"2024-04-26T01:51:25.383760+0000","last_undegraded":"2024-04-26T01:51:25.383760+0000","last_fullsized":"2024-04-26T01:51:25.383760+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:28:05.711795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.069074+0000","last_change":"2024-04-26T01:49:34.189193+0000","last_active":"2024-04-26T01:50:27.069074+0000","last_peered":"2024-04-26T01:50:27.069074+0000","last_clean":"2024-04-26T01:50:27.069074+0000","last_became_active":"2024-04-26T01:49:34.188827+0000","last_became_peered":"2024-04-26T01:49:34.188827+0000","last_unstale":"2024-04-26T01:50:27.069074+0000","last_undegraded":"2024-04-26T01:50:27.069074+0000","last_fullsized":"2024-04-26T01:50:27.069074+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:02:04.583215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.384293+0000","last_change":"2024-04-26T01:49:37.644208+0000","last_active":"2024-04-26T01:51:25.384293+0000","last_peered":"2024-04-26T01:51:25.384293+0000","last_clean":"2024-04-26T01:51:25.384293+0000","last_became_active":"2024-04-26T01:49:37.643787+0000","last_became_peered":"2024-04-26T01:49:37.643787+0000","last_unstale":"2024-04-26T01:51:25.384293+0000","last_undegraded":"2024-04-26T01:51:25.384293+0000","last_fullsized":"2024-04-26T01:51:25.384293+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:42:21.243628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"92'31","reported_seq":306,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:59.970180+0000","last_change":"2024-04-26T01:49:33.590965+0000","last_active":"2024-04-26T01:51:59.970180+0000","last_peered":"2024-04-26T01:51:59.970180+0000","last_clean":"2024-04-26T01:51:59.970180+0000","last_became_active":"2024-04-26T01:49:33.590543+0000","last_became_peered":"2024-04-26T01:49:33.590543+0000","last_unstale":"2024-04-26T01:51:59.970180+0000","last_undegraded":"2024-04-26T01:51:59.970180+0000","last_fullsized":"2024-04-26T01:51:59.970180+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:26.627549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60596,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":248,"num_read_kb":183,"num_write":32,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.376407+0000","last_change":"2024-04-26T01:49:37.641172+0000","last_active":"2024-04-26T01:51:25.376407+0000","last_peered":"2024-04-26T01:51:25.376407+0000","last_clean":"2024-04-26T01:51:25.376407+0000","last_became_active":"2024-04-26T01:49:37.640866+0000","last_became_peered":"2024-04-26T01:49:37.640866+0000","last_unstale":"2024-04-26T01:51:25.376407+0000","last_undegraded":"2024-04-26T01:51:25.376407+0000","last_fullsized":"2024-04-26T01:51:25.376407+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:29:12.461005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.999186+0000","last_change":"2024-04-26T01:49:33.598466+0000","last_active":"2024-04-26T01:50:26.999186+0000","last_peered":"2024-04-26T01:50:26.999186+0000","last_clean":"2024-04-26T01:50:26.999186+0000","last_became_active":"2024-04-26T01:49:33.598042+0000","last_became_peered":"2024-04-26T01:49:33.598042+0000","last_unstale":"2024-04-26T01:50:26.999186+0000","last_undegraded":"2024-04-26T01:50:26.999186+0000","last_fullsized":"2024-04-26T01:50:26.999186+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:55:54.901016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.412517+0000","last_change":"2024-04-26T01:49:37.665772+0000","last_active":"2024-04-26T01:51:25.412517+0000","last_peered":"2024-04-26T01:51:25.412517+0000","last_clean":"2024-04-26T01:51:25.412517+0000","last_became_active":"2024-04-26T01:49:37.665524+0000","last_became_peered":"2024-04-26T01:49:37.665524+0000","last_unstale":"2024-04-26T01:51:25.412517+0000","last_undegraded":"2024-04-26T01:51:25.412517+0000","last_fullsized":"2024-04-26T01:51:25.412517+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:00:19.729098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.068644+0000","last_change":"2024-04-26T01:49:33.599456+0000","last_active":"2024-04-26T01:50:27.068644+0000","last_peered":"2024-04-26T01:50:27.068644+0000","last_clean":"2024-04-26T01:50:27.068644+0000","last_became_active":"2024-04-26T01:49:33.599101+0000","last_became_peered":"2024-04-26T01:49:33.599101+0000","last_unstale":"2024-04-26T01:50:27.068644+0000","last_undegraded":"2024-04-26T01:50:27.068644+0000","last_fullsized":"2024-04-26T01:50:27.068644+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:14:35.923011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.400197+0000","last_change":"2024-04-26T01:49:37.647193+0000","last_active":"2024-04-26T01:51:25.400197+0000","last_peered":"2024-04-26T01:51:25.400197+0000","last_clean":"2024-04-26T01:51:25.400197+0000","last_became_active":"2024-04-26T01:49:37.646881+0000","last_became_peered":"2024-04-26T01:49:37.646881+0000","last_unstale":"2024-04-26T01:51:25.400197+0000","last_undegraded":"2024-04-26T01:51:25.400197+0000","last_fullsized":"2024-04-26T01:51:25.400197+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:50:10.006986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"92'5","reported_seq":153,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:57.025972+0000","last_change":"2024-04-26T01:49:33.590398+0000","last_active":"2024-04-26T01:51:57.025972+0000","last_peered":"2024-04-26T01:51:57.025972+0000","last_clean":"2024-04-26T01:51:57.025972+0000","last_became_active":"2024-04-26T01:49:33.590033+0000","last_became_peered":"2024-04-26T01:49:33.590033+0000","last_unstale":"2024-04-26T01:51:57.025972+0000","last_undegraded":"2024-04-26T01:51:57.025972+0000","last_fullsized":"2024-04-26T01:51:57.025972+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:27:43.203781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.377744+0000","last_change":"2024-04-26T01:49:37.649784+0000","last_active":"2024-04-26T01:51:25.377744+0000","last_peered":"2024-04-26T01:51:25.377744+0000","last_clean":"2024-04-26T01:51:25.377744+0000","last_became_active":"2024-04-26T01:49:37.649431+0000","last_became_peered":"2024-04-26T01:49:37.649431+0000","last_unstale":"2024-04-26T01:51:25.377744+0000","last_undegraded":"2024-04-26T01:51:25.377744+0000","last_fullsized":"2024-04-26T01:51:25.377744+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:45:19.045983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998248+0000","last_change":"2024-04-26T01:49:34.182987+0000","last_active":"2024-04-26T01:50:26.998248+0000","last_peered":"2024-04-26T01:50:26.998248+0000","last_clean":"2024-04-26T01:50:26.998248+0000","last_became_active":"2024-04-26T01:49:34.182600+0000","last_became_peered":"2024-04-26T01:49:34.182600+0000","last_unstale":"2024-04-26T01:50:26.998248+0000","last_undegraded":"2024-04-26T01:50:26.998248+0000","last_fullsized":"2024-04-26T01:50:26.998248+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:05:52.774170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.384199+0000","last_change":"2024-04-26T01:49:37.646097+0000","last_active":"2024-04-26T01:51:25.384199+0000","last_peered":"2024-04-26T01:51:25.384199+0000","last_clean":"2024-04-26T01:51:25.384199+0000","last_became_active":"2024-04-26T01:49:37.645763+0000","last_became_peered":"2024-04-26T01:49:37.645763+0000","last_unstale":"2024-04-26T01:51:25.384199+0000","last_undegraded":"2024-04-26T01:51:25.384199+0000","last_fullsized":"2024-04-26T01:51:25.384199+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:42:37.159950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.999051+0000","last_change":"2024-04-26T01:49:33.591457+0000","last_active":"2024-04-26T01:50:26.999051+0000","last_peered":"2024-04-26T01:50:26.999051+0000","last_clean":"2024-04-26T01:50:26.999051+0000","last_became_active":"2024-04-26T01:49:33.590655+0000","last_became_peered":"2024-04-26T01:49:33.590655+0000","last_unstale":"2024-04-26T01:50:26.999051+0000","last_undegraded":"2024-04-26T01:50:26.999051+0000","last_fullsized":"2024-04-26T01:50:26.999051+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:33:18.803796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.380099+0000","last_change":"2024-04-26T01:49:37.646423+0000","last_active":"2024-04-26T01:51:25.380099+0000","last_peered":"2024-04-26T01:51:25.380099+0000","last_clean":"2024-04-26T01:51:25.380099+0000","last_became_active":"2024-04-26T01:49:37.646046+0000","last_became_peered":"2024-04-26T01:49:37.646046+0000","last_unstale":"2024-04-26T01:51:25.380099+0000","last_undegraded":"2024-04-26T01:51:25.380099+0000","last_fullsized":"2024-04-26T01:51:25.380099+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:26:03.570479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.997386+0000","last_change":"2024-04-26T01:49:34.192848+0000","last_active":"2024-04-26T01:50:26.997386+0000","last_peered":"2024-04-26T01:50:26.997386+0000","last_clean":"2024-04-26T01:50:26.997386+0000","last_became_active":"2024-04-26T01:49:34.190449+0000","last_became_peered":"2024-04-26T01:49:34.190449+0000","last_unstale":"2024-04-26T01:50:26.997386+0000","last_undegraded":"2024-04-26T01:50:26.997386+0000","last_fullsized":"2024-04-26T01:50:26.997386+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:05:22.094991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.402950+0000","last_change":"2024-04-26T01:49:37.667147+0000","last_active":"2024-04-26T01:51:25.402950+0000","last_peered":"2024-04-26T01:51:25.402950+0000","last_clean":"2024-04-26T01:51:25.402950+0000","last_became_active":"2024-04-26T01:49:37.666752+0000","last_became_peered":"2024-04-26T01:49:37.666752+0000","last_unstale":"2024-04-26T01:51:25.402950+0000","last_undegraded":"2024-04-26T01:51:25.402950+0000","last_fullsized":"2024-04-26T01:51:25.402950+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:18:31.778916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998501+0000","last_change":"2024-04-26T01:49:33.596155+0000","last_active":"2024-04-26T01:50:26.998501+0000","last_peered":"2024-04-26T01:50:26.998501+0000","last_clean":"2024-04-26T01:50:26.998501+0000","last_became_active":"2024-04-26T01:49:33.595760+0000","last_became_peered":"2024-04-26T01:49:33.595760+0000","last_unstale":"2024-04-26T01:50:26.998501+0000","last_undegraded":"2024-04-26T01:50:26.998501+0000","last_fullsized":"2024-04-26T01:50:26.998501+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:07:57.208116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.375222+0000","last_change":"2024-04-26T01:49:37.641660+0000","last_active":"2024-04-26T01:51:25.375222+0000","last_peered":"2024-04-26T01:51:25.375222+0000","last_clean":"2024-04-26T01:51:25.375222+0000","last_became_active":"2024-04-26T01:49:37.641302+0000","last_became_peered":"2024-04-26T01:49:37.641302+0000","last_unstale":"2024-04-26T01:51:25.375222+0000","last_undegraded":"2024-04-26T01:51:25.375222+0000","last_fullsized":"2024-04-26T01:51:25.375222+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:32:59.889189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.380794+0000","last_change":"2024-04-26T01:49:37.643083+0000","last_active":"2024-04-26T01:51:25.380794+0000","last_peered":"2024-04-26T01:51:25.380794+0000","last_clean":"2024-04-26T01:51:25.380794+0000","last_became_active":"2024-04-26T01:49:37.642548+0000","last_became_peered":"2024-04-26T01:49:37.642548+0000","last_unstale":"2024-04-26T01:51:25.380794+0000","last_undegraded":"2024-04-26T01:51:25.380794+0000","last_fullsized":"2024-04-26T01:51:25.380794+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:09:06.136510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'5","reported_seq":151,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:59.029912+0000","last_change":"2024-04-26T01:49:34.185069+0000","last_active":"2024-04-26T01:51:59.029912+0000","last_peered":"2024-04-26T01:51:59.029912+0000","last_clean":"2024-04-26T01:51:59.029912+0000","last_became_active":"2024-04-26T01:49:34.183989+0000","last_became_peered":"2024-04-26T01:49:34.183989+0000","last_unstale":"2024-04-26T01:51:59.029912+0000","last_undegraded":"2024-04-26T01:51:59.029912+0000","last_fullsized":"2024-04-26T01:51:59.029912+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:04:26.327960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":119,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.408158+0000","last_change":"2024-04-26T01:49:37.655512+0000","last_active":"2024-04-26T01:51:25.408158+0000","last_peered":"2024-04-26T01:51:25.408158+0000","last_clean":"2024-04-26T01:51:25.408158+0000","last_became_active":"2024-04-26T01:49:37.655142+0000","last_became_peered":"2024-04-26T01:49:37.655142+0000","last_unstale":"2024-04-26T01:51:25.408158+0000","last_undegraded":"2024-04-26T01:51:25.408158+0000","last_fullsized":"2024-04-26T01:51:25.408158+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:02:58.090885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.485822+0000","last_change":"2024-04-26T01:49:33.599578+0000","last_active":"2024-04-26T01:50:27.485822+0000","last_peered":"2024-04-26T01:50:27.485822+0000","last_clean":"2024-04-26T01:50:27.485822+0000","last_became_active":"2024-04-26T01:49:33.599010+0000","last_became_peered":"2024-04-26T01:49:33.599010+0000","last_unstale":"2024-04-26T01:50:27.485822+0000","last_undegraded":"2024-04-26T01:50:27.485822+0000","last_fullsized":"2024-04-26T01:50:27.485822+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:16:09.925304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":360,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998346+0000","last_change":"2024-04-26T01:47:59.584819+0000","last_active":"2024-04-26T01:50:26.998346+0000","last_peered":"2024-04-26T01:50:26.998346+0000","last_clean":"2024-04-26T01:50:26.998346+0000","last_became_active":"2024-04-26T01:47:59.584356+0000","last_became_peered":"2024-04-26T01:47:59.584356+0000","last_unstale":"2024-04-26T01:50:26.998346+0000","last_undegraded":"2024-04-26T01:50:26.998346+0000","last_fullsized":"2024-04-26T01:50:26.998346+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:44:11.653568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:44:11.653568+0000","last_clean_scrub_stamp":"2024-04-26T01:44:11.653568+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:25:11.168583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.403915+0000","last_change":"2024-04-26T01:50:27.008911+0000","last_active":"2024-04-26T01:51:25.403915+0000","last_peered":"2024-04-26T01:51:25.403915+0000","last_clean":"2024-04-26T01:51:25.403915+0000","last_became_active":"2024-04-26T01:50:27.008507+0000","last_became_peered":"2024-04-26T01:50:27.008507+0000","last_unstale":"2024-04-26T01:51:25.403915+0000","last_undegraded":"2024-04-26T01:51:25.403915+0000","last_fullsized":"2024-04-26T01:51:25.403915+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:21:14.985935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998684+0000","last_change":"2024-04-26T01:49:33.590171+0000","last_active":"2024-04-26T01:50:26.998684+0000","last_peered":"2024-04-26T01:50:26.998684+0000","last_clean":"2024-04-26T01:50:26.998684+0000","last_became_active":"2024-04-26T01:49:33.589312+0000","last_became_peered":"2024-04-26T01:49:33.589312+0000","last_unstale":"2024-04-26T01:50:26.998684+0000","last_undegraded":"2024-04-26T01:50:26.998684+0000","last_fullsized":"2024-04-26T01:50:26.998684+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:01:25.441527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998802+0000","last_change":"2024-04-26T01:49:34.190338+0000","last_active":"2024-04-26T01:50:26.998802+0000","last_peered":"2024-04-26T01:50:26.998802+0000","last_clean":"2024-04-26T01:50:26.998802+0000","last_became_active":"2024-04-26T01:49:34.189984+0000","last_became_peered":"2024-04-26T01:49:34.189984+0000","last_unstale":"2024-04-26T01:50:26.998802+0000","last_undegraded":"2024-04-26T01:50:26.998802+0000","last_fullsized":"2024-04-26T01:50:26.998802+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:58.588012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.406676+0000","last_change":"2024-04-26T01:49:37.649368+0000","last_active":"2024-04-26T01:51:25.406676+0000","last_peered":"2024-04-26T01:51:25.406676+0000","last_clean":"2024-04-26T01:51:25.406676+0000","last_became_active":"2024-04-26T01:49:37.648987+0000","last_became_peered":"2024-04-26T01:49:37.648987+0000","last_unstale":"2024-04-26T01:51:25.406676+0000","last_undegraded":"2024-04-26T01:51:25.406676+0000","last_fullsized":"2024-04-26T01:51:25.406676+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:04:59.080703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"92'12","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.696507+0000","last_change":"2024-04-26T01:49:33.600813+0000","last_active":"2024-04-26T01:51:25.696507+0000","last_peered":"2024-04-26T01:51:25.696507+0000","last_clean":"2024-04-26T01:51:25.696507+0000","last_became_active":"2024-04-26T01:49:33.600361+0000","last_became_peered":"2024-04-26T01:49:33.600361+0000","last_unstale":"2024-04-26T01:51:25.696507+0000","last_undegraded":"2024-04-26T01:51:25.696507+0000","last_fullsized":"2024-04-26T01:51:25.696507+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:02:58.638449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.414301+0000","last_change":"2024-04-26T01:49:37.659381+0000","last_active":"2024-04-26T01:51:25.414301+0000","last_peered":"2024-04-26T01:51:25.414301+0000","last_clean":"2024-04-26T01:51:25.414301+0000","last_became_active":"2024-04-26T01:49:37.656205+0000","last_became_peered":"2024-04-26T01:49:37.656205+0000","last_unstale":"2024-04-26T01:51:25.414301+0000","last_undegraded":"2024-04-26T01:51:25.414301+0000","last_fullsized":"2024-04-26T01:51:25.414301+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:02:10.103330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.486077+0000","last_change":"2024-04-26T01:49:33.854732+0000","last_active":"2024-04-26T01:50:27.486077+0000","last_peered":"2024-04-26T01:50:27.486077+0000","last_clean":"2024-04-26T01:50:27.486077+0000","last_became_active":"2024-04-26T01:49:33.854396+0000","last_became_peered":"2024-04-26T01:49:33.854396+0000","last_unstale":"2024-04-26T01:50:27.486077+0000","last_undegraded":"2024-04-26T01:50:27.486077+0000","last_fullsized":"2024-04-26T01:50:27.486077+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:12:37.183725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.403468+0000","last_change":"2024-04-26T01:49:37.656366+0000","last_active":"2024-04-26T01:51:25.403468+0000","last_peered":"2024-04-26T01:51:25.403468+0000","last_clean":"2024-04-26T01:51:25.403468+0000","last_became_active":"2024-04-26T01:49:37.655869+0000","last_became_peered":"2024-04-26T01:49:37.655869+0000","last_unstale":"2024-04-26T01:51:25.403468+0000","last_undegraded":"2024-04-26T01:51:25.403468+0000","last_fullsized":"2024-04-26T01:51:25.403468+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:01:57.083315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.837207+0000","last_change":"2024-04-26T01:49:34.186070+0000","last_active":"2024-04-26T01:50:27.837207+0000","last_peered":"2024-04-26T01:50:27.837207+0000","last_clean":"2024-04-26T01:50:27.837207+0000","last_became_active":"2024-04-26T01:49:34.185742+0000","last_became_peered":"2024-04-26T01:49:34.185742+0000","last_unstale":"2024-04-26T01:50:27.837207+0000","last_undegraded":"2024-04-26T01:50:27.837207+0000","last_fullsized":"2024-04-26T01:50:27.837207+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:05.870056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.376526+0000","last_change":"2024-04-26T01:49:37.640938+0000","last_active":"2024-04-26T01:51:25.376526+0000","last_peered":"2024-04-26T01:51:25.376526+0000","last_clean":"2024-04-26T01:51:25.376526+0000","last_became_active":"2024-04-26T01:49:37.640608+0000","last_became_peered":"2024-04-26T01:49:37.640608+0000","last_unstale":"2024-04-26T01:51:25.376526+0000","last_undegraded":"2024-04-26T01:51:25.376526+0000","last_fullsized":"2024-04-26T01:51:25.376526+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:30:30.460641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.991984+0000","last_change":"2024-04-26T01:49:33.604152+0000","last_active":"2024-04-26T01:50:26.991984+0000","last_peered":"2024-04-26T01:50:26.991984+0000","last_clean":"2024-04-26T01:50:26.991984+0000","last_became_active":"2024-04-26T01:49:33.603479+0000","last_became_peered":"2024-04-26T01:49:33.603479+0000","last_unstale":"2024-04-26T01:50:26.991984+0000","last_undegraded":"2024-04-26T01:50:26.991984+0000","last_fullsized":"2024-04-26T01:50:26.991984+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:25:13.685392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.379393+0000","last_change":"2024-04-26T01:49:37.645152+0000","last_active":"2024-04-26T01:51:25.379393+0000","last_peered":"2024-04-26T01:51:25.379393+0000","last_clean":"2024-04-26T01:51:25.379393+0000","last_became_active":"2024-04-26T01:49:37.644789+0000","last_became_peered":"2024-04-26T01:49:37.644789+0000","last_unstale":"2024-04-26T01:51:25.379393+0000","last_undegraded":"2024-04-26T01:51:25.379393+0000","last_fullsized":"2024-04-26T01:51:25.379393+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:20:01.494222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.010485+0000","last_change":"2024-04-26T01:50:27.009973+0000","last_active":"2024-04-26T01:50:27.010485+0000","last_peered":"2024-04-26T01:50:27.010485+0000","last_clean":"2024-04-26T01:50:27.010485+0000","last_became_active":"2024-04-26T01:50:27.001678+0000","last_became_peered":"2024-04-26T01:50:27.001678+0000","last_unstale":"2024-04-26T01:50:27.010485+0000","last_undegraded":"2024-04-26T01:50:27.010485+0000","last_fullsized":"2024-04-26T01:50:27.010485+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:59:54.807792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.413791+0000","last_change":"2024-04-26T01:49:37.662665+0000","last_active":"2024-04-26T01:51:25.413791+0000","last_peered":"2024-04-26T01:51:25.413791+0000","last_clean":"2024-04-26T01:51:25.413791+0000","last_became_active":"2024-04-26T01:49:37.662216+0000","last_became_peered":"2024-04-26T01:49:37.662216+0000","last_unstale":"2024-04-26T01:51:25.413791+0000","last_undegraded":"2024-04-26T01:51:25.413791+0000","last_fullsized":"2024-04-26T01:51:25.413791+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:10:54.724858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998379+0000","last_change":"2024-04-26T01:49:33.592078+0000","last_active":"2024-04-26T01:50:26.998379+0000","last_peered":"2024-04-26T01:50:26.998379+0000","last_clean":"2024-04-26T01:50:26.998379+0000","last_became_active":"2024-04-26T01:49:33.590265+0000","last_became_peered":"2024-04-26T01:49:33.590265+0000","last_unstale":"2024-04-26T01:50:26.998379+0000","last_undegraded":"2024-04-26T01:50:26.998379+0000","last_fullsized":"2024-04-26T01:50:26.998379+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:44:27.163705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.392973+0000","last_change":"2024-04-26T01:49:37.645943+0000","last_active":"2024-04-26T01:51:25.392973+0000","last_peered":"2024-04-26T01:51:25.392973+0000","last_clean":"2024-04-26T01:51:25.392973+0000","last_became_active":"2024-04-26T01:49:37.645586+0000","last_became_peered":"2024-04-26T01:49:37.645586+0000","last_unstale":"2024-04-26T01:51:25.392973+0000","last_undegraded":"2024-04-26T01:51:25.392973+0000","last_fullsized":"2024-04-26T01:51:25.392973+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:40.682344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.000264+0000","last_change":"2024-04-26T01:49:34.187115+0000","last_active":"2024-04-26T01:50:27.000264+0000","last_peered":"2024-04-26T01:50:27.000264+0000","last_clean":"2024-04-26T01:50:27.000264+0000","last_became_active":"2024-04-26T01:49:34.186660+0000","last_became_peered":"2024-04-26T01:49:34.186660+0000","last_unstale":"2024-04-26T01:50:27.000264+0000","last_undegraded":"2024-04-26T01:50:27.000264+0000","last_fullsized":"2024-04-26T01:50:27.000264+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:56:32.274752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.384545+0000","last_change":"2024-04-26T01:49:37.650382+0000","last_active":"2024-04-26T01:51:25.384545+0000","last_peered":"2024-04-26T01:51:25.384545+0000","last_clean":"2024-04-26T01:51:25.384545+0000","last_became_active":"2024-04-26T01:49:37.649545+0000","last_became_peered":"2024-04-26T01:49:37.649545+0000","last_unstale":"2024-04-26T01:51:25.384545+0000","last_undegraded":"2024-04-26T01:51:25.384545+0000","last_fullsized":"2024-04-26T01:51:25.384545+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:43:16.291311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'1","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.069019+0000","last_change":"2024-04-26T01:49:33.595881+0000","last_active":"2024-04-26T01:50:27.069019+0000","last_peered":"2024-04-26T01:50:27.069019+0000","last_clean":"2024-04-26T01:50:27.069019+0000","last_became_active":"2024-04-26T01:49:33.595480+0000","last_became_peered":"2024-04-26T01:49:33.595480+0000","last_unstale":"2024-04-26T01:50:27.069019+0000","last_undegraded":"2024-04-26T01:50:27.069019+0000","last_fullsized":"2024-04-26T01:50:27.069019+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:30:58.028230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.403827+0000","last_change":"2024-04-26T01:49:37.660765+0000","last_active":"2024-04-26T01:51:25.403827+0000","last_peered":"2024-04-26T01:51:25.403827+0000","last_clean":"2024-04-26T01:51:25.403827+0000","last_became_active":"2024-04-26T01:49:37.660475+0000","last_became_peered":"2024-04-26T01:49:37.660475+0000","last_unstale":"2024-04-26T01:51:25.403827+0000","last_undegraded":"2024-04-26T01:51:25.403827+0000","last_fullsized":"2024-04-26T01:51:25.403827+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:36:00.483005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.006246+0000","last_change":"2024-04-26T01:49:33.598247+0000","last_active":"2024-04-26T01:50:27.006246+0000","last_peered":"2024-04-26T01:50:27.006246+0000","last_clean":"2024-04-26T01:50:27.006246+0000","last_became_active":"2024-04-26T01:49:33.597796+0000","last_became_peered":"2024-04-26T01:49:33.597796+0000","last_unstale":"2024-04-26T01:50:27.006246+0000","last_undegraded":"2024-04-26T01:50:27.006246+0000","last_fullsized":"2024-04-26T01:50:27.006246+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:53:38.994798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.380230+0000","last_change":"2024-04-26T01:49:37.648903+0000","last_active":"2024-04-26T01:51:25.380230+0000","last_peered":"2024-04-26T01:51:25.380230+0000","last_clean":"2024-04-26T01:51:25.380230+0000","last_became_active":"2024-04-26T01:49:37.648570+0000","last_became_peered":"2024-04-26T01:49:37.648570+0000","last_unstale":"2024-04-26T01:51:25.380230+0000","last_undegraded":"2024-04-26T01:51:25.380230+0000","last_fullsized":"2024-04-26T01:51:25.380230+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:33:12.678513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.379320+0000","last_change":"2024-04-26T01:49:37.648485+0000","last_active":"2024-04-26T01:51:25.379320+0000","last_peered":"2024-04-26T01:51:25.379320+0000","last_clean":"2024-04-26T01:51:25.379320+0000","last_became_active":"2024-04-26T01:49:37.648129+0000","last_became_peered":"2024-04-26T01:49:37.648129+0000","last_unstale":"2024-04-26T01:51:25.379320+0000","last_undegraded":"2024-04-26T01:51:25.379320+0000","last_fullsized":"2024-04-26T01:51:25.379320+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:56:12.047821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.376376+0000","last_change":"2024-04-26T01:49:37.645929+0000","last_active":"2024-04-26T01:51:25.376376+0000","last_peered":"2024-04-26T01:51:25.376376+0000","last_clean":"2024-04-26T01:51:25.376376+0000","last_became_active":"2024-04-26T01:49:37.645473+0000","last_became_peered":"2024-04-26T01:49:37.645473+0000","last_unstale":"2024-04-26T01:51:25.376376+0000","last_undegraded":"2024-04-26T01:51:25.376376+0000","last_fullsized":"2024-04-26T01:51:25.376376+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:39:56.687973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.403322+0000","last_change":"2024-04-26T01:49:37.657494+0000","last_active":"2024-04-26T01:51:25.403322+0000","last_peered":"2024-04-26T01:51:25.403322+0000","last_clean":"2024-04-26T01:51:25.403322+0000","last_became_active":"2024-04-26T01:49:37.657121+0000","last_became_peered":"2024-04-26T01:49:37.657121+0000","last_unstale":"2024-04-26T01:51:25.403322+0000","last_undegraded":"2024-04-26T01:51:25.403322+0000","last_fullsized":"2024-04-26T01:51:25.403322+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:50:44.691179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.399871+0000","last_change":"2024-04-26T01:49:37.646012+0000","last_active":"2024-04-26T01:51:25.399871+0000","last_peered":"2024-04-26T01:51:25.399871+0000","last_clean":"2024-04-26T01:51:25.399871+0000","last_became_active":"2024-04-26T01:49:37.645695+0000","last_became_peered":"2024-04-26T01:49:37.645695+0000","last_unstale":"2024-04-26T01:51:25.399871+0000","last_undegraded":"2024-04-26T01:51:25.399871+0000","last_fullsized":"2024-04-26T01:51:25.399871+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:40:51.117523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.402038+0000","last_change":"2024-04-26T01:49:37.646503+0000","last_active":"2024-04-26T01:51:25.402038+0000","last_peered":"2024-04-26T01:51:25.402038+0000","last_clean":"2024-04-26T01:51:25.402038+0000","last_became_active":"2024-04-26T01:49:37.645200+0000","last_became_peered":"2024-04-26T01:49:37.645200+0000","last_unstale":"2024-04-26T01:51:25.402038+0000","last_undegraded":"2024-04-26T01:51:25.402038+0000","last_fullsized":"2024-04-26T01:51:25.402038+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:31:10.957199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.397650+0000","last_change":"2024-04-26T01:50:27.002908+0000","last_active":"2024-04-26T01:51:25.397650+0000","last_peered":"2024-04-26T01:51:25.397650+0000","last_clean":"2024-04-26T01:51:25.397650+0000","last_became_active":"2024-04-26T01:50:27.002454+0000","last_became_peered":"2024-04-26T01:50:27.002454+0000","last_unstale":"2024-04-26T01:51:25.397650+0000","last_undegraded":"2024-04-26T01:51:25.397650+0000","last_fullsized":"2024-04-26T01:51:25.397650+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:18:01.848427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.383409+0000","last_change":"2024-04-26T01:49:37.642676+0000","last_active":"2024-04-26T01:51:25.383409+0000","last_peered":"2024-04-26T01:51:25.383409+0000","last_clean":"2024-04-26T01:51:25.383409+0000","last_became_active":"2024-04-26T01:49:37.642302+0000","last_became_peered":"2024-04-26T01:49:37.642302+0000","last_unstale":"2024-04-26T01:51:25.383409+0000","last_undegraded":"2024-04-26T01:51:25.383409+0000","last_fullsized":"2024-04-26T01:51:25.383409+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:51:43.454649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.413280+0000","last_change":"2024-04-26T01:50:27.003914+0000","last_active":"2024-04-26T01:51:25.413280+0000","last_peered":"2024-04-26T01:51:25.413280+0000","last_clean":"2024-04-26T01:51:25.413280+0000","last_became_active":"2024-04-26T01:50:27.002344+0000","last_became_peered":"2024-04-26T01:50:27.002344+0000","last_unstale":"2024-04-26T01:51:25.413280+0000","last_undegraded":"2024-04-26T01:51:25.413280+0000","last_fullsized":"2024-04-26T01:51:25.413280+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:46:02.101508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.416372+0000","last_change":"2024-04-26T01:49:37.663306+0000","last_active":"2024-04-26T01:51:25.416372+0000","last_peered":"2024-04-26T01:51:25.416372+0000","last_clean":"2024-04-26T01:51:25.416372+0000","last_became_active":"2024-04-26T01:49:37.662999+0000","last_became_peered":"2024-04-26T01:49:37.662999+0000","last_unstale":"2024-04-26T01:51:25.416372+0000","last_undegraded":"2024-04-26T01:51:25.416372+0000","last_fullsized":"2024-04-26T01:51:25.416372+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:55.155096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.414001+0000","last_change":"2024-04-26T01:49:37.655157+0000","last_active":"2024-04-26T01:51:25.414001+0000","last_peered":"2024-04-26T01:51:25.414001+0000","last_clean":"2024-04-26T01:51:25.414001+0000","last_became_active":"2024-04-26T01:49:37.654810+0000","last_became_peered":"2024-04-26T01:49:37.654810+0000","last_unstale":"2024-04-26T01:51:25.414001+0000","last_undegraded":"2024-04-26T01:51:25.414001+0000","last_fullsized":"2024-04-26T01:51:25.414001+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:28:04.012382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.412653+0000","last_change":"2024-04-26T01:49:37.667228+0000","last_active":"2024-04-26T01:51:25.412653+0000","last_peered":"2024-04-26T01:51:25.412653+0000","last_clean":"2024-04-26T01:51:25.412653+0000","last_became_active":"2024-04-26T01:49:37.666835+0000","last_became_peered":"2024-04-26T01:49:37.666835+0000","last_unstale":"2024-04-26T01:51:25.412653+0000","last_undegraded":"2024-04-26T01:51:25.412653+0000","last_fullsized":"2024-04-26T01:51:25.412653+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:54:16.902836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.400335+0000","last_change":"2024-04-26T01:49:37.658231+0000","last_active":"2024-04-26T01:51:25.400335+0000","last_peered":"2024-04-26T01:51:25.400335+0000","last_clean":"2024-04-26T01:51:25.400335+0000","last_became_active":"2024-04-26T01:49:37.657842+0000","last_became_peered":"2024-04-26T01:49:37.657842+0000","last_unstale":"2024-04-26T01:51:25.400335+0000","last_undegraded":"2024-04-26T01:51:25.400335+0000","last_fullsized":"2024-04-26T01:51:25.400335+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:02:01.445262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.380571+0000","last_change":"2024-04-26T01:49:37.645221+0000","last_active":"2024-04-26T01:51:25.380571+0000","last_peered":"2024-04-26T01:51:25.380571+0000","last_clean":"2024-04-26T01:51:25.380571+0000","last_became_active":"2024-04-26T01:49:37.644814+0000","last_became_peered":"2024-04-26T01:49:37.644814+0000","last_unstale":"2024-04-26T01:51:25.380571+0000","last_undegraded":"2024-04-26T01:51:25.380571+0000","last_fullsized":"2024-04-26T01:51:25.380571+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:03:17.850633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65270,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":741,"num_read_kb":443,"num_write":173,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":195810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147,"ondisk_log_size":147,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318320,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":995328,"data_stored":486380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579955,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28668,"kb_used_data":1652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716484,"statfs":{"total":95995035648,"available":95965679616,"internally_reserved":0,"allocated":1691648,"data_stored":1138989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743481,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1011712,"data_stored":486616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005120,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":486536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266756,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28584,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716568,"statfs":{"total":95995035648,"available":95965765632,"internally_reserved":0,"allocated":1605632,"data_stored":1079139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430283,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":486005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574950,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1060864,"data_stored":546394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738474,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1019904,"data_stored":487276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411426,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1085440,"data_stored":548276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117082,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1015808,"data_stored":486316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855447,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":1003520,"data_stored":486974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691920,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28596,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716556,"statfs":{"total":95995035648,"available":95965753344,"internally_reserved":0,"allocated":1617920,"data_stored":1078291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":60596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T01:52:03.933 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T01:52:04.010 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:52:03 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:52:04.010 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:52:03 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:52:04.011 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:52:03 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:52:04.011 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:52:03 smithi151 ceph-mon[28057]: from='client.24826 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:52:04.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:52:03 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:52:04.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:52:03 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:52:04.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:52:03 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:52:04.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:52:03 smithi172 ceph-mon[32785]: from='client.24826 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:52:04.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:52:03 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:52:04.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:52:03 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:52:04.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:52:03 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:52:04.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:52:03 smithi165 ceph-mon[32648]: from='client.24826 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:52:04.383 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:52:05.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:52:04 smithi151 ceph-mon[28057]: pgmap v401: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:52:05.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:52:04 smithi172 ceph-mon[32785]: pgmap v401: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:52:05.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:52:04 smithi165 ceph-mon[32648]: pgmap v401: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:52:06.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:06.491+0000 7f04baa00700 1 -- 172.21.15.151:0/2892919198 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f04b4109fa0 msgr2=0x7f04b4111b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:06.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:06.491+0000 7f04baa00700 1 --2- 172.21.15.151:0/2892919198 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f04b4109fa0 0x7f04b4111b70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f04ac00b0d0 tx=0x7f04ac00b3e0 comp rx=0 tx=0).stop 2024-04-26T01:52:06.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:06.491+0000 7f04baa00700 1 -- 172.21.15.151:0/2892919198 shutdown_connections 2024-04-26T01:52:06.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:06.491+0000 7f04baa00700 1 --2- 172.21.15.151:0/2892919198 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f04b4109fa0 0x7f04b4111b70 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:06.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:06.491+0000 7f04baa00700 1 --2- 172.21.15.151:0/2892919198 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04b4104ec0 0x7f04b41052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:06.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:06.491+0000 7f04baa00700 1 --2- 172.21.15.151:0/2892919198 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04b4105870 0x7f04b41098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:06.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:06.491+0000 7f04baa00700 1 -- 172.21.15.151:0/2892919198 >> 172.21.15.151:0/2892919198 conn(0x7f04b4100750 msgr2=0x7f04b4102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:06.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:06.491+0000 7f04baa00700 1 -- 172.21.15.151:0/2892919198 shutdown_connections 2024-04-26T01:52:06.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:06.492+0000 7f04baa00700 1 -- 172.21.15.151:0/2892919198 wait complete. 2024-04-26T01:52:06.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:06.492+0000 7f04baa00700 1 Processor -- start 2024-04-26T01:52:06.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:06.493+0000 7f04baa00700 1 -- start start 2024-04-26T01:52:06.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:06.493+0000 7f04baa00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04b4104ec0 0x7f04b419b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:06.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:06.493+0000 7f04baa00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04b4105870 0x7f04b419b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:06.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:06.493+0000 7f04baa00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f04b4109fa0 0x7f04b419fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:06.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:06.493+0000 7f04baa00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f04b4114e90 con 0x7f04b4104ec0 2024-04-26T01:52:06.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:06.493+0000 7f04baa00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f04b4114d10 con 0x7f04b4105870 2024-04-26T01:52:06.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:06.493+0000 7f04baa00700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f04b4115010 con 0x7f04b4109fa0 2024-04-26T01:52:06.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:06.493+0000 7f04b3e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04b4104ec0 0x7f04b419b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:06.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:06.493+0000 7f04b3e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04b4104ec0 0x7f04b419b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49144/0 (socket says 172.21.15.151:49144) 2024-04-26T01:52:06.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:06.493+0000 7f04b3e00700 1 -- 172.21.15.151:0/858456309 learned_addr learned my addr 172.21.15.151:0/858456309 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:52:06.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:06.493+0000 7f04b8a00700 1 --2- 172.21.15.151:0/858456309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f04b4109fa0 0x7f04b419fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:06.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:06.494+0000 7f04b3400700 1 --2- 172.21.15.151:0/858456309 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04b4105870 0x7f04b419b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:06.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:06.494+0000 7f04b8a00700 1 -- 172.21.15.151:0/858456309 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04b4105870 msgr2=0x7f04b419b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:06.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:06.494+0000 7f04b8a00700 1 --2- 172.21.15.151:0/858456309 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04b4105870 0x7f04b419b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:06.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:06.494+0000 7f04b8a00700 1 -- 172.21.15.151:0/858456309 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04b4104ec0 msgr2=0x7f04b419b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:06.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:06.494+0000 7f04b8a00700 1 --2- 172.21.15.151:0/858456309 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04b4104ec0 0x7f04b419b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:06.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:06.494+0000 7f04b8a00700 1 -- 172.21.15.151:0/858456309 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04b41a0290 con 0x7f04b4109fa0 2024-04-26T01:52:06.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:06.494+0000 7f04b8a00700 1 --2- 172.21.15.151:0/858456309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f04b4109fa0 0x7f04b419fb50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f04ac00b770 tx=0x7f04ac0079a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:06.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:06.494+0000 7f04b0c00700 1 -- 172.21.15.151:0/858456309 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ac01f070 con 0x7f04b4109fa0 2024-04-26T01:52:06.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:06.494+0000 7f04b0c00700 1 -- 172.21.15.151:0/858456309 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04ac004c50 con 0x7f04b4109fa0 2024-04-26T01:52:06.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:06.494+0000 7f04baa00700 1 -- 172.21.15.151:0/858456309 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04b41a0520 con 0x7f04b4109fa0 2024-04-26T01:52:06.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:06.495+0000 7f04b0c00700 1 -- 172.21.15.151:0/858456309 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ac01b690 con 0x7f04b4109fa0 2024-04-26T01:52:06.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:06.495+0000 7f04baa00700 1 -- 172.21.15.151:0/858456309 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04b41a7e00 con 0x7f04b4109fa0 2024-04-26T01:52:06.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:06.496+0000 7f04baa00700 1 -- 172.21.15.151:0/858456309 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04b4069730 con 0x7f04b4109fa0 2024-04-26T01:52:06.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:06.498+0000 7f04b0c00700 1 -- 172.21.15.151:0/858456309 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f04ac022070 con 0x7f04b4109fa0 2024-04-26T01:52:06.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:06.499+0000 7f04b0c00700 1 --2- 172.21.15.151:0/858456309 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f04980a3cc0 0x7f04980a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:06.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:06.499+0000 7f04b0c00700 1 -- 172.21.15.151:0/858456309 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f04ac00f020 con 0x7f04b4109fa0 2024-04-26T01:52:06.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:06.499+0000 7f04b3e00700 1 --2- 172.21.15.151:0/858456309 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f04980a3cc0 0x7f04980a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:06.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:06.500+0000 7f04b3e00700 1 --2- 172.21.15.151:0/858456309 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f04980a3cc0 0x7f04980a6180 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f04a4009d20 tx=0x7f04a4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:06.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:06.503+0000 7f04b0c00700 1 -- 172.21.15.151:0/858456309 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f04ac029020 con 0x7f04b4109fa0 2024-04-26T01:52:06.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:06.724+0000 7f04baa00700 1 -- 172.21.15.151:0/858456309 --> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f04b4062e80 con 0x7f04980a3cc0 2024-04-26T01:52:06.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:06.732+0000 7f04b0c00700 1 -- 172.21.15.151:0/858456309 <== mgr.14152 v2:172.21.15.151:6800/2251256505 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258841 (secure 0 0 0) 0x7f04b4062e80 con 0x7f04980a3cc0 2024-04-26T01:52:06.734 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:52:06.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:06.737+0000 7f04baa00700 1 -- 172.21.15.151:0/858456309 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f04980a3cc0 msgr2=0x7f04980a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:06.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:06.737+0000 7f04baa00700 1 --2- 172.21.15.151:0/858456309 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f04980a3cc0 0x7f04980a6180 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f04a4009d20 tx=0x7f04a4009260 comp rx=0 tx=0).stop 2024-04-26T01:52:06.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:06.738+0000 7f04baa00700 1 -- 172.21.15.151:0/858456309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f04b4109fa0 msgr2=0x7f04b419fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:06.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:06.738+0000 7f04baa00700 1 --2- 172.21.15.151:0/858456309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f04b4109fa0 0x7f04b419fb50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f04ac00b770 tx=0x7f04ac0079a0 comp rx=0 tx=0).stop 2024-04-26T01:52:06.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:06.738+0000 7f04baa00700 1 -- 172.21.15.151:0/858456309 shutdown_connections 2024-04-26T01:52:06.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:06.738+0000 7f04baa00700 1 --2- 172.21.15.151:0/858456309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f04b4109fa0 0x7f04b419fb50 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:06.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:06.738+0000 7f04baa00700 1 --2- 172.21.15.151:0/858456309 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f04980a3cc0 0x7f04980a6180 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:06.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:06.738+0000 7f04baa00700 1 --2- 172.21.15.151:0/858456309 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04b4104ec0 0x7f04b419b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:06.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:06.738+0000 7f04baa00700 1 --2- 172.21.15.151:0/858456309 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04b4105870 0x7f04b419b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:06.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:06.738+0000 7f04baa00700 1 -- 172.21.15.151:0/858456309 >> 172.21.15.151:0/858456309 conn(0x7f04b4100750 msgr2=0x7f04b4101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:06.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:06.738+0000 7f04baa00700 1 -- 172.21.15.151:0/858456309 shutdown_connections 2024-04-26T01:52:06.738 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:06.738+0000 7f04baa00700 1 -- 172.21.15.151:0/858456309 wait complete. 2024-04-26T01:52:06.738 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-04-26T01:52:07.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:52:06 smithi172 ceph-mon[32785]: pgmap v402: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:52:07.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:52:06 smithi165 ceph-mon[32648]: pgmap v402: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:52:07.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:52:06 smithi151 ceph-mon[28057]: pgmap v402: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:52:07.530 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":402,"stamp":"2024-04-26T01:52:05.442677+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655872,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":887,"num_read_kb":562,"num_write":356,"num_write_kb":3058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":786,"ondisk_log_size":786,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":387900,"kb_used_data":13788,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124553924,"statfs":{"total":1151940427776,"available":1151543218176,"internally_reserved":0,"allocated":14118912,"data_stored":7797192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005738"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.381128+0000","last_change":"2024-04-26T01:49:37.641666+0000","last_active":"2024-04-26T01:51:25.381128+0000","last_peered":"2024-04-26T01:51:25.381128+0000","last_clean":"2024-04-26T01:51:25.381128+0000","last_became_active":"2024-04-26T01:49:37.641310+0000","last_became_peered":"2024-04-26T01:49:37.641310+0000","last_unstale":"2024-04-26T01:51:25.381128+0000","last_undegraded":"2024-04-26T01:51:25.381128+0000","last_fullsized":"2024-04-26T01:51:25.381128+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:28:02.952851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.374761+0000","last_change":"2024-04-26T01:49:37.637607+0000","last_active":"2024-04-26T01:51:25.374761+0000","last_peered":"2024-04-26T01:51:25.374761+0000","last_clean":"2024-04-26T01:51:25.374761+0000","last_became_active":"2024-04-26T01:49:37.637283+0000","last_became_peered":"2024-04-26T01:49:37.637283+0000","last_unstale":"2024-04-26T01:51:25.374761+0000","last_undegraded":"2024-04-26T01:51:25.374761+0000","last_fullsized":"2024-04-26T01:51:25.374761+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:16:46.519381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.387398+0000","last_change":"2024-04-26T01:49:37.647769+0000","last_active":"2024-04-26T01:51:25.387398+0000","last_peered":"2024-04-26T01:51:25.387398+0000","last_clean":"2024-04-26T01:51:25.387398+0000","last_became_active":"2024-04-26T01:49:37.643750+0000","last_became_peered":"2024-04-26T01:49:37.643750+0000","last_unstale":"2024-04-26T01:51:25.387398+0000","last_undegraded":"2024-04-26T01:51:25.387398+0000","last_fullsized":"2024-04-26T01:51:25.387398+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:34:18.801267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.372634+0000","last_change":"2024-04-26T01:49:37.643291+0000","last_active":"2024-04-26T01:51:25.372634+0000","last_peered":"2024-04-26T01:51:25.372634+0000","last_clean":"2024-04-26T01:51:25.372634+0000","last_became_active":"2024-04-26T01:49:37.642947+0000","last_became_peered":"2024-04-26T01:49:37.642947+0000","last_unstale":"2024-04-26T01:51:25.372634+0000","last_undegraded":"2024-04-26T01:51:25.372634+0000","last_fullsized":"2024-04-26T01:51:25.372634+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:31:21.600533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.411232+0000","last_change":"2024-04-26T01:49:37.665311+0000","last_active":"2024-04-26T01:51:25.411232+0000","last_peered":"2024-04-26T01:51:25.411232+0000","last_clean":"2024-04-26T01:51:25.411232+0000","last_became_active":"2024-04-26T01:49:37.664944+0000","last_became_peered":"2024-04-26T01:49:37.664944+0000","last_unstale":"2024-04-26T01:51:25.411232+0000","last_undegraded":"2024-04-26T01:51:25.411232+0000","last_fullsized":"2024-04-26T01:51:25.411232+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:49:50.744630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.388269+0000","last_change":"2024-04-26T01:49:37.638667+0000","last_active":"2024-04-26T01:51:25.388269+0000","last_peered":"2024-04-26T01:51:25.388269+0000","last_clean":"2024-04-26T01:51:25.388269+0000","last_became_active":"2024-04-26T01:49:37.638226+0000","last_became_peered":"2024-04-26T01:49:37.638226+0000","last_unstale":"2024-04-26T01:51:25.388269+0000","last_undegraded":"2024-04-26T01:51:25.388269+0000","last_fullsized":"2024-04-26T01:51:25.388269+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:46:17.786883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.392209+0000","last_change":"2024-04-26T01:49:37.650082+0000","last_active":"2024-04-26T01:51:25.392209+0000","last_peered":"2024-04-26T01:51:25.392209+0000","last_clean":"2024-04-26T01:51:25.392209+0000","last_became_active":"2024-04-26T01:49:37.649637+0000","last_became_peered":"2024-04-26T01:49:37.649637+0000","last_unstale":"2024-04-26T01:51:25.392209+0000","last_undegraded":"2024-04-26T01:51:25.392209+0000","last_fullsized":"2024-04-26T01:51:25.392209+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:21:04.686398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.416839+0000","last_change":"2024-04-26T01:49:37.661255+0000","last_active":"2024-04-26T01:51:25.416839+0000","last_peered":"2024-04-26T01:51:25.416839+0000","last_clean":"2024-04-26T01:51:25.416839+0000","last_became_active":"2024-04-26T01:49:37.658882+0000","last_became_peered":"2024-04-26T01:49:37.658882+0000","last_unstale":"2024-04-26T01:51:25.416839+0000","last_undegraded":"2024-04-26T01:51:25.416839+0000","last_fullsized":"2024-04-26T01:51:25.416839+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:36:27.874191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.384359+0000","last_change":"2024-04-26T01:49:37.648874+0000","last_active":"2024-04-26T01:51:25.384359+0000","last_peered":"2024-04-26T01:51:25.384359+0000","last_clean":"2024-04-26T01:51:25.384359+0000","last_became_active":"2024-04-26T01:49:37.648490+0000","last_became_peered":"2024-04-26T01:49:37.648490+0000","last_unstale":"2024-04-26T01:51:25.384359+0000","last_undegraded":"2024-04-26T01:51:25.384359+0000","last_fullsized":"2024-04-26T01:51:25.384359+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:46:12.552861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.373500+0000","last_change":"2024-04-26T01:49:37.643842+0000","last_active":"2024-04-26T01:51:25.373500+0000","last_peered":"2024-04-26T01:51:25.373500+0000","last_clean":"2024-04-26T01:51:25.373500+0000","last_became_active":"2024-04-26T01:49:37.643084+0000","last_became_peered":"2024-04-26T01:49:37.643084+0000","last_unstale":"2024-04-26T01:51:25.373500+0000","last_undegraded":"2024-04-26T01:51:25.373500+0000","last_fullsized":"2024-04-26T01:51:25.373500+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:16:53.102318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.384964+0000","last_change":"2024-04-26T01:49:37.644209+0000","last_active":"2024-04-26T01:51:25.384964+0000","last_peered":"2024-04-26T01:51:25.384964+0000","last_clean":"2024-04-26T01:51:25.384964+0000","last_became_active":"2024-04-26T01:49:37.643659+0000","last_became_peered":"2024-04-26T01:49:37.643659+0000","last_unstale":"2024-04-26T01:51:25.384964+0000","last_undegraded":"2024-04-26T01:51:25.384964+0000","last_fullsized":"2024-04-26T01:51:25.384964+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:24:46.363873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.398749+0000","last_change":"2024-04-26T01:50:27.001615+0000","last_active":"2024-04-26T01:51:25.398749+0000","last_peered":"2024-04-26T01:51:25.398749+0000","last_clean":"2024-04-26T01:51:25.398749+0000","last_became_active":"2024-04-26T01:50:27.001039+0000","last_became_peered":"2024-04-26T01:50:27.001039+0000","last_unstale":"2024-04-26T01:51:25.398749+0000","last_undegraded":"2024-04-26T01:51:25.398749+0000","last_fullsized":"2024-04-26T01:51:25.398749+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:42:46.148500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.402097+0000","last_change":"2024-04-26T01:49:37.651085+0000","last_active":"2024-04-26T01:51:25.402097+0000","last_peered":"2024-04-26T01:51:25.402097+0000","last_clean":"2024-04-26T01:51:25.402097+0000","last_became_active":"2024-04-26T01:49:37.649662+0000","last_became_peered":"2024-04-26T01:49:37.649662+0000","last_unstale":"2024-04-26T01:51:25.402097+0000","last_undegraded":"2024-04-26T01:51:25.402097+0000","last_fullsized":"2024-04-26T01:51:25.402097+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:41:34.537533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.373700+0000","last_change":"2024-04-26T01:49:37.648432+0000","last_active":"2024-04-26T01:51:25.373700+0000","last_peered":"2024-04-26T01:51:25.373700+0000","last_clean":"2024-04-26T01:51:25.373700+0000","last_became_active":"2024-04-26T01:49:37.648047+0000","last_became_peered":"2024-04-26T01:49:37.648047+0000","last_unstale":"2024-04-26T01:51:25.373700+0000","last_undegraded":"2024-04-26T01:51:25.373700+0000","last_fullsized":"2024-04-26T01:51:25.373700+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:00:06.843384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.382303+0000","last_change":"2024-04-26T01:49:37.644203+0000","last_active":"2024-04-26T01:51:25.382303+0000","last_peered":"2024-04-26T01:51:25.382303+0000","last_clean":"2024-04-26T01:51:25.382303+0000","last_became_active":"2024-04-26T01:49:37.643772+0000","last_became_peered":"2024-04-26T01:49:37.643772+0000","last_unstale":"2024-04-26T01:51:25.382303+0000","last_undegraded":"2024-04-26T01:51:25.382303+0000","last_fullsized":"2024-04-26T01:51:25.382303+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:52:20.707845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.384724+0000","last_change":"2024-04-26T01:49:37.647364+0000","last_active":"2024-04-26T01:51:25.384724+0000","last_peered":"2024-04-26T01:51:25.384724+0000","last_clean":"2024-04-26T01:51:25.384724+0000","last_became_active":"2024-04-26T01:49:37.647054+0000","last_became_peered":"2024-04-26T01:49:37.647054+0000","last_unstale":"2024-04-26T01:51:25.384724+0000","last_undegraded":"2024-04-26T01:51:25.384724+0000","last_fullsized":"2024-04-26T01:51:25.384724+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:53:09.631463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.384289+0000","last_change":"2024-04-26T01:49:37.641843+0000","last_active":"2024-04-26T01:51:25.384289+0000","last_peered":"2024-04-26T01:51:25.384289+0000","last_clean":"2024-04-26T01:51:25.384289+0000","last_became_active":"2024-04-26T01:49:37.641514+0000","last_became_peered":"2024-04-26T01:49:37.641514+0000","last_unstale":"2024-04-26T01:51:25.384289+0000","last_undegraded":"2024-04-26T01:51:25.384289+0000","last_fullsized":"2024-04-26T01:51:25.384289+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:24:32.261834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.412843+0000","last_change":"2024-04-26T01:49:37.644750+0000","last_active":"2024-04-26T01:51:25.412843+0000","last_peered":"2024-04-26T01:51:25.412843+0000","last_clean":"2024-04-26T01:51:25.412843+0000","last_became_active":"2024-04-26T01:49:37.644086+0000","last_became_peered":"2024-04-26T01:49:37.644086+0000","last_unstale":"2024-04-26T01:51:25.412843+0000","last_undegraded":"2024-04-26T01:51:25.412843+0000","last_fullsized":"2024-04-26T01:51:25.412843+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:59:42.614846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.403738+0000","last_change":"2024-04-26T01:49:37.664204+0000","last_active":"2024-04-26T01:51:25.403738+0000","last_peered":"2024-04-26T01:51:25.403738+0000","last_clean":"2024-04-26T01:51:25.403738+0000","last_became_active":"2024-04-26T01:49:37.663878+0000","last_became_peered":"2024-04-26T01:49:37.663878+0000","last_unstale":"2024-04-26T01:51:25.403738+0000","last_undegraded":"2024-04-26T01:51:25.403738+0000","last_fullsized":"2024-04-26T01:51:25.403738+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:43:49.272308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.997103+0000","last_change":"2024-04-26T01:49:33.592017+0000","last_active":"2024-04-26T01:50:26.997103+0000","last_peered":"2024-04-26T01:50:26.997103+0000","last_clean":"2024-04-26T01:50:26.997103+0000","last_became_active":"2024-04-26T01:49:33.587928+0000","last_became_peered":"2024-04-26T01:49:33.587928+0000","last_unstale":"2024-04-26T01:50:26.997103+0000","last_undegraded":"2024-04-26T01:50:26.997103+0000","last_fullsized":"2024-04-26T01:50:26.997103+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:02:04.313785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.411761+0000","last_change":"2024-04-26T01:49:37.662010+0000","last_active":"2024-04-26T01:51:25.411761+0000","last_peered":"2024-04-26T01:51:25.411761+0000","last_clean":"2024-04-26T01:51:25.411761+0000","last_became_active":"2024-04-26T01:49:37.661673+0000","last_became_peered":"2024-04-26T01:49:37.661673+0000","last_unstale":"2024-04-26T01:51:25.411761+0000","last_undegraded":"2024-04-26T01:51:25.411761+0000","last_fullsized":"2024-04-26T01:51:25.411761+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:37:34.155998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.994246+0000","last_change":"2024-04-26T01:49:34.185758+0000","last_active":"2024-04-26T01:50:26.994246+0000","last_peered":"2024-04-26T01:50:26.994246+0000","last_clean":"2024-04-26T01:50:26.994246+0000","last_became_active":"2024-04-26T01:49:34.184669+0000","last_became_peered":"2024-04-26T01:49:34.184669+0000","last_unstale":"2024-04-26T01:50:26.994246+0000","last_undegraded":"2024-04-26T01:50:26.994246+0000","last_fullsized":"2024-04-26T01:50:26.994246+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:23:23.742032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.398961+0000","last_change":"2024-04-26T01:49:37.641307+0000","last_active":"2024-04-26T01:51:25.398961+0000","last_peered":"2024-04-26T01:51:25.398961+0000","last_clean":"2024-04-26T01:51:25.398961+0000","last_became_active":"2024-04-26T01:49:37.640677+0000","last_became_peered":"2024-04-26T01:49:37.640677+0000","last_unstale":"2024-04-26T01:51:25.398961+0000","last_undegraded":"2024-04-26T01:51:25.398961+0000","last_fullsized":"2024-04-26T01:51:25.398961+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:35:56.859045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.007585+0000","last_change":"2024-04-26T01:50:27.007034+0000","last_active":"2024-04-26T01:50:27.007585+0000","last_peered":"2024-04-26T01:50:27.007585+0000","last_clean":"2024-04-26T01:50:27.007585+0000","last_became_active":"2024-04-26T01:50:27.000945+0000","last_became_peered":"2024-04-26T01:50:27.000945+0000","last_unstale":"2024-04-26T01:50:27.007585+0000","last_undegraded":"2024-04-26T01:50:27.007585+0000","last_fullsized":"2024-04-26T01:50:27.007585+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:39:02.484719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.406751+0000","last_change":"2024-04-26T01:49:37.645936+0000","last_active":"2024-04-26T01:51:25.406751+0000","last_peered":"2024-04-26T01:51:25.406751+0000","last_clean":"2024-04-26T01:51:25.406751+0000","last_became_active":"2024-04-26T01:49:37.645535+0000","last_became_peered":"2024-04-26T01:49:37.645535+0000","last_unstale":"2024-04-26T01:51:25.406751+0000","last_undegraded":"2024-04-26T01:51:25.406751+0000","last_fullsized":"2024-04-26T01:51:25.406751+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:45:04.434415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.999363+0000","last_change":"2024-04-26T01:49:33.589904+0000","last_active":"2024-04-26T01:50:26.999363+0000","last_peered":"2024-04-26T01:50:26.999363+0000","last_clean":"2024-04-26T01:50:26.999363+0000","last_became_active":"2024-04-26T01:49:33.589332+0000","last_became_peered":"2024-04-26T01:49:33.589332+0000","last_unstale":"2024-04-26T01:50:26.999363+0000","last_undegraded":"2024-04-26T01:50:26.999363+0000","last_fullsized":"2024-04-26T01:50:26.999363+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:28:07.972583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.385634+0000","last_change":"2024-04-26T01:49:37.652093+0000","last_active":"2024-04-26T01:51:25.385634+0000","last_peered":"2024-04-26T01:51:25.385634+0000","last_clean":"2024-04-26T01:51:25.385634+0000","last_became_active":"2024-04-26T01:49:37.651506+0000","last_became_peered":"2024-04-26T01:49:37.651506+0000","last_unstale":"2024-04-26T01:51:25.385634+0000","last_undegraded":"2024-04-26T01:51:25.385634+0000","last_fullsized":"2024-04-26T01:51:25.385634+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:01:31.421498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"92'7","reported_seq":158,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:52:00.972305+0000","last_change":"2024-04-26T01:49:33.587652+0000","last_active":"2024-04-26T01:52:00.972305+0000","last_peered":"2024-04-26T01:52:00.972305+0000","last_clean":"2024-04-26T01:52:00.972305+0000","last_became_active":"2024-04-26T01:49:33.587294+0000","last_became_peered":"2024-04-26T01:49:33.587294+0000","last_unstale":"2024-04-26T01:52:00.972305+0000","last_undegraded":"2024-04-26T01:52:00.972305+0000","last_fullsized":"2024-04-26T01:52:00.972305+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:32:46.803415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":125,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.398412+0000","last_change":"2024-04-26T01:49:37.647899+0000","last_active":"2024-04-26T01:51:25.398412+0000","last_peered":"2024-04-26T01:51:25.398412+0000","last_clean":"2024-04-26T01:51:25.398412+0000","last_became_active":"2024-04-26T01:49:37.647420+0000","last_became_peered":"2024-04-26T01:49:37.647420+0000","last_unstale":"2024-04-26T01:51:25.398412+0000","last_undegraded":"2024-04-26T01:51:25.398412+0000","last_fullsized":"2024-04-26T01:51:25.398412+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:16:39.262842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.992761+0000","last_change":"2024-04-26T01:49:33.596096+0000","last_active":"2024-04-26T01:50:26.992761+0000","last_peered":"2024-04-26T01:50:26.992761+0000","last_clean":"2024-04-26T01:50:26.992761+0000","last_became_active":"2024-04-26T01:49:33.595604+0000","last_became_peered":"2024-04-26T01:49:33.595604+0000","last_unstale":"2024-04-26T01:50:26.992761+0000","last_undegraded":"2024-04-26T01:50:26.992761+0000","last_fullsized":"2024-04-26T01:50:26.992761+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:44:26.497448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.412127+0000","last_change":"2024-04-26T01:49:37.665001+0000","last_active":"2024-04-26T01:51:25.412127+0000","last_peered":"2024-04-26T01:51:25.412127+0000","last_clean":"2024-04-26T01:51:25.412127+0000","last_became_active":"2024-04-26T01:49:37.664598+0000","last_became_peered":"2024-04-26T01:49:37.664598+0000","last_unstale":"2024-04-26T01:51:25.412127+0000","last_undegraded":"2024-04-26T01:51:25.412127+0000","last_fullsized":"2024-04-26T01:51:25.412127+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:54:28.299606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.006003+0000","last_change":"2024-04-26T01:49:33.602593+0000","last_active":"2024-04-26T01:50:27.006003+0000","last_peered":"2024-04-26T01:50:27.006003+0000","last_clean":"2024-04-26T01:50:27.006003+0000","last_became_active":"2024-04-26T01:49:33.602223+0000","last_became_peered":"2024-04-26T01:49:33.602223+0000","last_unstale":"2024-04-26T01:50:27.006003+0000","last_undegraded":"2024-04-26T01:50:27.006003+0000","last_fullsized":"2024-04-26T01:50:27.006003+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:43:29.404719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.415275+0000","last_change":"2024-04-26T01:49:37.662351+0000","last_active":"2024-04-26T01:51:25.415275+0000","last_peered":"2024-04-26T01:51:25.415275+0000","last_clean":"2024-04-26T01:51:25.415275+0000","last_became_active":"2024-04-26T01:49:37.661900+0000","last_became_peered":"2024-04-26T01:49:37.661900+0000","last_unstale":"2024-04-26T01:51:25.415275+0000","last_undegraded":"2024-04-26T01:51:25.415275+0000","last_fullsized":"2024-04-26T01:51:25.415275+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:49:10.878580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.068468+0000","last_change":"2024-04-26T01:49:33.586538+0000","last_active":"2024-04-26T01:50:27.068468+0000","last_peered":"2024-04-26T01:50:27.068468+0000","last_clean":"2024-04-26T01:50:27.068468+0000","last_became_active":"2024-04-26T01:49:33.586127+0000","last_became_peered":"2024-04-26T01:49:33.586127+0000","last_unstale":"2024-04-26T01:50:27.068468+0000","last_undegraded":"2024-04-26T01:50:27.068468+0000","last_fullsized":"2024-04-26T01:50:27.068468+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:05.599479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.412509+0000","last_change":"2024-04-26T01:49:37.657154+0000","last_active":"2024-04-26T01:51:25.412509+0000","last_peered":"2024-04-26T01:51:25.412509+0000","last_clean":"2024-04-26T01:51:25.412509+0000","last_became_active":"2024-04-26T01:49:37.656814+0000","last_became_peered":"2024-04-26T01:49:37.656814+0000","last_unstale":"2024-04-26T01:51:25.412509+0000","last_undegraded":"2024-04-26T01:51:25.412509+0000","last_fullsized":"2024-04-26T01:51:25.412509+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:26:54.538267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"92'4","reported_seq":150,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:59.970024+0000","last_change":"2024-04-26T01:49:33.602441+0000","last_active":"2024-04-26T01:51:59.970024+0000","last_peered":"2024-04-26T01:51:59.970024+0000","last_clean":"2024-04-26T01:51:59.970024+0000","last_became_active":"2024-04-26T01:49:33.602116+0000","last_became_peered":"2024-04-26T01:49:33.602116+0000","last_unstale":"2024-04-26T01:51:59.970024+0000","last_undegraded":"2024-04-26T01:51:59.970024+0000","last_fullsized":"2024-04-26T01:51:59.970024+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:45:52.541043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":119,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.412565+0000","last_change":"2024-04-26T01:49:37.645921+0000","last_active":"2024-04-26T01:51:25.412565+0000","last_peered":"2024-04-26T01:51:25.412565+0000","last_clean":"2024-04-26T01:51:25.412565+0000","last_became_active":"2024-04-26T01:49:37.645590+0000","last_became_peered":"2024-04-26T01:49:37.645590+0000","last_unstale":"2024-04-26T01:51:25.412565+0000","last_undegraded":"2024-04-26T01:51:25.412565+0000","last_fullsized":"2024-04-26T01:51:25.412565+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:00:08.931051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998659+0000","last_change":"2024-04-26T01:49:33.589350+0000","last_active":"2024-04-26T01:50:26.998659+0000","last_peered":"2024-04-26T01:50:26.998659+0000","last_clean":"2024-04-26T01:50:26.998659+0000","last_became_active":"2024-04-26T01:49:33.588943+0000","last_became_peered":"2024-04-26T01:49:33.588943+0000","last_unstale":"2024-04-26T01:50:26.998659+0000","last_undegraded":"2024-04-26T01:50:26.998659+0000","last_fullsized":"2024-04-26T01:50:26.998659+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:36:36.104012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.384125+0000","last_change":"2024-04-26T01:49:37.648458+0000","last_active":"2024-04-26T01:51:25.384125+0000","last_peered":"2024-04-26T01:51:25.384125+0000","last_clean":"2024-04-26T01:51:25.384125+0000","last_became_active":"2024-04-26T01:49:37.648037+0000","last_became_peered":"2024-04-26T01:49:37.648037+0000","last_unstale":"2024-04-26T01:51:25.384125+0000","last_undegraded":"2024-04-26T01:51:25.384125+0000","last_fullsized":"2024-04-26T01:51:25.384125+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:28:05.711795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.069128+0000","last_change":"2024-04-26T01:49:34.189193+0000","last_active":"2024-04-26T01:50:27.069128+0000","last_peered":"2024-04-26T01:50:27.069128+0000","last_clean":"2024-04-26T01:50:27.069128+0000","last_became_active":"2024-04-26T01:49:34.188827+0000","last_became_peered":"2024-04-26T01:49:34.188827+0000","last_unstale":"2024-04-26T01:50:27.069128+0000","last_undegraded":"2024-04-26T01:50:27.069128+0000","last_fullsized":"2024-04-26T01:50:27.069128+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:02:04.583215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.384424+0000","last_change":"2024-04-26T01:49:37.644208+0000","last_active":"2024-04-26T01:51:25.384424+0000","last_peered":"2024-04-26T01:51:25.384424+0000","last_clean":"2024-04-26T01:51:25.384424+0000","last_became_active":"2024-04-26T01:49:37.643787+0000","last_became_peered":"2024-04-26T01:49:37.643787+0000","last_unstale":"2024-04-26T01:51:25.384424+0000","last_undegraded":"2024-04-26T01:51:25.384424+0000","last_fullsized":"2024-04-26T01:51:25.384424+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:42:21.243628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"92'31","reported_seq":306,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:59.970180+0000","last_change":"2024-04-26T01:49:33.590965+0000","last_active":"2024-04-26T01:51:59.970180+0000","last_peered":"2024-04-26T01:51:59.970180+0000","last_clean":"2024-04-26T01:51:59.970180+0000","last_became_active":"2024-04-26T01:49:33.590543+0000","last_became_peered":"2024-04-26T01:49:33.590543+0000","last_unstale":"2024-04-26T01:51:59.970180+0000","last_undegraded":"2024-04-26T01:51:59.970180+0000","last_fullsized":"2024-04-26T01:51:59.970180+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:26.627549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60596,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":248,"num_read_kb":183,"num_write":32,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.376539+0000","last_change":"2024-04-26T01:49:37.641172+0000","last_active":"2024-04-26T01:51:25.376539+0000","last_peered":"2024-04-26T01:51:25.376539+0000","last_clean":"2024-04-26T01:51:25.376539+0000","last_became_active":"2024-04-26T01:49:37.640866+0000","last_became_peered":"2024-04-26T01:49:37.640866+0000","last_unstale":"2024-04-26T01:51:25.376539+0000","last_undegraded":"2024-04-26T01:51:25.376539+0000","last_fullsized":"2024-04-26T01:51:25.376539+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:29:12.461005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.999186+0000","last_change":"2024-04-26T01:49:33.598466+0000","last_active":"2024-04-26T01:50:26.999186+0000","last_peered":"2024-04-26T01:50:26.999186+0000","last_clean":"2024-04-26T01:50:26.999186+0000","last_became_active":"2024-04-26T01:49:33.598042+0000","last_became_peered":"2024-04-26T01:49:33.598042+0000","last_unstale":"2024-04-26T01:50:26.999186+0000","last_undegraded":"2024-04-26T01:50:26.999186+0000","last_fullsized":"2024-04-26T01:50:26.999186+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:55:54.901016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.412517+0000","last_change":"2024-04-26T01:49:37.665772+0000","last_active":"2024-04-26T01:51:25.412517+0000","last_peered":"2024-04-26T01:51:25.412517+0000","last_clean":"2024-04-26T01:51:25.412517+0000","last_became_active":"2024-04-26T01:49:37.665524+0000","last_became_peered":"2024-04-26T01:49:37.665524+0000","last_unstale":"2024-04-26T01:51:25.412517+0000","last_undegraded":"2024-04-26T01:51:25.412517+0000","last_fullsized":"2024-04-26T01:51:25.412517+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:00:19.729098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.068644+0000","last_change":"2024-04-26T01:49:33.599456+0000","last_active":"2024-04-26T01:50:27.068644+0000","last_peered":"2024-04-26T01:50:27.068644+0000","last_clean":"2024-04-26T01:50:27.068644+0000","last_became_active":"2024-04-26T01:49:33.599101+0000","last_became_peered":"2024-04-26T01:49:33.599101+0000","last_unstale":"2024-04-26T01:50:27.068644+0000","last_undegraded":"2024-04-26T01:50:27.068644+0000","last_fullsized":"2024-04-26T01:50:27.068644+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:14:35.923011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.400197+0000","last_change":"2024-04-26T01:49:37.647193+0000","last_active":"2024-04-26T01:51:25.400197+0000","last_peered":"2024-04-26T01:51:25.400197+0000","last_clean":"2024-04-26T01:51:25.400197+0000","last_became_active":"2024-04-26T01:49:37.646881+0000","last_became_peered":"2024-04-26T01:49:37.646881+0000","last_unstale":"2024-04-26T01:51:25.400197+0000","last_undegraded":"2024-04-26T01:51:25.400197+0000","last_fullsized":"2024-04-26T01:51:25.400197+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:50:10.006986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"92'5","reported_seq":158,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:52:02.037151+0000","last_change":"2024-04-26T01:49:33.590398+0000","last_active":"2024-04-26T01:52:02.037151+0000","last_peered":"2024-04-26T01:52:02.037151+0000","last_clean":"2024-04-26T01:52:02.037151+0000","last_became_active":"2024-04-26T01:49:33.590033+0000","last_became_peered":"2024-04-26T01:49:33.590033+0000","last_unstale":"2024-04-26T01:52:02.037151+0000","last_undegraded":"2024-04-26T01:52:02.037151+0000","last_fullsized":"2024-04-26T01:52:02.037151+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:27:43.203781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":127,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.377744+0000","last_change":"2024-04-26T01:49:37.649784+0000","last_active":"2024-04-26T01:51:25.377744+0000","last_peered":"2024-04-26T01:51:25.377744+0000","last_clean":"2024-04-26T01:51:25.377744+0000","last_became_active":"2024-04-26T01:49:37.649431+0000","last_became_peered":"2024-04-26T01:49:37.649431+0000","last_unstale":"2024-04-26T01:51:25.377744+0000","last_undegraded":"2024-04-26T01:51:25.377744+0000","last_fullsized":"2024-04-26T01:51:25.377744+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:45:19.045983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998248+0000","last_change":"2024-04-26T01:49:34.182987+0000","last_active":"2024-04-26T01:50:26.998248+0000","last_peered":"2024-04-26T01:50:26.998248+0000","last_clean":"2024-04-26T01:50:26.998248+0000","last_became_active":"2024-04-26T01:49:34.182600+0000","last_became_peered":"2024-04-26T01:49:34.182600+0000","last_unstale":"2024-04-26T01:50:26.998248+0000","last_undegraded":"2024-04-26T01:50:26.998248+0000","last_fullsized":"2024-04-26T01:50:26.998248+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:05:52.774170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.384199+0000","last_change":"2024-04-26T01:49:37.646097+0000","last_active":"2024-04-26T01:51:25.384199+0000","last_peered":"2024-04-26T01:51:25.384199+0000","last_clean":"2024-04-26T01:51:25.384199+0000","last_became_active":"2024-04-26T01:49:37.645763+0000","last_became_peered":"2024-04-26T01:49:37.645763+0000","last_unstale":"2024-04-26T01:51:25.384199+0000","last_undegraded":"2024-04-26T01:51:25.384199+0000","last_fullsized":"2024-04-26T01:51:25.384199+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:42:37.159950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.999051+0000","last_change":"2024-04-26T01:49:33.591457+0000","last_active":"2024-04-26T01:50:26.999051+0000","last_peered":"2024-04-26T01:50:26.999051+0000","last_clean":"2024-04-26T01:50:26.999051+0000","last_became_active":"2024-04-26T01:49:33.590655+0000","last_became_peered":"2024-04-26T01:49:33.590655+0000","last_unstale":"2024-04-26T01:50:26.999051+0000","last_undegraded":"2024-04-26T01:50:26.999051+0000","last_fullsized":"2024-04-26T01:50:26.999051+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:33:18.803796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.380099+0000","last_change":"2024-04-26T01:49:37.646423+0000","last_active":"2024-04-26T01:51:25.380099+0000","last_peered":"2024-04-26T01:51:25.380099+0000","last_clean":"2024-04-26T01:51:25.380099+0000","last_became_active":"2024-04-26T01:49:37.646046+0000","last_became_peered":"2024-04-26T01:49:37.646046+0000","last_unstale":"2024-04-26T01:51:25.380099+0000","last_undegraded":"2024-04-26T01:51:25.380099+0000","last_fullsized":"2024-04-26T01:51:25.380099+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:26:03.570479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.997386+0000","last_change":"2024-04-26T01:49:34.192848+0000","last_active":"2024-04-26T01:50:26.997386+0000","last_peered":"2024-04-26T01:50:26.997386+0000","last_clean":"2024-04-26T01:50:26.997386+0000","last_became_active":"2024-04-26T01:49:34.190449+0000","last_became_peered":"2024-04-26T01:49:34.190449+0000","last_unstale":"2024-04-26T01:50:26.997386+0000","last_undegraded":"2024-04-26T01:50:26.997386+0000","last_fullsized":"2024-04-26T01:50:26.997386+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:05:22.094991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.402950+0000","last_change":"2024-04-26T01:49:37.667147+0000","last_active":"2024-04-26T01:51:25.402950+0000","last_peered":"2024-04-26T01:51:25.402950+0000","last_clean":"2024-04-26T01:51:25.402950+0000","last_became_active":"2024-04-26T01:49:37.666752+0000","last_became_peered":"2024-04-26T01:49:37.666752+0000","last_unstale":"2024-04-26T01:51:25.402950+0000","last_undegraded":"2024-04-26T01:51:25.402950+0000","last_fullsized":"2024-04-26T01:51:25.402950+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:18:31.778916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998501+0000","last_change":"2024-04-26T01:49:33.596155+0000","last_active":"2024-04-26T01:50:26.998501+0000","last_peered":"2024-04-26T01:50:26.998501+0000","last_clean":"2024-04-26T01:50:26.998501+0000","last_became_active":"2024-04-26T01:49:33.595760+0000","last_became_peered":"2024-04-26T01:49:33.595760+0000","last_unstale":"2024-04-26T01:50:26.998501+0000","last_undegraded":"2024-04-26T01:50:26.998501+0000","last_fullsized":"2024-04-26T01:50:26.998501+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:07:57.208116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.375222+0000","last_change":"2024-04-26T01:49:37.641660+0000","last_active":"2024-04-26T01:51:25.375222+0000","last_peered":"2024-04-26T01:51:25.375222+0000","last_clean":"2024-04-26T01:51:25.375222+0000","last_became_active":"2024-04-26T01:49:37.641302+0000","last_became_peered":"2024-04-26T01:49:37.641302+0000","last_unstale":"2024-04-26T01:51:25.375222+0000","last_undegraded":"2024-04-26T01:51:25.375222+0000","last_fullsized":"2024-04-26T01:51:25.375222+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:32:59.889189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.380794+0000","last_change":"2024-04-26T01:49:37.643083+0000","last_active":"2024-04-26T01:51:25.380794+0000","last_peered":"2024-04-26T01:51:25.380794+0000","last_clean":"2024-04-26T01:51:25.380794+0000","last_became_active":"2024-04-26T01:49:37.642548+0000","last_became_peered":"2024-04-26T01:49:37.642548+0000","last_unstale":"2024-04-26T01:51:25.380794+0000","last_undegraded":"2024-04-26T01:51:25.380794+0000","last_fullsized":"2024-04-26T01:51:25.380794+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:09:06.136510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'5","reported_seq":151,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:59.029912+0000","last_change":"2024-04-26T01:49:34.185069+0000","last_active":"2024-04-26T01:51:59.029912+0000","last_peered":"2024-04-26T01:51:59.029912+0000","last_clean":"2024-04-26T01:51:59.029912+0000","last_became_active":"2024-04-26T01:49:34.183989+0000","last_became_peered":"2024-04-26T01:49:34.183989+0000","last_unstale":"2024-04-26T01:51:59.029912+0000","last_undegraded":"2024-04-26T01:51:59.029912+0000","last_fullsized":"2024-04-26T01:51:59.029912+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:04:26.327960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":119,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.408158+0000","last_change":"2024-04-26T01:49:37.655512+0000","last_active":"2024-04-26T01:51:25.408158+0000","last_peered":"2024-04-26T01:51:25.408158+0000","last_clean":"2024-04-26T01:51:25.408158+0000","last_became_active":"2024-04-26T01:49:37.655142+0000","last_became_peered":"2024-04-26T01:49:37.655142+0000","last_unstale":"2024-04-26T01:51:25.408158+0000","last_undegraded":"2024-04-26T01:51:25.408158+0000","last_fullsized":"2024-04-26T01:51:25.408158+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:02:58.090885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.485822+0000","last_change":"2024-04-26T01:49:33.599578+0000","last_active":"2024-04-26T01:50:27.485822+0000","last_peered":"2024-04-26T01:50:27.485822+0000","last_clean":"2024-04-26T01:50:27.485822+0000","last_became_active":"2024-04-26T01:49:33.599010+0000","last_became_peered":"2024-04-26T01:49:33.599010+0000","last_unstale":"2024-04-26T01:50:27.485822+0000","last_undegraded":"2024-04-26T01:50:27.485822+0000","last_fullsized":"2024-04-26T01:50:27.485822+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:16:09.925304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":360,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998346+0000","last_change":"2024-04-26T01:47:59.584819+0000","last_active":"2024-04-26T01:50:26.998346+0000","last_peered":"2024-04-26T01:50:26.998346+0000","last_clean":"2024-04-26T01:50:26.998346+0000","last_became_active":"2024-04-26T01:47:59.584356+0000","last_became_peered":"2024-04-26T01:47:59.584356+0000","last_unstale":"2024-04-26T01:50:26.998346+0000","last_undegraded":"2024-04-26T01:50:26.998346+0000","last_fullsized":"2024-04-26T01:50:26.998346+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:44:11.653568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:44:11.653568+0000","last_clean_scrub_stamp":"2024-04-26T01:44:11.653568+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:25:11.168583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.403915+0000","last_change":"2024-04-26T01:50:27.008911+0000","last_active":"2024-04-26T01:51:25.403915+0000","last_peered":"2024-04-26T01:51:25.403915+0000","last_clean":"2024-04-26T01:51:25.403915+0000","last_became_active":"2024-04-26T01:50:27.008507+0000","last_became_peered":"2024-04-26T01:50:27.008507+0000","last_unstale":"2024-04-26T01:51:25.403915+0000","last_undegraded":"2024-04-26T01:51:25.403915+0000","last_fullsized":"2024-04-26T01:51:25.403915+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:21:14.985935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998684+0000","last_change":"2024-04-26T01:49:33.590171+0000","last_active":"2024-04-26T01:50:26.998684+0000","last_peered":"2024-04-26T01:50:26.998684+0000","last_clean":"2024-04-26T01:50:26.998684+0000","last_became_active":"2024-04-26T01:49:33.589312+0000","last_became_peered":"2024-04-26T01:49:33.589312+0000","last_unstale":"2024-04-26T01:50:26.998684+0000","last_undegraded":"2024-04-26T01:50:26.998684+0000","last_fullsized":"2024-04-26T01:50:26.998684+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:01:25.441527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998802+0000","last_change":"2024-04-26T01:49:34.190338+0000","last_active":"2024-04-26T01:50:26.998802+0000","last_peered":"2024-04-26T01:50:26.998802+0000","last_clean":"2024-04-26T01:50:26.998802+0000","last_became_active":"2024-04-26T01:49:34.189984+0000","last_became_peered":"2024-04-26T01:49:34.189984+0000","last_unstale":"2024-04-26T01:50:26.998802+0000","last_undegraded":"2024-04-26T01:50:26.998802+0000","last_fullsized":"2024-04-26T01:50:26.998802+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:58.588012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.406676+0000","last_change":"2024-04-26T01:49:37.649368+0000","last_active":"2024-04-26T01:51:25.406676+0000","last_peered":"2024-04-26T01:51:25.406676+0000","last_clean":"2024-04-26T01:51:25.406676+0000","last_became_active":"2024-04-26T01:49:37.648987+0000","last_became_peered":"2024-04-26T01:49:37.648987+0000","last_unstale":"2024-04-26T01:51:25.406676+0000","last_undegraded":"2024-04-26T01:51:25.406676+0000","last_fullsized":"2024-04-26T01:51:25.406676+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:04:59.080703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"92'12","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.696507+0000","last_change":"2024-04-26T01:49:33.600813+0000","last_active":"2024-04-26T01:51:25.696507+0000","last_peered":"2024-04-26T01:51:25.696507+0000","last_clean":"2024-04-26T01:51:25.696507+0000","last_became_active":"2024-04-26T01:49:33.600361+0000","last_became_peered":"2024-04-26T01:49:33.600361+0000","last_unstale":"2024-04-26T01:51:25.696507+0000","last_undegraded":"2024-04-26T01:51:25.696507+0000","last_fullsized":"2024-04-26T01:51:25.696507+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:02:58.638449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.414301+0000","last_change":"2024-04-26T01:49:37.659381+0000","last_active":"2024-04-26T01:51:25.414301+0000","last_peered":"2024-04-26T01:51:25.414301+0000","last_clean":"2024-04-26T01:51:25.414301+0000","last_became_active":"2024-04-26T01:49:37.656205+0000","last_became_peered":"2024-04-26T01:49:37.656205+0000","last_unstale":"2024-04-26T01:51:25.414301+0000","last_undegraded":"2024-04-26T01:51:25.414301+0000","last_fullsized":"2024-04-26T01:51:25.414301+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:02:10.103330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.486077+0000","last_change":"2024-04-26T01:49:33.854732+0000","last_active":"2024-04-26T01:50:27.486077+0000","last_peered":"2024-04-26T01:50:27.486077+0000","last_clean":"2024-04-26T01:50:27.486077+0000","last_became_active":"2024-04-26T01:49:33.854396+0000","last_became_peered":"2024-04-26T01:49:33.854396+0000","last_unstale":"2024-04-26T01:50:27.486077+0000","last_undegraded":"2024-04-26T01:50:27.486077+0000","last_fullsized":"2024-04-26T01:50:27.486077+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:12:37.183725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.403468+0000","last_change":"2024-04-26T01:49:37.656366+0000","last_active":"2024-04-26T01:51:25.403468+0000","last_peered":"2024-04-26T01:51:25.403468+0000","last_clean":"2024-04-26T01:51:25.403468+0000","last_became_active":"2024-04-26T01:49:37.655869+0000","last_became_peered":"2024-04-26T01:49:37.655869+0000","last_unstale":"2024-04-26T01:51:25.403468+0000","last_undegraded":"2024-04-26T01:51:25.403468+0000","last_fullsized":"2024-04-26T01:51:25.403468+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:01:57.083315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.837207+0000","last_change":"2024-04-26T01:49:34.186070+0000","last_active":"2024-04-26T01:50:27.837207+0000","last_peered":"2024-04-26T01:50:27.837207+0000","last_clean":"2024-04-26T01:50:27.837207+0000","last_became_active":"2024-04-26T01:49:34.185742+0000","last_became_peered":"2024-04-26T01:49:34.185742+0000","last_unstale":"2024-04-26T01:50:27.837207+0000","last_undegraded":"2024-04-26T01:50:27.837207+0000","last_fullsized":"2024-04-26T01:50:27.837207+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:05.870056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.376526+0000","last_change":"2024-04-26T01:49:37.640938+0000","last_active":"2024-04-26T01:51:25.376526+0000","last_peered":"2024-04-26T01:51:25.376526+0000","last_clean":"2024-04-26T01:51:25.376526+0000","last_became_active":"2024-04-26T01:49:37.640608+0000","last_became_peered":"2024-04-26T01:49:37.640608+0000","last_unstale":"2024-04-26T01:51:25.376526+0000","last_undegraded":"2024-04-26T01:51:25.376526+0000","last_fullsized":"2024-04-26T01:51:25.376526+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:30:30.460641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.991984+0000","last_change":"2024-04-26T01:49:33.604152+0000","last_active":"2024-04-26T01:50:26.991984+0000","last_peered":"2024-04-26T01:50:26.991984+0000","last_clean":"2024-04-26T01:50:26.991984+0000","last_became_active":"2024-04-26T01:49:33.603479+0000","last_became_peered":"2024-04-26T01:49:33.603479+0000","last_unstale":"2024-04-26T01:50:26.991984+0000","last_undegraded":"2024-04-26T01:50:26.991984+0000","last_fullsized":"2024-04-26T01:50:26.991984+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:25:13.685392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.379393+0000","last_change":"2024-04-26T01:49:37.645152+0000","last_active":"2024-04-26T01:51:25.379393+0000","last_peered":"2024-04-26T01:51:25.379393+0000","last_clean":"2024-04-26T01:51:25.379393+0000","last_became_active":"2024-04-26T01:49:37.644789+0000","last_became_peered":"2024-04-26T01:49:37.644789+0000","last_unstale":"2024-04-26T01:51:25.379393+0000","last_undegraded":"2024-04-26T01:51:25.379393+0000","last_fullsized":"2024-04-26T01:51:25.379393+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:20:01.494222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.010485+0000","last_change":"2024-04-26T01:50:27.009973+0000","last_active":"2024-04-26T01:50:27.010485+0000","last_peered":"2024-04-26T01:50:27.010485+0000","last_clean":"2024-04-26T01:50:27.010485+0000","last_became_active":"2024-04-26T01:50:27.001678+0000","last_became_peered":"2024-04-26T01:50:27.001678+0000","last_unstale":"2024-04-26T01:50:27.010485+0000","last_undegraded":"2024-04-26T01:50:27.010485+0000","last_fullsized":"2024-04-26T01:50:27.010485+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:59:54.807792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.413791+0000","last_change":"2024-04-26T01:49:37.662665+0000","last_active":"2024-04-26T01:51:25.413791+0000","last_peered":"2024-04-26T01:51:25.413791+0000","last_clean":"2024-04-26T01:51:25.413791+0000","last_became_active":"2024-04-26T01:49:37.662216+0000","last_became_peered":"2024-04-26T01:49:37.662216+0000","last_unstale":"2024-04-26T01:51:25.413791+0000","last_undegraded":"2024-04-26T01:51:25.413791+0000","last_fullsized":"2024-04-26T01:51:25.413791+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:10:54.724858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:26.998379+0000","last_change":"2024-04-26T01:49:33.592078+0000","last_active":"2024-04-26T01:50:26.998379+0000","last_peered":"2024-04-26T01:50:26.998379+0000","last_clean":"2024-04-26T01:50:26.998379+0000","last_became_active":"2024-04-26T01:49:33.590265+0000","last_became_peered":"2024-04-26T01:49:33.590265+0000","last_unstale":"2024-04-26T01:50:26.998379+0000","last_undegraded":"2024-04-26T01:50:26.998379+0000","last_fullsized":"2024-04-26T01:50:26.998379+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:44:27.163705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.392973+0000","last_change":"2024-04-26T01:49:37.645943+0000","last_active":"2024-04-26T01:51:25.392973+0000","last_peered":"2024-04-26T01:51:25.392973+0000","last_clean":"2024-04-26T01:51:25.392973+0000","last_became_active":"2024-04-26T01:49:37.645586+0000","last_became_peered":"2024-04-26T01:49:37.645586+0000","last_unstale":"2024-04-26T01:51:25.392973+0000","last_undegraded":"2024-04-26T01:51:25.392973+0000","last_fullsized":"2024-04-26T01:51:25.392973+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:40.682344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.000264+0000","last_change":"2024-04-26T01:49:34.187115+0000","last_active":"2024-04-26T01:50:27.000264+0000","last_peered":"2024-04-26T01:50:27.000264+0000","last_clean":"2024-04-26T01:50:27.000264+0000","last_became_active":"2024-04-26T01:49:34.186660+0000","last_became_peered":"2024-04-26T01:49:34.186660+0000","last_unstale":"2024-04-26T01:50:27.000264+0000","last_undegraded":"2024-04-26T01:50:27.000264+0000","last_fullsized":"2024-04-26T01:50:27.000264+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:56:32.274752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.384545+0000","last_change":"2024-04-26T01:49:37.650382+0000","last_active":"2024-04-26T01:51:25.384545+0000","last_peered":"2024-04-26T01:51:25.384545+0000","last_clean":"2024-04-26T01:51:25.384545+0000","last_became_active":"2024-04-26T01:49:37.649545+0000","last_became_peered":"2024-04-26T01:49:37.649545+0000","last_unstale":"2024-04-26T01:51:25.384545+0000","last_undegraded":"2024-04-26T01:51:25.384545+0000","last_fullsized":"2024-04-26T01:51:25.384545+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:43:16.291311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'1","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.069019+0000","last_change":"2024-04-26T01:49:33.595881+0000","last_active":"2024-04-26T01:50:27.069019+0000","last_peered":"2024-04-26T01:50:27.069019+0000","last_clean":"2024-04-26T01:50:27.069019+0000","last_became_active":"2024-04-26T01:49:33.595480+0000","last_became_peered":"2024-04-26T01:49:33.595480+0000","last_unstale":"2024-04-26T01:50:27.069019+0000","last_undegraded":"2024-04-26T01:50:27.069019+0000","last_fullsized":"2024-04-26T01:50:27.069019+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:30:58.028230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.403827+0000","last_change":"2024-04-26T01:49:37.660765+0000","last_active":"2024-04-26T01:51:25.403827+0000","last_peered":"2024-04-26T01:51:25.403827+0000","last_clean":"2024-04-26T01:51:25.403827+0000","last_became_active":"2024-04-26T01:49:37.660475+0000","last_became_peered":"2024-04-26T01:49:37.660475+0000","last_unstale":"2024-04-26T01:51:25.403827+0000","last_undegraded":"2024-04-26T01:51:25.403827+0000","last_fullsized":"2024-04-26T01:51:25.403827+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:36:00.483005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:27.006246+0000","last_change":"2024-04-26T01:49:33.598247+0000","last_active":"2024-04-26T01:50:27.006246+0000","last_peered":"2024-04-26T01:50:27.006246+0000","last_clean":"2024-04-26T01:50:27.006246+0000","last_became_active":"2024-04-26T01:49:33.597796+0000","last_became_peered":"2024-04-26T01:49:33.597796+0000","last_unstale":"2024-04-26T01:50:27.006246+0000","last_undegraded":"2024-04-26T01:50:27.006246+0000","last_fullsized":"2024-04-26T01:50:27.006246+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:53:38.994798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.380230+0000","last_change":"2024-04-26T01:49:37.648903+0000","last_active":"2024-04-26T01:51:25.380230+0000","last_peered":"2024-04-26T01:51:25.380230+0000","last_clean":"2024-04-26T01:51:25.380230+0000","last_became_active":"2024-04-26T01:49:37.648570+0000","last_became_peered":"2024-04-26T01:49:37.648570+0000","last_unstale":"2024-04-26T01:51:25.380230+0000","last_undegraded":"2024-04-26T01:51:25.380230+0000","last_fullsized":"2024-04-26T01:51:25.380230+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:33:12.678513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.379320+0000","last_change":"2024-04-26T01:49:37.648485+0000","last_active":"2024-04-26T01:51:25.379320+0000","last_peered":"2024-04-26T01:51:25.379320+0000","last_clean":"2024-04-26T01:51:25.379320+0000","last_became_active":"2024-04-26T01:49:37.648129+0000","last_became_peered":"2024-04-26T01:49:37.648129+0000","last_unstale":"2024-04-26T01:51:25.379320+0000","last_undegraded":"2024-04-26T01:51:25.379320+0000","last_fullsized":"2024-04-26T01:51:25.379320+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:56:12.047821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.376376+0000","last_change":"2024-04-26T01:49:37.645929+0000","last_active":"2024-04-26T01:51:25.376376+0000","last_peered":"2024-04-26T01:51:25.376376+0000","last_clean":"2024-04-26T01:51:25.376376+0000","last_became_active":"2024-04-26T01:49:37.645473+0000","last_became_peered":"2024-04-26T01:49:37.645473+0000","last_unstale":"2024-04-26T01:51:25.376376+0000","last_undegraded":"2024-04-26T01:51:25.376376+0000","last_fullsized":"2024-04-26T01:51:25.376376+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:39:56.687973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.403322+0000","last_change":"2024-04-26T01:49:37.657494+0000","last_active":"2024-04-26T01:51:25.403322+0000","last_peered":"2024-04-26T01:51:25.403322+0000","last_clean":"2024-04-26T01:51:25.403322+0000","last_became_active":"2024-04-26T01:49:37.657121+0000","last_became_peered":"2024-04-26T01:49:37.657121+0000","last_unstale":"2024-04-26T01:51:25.403322+0000","last_undegraded":"2024-04-26T01:51:25.403322+0000","last_fullsized":"2024-04-26T01:51:25.403322+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:50:44.691179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.399871+0000","last_change":"2024-04-26T01:49:37.646012+0000","last_active":"2024-04-26T01:51:25.399871+0000","last_peered":"2024-04-26T01:51:25.399871+0000","last_clean":"2024-04-26T01:51:25.399871+0000","last_became_active":"2024-04-26T01:49:37.645695+0000","last_became_peered":"2024-04-26T01:49:37.645695+0000","last_unstale":"2024-04-26T01:51:25.399871+0000","last_undegraded":"2024-04-26T01:51:25.399871+0000","last_fullsized":"2024-04-26T01:51:25.399871+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:40:51.117523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.402038+0000","last_change":"2024-04-26T01:49:37.646503+0000","last_active":"2024-04-26T01:51:25.402038+0000","last_peered":"2024-04-26T01:51:25.402038+0000","last_clean":"2024-04-26T01:51:25.402038+0000","last_became_active":"2024-04-26T01:49:37.645200+0000","last_became_peered":"2024-04-26T01:49:37.645200+0000","last_unstale":"2024-04-26T01:51:25.402038+0000","last_undegraded":"2024-04-26T01:51:25.402038+0000","last_fullsized":"2024-04-26T01:51:25.402038+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:31:10.957199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.397650+0000","last_change":"2024-04-26T01:50:27.002908+0000","last_active":"2024-04-26T01:51:25.397650+0000","last_peered":"2024-04-26T01:51:25.397650+0000","last_clean":"2024-04-26T01:51:25.397650+0000","last_became_active":"2024-04-26T01:50:27.002454+0000","last_became_peered":"2024-04-26T01:50:27.002454+0000","last_unstale":"2024-04-26T01:51:25.397650+0000","last_undegraded":"2024-04-26T01:51:25.397650+0000","last_fullsized":"2024-04-26T01:51:25.397650+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:18:01.848427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.383409+0000","last_change":"2024-04-26T01:49:37.642676+0000","last_active":"2024-04-26T01:51:25.383409+0000","last_peered":"2024-04-26T01:51:25.383409+0000","last_clean":"2024-04-26T01:51:25.383409+0000","last_became_active":"2024-04-26T01:49:37.642302+0000","last_became_peered":"2024-04-26T01:49:37.642302+0000","last_unstale":"2024-04-26T01:51:25.383409+0000","last_undegraded":"2024-04-26T01:51:25.383409+0000","last_fullsized":"2024-04-26T01:51:25.383409+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:51:43.454649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.413280+0000","last_change":"2024-04-26T01:50:27.003914+0000","last_active":"2024-04-26T01:51:25.413280+0000","last_peered":"2024-04-26T01:51:25.413280+0000","last_clean":"2024-04-26T01:51:25.413280+0000","last_became_active":"2024-04-26T01:50:27.002344+0000","last_became_peered":"2024-04-26T01:50:27.002344+0000","last_unstale":"2024-04-26T01:51:25.413280+0000","last_undegraded":"2024-04-26T01:51:25.413280+0000","last_fullsized":"2024-04-26T01:51:25.413280+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:46:02.101508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.416372+0000","last_change":"2024-04-26T01:49:37.663306+0000","last_active":"2024-04-26T01:51:25.416372+0000","last_peered":"2024-04-26T01:51:25.416372+0000","last_clean":"2024-04-26T01:51:25.416372+0000","last_became_active":"2024-04-26T01:49:37.662999+0000","last_became_peered":"2024-04-26T01:49:37.662999+0000","last_unstale":"2024-04-26T01:51:25.416372+0000","last_undegraded":"2024-04-26T01:51:25.416372+0000","last_fullsized":"2024-04-26T01:51:25.416372+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:55.155096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.414001+0000","last_change":"2024-04-26T01:49:37.655157+0000","last_active":"2024-04-26T01:51:25.414001+0000","last_peered":"2024-04-26T01:51:25.414001+0000","last_clean":"2024-04-26T01:51:25.414001+0000","last_became_active":"2024-04-26T01:49:37.654810+0000","last_became_peered":"2024-04-26T01:49:37.654810+0000","last_unstale":"2024-04-26T01:51:25.414001+0000","last_undegraded":"2024-04-26T01:51:25.414001+0000","last_fullsized":"2024-04-26T01:51:25.414001+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:28:04.012382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.412653+0000","last_change":"2024-04-26T01:49:37.667228+0000","last_active":"2024-04-26T01:51:25.412653+0000","last_peered":"2024-04-26T01:51:25.412653+0000","last_clean":"2024-04-26T01:51:25.412653+0000","last_became_active":"2024-04-26T01:49:37.666835+0000","last_became_peered":"2024-04-26T01:49:37.666835+0000","last_unstale":"2024-04-26T01:51:25.412653+0000","last_undegraded":"2024-04-26T01:51:25.412653+0000","last_fullsized":"2024-04-26T01:51:25.412653+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:54:16.902836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.400335+0000","last_change":"2024-04-26T01:49:37.658231+0000","last_active":"2024-04-26T01:51:25.400335+0000","last_peered":"2024-04-26T01:51:25.400335+0000","last_clean":"2024-04-26T01:51:25.400335+0000","last_became_active":"2024-04-26T01:49:37.657842+0000","last_became_peered":"2024-04-26T01:49:37.657842+0000","last_unstale":"2024-04-26T01:51:25.400335+0000","last_undegraded":"2024-04-26T01:51:25.400335+0000","last_fullsized":"2024-04-26T01:51:25.400335+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:02:01.445262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:51:25.380571+0000","last_change":"2024-04-26T01:49:37.645221+0000","last_active":"2024-04-26T01:51:25.380571+0000","last_peered":"2024-04-26T01:51:25.380571+0000","last_clean":"2024-04-26T01:51:25.380571+0000","last_became_active":"2024-04-26T01:49:37.644814+0000","last_became_peered":"2024-04-26T01:49:37.644814+0000","last_unstale":"2024-04-26T01:51:25.380571+0000","last_undegraded":"2024-04-26T01:51:25.380571+0000","last_fullsized":"2024-04-26T01:51:25.380571+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:03:17.850633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65270,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":751,"num_read_kb":443,"num_write":173,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":195810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147,"ondisk_log_size":147,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318321,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":995328,"data_stored":486380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579955,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28668,"kb_used_data":1652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716484,"statfs":{"total":95995035648,"available":95965679616,"internally_reserved":0,"allocated":1691648,"data_stored":1138989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743481,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1011712,"data_stored":486616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005121,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":486536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266757,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28584,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716568,"statfs":{"total":95995035648,"available":95965765632,"internally_reserved":0,"allocated":1605632,"data_stored":1079139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430283,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":486005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574951,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1060864,"data_stored":546394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738475,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1019904,"data_stored":487276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411426,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1085440,"data_stored":548276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117083,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1015808,"data_stored":486316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855448,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":1003520,"data_stored":486974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691921,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28596,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716556,"statfs":{"total":95995035648,"available":95965753344,"internally_reserved":0,"allocated":1617920,"data_stored":1078291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":60596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T01:52:07.533 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-26T01:52:07.533 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-26T01:52:07.533 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-26T01:52:07.534 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-26T01:52:08.020 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:52:08.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:52:07 smithi172 ceph-mon[32785]: from='client.24832 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:52:08.330 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:52:07 smithi151 ceph-mon[28057]: from='client.24832 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:52:08.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:52:07 smithi165 ceph-mon[32648]: from='client.24832 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:52:09.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:52:08 smithi172 ceph-mon[32785]: pgmap v403: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T01:52:09.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:52:08 smithi165 ceph-mon[32648]: pgmap v403: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T01:52:09.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:52:08 smithi151 ceph-mon[28057]: pgmap v403: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T01:52:10.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:10.367+0000 7fd07f000700 1 -- 172.21.15.151:0/3339938444 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd078101030 msgr2=0x7fd0781014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:10.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:10.367+0000 7fd07f000700 1 --2- 172.21.15.151:0/3339938444 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd078101030 0x7fd0781014b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd064009990 tx=0x7fd064009ca0 comp rx=0 tx=0).stop 2024-04-26T01:52:10.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:10.367+0000 7fd07f000700 1 -- 172.21.15.151:0/3339938444 shutdown_connections 2024-04-26T01:52:10.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:10.367+0000 7fd07f000700 1 --2- 172.21.15.151:0/3339938444 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd078101ad0 0x7fd0781117c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:10.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:10.367+0000 7fd07f000700 1 --2- 172.21.15.151:0/3339938444 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd078103950 0x7fd078103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:10.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:10.367+0000 7fd07f000700 1 --2- 172.21.15.151:0/3339938444 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd078101030 0x7fd0781014b0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:10.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:10.367+0000 7fd07f000700 1 -- 172.21.15.151:0/3339938444 >> 172.21.15.151:0/3339938444 conn(0x7fd078077fb0 msgr2=0x7fd0780ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:10.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:10.367+0000 7fd07f000700 1 -- 172.21.15.151:0/3339938444 shutdown_connections 2024-04-26T01:52:10.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:10.367+0000 7fd07f000700 1 -- 172.21.15.151:0/3339938444 wait complete. 2024-04-26T01:52:10.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:10.368+0000 7fd07f000700 1 Processor -- start 2024-04-26T01:52:10.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:10.369+0000 7fd07f000700 1 -- start start 2024-04-26T01:52:10.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:10.369+0000 7fd07f000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd078101030 0x7fd07819b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:10.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:10.369+0000 7fd07f000700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd078101ad0 0x7fd07819b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:10.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:10.369+0000 7fd07f000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd078103950 0x7fd07819fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:10.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:10.369+0000 7fd07f000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd078114df0 con 0x7fd078101030 2024-04-26T01:52:10.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:10.369+0000 7fd07f000700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd078114c70 con 0x7fd078103950 2024-04-26T01:52:10.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:10.369+0000 7fd07f000700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd078114f70 con 0x7fd078101ad0 2024-04-26T01:52:10.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:10.370+0000 7fd077e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd078101030 0x7fd07819b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:10.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:10.370+0000 7fd077400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd078101ad0 0x7fd07819b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:10.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:10.370+0000 7fd07d000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd078103950 0x7fd07819fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:10.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:10.370+0000 7fd077e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd078101030 0x7fd07819b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49106/0 (socket says 172.21.15.151:49106) 2024-04-26T01:52:10.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:10.370+0000 7fd077e00700 1 -- 172.21.15.151:0/3915682981 learned_addr learned my addr 172.21.15.151:0/3915682981 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:52:10.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:10.370+0000 7fd07d000700 1 -- 172.21.15.151:0/3915682981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd078101ad0 msgr2=0x7fd07819b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:10.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:10.370+0000 7fd07d000700 1 --2- 172.21.15.151:0/3915682981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd078101ad0 0x7fd07819b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:10.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:10.370+0000 7fd07d000700 1 -- 172.21.15.151:0/3915682981 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd078101030 msgr2=0x7fd07819b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:10.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:10.370+0000 7fd07d000700 1 --2- 172.21.15.151:0/3915682981 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd078101030 0x7fd07819b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:10.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:10.370+0000 7fd07d000700 1 -- 172.21.15.151:0/3915682981 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0781a02d0 con 0x7fd078103950 2024-04-26T01:52:10.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:10.370+0000 7fd077400700 1 --2- 172.21.15.151:0/3915682981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd078101ad0 0x7fd07819b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:52:10.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:10.370+0000 7fd07d000700 1 --2- 172.21.15.151:0/3915682981 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd078103950 0x7fd07819fb90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd07000efc0 tx=0x7fd07000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:10.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:10.371+0000 7fd074c00700 1 -- 172.21.15.151:0/3915682981 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0700092e0 con 0x7fd078103950 2024-04-26T01:52:10.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:10.371+0000 7fd074c00700 1 -- 172.21.15.151:0/3915682981 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd070004bb0 con 0x7fd078103950 2024-04-26T01:52:10.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:10.371+0000 7fd07f000700 1 -- 172.21.15.151:0/3915682981 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0781a0560 con 0x7fd078103950 2024-04-26T01:52:10.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:10.371+0000 7fd074c00700 1 -- 172.21.15.151:0/3915682981 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd070004020 con 0x7fd078103950 2024-04-26T01:52:10.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:10.371+0000 7fd07f000700 1 -- 172.21.15.151:0/3915682981 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0781a7e40 con 0x7fd078103950 2024-04-26T01:52:10.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:10.372+0000 7fd07f000700 1 -- 172.21.15.151:0/3915682981 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd078102040 con 0x7fd078103950 2024-04-26T01:52:10.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:10.374+0000 7fd074c00700 1 -- 172.21.15.151:0/3915682981 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7fd07001b030 con 0x7fd078103950 2024-04-26T01:52:10.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:10.375+0000 7fd074c00700 1 --2- 172.21.15.151:0/3915682981 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fd05c0a3c40 0x7fd05c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:10.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:10.376+0000 7fd074c00700 1 -- 172.21.15.151:0/3915682981 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7fd0700d0110 con 0x7fd078103950 2024-04-26T01:52:10.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:10.376+0000 7fd077e00700 1 --2- 172.21.15.151:0/3915682981 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fd05c0a3c40 0x7fd05c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:10.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:10.376+0000 7fd077e00700 1 --2- 172.21.15.151:0/3915682981 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fd05c0a3c40 0x7fd05c0a6100 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd068009680 tx=0x7fd068009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:10.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:10.378+0000 7fd074c00700 1 -- 172.21.15.151:0/3915682981 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd070022020 con 0x7fd078103950 2024-04-26T01:52:10.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:10.660+0000 7fd07f000700 1 -- 172.21.15.151:0/3915682981 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fd07819c620 con 0x7fd078103950 2024-04-26T01:52:10.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:10.661+0000 7fd074c00700 1 -- 172.21.15.151:0/3915682981 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fd070010040 con 0x7fd078103950 2024-04-26T01:52:10.660 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:52:10.660 INFO:teuthology.orchestra.run.smithi151.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-26T01:52:10.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:10.664+0000 7fd07f000700 1 -- 172.21.15.151:0/3915682981 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fd05c0a3c40 msgr2=0x7fd05c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:10.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:10.664+0000 7fd07f000700 1 --2- 172.21.15.151:0/3915682981 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fd05c0a3c40 0x7fd05c0a6100 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd068009680 tx=0x7fd068009260 comp rx=0 tx=0).stop 2024-04-26T01:52:10.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:10.664+0000 7fd07f000700 1 -- 172.21.15.151:0/3915682981 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd078103950 msgr2=0x7fd07819fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:10.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:10.664+0000 7fd07f000700 1 --2- 172.21.15.151:0/3915682981 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd078103950 0x7fd07819fb90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd07000efc0 tx=0x7fd07000c3a0 comp rx=0 tx=0).stop 2024-04-26T01:52:10.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:10.664+0000 7fd07f000700 1 -- 172.21.15.151:0/3915682981 shutdown_connections 2024-04-26T01:52:10.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:10.664+0000 7fd07f000700 1 --2- 172.21.15.151:0/3915682981 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd078103950 0x7fd07819fb90 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:10.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:10.664+0000 7fd07f000700 1 --2- 172.21.15.151:0/3915682981 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fd05c0a3c40 0x7fd05c0a6100 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:10.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:10.664+0000 7fd07f000700 1 --2- 172.21.15.151:0/3915682981 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd078101030 0x7fd07819b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:10.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:10.664+0000 7fd07f000700 1 --2- 172.21.15.151:0/3915682981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd078101ad0 0x7fd07819b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:10.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:10.664+0000 7fd07f000700 1 -- 172.21.15.151:0/3915682981 >> 172.21.15.151:0/3915682981 conn(0x7fd078077fb0 msgr2=0x7fd0780fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:10.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:10.664+0000 7fd07f000700 1 -- 172.21.15.151:0/3915682981 shutdown_connections 2024-04-26T01:52:10.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:10.664+0000 7fd07f000700 1 -- 172.21.15.151:0/3915682981 wait complete. 2024-04-26T01:52:11.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:52:10 smithi172 ceph-mon[32785]: pgmap v404: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:52:11.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:52:10 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/3915682981' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-26T01:52:11.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:52:10 smithi165 ceph-mon[32648]: pgmap v404: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:52:11.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:52:10 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/3915682981' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-26T01:52:11.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:52:10 smithi151 ceph-mon[28057]: pgmap v404: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:52:11.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:52:10 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/3915682981' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-26T01:52:11.548 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-26T01:52:11.549 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:52:12.003 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:52:13.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:52:13 smithi172 ceph-mon[32785]: pgmap v405: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:52:13.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:52:13 smithi165 ceph-mon[32648]: pgmap v405: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:52:13.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:52:13 smithi151 ceph-mon[28057]: pgmap v405: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:52:14.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:14.404+0000 7f219d400700 1 -- 172.21.15.151:0/4039078229 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f21981092d0 msgr2=0x7f21981096b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:14.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:14.404+0000 7f219d400700 1 --2- 172.21.15.151:0/4039078229 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f21981092d0 0x7f21981096b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f21880098f0 tx=0x7f2188009c00 comp rx=0 tx=0).stop 2024-04-26T01:52:14.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:14.404+0000 7f219d400700 1 -- 172.21.15.151:0/4039078229 shutdown_connections 2024-04-26T01:52:14.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:14.404+0000 7f219d400700 1 --2- 172.21.15.151:0/4039078229 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f219810e3b0 0x7f2198115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:14.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:14.404+0000 7f219d400700 1 --2- 172.21.15.151:0/4039078229 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f21981092d0 0x7f21981096b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:14.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:14.404+0000 7f219d400700 1 --2- 172.21.15.151:0/4039078229 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2198109c80 0x7f219810dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:14.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:14.404+0000 7f219d400700 1 -- 172.21.15.151:0/4039078229 >> 172.21.15.151:0/4039078229 conn(0x7f2198076210 msgr2=0x7f2198107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:14.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:14.404+0000 7f219d400700 1 -- 172.21.15.151:0/4039078229 shutdown_connections 2024-04-26T01:52:14.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:14.404+0000 7f219d400700 1 -- 172.21.15.151:0/4039078229 wait complete. 2024-04-26T01:52:14.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:14.405+0000 7f219d400700 1 Processor -- start 2024-04-26T01:52:14.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:14.405+0000 7f219d400700 1 -- start start 2024-04-26T01:52:14.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:14.406+0000 7f219d400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f21981092d0 0x7f21981a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:14.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:14.406+0000 7f219d400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2198109c80 0x7f21981a5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:14.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:14.406+0000 7f219d400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f219810e3b0 0x7f219819f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:14.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:14.406+0000 7f219d400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2198119290 con 0x7f219810e3b0 2024-04-26T01:52:14.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:14.406+0000 7f219d400700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2198119110 con 0x7f21981092d0 2024-04-26T01:52:14.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:14.406+0000 7f219d400700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2198119410 con 0x7f2198109c80 2024-04-26T01:52:14.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:14.406+0000 7f2196800700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f21981092d0 0x7f21981a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:14.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:14.406+0000 7f2195e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2198109c80 0x7f21981a5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:14.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:14.406+0000 7f2197200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f219810e3b0 0x7f219819f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:14.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:14.406+0000 7f2196800700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f21981092d0 0x7f21981a5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:43852/0 (socket says 172.21.15.151:43852) 2024-04-26T01:52:14.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:14.406+0000 7f2195e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2198109c80 0x7f21981a5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:41794/0 (socket says 172.21.15.151:41794) 2024-04-26T01:52:14.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:14.406+0000 7f2196800700 1 -- 172.21.15.151:0/2851156572 learned_addr learned my addr 172.21.15.151:0/2851156572 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:52:14.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:14.407+0000 7f2196800700 1 -- 172.21.15.151:0/2851156572 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2198109c80 msgr2=0x7f21981a5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:14.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:14.407+0000 7f2196800700 1 --2- 172.21.15.151:0/2851156572 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2198109c80 0x7f21981a5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:14.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:14.407+0000 7f2196800700 1 -- 172.21.15.151:0/2851156572 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f219810e3b0 msgr2=0x7f219819f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:14.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:14.407+0000 7f2196800700 1 --2- 172.21.15.151:0/2851156572 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f219810e3b0 0x7f219819f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:14.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:14.407+0000 7f2196800700 1 -- 172.21.15.151:0/2851156572 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f219819fd30 con 0x7f21981092d0 2024-04-26T01:52:14.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:14.407+0000 7f2196800700 1 --2- 172.21.15.151:0/2851156572 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f21981092d0 0x7f21981a5410 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2188009230 tx=0x7f218800fb40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:14.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:14.407+0000 7f218f400700 1 -- 172.21.15.151:0/2851156572 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2188005220 con 0x7f21981092d0 2024-04-26T01:52:14.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:14.407+0000 7f218f400700 1 -- 172.21.15.151:0/2851156572 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f218800b560 con 0x7f21981092d0 2024-04-26T01:52:14.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:14.407+0000 7f219d400700 1 -- 172.21.15.151:0/2851156572 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f219819ffc0 con 0x7f21981092d0 2024-04-26T01:52:14.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:14.407+0000 7f218f400700 1 -- 172.21.15.151:0/2851156572 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2188003a60 con 0x7f21981092d0 2024-04-26T01:52:14.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:14.408+0000 7f219d400700 1 -- 172.21.15.151:0/2851156572 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21981ac2e0 con 0x7f21981092d0 2024-04-26T01:52:14.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:14.409+0000 7f219d400700 1 -- 172.21.15.151:0/2851156572 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2198069730 con 0x7f21981092d0 2024-04-26T01:52:14.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:14.411+0000 7f218f400700 1 -- 172.21.15.151:0/2851156572 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f2188003c80 con 0x7f21981092d0 2024-04-26T01:52:14.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:14.412+0000 7f218f400700 1 --2- 172.21.15.151:0/2851156572 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f217c0a3cc0 0x7f217c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:14.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:14.412+0000 7f218f400700 1 -- 172.21.15.151:0/2851156572 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f21880d1350 con 0x7f21981092d0 2024-04-26T01:52:14.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:14.412+0000 7f2195e00700 1 --2- 172.21.15.151:0/2851156572 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f217c0a3cc0 0x7f217c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:14.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:14.413+0000 7f2195e00700 1 --2- 172.21.15.151:0/2851156572 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f217c0a3cc0 0x7f217c0a6180 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f21980782a0 tx=0x7f2184009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:14.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:14.414+0000 7f218f400700 1 -- 172.21.15.151:0/2851156572 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2188013030 con 0x7f21981092d0 2024-04-26T01:52:14.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:14.689+0000 7f219d400700 1 -- 172.21.15.151:0/2851156572 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f219802df80 con 0x7f21981092d0 2024-04-26T01:52:14.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:14.691+0000 7f218f400700 1 -- 172.21.15.151:0/2851156572 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11889 (secure 0 0 0) 0x7f218809ddc0 con 0x7f21981092d0 2024-04-26T01:52:14.689 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:52:14.689 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2937864097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2937864097},{"type":"v1","addr":"172.21.15.151:6835","nonce":2937864097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14373,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/2947090802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2947090802},{"type":"v1","addr":"172.21.15.151:6837","nonce":2947090802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/1356563280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1356563280},{"type":"v1","addr":"172.21.15.151:6839","nonce":1356563280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6841/1075240552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1075240552},{"type":"v1","addr":"172.21.15.151:6841","nonce":1075240552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/504675912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":504675912},{"type":"v1","addr":"172.21.15.165:6833","nonce":504675912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1207580015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1207580015},{"type":"v1","addr":"172.21.15.172:6833","nonce":1207580015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:49:40.746876+0000","modified":"2024-04-26T01:50:05.608353+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24505,"mds_2":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/3598138294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3598138294},{"type":"v1","addr":"172.21.15.165:6835","nonce":3598138294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/1847859054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":1847859054},{"type":"v1","addr":"172.21.15.165:6837","nonce":1847859054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/2309751171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2309751171},{"type":"v1","addr":"172.21.15.172:6835","nonce":2309751171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.172:6837/55972537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":55972537},{"type":"v1","addr":"172.21.15.172:6837","nonce":55972537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.172:6839/1242302810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1242302810},{"type":"v1","addr":"172.21.15.172:6839","nonce":1242302810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/1969716254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1969716254},{"type":"v1","addr":"172.21.15.165:6839","nonce":1969716254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24505,24509,24493]},"id":1}]} 2024-04-26T01:52:14.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:14.693+0000 7f219d400700 1 -- 172.21.15.151:0/2851156572 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f217c0a3cc0 msgr2=0x7f217c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:14.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:14.693+0000 7f219d400700 1 --2- 172.21.15.151:0/2851156572 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f217c0a3cc0 0x7f217c0a6180 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f21980782a0 tx=0x7f2184009310 comp rx=0 tx=0).stop 2024-04-26T01:52:14.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:14.693+0000 7f219d400700 1 -- 172.21.15.151:0/2851156572 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f21981092d0 msgr2=0x7f21981a5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:14.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:14.693+0000 7f219d400700 1 --2- 172.21.15.151:0/2851156572 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f21981092d0 0x7f21981a5410 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2188009230 tx=0x7f218800fb40 comp rx=0 tx=0).stop 2024-04-26T01:52:14.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:14.694+0000 7f219d400700 1 -- 172.21.15.151:0/2851156572 shutdown_connections 2024-04-26T01:52:14.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:14.694+0000 7f219d400700 1 --2- 172.21.15.151:0/2851156572 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f217c0a3cc0 0x7f217c0a6180 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:14.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:14.694+0000 7f219d400700 1 --2- 172.21.15.151:0/2851156572 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f219810e3b0 0x7f219819f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:14.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:14.694+0000 7f219d400700 1 --2- 172.21.15.151:0/2851156572 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f21981092d0 0x7f21981a5410 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:14.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:14.694+0000 7f219d400700 1 --2- 172.21.15.151:0/2851156572 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2198109c80 0x7f21981a5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:14.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:14.694+0000 7f219d400700 1 -- 172.21.15.151:0/2851156572 >> 172.21.15.151:0/2851156572 conn(0x7f2198076210 msgr2=0x7f219810d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:14.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:14.694+0000 7f219d400700 1 -- 172.21.15.151:0/2851156572 shutdown_connections 2024-04-26T01:52:14.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:14.694+0000 7f219d400700 1 -- 172.21.15.151:0/2851156572 wait complete. 2024-04-26T01:52:14.693 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 25 2024-04-26T01:52:15.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:52:15 smithi172 ceph-mon[32785]: pgmap v406: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:52:15.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:52:15 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/2851156572' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:52:15.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:52:15 smithi165 ceph-mon[32648]: pgmap v406: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:52:15.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:52:15 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/2851156572' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:52:15.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:52:15 smithi151 ceph-mon[28057]: pgmap v406: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:52:15.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:52:15 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/2851156572' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:52:15.525 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:52:15.975 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:52:17.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:52:17 smithi172 ceph-mon[32785]: pgmap v407: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:52:17.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:52:17 smithi165 ceph-mon[32648]: pgmap v407: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:52:17.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:52:17 smithi151 ceph-mon[28057]: pgmap v407: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:52:18.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:18.475+0000 7f343b000700 1 -- 172.21.15.151:0/347547259 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3434074c70 msgr2=0x7f3434075050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:18.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:18.475+0000 7f343b000700 1 --2- 172.21.15.151:0/347547259 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3434074c70 0x7f3434075050 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f34240098f0 tx=0x7f3424009c00 comp rx=0 tx=0).stop 2024-04-26T01:52:18.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:18.476+0000 7f343b000700 1 -- 172.21.15.151:0/347547259 shutdown_connections 2024-04-26T01:52:18.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:18.476+0000 7f343b000700 1 --2- 172.21.15.151:0/347547259 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f343410f470 0x7f3434111860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:18.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:18.476+0000 7f343b000700 1 --2- 172.21.15.151:0/347547259 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3434074c70 0x7f3434075050 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:18.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:18.476+0000 7f343b000700 1 --2- 172.21.15.151:0/347547259 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3434075590 0x7f343410ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:18.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:18.476+0000 7f343b000700 1 -- 172.21.15.151:0/347547259 >> 172.21.15.151:0/347547259 conn(0x7f3434079b10 msgr2=0x7f3434079f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:18.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:18.476+0000 7f343b000700 1 -- 172.21.15.151:0/347547259 shutdown_connections 2024-04-26T01:52:18.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:18.476+0000 7f343b000700 1 -- 172.21.15.151:0/347547259 wait complete. 2024-04-26T01:52:18.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:18.477+0000 7f343b000700 1 Processor -- start 2024-04-26T01:52:18.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:18.477+0000 7f343b000700 1 -- start start 2024-04-26T01:52:18.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:18.478+0000 7f343b000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3434074c70 0x7f343419b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:18.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:18.478+0000 7f343b000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3434075590 0x7f343419b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:18.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:18.478+0000 7f343b000700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f343410f470 0x7f343419fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:18.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:18.478+0000 7f343b000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3434114e40 con 0x7f3434074c70 2024-04-26T01:52:18.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:18.478+0000 7f343b000700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3434114cc0 con 0x7f3434075590 2024-04-26T01:52:18.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:18.478+0000 7f343b000700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3434114fc0 con 0x7f343410f470 2024-04-26T01:52:18.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:18.478+0000 7f3439000700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f343410f470 0x7f343419fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:18.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:18.478+0000 7f3433400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3434075590 0x7f343419b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:18.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:18.478+0000 7f3433e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3434074c70 0x7f343419b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:18.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:18.478+0000 7f3439000700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f343410f470 0x7f343419fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:41800/0 (socket says 172.21.15.151:41800) 2024-04-26T01:52:18.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:18.478+0000 7f3439000700 1 -- 172.21.15.151:0/542661673 learned_addr learned my addr 172.21.15.151:0/542661673 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:52:18.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:18.479+0000 7f3439000700 1 -- 172.21.15.151:0/542661673 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3434075590 msgr2=0x7f343419b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:18.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:18.479+0000 7f3439000700 1 --2- 172.21.15.151:0/542661673 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3434075590 0x7f343419b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:18.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:18.479+0000 7f3439000700 1 -- 172.21.15.151:0/542661673 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3434074c70 msgr2=0x7f343419b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:18.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:18.479+0000 7f3439000700 1 --2- 172.21.15.151:0/542661673 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3434074c70 0x7f343419b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:18.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:18.479+0000 7f3439000700 1 -- 172.21.15.151:0/542661673 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34341a0300 con 0x7f343410f470 2024-04-26T01:52:18.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:18.479+0000 7f3433400700 1 --2- 172.21.15.151:0/542661673 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3434075590 0x7f343419b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:52:18.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:18.479+0000 7f3439000700 1 --2- 172.21.15.151:0/542661673 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f343410f470 0x7f343419fbc0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f342c00ef90 tx=0x7f342c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:18.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:18.479+0000 7f3430c00700 1 -- 172.21.15.151:0/542661673 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342c0092e0 con 0x7f343410f470 2024-04-26T01:52:18.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:18.480+0000 7f343b000700 1 -- 172.21.15.151:0/542661673 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34341a0590 con 0x7f343410f470 2024-04-26T01:52:18.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:18.480+0000 7f3430c00700 1 -- 172.21.15.151:0/542661673 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f342c004bb0 con 0x7f343410f470 2024-04-26T01:52:18.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:18.480+0000 7f3430c00700 1 -- 172.21.15.151:0/542661673 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342c004020 con 0x7f343410f470 2024-04-26T01:52:18.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:18.480+0000 7f343b000700 1 -- 172.21.15.151:0/542661673 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34341a7e70 con 0x7f343410f470 2024-04-26T01:52:18.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:18.481+0000 7f343b000700 1 -- 172.21.15.151:0/542661673 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3434069730 con 0x7f343410f470 2024-04-26T01:52:18.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:18.483+0000 7f3430c00700 1 -- 172.21.15.151:0/542661673 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f342c01b030 con 0x7f343410f470 2024-04-26T01:52:18.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:18.484+0000 7f3430c00700 1 --2- 172.21.15.151:0/542661673 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f34180a3c40 0x7f34180a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:18.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:18.484+0000 7f3430c00700 1 -- 172.21.15.151:0/542661673 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f342c0d0130 con 0x7f343410f470 2024-04-26T01:52:18.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:18.484+0000 7f3433e00700 1 --2- 172.21.15.151:0/542661673 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f34180a3c40 0x7f34180a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:18.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:18.485+0000 7f3433e00700 1 --2- 172.21.15.151:0/542661673 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f34180a3c40 0x7f34180a6100 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3424009fd0 tx=0x7f3424019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:18.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:18.487+0000 7f3430c00700 1 -- 172.21.15.151:0/542661673 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f342c022020 con 0x7f343410f470 2024-04-26T01:52:18.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:18.759+0000 7f343b000700 1 -- 172.21.15.151:0/542661673 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f343402df80 con 0x7f343410f470 2024-04-26T01:52:18.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:18.760+0000 7f3430c00700 1 -- 172.21.15.151:0/542661673 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11889 (secure 0 0 0) 0x7f342c010040 con 0x7f343410f470 2024-04-26T01:52:18.758 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:52:18.759 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2937864097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2937864097},{"type":"v1","addr":"172.21.15.151:6835","nonce":2937864097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14373,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/2947090802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2947090802},{"type":"v1","addr":"172.21.15.151:6837","nonce":2947090802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/1356563280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1356563280},{"type":"v1","addr":"172.21.15.151:6839","nonce":1356563280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6841/1075240552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1075240552},{"type":"v1","addr":"172.21.15.151:6841","nonce":1075240552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/504675912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":504675912},{"type":"v1","addr":"172.21.15.165:6833","nonce":504675912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1207580015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1207580015},{"type":"v1","addr":"172.21.15.172:6833","nonce":1207580015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:49:40.746876+0000","modified":"2024-04-26T01:50:05.608353+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24505,"mds_2":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/3598138294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3598138294},{"type":"v1","addr":"172.21.15.165:6835","nonce":3598138294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/1847859054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":1847859054},{"type":"v1","addr":"172.21.15.165:6837","nonce":1847859054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/2309751171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2309751171},{"type":"v1","addr":"172.21.15.172:6835","nonce":2309751171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.172:6837/55972537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":55972537},{"type":"v1","addr":"172.21.15.172:6837","nonce":55972537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.172:6839/1242302810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1242302810},{"type":"v1","addr":"172.21.15.172:6839","nonce":1242302810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/1969716254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1969716254},{"type":"v1","addr":"172.21.15.165:6839","nonce":1969716254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24505,24509,24493]},"id":1}]} 2024-04-26T01:52:18.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:18.763+0000 7f343b000700 1 -- 172.21.15.151:0/542661673 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f34180a3c40 msgr2=0x7f34180a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:18.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:18.763+0000 7f343b000700 1 --2- 172.21.15.151:0/542661673 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f34180a3c40 0x7f34180a6100 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3424009fd0 tx=0x7f3424019040 comp rx=0 tx=0).stop 2024-04-26T01:52:18.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:18.763+0000 7f343b000700 1 -- 172.21.15.151:0/542661673 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f343410f470 msgr2=0x7f343419fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:18.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:18.763+0000 7f343b000700 1 --2- 172.21.15.151:0/542661673 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f343410f470 0x7f343419fbc0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f342c00ef90 tx=0x7f342c00c3a0 comp rx=0 tx=0).stop 2024-04-26T01:52:18.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:18.764+0000 7f343b000700 1 -- 172.21.15.151:0/542661673 shutdown_connections 2024-04-26T01:52:18.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:18.764+0000 7f343b000700 1 --2- 172.21.15.151:0/542661673 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f343410f470 0x7f343419fbc0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:18.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:18.764+0000 7f343b000700 1 --2- 172.21.15.151:0/542661673 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f34180a3c40 0x7f34180a6100 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:18.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:18.764+0000 7f343b000700 1 --2- 172.21.15.151:0/542661673 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3434074c70 0x7f343419b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:18.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:18.764+0000 7f343b000700 1 --2- 172.21.15.151:0/542661673 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3434075590 0x7f343419b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:18.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:18.764+0000 7f343b000700 1 -- 172.21.15.151:0/542661673 >> 172.21.15.151:0/542661673 conn(0x7f3434079b10 msgr2=0x7f34340ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:18.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:18.764+0000 7f343b000700 1 -- 172.21.15.151:0/542661673 shutdown_connections 2024-04-26T01:52:18.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:18.764+0000 7f343b000700 1 -- 172.21.15.151:0/542661673 wait complete. 2024-04-26T01:52:18.764 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 25 2024-04-26T01:52:19.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:52:19 smithi172 ceph-mon[32785]: pgmap v408: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T01:52:19.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:52:19 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/542661673' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:52:19.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:52:19 smithi165 ceph-mon[32648]: pgmap v408: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T01:52:19.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:52:19 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/542661673' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:52:19.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:52:19 smithi151 ceph-mon[28057]: pgmap v408: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T01:52:19.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:52:19 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/542661673' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:52:19.585 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T01:52:20.021 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:52:21.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:52:21 smithi172 ceph-mon[32785]: pgmap v409: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:52:21.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:52:21 smithi165 ceph-mon[32648]: pgmap v409: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:52:21.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:52:21 smithi151 ceph-mon[28057]: pgmap v409: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:52:22.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:22.347+0000 7efe32800700 1 -- 172.21.15.151:0/137545505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe2c105780 msgr2=0x7efe2c1097d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:22.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:22.347+0000 7efe32800700 1 --2- 172.21.15.151:0/137545505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe2c105780 0x7efe2c1097d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7efe180099c0 tx=0x7efe18009cd0 comp rx=0 tx=0).stop 2024-04-26T01:52:22.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:22.348+0000 7efe32800700 1 -- 172.21.15.151:0/137545505 shutdown_connections 2024-04-26T01:52:22.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:22.348+0000 7efe32800700 1 --2- 172.21.15.151:0/137545505 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe2c109eb0 0x7efe2c111a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:22.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:22.348+0000 7efe32800700 1 --2- 172.21.15.151:0/137545505 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe2c104dd0 0x7efe2c1051b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:22.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:22.348+0000 7efe32800700 1 --2- 172.21.15.151:0/137545505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe2c105780 0x7efe2c1097d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:22.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:22.348+0000 7efe32800700 1 -- 172.21.15.151:0/137545505 >> 172.21.15.151:0/137545505 conn(0x7efe2c100660 msgr2=0x7efe2c102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:22.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:22.348+0000 7efe32800700 1 -- 172.21.15.151:0/137545505 shutdown_connections 2024-04-26T01:52:22.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:22.348+0000 7efe32800700 1 -- 172.21.15.151:0/137545505 wait complete. 2024-04-26T01:52:22.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:22.349+0000 7efe32800700 1 Processor -- start 2024-04-26T01:52:22.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:22.349+0000 7efe32800700 1 -- start start 2024-04-26T01:52:22.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:22.349+0000 7efe32800700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe2c104dd0 0x7efe2c19b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:22.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:22.349+0000 7efe32800700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe2c105780 0x7efe2c19b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:22.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:22.349+0000 7efe32800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe2c109eb0 0x7efe2c19fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:22.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:22.349+0000 7efe32800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efe2c114d70 con 0x7efe2c109eb0 2024-04-26T01:52:22.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:22.349+0000 7efe32800700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7efe2c114bf0 con 0x7efe2c105780 2024-04-26T01:52:22.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:22.349+0000 7efe32800700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efe2c114ef0 con 0x7efe2c104dd0 2024-04-26T01:52:22.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:22.350+0000 7efe31e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe2c109eb0 0x7efe2c19fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:22.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:22.350+0000 7efe31e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe2c109eb0 0x7efe2c19fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:45314/0 (socket says 172.21.15.151:45314) 2024-04-26T01:52:22.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:22.350+0000 7efe31e00700 1 -- 172.21.15.151:0/2289110909 learned_addr learned my addr 172.21.15.151:0/2289110909 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:52:22.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:22.350+0000 7efe31400700 1 --2- 172.21.15.151:0/2289110909 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe2c104dd0 0x7efe2c19b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:22.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:22.350+0000 7efe30a00700 1 --2- 172.21.15.151:0/2289110909 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe2c105780 0x7efe2c19b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:22.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:22.350+0000 7efe31400700 1 -- 172.21.15.151:0/2289110909 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe2c105780 msgr2=0x7efe2c19b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:22.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:22.350+0000 7efe31400700 1 --2- 172.21.15.151:0/2289110909 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe2c105780 0x7efe2c19b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:22.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:22.350+0000 7efe31400700 1 -- 172.21.15.151:0/2289110909 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe2c109eb0 msgr2=0x7efe2c19fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:22.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:22.350+0000 7efe31400700 1 --2- 172.21.15.151:0/2289110909 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe2c109eb0 0x7efe2c19fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:22.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:22.350+0000 7efe31400700 1 -- 172.21.15.151:0/2289110909 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe2c1a0260 con 0x7efe2c104dd0 2024-04-26T01:52:22.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:22.351+0000 7efe31400700 1 --2- 172.21.15.151:0/2289110909 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe2c104dd0 0x7efe2c19b2a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7efe2000e9f0 tx=0x7efe2000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:22.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:22.351+0000 7efe2a000700 1 -- 172.21.15.151:0/2289110909 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe2000cb70 con 0x7efe2c104dd0 2024-04-26T01:52:22.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:22.351+0000 7efe32800700 1 -- 172.21.15.151:0/2289110909 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe2c1a0550 con 0x7efe2c104dd0 2024-04-26T01:52:22.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:22.351+0000 7efe2a000700 1 -- 172.21.15.151:0/2289110909 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe2000ee90 con 0x7efe2c104dd0 2024-04-26T01:52:22.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:22.351+0000 7efe2a000700 1 -- 172.21.15.151:0/2289110909 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe20003d40 con 0x7efe2c104dd0 2024-04-26T01:52:22.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:22.351+0000 7efe32800700 1 -- 172.21.15.151:0/2289110909 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe2c1a7dd0 con 0x7efe2c104dd0 2024-04-26T01:52:22.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:22.352+0000 7efe32800700 1 -- 172.21.15.151:0/2289110909 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe2c069730 con 0x7efe2c104dd0 2024-04-26T01:52:22.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:22.354+0000 7efe2a000700 1 -- 172.21.15.151:0/2289110909 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7efe20003ee0 con 0x7efe2c104dd0 2024-04-26T01:52:22.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:22.355+0000 7efe2a000700 1 --2- 172.21.15.151:0/2289110909 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7efe140a3c40 0x7efe140a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:22.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:22.355+0000 7efe2a000700 1 -- 172.21.15.151:0/2289110909 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7efe200d0530 con 0x7efe2c104dd0 2024-04-26T01:52:22.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:22.356+0000 7efe30a00700 1 --2- 172.21.15.151:0/2289110909 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7efe140a3c40 0x7efe140a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:22.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:22.356+0000 7efe30a00700 1 --2- 172.21.15.151:0/2289110909 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7efe140a3c40 0x7efe140a6100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7efe18000c00 tx=0x7efe18005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:22.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:22.360+0000 7efe2a000700 1 -- 172.21.15.151:0/2289110909 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efe2009cf10 con 0x7efe2c104dd0 2024-04-26T01:52:22.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:22.589+0000 7efe32800700 1 -- 172.21.15.151:0/2289110909 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efe2c19c5b0 con 0x7efe2c104dd0 2024-04-26T01:52:22.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:22.591+0000 7efe2a000700 1 -- 172.21.15.151:0/2289110909 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+22599 (secure 0 0 0) 0x7efe2009c810 con 0x7efe2c104dd0 2024-04-26T01:52:22.590 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:52:22.591 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":92,"fsid":"c0e49e64-036d-11ef-bc93-c7b262605968","created":"2024-04-26T01:40:07.483197+0000","modified":"2024-04-26T01:50:26.985192+0000","last_up_change":"2024-04-26T01:48:21.167691+0000","last_in_change":"2024-04-26T01:48:00.556990+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T01:44:11.411722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T01:49:31.756032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T01:49:36.529819+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ca27e814-a982-482c-ae1a-e596aac5c59a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":669937998},{"type":"v1","addr":"172.21.15.151:6803","nonce":669937998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":669937998},{"type":"v1","addr":"172.21.15.151:6805","nonce":669937998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":669937998},{"type":"v1","addr":"172.21.15.151:6809","nonce":669937998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":669937998},{"type":"v1","addr":"172.21.15.151:6807","nonce":669937998}]},"public_addr":"172.21.15.151:6803/669937998","cluster_addr":"172.21.15.151:6805/669937998","heartbeat_back_addr":"172.21.15.151:6809/669937998","heartbeat_front_addr":"172.21.15.151:6807/669937998","state":["exists","up"]},{"osd":1,"uuid":"e7d192c3-9624-4625-9ccb-a76c114d8aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":1356168171},{"type":"v1","addr":"172.21.15.165:6801","nonce":1356168171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":1356168171},{"type":"v1","addr":"172.21.15.165:6803","nonce":1356168171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":1356168171},{"type":"v1","addr":"172.21.15.165:6807","nonce":1356168171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":1356168171},{"type":"v1","addr":"172.21.15.165:6805","nonce":1356168171}]},"public_addr":"172.21.15.165:6801/1356168171","cluster_addr":"172.21.15.165:6803/1356168171","heartbeat_back_addr":"172.21.15.165:6807/1356168171","heartbeat_front_addr":"172.21.15.165:6805/1356168171","state":["exists","up"]},{"osd":2,"uuid":"0631df59-1ecc-48b4-b00a-48b0118dfc24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":237502635},{"type":"v1","addr":"172.21.15.172:6801","nonce":237502635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":237502635},{"type":"v1","addr":"172.21.15.172:6803","nonce":237502635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":237502635},{"type":"v1","addr":"172.21.15.172:6807","nonce":237502635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":237502635},{"type":"v1","addr":"172.21.15.172:6805","nonce":237502635}]},"public_addr":"172.21.15.172:6801/237502635","cluster_addr":"172.21.15.172:6803/237502635","heartbeat_back_addr":"172.21.15.172:6807/237502635","heartbeat_front_addr":"172.21.15.172:6805/237502635","state":["exists","up"]},{"osd":3,"uuid":"73076e49-2f8a-4a3d-99eb-7b50aff448aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3442279476},{"type":"v1","addr":"172.21.15.151:6811","nonce":3442279476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3442279476},{"type":"v1","addr":"172.21.15.151:6813","nonce":3442279476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":3442279476},{"type":"v1","addr":"172.21.15.151:6817","nonce":3442279476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3442279476},{"type":"v1","addr":"172.21.15.151:6815","nonce":3442279476}]},"public_addr":"172.21.15.151:6811/3442279476","cluster_addr":"172.21.15.151:6813/3442279476","heartbeat_back_addr":"172.21.15.151:6817/3442279476","heartbeat_front_addr":"172.21.15.151:6815/3442279476","state":["exists","up"]},{"osd":4,"uuid":"0230d615-548c-43f0-b8aa-7130f4cbbc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4029980723},{"type":"v1","addr":"172.21.15.165:6809","nonce":4029980723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4029980723},{"type":"v1","addr":"172.21.15.165:6811","nonce":4029980723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4029980723},{"type":"v1","addr":"172.21.15.165:6815","nonce":4029980723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4029980723},{"type":"v1","addr":"172.21.15.165:6813","nonce":4029980723}]},"public_addr":"172.21.15.165:6809/4029980723","cluster_addr":"172.21.15.165:6811/4029980723","heartbeat_back_addr":"172.21.15.165:6815/4029980723","heartbeat_front_addr":"172.21.15.165:6813/4029980723","state":["exists","up"]},{"osd":5,"uuid":"9739f44b-2605-47dd-82fb-d30034742bf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2216578687},{"type":"v1","addr":"172.21.15.172:6809","nonce":2216578687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2216578687},{"type":"v1","addr":"172.21.15.172:6811","nonce":2216578687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2216578687},{"type":"v1","addr":"172.21.15.172:6815","nonce":2216578687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2216578687},{"type":"v1","addr":"172.21.15.172:6813","nonce":2216578687}]},"public_addr":"172.21.15.172:6809/2216578687","cluster_addr":"172.21.15.172:6811/2216578687","heartbeat_back_addr":"172.21.15.172:6815/2216578687","heartbeat_front_addr":"172.21.15.172:6813/2216578687","state":["exists","up"]},{"osd":6,"uuid":"4290baaa-6160-49cb-a4e0-c96f070f2ef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1723252315},{"type":"v1","addr":"172.21.15.151:6819","nonce":1723252315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1723252315},{"type":"v1","addr":"172.21.15.151:6821","nonce":1723252315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":1723252315},{"type":"v1","addr":"172.21.15.151:6825","nonce":1723252315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1723252315},{"type":"v1","addr":"172.21.15.151:6823","nonce":1723252315}]},"public_addr":"172.21.15.151:6819/1723252315","cluster_addr":"172.21.15.151:6821/1723252315","heartbeat_back_addr":"172.21.15.151:6825/1723252315","heartbeat_front_addr":"172.21.15.151:6823/1723252315","state":["exists","up"]},{"osd":7,"uuid":"9f5315f2-036c-4e2f-92ed-3321a9e3c860","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":1977803781},{"type":"v1","addr":"172.21.15.165:6817","nonce":1977803781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":1977803781},{"type":"v1","addr":"172.21.15.165:6819","nonce":1977803781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":1977803781},{"type":"v1","addr":"172.21.15.165:6823","nonce":1977803781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":1977803781},{"type":"v1","addr":"172.21.15.165:6821","nonce":1977803781}]},"public_addr":"172.21.15.165:6817/1977803781","cluster_addr":"172.21.15.165:6819/1977803781","heartbeat_back_addr":"172.21.15.165:6823/1977803781","heartbeat_front_addr":"172.21.15.165:6821/1977803781","state":["exists","up"]},{"osd":8,"uuid":"8ddbe36d-cea7-45ef-a3ca-044a0e64ff85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":491199512},{"type":"v1","addr":"172.21.15.172:6817","nonce":491199512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":491199512},{"type":"v1","addr":"172.21.15.172:6819","nonce":491199512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":491199512},{"type":"v1","addr":"172.21.15.172:6823","nonce":491199512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":491199512},{"type":"v1","addr":"172.21.15.172:6821","nonce":491199512}]},"public_addr":"172.21.15.172:6817/491199512","cluster_addr":"172.21.15.172:6819/491199512","heartbeat_back_addr":"172.21.15.172:6823/491199512","heartbeat_front_addr":"172.21.15.172:6821/491199512","state":["exists","up"]},{"osd":9,"uuid":"40654ba0-1bd9-4b54-97ae-f1ed6c4785d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":1818923828},{"type":"v1","addr":"172.21.15.151:6827","nonce":1818923828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":1818923828},{"type":"v1","addr":"172.21.15.151:6829","nonce":1818923828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1818923828},{"type":"v1","addr":"172.21.15.151:6833","nonce":1818923828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":1818923828},{"type":"v1","addr":"172.21.15.151:6831","nonce":1818923828}]},"public_addr":"172.21.15.151:6827/1818923828","cluster_addr":"172.21.15.151:6829/1818923828","heartbeat_back_addr":"172.21.15.151:6833/1818923828","heartbeat_front_addr":"172.21.15.151:6831/1818923828","state":["exists","up"]},{"osd":10,"uuid":"19e75a1a-7bd0-41d9-b0cc-0fa4d1475dda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":2632621809},{"type":"v1","addr":"172.21.15.165:6825","nonce":2632621809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":2632621809},{"type":"v1","addr":"172.21.15.165:6827","nonce":2632621809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":2632621809},{"type":"v1","addr":"172.21.15.165:6831","nonce":2632621809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":2632621809},{"type":"v1","addr":"172.21.15.165:6829","nonce":2632621809}]},"public_addr":"172.21.15.165:6825/2632621809","cluster_addr":"172.21.15.165:6827/2632621809","heartbeat_back_addr":"172.21.15.165:6831/2632621809","heartbeat_front_addr":"172.21.15.165:6829/2632621809","state":["exists","up"]},{"osd":11,"uuid":"aa84e5f5-93a3-4375-8544-a1175f0eb06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":886604365},{"type":"v1","addr":"172.21.15.172:6825","nonce":886604365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":886604365},{"type":"v1","addr":"172.21.15.172:6827","nonce":886604365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":886604365},{"type":"v1","addr":"172.21.15.172:6831","nonce":886604365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":886604365},{"type":"v1","addr":"172.21.15.172:6829","nonce":886604365}]},"public_addr":"172.21.15.172:6825/886604365","cluster_addr":"172.21.15.172:6827/886604365","heartbeat_back_addr":"172.21.15.172:6831/886604365","heartbeat_front_addr":"172.21.15.172:6829/886604365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:43:18.296278+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:43:42.860620+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:44:07.073601+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:44:39.631760+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:45:04.771214+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:45:29.606674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:46:03.101851+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:46:28.351995+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:46:53.445435+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:47:27.391179+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:47:53.543638+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:48:18.687811+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.151:6801/796314699":"2024-04-27T01:41:25.234322+0000","172.21.15.151:6800/796314699":"2024-04-27T01:41:25.234322+0000","172.21.15.151:0/1173525554":"2024-04-27T01:41:25.234322+0000","172.21.15.151:0/1342496801":"2024-04-27T01:40:50.492554+0000","172.21.15.151:0/11665926":"2024-04-27T01:40:50.492554+0000","172.21.15.151:0/1765448952":"2024-04-27T01:40:50.492554+0000","172.21.15.151:0/46007430":"2024-04-27T01:41:25.234322+0000","172.21.15.151:6800/244725060":"2024-04-27T01:40:50.492554+0000","172.21.15.151:0/3703862543":"2024-04-27T01:41:25.234322+0000","172.21.15.151:6801/244725060":"2024-04-27T01:40:50.492554+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T01:52:22.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:22.594+0000 7efe32800700 1 -- 172.21.15.151:0/2289110909 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7efe140a3c40 msgr2=0x7efe140a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:22.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:22.594+0000 7efe32800700 1 --2- 172.21.15.151:0/2289110909 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7efe140a3c40 0x7efe140a6100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7efe18000c00 tx=0x7efe18005fb0 comp rx=0 tx=0).stop 2024-04-26T01:52:22.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:22.594+0000 7efe32800700 1 -- 172.21.15.151:0/2289110909 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe2c104dd0 msgr2=0x7efe2c19b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:22.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:22.594+0000 7efe32800700 1 --2- 172.21.15.151:0/2289110909 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe2c104dd0 0x7efe2c19b2a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7efe2000e9f0 tx=0x7efe2000ed00 comp rx=0 tx=0).stop 2024-04-26T01:52:22.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:22.594+0000 7efe32800700 1 -- 172.21.15.151:0/2289110909 shutdown_connections 2024-04-26T01:52:22.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:22.594+0000 7efe32800700 1 --2- 172.21.15.151:0/2289110909 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7efe140a3c40 0x7efe140a6100 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:22.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:22.594+0000 7efe32800700 1 --2- 172.21.15.151:0/2289110909 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe2c109eb0 0x7efe2c19fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:22.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:22.594+0000 7efe32800700 1 --2- 172.21.15.151:0/2289110909 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe2c104dd0 0x7efe2c19b2a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:22.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:22.594+0000 7efe32800700 1 --2- 172.21.15.151:0/2289110909 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe2c105780 0x7efe2c19b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:22.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:22.594+0000 7efe32800700 1 -- 172.21.15.151:0/2289110909 >> 172.21.15.151:0/2289110909 conn(0x7efe2c100660 msgr2=0x7efe2c102990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:22.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:22.595+0000 7efe32800700 1 -- 172.21.15.151:0/2289110909 shutdown_connections 2024-04-26T01:52:22.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:22.595+0000 7efe32800700 1 -- 172.21.15.151:0/2289110909 wait complete. 2024-04-26T01:52:23.291 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:52:23 smithi172 ceph-mon[32785]: pgmap v410: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:52:23.292 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:52:23 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/2289110909' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:52:23.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:52:23 smithi165 ceph-mon[32648]: pgmap v410: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:52:23.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:52:23 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/2289110909' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:52:23.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:52:23 smithi151 ceph-mon[28057]: pgmap v410: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:52:23.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:52:23 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/2289110909' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:52:23.535 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:52:23.989 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:52:25.195 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:52:25 smithi151 ceph-mon[28057]: pgmap v411: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:52:25.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:52:25 smithi172 ceph-mon[32785]: pgmap v411: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:52:25.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:52:25 smithi165 ceph-mon[32648]: pgmap v411: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:52:26.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:26.151+0000 7f4bed400700 1 -- 172.21.15.151:0/939386540 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4be806a750 msgr2=0x7f4be806abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:26.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:26.151+0000 7f4bed400700 1 --2- 172.21.15.151:0/939386540 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4be806a750 0x7f4be806abb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0009990 tx=0x7f4bd0009ca0 comp rx=0 tx=0).stop 2024-04-26T01:52:26.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:26.151+0000 7f4bed400700 1 -- 172.21.15.151:0/939386540 shutdown_connections 2024-04-26T01:52:26.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:26.151+0000 7f4bed400700 1 --2- 172.21.15.151:0/939386540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4be806b0f0 0x7f4be810f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:26.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:26.151+0000 7f4bed400700 1 --2- 172.21.15.151:0/939386540 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4be8108a30 0x7f4be8108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:26.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:26.151+0000 7f4bed400700 1 --2- 172.21.15.151:0/939386540 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4be806a750 0x7f4be806abb0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:26.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:26.151+0000 7f4bed400700 1 -- 172.21.15.151:0/939386540 >> 172.21.15.151:0/939386540 conn(0x7f4be80788f0 msgr2=0x7f4be8078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:26.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:26.152+0000 7f4bed400700 1 -- 172.21.15.151:0/939386540 shutdown_connections 2024-04-26T01:52:26.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:26.152+0000 7f4bed400700 1 -- 172.21.15.151:0/939386540 wait complete. 2024-04-26T01:52:26.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:26.153+0000 7f4bed400700 1 Processor -- start 2024-04-26T01:52:26.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:26.153+0000 7f4bed400700 1 -- start start 2024-04-26T01:52:26.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:26.153+0000 7f4bed400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4be806a750 0x7f4be8101a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:26.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:26.153+0000 7f4bed400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4be806b0f0 0x7f4be8101f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:26.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:26.153+0000 7f4bed400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4be8108a30 0x7f4be8102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:26.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:26.153+0000 7f4bed400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4be8112c20 con 0x7f4be806a750 2024-04-26T01:52:26.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:26.153+0000 7f4bed400700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4be8112aa0 con 0x7f4be806b0f0 2024-04-26T01:52:26.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:26.153+0000 7f4bed400700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4be8112da0 con 0x7f4be8108a30 2024-04-26T01:52:26.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:26.154+0000 7f4be6800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4be806a750 0x7f4be8101a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:26.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:26.154+0000 7f4bdde00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4be806b0f0 0x7f4be8101f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:26.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:26.154+0000 7f4be6800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4be806a750 0x7f4be8101a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:45330/0 (socket says 172.21.15.151:45330) 2024-04-26T01:52:26.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:26.154+0000 7f4be6800700 1 -- 172.21.15.151:0/954236206 learned_addr learned my addr 172.21.15.151:0/954236206 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:52:26.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:26.154+0000 7f4be7200700 1 --2- 172.21.15.151:0/954236206 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4be8108a30 0x7f4be8102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:26.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:26.154+0000 7f4bdde00700 1 -- 172.21.15.151:0/954236206 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4be8108a30 msgr2=0x7f4be8102490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:26.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:26.154+0000 7f4bdde00700 1 --2- 172.21.15.151:0/954236206 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4be8108a30 0x7f4be8102490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:26.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:26.154+0000 7f4bdde00700 1 -- 172.21.15.151:0/954236206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4be806a750 msgr2=0x7f4be8101a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:26.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:26.154+0000 7f4bdde00700 1 --2- 172.21.15.151:0/954236206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4be806a750 0x7f4be8101a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:26.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:26.154+0000 7f4bdde00700 1 -- 172.21.15.151:0/954236206 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4be81a76a0 con 0x7f4be806b0f0 2024-04-26T01:52:26.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:26.154+0000 7f4be6800700 1 --2- 172.21.15.151:0/954236206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4be806a750 0x7f4be8101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:52:26.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:26.155+0000 7f4bdde00700 1 --2- 172.21.15.151:0/954236206 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4be806b0f0 0x7f4be8101f50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4bd00098e0 tx=0x7f4bd00057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:26.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:26.155+0000 7f4bdfe00700 1 -- 172.21.15.151:0/954236206 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd0027070 con 0x7f4be806b0f0 2024-04-26T01:52:26.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:26.155+0000 7f4bed400700 1 -- 172.21.15.151:0/954236206 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4be81a7930 con 0x7f4be806b0f0 2024-04-26T01:52:26.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:26.155+0000 7f4bdfe00700 1 -- 172.21.15.151:0/954236206 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4bd0022470 con 0x7f4be806b0f0 2024-04-26T01:52:26.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:26.155+0000 7f4bdfe00700 1 -- 172.21.15.151:0/954236206 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd0017650 con 0x7f4be806b0f0 2024-04-26T01:52:26.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:26.155+0000 7f4bed400700 1 -- 172.21.15.151:0/954236206 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4be81a7de0 con 0x7f4be806b0f0 2024-04-26T01:52:26.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:26.156+0000 7f4bed400700 1 -- 172.21.15.151:0/954236206 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4be8069730 con 0x7f4be806b0f0 2024-04-26T01:52:26.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:26.158+0000 7f4bdfe00700 1 -- 172.21.15.151:0/954236206 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f4bd002a070 con 0x7f4be806b0f0 2024-04-26T01:52:26.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:26.159+0000 7f4bdfe00700 1 --2- 172.21.15.151:0/954236206 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f4bcc0a3cc0 0x7f4bcc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:26.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:26.160+0000 7f4bdfe00700 1 -- 172.21.15.151:0/954236206 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f4bd00d10e0 con 0x7f4be806b0f0 2024-04-26T01:52:26.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:26.160+0000 7f4be6800700 1 --2- 172.21.15.151:0/954236206 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f4bcc0a3cc0 0x7f4bcc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:26.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:26.160+0000 7f4be6800700 1 --2- 172.21.15.151:0/954236206 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f4bcc0a3cc0 0x7f4bcc0a6180 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4bd8009c10 tx=0x7f4bd800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:26.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:26.164+0000 7f4bdfe00700 1 -- 172.21.15.151:0/954236206 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4bd005eff0 con 0x7f4be806b0f0 2024-04-26T01:52:26.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:26.436+0000 7f4bed400700 1 -- 172.21.15.151:0/954236206 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4be802e140 con 0x7f4be806b0f0 2024-04-26T01:52:26.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:26.437+0000 7f4bdfe00700 1 -- 172.21.15.151:0/954236206 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11889 (secure 0 0 0) 0x7f4bd0013070 con 0x7f4be806b0f0 2024-04-26T01:52:26.436 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:52:26.436 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2937864097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2937864097},{"type":"v1","addr":"172.21.15.151:6835","nonce":2937864097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14373,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/2947090802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2947090802},{"type":"v1","addr":"172.21.15.151:6837","nonce":2947090802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/1356563280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1356563280},{"type":"v1","addr":"172.21.15.151:6839","nonce":1356563280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6841/1075240552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1075240552},{"type":"v1","addr":"172.21.15.151:6841","nonce":1075240552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/504675912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":504675912},{"type":"v1","addr":"172.21.15.165:6833","nonce":504675912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1207580015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1207580015},{"type":"v1","addr":"172.21.15.172:6833","nonce":1207580015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:49:40.746876+0000","modified":"2024-04-26T01:50:05.608353+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24505,"mds_2":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/3598138294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3598138294},{"type":"v1","addr":"172.21.15.165:6835","nonce":3598138294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/1847859054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":1847859054},{"type":"v1","addr":"172.21.15.165:6837","nonce":1847859054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/2309751171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2309751171},{"type":"v1","addr":"172.21.15.172:6835","nonce":2309751171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.172:6837/55972537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":55972537},{"type":"v1","addr":"172.21.15.172:6837","nonce":55972537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.172:6839/1242302810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1242302810},{"type":"v1","addr":"172.21.15.172:6839","nonce":1242302810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/1969716254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1969716254},{"type":"v1","addr":"172.21.15.165:6839","nonce":1969716254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24505,24509,24493]},"id":1}]} 2024-04-26T01:52:26.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:26.441+0000 7f4bed400700 1 -- 172.21.15.151:0/954236206 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f4bcc0a3cc0 msgr2=0x7f4bcc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:26.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:26.441+0000 7f4bed400700 1 --2- 172.21.15.151:0/954236206 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f4bcc0a3cc0 0x7f4bcc0a6180 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4bd8009c10 tx=0x7f4bd800b040 comp rx=0 tx=0).stop 2024-04-26T01:52:26.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:26.441+0000 7f4bed400700 1 -- 172.21.15.151:0/954236206 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4be806b0f0 msgr2=0x7f4be8101f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:26.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:26.441+0000 7f4bed400700 1 --2- 172.21.15.151:0/954236206 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4be806b0f0 0x7f4be8101f50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4bd00098e0 tx=0x7f4bd00057c0 comp rx=0 tx=0).stop 2024-04-26T01:52:26.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:26.441+0000 7f4bed400700 1 -- 172.21.15.151:0/954236206 shutdown_connections 2024-04-26T01:52:26.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:26.441+0000 7f4bed400700 1 --2- 172.21.15.151:0/954236206 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4be8108a30 0x7f4be8102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:26.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:26.441+0000 7f4bed400700 1 --2- 172.21.15.151:0/954236206 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f4bcc0a3cc0 0x7f4bcc0a6180 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:26.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:26.441+0000 7f4bed400700 1 --2- 172.21.15.151:0/954236206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4be806a750 0x7f4be8101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:26.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:26.441+0000 7f4bed400700 1 --2- 172.21.15.151:0/954236206 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4be806b0f0 0x7f4be8101f50 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:26.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:26.441+0000 7f4bed400700 1 -- 172.21.15.151:0/954236206 >> 172.21.15.151:0/954236206 conn(0x7f4be80788f0 msgr2=0x7f4be80ffa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:26.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:26.441+0000 7f4bed400700 1 -- 172.21.15.151:0/954236206 shutdown_connections 2024-04-26T01:52:26.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:26.441+0000 7f4bed400700 1 -- 172.21.15.151:0/954236206 wait complete. 2024-04-26T01:52:26.441 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 25 2024-04-26T01:52:27.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:52:27 smithi151 ceph-mon[28057]: pgmap v412: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:52:27.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:52:27 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/954236206' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:52:27.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:52:27 smithi165 ceph-mon[32648]: pgmap v412: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:52:27.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:52:27 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/954236206' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:52:27.461 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-26T01:49:40.746876+0000', 'modified': '2024-04-26T01:50:05.608353+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24509, 'mds_1': 24505, 'mds_2': 24493}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24458': {'gid': 24458, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.165:6835/3598138294', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6834', 'nonce': 3598138294}, {'type': 'v1', 'addr': '172.21.15.165:6835', 'nonce': 3598138294}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24470': {'gid': 24470, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.165:6837/1847859054', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6836', 'nonce': 1847859054}, {'type': 'v1', 'addr': '172.21.15.165:6837', 'nonce': 1847859054}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24481': {'gid': 24481, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.172:6835/2309751171', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6834', 'nonce': 2309751171}, {'type': 'v1', 'addr': '172.21.15.172:6835', 'nonce': 2309751171}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24493': {'gid': 24493, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.172:6837/55972537', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6836', 'nonce': 55972537}, {'type': 'v1', 'addr': '172.21.15.172:6837', 'nonce': 55972537}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24505': {'gid': 24505, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.172:6839/1242302810', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6838', 'nonce': 1242302810}, {'type': 'v1', 'addr': '172.21.15.172:6839', 'nonce': 1242302810}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24509': {'gid': 24509, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.165:6839/1969716254', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6838', 'nonce': 1969716254}, {'type': 'v1', 'addr': '172.21.15.165:6839', 'nonce': 1969716254}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24509, 'qdb_cluster': [24505, 24509, 24493]} 2024-04-26T01:52:27.461 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-26T01:52:27.461 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph tell mds.c status 2024-04-26T01:52:27.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:52:27 smithi172 ceph-mon[32785]: pgmap v412: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:52:27.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:52:27 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/954236206' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:52:27.883 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:52:29.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:52:29 smithi165 ceph-mon[32648]: pgmap v413: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T01:52:29.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:52:29 smithi151 ceph-mon[28057]: pgmap v413: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T01:52:29.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:52:29 smithi172 ceph-mon[32785]: pgmap v413: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T01:52:30.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.249+0000 7f274a200700 1 -- 172.21.15.151:0/1561298023 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2744109f70 msgr2=0x7f2744111b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:30.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.249+0000 7f274a200700 1 --2- 172.21.15.151:0/1561298023 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2744109f70 0x7f2744111b40 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f273c00b0d0 tx=0x7f273c00b3e0 comp rx=0 tx=0).stop 2024-04-26T01:52:30.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.249+0000 7f274a200700 1 -- 172.21.15.151:0/1561298023 shutdown_connections 2024-04-26T01:52:30.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.249+0000 7f274a200700 1 --2- 172.21.15.151:0/1561298023 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2744109f70 0x7f2744111b40 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:30.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.249+0000 7f274a200700 1 --2- 172.21.15.151:0/1561298023 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2744104e90 0x7f2744105270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:30.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.249+0000 7f274a200700 1 --2- 172.21.15.151:0/1561298023 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2744105840 0x7f2744109890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:30.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.249+0000 7f274a200700 1 -- 172.21.15.151:0/1561298023 >> 172.21.15.151:0/1561298023 conn(0x7f2744100720 msgr2=0x7f2744102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:30.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.249+0000 7f274a200700 1 -- 172.21.15.151:0/1561298023 shutdown_connections 2024-04-26T01:52:30.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.249+0000 7f274a200700 1 -- 172.21.15.151:0/1561298023 wait complete. 2024-04-26T01:52:30.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.251+0000 7f274a200700 1 Processor -- start 2024-04-26T01:52:30.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.251+0000 7f274a200700 1 -- start start 2024-04-26T01:52:30.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.251+0000 7f274a200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2744104e90 0x7f274419b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:30.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.251+0000 7f274a200700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2744105840 0x7f274419b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:30.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.251+0000 7f274a200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2744109f70 0x7f274419fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:30.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.251+0000 7f274a200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2744114e80 con 0x7f2744109f70 2024-04-26T01:52:30.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.251+0000 7f274a200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2744114d00 con 0x7f2744104e90 2024-04-26T01:52:30.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.251+0000 7f274a200700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2744115000 con 0x7f2744105840 2024-04-26T01:52:30.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.252+0000 7f2743e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2744109f70 0x7f274419fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:30.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.252+0000 7f2743400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2744104e90 0x7f274419b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:30.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.252+0000 7f2743400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2744104e90 0x7f274419b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:45614/0 (socket says 172.21.15.151:45614) 2024-04-26T01:52:30.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.252+0000 7f2743400700 1 -- 172.21.15.151:0/2432903321 learned_addr learned my addr 172.21.15.151:0/2432903321 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:52:30.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.252+0000 7f2743e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2744109f70 0x7f274419fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42518/0 (socket says 172.21.15.151:42518) 2024-04-26T01:52:30.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.252+0000 7f2742a00700 1 --2- 172.21.15.151:0/2432903321 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2744105840 0x7f274419b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:30.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.252+0000 7f2743400700 1 -- 172.21.15.151:0/2432903321 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2744105840 msgr2=0x7f274419b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:30.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.252+0000 7f2743400700 1 --2- 172.21.15.151:0/2432903321 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2744105840 0x7f274419b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:30.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.252+0000 7f2743400700 1 -- 172.21.15.151:0/2432903321 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2744109f70 msgr2=0x7f274419fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:30.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.252+0000 7f2743400700 1 --2- 172.21.15.151:0/2432903321 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2744109f70 0x7f274419fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:30.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.252+0000 7f2743400700 1 -- 172.21.15.151:0/2432903321 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27441a02d0 con 0x7f2744104e90 2024-04-26T01:52:30.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.252+0000 7f2743e00700 1 --2- 172.21.15.151:0/2432903321 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2744109f70 0x7f274419fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:52:30.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.253+0000 7f2743400700 1 --2- 172.21.15.151:0/2432903321 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2744104e90 0x7f274419b310 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f273400e9a0 tx=0x7f273400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:30.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.253+0000 7f273be00700 1 -- 172.21.15.151:0/2432903321 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273400ca10 con 0x7f2744104e90 2024-04-26T01:52:30.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.253+0000 7f273be00700 1 -- 172.21.15.151:0/2432903321 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f273400cbb0 con 0x7f2744104e90 2024-04-26T01:52:30.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.253+0000 7f274a200700 1 -- 172.21.15.151:0/2432903321 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27441a05c0 con 0x7f2744104e90 2024-04-26T01:52:30.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.253+0000 7f273be00700 1 -- 172.21.15.151:0/2432903321 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2734018680 con 0x7f2744104e90 2024-04-26T01:52:30.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.253+0000 7f274a200700 1 -- 172.21.15.151:0/2432903321 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27441a7e40 con 0x7f2744104e90 2024-04-26T01:52:30.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.256+0000 7f273be00700 1 -- 172.21.15.151:0/2432903321 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f2734010430 con 0x7f2744104e90 2024-04-26T01:52:30.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.258+0000 7f273be00700 1 --2- 172.21.15.151:0/2432903321 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f27280a3c40 0x7f27280a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:30.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.258+0000 7f2754b01b80 1 Processor -- start 2024-04-26T01:52:30.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.258+0000 7f273be00700 1 -- 172.21.15.151:0/2432903321 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f27340d0580 con 0x7f2744104e90 2024-04-26T01:52:30.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.258+0000 7f2742a00700 1 --2- 172.21.15.151:0/2432903321 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f27280a3c40 0x7f27280a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:30.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.258+0000 7f2754b01b80 1 -- start start 2024-04-26T01:52:30.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.258+0000 7f2754b01b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634f2d27780 0x5634f2d27b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:30.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.258+0000 7f2754b01b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634f2d280a0 0x5634f2d219b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:30.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.259+0000 7f2754b01b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634f2d21ef0 0x5634f2d22330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:30.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.259+0000 7f2754b01b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5634f2c73e40 con 0x5634f2d27780 2024-04-26T01:52:30.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.259+0000 7f2754b01b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5634f2c1a6e0 con 0x5634f2d21ef0 2024-04-26T01:52:30.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.259+0000 7f2754b01b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5634f2c6fe00 con 0x5634f2d280a0 2024-04-26T01:52:30.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.259+0000 7f2742a00700 1 --2- 172.21.15.151:0/2432903321 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f27280a3c40 0x7f27280a6100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f274410b510 tx=0x7f2730009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:30.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.259+0000 7f2743400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634f2d27780 0x5634f2d27b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:30.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.259+0000 7f2742a00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634f2d280a0 0x5634f2d219b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:30.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.259+0000 7f2743400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634f2d27780 0x5634f2d27b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42526/0 (socket says 172.21.15.151:42526) 2024-04-26T01:52:30.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.259+0000 7f2743400700 1 -- 172.21.15.151:0/1662794725 learned_addr learned my addr 172.21.15.151:0/1662794725 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:52:30.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.259+0000 7f2743e00700 1 --2- 172.21.15.151:0/1662794725 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634f2d21ef0 0x5634f2d22330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:30.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.259+0000 7f2742a00700 1 -- 172.21.15.151:0/1662794725 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634f2d21ef0 msgr2=0x5634f2d22330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:30.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.259+0000 7f2742a00700 1 --2- 172.21.15.151:0/1662794725 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634f2d21ef0 0x5634f2d22330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:30.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.259+0000 7f2742a00700 1 -- 172.21.15.151:0/1662794725 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634f2d27780 msgr2=0x5634f2d27b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:30.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.259+0000 7f2742a00700 1 --2- 172.21.15.151:0/1662794725 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634f2d27780 0x5634f2d27b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:30.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.259+0000 7f2742a00700 1 -- 172.21.15.151:0/1662794725 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634f2c10630 con 0x5634f2d280a0 2024-04-26T01:52:30.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.260+0000 7f2742a00700 1 --2- 172.21.15.151:0/1662794725 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634f2d280a0 0x5634f2d219b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f274406c2e0 tx=0x7f2730014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d41470174347d753 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:30.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.260+0000 7f2738c00700 1 -- 172.21.15.151:0/1662794725 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2730022070 con 0x5634f2d280a0 2024-04-26T01:52:30.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.260+0000 7f2738c00700 1 -- 172.21.15.151:0/1662794725 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2730004500 con 0x5634f2d280a0 2024-04-26T01:52:30.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.260+0000 7f2738c00700 1 -- 172.21.15.151:0/1662794725 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2730003970 con 0x5634f2d280a0 2024-04-26T01:52:30.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.261+0000 7f2754b01b80 1 -- 172.21.15.151:0/1662794725 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634f2d280a0 msgr2=0x5634f2d219b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:30.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.261+0000 7f2754b01b80 1 --2- 172.21.15.151:0/1662794725 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634f2d280a0 0x5634f2d219b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f274406c2e0 tx=0x7f2730014000 comp rx=0 tx=0).stop 2024-04-26T01:52:30.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.261+0000 7f2754b01b80 1 -- 172.21.15.151:0/1662794725 shutdown_connections 2024-04-26T01:52:30.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.261+0000 7f2754b01b80 1 --2- 172.21.15.151:0/1662794725 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634f2d21ef0 0x5634f2d22330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:30.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.261+0000 7f2754b01b80 1 --2- 172.21.15.151:0/1662794725 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634f2d27780 0x5634f2d27b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:30.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.261+0000 7f2754b01b80 1 --2- 172.21.15.151:0/1662794725 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634f2d280a0 0x5634f2d219b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:30.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.261+0000 7f2754b01b80 1 -- 172.21.15.151:0/1662794725 >> 172.21.15.151:0/1662794725 conn(0x5634f2c4ff00 msgr2=0x5634f2d1c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:30.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.261+0000 7f2754b01b80 1 -- 172.21.15.151:0/1662794725 shutdown_connections 2024-04-26T01:52:30.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.261+0000 7f2754b01b80 1 -- 172.21.15.151:0/1662794725 wait complete. 2024-04-26T01:52:30.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.261+0000 7f2754b01b80 1 -- start start 2024-04-26T01:52:30.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.262+0000 7f2754b01b80 1 Processor -- start 2024-04-26T01:52:30.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.262+0000 7f2754b01b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634f2d21ef0 0x5634f2d2dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:30.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.262+0000 7f2754b01b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634f2d27780 0x5634f2d2afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:30.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.262+0000 7f2754b01b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634f2d2b4e0 0x5634f2d2b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:30.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.262+0000 7f2754b01b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5634f2d2e920 con 0x5634f2d21ef0 2024-04-26T01:52:30.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.262+0000 7f2754b01b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5634f2c2d810 con 0x5634f2d27780 2024-04-26T01:52:30.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.262+0000 7f2754b01b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5634f2d2eaa0 con 0x5634f2d2b4e0 2024-04-26T01:52:30.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.262+0000 7f2754b01b80 10 client.0 _ll_register_callbacks cb 0x5634f2c0c6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:52:30.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.263+0000 7f2743400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634f2d21ef0 0x5634f2d2dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:30.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.263+0000 7f2743400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634f2d21ef0 0x5634f2d2dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42530/0 (socket says 172.21.15.151:42530) 2024-04-26T01:52:30.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.263+0000 7f2743400700 1 -- 172.21.15.151:0/2897711705 learned_addr learned my addr 172.21.15.151:0/2897711705 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:52:30.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.263+0000 7f2742a00700 1 --2- 172.21.15.151:0/2897711705 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634f2d27780 0x5634f2d2afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:30.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.263+0000 7f2743e00700 1 --2- 172.21.15.151:0/2897711705 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634f2d2b4e0 0x5634f2d2b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:30.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.263+0000 7f2743400700 1 -- 172.21.15.151:0/2897711705 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634f2d2b4e0 msgr2=0x5634f2d2b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:30.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.263+0000 7f2743400700 1 --2- 172.21.15.151:0/2897711705 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634f2d2b4e0 0x5634f2d2b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:30.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.263+0000 7f2743400700 1 -- 172.21.15.151:0/2897711705 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634f2d27780 msgr2=0x5634f2d2afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:30.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.263+0000 7f2743400700 1 --2- 172.21.15.151:0/2897711705 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634f2d27780 0x5634f2d2afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:30.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.263+0000 7f2743400700 1 -- 172.21.15.151:0/2897711705 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634f2bc1880 con 0x5634f2d21ef0 2024-04-26T01:52:30.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.263+0000 7f2743e00700 1 --2- 172.21.15.151:0/2897711705 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634f2d2b4e0 0x5634f2d2b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:52:30.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.263+0000 7f2743400700 1 --2- 172.21.15.151:0/2897711705 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634f2d21ef0 0x5634f2d2dca0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f27340098c0 tx=0x7f2734087390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ab9198af195ed6e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:30.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.263+0000 7f272d600700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-04-26T01:52:30.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.264+0000 7f272d600700 1 -- 172.21.15.151:0/2897711705 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27340a0c10 con 0x5634f2d21ef0 2024-04-26T01:52:30.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.264+0000 7f272d600700 1 -- 172.21.15.151:0/2897711705 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f273409ab60 con 0x5634f2d21ef0 2024-04-26T01:52:30.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.264+0000 7f2754b01b80 1 -- 172.21.15.151:0/2897711705 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2730004500 con 0x5634f2d21ef0 2024-04-26T01:52:30.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.264+0000 7f272d600700 1 -- 172.21.15.151:0/2897711705 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f27340a54d0 con 0x5634f2d21ef0 2024-04-26T01:52:30.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.264+0000 7f2754b01b80 10 client.14754 fetch_fsmap learned FSMap version 25 2024-04-26T01:52:30.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.264+0000 7f2754b01b80 1 -- 172.21.15.151:0/2897711705 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5634f2c0aaa0 con 0x5634f2d21ef0 2024-04-26T01:52:30.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.264+0000 7f272d600700 1 -- 172.21.15.151:0/2897711705 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2730004500 con 0x5634f2d21ef0 2024-04-26T01:52:30.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.264+0000 7f272d600700 1 -- 172.21.15.151:0/2897711705 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f273409dcb0 con 0x5634f2d21ef0 2024-04-26T01:52:30.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.265+0000 7f2754b01b80 10 client.14754 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T01:52:30.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.265+0000 7f2754b01b80 10 client.14754 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T01:52:30.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.265+0000 7f2754b01b80 20 client.14754 populate_metadata read hostname 'smithi151' 2024-04-26T01:52:30.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.265+0000 7f2754b01b80 1 --2- 172.21.15.151:0/2897711705 >> [v2:172.21.15.172:6836/55972537,v1:172.21.15.172:6837/55972537] conn(0x5634f2d3ba50 0x5634f2d3cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:30.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.265+0000 7f2754b01b80 4 client.14754 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:52:30.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.265+0000 7f2754b01b80 1 -- 172.21.15.151:0/2897711705 --> [v2:172.21.15.172:6836/55972537,v1:172.21.15.172:6837/55972537] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634f2c81630 con 0x5634f2d3ba50 2024-04-26T01:52:30.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.265+0000 7f2742a00700 1 --2- 172.21.15.151:0/2897711705 >> [v2:172.21.15.172:6836/55972537,v1:172.21.15.172:6837/55972537] conn(0x5634f2d3ba50 0x5634f2d3cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:30.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.266+0000 7f272d600700 10 client.14754 ms_handle_connect on v2:172.21.15.172:6836/55972537 2024-04-26T01:52:30.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.265+0000 7f2742a00700 1 --2- 172.21.15.151:0/2897711705 >> [v2:172.21.15.172:6836/55972537,v1:172.21.15.172:6837/55972537] conn(0x5634f2d3ba50 0x5634f2d3cec0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=48aaf3dc7c36696 server_cookie=6cf3bde138a0be0a in_seq=0 out_seq=0 2024-04-26T01:52:30.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.266+0000 7f272d600700 1 -- 172.21.15.151:0/2897711705 <== mds.2 v2:172.21.15.172:6836/55972537 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5634f2c81630 con 0x5634f2d3ba50 2024-04-26T01:52:30.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.266+0000 7f272d600700 10 client.14754 handle_command_reply: tid=0 2024-04-26T01:52:30.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.267+0000 7f2754b01b80 1 client.14754 shutdown 2024-04-26T01:52:30.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.267+0000 7f2754b01b80 1 -- 172.21.15.151:0/2897711705 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634f2d21ef0 msgr2=0x5634f2d2dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:30.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.267+0000 7f2754b01b80 1 --2- 172.21.15.151:0/2897711705 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634f2d21ef0 0x5634f2d2dca0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f27340098c0 tx=0x7f2734087390 comp rx=0 tx=0).stop 2024-04-26T01:52:30.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.267+0000 7f2754b01b80 1 -- 172.21.15.151:0/2897711705 shutdown_connections 2024-04-26T01:52:30.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.267+0000 7f2754b01b80 1 --2- 172.21.15.151:0/2897711705 >> [v2:172.21.15.172:6836/55972537,v1:172.21.15.172:6837/55972537] conn(0x5634f2d3ba50 0x5634f2d3cec0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:30.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.267+0000 7f2754b01b80 1 --2- 172.21.15.151:0/2897711705 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634f2d2b4e0 0x5634f2d2b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:30.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.267+0000 7f2754b01b80 1 --2- 172.21.15.151:0/2897711705 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634f2d21ef0 0x5634f2d2dca0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:30.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.267+0000 7f2754b01b80 1 --2- 172.21.15.151:0/2897711705 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634f2d27780 0x5634f2d2afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:30.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.267+0000 7f2754b01b80 1 -- 172.21.15.151:0/2897711705 >> 172.21.15.151:0/2897711705 conn(0x5634f2c4ff00 msgr2=0x5634f2d1be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:30.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.267+0000 7f272d600700 0 client.14754 ms_handle_reset on v2:172.21.15.172:6836/55972537 2024-04-26T01:52:30.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.267+0000 7f2754b01b80 1 -- 172.21.15.151:0/2897711705 shutdown_connections 2024-04-26T01:52:30.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.267+0000 7f2754b01b80 1 -- 172.21.15.151:0/2897711705 wait complete. 2024-04-26T01:52:30.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.267+0000 7f2754b01b80 20 client.14754 trim_cache size 0 max 16384 2024-04-26T01:52:30.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.289+0000 7f2754b01b80 1 Processor -- start 2024-04-26T01:52:30.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.289+0000 7f2754b01b80 1 -- start start 2024-04-26T01:52:30.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.290+0000 7f2754b01b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634f2d21ef0 0x5634f2d219b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:30.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.290+0000 7f2754b01b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634f2d27780 0x5634f2d2af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:30.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.290+0000 7f2754b01b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634f2d2b4e0 0x5634f2d28ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:30.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.290+0000 7f2754b01b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5634f2d3b200 con 0x5634f2d2b4e0 2024-04-26T01:52:30.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.290+0000 7f2754b01b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5634f2d3b8d0 con 0x5634f2d21ef0 2024-04-26T01:52:30.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.290+0000 7f2754b01b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5634f2d3bfa0 con 0x5634f2d27780 2024-04-26T01:52:30.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.290+0000 7f2743e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634f2d2b4e0 0x5634f2d28ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:30.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.290+0000 7f2743e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634f2d2b4e0 0x5634f2d28ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42536/0 (socket says 172.21.15.151:42536) 2024-04-26T01:52:30.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.290+0000 7f2743e00700 1 -- 172.21.15.151:0/2042712531 learned_addr learned my addr 172.21.15.151:0/2042712531 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:52:30.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.290+0000 7f2743400700 1 --2- 172.21.15.151:0/2042712531 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634f2d21ef0 0x5634f2d219b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:30.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.290+0000 7f2742a00700 1 --2- 172.21.15.151:0/2042712531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634f2d27780 0x5634f2d2af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:30.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.291+0000 7f2743400700 1 -- 172.21.15.151:0/2042712531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634f2d27780 msgr2=0x5634f2d2af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:30.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.291+0000 7f2743400700 1 --2- 172.21.15.151:0/2042712531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634f2d27780 0x5634f2d2af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:30.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.291+0000 7f2743400700 1 -- 172.21.15.151:0/2042712531 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634f2d2b4e0 msgr2=0x5634f2d28ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:30.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.291+0000 7f2743400700 1 --2- 172.21.15.151:0/2042712531 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634f2d2b4e0 0x5634f2d28ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:30.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.291+0000 7f2743400700 1 -- 172.21.15.151:0/2042712531 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634f2d2d8e0 con 0x5634f2d21ef0 2024-04-26T01:52:30.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.291+0000 7f2743e00700 1 --2- 172.21.15.151:0/2042712531 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634f2d2b4e0 0x5634f2d28ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:52:30.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.291+0000 7f2743400700 1 --2- 172.21.15.151:0/2042712531 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634f2d21ef0 0x5634f2d219b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f273409c7f0 tx=0x7f273409d0a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a0f1647e0c4fdd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:30.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.291+0000 7f272fe00700 1 -- 172.21.15.151:0/2042712531 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273400cd60 con 0x5634f2d21ef0 2024-04-26T01:52:30.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.291+0000 7f272fe00700 1 -- 172.21.15.151:0/2042712531 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f273409ab60 con 0x5634f2d21ef0 2024-04-26T01:52:30.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.291+0000 7f272fe00700 1 -- 172.21.15.151:0/2042712531 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2734010a50 con 0x5634f2d21ef0 2024-04-26T01:52:30.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.292+0000 7f2754b01b80 1 -- 172.21.15.151:0/2042712531 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634f2d21ef0 msgr2=0x5634f2d219b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:30.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.292+0000 7f2754b01b80 1 --2- 172.21.15.151:0/2042712531 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634f2d21ef0 0x5634f2d219b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f273409c7f0 tx=0x7f273409d0a0 comp rx=0 tx=0).stop 2024-04-26T01:52:30.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.292+0000 7f2754b01b80 1 -- 172.21.15.151:0/2042712531 shutdown_connections 2024-04-26T01:52:30.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.292+0000 7f2754b01b80 1 --2- 172.21.15.151:0/2042712531 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634f2d2b4e0 0x5634f2d28ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:30.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.292+0000 7f2754b01b80 1 --2- 172.21.15.151:0/2042712531 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634f2d21ef0 0x5634f2d219b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:30.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.292+0000 7f2754b01b80 1 --2- 172.21.15.151:0/2042712531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634f2d27780 0x5634f2d2af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:30.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.292+0000 7f2754b01b80 1 -- 172.21.15.151:0/2042712531 >> 172.21.15.151:0/2042712531 conn(0x5634f2c4ff00 msgr2=0x5634f2d28490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:30.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.292+0000 7f2754b01b80 1 -- 172.21.15.151:0/2042712531 shutdown_connections 2024-04-26T01:52:30.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.292+0000 7f2754b01b80 1 -- 172.21.15.151:0/2042712531 wait complete. 2024-04-26T01:52:30.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.292+0000 7f2754b01b80 1 -- start start 2024-04-26T01:52:30.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.293+0000 7f2754b01b80 1 Processor -- start 2024-04-26T01:52:30.294 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.293+0000 7f2754b01b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634f2d21ef0 0x5634f2d46340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:30.294 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.293+0000 7f2754b01b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634f2d27780 0x5634f2d46880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:30.294 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.293+0000 7f2754b01b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634f2d2b4e0 0x5634f2d414f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:30.294 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.293+0000 7f2754b01b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5634f2d3cd40 con 0x5634f2d2b4e0 2024-04-26T01:52:30.294 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.293+0000 7f2754b01b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5634f2d3c670 con 0x5634f2d21ef0 2024-04-26T01:52:30.294 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.293+0000 7f2754b01b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5634f2d3d410 con 0x5634f2d27780 2024-04-26T01:52:30.294 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.293+0000 7f2754b01b80 10 client.0 _ll_register_callbacks cb 0x5634f2c0c6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:52:30.294 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.293+0000 7f2742a00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634f2d27780 0x5634f2d46880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:30.294 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.293+0000 7f2742a00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634f2d27780 0x5634f2d46880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:37610/0 (socket says 172.21.15.151:37610) 2024-04-26T01:52:30.294 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.293+0000 7f2742a00700 1 -- 172.21.15.151:0/1774741174 learned_addr learned my addr 172.21.15.151:0/1774741174 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:52:30.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.294+0000 7f2743400700 1 --2- 172.21.15.151:0/1774741174 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634f2d21ef0 0x5634f2d46340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:30.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.294+0000 7f2743e00700 1 --2- 172.21.15.151:0/1774741174 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634f2d2b4e0 0x5634f2d414f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:30.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.294+0000 7f2742a00700 1 -- 172.21.15.151:0/1774741174 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634f2d21ef0 msgr2=0x5634f2d46340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:30.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.294+0000 7f2742a00700 1 --2- 172.21.15.151:0/1774741174 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634f2d21ef0 0x5634f2d46340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:30.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.294+0000 7f2742a00700 1 -- 172.21.15.151:0/1774741174 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634f2d2b4e0 msgr2=0x5634f2d414f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:30.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.294+0000 7f2742a00700 1 --2- 172.21.15.151:0/1774741174 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634f2d2b4e0 0x5634f2d414f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:30.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.294+0000 7f2742a00700 1 -- 172.21.15.151:0/1774741174 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634f2d41d30 con 0x5634f2d27780 2024-04-26T01:52:30.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.294+0000 7f2742a00700 1 --2- 172.21.15.151:0/1774741174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634f2d27780 0x5634f2d46880 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2730013fd0 tx=0x7f27300091e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b976c120612a4053 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:30.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.294+0000 7f272d600700 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-26T01:52:30.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.294+0000 7f272d600700 1 -- 172.21.15.151:0/1774741174 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2730022040 con 0x5634f2d27780 2024-04-26T01:52:30.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.294+0000 7f2754b01b80 1 -- 172.21.15.151:0/1774741174 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f273409ab60 con 0x5634f2d27780 2024-04-26T01:52:30.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.294+0000 7f272d600700 1 -- 172.21.15.151:0/1774741174 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f273000ebd0 con 0x5634f2d27780 2024-04-26T01:52:30.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.295+0000 7f272d600700 1 -- 172.21.15.151:0/1774741174 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2730005110 con 0x5634f2d27780 2024-04-26T01:52:30.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.295+0000 7f2754b01b80 10 client.24880 fetch_fsmap learned FSMap version 25 2024-04-26T01:52:30.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.295+0000 7f2754b01b80 1 -- 172.21.15.151:0/1774741174 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5634f2dac020 con 0x5634f2d27780 2024-04-26T01:52:30.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.295+0000 7f272d600700 1 -- 172.21.15.151:0/1774741174 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f273409ab60 con 0x5634f2d27780 2024-04-26T01:52:30.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.295+0000 7f272d600700 1 -- 172.21.15.151:0/1774741174 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f2730003be0 con 0x5634f2d27780 2024-04-26T01:52:30.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.295+0000 7f2754b01b80 10 client.24880 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T01:52:30.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.295+0000 7f2754b01b80 10 client.24880 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T01:52:30.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.295+0000 7f2754b01b80 20 client.24880 populate_metadata read hostname 'smithi151' 2024-04-26T01:52:30.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.295+0000 7f2754b01b80 1 --2- 172.21.15.151:0/1774741174 >> [v2:172.21.15.172:6836/55972537,v1:172.21.15.172:6837/55972537] conn(0x5634f2db08c0 0x5634f2daf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:30.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.295+0000 7f2754b01b80 4 client.24880 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T01:52:30.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.295+0000 7f2754b01b80 1 -- 172.21.15.151:0/1774741174 --> [v2:172.21.15.172:6836/55972537,v1:172.21.15.172:6837/55972537] -- command(tid 0: {"prefix": "status"}) -- 0x5634f2c7b6d0 con 0x5634f2db08c0 2024-04-26T01:52:30.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.296+0000 7f2743400700 1 --2- 172.21.15.151:0/1774741174 >> [v2:172.21.15.172:6836/55972537,v1:172.21.15.172:6837/55972537] conn(0x5634f2db08c0 0x5634f2daf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:30.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.296+0000 7f2743400700 1 --2- 172.21.15.151:0/1774741174 >> [v2:172.21.15.172:6836/55972537,v1:172.21.15.172:6837/55972537] conn(0x5634f2db08c0 0x5634f2daf5d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f7bcd6093e208b80 server_cookie=7e2475d89ed9e55d in_seq=0 out_seq=0 2024-04-26T01:52:30.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.296+0000 7f272d600700 10 client.24880 ms_handle_connect on v2:172.21.15.172:6836/55972537 2024-04-26T01:52:30.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.296+0000 7f272d600700 1 -- 172.21.15.151:0/1774741174 <== mds.2 v2:172.21.15.172:6836/55972537 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5634f2c7b6d0 con 0x5634f2db08c0 2024-04-26T01:52:30.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.296+0000 7f272d600700 10 client.24880 handle_command_reply: tid=0 2024-04-26T01:52:30.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.297+0000 7f2754b01b80 1 client.24880 shutdown 2024-04-26T01:52:30.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.297+0000 7f2754b01b80 1 -- 172.21.15.151:0/1774741174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634f2d27780 msgr2=0x5634f2d46880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:30.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.297+0000 7f2754b01b80 1 --2- 172.21.15.151:0/1774741174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634f2d27780 0x5634f2d46880 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2730013fd0 tx=0x7f27300091e0 comp rx=0 tx=0).stop 2024-04-26T01:52:30.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.297+0000 7f2754b01b80 1 -- 172.21.15.151:0/1774741174 shutdown_connections 2024-04-26T01:52:30.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.297+0000 7f2754b01b80 1 --2- 172.21.15.151:0/1774741174 >> [v2:172.21.15.172:6836/55972537,v1:172.21.15.172:6837/55972537] conn(0x5634f2db08c0 0x5634f2daf5d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:30.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.297+0000 7f2754b01b80 1 --2- 172.21.15.151:0/1774741174 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634f2d2b4e0 0x5634f2d414f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:30.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.297+0000 7f2754b01b80 1 --2- 172.21.15.151:0/1774741174 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634f2d21ef0 0x5634f2d46340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:30.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.297+0000 7f272d600700 0 client.24880 ms_handle_reset on v2:172.21.15.172:6836/55972537 2024-04-26T01:52:30.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.297+0000 7f2754b01b80 1 --2- 172.21.15.151:0/1774741174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634f2d27780 0x5634f2d46880 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:30.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.297+0000 7f2754b01b80 1 -- 172.21.15.151:0/1774741174 >> 172.21.15.151:0/1774741174 conn(0x5634f2c4ff00 msgr2=0x5634f2d29890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:30.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.297+0000 7f2754b01b80 1 -- 172.21.15.151:0/1774741174 shutdown_connections 2024-04-26T01:52:30.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.297+0000 7f2754b01b80 1 -- 172.21.15.151:0/1774741174 wait complete. 2024-04-26T01:52:30.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.297+0000 7f2754b01b80 20 client.24880 trim_cache size 0 max 16384 2024-04-26T01:52:30.299 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-26T01:52:30.299 INFO:teuthology.orchestra.run.smithi151.stdout: "cluster_fsid": "c0e49e64-036d-11ef-bc93-c7b262605968", 2024-04-26T01:52:30.299 INFO:teuthology.orchestra.run.smithi151.stdout: "whoami": 2, 2024-04-26T01:52:30.299 INFO:teuthology.orchestra.run.smithi151.stdout: "id": 24493, 2024-04-26T01:52:30.299 INFO:teuthology.orchestra.run.smithi151.stdout: "want_state": "up:active", 2024-04-26T01:52:30.299 INFO:teuthology.orchestra.run.smithi151.stdout: "state": "up:active", 2024-04-26T01:52:30.299 INFO:teuthology.orchestra.run.smithi151.stdout: "fs_name": "cephfs", 2024-04-26T01:52:30.299 INFO:teuthology.orchestra.run.smithi151.stdout: "rank_uptime": 151.52675289800001, 2024-04-26T01:52:30.299 INFO:teuthology.orchestra.run.smithi151.stdout: "mdsmap_epoch": 25, 2024-04-26T01:52:30.299 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch": 92, 2024-04-26T01:52:30.300 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch_barrier": 90, 2024-04-26T01:52:30.300 INFO:teuthology.orchestra.run.smithi151.stdout: "uptime": 210.03899207200001 2024-04-26T01:52:30.300 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-26T01:52:30.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.298+0000 7f274a200700 1 -- 172.21.15.151:0/2432903321 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f27280a3c40 msgr2=0x7f27280a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:30.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.298+0000 7f274a200700 1 --2- 172.21.15.151:0/2432903321 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f27280a3c40 0x7f27280a6100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f274410b510 tx=0x7f2730009260 comp rx=0 tx=0).stop 2024-04-26T01:52:30.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.298+0000 7f274a200700 1 -- 172.21.15.151:0/2432903321 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2744104e90 msgr2=0x7f274419b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:30.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.298+0000 7f274a200700 1 --2- 172.21.15.151:0/2432903321 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2744104e90 0x7f274419b310 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f273400e9a0 tx=0x7f273400ecb0 comp rx=0 tx=0).stop 2024-04-26T01:52:30.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.298+0000 7f274a200700 1 -- 172.21.15.151:0/2432903321 shutdown_connections 2024-04-26T01:52:30.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.298+0000 7f274a200700 1 --2- 172.21.15.151:0/2432903321 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f27280a3c40 0x7f27280a6100 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:30.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.298+0000 7f274a200700 1 --2- 172.21.15.151:0/2432903321 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2744109f70 0x7f274419fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:30.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.298+0000 7f274a200700 1 --2- 172.21.15.151:0/2432903321 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2744104e90 0x7f274419b310 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:30.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.298+0000 7f274a200700 1 --2- 172.21.15.151:0/2432903321 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2744105840 0x7f274419b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:30.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.298+0000 7f274a200700 1 -- 172.21.15.151:0/2432903321 >> 172.21.15.151:0/2432903321 conn(0x7f2744100720 msgr2=0x7f2744101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:30.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.298+0000 7f274a200700 1 -- 172.21.15.151:0/2432903321 shutdown_connections 2024-04-26T01:52:30.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:30.298+0000 7f274a200700 1 -- 172.21.15.151:0/2432903321 wait complete. 2024-04-26T01:52:31.222 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph tell mds.i status 2024-04-26T01:52:31.300 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:52:31 smithi151 ceph-mon[28057]: pgmap v414: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:52:31.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:52:31 smithi165 ceph-mon[32648]: pgmap v414: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:52:31.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:52:31 smithi172 ceph-mon[32785]: pgmap v414: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:52:31.672 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:52:33.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:52:33 smithi165 ceph-mon[32648]: pgmap v415: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:52:33.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:52:33 smithi151 ceph-mon[28057]: pgmap v415: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:52:33.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:52:33 smithi172 ceph-mon[32785]: pgmap v415: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:52:34.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.016+0000 7f170f600700 1 -- 172.21.15.151:0/1245054337 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1708108a30 msgr2=0x7f1708108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:34.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.016+0000 7f170f600700 1 --2- 172.21.15.151:0/1245054337 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1708108a30 0x7f1708108e10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f16fc009f90 tx=0x7f16fc00b330 comp rx=0 tx=0).stop 2024-04-26T01:52:34.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.016+0000 7f170f600700 1 -- 172.21.15.151:0/1245054337 shutdown_connections 2024-04-26T01:52:34.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.016+0000 7f170f600700 1 --2- 172.21.15.151:0/1245054337 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f170806b0f0 0x7f170810f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.016+0000 7f170f600700 1 --2- 172.21.15.151:0/1245054337 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1708108a30 0x7f1708108e10 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.016+0000 7f170f600700 1 --2- 172.21.15.151:0/1245054337 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f170806a750 0x7f170806abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.016+0000 7f170f600700 1 -- 172.21.15.151:0/1245054337 >> 172.21.15.151:0/1245054337 conn(0x7f17080788f0 msgr2=0x7f1708078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:34.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.016+0000 7f170f600700 1 -- 172.21.15.151:0/1245054337 shutdown_connections 2024-04-26T01:52:34.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.016+0000 7f170f600700 1 -- 172.21.15.151:0/1245054337 wait complete. 2024-04-26T01:52:34.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.017+0000 7f170f600700 1 Processor -- start 2024-04-26T01:52:34.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.017+0000 7f170f600700 1 -- start start 2024-04-26T01:52:34.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.018+0000 7f170f600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f170806a750 0x7f170819b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:34.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.018+0000 7f170f600700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f170806b0f0 0x7f170819b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:34.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.018+0000 7f170f600700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1708108a30 0x7f170819fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:34.017 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.018+0000 7f170f600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1708112c00 con 0x7f170806a750 2024-04-26T01:52:34.017 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.018+0000 7f170f600700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1708112a80 con 0x7f1708108a30 2024-04-26T01:52:34.017 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.018+0000 7f170f600700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1708112d80 con 0x7f170806b0f0 2024-04-26T01:52:34.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.018+0000 7f170cc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f170806a750 0x7f170819b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:34.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.018+0000 7f170d600700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1708108a30 0x7f170819fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:34.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.018+0000 7f170cc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f170806a750 0x7f170819b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42570/0 (socket says 172.21.15.151:42570) 2024-04-26T01:52:34.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.018+0000 7f170cc00700 1 -- 172.21.15.151:0/2560643425 learned_addr learned my addr 172.21.15.151:0/2560643425 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:52:34.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.018+0000 7f1707e00700 1 --2- 172.21.15.151:0/2560643425 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f170806b0f0 0x7f170819b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:34.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.019+0000 7f170cc00700 1 -- 172.21.15.151:0/2560643425 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f170806b0f0 msgr2=0x7f170819b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:34.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.019+0000 7f170cc00700 1 --2- 172.21.15.151:0/2560643425 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f170806b0f0 0x7f170819b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.019+0000 7f170cc00700 1 -- 172.21.15.151:0/2560643425 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1708108a30 msgr2=0x7f170819fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:34.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.019+0000 7f170cc00700 1 --2- 172.21.15.151:0/2560643425 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1708108a30 0x7f170819fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.019+0000 7f170cc00700 1 -- 172.21.15.151:0/2560643425 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17081a03a0 con 0x7f170806a750 2024-04-26T01:52:34.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.019+0000 7f170d600700 1 --2- 172.21.15.151:0/2560643425 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1708108a30 0x7f170819fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:52:34.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.019+0000 7f170cc00700 1 --2- 172.21.15.151:0/2560643425 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f170806a750 0x7f170819b390 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f16fc009f60 tx=0x7f16fc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:34.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.019+0000 7f1705600700 1 -- 172.21.15.151:0/2560643425 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16fc021070 con 0x7f170806a750 2024-04-26T01:52:34.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.019+0000 7f170f600700 1 -- 172.21.15.151:0/2560643425 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17081a0630 con 0x7f170806a750 2024-04-26T01:52:34.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.019+0000 7f1705600700 1 -- 172.21.15.151:0/2560643425 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16fc007ac0 con 0x7f170806a750 2024-04-26T01:52:34.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.019+0000 7f1705600700 1 -- 172.21.15.151:0/2560643425 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16fc005040 con 0x7f170806a750 2024-04-26T01:52:34.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.020+0000 7f170f600700 1 -- 172.21.15.151:0/2560643425 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17081a7f10 con 0x7f170806a750 2024-04-26T01:52:34.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.023+0000 7f1705600700 1 -- 172.21.15.151:0/2560643425 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f16fc024070 con 0x7f170806a750 2024-04-26T01:52:34.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.024+0000 7f1705600700 1 --2- 172.21.15.151:0/2560643425 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f16f00a3cc0 0x7f16f00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:34.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.024+0000 7f1705600700 1 -- 172.21.15.151:0/2560643425 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f16fc00f020 con 0x7f170806a750 2024-04-26T01:52:34.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.024+0000 7f1719f4ab80 1 Processor -- start 2024-04-26T01:52:34.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.024+0000 7f1707e00700 1 --2- 172.21.15.151:0/2560643425 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f16f00a3cc0 0x7f16f00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:34.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.025+0000 7f1719f4ab80 1 -- start start 2024-04-26T01:52:34.024 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.025+0000 7f1719f4ab80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562221aec2f0 0x562221aec6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:34.024 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.025+0000 7f1719f4ab80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562221aecc10 0x562221af7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:34.024 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.025+0000 7f1719f4ab80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562221af7ab0 0x562221af9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:34.024 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.025+0000 7f1719f4ab80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562221a3fe40 con 0x562221aec2f0 2024-04-26T01:52:34.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.025+0000 7f1719f4ab80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5622219e66e0 con 0x562221aecc10 2024-04-26T01:52:34.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.025+0000 7f1719f4ab80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562221a3be00 con 0x562221af7ab0 2024-04-26T01:52:34.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.025+0000 7f1707e00700 1 --2- 172.21.15.151:0/2560643425 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f16f00a3cc0 0x7f16f00a6180 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1708079030 tx=0x7f16f4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:34.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.025+0000 7f170cc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562221aec2f0 0x562221aec6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:34.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.026+0000 7f170cc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562221aec2f0 0x562221aec6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42586/0 (socket says 172.21.15.151:42586) 2024-04-26T01:52:34.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.026+0000 7f170cc00700 1 -- 172.21.15.151:0/289425504 learned_addr learned my addr 172.21.15.151:0/289425504 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:52:34.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.026+0000 7f1707e00700 1 --2- 172.21.15.151:0/289425504 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562221aecc10 0x562221af7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:34.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.026+0000 7f170d600700 1 --2- 172.21.15.151:0/289425504 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562221af7ab0 0x562221af9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:34.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.026+0000 7f170cc00700 1 -- 172.21.15.151:0/289425504 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562221af7ab0 msgr2=0x562221af9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:34.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.026+0000 7f170cc00700 1 --2- 172.21.15.151:0/289425504 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562221af7ab0 0x562221af9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.026+0000 7f170cc00700 1 -- 172.21.15.151:0/289425504 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562221aecc10 msgr2=0x562221af7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:34.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.026+0000 7f170cc00700 1 --2- 172.21.15.151:0/289425504 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562221aecc10 0x562221af7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.026+0000 7f170cc00700 1 -- 172.21.15.151:0/289425504 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622219dc630 con 0x562221aec2f0 2024-04-26T01:52:34.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.026+0000 7f170cc00700 1 --2- 172.21.15.151:0/289425504 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562221aec2f0 0x562221aec6d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f16fc099f00 tx=0x7f16fc00cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b7a320298ea6c1e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:34.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.027+0000 7f16fa000700 1 -- 172.21.15.151:0/289425504 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16fc0a1e50 con 0x562221aec2f0 2024-04-26T01:52:34.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.027+0000 7f16fa000700 1 -- 172.21.15.151:0/289425504 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16fc0a6680 con 0x562221aec2f0 2024-04-26T01:52:34.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.027+0000 7f16fa000700 1 -- 172.21.15.151:0/289425504 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16fc09df60 con 0x562221aec2f0 2024-04-26T01:52:34.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.027+0000 7f1719f4ab80 1 -- 172.21.15.151:0/289425504 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562221aec2f0 msgr2=0x562221aec6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:34.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.027+0000 7f1719f4ab80 1 --2- 172.21.15.151:0/289425504 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562221aec2f0 0x562221aec6d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f16fc099f00 tx=0x7f16fc00cf50 comp rx=0 tx=0).stop 2024-04-26T01:52:34.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.028+0000 7f1719f4ab80 1 -- 172.21.15.151:0/289425504 shutdown_connections 2024-04-26T01:52:34.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.028+0000 7f1719f4ab80 1 --2- 172.21.15.151:0/289425504 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562221af7ab0 0x562221af9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.028+0000 7f1719f4ab80 1 --2- 172.21.15.151:0/289425504 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562221aec2f0 0x562221aec6d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.028+0000 7f1719f4ab80 1 --2- 172.21.15.151:0/289425504 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562221aecc10 0x562221af7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.028+0000 7f1719f4ab80 1 -- 172.21.15.151:0/289425504 >> 172.21.15.151:0/289425504 conn(0x562221a1b550 msgr2=0x562221ae8790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:34.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.028+0000 7f1719f4ab80 1 -- 172.21.15.151:0/289425504 shutdown_connections 2024-04-26T01:52:34.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.028+0000 7f1719f4ab80 1 -- 172.21.15.151:0/289425504 wait complete. 2024-04-26T01:52:34.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.028+0000 7f1719f4ab80 1 -- start start 2024-04-26T01:52:34.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.029+0000 7f1719f4ab80 1 Processor -- start 2024-04-26T01:52:34.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.029+0000 7f1719f4ab80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562221aec2f0 0x562221af7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:34.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.029+0000 7f1719f4ab80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562221aecc10 0x562221af4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:34.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.029+0000 7f1719f4ab80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562221af7ab0 0x562221af4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:34.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.029+0000 7f1719f4ab80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562221a3fe40 con 0x562221aec2f0 2024-04-26T01:52:34.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.029+0000 7f1719f4ab80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5622219f9810 con 0x562221af7ab0 2024-04-26T01:52:34.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.029+0000 7f1719f4ab80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562221afa3e0 con 0x562221aecc10 2024-04-26T01:52:34.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.029+0000 7f1719f4ab80 10 client.0 _ll_register_callbacks cb 0x5622219d86d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:52:34.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.029+0000 7f170cc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562221aec2f0 0x562221af7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:34.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.029+0000 7f170cc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562221aec2f0 0x562221af7270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42598/0 (socket says 172.21.15.151:42598) 2024-04-26T01:52:34.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.029+0000 7f170cc00700 1 -- 172.21.15.151:0/3056627875 learned_addr learned my addr 172.21.15.151:0/3056627875 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:52:34.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.030+0000 7f1707e00700 1 --2- 172.21.15.151:0/3056627875 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562221aecc10 0x562221af4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:34.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.030+0000 7f170d600700 1 --2- 172.21.15.151:0/3056627875 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562221af7ab0 0x562221af4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:34.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.030+0000 7f170cc00700 1 -- 172.21.15.151:0/3056627875 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562221aecc10 msgr2=0x562221af4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:34.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.030+0000 7f170cc00700 1 --2- 172.21.15.151:0/3056627875 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562221aecc10 0x562221af4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.030+0000 7f170cc00700 1 -- 172.21.15.151:0/3056627875 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562221af7ab0 msgr2=0x562221af4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:34.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.030+0000 7f170cc00700 1 --2- 172.21.15.151:0/3056627875 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562221af7ab0 0x562221af4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.030+0000 7f170cc00700 1 -- 172.21.15.151:0/3056627875 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56222198d880 con 0x562221aec2f0 2024-04-26T01:52:34.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.030+0000 7f170d600700 1 --2- 172.21.15.151:0/3056627875 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562221af7ab0 0x562221af4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T01:52:34.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.030+0000 7f170cc00700 1 --2- 172.21.15.151:0/3056627875 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562221aec2f0 0x562221af7270 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f16fc088040 tx=0x7f16fc009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1da81cb6194a1054 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:34.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.030+0000 7f16eea00700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-04-26T01:52:34.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.030+0000 7f16eea00700 1 -- 172.21.15.151:0/3056627875 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16fc0a1e50 con 0x562221aec2f0 2024-04-26T01:52:34.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.030+0000 7f16eea00700 1 -- 172.21.15.151:0/3056627875 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f16fc007420 con 0x562221aec2f0 2024-04-26T01:52:34.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.030+0000 7f1719f4ab80 1 -- 172.21.15.151:0/3056627875 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16fc0a6680 con 0x562221aec2f0 2024-04-26T01:52:34.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.031+0000 7f16eea00700 1 -- 172.21.15.151:0/3056627875 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f16fc0bcbd0 con 0x562221aec2f0 2024-04-26T01:52:34.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.031+0000 7f1719f4ab80 10 client.14772 fetch_fsmap learned FSMap version 25 2024-04-26T01:52:34.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.031+0000 7f1719f4ab80 1 -- 172.21.15.151:0/3056627875 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5622219d6aa0 con 0x562221aec2f0 2024-04-26T01:52:34.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.031+0000 7f16eea00700 1 -- 172.21.15.151:0/3056627875 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16fc0a6680 con 0x562221aec2f0 2024-04-26T01:52:34.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.031+0000 7f16eea00700 1 -- 172.21.15.151:0/3056627875 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f16fc011070 con 0x562221aec2f0 2024-04-26T01:52:34.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.031+0000 7f1719f4ab80 10 client.14772 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T01:52:34.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.031+0000 7f1719f4ab80 10 client.14772 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-26T01:52:34.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.031+0000 7f1719f4ab80 20 client.14772 populate_metadata read hostname 'smithi151' 2024-04-26T01:52:34.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.031+0000 7f1719f4ab80 1 --2- 172.21.15.151:0/3056627875 >> [v2:172.21.15.172:6838/1242302810,v1:172.21.15.172:6839/1242302810] conn(0x562221b06d80 0x562221b081f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:34.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.031+0000 7f1719f4ab80 4 client.14772 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:52:34.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.031+0000 7f1719f4ab80 1 -- 172.21.15.151:0/3056627875 --> [v2:172.21.15.172:6838/1242302810,v1:172.21.15.172:6839/1242302810] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562221a4d630 con 0x562221b06d80 2024-04-26T01:52:34.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.032+0000 7f1707e00700 1 --2- 172.21.15.151:0/3056627875 >> [v2:172.21.15.172:6838/1242302810,v1:172.21.15.172:6839/1242302810] conn(0x562221b06d80 0x562221b081f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:34.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.032+0000 7f1707e00700 1 --2- 172.21.15.151:0/3056627875 >> [v2:172.21.15.172:6838/1242302810,v1:172.21.15.172:6839/1242302810] conn(0x562221b06d80 0x562221b081f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=916bbcc3aabefbb8 server_cookie=71d98c22bb9a2825 in_seq=0 out_seq=0 2024-04-26T01:52:34.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.032+0000 7f16eea00700 10 client.14772 ms_handle_connect on v2:172.21.15.172:6838/1242302810 2024-04-26T01:52:34.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.033+0000 7f16eea00700 1 -- 172.21.15.151:0/3056627875 <== mds.1 v2:172.21.15.172:6838/1242302810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562221a4d630 con 0x562221b06d80 2024-04-26T01:52:34.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.033+0000 7f16eea00700 10 client.14772 handle_command_reply: tid=0 2024-04-26T01:52:34.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.033+0000 7f1719f4ab80 1 client.14772 shutdown 2024-04-26T01:52:34.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.034+0000 7f1719f4ab80 1 -- 172.21.15.151:0/3056627875 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562221aec2f0 msgr2=0x562221af7270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:34.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.034+0000 7f1719f4ab80 1 --2- 172.21.15.151:0/3056627875 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562221aec2f0 0x562221af7270 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f16fc088040 tx=0x7f16fc009920 comp rx=0 tx=0).stop 2024-04-26T01:52:34.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.034+0000 7f1719f4ab80 1 -- 172.21.15.151:0/3056627875 shutdown_connections 2024-04-26T01:52:34.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.034+0000 7f1719f4ab80 1 --2- 172.21.15.151:0/3056627875 >> [v2:172.21.15.172:6838/1242302810,v1:172.21.15.172:6839/1242302810] conn(0x562221b06d80 0x562221b081f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.034+0000 7f1719f4ab80 1 --2- 172.21.15.151:0/3056627875 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562221af7ab0 0x562221af4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.034+0000 7f1719f4ab80 1 --2- 172.21.15.151:0/3056627875 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562221aec2f0 0x562221af7270 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.034+0000 7f1719f4ab80 1 --2- 172.21.15.151:0/3056627875 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562221aecc10 0x562221af4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.034+0000 7f16eea00700 0 client.14772 ms_handle_reset on v2:172.21.15.172:6838/1242302810 2024-04-26T01:52:34.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.034+0000 7f1719f4ab80 1 -- 172.21.15.151:0/3056627875 >> 172.21.15.151:0/3056627875 conn(0x562221a1b550 msgr2=0x562221aeec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:34.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.034+0000 7f1719f4ab80 1 -- 172.21.15.151:0/3056627875 shutdown_connections 2024-04-26T01:52:34.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.034+0000 7f1719f4ab80 1 -- 172.21.15.151:0/3056627875 wait complete. 2024-04-26T01:52:34.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.034+0000 7f1719f4ab80 20 client.14772 trim_cache size 0 max 16384 2024-04-26T01:52:34.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.056+0000 7f1719f4ab80 1 Processor -- start 2024-04-26T01:52:34.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.057+0000 7f1719f4ab80 1 -- start start 2024-04-26T01:52:34.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.057+0000 7f1719f4ab80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562221aec2f0 0x562221af4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:34.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.057+0000 7f1719f4ab80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562221aecc10 0x562221af1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:34.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.057+0000 7f1719f4ab80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562221af7ab0 0x562221af1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:34.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.057+0000 7f1719f4ab80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562221b06530 con 0x562221af7ab0 2024-04-26T01:52:34.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.057+0000 7f1719f4ab80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562221b06c00 con 0x562221aec2f0 2024-04-26T01:52:34.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.057+0000 7f1719f4ab80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562221b072d0 con 0x562221aecc10 2024-04-26T01:52:34.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.057+0000 7f170cc00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562221aec2f0 0x562221af4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:34.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.057+0000 7f170cc00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562221aec2f0 0x562221af4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:45698/0 (socket says 172.21.15.151:45698) 2024-04-26T01:52:34.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.057+0000 7f170cc00700 1 -- 172.21.15.151:0/3787391344 learned_addr learned my addr 172.21.15.151:0/3787391344 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:52:34.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.057+0000 7f1707e00700 1 --2- 172.21.15.151:0/3787391344 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562221aecc10 0x562221af1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:34.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.057+0000 7f170d600700 1 --2- 172.21.15.151:0/3787391344 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562221af7ab0 0x562221af1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:34.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.058+0000 7f1707e00700 1 -- 172.21.15.151:0/3787391344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562221aec2f0 msgr2=0x562221af4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:34.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.058+0000 7f1707e00700 1 --2- 172.21.15.151:0/3787391344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562221aec2f0 0x562221af4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.058+0000 7f1707e00700 1 -- 172.21.15.151:0/3787391344 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562221af7ab0 msgr2=0x562221af1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:34.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.058+0000 7f1707e00700 1 --2- 172.21.15.151:0/3787391344 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562221af7ab0 0x562221af1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.058+0000 7f1707e00700 1 -- 172.21.15.151:0/3787391344 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562221aedb60 con 0x562221aecc10 2024-04-26T01:52:34.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.058+0000 7f1707e00700 1 --2- 172.21.15.151:0/3787391344 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562221aecc10 0x562221af1810 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x562221b08bb0 tx=0x7f16f4014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb6e4c57fef6dd11 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:34.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.058+0000 7f16f9600700 1 -- 172.21.15.151:0/3787391344 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f4032070 con 0x562221aecc10 2024-04-26T01:52:34.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.058+0000 7f16f9600700 1 -- 172.21.15.151:0/3787391344 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16f402c8e0 con 0x562221aecc10 2024-04-26T01:52:34.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.058+0000 7f16f9600700 1 -- 172.21.15.151:0/3787391344 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f401d670 con 0x562221aecc10 2024-04-26T01:52:34.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.059+0000 7f1719f4ab80 1 -- 172.21.15.151:0/3787391344 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562221aecc10 msgr2=0x562221af1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:34.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.059+0000 7f1719f4ab80 1 --2- 172.21.15.151:0/3787391344 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562221aecc10 0x562221af1810 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x562221b08bb0 tx=0x7f16f4014000 comp rx=0 tx=0).stop 2024-04-26T01:52:34.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.059+0000 7f1719f4ab80 1 -- 172.21.15.151:0/3787391344 shutdown_connections 2024-04-26T01:52:34.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.059+0000 7f1719f4ab80 1 --2- 172.21.15.151:0/3787391344 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562221af7ab0 0x562221af1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.059+0000 7f1719f4ab80 1 --2- 172.21.15.151:0/3787391344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562221aec2f0 0x562221af4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.059+0000 7f1719f4ab80 1 --2- 172.21.15.151:0/3787391344 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562221aecc10 0x562221af1810 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.059+0000 7f1719f4ab80 1 -- 172.21.15.151:0/3787391344 >> 172.21.15.151:0/3787391344 conn(0x562221a1b550 msgr2=0x562221af4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:34.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.060+0000 7f1719f4ab80 1 -- 172.21.15.151:0/3787391344 shutdown_connections 2024-04-26T01:52:34.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.060+0000 7f1719f4ab80 1 -- 172.21.15.151:0/3787391344 wait complete. 2024-04-26T01:52:34.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.060+0000 7f1719f4ab80 1 -- start start 2024-04-26T01:52:34.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.060+0000 7f1719f4ab80 1 Processor -- start 2024-04-26T01:52:34.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.061+0000 7f1719f4ab80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562221aec2f0 0x562221b115f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:34.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.061+0000 7f1719f4ab80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562221aecc10 0x562221b11b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:34.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.061+0000 7f1719f4ab80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562221af7ab0 0x562221b0dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:34.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.061+0000 7f1719f4ab80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562221b08070 con 0x562221aec2f0 2024-04-26T01:52:34.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.061+0000 7f1719f4ab80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562221b079a0 con 0x562221af7ab0 2024-04-26T01:52:34.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.061+0000 7f1719f4ab80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562221b08740 con 0x562221aecc10 2024-04-26T01:52:34.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.061+0000 7f1719f4ab80 10 client.0 _ll_register_callbacks cb 0x5622219d86d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:52:34.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.061+0000 7f170cc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562221aec2f0 0x562221b115f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:34.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.061+0000 7f170cc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562221aec2f0 0x562221b115f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42626/0 (socket says 172.21.15.151:42626) 2024-04-26T01:52:34.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.061+0000 7f170cc00700 1 -- 172.21.15.151:0/2637470680 learned_addr learned my addr 172.21.15.151:0/2637470680 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:52:34.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.061+0000 7f170d600700 1 --2- 172.21.15.151:0/2637470680 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562221af7ab0 0x562221b0dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:34.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.061+0000 7f1707e00700 1 --2- 172.21.15.151:0/2637470680 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562221aecc10 0x562221b11b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:34.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.061+0000 7f170cc00700 1 -- 172.21.15.151:0/2637470680 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562221aecc10 msgr2=0x562221b11b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:34.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.061+0000 7f170cc00700 1 --2- 172.21.15.151:0/2637470680 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562221aecc10 0x562221b11b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.061+0000 7f170cc00700 1 -- 172.21.15.151:0/2637470680 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562221af7ab0 msgr2=0x562221b0dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:34.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.061+0000 7f170cc00700 1 --2- 172.21.15.151:0/2637470680 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562221af7ab0 0x562221b0dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.061+0000 7f170cc00700 1 -- 172.21.15.151:0/2637470680 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562221b0e410 con 0x562221aec2f0 2024-04-26T01:52:34.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.061+0000 7f170d600700 1 --2- 172.21.15.151:0/2637470680 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562221af7ab0 0x562221b0dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:52:34.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.062+0000 7f170cc00700 1 --2- 172.21.15.151:0/2637470680 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562221aec2f0 0x562221b115f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f16fc09bd50 tx=0x7f16fc00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6f74be06f529943 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:34.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.062+0000 7f16eea00700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-04-26T01:52:34.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.062+0000 7f16eea00700 1 -- 172.21.15.151:0/2637470680 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16fc09d400 con 0x562221aec2f0 2024-04-26T01:52:34.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.062+0000 7f16eea00700 1 -- 172.21.15.151:0/2637470680 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f16fc0a63f0 con 0x562221aec2f0 2024-04-26T01:52:34.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.062+0000 7f1719f4ab80 1 -- 172.21.15.151:0/2637470680 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16f402c8e0 con 0x562221aec2f0 2024-04-26T01:52:34.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.062+0000 7f16eea00700 1 -- 172.21.15.151:0/2637470680 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f16fc09cef0 con 0x562221aec2f0 2024-04-26T01:52:34.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.062+0000 7f1719f4ab80 10 client.14778 fetch_fsmap learned FSMap version 25 2024-04-26T01:52:34.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.062+0000 7f1719f4ab80 1 -- 172.21.15.151:0/2637470680 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562221b0e410 con 0x562221aec2f0 2024-04-26T01:52:34.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.063+0000 7f16eea00700 1 -- 172.21.15.151:0/2637470680 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16f402c8e0 con 0x562221aec2f0 2024-04-26T01:52:34.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.063+0000 7f16eea00700 1 -- 172.21.15.151:0/2637470680 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f16fc011040 con 0x562221aec2f0 2024-04-26T01:52:34.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.063+0000 7f1719f4ab80 10 client.14778 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T01:52:34.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.063+0000 7f1719f4ab80 10 client.14778 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-26T01:52:34.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.063+0000 7f1719f4ab80 20 client.14778 populate_metadata read hostname 'smithi151' 2024-04-26T01:52:34.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.063+0000 7f1719f4ab80 1 --2- 172.21.15.151:0/2637470680 >> [v2:172.21.15.172:6838/1242302810,v1:172.21.15.172:6839/1242302810] conn(0x562221b73230 0x562221b71f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:34.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.063+0000 7f1719f4ab80 4 client.14778 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T01:52:34.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.063+0000 7f1719f4ab80 1 -- 172.21.15.151:0/2637470680 --> [v2:172.21.15.172:6838/1242302810,v1:172.21.15.172:6839/1242302810] -- command(tid 0: {"prefix": "status"}) -- 0x562221a476d0 con 0x562221b73230 2024-04-26T01:52:34.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.063+0000 7f1707e00700 1 --2- 172.21.15.151:0/2637470680 >> [v2:172.21.15.172:6838/1242302810,v1:172.21.15.172:6839/1242302810] conn(0x562221b73230 0x562221b71f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:34.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.064+0000 7f1707e00700 1 --2- 172.21.15.151:0/2637470680 >> [v2:172.21.15.172:6838/1242302810,v1:172.21.15.172:6839/1242302810] conn(0x562221b73230 0x562221b71f40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=52f104514ee24fbc server_cookie=6605a9d263a18e in_seq=0 out_seq=0 2024-04-26T01:52:34.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.064+0000 7f16eea00700 10 client.14778 ms_handle_connect on v2:172.21.15.172:6838/1242302810 2024-04-26T01:52:34.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.064+0000 7f16eea00700 1 -- 172.21.15.151:0/2637470680 <== mds.1 v2:172.21.15.172:6838/1242302810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x562221a476d0 con 0x562221b73230 2024-04-26T01:52:34.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.064+0000 7f16eea00700 10 client.14778 handle_command_reply: tid=0 2024-04-26T01:52:34.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.064+0000 7f1719f4ab80 1 client.14778 shutdown 2024-04-26T01:52:34.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.064+0000 7f1719f4ab80 1 -- 172.21.15.151:0/2637470680 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562221aec2f0 msgr2=0x562221b115f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:34.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.064+0000 7f1719f4ab80 1 --2- 172.21.15.151:0/2637470680 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562221aec2f0 0x562221b115f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f16fc09bd50 tx=0x7f16fc00b870 comp rx=0 tx=0).stop 2024-04-26T01:52:34.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.064+0000 7f1719f4ab80 1 -- 172.21.15.151:0/2637470680 shutdown_connections 2024-04-26T01:52:34.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.064+0000 7f1719f4ab80 1 --2- 172.21.15.151:0/2637470680 >> [v2:172.21.15.172:6838/1242302810,v1:172.21.15.172:6839/1242302810] conn(0x562221b73230 0x562221b71f40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.064+0000 7f1719f4ab80 1 --2- 172.21.15.151:0/2637470680 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562221af7ab0 0x562221b0dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.064+0000 7f1719f4ab80 1 --2- 172.21.15.151:0/2637470680 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562221aec2f0 0x562221b115f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.064+0000 7f1719f4ab80 1 --2- 172.21.15.151:0/2637470680 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562221aecc10 0x562221b11b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.064+0000 7f1719f4ab80 1 -- 172.21.15.151:0/2637470680 >> 172.21.15.151:0/2637470680 conn(0x562221a1b550 msgr2=0x562221ae70c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:34.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.064+0000 7f16eea00700 0 client.14778 ms_handle_reset on v2:172.21.15.172:6838/1242302810 2024-04-26T01:52:34.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.065+0000 7f1719f4ab80 1 -- 172.21.15.151:0/2637470680 shutdown_connections 2024-04-26T01:52:34.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.065+0000 7f1719f4ab80 1 -- 172.21.15.151:0/2637470680 wait complete. 2024-04-26T01:52:34.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.065+0000 7f1719f4ab80 20 client.14778 trim_cache size 0 max 16384 2024-04-26T01:52:34.067 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-26T01:52:34.067 INFO:teuthology.orchestra.run.smithi151.stdout: "cluster_fsid": "c0e49e64-036d-11ef-bc93-c7b262605968", 2024-04-26T01:52:34.067 INFO:teuthology.orchestra.run.smithi151.stdout: "whoami": 1, 2024-04-26T01:52:34.068 INFO:teuthology.orchestra.run.smithi151.stdout: "id": 24505, 2024-04-26T01:52:34.068 INFO:teuthology.orchestra.run.smithi151.stdout: "want_state": "up:active", 2024-04-26T01:52:34.068 INFO:teuthology.orchestra.run.smithi151.stdout: "state": "up:active", 2024-04-26T01:52:34.068 INFO:teuthology.orchestra.run.smithi151.stdout: "fs_name": "cephfs", 2024-04-26T01:52:34.068 INFO:teuthology.orchestra.run.smithi151.stdout: "rank_uptime": 156.306335905, 2024-04-26T01:52:34.068 INFO:teuthology.orchestra.run.smithi151.stdout: "mdsmap_epoch": 25, 2024-04-26T01:52:34.068 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch": 92, 2024-04-26T01:52:34.068 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch_barrier": 90, 2024-04-26T01:52:34.068 INFO:teuthology.orchestra.run.smithi151.stdout: "uptime": 208.27906371399999 2024-04-26T01:52:34.068 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-26T01:52:34.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.065+0000 7f170f600700 1 -- 172.21.15.151:0/2560643425 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f16f00a3cc0 msgr2=0x7f16f00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:34.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.065+0000 7f170f600700 1 --2- 172.21.15.151:0/2560643425 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f16f00a3cc0 0x7f16f00a6180 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1708079030 tx=0x7f16f4009260 comp rx=0 tx=0).stop 2024-04-26T01:52:34.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.065+0000 7f170f600700 1 -- 172.21.15.151:0/2560643425 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f170806a750 msgr2=0x7f170819b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:34.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.065+0000 7f170f600700 1 --2- 172.21.15.151:0/2560643425 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f170806a750 0x7f170819b390 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f16fc009f60 tx=0x7f16fc00ccb0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.066+0000 7f170f600700 1 -- 172.21.15.151:0/2560643425 shutdown_connections 2024-04-26T01:52:34.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.066+0000 7f170f600700 1 --2- 172.21.15.151:0/2560643425 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1708108a30 0x7f170819fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.066+0000 7f170f600700 1 --2- 172.21.15.151:0/2560643425 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f16f00a3cc0 0x7f16f00a6180 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.066+0000 7f170f600700 1 --2- 172.21.15.151:0/2560643425 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f170806a750 0x7f170819b390 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.066+0000 7f170f600700 1 --2- 172.21.15.151:0/2560643425 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f170806b0f0 0x7f170819b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.066+0000 7f170f600700 1 -- 172.21.15.151:0/2560643425 >> 172.21.15.151:0/2560643425 conn(0x7f17080788f0 msgr2=0x7f170810df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:34.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.066+0000 7f170f600700 1 -- 172.21.15.151:0/2560643425 shutdown_connections 2024-04-26T01:52:34.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:34.066+0000 7f170f600700 1 -- 172.21.15.151:0/2560643425 wait complete. 2024-04-26T01:52:34.816 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph tell mds.b status 2024-04-26T01:52:35.270 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:52:35.338 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:52:35 smithi172 ceph-mon[32785]: pgmap v416: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:52:35.345 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:52:35 smithi165 ceph-mon[32648]: pgmap v416: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:52:35.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:52:35 smithi151 ceph-mon[28057]: pgmap v416: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:52:37.232 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:52:37 smithi151 ceph-mon[28057]: pgmap v417: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:52:37.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:52:37 smithi165 ceph-mon[32648]: pgmap v417: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:52:37.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.406+0000 7f8374600700 1 -- 172.21.15.151:0/1128878929 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f836c06b110 msgr2=0x7f836c06b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:37.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.406+0000 7f8374600700 1 --2- 172.21.15.151:0/1128878929 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f836c06b110 0x7f836c06b570 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8358009990 tx=0x7f8358009ca0 comp rx=0 tx=0).stop 2024-04-26T01:52:37.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.406+0000 7f8374600700 1 -- 172.21.15.151:0/1128878929 shutdown_connections 2024-04-26T01:52:37.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.406+0000 7f8374600700 1 --2- 172.21.15.151:0/1128878929 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f836c06bab0 0x7f836c10f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:37.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.406+0000 7f8374600700 1 --2- 172.21.15.151:0/1128878929 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f836c108a50 0x7f836c108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:37.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.406+0000 7f8374600700 1 --2- 172.21.15.151:0/1128878929 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f836c06b110 0x7f836c06b570 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:37.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.406+0000 7f8374600700 1 -- 172.21.15.151:0/1128878929 >> 172.21.15.151:0/1128878929 conn(0x7f836c0788f0 msgr2=0x7f836c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:37.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.406+0000 7f8374600700 1 -- 172.21.15.151:0/1128878929 shutdown_connections 2024-04-26T01:52:37.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.406+0000 7f8374600700 1 -- 172.21.15.151:0/1128878929 wait complete. 2024-04-26T01:52:37.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.407+0000 7f8374600700 1 Processor -- start 2024-04-26T01:52:37.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.407+0000 7f8374600700 1 -- start start 2024-04-26T01:52:37.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.407+0000 7f8374600700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f836c06b110 0x7f836c1991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:37.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.407+0000 7f8374600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f836c06bab0 0x7f836c1996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:37.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.407+0000 7f8374600700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f836c108a50 0x7f836c19da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:37.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.407+0000 7f8374600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f836c112c20 con 0x7f836c06bab0 2024-04-26T01:52:37.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.407+0000 7f8374600700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f836c112aa0 con 0x7f836c108a50 2024-04-26T01:52:37.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.407+0000 7f8374600700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f836c112da0 con 0x7f836c06b110 2024-04-26T01:52:37.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.408+0000 7f8371200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f836c06bab0 0x7f836c1996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:37.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.408+0000 7f8371c00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f836c06b110 0x7f836c1991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:37.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.408+0000 7f8371c00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f836c06b110 0x7f836c1991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:37698/0 (socket says 172.21.15.151:37698) 2024-04-26T01:52:37.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.408+0000 7f8372600700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f836c108a50 0x7f836c19da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:37.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.408+0000 7f8371c00700 1 -- 172.21.15.151:0/1142781720 learned_addr learned my addr 172.21.15.151:0/1142781720 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:52:37.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.408+0000 7f8371200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f836c06bab0 0x7f836c1996f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42634/0 (socket says 172.21.15.151:42634) 2024-04-26T01:52:37.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.408+0000 7f8371c00700 1 -- 172.21.15.151:0/1142781720 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f836c108a50 msgr2=0x7f836c19da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:37.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.408+0000 7f8371c00700 1 --2- 172.21.15.151:0/1142781720 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f836c108a50 0x7f836c19da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:37.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.408+0000 7f8371c00700 1 -- 172.21.15.151:0/1142781720 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f836c06bab0 msgr2=0x7f836c1996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:37.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.408+0000 7f8371c00700 1 --2- 172.21.15.151:0/1142781720 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f836c06bab0 0x7f836c1996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:37.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.408+0000 7f8371c00700 1 -- 172.21.15.151:0/1142781720 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f836c19e1c0 con 0x7f836c06b110 2024-04-26T01:52:37.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.408+0000 7f8372600700 1 --2- 172.21.15.151:0/1142781720 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f836c108a50 0x7f836c19da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:52:37.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.408+0000 7f8371200700 1 --2- 172.21.15.151:0/1142781720 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f836c06bab0 0x7f836c1996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:52:37.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.409+0000 7f8371c00700 1 --2- 172.21.15.151:0/1142781720 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f836c06b110 0x7f836c1991b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f836000ef10 tx=0x7f836000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:37.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.409+0000 7f836a000700 1 -- 172.21.15.151:0/1142781720 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836000c9c0 con 0x7f836c06b110 2024-04-26T01:52:37.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.409+0000 7f836a000700 1 -- 172.21.15.151:0/1142781720 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8360019070 con 0x7f836c06b110 2024-04-26T01:52:37.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.409+0000 7f8374600700 1 -- 172.21.15.151:0/1142781720 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f836c19e450 con 0x7f836c06b110 2024-04-26T01:52:37.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.409+0000 7f836a000700 1 -- 172.21.15.151:0/1142781720 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8360014480 con 0x7f836c06b110 2024-04-26T01:52:37.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.409+0000 7f8374600700 1 -- 172.21.15.151:0/1142781720 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f836c1a5d30 con 0x7f836c06b110 2024-04-26T01:52:37.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.412+0000 7f836a000700 1 -- 172.21.15.151:0/1142781720 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f8360012050 con 0x7f836c06b110 2024-04-26T01:52:37.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.413+0000 7f836a000700 1 --2- 172.21.15.151:0/1142781720 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f83540a3cc0 0x7f83540a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:37.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.414+0000 7f837efb7b80 1 Processor -- start 2024-04-26T01:52:37.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.414+0000 7f836a000700 1 -- 172.21.15.151:0/1142781720 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f83600d0200 con 0x7f836c06b110 2024-04-26T01:52:37.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.414+0000 7f8371200700 1 --2- 172.21.15.151:0/1142781720 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f83540a3cc0 0x7f83540a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:37.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.414+0000 7f837efb7b80 1 -- start start 2024-04-26T01:52:37.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.414+0000 7f837efb7b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec28d002f0 0x55ec28d006d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:37.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.414+0000 7f837efb7b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec28d00c10 0x55ec28d0b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:37.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.414+0000 7f837efb7b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ec28d0bab0 0x55ec28d0dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:37.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.414+0000 7f837efb7b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ec28c53e40 con 0x55ec28d002f0 2024-04-26T01:52:37.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.414+0000 7f837efb7b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ec28bfa6e0 con 0x55ec28d0bab0 2024-04-26T01:52:37.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.414+0000 7f837efb7b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ec28c4fe00 con 0x55ec28d00c10 2024-04-26T01:52:37.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.415+0000 7f8371200700 1 --2- 172.21.15.151:0/1142781720 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f83540a3cc0 0x7f83540a6180 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f836c079030 tx=0x7f835801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:37.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.415+0000 7f8371c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec28d002f0 0x55ec28d006d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:37.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.415+0000 7f8371200700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec28d00c10 0x55ec28d0b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:37.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.415+0000 7f8372600700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ec28d0bab0 0x55ec28d0dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:37.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.415+0000 7f8371200700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec28d00c10 0x55ec28d0b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:37712/0 (socket says 172.21.15.151:37712) 2024-04-26T01:52:37.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.415+0000 7f8371200700 1 -- 172.21.15.151:0/1191536889 learned_addr learned my addr 172.21.15.151:0/1191536889 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:52:37.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.415+0000 7f8371c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec28d002f0 0x55ec28d006d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42636/0 (socket says 172.21.15.151:42636) 2024-04-26T01:52:37.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.415+0000 7f8371200700 1 -- 172.21.15.151:0/1191536889 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ec28d0bab0 msgr2=0x55ec28d0dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:37.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.415+0000 7f8371200700 1 --2- 172.21.15.151:0/1191536889 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ec28d0bab0 0x55ec28d0dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:37.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.415+0000 7f8371200700 1 -- 172.21.15.151:0/1191536889 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec28d002f0 msgr2=0x55ec28d006d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:37.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.415+0000 7f8371200700 1 --2- 172.21.15.151:0/1191536889 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec28d002f0 0x55ec28d006d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:37.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.415+0000 7f8371200700 1 -- 172.21.15.151:0/1191536889 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec28bf0630 con 0x55ec28d00c10 2024-04-26T01:52:37.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.415+0000 7f8371c00700 1 --2- 172.21.15.151:0/1191536889 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec28d002f0 0x55ec28d006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:52:37.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.416+0000 7f8371200700 1 --2- 172.21.15.151:0/1191536889 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec28d00c10 0x55ec28d0b570 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8358015000 tx=0x7f835801f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fe5f36b440abeb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:37.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.416+0000 7f835ea00700 1 -- 172.21.15.151:0/1191536889 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8358005560 con 0x55ec28d00c10 2024-04-26T01:52:37.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.416+0000 7f835ea00700 1 -- 172.21.15.151:0/1191536889 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8358003b30 con 0x55ec28d00c10 2024-04-26T01:52:37.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.416+0000 7f835ea00700 1 -- 172.21.15.151:0/1191536889 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8358030ab0 con 0x55ec28d00c10 2024-04-26T01:52:37.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.417+0000 7f837efb7b80 1 -- 172.21.15.151:0/1191536889 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec28d00c10 msgr2=0x55ec28d0b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:37.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.417+0000 7f837efb7b80 1 --2- 172.21.15.151:0/1191536889 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec28d00c10 0x55ec28d0b570 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8358015000 tx=0x7f835801f000 comp rx=0 tx=0).stop 2024-04-26T01:52:37.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.417+0000 7f837efb7b80 1 -- 172.21.15.151:0/1191536889 shutdown_connections 2024-04-26T01:52:37.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.417+0000 7f837efb7b80 1 --2- 172.21.15.151:0/1191536889 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ec28d0bab0 0x55ec28d0dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:37.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.417+0000 7f837efb7b80 1 --2- 172.21.15.151:0/1191536889 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec28d002f0 0x55ec28d006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:37.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.417+0000 7f837efb7b80 1 --2- 172.21.15.151:0/1191536889 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec28d00c10 0x55ec28d0b570 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:37.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.417+0000 7f837efb7b80 1 -- 172.21.15.151:0/1191536889 >> 172.21.15.151:0/1191536889 conn(0x55ec28c2f550 msgr2=0x55ec28cfc790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:37.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.417+0000 7f837efb7b80 1 -- 172.21.15.151:0/1191536889 shutdown_connections 2024-04-26T01:52:37.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.417+0000 7f837efb7b80 1 -- 172.21.15.151:0/1191536889 wait complete. 2024-04-26T01:52:37.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.418+0000 7f837efb7b80 1 -- start start 2024-04-26T01:52:37.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.418+0000 7f837efb7b80 1 Processor -- start 2024-04-26T01:52:37.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.418+0000 7f837efb7b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec28d002f0 0x55ec28d0b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:37.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.418+0000 7f837efb7b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec28d00c10 0x55ec28d08520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:37.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.418+0000 7f837efb7b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ec28d0bab0 0x55ec28d08a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:37.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.418+0000 7f837efb7b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ec28d0e540 con 0x55ec28d002f0 2024-04-26T01:52:37.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.418+0000 7f837efb7b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ec28c0d810 con 0x55ec28d0bab0 2024-04-26T01:52:37.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.418+0000 7f837efb7b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ec28d0e6c0 con 0x55ec28d00c10 2024-04-26T01:52:37.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.418+0000 7f837efb7b80 10 client.0 _ll_register_callbacks cb 0x55ec28bec6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:52:37.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.419+0000 7f8371c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec28d002f0 0x55ec28d0b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:37.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.419+0000 7f8371200700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec28d00c10 0x55ec28d08520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:37.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.419+0000 7f8371c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec28d002f0 0x55ec28d0b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42650/0 (socket says 172.21.15.151:42650) 2024-04-26T01:52:37.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.419+0000 7f8371c00700 1 -- 172.21.15.151:0/3454128675 learned_addr learned my addr 172.21.15.151:0/3454128675 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:52:37.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.419+0000 7f8372600700 1 --2- 172.21.15.151:0/3454128675 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ec28d0bab0 0x55ec28d08a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:37.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.419+0000 7f8371c00700 1 -- 172.21.15.151:0/3454128675 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec28d00c10 msgr2=0x55ec28d08520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:37.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.419+0000 7f8371c00700 1 --2- 172.21.15.151:0/3454128675 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec28d00c10 0x55ec28d08520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:37.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.419+0000 7f8371c00700 1 -- 172.21.15.151:0/3454128675 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ec28d0bab0 msgr2=0x55ec28d08a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:37.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.419+0000 7f8371c00700 1 --2- 172.21.15.151:0/3454128675 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ec28d0bab0 0x55ec28d08a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:37.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.419+0000 7f8371c00700 1 -- 172.21.15.151:0/3454128675 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec28ba1880 con 0x55ec28d002f0 2024-04-26T01:52:37.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.419+0000 7f8371c00700 1 --2- 172.21.15.151:0/3454128675 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec28d002f0 0x55ec28d0b220 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8360007fd0 tx=0x7f836009c110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8c3bea63329fcf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:37.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.419+0000 7f8353400700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-04-26T01:52:37.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.420+0000 7f8353400700 1 -- 172.21.15.151:0/3454128675 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83600b5060 con 0x55ec28d002f0 2024-04-26T01:52:37.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.420+0000 7f8353400700 1 -- 172.21.15.151:0/3454128675 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f83600a4cc0 con 0x55ec28d002f0 2024-04-26T01:52:37.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.420+0000 7f837efb7b80 1 -- 172.21.15.151:0/3454128675 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8358003b30 con 0x55ec28d002f0 2024-04-26T01:52:37.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.420+0000 7f8353400700 1 -- 172.21.15.151:0/3454128675 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f836009cbe0 con 0x55ec28d002f0 2024-04-26T01:52:37.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.420+0000 7f837efb7b80 10 client.14790 fetch_fsmap learned FSMap version 25 2024-04-26T01:52:37.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.420+0000 7f837efb7b80 1 -- 172.21.15.151:0/3454128675 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ec28beaaa0 con 0x55ec28d002f0 2024-04-26T01:52:37.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.420+0000 7f8353400700 1 -- 172.21.15.151:0/3454128675 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8358003b30 con 0x55ec28d002f0 2024-04-26T01:52:37.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.420+0000 7f8353400700 1 -- 172.21.15.151:0/3454128675 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f836009b930 con 0x55ec28d002f0 2024-04-26T01:52:37.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.420+0000 7f837efb7b80 10 client.14790 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T01:52:37.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.421+0000 7f837efb7b80 10 client.14790 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T01:52:37.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.421+0000 7f837efb7b80 20 client.14790 populate_metadata read hostname 'smithi151' 2024-04-26T01:52:37.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.421+0000 7f837efb7b80 1 --2- 172.21.15.151:0/3454128675 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55ec28d1eea0 0x55ec28d20990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:37.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.421+0000 7f837efb7b80 4 client.14790 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:52:37.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.421+0000 7f837efb7b80 1 -- 172.21.15.151:0/3454128675 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec28c61630 con 0x55ec28d1eea0 2024-04-26T01:52:37.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.421+0000 7f8371200700 1 --2- 172.21.15.151:0/3454128675 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55ec28d1eea0 0x55ec28d20990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:37.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.421+0000 7f8371200700 1 --2- 172.21.15.151:0/3454128675 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55ec28d1eea0 0x55ec28d20990 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=750eb6b0ddbf227b server_cookie=aeae1b5b165639ac in_seq=0 out_seq=0 2024-04-26T01:52:37.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.421+0000 7f8353400700 10 client.14790 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:52:37.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.422+0000 7f8353400700 1 -- 172.21.15.151:0/3454128675 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ec28c61630 con 0x55ec28d1eea0 2024-04-26T01:52:37.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.422+0000 7f8353400700 10 client.14790 handle_command_reply: tid=0 2024-04-26T01:52:37.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.422+0000 7f837efb7b80 1 client.14790 shutdown 2024-04-26T01:52:37.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.423+0000 7f837efb7b80 1 -- 172.21.15.151:0/3454128675 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec28d002f0 msgr2=0x55ec28d0b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:37.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.423+0000 7f837efb7b80 1 --2- 172.21.15.151:0/3454128675 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec28d002f0 0x55ec28d0b220 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8360007fd0 tx=0x7f836009c110 comp rx=0 tx=0).stop 2024-04-26T01:52:37.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.423+0000 7f837efb7b80 1 -- 172.21.15.151:0/3454128675 shutdown_connections 2024-04-26T01:52:37.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.423+0000 7f837efb7b80 1 --2- 172.21.15.151:0/3454128675 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55ec28d1eea0 0x55ec28d20990 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:37.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.423+0000 7f837efb7b80 1 --2- 172.21.15.151:0/3454128675 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ec28d0bab0 0x55ec28d08a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:37.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.423+0000 7f837efb7b80 1 --2- 172.21.15.151:0/3454128675 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec28d002f0 0x55ec28d0b220 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:37.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.423+0000 7f837efb7b80 1 --2- 172.21.15.151:0/3454128675 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec28d00c10 0x55ec28d08520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:37.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.423+0000 7f837efb7b80 1 -- 172.21.15.151:0/3454128675 >> 172.21.15.151:0/3454128675 conn(0x55ec28c2f550 msgr2=0x55ec28d02c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:37.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.423+0000 7f8353400700 0 client.14790 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:52:37.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.423+0000 7f837efb7b80 1 -- 172.21.15.151:0/3454128675 shutdown_connections 2024-04-26T01:52:37.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.423+0000 7f837efb7b80 1 -- 172.21.15.151:0/3454128675 wait complete. 2024-04-26T01:52:37.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.423+0000 7f837efb7b80 20 client.14790 trim_cache size 0 max 16384 2024-04-26T01:52:37.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.445+0000 7f837efb7b80 1 Processor -- start 2024-04-26T01:52:37.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.445+0000 7f837efb7b80 1 -- start start 2024-04-26T01:52:37.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.446+0000 7f837efb7b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec28d002f0 0x55ec28d0df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:37.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.446+0000 7f837efb7b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ec28d00c10 0x55ec28d08300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:37.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.446+0000 7f837efb7b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec28d0bab0 0x55ec28d08840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:37.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.446+0000 7f837efb7b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ec28d1ed20 con 0x55ec28d0bab0 2024-04-26T01:52:37.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.446+0000 7f837efb7b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ec28d1f3a0 con 0x55ec28d00c10 2024-04-26T01:52:37.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.446+0000 7f837efb7b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ec28d1fa70 con 0x55ec28d002f0 2024-04-26T01:52:37.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.446+0000 7f8372600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec28d0bab0 0x55ec28d08840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:37.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.446+0000 7f8371c00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec28d002f0 0x55ec28d0df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:37.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.446+0000 7f8371c00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec28d002f0 0x55ec28d0df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:37730/0 (socket says 172.21.15.151:37730) 2024-04-26T01:52:37.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.446+0000 7f8371c00700 1 -- 172.21.15.151:0/2763367055 learned_addr learned my addr 172.21.15.151:0/2763367055 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:52:37.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.446+0000 7f8371200700 1 --2- 172.21.15.151:0/2763367055 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ec28d00c10 0x55ec28d08300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:37.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.446+0000 7f8371c00700 1 -- 172.21.15.151:0/2763367055 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ec28d00c10 msgr2=0x55ec28d08300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:37.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.446+0000 7f8371c00700 1 --2- 172.21.15.151:0/2763367055 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ec28d00c10 0x55ec28d08300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:37.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.446+0000 7f8371c00700 1 -- 172.21.15.151:0/2763367055 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec28d0bab0 msgr2=0x55ec28d08840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:37.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.446+0000 7f8371c00700 1 --2- 172.21.15.151:0/2763367055 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec28d0bab0 0x55ec28d08840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:37.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.447+0000 7f8371c00700 1 -- 172.21.15.151:0/2763367055 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec28d1f890 con 0x55ec28d002f0 2024-04-26T01:52:37.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.447+0000 7f8371200700 1 --2- 172.21.15.151:0/2763367055 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ec28d00c10 0x55ec28d08300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:52:37.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.447+0000 7f8371c00700 1 --2- 172.21.15.151:0/2763367055 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec28d002f0 0x55ec28d0df30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f836009c1e0 tx=0x7f8360009eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16d646b7c020f244 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:37.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.447+0000 7f835e000700 1 -- 172.21.15.151:0/2763367055 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83600b5070 con 0x55ec28d002f0 2024-04-26T01:52:37.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.447+0000 7f835e000700 1 -- 172.21.15.151:0/2763367055 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f836009c4e0 con 0x55ec28d002f0 2024-04-26T01:52:37.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.447+0000 7f835e000700 1 -- 172.21.15.151:0/2763367055 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836009b3a0 con 0x55ec28d002f0 2024-04-26T01:52:37.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.448+0000 7f837efb7b80 1 -- 172.21.15.151:0/2763367055 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec28d002f0 msgr2=0x55ec28d0df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:37.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.448+0000 7f837efb7b80 1 --2- 172.21.15.151:0/2763367055 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec28d002f0 0x55ec28d0df30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f836009c1e0 tx=0x7f8360009eb0 comp rx=0 tx=0).stop 2024-04-26T01:52:37.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.448+0000 7f837efb7b80 1 -- 172.21.15.151:0/2763367055 shutdown_connections 2024-04-26T01:52:37.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.448+0000 7f837efb7b80 1 --2- 172.21.15.151:0/2763367055 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec28d0bab0 0x55ec28d08840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:37.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.448+0000 7f837efb7b80 1 --2- 172.21.15.151:0/2763367055 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec28d002f0 0x55ec28d0df30 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:37.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.448+0000 7f837efb7b80 1 --2- 172.21.15.151:0/2763367055 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ec28d00c10 0x55ec28d08300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:37.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.448+0000 7f837efb7b80 1 -- 172.21.15.151:0/2763367055 >> 172.21.15.151:0/2763367055 conn(0x55ec28c2f550 msgr2=0x55ec28d1e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:37.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.448+0000 7f837efb7b80 1 -- 172.21.15.151:0/2763367055 shutdown_connections 2024-04-26T01:52:37.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.448+0000 7f837efb7b80 1 -- 172.21.15.151:0/2763367055 wait complete. 2024-04-26T01:52:37.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.449+0000 7f837efb7b80 1 -- start start 2024-04-26T01:52:37.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.449+0000 7f837efb7b80 1 Processor -- start 2024-04-26T01:52:37.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.449+0000 7f837efb7b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec28d002f0 0x55ec28d25bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:37.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.449+0000 7f837efb7b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec28d00c10 0x55ec28d26130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:37.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.449+0000 7f837efb7b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ec28d0bab0 0x55ec28d85650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:37.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.449+0000 7f837efb7b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ec28d20810 con 0x55ec28d002f0 2024-04-26T01:52:37.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.449+0000 7f837efb7b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ec28d20140 con 0x55ec28d0bab0 2024-04-26T01:52:37.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.450+0000 7f837efb7b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ec28d20ee0 con 0x55ec28d00c10 2024-04-26T01:52:37.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.450+0000 7f837efb7b80 10 client.0 _ll_register_callbacks cb 0x55ec28bec6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:52:37.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.450+0000 7f8371c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec28d002f0 0x55ec28d25bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:37.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.450+0000 7f8371c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec28d002f0 0x55ec28d25bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42672/0 (socket says 172.21.15.151:42672) 2024-04-26T01:52:37.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.450+0000 7f8371c00700 1 -- 172.21.15.151:0/3537287505 learned_addr learned my addr 172.21.15.151:0/3537287505 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:52:37.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.450+0000 7f8371200700 1 --2- 172.21.15.151:0/3537287505 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec28d00c10 0x55ec28d26130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:37.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.450+0000 7f8372600700 1 --2- 172.21.15.151:0/3537287505 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ec28d0bab0 0x55ec28d85650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:37.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.450+0000 7f8371c00700 1 -- 172.21.15.151:0/3537287505 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec28d00c10 msgr2=0x55ec28d26130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:37.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.450+0000 7f8371c00700 1 --2- 172.21.15.151:0/3537287505 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec28d00c10 0x55ec28d26130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:37.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.450+0000 7f8371c00700 1 -- 172.21.15.151:0/3537287505 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ec28d0bab0 msgr2=0x55ec28d85650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:37.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.450+0000 7f8371c00700 1 --2- 172.21.15.151:0/3537287505 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ec28d0bab0 0x55ec28d85650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:37.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.450+0000 7f8371c00700 1 -- 172.21.15.151:0/3537287505 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec28d85b90 con 0x55ec28d002f0 2024-04-26T01:52:37.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.450+0000 7f8372600700 1 --2- 172.21.15.151:0/3537287505 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ec28d0bab0 0x55ec28d85650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T01:52:37.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.451+0000 7f8371c00700 1 --2- 172.21.15.151:0/3537287505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec28d002f0 0x55ec28d25bf0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f836009a790 tx=0x7f83600a5070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ea1fc327ee2883c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:37.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.451+0000 7f8353400700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-04-26T01:52:37.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.451+0000 7f8353400700 1 -- 172.21.15.151:0/3537287505 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83600b5050 con 0x55ec28d002f0 2024-04-26T01:52:37.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.451+0000 7f8353400700 1 -- 172.21.15.151:0/3537287505 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f83600bb370 con 0x55ec28d002f0 2024-04-26T01:52:37.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.451+0000 7f837efb7b80 1 -- 172.21.15.151:0/3537287505 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f836009c4e0 con 0x55ec28d002f0 2024-04-26T01:52:37.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.451+0000 7f8353400700 1 -- 172.21.15.151:0/3537287505 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f83600a0890 con 0x55ec28d002f0 2024-04-26T01:52:37.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.451+0000 7f837efb7b80 10 client.14796 fetch_fsmap learned FSMap version 25 2024-04-26T01:52:37.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.451+0000 7f837efb7b80 1 -- 172.21.15.151:0/3537287505 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ec28d86d20 con 0x55ec28d002f0 2024-04-26T01:52:37.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.451+0000 7f8353400700 1 -- 172.21.15.151:0/3537287505 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f836009c4e0 con 0x55ec28d002f0 2024-04-26T01:52:37.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.452+0000 7f8353400700 1 -- 172.21.15.151:0/3537287505 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f8360085a70 con 0x55ec28d002f0 2024-04-26T01:52:37.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.452+0000 7f837efb7b80 10 client.14796 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T01:52:37.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.452+0000 7f837efb7b80 10 client.14796 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T01:52:37.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.452+0000 7f837efb7b80 20 client.14796 populate_metadata read hostname 'smithi151' 2024-04-26T01:52:37.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.452+0000 7f837efb7b80 1 --2- 172.21.15.151:0/3537287505 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55ec28d8b300 0x55ec28d8a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:37.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.452+0000 7f837efb7b80 4 client.14796 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T01:52:37.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.452+0000 7f837efb7b80 1 -- 172.21.15.151:0/3537287505 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "status"}) -- 0x55ec28c5b6d0 con 0x55ec28d8b300 2024-04-26T01:52:37.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.452+0000 7f8371200700 1 --2- 172.21.15.151:0/3537287505 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55ec28d8b300 0x55ec28d8a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:37.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.453+0000 7f8371200700 1 --2- 172.21.15.151:0/3537287505 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55ec28d8b300 0x55ec28d8a010 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79085a9542c6741c server_cookie=df44dc00c6be24 in_seq=0 out_seq=0 2024-04-26T01:52:37.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.453+0000 7f8353400700 10 client.14796 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:52:37.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.453+0000 7f8353400700 1 -- 172.21.15.151:0/3537287505 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55ec28c5b6d0 con 0x55ec28d8b300 2024-04-26T01:52:37.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.453+0000 7f8353400700 10 client.14796 handle_command_reply: tid=0 2024-04-26T01:52:37.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.453+0000 7f837efb7b80 1 client.14796 shutdown 2024-04-26T01:52:37.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.453+0000 7f837efb7b80 1 -- 172.21.15.151:0/3537287505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec28d002f0 msgr2=0x55ec28d25bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:37.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.453+0000 7f837efb7b80 1 --2- 172.21.15.151:0/3537287505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec28d002f0 0x55ec28d25bf0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f836009a790 tx=0x7f83600a5070 comp rx=0 tx=0).stop 2024-04-26T01:52:37.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.453+0000 7f837efb7b80 1 -- 172.21.15.151:0/3537287505 shutdown_connections 2024-04-26T01:52:37.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.453+0000 7f837efb7b80 1 --2- 172.21.15.151:0/3537287505 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55ec28d8b300 0x55ec28d8a010 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:37.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.453+0000 7f837efb7b80 1 --2- 172.21.15.151:0/3537287505 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ec28d0bab0 0x55ec28d85650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:37.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.453+0000 7f8353400700 0 client.14796 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:52:37.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.453+0000 7f837efb7b80 1 --2- 172.21.15.151:0/3537287505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ec28d002f0 0x55ec28d25bf0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:37.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.453+0000 7f837efb7b80 1 --2- 172.21.15.151:0/3537287505 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec28d00c10 0x55ec28d26130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:37.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.453+0000 7f837efb7b80 1 -- 172.21.15.151:0/3537287505 >> 172.21.15.151:0/3537287505 conn(0x55ec28c2f550 msgr2=0x55ec28d08300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:37.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.453+0000 7f837efb7b80 1 -- 172.21.15.151:0/3537287505 shutdown_connections 2024-04-26T01:52:37.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.454+0000 7f837efb7b80 1 -- 172.21.15.151:0/3537287505 wait complete. 2024-04-26T01:52:37.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.454+0000 7f837efb7b80 20 client.14796 trim_cache size 0 max 16384 2024-04-26T01:52:37.455 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-26T01:52:37.455 INFO:teuthology.orchestra.run.smithi151.stdout: "cluster_fsid": "c0e49e64-036d-11ef-bc93-c7b262605968", 2024-04-26T01:52:37.455 INFO:teuthology.orchestra.run.smithi151.stdout: "whoami": 0, 2024-04-26T01:52:37.455 INFO:teuthology.orchestra.run.smithi151.stdout: "id": 24509, 2024-04-26T01:52:37.455 INFO:teuthology.orchestra.run.smithi151.stdout: "want_state": "up:active", 2024-04-26T01:52:37.456 INFO:teuthology.orchestra.run.smithi151.stdout: "state": "up:active", 2024-04-26T01:52:37.456 INFO:teuthology.orchestra.run.smithi151.stdout: "fs_name": "cephfs", 2024-04-26T01:52:37.456 INFO:teuthology.orchestra.run.smithi151.stdout: "rank_uptime": 175.77797016599999, 2024-04-26T01:52:37.456 INFO:teuthology.orchestra.run.smithi151.stdout: "mdsmap_epoch": 25, 2024-04-26T01:52:37.456 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch": 92, 2024-04-26T01:52:37.456 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch_barrier": 90, 2024-04-26T01:52:37.456 INFO:teuthology.orchestra.run.smithi151.stdout: "uptime": 199.33753484299999 2024-04-26T01:52:37.456 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-26T01:52:37.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.454+0000 7f8374600700 1 -- 172.21.15.151:0/1142781720 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f83540a3cc0 msgr2=0x7f83540a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:37.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.454+0000 7f8374600700 1 --2- 172.21.15.151:0/1142781720 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f83540a3cc0 0x7f83540a6180 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f836c079030 tx=0x7f835801a040 comp rx=0 tx=0).stop 2024-04-26T01:52:37.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.454+0000 7f8374600700 1 -- 172.21.15.151:0/1142781720 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f836c06b110 msgr2=0x7f836c1991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:37.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.454+0000 7f8374600700 1 --2- 172.21.15.151:0/1142781720 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f836c06b110 0x7f836c1991b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f836000ef10 tx=0x7f836000c3a0 comp rx=0 tx=0).stop 2024-04-26T01:52:37.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.455+0000 7f8374600700 1 -- 172.21.15.151:0/1142781720 shutdown_connections 2024-04-26T01:52:37.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.455+0000 7f8374600700 1 --2- 172.21.15.151:0/1142781720 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f836c108a50 0x7f836c19da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:37.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.455+0000 7f8374600700 1 --2- 172.21.15.151:0/1142781720 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f836c06b110 0x7f836c1991b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:37.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.455+0000 7f8374600700 1 --2- 172.21.15.151:0/1142781720 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f83540a3cc0 0x7f83540a6180 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:37.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.455+0000 7f8374600700 1 --2- 172.21.15.151:0/1142781720 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f836c06bab0 0x7f836c1996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:37.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.455+0000 7f8374600700 1 -- 172.21.15.151:0/1142781720 >> 172.21.15.151:0/1142781720 conn(0x7f836c0788f0 msgr2=0x7f836c0fff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:37.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.455+0000 7f8374600700 1 -- 172.21.15.151:0/1142781720 shutdown_connections 2024-04-26T01:52:37.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:37.455+0000 7f8374600700 1 -- 172.21.15.151:0/1142781720 wait complete. 2024-04-26T01:52:37.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:52:37 smithi172 ceph-mon[32785]: pgmap v417: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:52:38.327 INFO:tasks.cephadm:Setup complete, yielding 2024-04-26T01:52:38.328 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-26T01:52:38.337 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi151.front.sepia.ceph.com 2024-04-26T01:52:38.337 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-26T01:52:38.771 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:52:39.298 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:52:39 smithi151 ceph-mon[28057]: pgmap v418: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T01:52:39.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:52:39 smithi165 ceph-mon[32648]: pgmap v418: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T01:52:39.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:52:39 smithi172 ceph-mon[32785]: pgmap v418: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T01:52:41.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:41.038+0000 7f2762600700 1 -- 172.21.15.151:0/431057952 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f275c104ce0 msgr2=0x7f275c105100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:41.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:41.038+0000 7f2762600700 1 --2- 172.21.15.151:0/431057952 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f275c104ce0 0x7f275c105100 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f274c0099c0 tx=0x7f274c009cd0 comp rx=0 tx=0).stop 2024-04-26T01:52:41.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:41.038+0000 7f2762600700 1 -- 172.21.15.151:0/431057952 shutdown_connections 2024-04-26T01:52:41.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:41.038+0000 7f2762600700 1 --2- 172.21.15.151:0/431057952 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f275c106880 0x7f275c10d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:41.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:41.038+0000 7f2762600700 1 --2- 172.21.15.151:0/431057952 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f275c104ce0 0x7f275c105100 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:41.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:41.038+0000 7f2762600700 1 --2- 172.21.15.151:0/431057952 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f275c105ee0 0x7f275c106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:41.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:41.038+0000 7f2762600700 1 -- 172.21.15.151:0/431057952 >> 172.21.15.151:0/431057952 conn(0x7f275c1002a0 msgr2=0x7f275c1026c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:41.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:41.038+0000 7f2762600700 1 -- 172.21.15.151:0/431057952 shutdown_connections 2024-04-26T01:52:41.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:41.038+0000 7f2762600700 1 -- 172.21.15.151:0/431057952 wait complete. 2024-04-26T01:52:41.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:41.039+0000 7f2762600700 1 Processor -- start 2024-04-26T01:52:41.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:41.039+0000 7f2762600700 1 -- start start 2024-04-26T01:52:41.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:41.040+0000 7f2762600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f275c104ce0 0x7f275c19ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:41.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:41.040+0000 7f2762600700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f275c105ee0 0x7f275c19b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:41.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:41.040+0000 7f2762600700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f275c106880 0x7f275c1a23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:41.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:41.040+0000 7f2762600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f275c1107c0 con 0x7f275c104ce0 2024-04-26T01:52:41.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:41.040+0000 7f2762600700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f275c110640 con 0x7f275c106880 2024-04-26T01:52:41.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:41.040+0000 7f2762600700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f275c110940 con 0x7f275c105ee0 2024-04-26T01:52:41.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:41.040+0000 7f275b400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f275c104ce0 0x7f275c19ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:41.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:41.040+0000 7f275aa00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f275c105ee0 0x7f275c19b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:41.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:41.040+0000 7f275be00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f275c106880 0x7f275c1a23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:41.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:41.040+0000 7f275b400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f275c104ce0 0x7f275c19ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41584/0 (socket says 172.21.15.151:41584) 2024-04-26T01:52:41.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:41.040+0000 7f275b400700 1 -- 172.21.15.151:0/273195700 learned_addr learned my addr 172.21.15.151:0/273195700 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:52:41.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:41.041+0000 7f275be00700 1 -- 172.21.15.151:0/273195700 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f275c105ee0 msgr2=0x7f275c19b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:41.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:41.041+0000 7f275be00700 1 --2- 172.21.15.151:0/273195700 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f275c105ee0 0x7f275c19b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:41.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:41.041+0000 7f275be00700 1 -- 172.21.15.151:0/273195700 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f275c104ce0 msgr2=0x7f275c19ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:41.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:41.041+0000 7f275be00700 1 --2- 172.21.15.151:0/273195700 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f275c104ce0 0x7f275c19ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:41.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:41.041+0000 7f275be00700 1 -- 172.21.15.151:0/273195700 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f275c1a2a60 con 0x7f275c106880 2024-04-26T01:52:41.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:41.041+0000 7f275be00700 1 --2- 172.21.15.151:0/273195700 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f275c106880 0x7f275c1a23c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f275400efc0 tx=0x7f275400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:41.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:41.041+0000 7f2753e00700 1 -- 172.21.15.151:0/273195700 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27540092e0 con 0x7f275c106880 2024-04-26T01:52:41.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:41.041+0000 7f2762600700 1 -- 172.21.15.151:0/273195700 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f275c1a2cf0 con 0x7f275c106880 2024-04-26T01:52:41.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:41.041+0000 7f2753e00700 1 -- 172.21.15.151:0/273195700 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2754004bb0 con 0x7f275c106880 2024-04-26T01:52:41.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:41.041+0000 7f2753e00700 1 -- 172.21.15.151:0/273195700 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2754004020 con 0x7f275c106880 2024-04-26T01:52:41.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:41.041+0000 7f2762600700 1 -- 172.21.15.151:0/273195700 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f275c1a3220 con 0x7f275c106880 2024-04-26T01:52:41.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:41.042+0000 7f2762600700 1 -- 172.21.15.151:0/273195700 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f275c069730 con 0x7f275c106880 2024-04-26T01:52:41.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:41.044+0000 7f2753e00700 1 -- 172.21.15.151:0/273195700 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f275401b030 con 0x7f275c106880 2024-04-26T01:52:41.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:41.046+0000 7f2753e00700 1 --2- 172.21.15.151:0/273195700 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f27400a3c40 0x7f27400a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:41.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:41.046+0000 7f2753e00700 1 -- 172.21.15.151:0/273195700 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f27540cfdb0 con 0x7f275c106880 2024-04-26T01:52:41.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:41.046+0000 7f275b400700 1 --2- 172.21.15.151:0/273195700 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f27400a3c40 0x7f27400a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:41.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:41.047+0000 7f275b400700 1 --2- 172.21.15.151:0/273195700 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f27400a3c40 0x7f27400a6100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f274c003820 tx=0x7f274c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:41.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:41.050+0000 7f2753e00700 1 -- 172.21.15.151:0/273195700 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f27540d4070 con 0x7f275c106880 2024-04-26T01:52:41.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:52:41 smithi151 ceph-mon[28057]: pgmap v419: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:52:41.294 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:41.296+0000 7f2762600700 1 -- 172.21.15.151:0/273195700 --> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f275c062e80 con 0x7f27400a3c40 2024-04-26T01:52:41.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:41.299+0000 7f2753e00700 1 -- 172.21.15.151:0/273195700 <== mgr.14152 v2:172.21.15.151:6800/2251256505 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f275c062e80 con 0x7f27400a3c40 2024-04-26T01:52:41.296 INFO:teuthology.orchestra.run.smithi151.stdout:Backend: cephadm 2024-04-26T01:52:41.297 INFO:teuthology.orchestra.run.smithi151.stdout:Available: Yes 2024-04-26T01:52:41.297 INFO:teuthology.orchestra.run.smithi151.stdout:Paused: No 2024-04-26T01:52:41.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:41.302+0000 7f2762600700 1 -- 172.21.15.151:0/273195700 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f27400a3c40 msgr2=0x7f27400a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:41.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:41.302+0000 7f2762600700 1 --2- 172.21.15.151:0/273195700 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f27400a3c40 0x7f27400a6100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f274c003820 tx=0x7f274c005fb0 comp rx=0 tx=0).stop 2024-04-26T01:52:41.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:41.302+0000 7f2762600700 1 -- 172.21.15.151:0/273195700 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f275c106880 msgr2=0x7f275c1a23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:41.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:41.302+0000 7f2762600700 1 --2- 172.21.15.151:0/273195700 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f275c106880 0x7f275c1a23c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f275400efc0 tx=0x7f275400c3a0 comp rx=0 tx=0).stop 2024-04-26T01:52:41.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:41.302+0000 7f2762600700 1 -- 172.21.15.151:0/273195700 shutdown_connections 2024-04-26T01:52:41.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:41.302+0000 7f2762600700 1 --2- 172.21.15.151:0/273195700 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f275c106880 0x7f275c1a23c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:41.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:41.302+0000 7f2762600700 1 --2- 172.21.15.151:0/273195700 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f27400a3c40 0x7f27400a6100 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:41.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:41.302+0000 7f2762600700 1 --2- 172.21.15.151:0/273195700 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f275c104ce0 0x7f275c19ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:41.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:41.302+0000 7f2762600700 1 --2- 172.21.15.151:0/273195700 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f275c105ee0 0x7f275c19b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:41.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:41.302+0000 7f2762600700 1 -- 172.21.15.151:0/273195700 >> 172.21.15.151:0/273195700 conn(0x7f275c1002a0 msgr2=0x7f275c109230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:41.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:41.302+0000 7f2762600700 1 -- 172.21.15.151:0/273195700 shutdown_connections 2024-04-26T01:52:41.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:41.302+0000 7f2762600700 1 -- 172.21.15.151:0/273195700 wait complete. 2024-04-26T01:52:41.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:52:41 smithi165 ceph-mon[32648]: pgmap v419: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:52:41.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:52:41 smithi172 ceph-mon[32785]: pgmap v419: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:52:42.140 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-26T01:52:42.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:52:42 smithi165 ceph-mon[32648]: from='client.24896 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:52:42.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:52:42 smithi151 ceph-mon[28057]: from='client.24896 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:52:42.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:52:42 smithi172 ceph-mon[32785]: from='client.24896 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:52:42.591 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:52:43.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:52:43 smithi165 ceph-mon[32648]: pgmap v420: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:52:43.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:52:43 smithi151 ceph-mon[28057]: pgmap v420: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:52:43.538 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:52:43 smithi172 ceph-mon[32785]: pgmap v420: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:52:44.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:44.759+0000 7f14ca800700 1 -- 172.21.15.151:0/1002492074 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14c4104ce0 msgr2=0x7f14c4105100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:44.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:44.759+0000 7f14ca800700 1 --2- 172.21.15.151:0/1002492074 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14c4104ce0 0x7f14c4105100 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f14b40099c0 tx=0x7f14b4009cd0 comp rx=0 tx=0).stop 2024-04-26T01:52:44.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:44.759+0000 7f14ca800700 1 -- 172.21.15.151:0/1002492074 shutdown_connections 2024-04-26T01:52:44.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:44.759+0000 7f14ca800700 1 --2- 172.21.15.151:0/1002492074 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14c4106880 0x7f14c410d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:44.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:44.759+0000 7f14ca800700 1 --2- 172.21.15.151:0/1002492074 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14c4104ce0 0x7f14c4105100 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:44.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:44.759+0000 7f14ca800700 1 --2- 172.21.15.151:0/1002492074 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14c4105ee0 0x7f14c4106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:44.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:44.759+0000 7f14ca800700 1 -- 172.21.15.151:0/1002492074 >> 172.21.15.151:0/1002492074 conn(0x7f14c41002a0 msgr2=0x7f14c41026c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:44.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:44.759+0000 7f14ca800700 1 -- 172.21.15.151:0/1002492074 shutdown_connections 2024-04-26T01:52:44.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:44.759+0000 7f14ca800700 1 -- 172.21.15.151:0/1002492074 wait complete. 2024-04-26T01:52:44.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:44.760+0000 7f14ca800700 1 Processor -- start 2024-04-26T01:52:44.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:44.761+0000 7f14ca800700 1 -- start start 2024-04-26T01:52:44.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:44.761+0000 7f14ca800700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14c4104ce0 0x7f14c419ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:44.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:44.761+0000 7f14ca800700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14c4105ee0 0x7f14c419b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:44.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:44.761+0000 7f14ca800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14c4106880 0x7f14c41a2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:44.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:44.761+0000 7f14ca800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f14c41107c0 con 0x7f14c4106880 2024-04-26T01:52:44.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:44.761+0000 7f14ca800700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f14c4110640 con 0x7f14c4104ce0 2024-04-26T01:52:44.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:44.761+0000 7f14ca800700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f14c4110940 con 0x7f14c4105ee0 2024-04-26T01:52:44.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:44.761+0000 7f14c3400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14c4104ce0 0x7f14c419ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:44.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:44.761+0000 7f14c3400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14c4104ce0 0x7f14c419ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:33132/0 (socket says 172.21.15.151:33132) 2024-04-26T01:52:44.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:44.761+0000 7f14c3400700 1 -- 172.21.15.151:0/2864387988 learned_addr learned my addr 172.21.15.151:0/2864387988 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:52:44.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:44.761+0000 7f14c3e00700 1 --2- 172.21.15.151:0/2864387988 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14c4106880 0x7f14c41a2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:44.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:44.761+0000 7f14c2a00700 1 --2- 172.21.15.151:0/2864387988 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14c4105ee0 0x7f14c419b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:44.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:44.762+0000 7f14c3400700 1 -- 172.21.15.151:0/2864387988 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14c4105ee0 msgr2=0x7f14c419b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:44.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:44.762+0000 7f14c3400700 1 --2- 172.21.15.151:0/2864387988 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14c4105ee0 0x7f14c419b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:44.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:44.762+0000 7f14c3400700 1 -- 172.21.15.151:0/2864387988 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14c4106880 msgr2=0x7f14c41a2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:44.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:44.762+0000 7f14c3400700 1 --2- 172.21.15.151:0/2864387988 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14c4106880 0x7f14c41a2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:44.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:44.762+0000 7f14c3400700 1 -- 172.21.15.151:0/2864387988 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14c41a2af0 con 0x7f14c4104ce0 2024-04-26T01:52:44.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:44.762+0000 7f14c3400700 1 --2- 172.21.15.151:0/2864387988 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14c4104ce0 0x7f14c419ae90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f14b40036f0 tx=0x7f14b4003af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:44.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:44.762+0000 7f14bbe00700 1 -- 172.21.15.151:0/2864387988 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b4009e60 con 0x7f14c4104ce0 2024-04-26T01:52:44.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:44.762+0000 7f14bbe00700 1 -- 172.21.15.151:0/2864387988 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14b4005570 con 0x7f14c4104ce0 2024-04-26T01:52:44.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:44.762+0000 7f14ca800700 1 -- 172.21.15.151:0/2864387988 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14c41a2d80 con 0x7f14c4104ce0 2024-04-26T01:52:44.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:44.762+0000 7f14bbe00700 1 -- 172.21.15.151:0/2864387988 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b400fd50 con 0x7f14c4104ce0 2024-04-26T01:52:44.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:44.763+0000 7f14ca800700 1 -- 172.21.15.151:0/2864387988 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14c41a3180 con 0x7f14c4104ce0 2024-04-26T01:52:44.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:44.764+0000 7f14ca800700 1 -- 172.21.15.151:0/2864387988 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14c402df80 con 0x7f14c4104ce0 2024-04-26T01:52:44.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:44.766+0000 7f14bbe00700 1 -- 172.21.15.151:0/2864387988 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f14b400f460 con 0x7f14c4104ce0 2024-04-26T01:52:44.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:44.767+0000 7f14bbe00700 1 --2- 172.21.15.151:0/2864387988 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f14a80a3c40 0x7f14a80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:44.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:44.767+0000 7f14bbe00700 1 -- 172.21.15.151:0/2864387988 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f14b40d1180 con 0x7f14c4104ce0 2024-04-26T01:52:44.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:44.767+0000 7f14c2a00700 1 --2- 172.21.15.151:0/2864387988 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f14a80a3c40 0x7f14a80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:44.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:44.768+0000 7f14c2a00700 1 --2- 172.21.15.151:0/2864387988 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f14a80a3c40 0x7f14a80a6100 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f14b0009c60 tx=0x7f14b0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:44.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:44.771+0000 7f14bbe00700 1 -- 172.21.15.151:0/2864387988 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f14b4013070 con 0x7f14c4104ce0 2024-04-26T01:52:45.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:45.011+0000 7f14ca800700 1 -- 172.21.15.151:0/2864387988 --> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f14c4062e80 con 0x7f14a80a3c40 2024-04-26T01:52:45.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:45.050+0000 7f14bbe00700 1 -- 172.21.15.151:0/2864387988 <== mgr.14152 v2:172.21.15.151:6800/2251256505 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f14c4062e80 con 0x7f14a80a3c40 2024-04-26T01:52:45.048 INFO:teuthology.orchestra.run.smithi151.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T01:52:45.048 INFO:teuthology.orchestra.run.smithi151.stdout:mds.a smithi151 running (3m) 43s ago 3m 13.0M - 19.0.0-3156-gd69f13d8 da4b004c09e8 b6ec99fbb17c 2024-04-26T01:52:45.048 INFO:teuthology.orchestra.run.smithi151.stdout:mds.b smithi165 running (3m) 69s ago 3m 29.9M - 19.0.0-3156-gd69f13d8 da4b004c09e8 749c302b6919 2024-04-26T01:52:45.048 INFO:teuthology.orchestra.run.smithi151.stdout:mds.c smithi172 running (3m) 68s ago 3m 19.1M - 19.0.0-3156-gd69f13d8 da4b004c09e8 4da7b90b621d 2024-04-26T01:52:45.048 INFO:teuthology.orchestra.run.smithi151.stdout:mds.d smithi151 running (3m) 43s ago 3m 13.2M - 19.0.0-3156-gd69f13d8 da4b004c09e8 8d353b91b12c 2024-04-26T01:52:45.048 INFO:teuthology.orchestra.run.smithi151.stdout:mds.e smithi165 running (3m) 69s ago 3m 18.9M - 19.0.0-3156-gd69f13d8 da4b004c09e8 ddbe4960fbb2 2024-04-26T01:52:45.048 INFO:teuthology.orchestra.run.smithi151.stdout:mds.f smithi172 running (3m) 68s ago 3m 19.1M - 19.0.0-3156-gd69f13d8 da4b004c09e8 d8269a242bd3 2024-04-26T01:52:45.049 INFO:teuthology.orchestra.run.smithi151.stdout:mds.g smithi151 running (3m) 43s ago 3m 13.1M - 19.0.0-3156-gd69f13d8 da4b004c09e8 aa6c28e3310f 2024-04-26T01:52:45.049 INFO:teuthology.orchestra.run.smithi151.stdout:mds.h smithi165 running (3m) 69s ago 3m 20.3M - 19.0.0-3156-gd69f13d8 da4b004c09e8 a37b5d313664 2024-04-26T01:52:45.049 INFO:teuthology.orchestra.run.smithi151.stdout:mds.i smithi172 running (3m) 68s ago 3m 19.8M - 19.0.0-3156-gd69f13d8 da4b004c09e8 c32829abcd2d 2024-04-26T01:52:45.049 INFO:teuthology.orchestra.run.smithi151.stdout:mds.j smithi151 running (3m) 43s ago 3m 13.2M - 19.0.0-3156-gd69f13d8 da4b004c09e8 c1527278cf05 2024-04-26T01:52:45.049 INFO:teuthology.orchestra.run.smithi151.stdout:mds.k smithi165 running (4m) 69s ago 4m 14.5M - 19.0.0-3156-gd69f13d8 da4b004c09e8 78f7d2a05414 2024-04-26T01:52:45.049 INFO:teuthology.orchestra.run.smithi151.stdout:mds.l smithi172 running (4m) 68s ago 4m 17.2M - 19.0.0-3156-gd69f13d8 da4b004c09e8 5ca0c9a91dcb 2024-04-26T01:52:45.049 INFO:teuthology.orchestra.run.smithi151.stdout:mgr.x smithi151 *:9283,8765 running (12m) 43s ago 12m 516M - 19.0.0-3156-gd69f13d8 da4b004c09e8 463671658fa6 2024-04-26T01:52:45.049 INFO:teuthology.orchestra.run.smithi151.stdout:mgr.y smithi165 *:8443,8765 running (9m) 69s ago 9m 454M - 19.0.0-3156-gd69f13d8 da4b004c09e8 1faa05cec8e2 2024-04-26T01:52:45.049 INFO:teuthology.orchestra.run.smithi151.stdout:mgr.z smithi172 *:8443,8765 running (9m) 68s ago 9m 453M - 19.0.0-3156-gd69f13d8 da4b004c09e8 0171154f9dfe 2024-04-26T01:52:45.049 INFO:teuthology.orchestra.run.smithi151.stdout:mon.a smithi151 running (12m) 43s ago 12m 49.2M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 a43466754df1 2024-04-26T01:52:45.049 INFO:teuthology.orchestra.run.smithi151.stdout:mon.b smithi165 running (10m) 69s ago 10m 44.8M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 5a1fc8d54647 2024-04-26T01:52:45.049 INFO:teuthology.orchestra.run.smithi151.stdout:mon.c smithi172 running (10m) 68s ago 10m 45.1M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 655ff9c7ac33 2024-04-26T01:52:45.050 INFO:teuthology.orchestra.run.smithi151.stdout:osd.0 smithi151 running (9m) 43s ago 9m 58.1M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 0c6bf446b2c6 2024-04-26T01:52:45.050 INFO:teuthology.orchestra.run.smithi151.stdout:osd.1 smithi165 running (9m) 69s ago 9m 59.0M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 3898f9c76b60 2024-04-26T01:52:45.050 INFO:teuthology.orchestra.run.smithi151.stdout:osd.2 smithi172 running (8m) 68s ago 8m 58.4M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 b763478b7595 2024-04-26T01:52:45.050 INFO:teuthology.orchestra.run.smithi151.stdout:osd.3 smithi151 running (8m) 43s ago 8m 55.1M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 6af14d958ee1 2024-04-26T01:52:45.050 INFO:teuthology.orchestra.run.smithi151.stdout:osd.4 smithi165 running (7m) 69s ago 7m 55.2M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 fe0ca551b799 2024-04-26T01:52:45.050 INFO:teuthology.orchestra.run.smithi151.stdout:osd.5 smithi172 running (7m) 68s ago 7m 48.0M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 0dda7ef3f4f9 2024-04-26T01:52:45.050 INFO:teuthology.orchestra.run.smithi151.stdout:osd.6 smithi151 running (6m) 43s ago 6m 47.5M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 a910a3f4d95e 2024-04-26T01:52:45.050 INFO:teuthology.orchestra.run.smithi151.stdout:osd.7 smithi165 running (6m) 69s ago 6m 48.7M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 cabe7c356de2 2024-04-26T01:52:45.050 INFO:teuthology.orchestra.run.smithi151.stdout:osd.8 smithi172 running (5m) 68s ago 5m 47.7M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 39f6d303d045 2024-04-26T01:52:45.050 INFO:teuthology.orchestra.run.smithi151.stdout:osd.9 smithi151 running (5m) 43s ago 5m 52.3M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 6a09242a11d7 2024-04-26T01:52:45.050 INFO:teuthology.orchestra.run.smithi151.stdout:osd.10 smithi165 running (4m) 69s ago 4m 47.6M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 6eee6b8b80d7 2024-04-26T01:52:45.050 INFO:teuthology.orchestra.run.smithi151.stdout:osd.11 smithi172 running (4m) 68s ago 4m 54.2M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 7a6db04af718 2024-04-26T01:52:45.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:45.053+0000 7f14ca800700 1 -- 172.21.15.151:0/2864387988 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f14a80a3c40 msgr2=0x7f14a80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:45.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:45.053+0000 7f14ca800700 1 --2- 172.21.15.151:0/2864387988 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f14a80a3c40 0x7f14a80a6100 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f14b0009c60 tx=0x7f14b0009260 comp rx=0 tx=0).stop 2024-04-26T01:52:45.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:45.053+0000 7f14ca800700 1 -- 172.21.15.151:0/2864387988 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14c4104ce0 msgr2=0x7f14c419ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:45.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:45.053+0000 7f14ca800700 1 --2- 172.21.15.151:0/2864387988 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14c4104ce0 0x7f14c419ae90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f14b40036f0 tx=0x7f14b4003af0 comp rx=0 tx=0).stop 2024-04-26T01:52:45.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:45.054+0000 7f14ca800700 1 -- 172.21.15.151:0/2864387988 shutdown_connections 2024-04-26T01:52:45.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:45.054+0000 7f14ca800700 1 --2- 172.21.15.151:0/2864387988 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f14a80a3c40 0x7f14a80a6100 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:45.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:45.054+0000 7f14ca800700 1 --2- 172.21.15.151:0/2864387988 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14c4106880 0x7f14c41a2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:45.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:45.054+0000 7f14ca800700 1 --2- 172.21.15.151:0/2864387988 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14c4104ce0 0x7f14c419ae90 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:45.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:45.054+0000 7f14ca800700 1 --2- 172.21.15.151:0/2864387988 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14c4105ee0 0x7f14c419b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:45.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:45.054+0000 7f14ca800700 1 -- 172.21.15.151:0/2864387988 >> 172.21.15.151:0/2864387988 conn(0x7f14c41002a0 msgr2=0x7f14c4109300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:45.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:45.054+0000 7f14ca800700 1 -- 172.21.15.151:0/2864387988 shutdown_connections 2024-04-26T01:52:45.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:45.054+0000 7f14ca800700 1 -- 172.21.15.151:0/2864387988 wait complete. 2024-04-26T01:52:45.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:52:45 smithi165 ceph-mon[32648]: pgmap v421: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:52:45.365 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:52:45 smithi151 ceph-mon[28057]: pgmap v421: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:52:45.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:52:45 smithi172 ceph-mon[32785]: pgmap v421: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:52:45.796 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-26T01:52:46.242 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:52:46.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:52:46 smithi165 ceph-mon[32648]: from='client.24902 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:52:46.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:52:46 smithi151 ceph-mon[28057]: from='client.24902 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:52:46.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:52:46 smithi172 ceph-mon[32785]: from='client.24902 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:52:47.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:52:47 smithi151 ceph-mon[28057]: pgmap v422: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:52:47.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:52:47 smithi172 ceph-mon[32785]: pgmap v422: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:52:47.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:52:47 smithi165 ceph-mon[32648]: pgmap v422: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:52:48.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:48.403+0000 7f386dc00700 1 -- 172.21.15.151:0/3308860118 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3868108830 msgr2=0x7f386810ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:48.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:48.403+0000 7f386dc00700 1 --2- 172.21.15.151:0/3308860118 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3868108830 0x7f386810ac20 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f386000b0a0 tx=0x7f386000b3b0 comp rx=0 tx=0).stop 2024-04-26T01:52:48.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:48.403+0000 7f386dc00700 1 -- 172.21.15.151:0/3308860118 shutdown_connections 2024-04-26T01:52:48.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:48.403+0000 7f386dc00700 1 --2- 172.21.15.151:0/3308860118 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3868108830 0x7f386810ac20 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:48.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:48.403+0000 7f386dc00700 1 --2- 172.21.15.151:0/3308860118 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f38681035d0 0x7f38681059c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:48.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:48.403+0000 7f386dc00700 1 --2- 172.21.15.151:0/3308860118 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3868105f00 0x7f38681082f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:48.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:48.403+0000 7f386dc00700 1 -- 172.21.15.151:0/3308860118 >> 172.21.15.151:0/3308860118 conn(0x7f38680fcf40 msgr2=0x7f38680ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:48.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:48.403+0000 7f386dc00700 1 -- 172.21.15.151:0/3308860118 shutdown_connections 2024-04-26T01:52:48.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:48.403+0000 7f386dc00700 1 -- 172.21.15.151:0/3308860118 wait complete. 2024-04-26T01:52:48.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:48.404+0000 7f386dc00700 1 Processor -- start 2024-04-26T01:52:48.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:48.404+0000 7f386dc00700 1 -- start start 2024-04-26T01:52:48.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:48.405+0000 7f386dc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f38681035d0 0x7f386819adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:48.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:48.405+0000 7f386dc00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3868105f00 0x7f386819b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:48.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:48.405+0000 7f386dc00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3868108830 0x7f38681a23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:48.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:48.405+0000 7f386dc00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f386810e530 con 0x7f38681035d0 2024-04-26T01:52:48.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:48.405+0000 7f386dc00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f386810e3b0 con 0x7f3868108830 2024-04-26T01:52:48.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:48.405+0000 7f386dc00700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f386810e6b0 con 0x7f3868105f00 2024-04-26T01:52:48.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:48.405+0000 7f385fe00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3868105f00 0x7f386819b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:48.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:48.405+0000 7f385fe00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3868105f00 0x7f386819b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:42862/0 (socket says 172.21.15.151:42862) 2024-04-26T01:52:48.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:48.405+0000 7f385fe00700 1 -- 172.21.15.151:0/3253100769 learned_addr learned my addr 172.21.15.151:0/3253100769 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:52:48.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:48.405+0000 7f3866a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f38681035d0 0x7f386819adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:48.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:48.405+0000 7f3867400700 1 --2- 172.21.15.151:0/3253100769 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3868108830 0x7f38681a23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:48.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:48.406+0000 7f3867400700 1 -- 172.21.15.151:0/3253100769 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3868105f00 msgr2=0x7f386819b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:48.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:48.406+0000 7f3867400700 1 --2- 172.21.15.151:0/3253100769 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3868105f00 0x7f386819b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:48.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:48.406+0000 7f3867400700 1 -- 172.21.15.151:0/3253100769 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f38681035d0 msgr2=0x7f386819adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:48.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:48.406+0000 7f3867400700 1 --2- 172.21.15.151:0/3253100769 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f38681035d0 0x7f386819adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:48.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:48.406+0000 7f3867400700 1 -- 172.21.15.151:0/3253100769 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38681a2a50 con 0x7f3868108830 2024-04-26T01:52:48.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:48.406+0000 7f3867400700 1 --2- 172.21.15.151:0/3253100769 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3868108830 0x7f38681a23b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f386000b070 tx=0x7f3860004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:48.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:48.406+0000 7f385f400700 1 -- 172.21.15.151:0/3253100769 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3860021070 con 0x7f3868108830 2024-04-26T01:52:48.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:48.406+0000 7f386dc00700 1 -- 172.21.15.151:0/3253100769 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38681a2ce0 con 0x7f3868108830 2024-04-26T01:52:48.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:48.406+0000 7f385f400700 1 -- 172.21.15.151:0/3253100769 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3860007e30 con 0x7f3868108830 2024-04-26T01:52:48.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:48.406+0000 7f385f400700 1 -- 172.21.15.151:0/3253100769 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38600046f0 con 0x7f3868108830 2024-04-26T01:52:48.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:48.406+0000 7f386dc00700 1 -- 172.21.15.151:0/3253100769 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38681a3140 con 0x7f3868108830 2024-04-26T01:52:48.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:48.407+0000 7f386dc00700 1 -- 172.21.15.151:0/3253100769 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3868069730 con 0x7f3868108830 2024-04-26T01:52:48.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:48.409+0000 7f385f400700 1 -- 172.21.15.151:0/3253100769 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f3860003d70 con 0x7f3868108830 2024-04-26T01:52:48.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:48.411+0000 7f385f400700 1 --2- 172.21.15.151:0/3253100769 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f384c0a3c40 0x7f384c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:48.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:48.411+0000 7f385f400700 1 -- 172.21.15.151:0/3253100769 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f38600d11f0 con 0x7f3868108830 2024-04-26T01:52:48.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:48.411+0000 7f3866a00700 1 --2- 172.21.15.151:0/3253100769 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f384c0a3c40 0x7f384c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:48.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:48.412+0000 7f3866a00700 1 --2- 172.21.15.151:0/3253100769 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f384c0a3c40 0x7f384c0a6100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3858009c60 tx=0x7f3858009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:48.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:48.413+0000 7f385f400700 1 -- 172.21.15.151:0/3253100769 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f386009dc60 con 0x7f3868108830 2024-04-26T01:52:48.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:48.651+0000 7f386dc00700 1 -- 172.21.15.151:0/3253100769 --> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f3868062e80 con 0x7f384c0a3c40 2024-04-26T01:52:48.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:48.660+0000 7f385f400700 1 -- 172.21.15.151:0/3253100769 <== mgr.14152 v2:172.21.15.151:6800/2251256505 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f3868062e80 con 0x7f384c0a3c40 2024-04-26T01:52:48.658 INFO:teuthology.orchestra.run.smithi151.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-26T01:52:48.658 INFO:teuthology.orchestra.run.smithi151.stdout:mds.all 12/12 72s ago 4m smithi151=a;smithi151=d;smithi151=g;smithi151=j;smithi165=b;smithi165=e;smithi165=h;smithi165=k;smithi172=c;smithi172=f;smithi172=i;smithi172=l;count:12 2024-04-26T01:52:48.658 INFO:teuthology.orchestra.run.smithi151.stdout:mgr 3/3 72s ago 10m smithi151=x;smithi165=y;smithi172=z;count:3 2024-04-26T01:52:48.659 INFO:teuthology.orchestra.run.smithi151.stdout:mon 3/3 72s ago 10m smithi151:172.21.15.151=a;smithi165:172.21.15.165=b;smithi172:172.21.15.172=c;count:3 2024-04-26T01:52:48.659 INFO:teuthology.orchestra.run.smithi151.stdout:osd 12 72s ago - 2024-04-26T01:52:48.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:48.663+0000 7f386dc00700 1 -- 172.21.15.151:0/3253100769 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f384c0a3c40 msgr2=0x7f384c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:48.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:48.663+0000 7f386dc00700 1 --2- 172.21.15.151:0/3253100769 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f384c0a3c40 0x7f384c0a6100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3858009c60 tx=0x7f3858009260 comp rx=0 tx=0).stop 2024-04-26T01:52:48.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:48.663+0000 7f386dc00700 1 -- 172.21.15.151:0/3253100769 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3868108830 msgr2=0x7f38681a23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:48.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:48.663+0000 7f386dc00700 1 --2- 172.21.15.151:0/3253100769 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3868108830 0x7f38681a23b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f386000b070 tx=0x7f3860004d00 comp rx=0 tx=0).stop 2024-04-26T01:52:48.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:48.663+0000 7f386dc00700 1 -- 172.21.15.151:0/3253100769 shutdown_connections 2024-04-26T01:52:48.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:48.663+0000 7f386dc00700 1 --2- 172.21.15.151:0/3253100769 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3868108830 0x7f38681a23b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:48.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:48.663+0000 7f386dc00700 1 --2- 172.21.15.151:0/3253100769 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f384c0a3c40 0x7f384c0a6100 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:48.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:48.663+0000 7f386dc00700 1 --2- 172.21.15.151:0/3253100769 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f38681035d0 0x7f386819adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:48.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:48.663+0000 7f386dc00700 1 --2- 172.21.15.151:0/3253100769 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3868105f00 0x7f386819b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:48.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:48.663+0000 7f386dc00700 1 -- 172.21.15.151:0/3253100769 >> 172.21.15.151:0/3253100769 conn(0x7f38680fcf40 msgr2=0x7f3868102d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:48.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:48.663+0000 7f386dc00700 1 -- 172.21.15.151:0/3253100769 shutdown_connections 2024-04-26T01:52:48.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:48.663+0000 7f386dc00700 1 -- 172.21.15.151:0/3253100769 wait complete. 2024-04-26T01:52:49.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:52:49 smithi151 ceph-mon[28057]: pgmap v423: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T01:52:49.466 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-26T01:52:49.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:52:49 smithi172 ceph-mon[32785]: pgmap v423: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T01:52:49.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:52:49 smithi165 ceph-mon[32648]: pgmap v423: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T01:52:49.922 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:52:50.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:52:50 smithi151 ceph-mon[28057]: from='client.24908 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:52:50.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:52:50 smithi172 ceph-mon[32785]: from='client.24908 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:52:50.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:52:50 smithi165 ceph-mon[32648]: from='client.24908 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:52:51.237 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:52:51 smithi151 ceph-mon[28057]: pgmap v424: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:52:51.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:52:51 smithi172 ceph-mon[32785]: pgmap v424: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:52:51.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:52:51 smithi165 ceph-mon[32648]: pgmap v424: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:52:52.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:52.156+0000 7f4d80200700 1 -- 172.21.15.151:0/3729788033 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d78102aa0 msgr2=0x7f4d78102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:52.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:52.156+0000 7f4d80200700 1 --2- 172.21.15.151:0/3729788033 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d78102aa0 0x7f4d78102ec0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c0099c0 tx=0x7f4d6c009cd0 comp rx=0 tx=0).stop 2024-04-26T01:52:52.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:52.157+0000 7f4d80200700 1 -- 172.21.15.151:0/3729788033 shutdown_connections 2024-04-26T01:52:52.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:52.157+0000 7f4d80200700 1 --2- 172.21.15.151:0/3729788033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4d78104640 0x7f4d7810aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:52.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:52.157+0000 7f4d80200700 1 --2- 172.21.15.151:0/3729788033 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d78102aa0 0x7f4d78102ec0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:52.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:52.157+0000 7f4d80200700 1 --2- 172.21.15.151:0/3729788033 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d78103ca0 0x7f4d78104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:52.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:52.157+0000 7f4d80200700 1 -- 172.21.15.151:0/3729788033 >> 172.21.15.151:0/3729788033 conn(0x7f4d780fe040 msgr2=0x7f4d78100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:52.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:52.157+0000 7f4d80200700 1 -- 172.21.15.151:0/3729788033 shutdown_connections 2024-04-26T01:52:52.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:52.157+0000 7f4d80200700 1 -- 172.21.15.151:0/3729788033 wait complete. 2024-04-26T01:52:52.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:52.158+0000 7f4d80200700 1 Processor -- start 2024-04-26T01:52:52.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:52.158+0000 7f4d80200700 1 -- start start 2024-04-26T01:52:52.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:52.158+0000 7f4d80200700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d78102aa0 0x7f4d7819ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:52.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:52.158+0000 7f4d80200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4d78103ca0 0x7f4d7819b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:52.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:52.158+0000 7f4d80200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d78104640 0x7f4d781a23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:52.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:52.158+0000 7f4d80200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4d7810e510 con 0x7f4d78104640 2024-04-26T01:52:52.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:52.158+0000 7f4d80200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4d7810e390 con 0x7f4d78103ca0 2024-04-26T01:52:52.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:52.158+0000 7f4d80200700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4d7810e690 con 0x7f4d78102aa0 2024-04-26T01:52:52.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:52.159+0000 7f4d7e200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d78104640 0x7f4d781a23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:52.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:52.159+0000 7f4d7e200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d78104640 0x7f4d781a23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:57476/0 (socket says 172.21.15.151:57476) 2024-04-26T01:52:52.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:52.159+0000 7f4d7e200700 1 -- 172.21.15.151:0/177136651 learned_addr learned my addr 172.21.15.151:0/177136651 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:52:52.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:52.159+0000 7f4d7d800700 1 --2- 172.21.15.151:0/177136651 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d78102aa0 0x7f4d7819ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:52.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:52.159+0000 7f4d7ce00700 1 --2- 172.21.15.151:0/177136651 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4d78103ca0 0x7f4d7819b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:52.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:52.159+0000 7f4d7e200700 1 -- 172.21.15.151:0/177136651 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d78102aa0 msgr2=0x7f4d7819ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:52.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:52.159+0000 7f4d7e200700 1 --2- 172.21.15.151:0/177136651 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d78102aa0 0x7f4d7819ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:52.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:52.159+0000 7f4d7e200700 1 -- 172.21.15.151:0/177136651 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4d78103ca0 msgr2=0x7f4d7819b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:52.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:52.159+0000 7f4d7e200700 1 --2- 172.21.15.151:0/177136651 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4d78103ca0 0x7f4d7819b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:52.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:52.159+0000 7f4d7e200700 1 -- 172.21.15.151:0/177136651 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d781a2a60 con 0x7f4d78104640 2024-04-26T01:52:52.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:52.159+0000 7f4d7e200700 1 --2- 172.21.15.151:0/177136651 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d78104640 0x7f4d781a23c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4d7000ef60 tx=0x7f4d7000ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:52.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:52.160+0000 7f4d76000700 1 -- 172.21.15.151:0/177136651 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d700092e0 con 0x7f4d78104640 2024-04-26T01:52:52.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:52.160+0000 7f4d76000700 1 -- 172.21.15.151:0/177136651 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d70004bb0 con 0x7f4d78104640 2024-04-26T01:52:52.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:52.160+0000 7f4d76000700 1 -- 172.21.15.151:0/177136651 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d70004020 con 0x7f4d78104640 2024-04-26T01:52:52.160 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:52.160+0000 7f4d80200700 1 -- 172.21.15.151:0/177136651 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d781a2cf0 con 0x7f4d78104640 2024-04-26T01:52:52.160 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:52.160+0000 7f4d80200700 1 -- 172.21.15.151:0/177136651 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d781a3220 con 0x7f4d78104640 2024-04-26T01:52:52.160 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:52.161+0000 7f4d80200700 1 -- 172.21.15.151:0/177136651 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d78069760 con 0x7f4d78104640 2024-04-26T01:52:52.161 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:52.163+0000 7f4d76000700 1 -- 172.21.15.151:0/177136651 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f4d7001b030 con 0x7f4d78104640 2024-04-26T01:52:52.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:52.164+0000 7f4d76000700 1 --2- 172.21.15.151:0/177136651 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f4d600a3cc0 0x7f4d600a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:52.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:52.165+0000 7f4d76000700 1 -- 172.21.15.151:0/177136651 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f4d700d06f0 con 0x7f4d78104640 2024-04-26T01:52:52.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:52.165+0000 7f4d7d800700 1 --2- 172.21.15.151:0/177136651 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f4d600a3cc0 0x7f4d600a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:52.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:52.165+0000 7f4d7d800700 1 --2- 172.21.15.151:0/177136651 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f4d600a3cc0 0x7f4d600a6180 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c0036f0 tx=0x7f4d6c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:52.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:52.167+0000 7f4d76000700 1 -- 172.21.15.151:0/177136651 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4d78069760 con 0x7f4d78104640 2024-04-26T01:52:52.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:52.401+0000 7f4d80200700 1 -- 172.21.15.151:0/177136651 --> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f4d7810abe0 con 0x7f4d600a3cc0 2024-04-26T01:52:52.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:52.403+0000 7f4d76000700 1 -- 172.21.15.151:0/177136651 <== mgr.14152 v2:172.21.15.151:6800/2251256505 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f4d7810abe0 con 0x7f4d600a3cc0 2024-04-26T01:52:52.402 INFO:teuthology.orchestra.run.smithi151.stdout:HOST ADDR LABELS STATUS 2024-04-26T01:52:52.402 INFO:teuthology.orchestra.run.smithi151.stdout:smithi151 172.21.15.151 2024-04-26T01:52:52.402 INFO:teuthology.orchestra.run.smithi151.stdout:smithi165 172.21.15.165 2024-04-26T01:52:52.402 INFO:teuthology.orchestra.run.smithi151.stdout:smithi172 172.21.15.172 2024-04-26T01:52:52.403 INFO:teuthology.orchestra.run.smithi151.stdout:3 hosts in cluster 2024-04-26T01:52:52.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:52.406+0000 7f4d80200700 1 -- 172.21.15.151:0/177136651 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f4d600a3cc0 msgr2=0x7f4d600a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:52.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:52.406+0000 7f4d80200700 1 --2- 172.21.15.151:0/177136651 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f4d600a3cc0 0x7f4d600a6180 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c0036f0 tx=0x7f4d6c005fb0 comp rx=0 tx=0).stop 2024-04-26T01:52:52.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:52.407+0000 7f4d80200700 1 -- 172.21.15.151:0/177136651 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d78104640 msgr2=0x7f4d781a23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:52.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:52.407+0000 7f4d80200700 1 --2- 172.21.15.151:0/177136651 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d78104640 0x7f4d781a23c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4d7000ef60 tx=0x7f4d7000ef90 comp rx=0 tx=0).stop 2024-04-26T01:52:52.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:52.407+0000 7f4d80200700 1 -- 172.21.15.151:0/177136651 shutdown_connections 2024-04-26T01:52:52.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:52.407+0000 7f4d80200700 1 --2- 172.21.15.151:0/177136651 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f4d600a3cc0 0x7f4d600a6180 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:52.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:52.407+0000 7f4d80200700 1 --2- 172.21.15.151:0/177136651 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d78104640 0x7f4d781a23c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:52.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:52.407+0000 7f4d80200700 1 --2- 172.21.15.151:0/177136651 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d78102aa0 0x7f4d7819ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:52.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:52.407+0000 7f4d80200700 1 --2- 172.21.15.151:0/177136651 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4d78103ca0 0x7f4d7819b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:52.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:52.407+0000 7f4d80200700 1 -- 172.21.15.151:0/177136651 >> 172.21.15.151:0/177136651 conn(0x7f4d780fe040 msgr2=0x7f4d781094c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:52.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:52.407+0000 7f4d80200700 1 -- 172.21.15.151:0/177136651 shutdown_connections 2024-04-26T01:52:52.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:52.407+0000 7f4d80200700 1 -- 172.21.15.151:0/177136651 wait complete. 2024-04-26T01:52:53.206 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-26T01:52:53.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:52:53 smithi151 ceph-mon[28057]: pgmap v425: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:52:53.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:52:53 smithi172 ceph-mon[32785]: pgmap v425: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:52:53.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:52:53 smithi165 ceph-mon[32648]: pgmap v425: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:52:53.652 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:52:54.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:52:54 smithi172 ceph-mon[32785]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:52:54.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:52:54 smithi165 ceph-mon[32648]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:52:54.597 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:52:54 smithi151 ceph-mon[28057]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:52:55.451 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:52:55 smithi151 ceph-mon[28057]: pgmap v426: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:52:55.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:52:55 smithi172 ceph-mon[32785]: pgmap v426: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:52:55.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:52:55 smithi165 ceph-mon[32648]: pgmap v426: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:52:55.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:55.781+0000 7f7d5ac00700 1 -- 172.21.15.151:0/3743389038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d54101310 msgr2=0x7f7d54101730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:55.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:55.781+0000 7f7d5ac00700 1 --2- 172.21.15.151:0/3743389038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d54101310 0x7f7d54101730 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7d44009990 tx=0x7f7d44009ca0 comp rx=0 tx=0).stop 2024-04-26T01:52:55.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:55.781+0000 7f7d5ac00700 1 -- 172.21.15.151:0/3743389038 shutdown_connections 2024-04-26T01:52:55.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:55.781+0000 7f7d5ac00700 1 --2- 172.21.15.151:0/3743389038 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7d54102630 0x7f7d54102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:55.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:55.781+0000 7f7d5ac00700 1 --2- 172.21.15.151:0/3743389038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d54101310 0x7f7d54101730 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:55.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:55.781+0000 7f7d5ac00700 1 --2- 172.21.15.151:0/3743389038 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d54101c70 0x7f7d541020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:55.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:55.781+0000 7f7d5ac00700 1 -- 172.21.15.151:0/3743389038 >> 172.21.15.151:0/3743389038 conn(0x7f7d540fcf30 msgr2=0x7f7d540ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:55.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:55.781+0000 7f7d5ac00700 1 -- 172.21.15.151:0/3743389038 shutdown_connections 2024-04-26T01:52:55.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:55.781+0000 7f7d5ac00700 1 -- 172.21.15.151:0/3743389038 wait complete. 2024-04-26T01:52:55.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:55.782+0000 7f7d5ac00700 1 Processor -- start 2024-04-26T01:52:55.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:55.783+0000 7f7d5ac00700 1 -- start start 2024-04-26T01:52:55.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:55.783+0000 7f7d5ac00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d54101310 0x7f7d5419ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:55.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:55.783+0000 7f7d5ac00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d54101c70 0x7f7d5419b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:55.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:55.783+0000 7f7d5ac00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7d54102630 0x7f7d541a2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:55.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:55.783+0000 7f7d5ac00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7d54110750 con 0x7f7d54101c70 2024-04-26T01:52:55.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:55.783+0000 7f7d5ac00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7d541105d0 con 0x7f7d54102630 2024-04-26T01:52:55.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:55.783+0000 7f7d5ac00700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7d541108d0 con 0x7f7d54101310 2024-04-26T01:52:55.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:55.783+0000 7f7d53400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d54101c70 0x7f7d5419b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:55.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:55.783+0000 7f7d53400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d54101c70 0x7f7d5419b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:57506/0 (socket says 172.21.15.151:57506) 2024-04-26T01:52:55.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:55.783+0000 7f7d53e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d54101310 0x7f7d5419ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:55.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:55.783+0000 7f7d53400700 1 -- 172.21.15.151:0/2297078059 learned_addr learned my addr 172.21.15.151:0/2297078059 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:52:55.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:55.783+0000 7f7d58c00700 1 --2- 172.21.15.151:0/2297078059 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7d54102630 0x7f7d541a2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:55.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:55.784+0000 7f7d53400700 1 -- 172.21.15.151:0/2297078059 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d54101310 msgr2=0x7f7d5419ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:55.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:55.784+0000 7f7d53400700 1 --2- 172.21.15.151:0/2297078059 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d54101310 0x7f7d5419ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:55.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:55.784+0000 7f7d53400700 1 -- 172.21.15.151:0/2297078059 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7d54102630 msgr2=0x7f7d541a2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:55.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:55.784+0000 7f7d53400700 1 --2- 172.21.15.151:0/2297078059 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7d54102630 0x7f7d541a2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:55.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:55.784+0000 7f7d53400700 1 -- 172.21.15.151:0/2297078059 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d541a2ac0 con 0x7f7d54101c70 2024-04-26T01:52:55.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:55.784+0000 7f7d53400700 1 --2- 172.21.15.151:0/2297078059 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d54101c70 0x7f7d5419b3a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7d4000e9f0 tx=0x7f7d4000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:55.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:55.784+0000 7f7d50c00700 1 -- 172.21.15.151:0/2297078059 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4000cbd0 con 0x7f7d54101c70 2024-04-26T01:52:55.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:55.784+0000 7f7d5ac00700 1 -- 172.21.15.151:0/2297078059 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d541a2db0 con 0x7f7d54101c70 2024-04-26T01:52:55.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:55.784+0000 7f7d50c00700 1 -- 172.21.15.151:0/2297078059 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d4000ee90 con 0x7f7d54101c70 2024-04-26T01:52:55.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:55.784+0000 7f7d50c00700 1 -- 172.21.15.151:0/2297078059 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d40005590 con 0x7f7d54101c70 2024-04-26T01:52:55.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:55.785+0000 7f7d5ac00700 1 -- 172.21.15.151:0/2297078059 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d541a3210 con 0x7f7d54101c70 2024-04-26T01:52:55.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:55.786+0000 7f7d5ac00700 1 -- 172.21.15.151:0/2297078059 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d54069730 con 0x7f7d54101c70 2024-04-26T01:52:55.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:55.788+0000 7f7d50c00700 1 -- 172.21.15.151:0/2297078059 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f7d40004750 con 0x7f7d54101c70 2024-04-26T01:52:55.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:55.789+0000 7f7d50c00700 1 --2- 172.21.15.151:0/2297078059 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f7d380a3c40 0x7f7d380a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:55.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:55.789+0000 7f7d50c00700 1 -- 172.21.15.151:0/2297078059 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f7d400d05f0 con 0x7f7d54101c70 2024-04-26T01:52:55.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:55.789+0000 7f7d53e00700 1 --2- 172.21.15.151:0/2297078059 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f7d380a3c40 0x7f7d380a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:55.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:55.790+0000 7f7d53e00700 1 --2- 172.21.15.151:0/2297078059 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f7d380a3c40 0x7f7d380a6100 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7d4400b2e0 tx=0x7f7d44005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:55.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:55.794+0000 7f7d50c00700 1 -- 172.21.15.151:0/2297078059 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7d4009d070 con 0x7f7d54101c70 2024-04-26T01:52:56.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:56.031+0000 7f7d5ac00700 1 -- 172.21.15.151:0/2297078059 --> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f7d54062e80 con 0x7f7d380a3c40 2024-04-26T01:52:56.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:56.035+0000 7f7d50c00700 1 -- 172.21.15.151:0/2297078059 <== mgr.14152 v2:172.21.15.151:6800/2251256505 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f7d54062e80 con 0x7f7d380a3c40 2024-04-26T01:52:56.034 INFO:teuthology.orchestra.run.smithi151.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-26T01:52:56.034 INFO:teuthology.orchestra.run.smithi151.stdout:smithi151 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300JJ400BGN 372G No 5m ago Has a FileSystem, LVM detected 2024-04-26T01:52:56.034 INFO:teuthology.orchestra.run.smithi151.stdout:smithi165 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300CE400BGN 372G No 5m ago Has a FileSystem, LVM detected 2024-04-26T01:52:56.034 INFO:teuthology.orchestra.run.smithi151.stdout:smithi172 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233005M400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-04-26T01:52:56.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:56.038+0000 7f7d5ac00700 1 -- 172.21.15.151:0/2297078059 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f7d380a3c40 msgr2=0x7f7d380a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:56.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:56.038+0000 7f7d5ac00700 1 --2- 172.21.15.151:0/2297078059 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f7d380a3c40 0x7f7d380a6100 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7d4400b2e0 tx=0x7f7d44005fb0 comp rx=0 tx=0).stop 2024-04-26T01:52:56.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:56.038+0000 7f7d5ac00700 1 -- 172.21.15.151:0/2297078059 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d54101c70 msgr2=0x7f7d5419b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:56.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:56.038+0000 7f7d5ac00700 1 --2- 172.21.15.151:0/2297078059 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d54101c70 0x7f7d5419b3a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7d4000e9f0 tx=0x7f7d4000ed00 comp rx=0 tx=0).stop 2024-04-26T01:52:56.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:56.038+0000 7f7d5ac00700 1 -- 172.21.15.151:0/2297078059 shutdown_connections 2024-04-26T01:52:56.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:56.038+0000 7f7d5ac00700 1 --2- 172.21.15.151:0/2297078059 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7d54102630 0x7f7d541a2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:56.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:56.038+0000 7f7d5ac00700 1 --2- 172.21.15.151:0/2297078059 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d54101310 0x7f7d5419ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:56.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:56.038+0000 7f7d5ac00700 1 --2- 172.21.15.151:0/2297078059 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f7d380a3c40 0x7f7d380a6100 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:56.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:56.038+0000 7f7d5ac00700 1 --2- 172.21.15.151:0/2297078059 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d54101c70 0x7f7d5419b3a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:56.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:56.038+0000 7f7d5ac00700 1 -- 172.21.15.151:0/2297078059 >> 172.21.15.151:0/2297078059 conn(0x7f7d540fcf30 msgr2=0x7f7d540fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:56.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:56.038+0000 7f7d5ac00700 1 -- 172.21.15.151:0/2297078059 shutdown_connections 2024-04-26T01:52:56.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:56.039+0000 7f7d5ac00700 1 -- 172.21.15.151:0/2297078059 wait complete. 2024-04-26T01:52:56.855 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-26T01:52:56.869 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi151.front.sepia.ceph.com 2024-04-26T01:52:56.870 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-26T01:52:57.315 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:52:57.466 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:52:57 smithi151 ceph-mon[28057]: pgmap v427: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:52:57.466 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:52:57 smithi151 ceph-mon[28057]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:52:57.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:52:57 smithi172 ceph-mon[32785]: pgmap v427: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:52:57.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:52:57 smithi172 ceph-mon[32785]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:52:57.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:52:57 smithi165 ceph-mon[32648]: pgmap v427: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:52:57.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:52:57 smithi165 ceph-mon[32648]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:52:59.417 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:52:59 smithi151 ceph-mon[28057]: pgmap v428: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T01:52:59.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:59.487+0000 7f9a9e800700 1 -- 172.21.15.151:0/4148324962 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9a98102ba0 msgr2=0x7f9a98103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:59.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:59.487+0000 7f9a9e800700 1 --2- 172.21.15.151:0/4148324962 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9a98102ba0 0x7f9a98103000 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9a840099c0 tx=0x7f9a84009cd0 comp rx=0 tx=0).stop 2024-04-26T01:52:59.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:59.487+0000 7f9a9e800700 1 -- 172.21.15.151:0/4148324962 shutdown_connections 2024-04-26T01:52:59.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:59.487+0000 7f9a9e800700 1 --2- 172.21.15.151:0/4148324962 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a98103540 0x7f9a98109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:59.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:59.487+0000 7f9a9e800700 1 --2- 172.21.15.151:0/4148324962 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a981019a0 0x7f9a98101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:59.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:59.487+0000 7f9a9e800700 1 --2- 172.21.15.151:0/4148324962 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9a98102ba0 0x7f9a98103000 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:59.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:59.487+0000 7f9a9e800700 1 -- 172.21.15.151:0/4148324962 >> 172.21.15.151:0/4148324962 conn(0x7f9a980fcf40 msgr2=0x7f9a980ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:59.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:59.487+0000 7f9a9e800700 1 -- 172.21.15.151:0/4148324962 shutdown_connections 2024-04-26T01:52:59.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:59.487+0000 7f9a9e800700 1 -- 172.21.15.151:0/4148324962 wait complete. 2024-04-26T01:52:59.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:59.488+0000 7f9a9e800700 1 Processor -- start 2024-04-26T01:52:59.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:59.488+0000 7f9a9e800700 1 -- start start 2024-04-26T01:52:59.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:59.489+0000 7f9a9e800700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a981019a0 0x7f9a9819ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:59.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:59.489+0000 7f9a9e800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9a98102ba0 0x7f9a9819b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:59.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:59.489+0000 7f9a9e800700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a98103540 0x7f9a981a2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:59.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:59.489+0000 7f9a9e800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9810d460 con 0x7f9a98102ba0 2024-04-26T01:52:59.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:59.489+0000 7f9a9e800700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9810d2e0 con 0x7f9a981019a0 2024-04-26T01:52:59.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:59.489+0000 7f9a9e800700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9810d5e0 con 0x7f9a98103540 2024-04-26T01:52:59.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:59.489+0000 7f9a96a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9a98102ba0 0x7f9a9819b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:59.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:59.489+0000 7f9a97400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a981019a0 0x7f9a9819ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:59.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:59.489+0000 7f9a96a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9a98102ba0 0x7f9a9819b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:57516/0 (socket says 172.21.15.151:57516) 2024-04-26T01:52:59.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:59.489+0000 7f9a97400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a981019a0 0x7f9a9819ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:33974/0 (socket says 172.21.15.151:33974) 2024-04-26T01:52:59.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:59.489+0000 7f9a96a00700 1 -- 172.21.15.151:0/2560813519 learned_addr learned my addr 172.21.15.151:0/2560813519 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:52:59.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:59.489+0000 7f9a97e00700 1 --2- 172.21.15.151:0/2560813519 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a98103540 0x7f9a981a2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:59.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:59.490+0000 7f9a97400700 1 -- 172.21.15.151:0/2560813519 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a98103540 msgr2=0x7f9a981a2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:59.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:59.490+0000 7f9a97400700 1 --2- 172.21.15.151:0/2560813519 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a98103540 0x7f9a981a2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:59.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:59.490+0000 7f9a97400700 1 -- 172.21.15.151:0/2560813519 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9a98102ba0 msgr2=0x7f9a9819b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:59.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:59.490+0000 7f9a97400700 1 --2- 172.21.15.151:0/2560813519 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9a98102ba0 0x7f9a9819b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:59.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:59.490+0000 7f9a97400700 1 -- 172.21.15.151:0/2560813519 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a981a2aa0 con 0x7f9a981019a0 2024-04-26T01:52:59.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:59.490+0000 7f9a97400700 1 --2- 172.21.15.151:0/2560813519 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a981019a0 0x7f9a9819ae40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9a8800e9c0 tx=0x7f9a8800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:59.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:59.490+0000 7f9a8fe00700 1 -- 172.21.15.151:0/2560813519 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a88004220 con 0x7f9a981019a0 2024-04-26T01:52:59.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:59.491+0000 7f9a8fe00700 1 -- 172.21.15.151:0/2560813519 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a880043c0 con 0x7f9a981019a0 2024-04-26T01:52:59.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:59.491+0000 7f9a9e800700 1 -- 172.21.15.151:0/2560813519 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a981a2d90 con 0x7f9a981019a0 2024-04-26T01:52:59.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:59.491+0000 7f9a8fe00700 1 -- 172.21.15.151:0/2560813519 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a88003c30 con 0x7f9a981019a0 2024-04-26T01:52:59.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:59.491+0000 7f9a9e800700 1 -- 172.21.15.151:0/2560813519 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a981a3210 con 0x7f9a981019a0 2024-04-26T01:52:59.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:59.492+0000 7f9a9e800700 1 -- 172.21.15.151:0/2560813519 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a98069760 con 0x7f9a981019a0 2024-04-26T01:52:59.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:59.494+0000 7f9a8fe00700 1 -- 172.21.15.151:0/2560813519 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f9a8800c7c0 con 0x7f9a981019a0 2024-04-26T01:52:59.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:59.495+0000 7f9a8fe00700 1 --2- 172.21.15.151:0/2560813519 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f9a7c0a3c40 0x7f9a7c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:59.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:59.495+0000 7f9a8fe00700 1 -- 172.21.15.151:0/2560813519 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f9a880d05c0 con 0x7f9a981019a0 2024-04-26T01:52:59.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:59.495+0000 7f9a96a00700 1 --2- 172.21.15.151:0/2560813519 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f9a7c0a3c40 0x7f9a7c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:59.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:59.496+0000 7f9a96a00700 1 --2- 172.21.15.151:0/2560813519 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f9a7c0a3c40 0x7f9a7c0a6100 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9a84009990 tx=0x7f9a840037c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:59.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:59.500+0000 7f9a8fe00700 1 -- 172.21.15.151:0/2560813519 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9a98069760 con 0x7f9a981019a0 2024-04-26T01:52:59.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:52:59 smithi172 ceph-mon[32785]: pgmap v428: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T01:52:59.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:52:59 smithi165 ceph-mon[32648]: pgmap v428: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T01:52:59.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:59.779+0000 7f9a9e800700 1 -- 172.21.15.151:0/2560813519 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f9a981a3c90 con 0x7f9a981019a0 2024-04-26T01:52:59.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:59.780+0000 7f9a8fe00700 1 -- 172.21.15.151:0/2560813519 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+3062 (secure 0 0 0) 0x7f9a8809cfa0 con 0x7f9a981019a0 2024-04-26T01:52:59.780 INFO:teuthology.orchestra.run.smithi151.stdout:e25 2024-04-26T01:52:59.780 INFO:teuthology.orchestra.run.smithi151.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T01:52:59.780 INFO:teuthology.orchestra.run.smithi151.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T01:52:59.780 INFO:teuthology.orchestra.run.smithi151.stdout:legacy client fscid: 1 2024-04-26T01:52:59.780 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:52:59.780 INFO:teuthology.orchestra.run.smithi151.stdout:Filesystem 'cephfs' (1) 2024-04-26T01:52:59.780 INFO:teuthology.orchestra.run.smithi151.stdout:fs_name cephfs 2024-04-26T01:52:59.780 INFO:teuthology.orchestra.run.smithi151.stdout:epoch 25 2024-04-26T01:52:59.780 INFO:teuthology.orchestra.run.smithi151.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-26T01:52:59.780 INFO:teuthology.orchestra.run.smithi151.stdout:created 2024-04-26T01:49:40.746876+0000 2024-04-26T01:52:59.780 INFO:teuthology.orchestra.run.smithi151.stdout:modified 2024-04-26T01:50:05.608353+0000 2024-04-26T01:52:59.780 INFO:teuthology.orchestra.run.smithi151.stdout:tableserver 0 2024-04-26T01:52:59.780 INFO:teuthology.orchestra.run.smithi151.stdout:root 0 2024-04-26T01:52:59.781 INFO:teuthology.orchestra.run.smithi151.stdout:session_timeout 300 2024-04-26T01:52:59.781 INFO:teuthology.orchestra.run.smithi151.stdout:session_autoclose 300 2024-04-26T01:52:59.781 INFO:teuthology.orchestra.run.smithi151.stdout:max_file_size 1099511627776 2024-04-26T01:52:59.781 INFO:teuthology.orchestra.run.smithi151.stdout:max_xattr_size 65536 2024-04-26T01:52:59.781 INFO:teuthology.orchestra.run.smithi151.stdout:required_client_features {} 2024-04-26T01:52:59.781 INFO:teuthology.orchestra.run.smithi151.stdout:last_failure 0 2024-04-26T01:52:59.781 INFO:teuthology.orchestra.run.smithi151.stdout:last_failure_osd_epoch 0 2024-04-26T01:52:59.781 INFO:teuthology.orchestra.run.smithi151.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T01:52:59.781 INFO:teuthology.orchestra.run.smithi151.stdout:max_mds 3 2024-04-26T01:52:59.781 INFO:teuthology.orchestra.run.smithi151.stdout:in 0,1,2 2024-04-26T01:52:59.781 INFO:teuthology.orchestra.run.smithi151.stdout:up {0=24509,1=24505,2=24493} 2024-04-26T01:52:59.781 INFO:teuthology.orchestra.run.smithi151.stdout:failed 2024-04-26T01:52:59.781 INFO:teuthology.orchestra.run.smithi151.stdout:damaged 2024-04-26T01:52:59.782 INFO:teuthology.orchestra.run.smithi151.stdout:stopped 2024-04-26T01:52:59.782 INFO:teuthology.orchestra.run.smithi151.stdout:data_pools [3] 2024-04-26T01:52:59.782 INFO:teuthology.orchestra.run.smithi151.stdout:metadata_pool 2 2024-04-26T01:52:59.782 INFO:teuthology.orchestra.run.smithi151.stdout:inline_data disabled 2024-04-26T01:52:59.782 INFO:teuthology.orchestra.run.smithi151.stdout:balancer 2024-04-26T01:52:59.782 INFO:teuthology.orchestra.run.smithi151.stdout:bal_rank_mask -1 2024-04-26T01:52:59.782 INFO:teuthology.orchestra.run.smithi151.stdout:standby_count_wanted 0 2024-04-26T01:52:59.782 INFO:teuthology.orchestra.run.smithi151.stdout:qdb_cluster leader: 24509 members: 24505,24509,24493 2024-04-26T01:52:59.782 INFO:teuthology.orchestra.run.smithi151.stdout:[mds.b{0:24509} state up:active seq 7 addr [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T01:52:59.782 INFO:teuthology.orchestra.run.smithi151.stdout:[mds.f{0:24481} state up:standby-replay seq 1 addr [v2:172.21.15.172:6834/2309751171,v1:172.21.15.172:6835/2309751171] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T01:52:59.782 INFO:teuthology.orchestra.run.smithi151.stdout:[mds.i{1:24505} state up:active seq 14 addr [v2:172.21.15.172:6838/1242302810,v1:172.21.15.172:6839/1242302810] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T01:52:59.782 INFO:teuthology.orchestra.run.smithi151.stdout:[mds.h{1:24470} state up:standby-replay seq 1 addr [v2:172.21.15.165:6836/1847859054,v1:172.21.15.165:6837/1847859054] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T01:52:59.782 INFO:teuthology.orchestra.run.smithi151.stdout:[mds.c{2:24493} state up:active seq 16 addr [v2:172.21.15.172:6836/55972537,v1:172.21.15.172:6837/55972537] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T01:52:59.782 INFO:teuthology.orchestra.run.smithi151.stdout:[mds.e{2:24458} state up:standby-replay seq 1 addr [v2:172.21.15.165:6834/3598138294,v1:172.21.15.165:6835/3598138294] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T01:52:59.782 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:52:59.783 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:52:59.783 INFO:teuthology.orchestra.run.smithi151.stdout:Standby daemons: 2024-04-26T01:52:59.783 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:52:59.783 INFO:teuthology.orchestra.run.smithi151.stdout:[mds.j{-1:14367} state up:standby seq 1 addr [v2:172.21.15.151:6834/2937864097,v1:172.21.15.151:6835/2937864097] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T01:52:59.783 INFO:teuthology.orchestra.run.smithi151.stdout:[mds.d{-1:14373} state up:standby seq 1 addr [v2:172.21.15.151:6836/2947090802,v1:172.21.15.151:6837/2947090802] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T01:52:59.783 INFO:teuthology.orchestra.run.smithi151.stdout:[mds.g{-1:14397} state up:standby seq 1 addr [v2:172.21.15.151:6838/1356563280,v1:172.21.15.151:6839/1356563280] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T01:52:59.783 INFO:teuthology.orchestra.run.smithi151.stdout:[mds.a{-1:14409} state up:standby seq 1 addr [v2:172.21.15.151:6840/1075240552,v1:172.21.15.151:6841/1075240552] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T01:52:59.783 INFO:teuthology.orchestra.run.smithi151.stdout:[mds.k{-1:24428} state up:standby seq 1 addr [v2:172.21.15.165:6832/504675912,v1:172.21.15.165:6833/504675912] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T01:52:59.783 INFO:teuthology.orchestra.run.smithi151.stdout:[mds.l{-1:24442} state up:standby seq 1 addr [v2:172.21.15.172:6832/1207580015,v1:172.21.15.172:6833/1207580015] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T01:52:59.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:59.784+0000 7f9a9e800700 1 -- 172.21.15.151:0/2560813519 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f9a7c0a3c40 msgr2=0x7f9a7c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:59.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:59.784+0000 7f9a9e800700 1 --2- 172.21.15.151:0/2560813519 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f9a7c0a3c40 0x7f9a7c0a6100 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9a84009990 tx=0x7f9a840037c0 comp rx=0 tx=0).stop 2024-04-26T01:52:59.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:59.784+0000 7f9a9e800700 1 -- 172.21.15.151:0/2560813519 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a981019a0 msgr2=0x7f9a9819ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:59.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:59.784+0000 7f9a9e800700 1 --2- 172.21.15.151:0/2560813519 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a981019a0 0x7f9a9819ae40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9a8800e9c0 tx=0x7f9a8800ecd0 comp rx=0 tx=0).stop 2024-04-26T01:52:59.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:59.785+0000 7f9a9e800700 1 -- 172.21.15.151:0/2560813519 shutdown_connections 2024-04-26T01:52:59.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:59.785+0000 7f9a9e800700 1 --2- 172.21.15.151:0/2560813519 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a98103540 0x7f9a981a2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:59.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:59.785+0000 7f9a9e800700 1 --2- 172.21.15.151:0/2560813519 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a981019a0 0x7f9a9819ae40 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:59.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:59.785+0000 7f9a9e800700 1 --2- 172.21.15.151:0/2560813519 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f9a7c0a3c40 0x7f9a7c0a6100 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:59.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:59.785+0000 7f9a9e800700 1 --2- 172.21.15.151:0/2560813519 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9a98102ba0 0x7f9a9819b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:59.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:59.785+0000 7f9a9e800700 1 -- 172.21.15.151:0/2560813519 >> 172.21.15.151:0/2560813519 conn(0x7f9a980fcf40 msgr2=0x7f9a9806a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:59.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:59.785+0000 7f9a9e800700 1 -- 172.21.15.151:0/2560813519 shutdown_connections 2024-04-26T01:52:59.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:52:59.785+0000 7f9a9e800700 1 -- 172.21.15.151:0/2560813519 wait complete. 2024-04-26T01:52:59.785 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 25 2024-04-26T01:53:00.477 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:00 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/2560813519' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T01:53:00.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:00 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/2560813519' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T01:53:00.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:00 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/2560813519' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T01:53:00.633 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-26T01:53:01.090 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:53:01.452 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:01 smithi151 ceph-mon[28057]: pgmap v429: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:53:01.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:01 smithi172 ceph-mon[32785]: pgmap v429: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:53:01.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:01 smithi165 ceph-mon[32648]: pgmap v429: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:53:03.355 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:03 smithi151 ceph-mon[28057]: pgmap v430: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:53:03.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:03.411+0000 7f9defc00700 1 -- 172.21.15.151:0/170778443 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9de8076a70 msgr2=0x7f9de8074ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:03.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:03.411+0000 7f9defc00700 1 --2- 172.21.15.151:0/170778443 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9de8076a70 0x7f9de8074ed0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc009990 tx=0x7f9ddc009ca0 comp rx=0 tx=0).stop 2024-04-26T01:53:03.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:03.411+0000 7f9defc00700 1 -- 172.21.15.151:0/170778443 shutdown_connections 2024-04-26T01:53:03.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:03.411+0000 7f9defc00700 1 --2- 172.21.15.151:0/170778443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9de810a980 0x7f9de810ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:03.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:03.411+0000 7f9defc00700 1 --2- 172.21.15.151:0/170778443 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9de8076a70 0x7f9de8074ed0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:03.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:03.411+0000 7f9defc00700 1 --2- 172.21.15.151:0/170778443 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9de80754a0 0x7f9de8075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:03.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:03.411+0000 7f9defc00700 1 -- 172.21.15.151:0/170778443 >> 172.21.15.151:0/170778443 conn(0x7f9de80fe010 msgr2=0x7f9de8100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:03.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:03.411+0000 7f9defc00700 1 -- 172.21.15.151:0/170778443 shutdown_connections 2024-04-26T01:53:03.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:03.411+0000 7f9defc00700 1 -- 172.21.15.151:0/170778443 wait complete. 2024-04-26T01:53:03.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:03.412+0000 7f9defc00700 1 Processor -- start 2024-04-26T01:53:03.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:03.413+0000 7f9defc00700 1 -- start start 2024-04-26T01:53:03.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:03.413+0000 7f9defc00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9de80754a0 0x7f9de819ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:03.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:03.413+0000 7f9defc00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9de8076a70 0x7f9de819b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:03.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:03.413+0000 7f9defc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9de810a980 0x7f9de81a2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:03.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:03.413+0000 7f9defc00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9de8110770 con 0x7f9de810a980 2024-04-26T01:53:03.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:03.413+0000 7f9defc00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9de81105f0 con 0x7f9de80754a0 2024-04-26T01:53:03.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:03.413+0000 7f9defc00700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9de81108f0 con 0x7f9de8076a70 2024-04-26T01:53:03.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:03.413+0000 7f9de7e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9de8076a70 0x7f9de819b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:03.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:03.413+0000 7f9ded200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9de80754a0 0x7f9de819ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:03.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:03.413+0000 7f9de7e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9de8076a70 0x7f9de819b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:35712/0 (socket says 172.21.15.151:35712) 2024-04-26T01:53:03.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:03.413+0000 7f9de7e00700 1 -- 172.21.15.151:0/4060736743 learned_addr learned my addr 172.21.15.151:0/4060736743 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:53:03.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:03.414+0000 7f9dedc00700 1 --2- 172.21.15.151:0/4060736743 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9de810a980 0x7f9de81a2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:03.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:03.414+0000 7f9de7e00700 1 -- 172.21.15.151:0/4060736743 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9de80754a0 msgr2=0x7f9de819ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:03.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:03.414+0000 7f9de7e00700 1 --2- 172.21.15.151:0/4060736743 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9de80754a0 0x7f9de819ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:03.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:03.414+0000 7f9de7e00700 1 -- 172.21.15.151:0/4060736743 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9de810a980 msgr2=0x7f9de81a2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:03.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:03.414+0000 7f9de7e00700 1 --2- 172.21.15.151:0/4060736743 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9de810a980 0x7f9de81a2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:03.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:03.414+0000 7f9de7e00700 1 -- 172.21.15.151:0/4060736743 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9de81a2aa0 con 0x7f9de8076a70 2024-04-26T01:53:03.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:03.414+0000 7f9dedc00700 1 --2- 172.21.15.151:0/4060736743 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9de810a980 0x7f9de81a2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:53:03.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:03.414+0000 7f9ded200700 1 --2- 172.21.15.151:0/4060736743 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9de80754a0 0x7f9de819ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:53:03.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:03.414+0000 7f9de7e00700 1 --2- 172.21.15.151:0/4060736743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9de8076a70 0x7f9de819b3d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9dd400e9a0 tx=0x7f9dd400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:03.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:03.414+0000 7f9de5600700 1 -- 172.21.15.151:0/4060736743 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd400cbd0 con 0x7f9de8076a70 2024-04-26T01:53:03.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:03.415+0000 7f9de5600700 1 -- 172.21.15.151:0/4060736743 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dd40044d0 con 0x7f9de8076a70 2024-04-26T01:53:03.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:03.415+0000 7f9de5600700 1 -- 172.21.15.151:0/4060736743 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd4003d80 con 0x7f9de8076a70 2024-04-26T01:53:03.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:03.415+0000 7f9defc00700 1 -- 172.21.15.151:0/4060736743 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9de81a2d90 con 0x7f9de8076a70 2024-04-26T01:53:03.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:03.415+0000 7f9defc00700 1 -- 172.21.15.151:0/4060736743 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9de81a3220 con 0x7f9de8076a70 2024-04-26T01:53:03.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:03.416+0000 7f9defc00700 1 -- 172.21.15.151:0/4060736743 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9de8069770 con 0x7f9de8076a70 2024-04-26T01:53:03.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:03.418+0000 7f9de5600700 1 -- 172.21.15.151:0/4060736743 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f9dd4003f20 con 0x7f9de8076a70 2024-04-26T01:53:03.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:03.419+0000 7f9de5600700 1 --2- 172.21.15.151:0/4060736743 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f9dd00a3c40 0x7f9dd00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:03.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:03.419+0000 7f9de5600700 1 -- 172.21.15.151:0/4060736743 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f9dd40d0680 con 0x7f9de8076a70 2024-04-26T01:53:03.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:03.420+0000 7f9ded200700 1 --2- 172.21.15.151:0/4060736743 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f9dd00a3c40 0x7f9dd00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:03.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:03.420+0000 7f9ded200700 1 --2- 172.21.15.151:0/4060736743 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f9dd00a3c40 0x7f9dd00a6100 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc00b2e0 tx=0x7f9ddc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:03.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:03.426+0000 7f9de5600700 1 -- 172.21.15.151:0/4060736743 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9dd409d060 con 0x7f9de8076a70 2024-04-26T01:53:03.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:03 smithi172 ceph-mon[32785]: pgmap v430: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:53:03.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:03 smithi165 ceph-mon[32648]: pgmap v430: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:53:03.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:03.659+0000 7f9defc00700 1 -- 172.21.15.151:0/4060736743 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f9de802e1d0 con 0x7f9de8076a70 2024-04-26T01:53:03.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:03.660+0000 7f9de5600700 1 -- 172.21.15.151:0/4060736743 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v92) ==== 56+0+5248 (secure 0 0 0) 0x7f9dd409c960 con 0x7f9de8076a70 2024-04-26T01:53:03.659 INFO:teuthology.orchestra.run.smithi151.stdout:epoch 92 2024-04-26T01:53:03.660 INFO:teuthology.orchestra.run.smithi151.stdout:fsid c0e49e64-036d-11ef-bc93-c7b262605968 2024-04-26T01:53:03.660 INFO:teuthology.orchestra.run.smithi151.stdout:created 2024-04-26T01:40:07.483197+0000 2024-04-26T01:53:03.660 INFO:teuthology.orchestra.run.smithi151.stdout:modified 2024-04-26T01:50:26.985192+0000 2024-04-26T01:53:03.660 INFO:teuthology.orchestra.run.smithi151.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-26T01:53:03.660 INFO:teuthology.orchestra.run.smithi151.stdout:crush_version 26 2024-04-26T01:53:03.660 INFO:teuthology.orchestra.run.smithi151.stdout:full_ratio 0.95 2024-04-26T01:53:03.660 INFO:teuthology.orchestra.run.smithi151.stdout:backfillfull_ratio 0.9 2024-04-26T01:53:03.660 INFO:teuthology.orchestra.run.smithi151.stdout:nearfull_ratio 0.85 2024-04-26T01:53:03.661 INFO:teuthology.orchestra.run.smithi151.stdout:require_min_compat_client luminous 2024-04-26T01:53:03.661 INFO:teuthology.orchestra.run.smithi151.stdout:min_compat_client luminous 2024-04-26T01:53:03.661 INFO:teuthology.orchestra.run.smithi151.stdout:require_osd_release squid 2024-04-26T01:53:03.661 INFO:teuthology.orchestra.run.smithi151.stdout:stretch_mode_enabled false 2024-04-26T01:53:03.661 INFO:teuthology.orchestra.run.smithi151.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-26T01:53:03.661 INFO:teuthology.orchestra.run.smithi151.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 90 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-26T01:53:03.661 INFO:teuthology.orchestra.run.smithi151.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 90 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-04-26T01:53:03.662 INFO:teuthology.orchestra.run.smithi151.stdout:max_osd 12 2024-04-26T01:53:03.662 INFO:teuthology.orchestra.run.smithi151.stdout:osd.0 up in weight 1 up_from 8 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.151:6802/669937998,v1:172.21.15.151:6803/669937998] [v2:172.21.15.151:6804/669937998,v1:172.21.15.151:6805/669937998] exists,up ca27e814-a982-482c-ae1a-e596aac5c59a 2024-04-26T01:53:03.662 INFO:teuthology.orchestra.run.smithi151.stdout:osd.1 up in weight 1 up_from 13 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.165:6800/1356168171,v1:172.21.15.165:6801/1356168171] [v2:172.21.15.165:6802/1356168171,v1:172.21.15.165:6803/1356168171] exists,up e7d192c3-9624-4625-9ccb-a76c114d8aad 2024-04-26T01:53:03.662 INFO:teuthology.orchestra.run.smithi151.stdout:osd.2 up in weight 1 up_from 18 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.172:6800/237502635,v1:172.21.15.172:6801/237502635] [v2:172.21.15.172:6802/237502635,v1:172.21.15.172:6803/237502635] exists,up 0631df59-1ecc-48b4-b00a-48b0118dfc24 2024-04-26T01:53:03.662 INFO:teuthology.orchestra.run.smithi151.stdout:osd.3 up in weight 1 up_from 27 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.151:6810/3442279476,v1:172.21.15.151:6811/3442279476] [v2:172.21.15.151:6812/3442279476,v1:172.21.15.151:6813/3442279476] exists,up 73076e49-2f8a-4a3d-99eb-7b50aff448aa 2024-04-26T01:53:03.662 INFO:teuthology.orchestra.run.smithi151.stdout:osd.4 up in weight 1 up_from 35 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.165:6808/4029980723,v1:172.21.15.165:6809/4029980723] [v2:172.21.15.165:6810/4029980723,v1:172.21.15.165:6811/4029980723] exists,up 0230d615-548c-43f0-b8aa-7130f4cbbc1f 2024-04-26T01:53:03.662 INFO:teuthology.orchestra.run.smithi151.stdout:osd.5 up in weight 1 up_from 40 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.172:6808/2216578687,v1:172.21.15.172:6809/2216578687] [v2:172.21.15.172:6810/2216578687,v1:172.21.15.172:6811/2216578687] exists,up 9739f44b-2605-47dd-82fb-d30034742bf3 2024-04-26T01:53:03.663 INFO:teuthology.orchestra.run.smithi151.stdout:osd.6 up in weight 1 up_from 48 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.151:6818/1723252315,v1:172.21.15.151:6819/1723252315] [v2:172.21.15.151:6820/1723252315,v1:172.21.15.151:6821/1723252315] exists,up 4290baaa-6160-49cb-a4e0-c96f070f2ef5 2024-04-26T01:53:03.663 INFO:teuthology.orchestra.run.smithi151.stdout:osd.7 up in weight 1 up_from 53 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.165:6816/1977803781,v1:172.21.15.165:6817/1977803781] [v2:172.21.15.165:6818/1977803781,v1:172.21.15.165:6819/1977803781] exists,up 9f5315f2-036c-4e2f-92ed-3321a9e3c860 2024-04-26T01:53:03.663 INFO:teuthology.orchestra.run.smithi151.stdout:osd.8 up in weight 1 up_from 61 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.172:6816/491199512,v1:172.21.15.172:6817/491199512] [v2:172.21.15.172:6818/491199512,v1:172.21.15.172:6819/491199512] exists,up 8ddbe36d-cea7-45ef-a3ca-044a0e64ff85 2024-04-26T01:53:03.663 INFO:teuthology.orchestra.run.smithi151.stdout:osd.9 up in weight 1 up_from 69 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.151:6826/1818923828,v1:172.21.15.151:6827/1818923828] [v2:172.21.15.151:6828/1818923828,v1:172.21.15.151:6829/1818923828] exists,up 40654ba0-1bd9-4b54-97ae-f1ed6c4785d3 2024-04-26T01:53:03.663 INFO:teuthology.orchestra.run.smithi151.stdout:osd.10 up in weight 1 up_from 74 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.165:6824/2632621809,v1:172.21.15.165:6825/2632621809] [v2:172.21.15.165:6826/2632621809,v1:172.21.15.165:6827/2632621809] exists,up 19e75a1a-7bd0-41d9-b0cc-0fa4d1475dda 2024-04-26T01:53:03.663 INFO:teuthology.orchestra.run.smithi151.stdout:osd.11 up in weight 1 up_from 82 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.172:6824/886604365,v1:172.21.15.172:6825/886604365] [v2:172.21.15.172:6826/886604365,v1:172.21.15.172:6827/886604365] exists,up aa84e5f5-93a3-4375-8544-a1175f0eb06e 2024-04-26T01:53:03.663 INFO:teuthology.orchestra.run.smithi151.stdout:pg_upmap_items 2.a [2,11] 2024-04-26T01:53:03.664 INFO:teuthology.orchestra.run.smithi151.stdout:pg_upmap_items 2.1d [8,11] 2024-04-26T01:53:03.664 INFO:teuthology.orchestra.run.smithi151.stdout:pg_upmap_items 3.1 [4,9] 2024-04-26T01:53:03.664 INFO:teuthology.orchestra.run.smithi151.stdout:pg_upmap_items 3.25 [5,9] 2024-04-26T01:53:03.664 INFO:teuthology.orchestra.run.smithi151.stdout:pg_upmap_items 3.27 [4,8] 2024-04-26T01:53:03.664 INFO:teuthology.orchestra.run.smithi151.stdout:pg_upmap_items 3.34 [4,9] 2024-04-26T01:53:03.664 INFO:teuthology.orchestra.run.smithi151.stdout:blocklist 172.21.15.151:6801/796314699 expires 2024-04-27T01:41:25.234322+0000 2024-04-26T01:53:03.665 INFO:teuthology.orchestra.run.smithi151.stdout:blocklist 172.21.15.151:6800/796314699 expires 2024-04-27T01:41:25.234322+0000 2024-04-26T01:53:03.665 INFO:teuthology.orchestra.run.smithi151.stdout:blocklist 172.21.15.151:0/1173525554 expires 2024-04-27T01:41:25.234322+0000 2024-04-26T01:53:03.665 INFO:teuthology.orchestra.run.smithi151.stdout:blocklist 172.21.15.151:0/1342496801 expires 2024-04-27T01:40:50.492554+0000 2024-04-26T01:53:03.665 INFO:teuthology.orchestra.run.smithi151.stdout:blocklist 172.21.15.151:0/11665926 expires 2024-04-27T01:40:50.492554+0000 2024-04-26T01:53:03.665 INFO:teuthology.orchestra.run.smithi151.stdout:blocklist 172.21.15.151:0/1765448952 expires 2024-04-27T01:40:50.492554+0000 2024-04-26T01:53:03.665 INFO:teuthology.orchestra.run.smithi151.stdout:blocklist 172.21.15.151:0/46007430 expires 2024-04-27T01:41:25.234322+0000 2024-04-26T01:53:03.665 INFO:teuthology.orchestra.run.smithi151.stdout:blocklist 172.21.15.151:6800/244725060 expires 2024-04-27T01:40:50.492554+0000 2024-04-26T01:53:03.666 INFO:teuthology.orchestra.run.smithi151.stdout:blocklist 172.21.15.151:0/3703862543 expires 2024-04-27T01:41:25.234322+0000 2024-04-26T01:53:03.666 INFO:teuthology.orchestra.run.smithi151.stdout:blocklist 172.21.15.151:6801/244725060 expires 2024-04-27T01:40:50.492554+0000 2024-04-26T01:53:03.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:03.663+0000 7f9defc00700 1 -- 172.21.15.151:0/4060736743 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f9dd00a3c40 msgr2=0x7f9dd00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:03.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:03.663+0000 7f9defc00700 1 --2- 172.21.15.151:0/4060736743 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f9dd00a3c40 0x7f9dd00a6100 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc00b2e0 tx=0x7f9ddc005fb0 comp rx=0 tx=0).stop 2024-04-26T01:53:03.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:03.663+0000 7f9defc00700 1 -- 172.21.15.151:0/4060736743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9de8076a70 msgr2=0x7f9de819b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:03.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:03.663+0000 7f9defc00700 1 --2- 172.21.15.151:0/4060736743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9de8076a70 0x7f9de819b3d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9dd400e9a0 tx=0x7f9dd400ecb0 comp rx=0 tx=0).stop 2024-04-26T01:53:03.668 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:03.663+0000 7f9defc00700 1 -- 172.21.15.151:0/4060736743 shutdown_connections 2024-04-26T01:53:03.668 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:03.663+0000 7f9defc00700 1 --2- 172.21.15.151:0/4060736743 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f9dd00a3c40 0x7f9dd00a6100 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:03.668 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:03.663+0000 7f9defc00700 1 --2- 172.21.15.151:0/4060736743 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9de810a980 0x7f9de81a2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:03.668 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:03.663+0000 7f9defc00700 1 --2- 172.21.15.151:0/4060736743 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9de80754a0 0x7f9de819ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:03.668 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:03.663+0000 7f9defc00700 1 --2- 172.21.15.151:0/4060736743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9de8076a70 0x7f9de819b3d0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:03.668 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:03.663+0000 7f9defc00700 1 -- 172.21.15.151:0/4060736743 >> 172.21.15.151:0/4060736743 conn(0x7f9de80fe010 msgr2=0x7f9de8106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:03.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:03.663+0000 7f9defc00700 1 -- 172.21.15.151:0/4060736743 shutdown_connections 2024-04-26T01:53:03.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:03.663+0000 7f9defc00700 1 -- 172.21.15.151:0/4060736743 wait complete. 2024-04-26T01:53:04.358 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:04 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:53:04.358 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:04 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/4060736743' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-26T01:53:04.358 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:04 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:53:04.358 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:04 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:53:04.358 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:04 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:53:04.358 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:04 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:53:04.412 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-26T01:53:04.423 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:53:04.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:04 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:53:04.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:04 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/4060736743' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-26T01:53:04.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:04 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:53:04.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:04 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:53:04.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:04 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:53:04.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:04 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:53:04.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:04 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:53:04.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:04 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/4060736743' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-26T01:53:04.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:04 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:53:04.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:04 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:53:04.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:04 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:53:04.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:04 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:53:04.869 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:53:05.364 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:05 smithi151 ceph-mon[28057]: pgmap v431: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:53:05.364 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:05 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:53:05.364 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:05 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:53:05.364 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:05 smithi151 ceph-mon[28057]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:53:05.437 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:05 smithi172 ceph-mon[32785]: pgmap v431: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:53:05.437 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:05 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:53:05.438 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:05 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:53:05.438 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:05 smithi172 ceph-mon[32785]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:53:05.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:05 smithi165 ceph-mon[32648]: pgmap v431: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:53:05.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:05 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:53:05.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:05 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:53:05.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:05 smithi165 ceph-mon[32648]: from='mgr.14152 172.21.15.151:0/118002720' entity='mgr.x' 2024-04-26T01:53:07.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:07.172+0000 7fce2d000700 1 -- 172.21.15.151:0/1791899636 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce281092d0 msgr2=0x7fce281096b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:07.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:07.172+0000 7fce2d000700 1 --2- 172.21.15.151:0/1791899636 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce281092d0 0x7fce281096b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fce180099c0 tx=0x7fce18009cd0 comp rx=0 tx=0).stop 2024-04-26T01:53:07.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:07.172+0000 7fce2d000700 1 -- 172.21.15.151:0/1791899636 shutdown_connections 2024-04-26T01:53:07.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:07.172+0000 7fce2d000700 1 --2- 172.21.15.151:0/1791899636 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fce2810e3b0 0x7fce28115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:07.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:07.172+0000 7fce2d000700 1 --2- 172.21.15.151:0/1791899636 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce281092d0 0x7fce281096b0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:07.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:07.172+0000 7fce2d000700 1 --2- 172.21.15.151:0/1791899636 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce28109c80 0x7fce2810dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:07.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:07.172+0000 7fce2d000700 1 -- 172.21.15.151:0/1791899636 >> 172.21.15.151:0/1791899636 conn(0x7fce28076210 msgr2=0x7fce28078630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:07.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:07.172+0000 7fce2d000700 1 -- 172.21.15.151:0/1791899636 shutdown_connections 2024-04-26T01:53:07.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:07.173+0000 7fce2d000700 1 -- 172.21.15.151:0/1791899636 wait complete. 2024-04-26T01:53:07.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:07.174+0000 7fce2d000700 1 Processor -- start 2024-04-26T01:53:07.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:07.174+0000 7fce2d000700 1 -- start start 2024-04-26T01:53:07.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:07.174+0000 7fce2d000700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fce281092d0 0x7fce281a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:07.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:07.174+0000 7fce2d000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce28109c80 0x7fce281a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:07.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:07.174+0000 7fce2d000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce2810e3b0 0x7fce2819f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:07.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:07.174+0000 7fce2d000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fce281192c0 con 0x7fce2810e3b0 2024-04-26T01:53:07.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:07.174+0000 7fce2d000700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fce28119140 con 0x7fce28109c80 2024-04-26T01:53:07.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:07.174+0000 7fce2d000700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fce28119440 con 0x7fce281092d0 2024-04-26T01:53:07.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:07.175+0000 7fce25e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fce281092d0 0x7fce281a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:07.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:07.175+0000 7fce1fe00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce28109c80 0x7fce281a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:07.174 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:07.175+0000 7fce25e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fce281092d0 0x7fce281a5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:35732/0 (socket says 172.21.15.151:35732) 2024-04-26T01:53:07.174 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:07.175+0000 7fce25e00700 1 -- 172.21.15.151:0/3294512206 learned_addr learned my addr 172.21.15.151:0/3294512206 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:53:07.174 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:07.175+0000 7fce26800700 1 --2- 172.21.15.151:0/3294512206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce2810e3b0 0x7fce2819f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:07.174 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:07.175+0000 7fce1fe00700 1 -- 172.21.15.151:0/3294512206 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fce281092d0 msgr2=0x7fce281a5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:07.174 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:07.175+0000 7fce1fe00700 1 --2- 172.21.15.151:0/3294512206 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fce281092d0 0x7fce281a5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:07.174 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:07.175+0000 7fce1fe00700 1 -- 172.21.15.151:0/3294512206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce2810e3b0 msgr2=0x7fce2819f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:07.174 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:07.175+0000 7fce1fe00700 1 --2- 172.21.15.151:0/3294512206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce2810e3b0 0x7fce2819f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:07.174 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:07.175+0000 7fce1fe00700 1 -- 172.21.15.151:0/3294512206 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce2819fd60 con 0x7fce28109c80 2024-04-26T01:53:07.174 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:07.175+0000 7fce26800700 1 --2- 172.21.15.151:0/3294512206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce2810e3b0 0x7fce2819f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:53:07.174 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:07.175+0000 7fce1fe00700 1 --2- 172.21.15.151:0/3294512206 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce28109c80 0x7fce281a5980 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fce1000e9c0 tx=0x7fce1000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:07.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:07.176+0000 7fce1ea00700 1 -- 172.21.15.151:0/3294512206 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce1000cbd0 con 0x7fce28109c80 2024-04-26T01:53:07.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:07.176+0000 7fce1ea00700 1 -- 172.21.15.151:0/3294512206 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce1000cd70 con 0x7fce28109c80 2024-04-26T01:53:07.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:07.176+0000 7fce1ea00700 1 -- 172.21.15.151:0/3294512206 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce10018680 con 0x7fce28109c80 2024-04-26T01:53:07.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:07.176+0000 7fce2d000700 1 -- 172.21.15.151:0/3294512206 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce281a0050 con 0x7fce28109c80 2024-04-26T01:53:07.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:07.176+0000 7fce2d000700 1 -- 172.21.15.151:0/3294512206 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce281ac310 con 0x7fce28109c80 2024-04-26T01:53:07.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:07.177+0000 7fce2d000700 1 -- 172.21.15.151:0/3294512206 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce28069730 con 0x7fce28109c80 2024-04-26T01:53:07.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:07.179+0000 7fce1ea00700 1 -- 172.21.15.151:0/3294512206 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7fce10010430 con 0x7fce28109c80 2024-04-26T01:53:07.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:07.181+0000 7fce1ea00700 1 --2- 172.21.15.151:0/3294512206 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fce0c0a3c40 0x7fce0c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:07.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:07.181+0000 7fce1ea00700 1 -- 172.21.15.151:0/3294512206 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7fce100d04e0 con 0x7fce28109c80 2024-04-26T01:53:07.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:07.181+0000 7fce25e00700 1 --2- 172.21.15.151:0/3294512206 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fce0c0a3c40 0x7fce0c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:07.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:07.182+0000 7fce25e00700 1 --2- 172.21.15.151:0/3294512206 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fce0c0a3c40 0x7fce0c0a6100 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fce18009990 tx=0x7fce18005ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:07.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:07.186+0000 7fce1ea00700 1 -- 172.21.15.151:0/3294512206 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fce1009cf60 con 0x7fce28109c80 2024-04-26T01:53:07.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:07 smithi151 ceph-mon[28057]: pgmap v432: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:53:07.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:07.462+0000 7fce2d000700 1 -- 172.21.15.151:0/3294512206 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fce2802df80 con 0x7fce28109c80 2024-04-26T01:53:07.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:07.463+0000 7fce1ea00700 1 -- 172.21.15.151:0/3294512206 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11889 (secure 0 0 0) 0x7fce1009c860 con 0x7fce28109c80 2024-04-26T01:53:07.462 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:53:07.462 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2937864097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2937864097},{"type":"v1","addr":"172.21.15.151:6835","nonce":2937864097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14373,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/2947090802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2947090802},{"type":"v1","addr":"172.21.15.151:6837","nonce":2947090802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/1356563280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1356563280},{"type":"v1","addr":"172.21.15.151:6839","nonce":1356563280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6841/1075240552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1075240552},{"type":"v1","addr":"172.21.15.151:6841","nonce":1075240552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/504675912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":504675912},{"type":"v1","addr":"172.21.15.165:6833","nonce":504675912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1207580015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1207580015},{"type":"v1","addr":"172.21.15.172:6833","nonce":1207580015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:49:40.746876+0000","modified":"2024-04-26T01:50:05.608353+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24505,"mds_2":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/3598138294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3598138294},{"type":"v1","addr":"172.21.15.165:6835","nonce":3598138294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/1847859054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":1847859054},{"type":"v1","addr":"172.21.15.165:6837","nonce":1847859054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/2309751171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2309751171},{"type":"v1","addr":"172.21.15.172:6835","nonce":2309751171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.172:6837/55972537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":55972537},{"type":"v1","addr":"172.21.15.172:6837","nonce":55972537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.172:6839/1242302810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1242302810},{"type":"v1","addr":"172.21.15.172:6839","nonce":1242302810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/1969716254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1969716254},{"type":"v1","addr":"172.21.15.165:6839","nonce":1969716254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24505,24509,24493]},"id":1}]} 2024-04-26T01:53:07.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:07.466+0000 7fce2d000700 1 -- 172.21.15.151:0/3294512206 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fce0c0a3c40 msgr2=0x7fce0c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:07.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:07.466+0000 7fce2d000700 1 --2- 172.21.15.151:0/3294512206 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fce0c0a3c40 0x7fce0c0a6100 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fce18009990 tx=0x7fce18005ab0 comp rx=0 tx=0).stop 2024-04-26T01:53:07.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:07.466+0000 7fce2d000700 1 -- 172.21.15.151:0/3294512206 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce28109c80 msgr2=0x7fce281a5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:07.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:07.466+0000 7fce2d000700 1 --2- 172.21.15.151:0/3294512206 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce28109c80 0x7fce281a5980 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fce1000e9c0 tx=0x7fce1000ecd0 comp rx=0 tx=0).stop 2024-04-26T01:53:07.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:07.466+0000 7fce2d000700 1 -- 172.21.15.151:0/3294512206 shutdown_connections 2024-04-26T01:53:07.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:07.466+0000 7fce2d000700 1 --2- 172.21.15.151:0/3294512206 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fce0c0a3c40 0x7fce0c0a6100 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:07.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:07.466+0000 7fce2d000700 1 --2- 172.21.15.151:0/3294512206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce2810e3b0 0x7fce2819f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:07.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:07.466+0000 7fce2d000700 1 --2- 172.21.15.151:0/3294512206 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fce281092d0 0x7fce281a5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:07.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:07.466+0000 7fce2d000700 1 --2- 172.21.15.151:0/3294512206 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce28109c80 0x7fce281a5980 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:07.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:07.466+0000 7fce2d000700 1 -- 172.21.15.151:0/3294512206 >> 172.21.15.151:0/3294512206 conn(0x7fce28076210 msgr2=0x7fce28078570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:07.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:07.466+0000 7fce2d000700 1 -- 172.21.15.151:0/3294512206 shutdown_connections 2024-04-26T01:53:07.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:07.466+0000 7fce2d000700 1 -- 172.21.15.151:0/3294512206 wait complete. 2024-04-26T01:53:07.466 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 25 2024-04-26T01:53:07.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:07 smithi172 ceph-mon[32785]: pgmap v432: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:53:07.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:07 smithi165 ceph-mon[32648]: pgmap v432: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:53:08.293 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:53:08.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:08 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/3294512206' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:53:08.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:08 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/3294512206' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:53:08.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:08 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/3294512206' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:53:08.734 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:53:09.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:09 smithi172 ceph-mon[32785]: pgmap v433: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T01:53:09.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:09 smithi165 ceph-mon[32648]: pgmap v433: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T01:53:09.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:09 smithi151 ceph-mon[28057]: pgmap v433: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T01:53:10.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:10.873+0000 7f7cfea00700 1 -- 172.21.15.151:0/2445623464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7cf8074de0 msgr2=0x7f7cf80751c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:10.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:10.873+0000 7f7cfea00700 1 --2- 172.21.15.151:0/2445623464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7cf8074de0 0x7f7cf80751c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7ce8009990 tx=0x7f7ce8009ca0 comp rx=0 tx=0).stop 2024-04-26T01:53:10.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:10.873+0000 7f7cfea00700 1 -- 172.21.15.151:0/2445623464 shutdown_connections 2024-04-26T01:53:10.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:10.873+0000 7f7cfea00700 1 --2- 172.21.15.151:0/2445623464 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7cf8113960 0x7f7cf8115d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:10.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:10.873+0000 7f7cfea00700 1 --2- 172.21.15.151:0/2445623464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7cf8074de0 0x7f7cf80751c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:10.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:10.873+0000 7f7cfea00700 1 --2- 172.21.15.151:0/2445623464 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7cf8075700 0x7f7cf8113420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:10.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:10.873+0000 7f7cfea00700 1 -- 172.21.15.151:0/2445623464 >> 172.21.15.151:0/2445623464 conn(0x7f7cf80fe460 msgr2=0x7f7cf8100880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:10.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:10.873+0000 7f7cfea00700 1 -- 172.21.15.151:0/2445623464 shutdown_connections 2024-04-26T01:53:10.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:10.873+0000 7f7cfea00700 1 -- 172.21.15.151:0/2445623464 wait complete. 2024-04-26T01:53:10.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:10.874+0000 7f7cfea00700 1 Processor -- start 2024-04-26T01:53:10.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:10.874+0000 7f7cfea00700 1 -- start start 2024-04-26T01:53:10.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:10.875+0000 7f7cfea00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7cf8074de0 0x7f7cf819f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:10.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:10.875+0000 7f7cfea00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7cf8075700 0x7f7cf819fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:10.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:10.875+0000 7f7cfea00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7cf8113960 0x7f7cf81a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:10.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:10.875+0000 7f7cfea00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf8119180 con 0x7f7cf8075700 2024-04-26T01:53:10.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:10.875+0000 7f7cfea00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf8119000 con 0x7f7cf8074de0 2024-04-26T01:53:10.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:10.875+0000 7f7cfea00700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf8119300 con 0x7f7cf8113960 2024-04-26T01:53:10.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:10.875+0000 7f7cfd600700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7cf8074de0 0x7f7cf819f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:10.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:10.875+0000 7f7cfd600700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7cf8074de0 0x7f7cf819f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:36768/0 (socket says 172.21.15.151:36768) 2024-04-26T01:53:10.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:10.875+0000 7f7cfd600700 1 -- 172.21.15.151:0/2993094190 learned_addr learned my addr 172.21.15.151:0/2993094190 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:53:10.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:10.875+0000 7f7cfcc00700 1 --2- 172.21.15.151:0/2993094190 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7cf8075700 0x7f7cf819fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:10.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:10.875+0000 7f7cfe000700 1 --2- 172.21.15.151:0/2993094190 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7cf8113960 0x7f7cf81a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:10.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:10.876+0000 7f7cfcc00700 1 -- 172.21.15.151:0/2993094190 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7cf8113960 msgr2=0x7f7cf81a3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:10.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:10.876+0000 7f7cfcc00700 1 --2- 172.21.15.151:0/2993094190 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7cf8113960 0x7f7cf81a3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:10.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:10.876+0000 7f7cfcc00700 1 -- 172.21.15.151:0/2993094190 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7cf8074de0 msgr2=0x7f7cf819f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:10.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:10.876+0000 7f7cfcc00700 1 --2- 172.21.15.151:0/2993094190 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7cf8074de0 0x7f7cf819f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:10.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:10.876+0000 7f7cfcc00700 1 -- 172.21.15.151:0/2993094190 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cf81a46b0 con 0x7f7cf8075700 2024-04-26T01:53:10.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:10.876+0000 7f7cfd600700 1 --2- 172.21.15.151:0/2993094190 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7cf8074de0 0x7f7cf819f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:53:10.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:10.876+0000 7f7cfcc00700 1 --2- 172.21.15.151:0/2993094190 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7cf8075700 0x7f7cf819fc30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7ce400e9f0 tx=0x7f7ce400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:10.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:10.876+0000 7f7cf6000700 1 -- 172.21.15.151:0/2993094190 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce400c9c0 con 0x7f7cf8075700 2024-04-26T01:53:10.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:10.876+0000 7f7cf6000700 1 -- 172.21.15.151:0/2993094190 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ce400cb60 con 0x7f7cf8075700 2024-04-26T01:53:10.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:10.876+0000 7f7cfea00700 1 -- 172.21.15.151:0/2993094190 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cf81a49a0 con 0x7f7cf8075700 2024-04-26T01:53:10.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:10.876+0000 7f7cf6000700 1 -- 172.21.15.151:0/2993094190 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce4018680 con 0x7f7cf8075700 2024-04-26T01:53:10.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:10.877+0000 7f7cfea00700 1 -- 172.21.15.151:0/2993094190 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cf81ac220 con 0x7f7cf8075700 2024-04-26T01:53:10.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:10.878+0000 7f7cfea00700 1 -- 172.21.15.151:0/2993094190 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cf8069730 con 0x7f7cf8075700 2024-04-26T01:53:10.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:10.880+0000 7f7cf6000700 1 -- 172.21.15.151:0/2993094190 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f7ce4010430 con 0x7f7cf8075700 2024-04-26T01:53:10.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:10.881+0000 7f7cf6000700 1 --2- 172.21.15.151:0/2993094190 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f7cdc0a3cc0 0x7f7cdc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:10.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:10.881+0000 7f7cf6000700 1 -- 172.21.15.151:0/2993094190 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f7ce40d0800 con 0x7f7cf8075700 2024-04-26T01:53:10.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:10.882+0000 7f7cfd600700 1 --2- 172.21.15.151:0/2993094190 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f7cdc0a3cc0 0x7f7cdc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:10.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:10.882+0000 7f7cfd600700 1 --2- 172.21.15.151:0/2993094190 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f7cdc0a3cc0 0x7f7cdc0a6180 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7ce8006010 tx=0x7f7ce800b280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:10.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:10.884+0000 7f7cf6000700 1 -- 172.21.15.151:0/2993094190 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7ce409d1e0 con 0x7f7cf8075700 2024-04-26T01:53:11.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:11.161+0000 7f7cfea00700 1 -- 172.21.15.151:0/2993094190 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7cf81a0a00 con 0x7f7cf8075700 2024-04-26T01:53:11.161 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:11.162+0000 7f7cf6000700 1 -- 172.21.15.151:0/2993094190 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11889 (secure 0 0 0) 0x7f7ce409cae0 con 0x7f7cf8075700 2024-04-26T01:53:11.161 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:53:11.161 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2937864097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2937864097},{"type":"v1","addr":"172.21.15.151:6835","nonce":2937864097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14373,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/2947090802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2947090802},{"type":"v1","addr":"172.21.15.151:6837","nonce":2947090802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/1356563280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1356563280},{"type":"v1","addr":"172.21.15.151:6839","nonce":1356563280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6841/1075240552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1075240552},{"type":"v1","addr":"172.21.15.151:6841","nonce":1075240552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/504675912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":504675912},{"type":"v1","addr":"172.21.15.165:6833","nonce":504675912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1207580015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1207580015},{"type":"v1","addr":"172.21.15.172:6833","nonce":1207580015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:49:40.746876+0000","modified":"2024-04-26T01:50:05.608353+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24505,"mds_2":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/3598138294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3598138294},{"type":"v1","addr":"172.21.15.165:6835","nonce":3598138294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/1847859054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":1847859054},{"type":"v1","addr":"172.21.15.165:6837","nonce":1847859054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/2309751171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2309751171},{"type":"v1","addr":"172.21.15.172:6835","nonce":2309751171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.172:6837/55972537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":55972537},{"type":"v1","addr":"172.21.15.172:6837","nonce":55972537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.172:6839/1242302810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1242302810},{"type":"v1","addr":"172.21.15.172:6839","nonce":1242302810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/1969716254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1969716254},{"type":"v1","addr":"172.21.15.165:6839","nonce":1969716254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24505,24509,24493]},"id":1}]} 2024-04-26T01:53:11.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:11.165+0000 7f7cfea00700 1 -- 172.21.15.151:0/2993094190 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f7cdc0a3cc0 msgr2=0x7f7cdc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:11.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:11.165+0000 7f7cfea00700 1 --2- 172.21.15.151:0/2993094190 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f7cdc0a3cc0 0x7f7cdc0a6180 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7ce8006010 tx=0x7f7ce800b280 comp rx=0 tx=0).stop 2024-04-26T01:53:11.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:11.165+0000 7f7cfea00700 1 -- 172.21.15.151:0/2993094190 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7cf8075700 msgr2=0x7f7cf819fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:11.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:11.165+0000 7f7cfea00700 1 --2- 172.21.15.151:0/2993094190 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7cf8075700 0x7f7cf819fc30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7ce400e9f0 tx=0x7f7ce400ed00 comp rx=0 tx=0).stop 2024-04-26T01:53:11.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:11.165+0000 7f7cfea00700 1 -- 172.21.15.151:0/2993094190 shutdown_connections 2024-04-26T01:53:11.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:11.165+0000 7f7cfea00700 1 --2- 172.21.15.151:0/2993094190 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7cf8113960 0x7f7cf81a3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:11.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:11.165+0000 7f7cfea00700 1 --2- 172.21.15.151:0/2993094190 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7cf8074de0 0x7f7cf819f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:11.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:11.165+0000 7f7cfea00700 1 --2- 172.21.15.151:0/2993094190 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f7cdc0a3cc0 0x7f7cdc0a6180 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:11.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:11.165+0000 7f7cfea00700 1 --2- 172.21.15.151:0/2993094190 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7cf8075700 0x7f7cf819fc30 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:11.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:11.165+0000 7f7cfea00700 1 -- 172.21.15.151:0/2993094190 >> 172.21.15.151:0/2993094190 conn(0x7f7cf80fe460 msgr2=0x7f7cf80ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:11.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:11.166+0000 7f7cfea00700 1 -- 172.21.15.151:0/2993094190 shutdown_connections 2024-04-26T01:53:11.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:11.166+0000 7f7cfea00700 1 -- 172.21.15.151:0/2993094190 wait complete. 2024-04-26T01:53:11.166 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 25 2024-04-26T01:53:11.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:11 smithi172 ceph-mon[32785]: pgmap v434: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:53:11.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:11 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/2993094190' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:53:11.576 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:11 smithi151 ceph-mon[28057]: pgmap v434: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:53:11.576 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:11 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/2993094190' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:53:11.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:11 smithi165 ceph-mon[32648]: pgmap v434: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:53:11.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:11 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/2993094190' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:53:11.968 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T01:53:12.416 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:53:13.494 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:13 smithi151 ceph-mon[28057]: pgmap v435: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:53:13.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:13 smithi172 ceph-mon[32785]: pgmap v435: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:53:13.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:13 smithi165 ceph-mon[32648]: pgmap v435: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:53:14.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:14.381+0000 7f11e6a00700 1 -- 172.21.15.151:0/832371006 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11e0105870 msgr2=0x7f11e01098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:14.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:14.381+0000 7f11e6a00700 1 --2- 172.21.15.151:0/832371006 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11e0105870 0x7f11e01098c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f11c80099c0 tx=0x7f11c8009cd0 comp rx=0 tx=0).stop 2024-04-26T01:53:14.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:14.381+0000 7f11e6a00700 1 -- 172.21.15.151:0/832371006 shutdown_connections 2024-04-26T01:53:14.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:14.381+0000 7f11e6a00700 1 --2- 172.21.15.151:0/832371006 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11e0109fa0 0x7f11e0111b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:14.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:14.381+0000 7f11e6a00700 1 --2- 172.21.15.151:0/832371006 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11e0104ec0 0x7f11e01052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:14.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:14.381+0000 7f11e6a00700 1 --2- 172.21.15.151:0/832371006 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11e0105870 0x7f11e01098c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:14.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:14.381+0000 7f11e6a00700 1 -- 172.21.15.151:0/832371006 >> 172.21.15.151:0/832371006 conn(0x7f11e0100750 msgr2=0x7f11e0102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:14.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:14.382+0000 7f11e6a00700 1 -- 172.21.15.151:0/832371006 shutdown_connections 2024-04-26T01:53:14.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:14.382+0000 7f11e6a00700 1 -- 172.21.15.151:0/832371006 wait complete. 2024-04-26T01:53:14.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:14.383+0000 7f11e6a00700 1 Processor -- start 2024-04-26T01:53:14.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:14.383+0000 7f11e6a00700 1 -- start start 2024-04-26T01:53:14.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:14.383+0000 7f11e6a00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11e0104ec0 0x7f11e019b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:14.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:14.383+0000 7f11e6a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11e0105870 0x7f11e019b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:14.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:14.383+0000 7f11e6a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11e0109fa0 0x7f11e019fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:14.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:14.383+0000 7f11e6a00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f11e0114e80 con 0x7f11e0105870 2024-04-26T01:53:14.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:14.383+0000 7f11e6a00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f11e0114d00 con 0x7f11e0109fa0 2024-04-26T01:53:14.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:14.383+0000 7f11e6a00700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f11e0115000 con 0x7f11e0104ec0 2024-04-26T01:53:14.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:14.384+0000 7f11dfe00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11e0104ec0 0x7f11e019b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:14.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:14.384+0000 7f11dfe00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11e0104ec0 0x7f11e019b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:46562/0 (socket says 172.21.15.151:46562) 2024-04-26T01:53:14.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:14.384+0000 7f11dfe00700 1 -- 172.21.15.151:0/2998761897 learned_addr learned my addr 172.21.15.151:0/2998761897 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:53:14.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:14.384+0000 7f11e4a00700 1 --2- 172.21.15.151:0/2998761897 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11e0109fa0 0x7f11e019fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:14.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:14.384+0000 7f11d7e00700 1 --2- 172.21.15.151:0/2998761897 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11e0105870 0x7f11e019b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:14.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:14.384+0000 7f11dfe00700 1 -- 172.21.15.151:0/2998761897 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11e0109fa0 msgr2=0x7f11e019fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:14.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:14.384+0000 7f11dfe00700 1 --2- 172.21.15.151:0/2998761897 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11e0109fa0 0x7f11e019fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:14.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:14.384+0000 7f11dfe00700 1 -- 172.21.15.151:0/2998761897 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11e0105870 msgr2=0x7f11e019b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:14.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:14.384+0000 7f11dfe00700 1 --2- 172.21.15.151:0/2998761897 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11e0105870 0x7f11e019b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:14.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:14.384+0000 7f11dfe00700 1 -- 172.21.15.151:0/2998761897 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11e01a0360 con 0x7f11e0104ec0 2024-04-26T01:53:14.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:14.385+0000 7f11dfe00700 1 --2- 172.21.15.151:0/2998761897 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11e0104ec0 0x7f11e019b350 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f11d000e9a0 tx=0x7f11d000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:14.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:14.385+0000 7f11dd600700 1 -- 172.21.15.151:0/2998761897 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d000ca10 con 0x7f11e0104ec0 2024-04-26T01:53:14.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:14.385+0000 7f11e6a00700 1 -- 172.21.15.151:0/2998761897 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11e01a0650 con 0x7f11e0104ec0 2024-04-26T01:53:14.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:14.385+0000 7f11e6a00700 1 -- 172.21.15.151:0/2998761897 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11e01a7ed0 con 0x7f11e0104ec0 2024-04-26T01:53:14.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:14.385+0000 7f11dd600700 1 -- 172.21.15.151:0/2998761897 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11d000cbb0 con 0x7f11e0104ec0 2024-04-26T01:53:14.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:14.385+0000 7f11dd600700 1 -- 172.21.15.151:0/2998761897 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d0018420 con 0x7f11e0104ec0 2024-04-26T01:53:14.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:14.386+0000 7f11e6a00700 1 -- 172.21.15.151:0/2998761897 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11e0069730 con 0x7f11e0104ec0 2024-04-26T01:53:14.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:14.388+0000 7f11dd600700 1 -- 172.21.15.151:0/2998761897 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f11d0010430 con 0x7f11e0104ec0 2024-04-26T01:53:14.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:14.389+0000 7f11dd600700 1 --2- 172.21.15.151:0/2998761897 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f11c40a3cc0 0x7f11c40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:14.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:14.389+0000 7f11dd600700 1 -- 172.21.15.151:0/2998761897 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f11d00d0620 con 0x7f11e0104ec0 2024-04-26T01:53:14.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:14.390+0000 7f11d7e00700 1 --2- 172.21.15.151:0/2998761897 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f11c40a3cc0 0x7f11c40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:14.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:14.390+0000 7f11d7e00700 1 --2- 172.21.15.151:0/2998761897 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f11c40a3cc0 0x7f11c40a6180 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f11c8000c00 tx=0x7f11c8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:14.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:14.394+0000 7f11dd600700 1 -- 172.21.15.151:0/2998761897 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f11d000cd60 con 0x7f11e0104ec0 2024-04-26T01:53:14.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:14.623+0000 7f11e6a00700 1 -- 172.21.15.151:0/2998761897 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f11e002df80 con 0x7f11e0104ec0 2024-04-26T01:53:14.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:14.625+0000 7f11dd600700 1 -- 172.21.15.151:0/2998761897 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+22599 (secure 0 0 0) 0x7f11d009d000 con 0x7f11e0104ec0 2024-04-26T01:53:14.625 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:53:14.625 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":92,"fsid":"c0e49e64-036d-11ef-bc93-c7b262605968","created":"2024-04-26T01:40:07.483197+0000","modified":"2024-04-26T01:50:26.985192+0000","last_up_change":"2024-04-26T01:48:21.167691+0000","last_in_change":"2024-04-26T01:48:00.556990+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T01:44:11.411722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T01:49:31.756032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T01:49:36.529819+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ca27e814-a982-482c-ae1a-e596aac5c59a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":669937998},{"type":"v1","addr":"172.21.15.151:6803","nonce":669937998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":669937998},{"type":"v1","addr":"172.21.15.151:6805","nonce":669937998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":669937998},{"type":"v1","addr":"172.21.15.151:6809","nonce":669937998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":669937998},{"type":"v1","addr":"172.21.15.151:6807","nonce":669937998}]},"public_addr":"172.21.15.151:6803/669937998","cluster_addr":"172.21.15.151:6805/669937998","heartbeat_back_addr":"172.21.15.151:6809/669937998","heartbeat_front_addr":"172.21.15.151:6807/669937998","state":["exists","up"]},{"osd":1,"uuid":"e7d192c3-9624-4625-9ccb-a76c114d8aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":1356168171},{"type":"v1","addr":"172.21.15.165:6801","nonce":1356168171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":1356168171},{"type":"v1","addr":"172.21.15.165:6803","nonce":1356168171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":1356168171},{"type":"v1","addr":"172.21.15.165:6807","nonce":1356168171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":1356168171},{"type":"v1","addr":"172.21.15.165:6805","nonce":1356168171}]},"public_addr":"172.21.15.165:6801/1356168171","cluster_addr":"172.21.15.165:6803/1356168171","heartbeat_back_addr":"172.21.15.165:6807/1356168171","heartbeat_front_addr":"172.21.15.165:6805/1356168171","state":["exists","up"]},{"osd":2,"uuid":"0631df59-1ecc-48b4-b00a-48b0118dfc24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":237502635},{"type":"v1","addr":"172.21.15.172:6801","nonce":237502635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":237502635},{"type":"v1","addr":"172.21.15.172:6803","nonce":237502635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":237502635},{"type":"v1","addr":"172.21.15.172:6807","nonce":237502635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":237502635},{"type":"v1","addr":"172.21.15.172:6805","nonce":237502635}]},"public_addr":"172.21.15.172:6801/237502635","cluster_addr":"172.21.15.172:6803/237502635","heartbeat_back_addr":"172.21.15.172:6807/237502635","heartbeat_front_addr":"172.21.15.172:6805/237502635","state":["exists","up"]},{"osd":3,"uuid":"73076e49-2f8a-4a3d-99eb-7b50aff448aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3442279476},{"type":"v1","addr":"172.21.15.151:6811","nonce":3442279476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3442279476},{"type":"v1","addr":"172.21.15.151:6813","nonce":3442279476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":3442279476},{"type":"v1","addr":"172.21.15.151:6817","nonce":3442279476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3442279476},{"type":"v1","addr":"172.21.15.151:6815","nonce":3442279476}]},"public_addr":"172.21.15.151:6811/3442279476","cluster_addr":"172.21.15.151:6813/3442279476","heartbeat_back_addr":"172.21.15.151:6817/3442279476","heartbeat_front_addr":"172.21.15.151:6815/3442279476","state":["exists","up"]},{"osd":4,"uuid":"0230d615-548c-43f0-b8aa-7130f4cbbc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4029980723},{"type":"v1","addr":"172.21.15.165:6809","nonce":4029980723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4029980723},{"type":"v1","addr":"172.21.15.165:6811","nonce":4029980723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4029980723},{"type":"v1","addr":"172.21.15.165:6815","nonce":4029980723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4029980723},{"type":"v1","addr":"172.21.15.165:6813","nonce":4029980723}]},"public_addr":"172.21.15.165:6809/4029980723","cluster_addr":"172.21.15.165:6811/4029980723","heartbeat_back_addr":"172.21.15.165:6815/4029980723","heartbeat_front_addr":"172.21.15.165:6813/4029980723","state":["exists","up"]},{"osd":5,"uuid":"9739f44b-2605-47dd-82fb-d30034742bf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2216578687},{"type":"v1","addr":"172.21.15.172:6809","nonce":2216578687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2216578687},{"type":"v1","addr":"172.21.15.172:6811","nonce":2216578687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2216578687},{"type":"v1","addr":"172.21.15.172:6815","nonce":2216578687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2216578687},{"type":"v1","addr":"172.21.15.172:6813","nonce":2216578687}]},"public_addr":"172.21.15.172:6809/2216578687","cluster_addr":"172.21.15.172:6811/2216578687","heartbeat_back_addr":"172.21.15.172:6815/2216578687","heartbeat_front_addr":"172.21.15.172:6813/2216578687","state":["exists","up"]},{"osd":6,"uuid":"4290baaa-6160-49cb-a4e0-c96f070f2ef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1723252315},{"type":"v1","addr":"172.21.15.151:6819","nonce":1723252315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1723252315},{"type":"v1","addr":"172.21.15.151:6821","nonce":1723252315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":1723252315},{"type":"v1","addr":"172.21.15.151:6825","nonce":1723252315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1723252315},{"type":"v1","addr":"172.21.15.151:6823","nonce":1723252315}]},"public_addr":"172.21.15.151:6819/1723252315","cluster_addr":"172.21.15.151:6821/1723252315","heartbeat_back_addr":"172.21.15.151:6825/1723252315","heartbeat_front_addr":"172.21.15.151:6823/1723252315","state":["exists","up"]},{"osd":7,"uuid":"9f5315f2-036c-4e2f-92ed-3321a9e3c860","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":1977803781},{"type":"v1","addr":"172.21.15.165:6817","nonce":1977803781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":1977803781},{"type":"v1","addr":"172.21.15.165:6819","nonce":1977803781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":1977803781},{"type":"v1","addr":"172.21.15.165:6823","nonce":1977803781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":1977803781},{"type":"v1","addr":"172.21.15.165:6821","nonce":1977803781}]},"public_addr":"172.21.15.165:6817/1977803781","cluster_addr":"172.21.15.165:6819/1977803781","heartbeat_back_addr":"172.21.15.165:6823/1977803781","heartbeat_front_addr":"172.21.15.165:6821/1977803781","state":["exists","up"]},{"osd":8,"uuid":"8ddbe36d-cea7-45ef-a3ca-044a0e64ff85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":491199512},{"type":"v1","addr":"172.21.15.172:6817","nonce":491199512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":491199512},{"type":"v1","addr":"172.21.15.172:6819","nonce":491199512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":491199512},{"type":"v1","addr":"172.21.15.172:6823","nonce":491199512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":491199512},{"type":"v1","addr":"172.21.15.172:6821","nonce":491199512}]},"public_addr":"172.21.15.172:6817/491199512","cluster_addr":"172.21.15.172:6819/491199512","heartbeat_back_addr":"172.21.15.172:6823/491199512","heartbeat_front_addr":"172.21.15.172:6821/491199512","state":["exists","up"]},{"osd":9,"uuid":"40654ba0-1bd9-4b54-97ae-f1ed6c4785d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":1818923828},{"type":"v1","addr":"172.21.15.151:6827","nonce":1818923828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":1818923828},{"type":"v1","addr":"172.21.15.151:6829","nonce":1818923828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1818923828},{"type":"v1","addr":"172.21.15.151:6833","nonce":1818923828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":1818923828},{"type":"v1","addr":"172.21.15.151:6831","nonce":1818923828}]},"public_addr":"172.21.15.151:6827/1818923828","cluster_addr":"172.21.15.151:6829/1818923828","heartbeat_back_addr":"172.21.15.151:6833/1818923828","heartbeat_front_addr":"172.21.15.151:6831/1818923828","state":["exists","up"]},{"osd":10,"uuid":"19e75a1a-7bd0-41d9-b0cc-0fa4d1475dda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":2632621809},{"type":"v1","addr":"172.21.15.165:6825","nonce":2632621809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":2632621809},{"type":"v1","addr":"172.21.15.165:6827","nonce":2632621809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":2632621809},{"type":"v1","addr":"172.21.15.165:6831","nonce":2632621809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":2632621809},{"type":"v1","addr":"172.21.15.165:6829","nonce":2632621809}]},"public_addr":"172.21.15.165:6825/2632621809","cluster_addr":"172.21.15.165:6827/2632621809","heartbeat_back_addr":"172.21.15.165:6831/2632621809","heartbeat_front_addr":"172.21.15.165:6829/2632621809","state":["exists","up"]},{"osd":11,"uuid":"aa84e5f5-93a3-4375-8544-a1175f0eb06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":886604365},{"type":"v1","addr":"172.21.15.172:6825","nonce":886604365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":886604365},{"type":"v1","addr":"172.21.15.172:6827","nonce":886604365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":886604365},{"type":"v1","addr":"172.21.15.172:6831","nonce":886604365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":886604365},{"type":"v1","addr":"172.21.15.172:6829","nonce":886604365}]},"public_addr":"172.21.15.172:6825/886604365","cluster_addr":"172.21.15.172:6827/886604365","heartbeat_back_addr":"172.21.15.172:6831/886604365","heartbeat_front_addr":"172.21.15.172:6829/886604365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:43:18.296278+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:43:42.860620+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:44:07.073601+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:44:39.631760+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:45:04.771214+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:45:29.606674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:46:03.101851+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:46:28.351995+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:46:53.445435+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:47:27.391179+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:47:53.543638+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:48:18.687811+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.151:6801/796314699":"2024-04-27T01:41:25.234322+0000","172.21.15.151:6800/796314699":"2024-04-27T01:41:25.234322+0000","172.21.15.151:0/1173525554":"2024-04-27T01:41:25.234322+0000","172.21.15.151:0/1342496801":"2024-04-27T01:40:50.492554+0000","172.21.15.151:0/11665926":"2024-04-27T01:40:50.492554+0000","172.21.15.151:0/1765448952":"2024-04-27T01:40:50.492554+0000","172.21.15.151:0/46007430":"2024-04-27T01:41:25.234322+0000","172.21.15.151:6800/244725060":"2024-04-27T01:40:50.492554+0000","172.21.15.151:0/3703862543":"2024-04-27T01:41:25.234322+0000","172.21.15.151:6801/244725060":"2024-04-27T01:40:50.492554+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T01:53:14.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:14.628+0000 7f11e6a00700 1 -- 172.21.15.151:0/2998761897 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f11c40a3cc0 msgr2=0x7f11c40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:14.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:14.628+0000 7f11e6a00700 1 --2- 172.21.15.151:0/2998761897 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f11c40a3cc0 0x7f11c40a6180 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f11c8000c00 tx=0x7f11c8005fb0 comp rx=0 tx=0).stop 2024-04-26T01:53:14.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:14.628+0000 7f11e6a00700 1 -- 172.21.15.151:0/2998761897 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11e0104ec0 msgr2=0x7f11e019b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:14.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:14.628+0000 7f11e6a00700 1 --2- 172.21.15.151:0/2998761897 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11e0104ec0 0x7f11e019b350 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f11d000e9a0 tx=0x7f11d000ecb0 comp rx=0 tx=0).stop 2024-04-26T01:53:14.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:14.629+0000 7f11e6a00700 1 -- 172.21.15.151:0/2998761897 shutdown_connections 2024-04-26T01:53:14.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:14.629+0000 7f11e6a00700 1 --2- 172.21.15.151:0/2998761897 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11e0109fa0 0x7f11e019fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:14.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:14.629+0000 7f11e6a00700 1 --2- 172.21.15.151:0/2998761897 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11e0104ec0 0x7f11e019b350 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:14.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:14.629+0000 7f11e6a00700 1 --2- 172.21.15.151:0/2998761897 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f11c40a3cc0 0x7f11c40a6180 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:14.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:14.629+0000 7f11e6a00700 1 --2- 172.21.15.151:0/2998761897 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11e0105870 0x7f11e019b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:14.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:14.629+0000 7f11e6a00700 1 -- 172.21.15.151:0/2998761897 >> 172.21.15.151:0/2998761897 conn(0x7f11e0100750 msgr2=0x7f11e0102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:14.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:14.629+0000 7f11e6a00700 1 -- 172.21.15.151:0/2998761897 shutdown_connections 2024-04-26T01:53:14.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:14.629+0000 7f11e6a00700 1 -- 172.21.15.151:0/2998761897 wait complete. 2024-04-26T01:53:15.310 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:15 smithi151 ceph-mon[28057]: pgmap v436: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:53:15.310 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:15 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/2998761897' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:53:15.381 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-26T01:49:40.746876+0000', 'modified': '2024-04-26T01:50:05.608353+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24509, 'mds_1': 24505, 'mds_2': 24493}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24458': {'gid': 24458, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.165:6835/3598138294', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6834', 'nonce': 3598138294}, {'type': 'v1', 'addr': '172.21.15.165:6835', 'nonce': 3598138294}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24470': {'gid': 24470, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.165:6837/1847859054', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6836', 'nonce': 1847859054}, {'type': 'v1', 'addr': '172.21.15.165:6837', 'nonce': 1847859054}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24481': {'gid': 24481, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.172:6835/2309751171', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6834', 'nonce': 2309751171}, {'type': 'v1', 'addr': '172.21.15.172:6835', 'nonce': 2309751171}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24493': {'gid': 24493, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.172:6837/55972537', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6836', 'nonce': 55972537}, {'type': 'v1', 'addr': '172.21.15.172:6837', 'nonce': 55972537}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24505': {'gid': 24505, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.172:6839/1242302810', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6838', 'nonce': 1242302810}, {'type': 'v1', 'addr': '172.21.15.172:6839', 'nonce': 1242302810}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24509': {'gid': 24509, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.165:6839/1969716254', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6838', 'nonce': 1969716254}, {'type': 'v1', 'addr': '172.21.15.165:6839', 'nonce': 1969716254}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24509, 'qdb_cluster': [24505, 24509, 24493]} 2024-04-26T01:53:15.381 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-26T01:53:15.381 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph tell mds.c status 2024-04-26T01:53:15.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:15 smithi172 ceph-mon[32785]: pgmap v436: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:53:15.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:15 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/2998761897' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:53:15.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:15 smithi165 ceph-mon[32648]: pgmap v436: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:53:15.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:15 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/2998761897' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:53:15.836 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:53:17.337 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:17 smithi151 ceph-mon[28057]: pgmap v437: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:53:17.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:17 smithi172 ceph-mon[32785]: pgmap v437: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:53:17.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:17 smithi165 ceph-mon[32648]: pgmap v437: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:53:17.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.881+0000 7fb5bfc00700 1 -- 172.21.15.151:0/1794251759 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5b810e3b0 msgr2=0x7fb5b8115f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:17.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.881+0000 7fb5bfc00700 1 --2- 172.21.15.151:0/1794251759 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5b810e3b0 0x7fb5b8115f80 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb5b000b0d0 tx=0x7fb5b000b3e0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.881+0000 7fb5bfc00700 1 -- 172.21.15.151:0/1794251759 shutdown_connections 2024-04-26T01:53:17.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.881+0000 7fb5bfc00700 1 --2- 172.21.15.151:0/1794251759 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5b810e3b0 0x7fb5b8115f80 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.881+0000 7fb5bfc00700 1 --2- 172.21.15.151:0/1794251759 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5b81092d0 0x7fb5b81096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.881+0000 7fb5bfc00700 1 --2- 172.21.15.151:0/1794251759 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5b8109c80 0x7fb5b810dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.881+0000 7fb5bfc00700 1 -- 172.21.15.151:0/1794251759 >> 172.21.15.151:0/1794251759 conn(0x7fb5b8076210 msgr2=0x7fb5b8107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:17.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.881+0000 7fb5bfc00700 1 -- 172.21.15.151:0/1794251759 shutdown_connections 2024-04-26T01:53:17.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.881+0000 7fb5bfc00700 1 -- 172.21.15.151:0/1794251759 wait complete. 2024-04-26T01:53:17.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.882+0000 7fb5bfc00700 1 Processor -- start 2024-04-26T01:53:17.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.882+0000 7fb5bfc00700 1 -- start start 2024-04-26T01:53:17.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.883+0000 7fb5bfc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5b81092d0 0x7fb5b819f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:17.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.883+0000 7fb5bfc00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5b8109c80 0x7fb5b819fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:17.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.883+0000 7fb5bfc00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5b810e3b0 0x7fb5b81a3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:17.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.883+0000 7fb5bfc00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b81192c0 con 0x7fb5b81092d0 2024-04-26T01:53:17.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.883+0000 7fb5bfc00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b8119140 con 0x7fb5b8109c80 2024-04-26T01:53:17.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.883+0000 7fb5bfc00700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b8119440 con 0x7fb5b810e3b0 2024-04-26T01:53:17.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.883+0000 7fb5bd200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5b81092d0 0x7fb5b819f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:17.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.883+0000 7fb5bd200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5b81092d0 0x7fb5b819f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49626/0 (socket says 172.21.15.151:49626) 2024-04-26T01:53:17.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.883+0000 7fb5bd200700 1 -- 172.21.15.151:0/289877158 learned_addr learned my addr 172.21.15.151:0/289877158 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:53:17.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.883+0000 7fb5bdc00700 1 --2- 172.21.15.151:0/289877158 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5b810e3b0 0x7fb5b81a3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:17.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.883+0000 7fb5b7e00700 1 --2- 172.21.15.151:0/289877158 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5b8109c80 0x7fb5b819fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:17.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.884+0000 7fb5bd200700 1 -- 172.21.15.151:0/289877158 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5b810e3b0 msgr2=0x7fb5b81a3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:17.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.884+0000 7fb5bd200700 1 --2- 172.21.15.151:0/289877158 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5b810e3b0 0x7fb5b81a3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.884+0000 7fb5bd200700 1 -- 172.21.15.151:0/289877158 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5b8109c80 msgr2=0x7fb5b819fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:17.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.884+0000 7fb5bd200700 1 --2- 172.21.15.151:0/289877158 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5b8109c80 0x7fb5b819fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.884+0000 7fb5bd200700 1 -- 172.21.15.151:0/289877158 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5b81a4700 con 0x7fb5b81092d0 2024-04-26T01:53:17.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.884+0000 7fb5bdc00700 1 --2- 172.21.15.151:0/289877158 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5b810e3b0 0x7fb5b81a3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:53:17.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.884+0000 7fb5b7e00700 1 --2- 172.21.15.151:0/289877158 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5b8109c80 0x7fb5b819fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:53:17.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.884+0000 7fb5bd200700 1 --2- 172.21.15.151:0/289877158 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5b81092d0 0x7fb5b819f6f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac00e9a0 tx=0x7fb5ac00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:17.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.884+0000 7fb5b5600700 1 -- 172.21.15.151:0/289877158 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac00ca10 con 0x7fb5b81092d0 2024-04-26T01:53:17.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.884+0000 7fb5b5600700 1 -- 172.21.15.151:0/289877158 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5ac00cbb0 con 0x7fb5b81092d0 2024-04-26T01:53:17.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.884+0000 7fb5bfc00700 1 -- 172.21.15.151:0/289877158 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5b81a49f0 con 0x7fb5b81092d0 2024-04-26T01:53:17.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.884+0000 7fb5b5600700 1 -- 172.21.15.151:0/289877158 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac018680 con 0x7fb5b81092d0 2024-04-26T01:53:17.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.885+0000 7fb5bfc00700 1 -- 172.21.15.151:0/289877158 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5b81ac270 con 0x7fb5b81092d0 2024-04-26T01:53:17.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.888+0000 7fb5b5600700 1 -- 172.21.15.151:0/289877158 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7fb5ac010430 con 0x7fb5b81092d0 2024-04-26T01:53:17.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.890+0000 7fb5ca579b80 1 Processor -- start 2024-04-26T01:53:17.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.890+0000 7fb5b5600700 1 --2- 172.21.15.151:0/289877158 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fb5a00a3c40 0x7fb5a00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:17.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.890+0000 7fb5ca579b80 1 -- start start 2024-04-26T01:53:17.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.891+0000 7fb5b5600700 1 -- 172.21.15.151:0/289877158 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7fb5ac0d0560 con 0x7fb5b81092d0 2024-04-26T01:53:17.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.891+0000 7fb5bdc00700 1 --2- 172.21.15.151:0/289877158 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fb5a00a3c40 0x7fb5a00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:17.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.891+0000 7fb5ca579b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558450dfc2f0 0x558450dfc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:17.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.891+0000 7fb5ca579b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558450dfcc10 0x558450e07570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:17.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.891+0000 7fb5ca579b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558450e07ab0 0x558450e09ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:17.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.891+0000 7fb5ca579b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558450d4fe40 con 0x558450dfc2f0 2024-04-26T01:53:17.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.891+0000 7fb5ca579b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558450cf66e0 con 0x558450dfcc10 2024-04-26T01:53:17.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.891+0000 7fb5ca579b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558450d4be00 con 0x558450e07ab0 2024-04-26T01:53:17.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.891+0000 7fb5bd200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558450dfc2f0 0x558450dfc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:17.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.891+0000 7fb5bd200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558450dfc2f0 0x558450dfc6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49640/0 (socket says 172.21.15.151:49640) 2024-04-26T01:53:17.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.891+0000 7fb5bd200700 1 -- 172.21.15.151:0/638701447 learned_addr learned my addr 172.21.15.151:0/638701447 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:53:17.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.891+0000 7fb5b7e00700 1 --2- 172.21.15.151:0/638701447 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558450dfcc10 0x558450e07570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:17.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.892+0000 7fb5bdc00700 1 --2- 172.21.15.151:0/289877158 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fb5a00a3c40 0x7fb5a00a6100 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb5b000bfd0 tx=0x7fb5b000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:17.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.892+0000 7fb5bdc00700 1 --2- 172.21.15.151:0/638701447 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558450e07ab0 0x558450e09ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:17.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.892+0000 7fb5b7e00700 1 -- 172.21.15.151:0/638701447 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558450e07ab0 msgr2=0x558450e09ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:17.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.892+0000 7fb5b7e00700 1 --2- 172.21.15.151:0/638701447 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558450e07ab0 0x558450e09ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.892+0000 7fb5b7e00700 1 -- 172.21.15.151:0/638701447 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558450dfc2f0 msgr2=0x558450dfc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:17.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.892+0000 7fb5b7e00700 1 --2- 172.21.15.151:0/638701447 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558450dfc2f0 0x558450dfc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.892+0000 7fb5b7e00700 1 -- 172.21.15.151:0/638701447 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558450cec630 con 0x558450dfcc10 2024-04-26T01:53:17.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.892+0000 7fb5bdc00700 1 --2- 172.21.15.151:0/638701447 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558450e07ab0 0x558450e09ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T01:53:17.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.892+0000 7fb5b7e00700 1 --2- 172.21.15.151:0/638701447 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558450dfcc10 0x558450e07570 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb5b806c260 tx=0x7fb5a4009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae56c5bb4c46dab8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:17.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.892+0000 7fb5aa000700 1 -- 172.21.15.151:0/638701447 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a4004310 con 0x558450dfcc10 2024-04-26T01:53:17.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.893+0000 7fb5aa000700 1 -- 172.21.15.151:0/638701447 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5a40044b0 con 0x558450dfcc10 2024-04-26T01:53:17.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.893+0000 7fb5aa000700 1 -- 172.21.15.151:0/638701447 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a4003680 con 0x558450dfcc10 2024-04-26T01:53:17.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.893+0000 7fb5ca579b80 1 -- 172.21.15.151:0/638701447 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558450dfcc10 msgr2=0x558450e07570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:17.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.893+0000 7fb5ca579b80 1 --2- 172.21.15.151:0/638701447 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558450dfcc10 0x558450e07570 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb5b806c260 tx=0x7fb5a4009f90 comp rx=0 tx=0).stop 2024-04-26T01:53:17.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.893+0000 7fb5ca579b80 1 -- 172.21.15.151:0/638701447 shutdown_connections 2024-04-26T01:53:17.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.893+0000 7fb5ca579b80 1 --2- 172.21.15.151:0/638701447 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558450e07ab0 0x558450e09ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.893+0000 7fb5ca579b80 1 --2- 172.21.15.151:0/638701447 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558450dfc2f0 0x558450dfc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.893+0000 7fb5ca579b80 1 --2- 172.21.15.151:0/638701447 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558450dfcc10 0x558450e07570 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.894+0000 7fb5ca579b80 1 -- 172.21.15.151:0/638701447 >> 172.21.15.151:0/638701447 conn(0x558450d2b550 msgr2=0x558450df8790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:17.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.894+0000 7fb5ca579b80 1 -- 172.21.15.151:0/638701447 shutdown_connections 2024-04-26T01:53:17.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.894+0000 7fb5ca579b80 1 -- 172.21.15.151:0/638701447 wait complete. 2024-04-26T01:53:17.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.894+0000 7fb5ca579b80 1 -- start start 2024-04-26T01:53:17.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.894+0000 7fb5ca579b80 1 Processor -- start 2024-04-26T01:53:17.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.895+0000 7fb5ca579b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558450dfc2f0 0x558450e071c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:17.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.895+0000 7fb5ca579b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558450dfcc10 0x558450e044c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:17.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.895+0000 7fb5ca579b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558450e07ab0 0x558450e04a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:17.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.895+0000 7fb5ca579b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558450e0a5a0 con 0x558450e07ab0 2024-04-26T01:53:17.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.895+0000 7fb5ca579b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558450d09810 con 0x558450dfcc10 2024-04-26T01:53:17.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.895+0000 7fb5ca579b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558450e0a720 con 0x558450dfc2f0 2024-04-26T01:53:17.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.895+0000 7fb5ca579b80 10 client.0 _ll_register_callbacks cb 0x558450ce86d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:53:17.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.895+0000 7fb5b7e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558450dfcc10 0x558450e044c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:17.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.895+0000 7fb5b7e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558450dfcc10 0x558450e044c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:36814/0 (socket says 172.21.15.151:36814) 2024-04-26T01:53:17.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.895+0000 7fb5b7e00700 1 -- 172.21.15.151:0/1913974044 learned_addr learned my addr 172.21.15.151:0/1913974044 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:53:17.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.895+0000 7fb5bdc00700 1 --2- 172.21.15.151:0/1913974044 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558450e07ab0 0x558450e04a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:17.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.895+0000 7fb5bd200700 1 --2- 172.21.15.151:0/1913974044 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558450dfc2f0 0x558450e071c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:17.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.896+0000 7fb5b7e00700 1 -- 172.21.15.151:0/1913974044 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558450dfc2f0 msgr2=0x558450e071c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:17.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.896+0000 7fb5b7e00700 1 --2- 172.21.15.151:0/1913974044 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558450dfc2f0 0x558450e071c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.896+0000 7fb5b7e00700 1 -- 172.21.15.151:0/1913974044 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558450e07ab0 msgr2=0x558450e04a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:17.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.896+0000 7fb5b7e00700 1 --2- 172.21.15.151:0/1913974044 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558450e07ab0 0x558450e04a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.896+0000 7fb5b7e00700 1 -- 172.21.15.151:0/1913974044 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558450c9d880 con 0x558450dfcc10 2024-04-26T01:53:17.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.896+0000 7fb5b7e00700 1 --2- 172.21.15.151:0/1913974044 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558450dfcc10 0x558450e044c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb5a4005c50 tx=0x7fb5a40051e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92d8e1e6ee4ce3f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:17.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.896+0000 7fb59ea00700 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T01:53:17.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.896+0000 7fb59ea00700 1 -- 172.21.15.151:0/1913974044 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a4026020 con 0x558450dfcc10 2024-04-26T01:53:17.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.896+0000 7fb59ea00700 1 -- 172.21.15.151:0/1913974044 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7fb5a4004750 con 0x558450dfcc10 2024-04-26T01:53:17.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.896+0000 7fb5ca579b80 1 -- 172.21.15.151:0/1913974044 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5a40044b0 con 0x558450dfcc10 2024-04-26T01:53:17.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.897+0000 7fb59ea00700 1 -- 172.21.15.151:0/1913974044 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb5a4004950 con 0x558450dfcc10 2024-04-26T01:53:17.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.897+0000 7fb5ca579b80 10 client.24962 fetch_fsmap learned FSMap version 25 2024-04-26T01:53:17.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.897+0000 7fb5ca579b80 1 -- 172.21.15.151:0/1913974044 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558450ce6aa0 con 0x558450dfcc10 2024-04-26T01:53:17.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.897+0000 7fb59ea00700 1 -- 172.21.15.151:0/1913974044 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5a40044b0 con 0x558450dfcc10 2024-04-26T01:53:17.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.897+0000 7fb59ea00700 1 -- 172.21.15.151:0/1913974044 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fb5a4018070 con 0x558450dfcc10 2024-04-26T01:53:17.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.898+0000 7fb5ca579b80 10 client.24962 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T01:53:17.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.898+0000 7fb5ca579b80 10 client.24962 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T01:53:17.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.898+0000 7fb5ca579b80 20 client.24962 populate_metadata read hostname 'smithi151' 2024-04-26T01:53:17.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.898+0000 7fb5ca579b80 1 --2- 172.21.15.151:0/1913974044 >> [v2:172.21.15.172:6836/55972537,v1:172.21.15.172:6837/55972537] conn(0x558450e16ec0 0x558450e18330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:17.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.898+0000 7fb5ca579b80 4 client.24962 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:53:17.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.898+0000 7fb5ca579b80 1 -- 172.21.15.151:0/1913974044 --> [v2:172.21.15.172:6836/55972537,v1:172.21.15.172:6837/55972537] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558450d5d630 con 0x558450e16ec0 2024-04-26T01:53:17.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.898+0000 7fb5bd200700 1 --2- 172.21.15.151:0/1913974044 >> [v2:172.21.15.172:6836/55972537,v1:172.21.15.172:6837/55972537] conn(0x558450e16ec0 0x558450e18330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:17.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.898+0000 7fb5bd200700 1 --2- 172.21.15.151:0/1913974044 >> [v2:172.21.15.172:6836/55972537,v1:172.21.15.172:6837/55972537] conn(0x558450e16ec0 0x558450e18330 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=44c0f3901c6a64bf server_cookie=5b503be973db3242 in_seq=0 out_seq=0 2024-04-26T01:53:17.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.898+0000 7fb59ea00700 10 client.24962 ms_handle_connect on v2:172.21.15.172:6836/55972537 2024-04-26T01:53:17.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.899+0000 7fb59ea00700 1 -- 172.21.15.151:0/1913974044 <== mds.2 v2:172.21.15.172:6836/55972537 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558450d5d630 con 0x558450e16ec0 2024-04-26T01:53:17.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.899+0000 7fb59ea00700 10 client.24962 handle_command_reply: tid=0 2024-04-26T01:53:17.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.900+0000 7fb5ca579b80 1 client.24962 shutdown 2024-04-26T01:53:17.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.900+0000 7fb5ca579b80 1 -- 172.21.15.151:0/1913974044 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558450dfcc10 msgr2=0x558450e044c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:17.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.900+0000 7fb5ca579b80 1 --2- 172.21.15.151:0/1913974044 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558450dfcc10 0x558450e044c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb5a4005c50 tx=0x7fb5a40051e0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.900+0000 7fb5ca579b80 1 -- 172.21.15.151:0/1913974044 shutdown_connections 2024-04-26T01:53:17.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.900+0000 7fb5ca579b80 1 --2- 172.21.15.151:0/1913974044 >> [v2:172.21.15.172:6836/55972537,v1:172.21.15.172:6837/55972537] conn(0x558450e16ec0 0x558450e18330 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.900+0000 7fb5ca579b80 1 --2- 172.21.15.151:0/1913974044 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558450e07ab0 0x558450e04a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.900+0000 7fb5ca579b80 1 --2- 172.21.15.151:0/1913974044 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558450dfc2f0 0x558450e071c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.900+0000 7fb5ca579b80 1 --2- 172.21.15.151:0/1913974044 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558450dfcc10 0x558450e044c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.900+0000 7fb59ea00700 0 client.24962 ms_handle_reset on v2:172.21.15.172:6836/55972537 2024-04-26T01:53:17.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.900+0000 7fb5ca579b80 1 -- 172.21.15.151:0/1913974044 >> 172.21.15.151:0/1913974044 conn(0x558450d2b550 msgr2=0x558450df7950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:17.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.900+0000 7fb5ca579b80 1 -- 172.21.15.151:0/1913974044 shutdown_connections 2024-04-26T01:53:17.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.900+0000 7fb5ca579b80 1 -- 172.21.15.151:0/1913974044 wait complete. 2024-04-26T01:53:17.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.900+0000 7fb5ca579b80 20 client.24962 trim_cache size 0 max 16384 2024-04-26T01:53:17.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.922+0000 7fb5ca579b80 1 Processor -- start 2024-04-26T01:53:17.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.923+0000 7fb5ca579b80 1 -- start start 2024-04-26T01:53:17.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.923+0000 7fb5ca579b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558450dfc2f0 0x558450e071c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:17.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.923+0000 7fb5ca579b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558450dfcc10 0x558450dff4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:17.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.923+0000 7fb5ca579b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558450e07ab0 0x558450dffa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:17.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.923+0000 7fb5ca579b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558450e16670 con 0x558450dfc2f0 2024-04-26T01:53:17.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.923+0000 7fb5ca579b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558450e16d40 con 0x558450e07ab0 2024-04-26T01:53:17.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.923+0000 7fb5ca579b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558450e17410 con 0x558450dfcc10 2024-04-26T01:53:17.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.923+0000 7fb5bd200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558450dfc2f0 0x558450e071c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:17.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.923+0000 7fb5bd200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558450dfc2f0 0x558450e071c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49662/0 (socket says 172.21.15.151:49662) 2024-04-26T01:53:17.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.923+0000 7fb5bd200700 1 -- 172.21.15.151:0/2796401471 learned_addr learned my addr 172.21.15.151:0/2796401471 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:53:17.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.923+0000 7fb5b7e00700 1 --2- 172.21.15.151:0/2796401471 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558450dfcc10 0x558450dff4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:17.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.923+0000 7fb5bdc00700 1 --2- 172.21.15.151:0/2796401471 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558450e07ab0 0x558450dffa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:17.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.924+0000 7fb5bd200700 1 -- 172.21.15.151:0/2796401471 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558450dfcc10 msgr2=0x558450dff4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:17.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.924+0000 7fb5bd200700 1 --2- 172.21.15.151:0/2796401471 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558450dfcc10 0x558450dff4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.924+0000 7fb5bd200700 1 -- 172.21.15.151:0/2796401471 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558450e07ab0 msgr2=0x558450dffa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:17.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.924+0000 7fb5bd200700 1 --2- 172.21.15.151:0/2796401471 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558450e07ab0 0x558450dffa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.924+0000 7fb5bd200700 1 -- 172.21.15.151:0/2796401471 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558450e02a30 con 0x558450dfc2f0 2024-04-26T01:53:17.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.924+0000 7fb5bd200700 1 --2- 172.21.15.151:0/2796401471 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558450dfc2f0 0x558450e071c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac00c2d0 tx=0x7fb5ac0cdf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59a306e0d5a68295 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:17.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.924+0000 7fb5a9600700 1 -- 172.21.15.151:0/2796401471 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac0ba070 con 0x558450dfc2f0 2024-04-26T01:53:17.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.924+0000 7fb5a9600700 1 -- 172.21.15.151:0/2796401471 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5ac0a54b0 con 0x558450dfc2f0 2024-04-26T01:53:17.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.924+0000 7fb5a9600700 1 -- 172.21.15.151:0/2796401471 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac09cd00 con 0x558450dfc2f0 2024-04-26T01:53:17.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.925+0000 7fb5ca579b80 1 -- 172.21.15.151:0/2796401471 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558450dfc2f0 msgr2=0x558450e071c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:17.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.925+0000 7fb5ca579b80 1 --2- 172.21.15.151:0/2796401471 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558450dfc2f0 0x558450e071c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac00c2d0 tx=0x7fb5ac0cdf30 comp rx=0 tx=0).stop 2024-04-26T01:53:17.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.925+0000 7fb5ca579b80 1 -- 172.21.15.151:0/2796401471 shutdown_connections 2024-04-26T01:53:17.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.925+0000 7fb5ca579b80 1 --2- 172.21.15.151:0/2796401471 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558450e07ab0 0x558450dffa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.925+0000 7fb5ca579b80 1 --2- 172.21.15.151:0/2796401471 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558450dfc2f0 0x558450e071c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.925+0000 7fb5ca579b80 1 --2- 172.21.15.151:0/2796401471 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558450dfcc10 0x558450dff4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.925+0000 7fb5ca579b80 1 -- 172.21.15.151:0/2796401471 >> 172.21.15.151:0/2796401471 conn(0x558450d2b550 msgr2=0x558450e15f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:17.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.925+0000 7fb5ca579b80 1 -- 172.21.15.151:0/2796401471 shutdown_connections 2024-04-26T01:53:17.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.925+0000 7fb5ca579b80 1 -- 172.21.15.151:0/2796401471 wait complete. 2024-04-26T01:53:17.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.925+0000 7fb5ca579b80 1 -- start start 2024-04-26T01:53:17.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.926+0000 7fb5ca579b80 1 Processor -- start 2024-04-26T01:53:17.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.926+0000 7fb5ca579b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558450dfc2f0 0x558450e1ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:17.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.926+0000 7fb5ca579b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558450dfcc10 0x558450e1f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:17.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.926+0000 7fb5ca579b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558450e07ab0 0x558450e08c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:17.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.926+0000 7fb5ca579b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558450e181b0 con 0x558450dfc2f0 2024-04-26T01:53:17.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.926+0000 7fb5ca579b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558450e17ae0 con 0x558450e07ab0 2024-04-26T01:53:17.928 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.926+0000 7fb5ca579b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558450e18880 con 0x558450dfcc10 2024-04-26T01:53:17.928 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.926+0000 7fb5ca579b80 10 client.0 _ll_register_callbacks cb 0x558450ce86d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:53:17.928 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.926+0000 7fb5bd200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558450dfc2f0 0x558450e1ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:17.928 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.927+0000 7fb5bd200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558450dfc2f0 0x558450e1ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49666/0 (socket says 172.21.15.151:49666) 2024-04-26T01:53:17.928 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.927+0000 7fb5bd200700 1 -- 172.21.15.151:0/937972085 learned_addr learned my addr 172.21.15.151:0/937972085 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:53:17.928 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.927+0000 7fb5bdc00700 1 --2- 172.21.15.151:0/937972085 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558450e07ab0 0x558450e08c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:17.928 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.927+0000 7fb5b7e00700 1 --2- 172.21.15.151:0/937972085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558450dfcc10 0x558450e1f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:17.928 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.927+0000 7fb5bd200700 1 -- 172.21.15.151:0/937972085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558450dfcc10 msgr2=0x558450e1f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:17.928 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.927+0000 7fb5bd200700 1 --2- 172.21.15.151:0/937972085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558450dfcc10 0x558450e1f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.928 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.927+0000 7fb5bd200700 1 -- 172.21.15.151:0/937972085 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558450e07ab0 msgr2=0x558450e08c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:17.928 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.927+0000 7fb5bd200700 1 --2- 172.21.15.151:0/937972085 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558450e07ab0 0x558450e08c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.928 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.927+0000 7fb5bd200700 1 -- 172.21.15.151:0/937972085 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558450e09160 con 0x558450dfc2f0 2024-04-26T01:53:17.928 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.927+0000 7fb5bd200700 1 --2- 172.21.15.151:0/937972085 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558450dfc2f0 0x558450e1ee80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac0b6fd0 tx=0x7fb5ac09d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac2d25ae5f57edbe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:17.928 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.927+0000 7fb59ea00700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-04-26T01:53:17.928 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.927+0000 7fb59ea00700 1 -- 172.21.15.151:0/937972085 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac0ba070 con 0x558450dfc2f0 2024-04-26T01:53:17.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.928+0000 7fb59ea00700 1 -- 172.21.15.151:0/937972085 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7fb5ac0d00f0 con 0x558450dfc2f0 2024-04-26T01:53:17.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.928+0000 7fb5ca579b80 1 -- 172.21.15.151:0/937972085 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5ac0a54b0 con 0x558450dfc2f0 2024-04-26T01:53:17.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.928+0000 7fb59ea00700 1 -- 172.21.15.151:0/937972085 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb5ac0d02f0 con 0x558450dfc2f0 2024-04-26T01:53:17.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.928+0000 7fb5ca579b80 10 client.14874 fetch_fsmap learned FSMap version 25 2024-04-26T01:53:17.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.928+0000 7fb5ca579b80 1 -- 172.21.15.151:0/937972085 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558450e7e6b0 con 0x558450dfc2f0 2024-04-26T01:53:17.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.928+0000 7fb59ea00700 1 -- 172.21.15.151:0/937972085 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5ac0a54b0 con 0x558450dfc2f0 2024-04-26T01:53:17.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.928+0000 7fb59ea00700 1 -- 172.21.15.151:0/937972085 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fb5ac0b4070 con 0x558450dfc2f0 2024-04-26T01:53:17.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.928+0000 7fb5ca579b80 10 client.14874 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T01:53:17.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.928+0000 7fb5ca579b80 10 client.14874 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T01:53:17.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.928+0000 7fb5ca579b80 20 client.14874 populate_metadata read hostname 'smithi151' 2024-04-26T01:53:17.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.928+0000 7fb5ca579b80 1 --2- 172.21.15.151:0/937972085 >> [v2:172.21.15.172:6836/55972537,v1:172.21.15.172:6837/55972537] conn(0x558450e82c60 0x558450e81970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:17.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.929+0000 7fb5ca579b80 4 client.14874 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T01:53:17.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.929+0000 7fb5ca579b80 1 -- 172.21.15.151:0/937972085 --> [v2:172.21.15.172:6836/55972537,v1:172.21.15.172:6837/55972537] -- command(tid 0: {"prefix": "status"}) -- 0x558450d576d0 con 0x558450e82c60 2024-04-26T01:53:17.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.929+0000 7fb5b7e00700 1 --2- 172.21.15.151:0/937972085 >> [v2:172.21.15.172:6836/55972537,v1:172.21.15.172:6837/55972537] conn(0x558450e82c60 0x558450e81970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:17.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.929+0000 7fb5b7e00700 1 --2- 172.21.15.151:0/937972085 >> [v2:172.21.15.172:6836/55972537,v1:172.21.15.172:6837/55972537] conn(0x558450e82c60 0x558450e81970 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7be1520b1c1f6124 server_cookie=b04d5d878d3c21b3 in_seq=0 out_seq=0 2024-04-26T01:53:17.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.929+0000 7fb59ea00700 10 client.14874 ms_handle_connect on v2:172.21.15.172:6836/55972537 2024-04-26T01:53:17.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.930+0000 7fb59ea00700 1 -- 172.21.15.151:0/937972085 <== mds.2 v2:172.21.15.172:6836/55972537 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x558450d576d0 con 0x558450e82c60 2024-04-26T01:53:17.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.930+0000 7fb59ea00700 10 client.14874 handle_command_reply: tid=0 2024-04-26T01:53:17.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.930+0000 7fb5ca579b80 1 client.14874 shutdown 2024-04-26T01:53:17.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.930+0000 7fb5ca579b80 1 -- 172.21.15.151:0/937972085 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558450dfc2f0 msgr2=0x558450e1ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:17.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.930+0000 7fb5ca579b80 1 --2- 172.21.15.151:0/937972085 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558450dfc2f0 0x558450e1ee80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac0b6fd0 tx=0x7fb5ac09d040 comp rx=0 tx=0).stop 2024-04-26T01:53:17.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.930+0000 7fb5ca579b80 1 -- 172.21.15.151:0/937972085 shutdown_connections 2024-04-26T01:53:17.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.930+0000 7fb5ca579b80 1 --2- 172.21.15.151:0/937972085 >> [v2:172.21.15.172:6836/55972537,v1:172.21.15.172:6837/55972537] conn(0x558450e82c60 0x558450e81970 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.930+0000 7fb5ca579b80 1 --2- 172.21.15.151:0/937972085 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558450e07ab0 0x558450e08c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.930+0000 7fb5ca579b80 1 --2- 172.21.15.151:0/937972085 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558450dfc2f0 0x558450e1ee80 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.930+0000 7fb59ea00700 0 client.14874 ms_handle_reset on v2:172.21.15.172:6836/55972537 2024-04-26T01:53:17.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.930+0000 7fb5ca579b80 1 --2- 172.21.15.151:0/937972085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558450dfcc10 0x558450e1f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.930+0000 7fb5ca579b80 1 -- 172.21.15.151:0/937972085 >> 172.21.15.151:0/937972085 conn(0x558450d2b550 msgr2=0x558450dff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:17.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.930+0000 7fb5ca579b80 1 -- 172.21.15.151:0/937972085 shutdown_connections 2024-04-26T01:53:17.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.930+0000 7fb5ca579b80 1 -- 172.21.15.151:0/937972085 wait complete. 2024-04-26T01:53:17.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.930+0000 7fb5ca579b80 20 client.14874 trim_cache size 0 max 16384 2024-04-26T01:53:17.932 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-26T01:53:17.932 INFO:teuthology.orchestra.run.smithi151.stdout: "cluster_fsid": "c0e49e64-036d-11ef-bc93-c7b262605968", 2024-04-26T01:53:17.932 INFO:teuthology.orchestra.run.smithi151.stdout: "whoami": 2, 2024-04-26T01:53:17.932 INFO:teuthology.orchestra.run.smithi151.stdout: "id": 24493, 2024-04-26T01:53:17.932 INFO:teuthology.orchestra.run.smithi151.stdout: "want_state": "up:active", 2024-04-26T01:53:17.932 INFO:teuthology.orchestra.run.smithi151.stdout: "state": "up:active", 2024-04-26T01:53:17.932 INFO:teuthology.orchestra.run.smithi151.stdout: "fs_name": "cephfs", 2024-04-26T01:53:17.932 INFO:teuthology.orchestra.run.smithi151.stdout: "rank_uptime": 199.15981332499999, 2024-04-26T01:53:17.932 INFO:teuthology.orchestra.run.smithi151.stdout: "mdsmap_epoch": 25, 2024-04-26T01:53:17.932 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch": 92, 2024-04-26T01:53:17.932 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch_barrier": 90, 2024-04-26T01:53:17.933 INFO:teuthology.orchestra.run.smithi151.stdout: "uptime": 257.67205249099999 2024-04-26T01:53:17.933 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-26T01:53:17.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.931+0000 7fb5bfc00700 1 -- 172.21.15.151:0/289877158 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fb5a00a3c40 msgr2=0x7fb5a00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:17.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.931+0000 7fb5bfc00700 1 --2- 172.21.15.151:0/289877158 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fb5a00a3c40 0x7fb5a00a6100 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb5b000bfd0 tx=0x7fb5b000b040 comp rx=0 tx=0).stop 2024-04-26T01:53:17.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.931+0000 7fb5bfc00700 1 -- 172.21.15.151:0/289877158 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5b81092d0 msgr2=0x7fb5b819f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:17.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.931+0000 7fb5bfc00700 1 --2- 172.21.15.151:0/289877158 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5b81092d0 0x7fb5b819f6f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac00e9a0 tx=0x7fb5ac00ecb0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.931+0000 7fb5bfc00700 1 -- 172.21.15.151:0/289877158 shutdown_connections 2024-04-26T01:53:17.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.931+0000 7fb5bfc00700 1 --2- 172.21.15.151:0/289877158 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5b810e3b0 0x7fb5b81a3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.931+0000 7fb5bfc00700 1 --2- 172.21.15.151:0/289877158 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fb5a00a3c40 0x7fb5a00a6100 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.931+0000 7fb5bfc00700 1 --2- 172.21.15.151:0/289877158 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5b81092d0 0x7fb5b819f6f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.931+0000 7fb5bfc00700 1 --2- 172.21.15.151:0/289877158 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5b8109c80 0x7fb5b819fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.931+0000 7fb5bfc00700 1 -- 172.21.15.151:0/289877158 >> 172.21.15.151:0/289877158 conn(0x7fb5b8076210 msgr2=0x7fb5b8076750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:17.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.931+0000 7fb5bfc00700 1 -- 172.21.15.151:0/289877158 shutdown_connections 2024-04-26T01:53:17.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:17.931+0000 7fb5bfc00700 1 -- 172.21.15.151:0/289877158 wait complete. 2024-04-26T01:53:18.849 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph tell mds.i status 2024-04-26T01:53:19.303 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:53:19.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:19 smithi172 ceph-mon[32785]: pgmap v438: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T01:53:19.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:19 smithi165 ceph-mon[32648]: pgmap v438: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T01:53:19.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:19 smithi151 ceph-mon[28057]: pgmap v438: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T01:53:21.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:21 smithi151 ceph-mon[28057]: pgmap v439: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:53:21.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:21 smithi172 ceph-mon[32785]: pgmap v439: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:53:21.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:21 smithi165 ceph-mon[32648]: pgmap v439: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:53:21.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.602+0000 7fd8e4800700 1 -- 172.21.15.151:0/4147852911 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8dc109fa0 msgr2=0x7fd8dc111b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:21.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.602+0000 7fd8e4800700 1 --2- 172.21.15.151:0/4147852911 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8dc109fa0 0x7fd8dc111b70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd8d8009f90 tx=0x7fd8d800b330 comp rx=0 tx=0).stop 2024-04-26T01:53:21.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.602+0000 7fd8e4800700 1 -- 172.21.15.151:0/4147852911 shutdown_connections 2024-04-26T01:53:21.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.602+0000 7fd8e4800700 1 --2- 172.21.15.151:0/4147852911 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8dc109fa0 0x7fd8dc111b70 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:21.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.602+0000 7fd8e4800700 1 --2- 172.21.15.151:0/4147852911 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8dc104ec0 0x7fd8dc1052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:21.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.602+0000 7fd8e4800700 1 --2- 172.21.15.151:0/4147852911 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd8dc105870 0x7fd8dc1098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:21.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.602+0000 7fd8e4800700 1 -- 172.21.15.151:0/4147852911 >> 172.21.15.151:0/4147852911 conn(0x7fd8dc100750 msgr2=0x7fd8dc102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:21.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.603+0000 7fd8e4800700 1 -- 172.21.15.151:0/4147852911 shutdown_connections 2024-04-26T01:53:21.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.603+0000 7fd8e4800700 1 -- 172.21.15.151:0/4147852911 wait complete. 2024-04-26T01:53:21.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.604+0000 7fd8e4800700 1 Processor -- start 2024-04-26T01:53:21.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.604+0000 7fd8e4800700 1 -- start start 2024-04-26T01:53:21.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.604+0000 7fd8e4800700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd8dc104ec0 0x7fd8dc19b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:21.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.604+0000 7fd8e4800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8dc105870 0x7fd8dc19b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:21.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.604+0000 7fd8e4800700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8dc109fa0 0x7fd8dc19fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:21.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.604+0000 7fd8e4800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd8dc114ea0 con 0x7fd8dc105870 2024-04-26T01:53:21.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.604+0000 7fd8e4800700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd8dc114d20 con 0x7fd8dc109fa0 2024-04-26T01:53:21.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.604+0000 7fd8e4800700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd8dc115020 con 0x7fd8dc104ec0 2024-04-26T01:53:21.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.604+0000 7fd8e1400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8dc105870 0x7fd8dc19b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:21.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.604+0000 7fd8e2800700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8dc109fa0 0x7fd8dc19fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:21.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.605+0000 7fd8e2800700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8dc109fa0 0x7fd8dc19fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:43654/0 (socket says 172.21.15.151:43654) 2024-04-26T01:53:21.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.605+0000 7fd8e1400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8dc105870 0x7fd8dc19b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36780/0 (socket says 172.21.15.151:36780) 2024-04-26T01:53:21.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.605+0000 7fd8e2800700 1 -- 172.21.15.151:0/4127684636 learned_addr learned my addr 172.21.15.151:0/4127684636 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:53:21.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.605+0000 7fd8e1e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd8dc104ec0 0x7fd8dc19b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:21.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.605+0000 7fd8e2800700 1 -- 172.21.15.151:0/4127684636 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd8dc104ec0 msgr2=0x7fd8dc19b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:21.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.605+0000 7fd8e2800700 1 --2- 172.21.15.151:0/4127684636 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd8dc104ec0 0x7fd8dc19b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:21.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.605+0000 7fd8e2800700 1 -- 172.21.15.151:0/4127684636 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8dc105870 msgr2=0x7fd8dc19b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:21.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.605+0000 7fd8e2800700 1 --2- 172.21.15.151:0/4127684636 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8dc105870 0x7fd8dc19b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:21.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.605+0000 7fd8e2800700 1 -- 172.21.15.151:0/4127684636 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8dc1a02e0 con 0x7fd8dc109fa0 2024-04-26T01:53:21.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.605+0000 7fd8e2800700 1 --2- 172.21.15.151:0/4127684636 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8dc109fa0 0x7fd8dc19fba0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd8d8009c10 tx=0x7fd8d800cca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:21.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.605+0000 7fd8d6a00700 1 -- 172.21.15.151:0/4127684636 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d8021070 con 0x7fd8dc109fa0 2024-04-26T01:53:21.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.606+0000 7fd8d6a00700 1 -- 172.21.15.151:0/4127684636 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8d801d450 con 0x7fd8dc109fa0 2024-04-26T01:53:21.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.606+0000 7fd8d6a00700 1 -- 172.21.15.151:0/4127684636 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d80052a0 con 0x7fd8dc109fa0 2024-04-26T01:53:21.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.606+0000 7fd8e4800700 1 -- 172.21.15.151:0/4127684636 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8dc1a0570 con 0x7fd8dc109fa0 2024-04-26T01:53:21.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.606+0000 7fd8e4800700 1 -- 172.21.15.151:0/4127684636 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8dc1a7e50 con 0x7fd8dc109fa0 2024-04-26T01:53:21.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.609+0000 7fd8d6a00700 1 -- 172.21.15.151:0/4127684636 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7fd8d8024070 con 0x7fd8dc109fa0 2024-04-26T01:53:21.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.611+0000 7fd8d6a00700 1 --2- 172.21.15.151:0/4127684636 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fd8c40a3cc0 0x7fd8c40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:21.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.611+0000 7fd8ef23fb80 1 Processor -- start 2024-04-26T01:53:21.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.612+0000 7fd8d6a00700 1 -- 172.21.15.151:0/4127684636 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7fd8d800f020 con 0x7fd8dc109fa0 2024-04-26T01:53:21.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.612+0000 7fd8ef23fb80 1 -- start start 2024-04-26T01:53:21.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.612+0000 7fd8e1e00700 1 --2- 172.21.15.151:0/4127684636 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fd8c40a3cc0 0x7fd8c40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:21.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.612+0000 7fd8ef23fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561330962f0 0x5561330966d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:21.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.612+0000 7fd8ef23fb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556133096c10 0x5561330a1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:21.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.612+0000 7fd8ef23fb80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561330a1ab0 0x5561330a3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:21.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.612+0000 7fd8ef23fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x556132fe9e40 con 0x5561330962f0 2024-04-26T01:53:21.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.612+0000 7fd8ef23fb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556132f906e0 con 0x556133096c10 2024-04-26T01:53:21.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.612+0000 7fd8ef23fb80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556132fe5e00 con 0x5561330a1ab0 2024-04-26T01:53:21.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.612+0000 7fd8e1e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561330962f0 0x5561330966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:21.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.612+0000 7fd8e1400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556133096c10 0x5561330a1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:21.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.612+0000 7fd8e1e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561330962f0 0x5561330966d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36790/0 (socket says 172.21.15.151:36790) 2024-04-26T01:53:21.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.612+0000 7fd8e1400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556133096c10 0x5561330a1570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:43662/0 (socket says 172.21.15.151:43662) 2024-04-26T01:53:21.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.612+0000 7fd8e1e00700 1 -- 172.21.15.151:0/455331744 learned_addr learned my addr 172.21.15.151:0/455331744 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:53:21.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.613+0000 7fd8e2800700 1 --2- 172.21.15.151:0/455331744 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561330a1ab0 0x5561330a3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:21.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.613+0000 7fd8e1e00700 1 --2- 172.21.15.151:0/4127684636 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fd8c40a3cc0 0x7fd8c40a6180 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd8d0005fd0 tx=0x7fd8d0011720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:21.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.613+0000 7fd8e1e00700 1 -- 172.21.15.151:0/455331744 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561330a1ab0 msgr2=0x5561330a3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:21.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.613+0000 7fd8e1e00700 1 --2- 172.21.15.151:0/455331744 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561330a1ab0 0x5561330a3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:21.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.613+0000 7fd8e1e00700 1 -- 172.21.15.151:0/455331744 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556133096c10 msgr2=0x5561330a1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:21.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.613+0000 7fd8e1e00700 1 --2- 172.21.15.151:0/455331744 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556133096c10 0x5561330a1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:21.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.613+0000 7fd8e1e00700 1 -- 172.21.15.151:0/455331744 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556132f86630 con 0x5561330962f0 2024-04-26T01:53:21.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.613+0000 7fd8e1400700 1 --2- 172.21.15.151:0/455331744 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556133096c10 0x5561330a1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:53:21.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.613+0000 7fd8e1e00700 1 --2- 172.21.15.151:0/455331744 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561330962f0 0x5561330966d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd8d0005d10 tx=0x7fd8d001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80e6b99d53af4e25 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:21.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.613+0000 7fd8cb400700 1 -- 172.21.15.151:0/455331744 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d0017040 con 0x5561330962f0 2024-04-26T01:53:21.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.614+0000 7fd8cb400700 1 -- 172.21.15.151:0/455331744 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8d0019050 con 0x5561330962f0 2024-04-26T01:53:21.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.614+0000 7fd8cb400700 1 -- 172.21.15.151:0/455331744 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d0005620 con 0x5561330962f0 2024-04-26T01:53:21.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.614+0000 7fd8ef23fb80 1 -- 172.21.15.151:0/455331744 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561330962f0 msgr2=0x5561330966d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:21.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.614+0000 7fd8ef23fb80 1 --2- 172.21.15.151:0/455331744 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561330962f0 0x5561330966d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd8d0005d10 tx=0x7fd8d001c000 comp rx=0 tx=0).stop 2024-04-26T01:53:21.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.614+0000 7fd8ef23fb80 1 -- 172.21.15.151:0/455331744 shutdown_connections 2024-04-26T01:53:21.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.615+0000 7fd8ef23fb80 1 --2- 172.21.15.151:0/455331744 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561330a1ab0 0x5561330a3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:21.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.615+0000 7fd8ef23fb80 1 --2- 172.21.15.151:0/455331744 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561330962f0 0x5561330966d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:21.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.615+0000 7fd8ef23fb80 1 --2- 172.21.15.151:0/455331744 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556133096c10 0x5561330a1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:21.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.615+0000 7fd8ef23fb80 1 -- 172.21.15.151:0/455331744 >> 172.21.15.151:0/455331744 conn(0x556132fc5550 msgr2=0x556133092790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:21.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.615+0000 7fd8ef23fb80 1 -- 172.21.15.151:0/455331744 shutdown_connections 2024-04-26T01:53:21.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.615+0000 7fd8ef23fb80 1 -- 172.21.15.151:0/455331744 wait complete. 2024-04-26T01:53:21.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.615+0000 7fd8ef23fb80 1 -- start start 2024-04-26T01:53:21.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.615+0000 7fd8ef23fb80 1 Processor -- start 2024-04-26T01:53:21.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.616+0000 7fd8ef23fb80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561330962f0 0x5561330a1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:21.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.616+0000 7fd8ef23fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556133096c10 0x55613309e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:21.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.616+0000 7fd8ef23fb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5561330a1ab0 0x55613309ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:21.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.616+0000 7fd8ef23fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5561330a45a0 con 0x556133096c10 2024-04-26T01:53:21.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.616+0000 7fd8ef23fb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556132fa3810 con 0x5561330a1ab0 2024-04-26T01:53:21.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.616+0000 7fd8ef23fb80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5561330a4720 con 0x5561330962f0 2024-04-26T01:53:21.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.616+0000 7fd8ef23fb80 10 client.0 _ll_register_callbacks cb 0x556132f826d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:53:21.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.616+0000 7fd8e1400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556133096c10 0x55613309e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:21.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.616+0000 7fd8e1400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556133096c10 0x55613309e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36794/0 (socket says 172.21.15.151:36794) 2024-04-26T01:53:21.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.616+0000 7fd8e1400700 1 -- 172.21.15.151:0/2904943606 learned_addr learned my addr 172.21.15.151:0/2904943606 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:53:21.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.616+0000 7fd8e1e00700 1 --2- 172.21.15.151:0/2904943606 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561330962f0 0x5561330a1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:21.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.616+0000 7fd8e2800700 1 --2- 172.21.15.151:0/2904943606 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5561330a1ab0 0x55613309ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:21.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.617+0000 7fd8e1400700 1 -- 172.21.15.151:0/2904943606 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561330962f0 msgr2=0x5561330a1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:21.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.617+0000 7fd8e1400700 1 --2- 172.21.15.151:0/2904943606 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561330962f0 0x5561330a1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:21.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.617+0000 7fd8e1400700 1 -- 172.21.15.151:0/2904943606 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5561330a1ab0 msgr2=0x55613309ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:21.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.617+0000 7fd8e1400700 1 --2- 172.21.15.151:0/2904943606 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5561330a1ab0 0x55613309ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:21.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.617+0000 7fd8e1400700 1 -- 172.21.15.151:0/2904943606 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556132f37880 con 0x556133096c10 2024-04-26T01:53:21.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.617+0000 7fd8e1400700 1 --2- 172.21.15.151:0/2904943606 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556133096c10 0x55613309e510 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc012980 tx=0x7fd8cc012c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7200e1e1680dbd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:21.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.617+0000 7fd8bfe00700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-04-26T01:53:21.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.617+0000 7fd8bfe00700 1 -- 172.21.15.151:0/2904943606 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8cc00f5f0 con 0x556133096c10 2024-04-26T01:53:21.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.617+0000 7fd8ef23fb80 1 -- 172.21.15.151:0/2904943606 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8d0019050 con 0x556133096c10 2024-04-26T01:53:21.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.617+0000 7fd8bfe00700 1 -- 172.21.15.151:0/2904943606 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7fd8cc00f790 con 0x556133096c10 2024-04-26T01:53:21.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.618+0000 7fd8bfe00700 1 -- 172.21.15.151:0/2904943606 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd8cc00f990 con 0x556133096c10 2024-04-26T01:53:21.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.618+0000 7fd8ef23fb80 10 client.14886 fetch_fsmap learned FSMap version 25 2024-04-26T01:53:21.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.618+0000 7fd8ef23fb80 1 -- 172.21.15.151:0/2904943606 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556132f80aa0 con 0x556133096c10 2024-04-26T01:53:21.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.618+0000 7fd8bfe00700 1 -- 172.21.15.151:0/2904943606 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8d0019050 con 0x556133096c10 2024-04-26T01:53:21.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.618+0000 7fd8bfe00700 1 -- 172.21.15.151:0/2904943606 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fd8cc010c10 con 0x556133096c10 2024-04-26T01:53:21.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.618+0000 7fd8ef23fb80 10 client.14886 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T01:53:21.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.618+0000 7fd8ef23fb80 10 client.14886 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-26T01:53:21.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.618+0000 7fd8ef23fb80 20 client.14886 populate_metadata read hostname 'smithi151' 2024-04-26T01:53:21.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.618+0000 7fd8ef23fb80 1 --2- 172.21.15.151:0/2904943606 >> [v2:172.21.15.172:6838/1242302810,v1:172.21.15.172:6839/1242302810] conn(0x5561330b1300 0x5561330b2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:21.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.618+0000 7fd8ef23fb80 4 client.14886 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:53:21.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.618+0000 7fd8ef23fb80 1 -- 172.21.15.151:0/2904943606 --> [v2:172.21.15.172:6838/1242302810,v1:172.21.15.172:6839/1242302810] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556132ff7630 con 0x5561330b1300 2024-04-26T01:53:21.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.619+0000 7fd8e1e00700 1 --2- 172.21.15.151:0/2904943606 >> [v2:172.21.15.172:6838/1242302810,v1:172.21.15.172:6839/1242302810] conn(0x5561330b1300 0x5561330b2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:21.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.619+0000 7fd8e1e00700 1 --2- 172.21.15.151:0/2904943606 >> [v2:172.21.15.172:6838/1242302810,v1:172.21.15.172:6839/1242302810] conn(0x5561330b1300 0x5561330b2770 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7250121f046cd776 server_cookie=6f1c1bbc0052f28f in_seq=0 out_seq=0 2024-04-26T01:53:21.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.619+0000 7fd8bfe00700 10 client.14886 ms_handle_connect on v2:172.21.15.172:6838/1242302810 2024-04-26T01:53:21.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.620+0000 7fd8bfe00700 1 -- 172.21.15.151:0/2904943606 <== mds.1 v2:172.21.15.172:6838/1242302810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556132ff7630 con 0x5561330b1300 2024-04-26T01:53:21.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.620+0000 7fd8bfe00700 10 client.14886 handle_command_reply: tid=0 2024-04-26T01:53:21.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.620+0000 7fd8ef23fb80 1 client.14886 shutdown 2024-04-26T01:53:21.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.620+0000 7fd8ef23fb80 1 -- 172.21.15.151:0/2904943606 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556133096c10 msgr2=0x55613309e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:21.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.620+0000 7fd8ef23fb80 1 --2- 172.21.15.151:0/2904943606 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556133096c10 0x55613309e510 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc012980 tx=0x7fd8cc012c90 comp rx=0 tx=0).stop 2024-04-26T01:53:21.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.621+0000 7fd8ef23fb80 1 -- 172.21.15.151:0/2904943606 shutdown_connections 2024-04-26T01:53:21.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.621+0000 7fd8ef23fb80 1 --2- 172.21.15.151:0/2904943606 >> [v2:172.21.15.172:6838/1242302810,v1:172.21.15.172:6839/1242302810] conn(0x5561330b1300 0x5561330b2770 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:21.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.621+0000 7fd8ef23fb80 1 --2- 172.21.15.151:0/2904943606 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5561330a1ab0 0x55613309ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:21.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.621+0000 7fd8ef23fb80 1 --2- 172.21.15.151:0/2904943606 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561330962f0 0x5561330a1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:21.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.621+0000 7fd8ef23fb80 1 --2- 172.21.15.151:0/2904943606 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556133096c10 0x55613309e510 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:21.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.621+0000 7fd8ef23fb80 1 -- 172.21.15.151:0/2904943606 >> 172.21.15.151:0/2904943606 conn(0x556132fc5550 msgr2=0x556133091950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:21.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.621+0000 7fd8bfe00700 0 client.14886 ms_handle_reset on v2:172.21.15.172:6838/1242302810 2024-04-26T01:53:21.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.621+0000 7fd8ef23fb80 1 -- 172.21.15.151:0/2904943606 shutdown_connections 2024-04-26T01:53:21.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.621+0000 7fd8ef23fb80 1 -- 172.21.15.151:0/2904943606 wait complete. 2024-04-26T01:53:21.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.621+0000 7fd8ef23fb80 20 client.14886 trim_cache size 0 max 16384 2024-04-26T01:53:21.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.643+0000 7fd8ef23fb80 1 Processor -- start 2024-04-26T01:53:21.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.643+0000 7fd8ef23fb80 1 -- start start 2024-04-26T01:53:21.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.643+0000 7fd8ef23fb80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561330962f0 0x5561330b0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:21.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.643+0000 7fd8ef23fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556133096c10 0x55613309ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:21.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.643+0000 7fd8ef23fb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5561330a1ab0 0x5561330920d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:21.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.644+0000 7fd8ef23fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5561330b0ab0 con 0x556133096c10 2024-04-26T01:53:21.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.644+0000 7fd8ef23fb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5561330b1180 con 0x5561330a1ab0 2024-04-26T01:53:21.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.644+0000 7fd8ef23fb80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5561330b1850 con 0x5561330962f0 2024-04-26T01:53:21.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.644+0000 7fd8e1400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556133096c10 0x55613309ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:21.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.644+0000 7fd8e1e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561330962f0 0x5561330b0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:21.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.644+0000 7fd8e1400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556133096c10 0x55613309ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36796/0 (socket says 172.21.15.151:36796) 2024-04-26T01:53:21.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.644+0000 7fd8e1400700 1 -- 172.21.15.151:0/173504664 learned_addr learned my addr 172.21.15.151:0/173504664 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:53:21.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.644+0000 7fd8e1e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561330962f0 0x5561330b0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:43204/0 (socket says 172.21.15.151:43204) 2024-04-26T01:53:21.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.644+0000 7fd8e2800700 1 --2- 172.21.15.151:0/173504664 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5561330a1ab0 0x5561330920d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:21.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.644+0000 7fd8e1e00700 1 -- 172.21.15.151:0/173504664 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5561330a1ab0 msgr2=0x5561330920d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:21.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.644+0000 7fd8e1e00700 1 --2- 172.21.15.151:0/173504664 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5561330a1ab0 0x5561330920d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:21.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.644+0000 7fd8e1e00700 1 -- 172.21.15.151:0/173504664 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556133096c10 msgr2=0x55613309ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:21.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.644+0000 7fd8e1e00700 1 --2- 172.21.15.151:0/173504664 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556133096c10 0x55613309ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:21.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.644+0000 7fd8e1e00700 1 -- 172.21.15.151:0/173504664 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55613309ca80 con 0x5561330962f0 2024-04-26T01:53:21.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.645+0000 7fd8e1e00700 1 --2- 172.21.15.151:0/173504664 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561330962f0 0x5561330b0c30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd8d0009930 tx=0x7fd8d0028040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=adcf5441d60ee08e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:21.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.645+0000 7fd8caa00700 1 -- 172.21.15.151:0/173504664 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d0035070 con 0x5561330962f0 2024-04-26T01:53:21.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.645+0000 7fd8caa00700 1 -- 172.21.15.151:0/173504664 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8d001dbb0 con 0x5561330962f0 2024-04-26T01:53:21.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.645+0000 7fd8caa00700 1 -- 172.21.15.151:0/173504664 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d0017040 con 0x5561330962f0 2024-04-26T01:53:21.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.646+0000 7fd8ef23fb80 1 -- 172.21.15.151:0/173504664 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561330962f0 msgr2=0x5561330b0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:21.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.646+0000 7fd8ef23fb80 1 --2- 172.21.15.151:0/173504664 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561330962f0 0x5561330b0c30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd8d0009930 tx=0x7fd8d0028040 comp rx=0 tx=0).stop 2024-04-26T01:53:21.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.646+0000 7fd8ef23fb80 1 -- 172.21.15.151:0/173504664 shutdown_connections 2024-04-26T01:53:21.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.646+0000 7fd8ef23fb80 1 --2- 172.21.15.151:0/173504664 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5561330a1ab0 0x5561330920d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:21.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.646+0000 7fd8ef23fb80 1 --2- 172.21.15.151:0/173504664 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561330962f0 0x5561330b0c30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:21.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.646+0000 7fd8ef23fb80 1 --2- 172.21.15.151:0/173504664 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556133096c10 0x55613309ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:21.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.646+0000 7fd8ef23fb80 1 -- 172.21.15.151:0/173504664 >> 172.21.15.151:0/173504664 conn(0x556132fc5550 msgr2=0x55613309e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:21.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.646+0000 7fd8ef23fb80 1 -- 172.21.15.151:0/173504664 shutdown_connections 2024-04-26T01:53:21.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.646+0000 7fd8ef23fb80 1 -- 172.21.15.151:0/173504664 wait complete. 2024-04-26T01:53:21.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.646+0000 7fd8ef23fb80 1 -- start start 2024-04-26T01:53:21.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.647+0000 7fd8ef23fb80 1 Processor -- start 2024-04-26T01:53:21.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.647+0000 7fd8ef23fb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5561330962f0 0x55613309bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:21.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.647+0000 7fd8ef23fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556133096c10 0x5561330bbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:21.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.647+0000 7fd8ef23fb80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561330a1ab0 0x5561330bc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:21.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.647+0000 7fd8ef23fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5561330b25f0 con 0x556133096c10 2024-04-26T01:53:21.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.647+0000 7fd8ef23fb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5561330b1f20 con 0x5561330962f0 2024-04-26T01:53:21.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.647+0000 7fd8ef23fb80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5561330b2cc0 con 0x5561330a1ab0 2024-04-26T01:53:21.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.647+0000 7fd8ef23fb80 10 client.0 _ll_register_callbacks cb 0x556132f826d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:53:21.648 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.647+0000 7fd8e1e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5561330962f0 0x55613309bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:21.648 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.647+0000 7fd8e1400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556133096c10 0x5561330bbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:21.648 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.647+0000 7fd8e1e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5561330962f0 0x55613309bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:43694/0 (socket says 172.21.15.151:43694) 2024-04-26T01:53:21.648 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.647+0000 7fd8e1e00700 1 -- 172.21.15.151:0/3792529941 learned_addr learned my addr 172.21.15.151:0/3792529941 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:53:21.648 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.647+0000 7fd8e2800700 1 --2- 172.21.15.151:0/3792529941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561330a1ab0 0x5561330bc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:21.648 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.648+0000 7fd8e1400700 1 -- 172.21.15.151:0/3792529941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561330a1ab0 msgr2=0x5561330bc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:21.648 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.648+0000 7fd8e1400700 1 --2- 172.21.15.151:0/3792529941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561330a1ab0 0x5561330bc120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:21.648 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.648+0000 7fd8e1400700 1 -- 172.21.15.151:0/3792529941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5561330962f0 msgr2=0x55613309bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:21.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.648+0000 7fd8e1400700 1 --2- 172.21.15.151:0/3792529941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5561330962f0 0x55613309bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:21.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.648+0000 7fd8e1400700 1 -- 172.21.15.151:0/3792529941 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561330a2850 con 0x556133096c10 2024-04-26T01:53:21.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.648+0000 7fd8e1400700 1 --2- 172.21.15.151:0/3792529941 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556133096c10 0x5561330bbbe0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc005a80 tx=0x7fd8cc00f870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f575ff1655b042b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:21.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.648+0000 7fd8bfe00700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-04-26T01:53:21.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.649+0000 7fd8bfe00700 1 -- 172.21.15.151:0/3792529941 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8cc005040 con 0x556133096c10 2024-04-26T01:53:21.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.649+0000 7fd8ef23fb80 1 -- 172.21.15.151:0/3792529941 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556132ff16d0 con 0x556133096c10 2024-04-26T01:53:21.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.649+0000 7fd8bfe00700 1 -- 172.21.15.151:0/3792529941 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7fd8cc0051e0 con 0x556133096c10 2024-04-26T01:53:21.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.649+0000 7fd8bfe00700 1 -- 172.21.15.151:0/3792529941 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd8cc037050 con 0x556133096c10 2024-04-26T01:53:21.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.649+0000 7fd8ef23fb80 10 client.14892 fetch_fsmap learned FSMap version 25 2024-04-26T01:53:21.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.649+0000 7fd8ef23fb80 1 -- 172.21.15.151:0/3792529941 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5561331186e0 con 0x556133096c10 2024-04-26T01:53:21.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.649+0000 7fd8bfe00700 1 -- 172.21.15.151:0/3792529941 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x556132ff16d0 con 0x556133096c10 2024-04-26T01:53:21.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.650+0000 7fd8bfe00700 1 -- 172.21.15.151:0/3792529941 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fd8cc003bb0 con 0x556133096c10 2024-04-26T01:53:21.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.650+0000 7fd8ef23fb80 10 client.14892 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T01:53:21.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.650+0000 7fd8ef23fb80 10 client.14892 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-26T01:53:21.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.650+0000 7fd8ef23fb80 20 client.14892 populate_metadata read hostname 'smithi151' 2024-04-26T01:53:21.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.650+0000 7fd8ef23fb80 1 --2- 172.21.15.151:0/3792529941 >> [v2:172.21.15.172:6838/1242302810,v1:172.21.15.172:6839/1242302810] conn(0x55613311d260 0x55613311bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:21.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.650+0000 7fd8ef23fb80 4 client.14892 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T01:53:21.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.650+0000 7fd8ef23fb80 1 -- 172.21.15.151:0/3792529941 --> [v2:172.21.15.172:6838/1242302810,v1:172.21.15.172:6839/1242302810] -- command(tid 0: {"prefix": "status"}) -- 0x556132e6a200 con 0x55613311d260 2024-04-26T01:53:21.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.650+0000 7fd8e1e00700 1 --2- 172.21.15.151:0/3792529941 >> [v2:172.21.15.172:6838/1242302810,v1:172.21.15.172:6839/1242302810] conn(0x55613311d260 0x55613311bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:21.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.651+0000 7fd8e1e00700 1 --2- 172.21.15.151:0/3792529941 >> [v2:172.21.15.172:6838/1242302810,v1:172.21.15.172:6839/1242302810] conn(0x55613311d260 0x55613311bf70 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5f701be974b1f67d server_cookie=aa99877e60b9a240 in_seq=0 out_seq=0 2024-04-26T01:53:21.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.651+0000 7fd8bfe00700 10 client.14892 ms_handle_connect on v2:172.21.15.172:6838/1242302810 2024-04-26T01:53:21.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.651+0000 7fd8bfe00700 1 -- 172.21.15.151:0/3792529941 <== mds.1 v2:172.21.15.172:6838/1242302810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x556132e6a200 con 0x55613311d260 2024-04-26T01:53:21.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.651+0000 7fd8bfe00700 10 client.14892 handle_command_reply: tid=0 2024-04-26T01:53:21.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.651+0000 7fd8ef23fb80 1 client.14892 shutdown 2024-04-26T01:53:21.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.651+0000 7fd8ef23fb80 1 -- 172.21.15.151:0/3792529941 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556133096c10 msgr2=0x5561330bbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:21.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.651+0000 7fd8ef23fb80 1 --2- 172.21.15.151:0/3792529941 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556133096c10 0x5561330bbbe0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc005a80 tx=0x7fd8cc00f870 comp rx=0 tx=0).stop 2024-04-26T01:53:21.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.651+0000 7fd8ef23fb80 1 -- 172.21.15.151:0/3792529941 shutdown_connections 2024-04-26T01:53:21.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.651+0000 7fd8ef23fb80 1 --2- 172.21.15.151:0/3792529941 >> [v2:172.21.15.172:6838/1242302810,v1:172.21.15.172:6839/1242302810] conn(0x55613311d260 0x55613311bf70 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:21.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.651+0000 7fd8ef23fb80 1 --2- 172.21.15.151:0/3792529941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561330a1ab0 0x5561330bc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:21.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.651+0000 7fd8ef23fb80 1 --2- 172.21.15.151:0/3792529941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5561330962f0 0x55613309bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:21.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.651+0000 7fd8ef23fb80 1 --2- 172.21.15.151:0/3792529941 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556133096c10 0x5561330bbbe0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:21.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.651+0000 7fd8ef23fb80 1 -- 172.21.15.151:0/3792529941 >> 172.21.15.151:0/3792529941 conn(0x556132fc5550 msgr2=0x5561330993e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:21.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.651+0000 7fd8bfe00700 0 client.14892 ms_handle_reset on v2:172.21.15.172:6838/1242302810 2024-04-26T01:53:21.653 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.651+0000 7fd8ef23fb80 1 -- 172.21.15.151:0/3792529941 shutdown_connections 2024-04-26T01:53:21.653 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.652+0000 7fd8ef23fb80 1 -- 172.21.15.151:0/3792529941 wait complete. 2024-04-26T01:53:21.653 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.652+0000 7fd8ef23fb80 20 client.14892 trim_cache size 0 max 16384 2024-04-26T01:53:21.653 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-26T01:53:21.653 INFO:teuthology.orchestra.run.smithi151.stdout: "cluster_fsid": "c0e49e64-036d-11ef-bc93-c7b262605968", 2024-04-26T01:53:21.653 INFO:teuthology.orchestra.run.smithi151.stdout: "whoami": 1, 2024-04-26T01:53:21.653 INFO:teuthology.orchestra.run.smithi151.stdout: "id": 24505, 2024-04-26T01:53:21.653 INFO:teuthology.orchestra.run.smithi151.stdout: "want_state": "up:active", 2024-04-26T01:53:21.653 INFO:teuthology.orchestra.run.smithi151.stdout: "state": "up:active", 2024-04-26T01:53:21.653 INFO:teuthology.orchestra.run.smithi151.stdout: "fs_name": "cephfs", 2024-04-26T01:53:21.653 INFO:teuthology.orchestra.run.smithi151.stdout: "rank_uptime": 203.89309885899999, 2024-04-26T01:53:21.653 INFO:teuthology.orchestra.run.smithi151.stdout: "mdsmap_epoch": 25, 2024-04-26T01:53:21.654 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch": 92, 2024-04-26T01:53:21.654 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch_barrier": 90, 2024-04-26T01:53:21.654 INFO:teuthology.orchestra.run.smithi151.stdout: "uptime": 255.865831316 2024-04-26T01:53:21.654 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-26T01:53:21.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.652+0000 7fd8e4800700 1 -- 172.21.15.151:0/4127684636 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fd8c40a3cc0 msgr2=0x7fd8c40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:21.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.652+0000 7fd8e4800700 1 --2- 172.21.15.151:0/4127684636 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fd8c40a3cc0 0x7fd8c40a6180 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd8d0005fd0 tx=0x7fd8d0011720 comp rx=0 tx=0).stop 2024-04-26T01:53:21.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.652+0000 7fd8e4800700 1 -- 172.21.15.151:0/4127684636 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8dc109fa0 msgr2=0x7fd8dc19fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:21.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.652+0000 7fd8e4800700 1 --2- 172.21.15.151:0/4127684636 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8dc109fa0 0x7fd8dc19fba0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd8d8009c10 tx=0x7fd8d800cca0 comp rx=0 tx=0).stop 2024-04-26T01:53:21.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.653+0000 7fd8e4800700 1 -- 172.21.15.151:0/4127684636 shutdown_connections 2024-04-26T01:53:21.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.653+0000 7fd8e4800700 1 --2- 172.21.15.151:0/4127684636 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8dc109fa0 0x7fd8dc19fba0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:21.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.653+0000 7fd8e4800700 1 --2- 172.21.15.151:0/4127684636 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd8dc104ec0 0x7fd8dc19b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:21.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.653+0000 7fd8e4800700 1 --2- 172.21.15.151:0/4127684636 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fd8c40a3cc0 0x7fd8c40a6180 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:21.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.653+0000 7fd8e4800700 1 --2- 172.21.15.151:0/4127684636 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8dc105870 0x7fd8dc19b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:21.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.653+0000 7fd8e4800700 1 -- 172.21.15.151:0/4127684636 >> 172.21.15.151:0/4127684636 conn(0x7fd8dc100750 msgr2=0x7fd8dc1012d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:21.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.653+0000 7fd8e4800700 1 -- 172.21.15.151:0/4127684636 shutdown_connections 2024-04-26T01:53:21.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:21.653+0000 7fd8e4800700 1 -- 172.21.15.151:0/4127684636 wait complete. 2024-04-26T01:53:22.610 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph tell mds.b status 2024-04-26T01:53:23.060 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:53:23.525 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:23 smithi151 ceph-mon[28057]: pgmap v440: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:53:23.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:23 smithi172 ceph-mon[32785]: pgmap v440: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:53:23.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:23 smithi165 ceph-mon[32648]: pgmap v440: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:53:25.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.337+0000 7fc8b9e00700 1 -- 172.21.15.151:0/188795596 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8b410e3b0 msgr2=0x7fc8b4115f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:25.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.337+0000 7fc8b9e00700 1 --2- 172.21.15.151:0/188795596 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8b410e3b0 0x7fc8b4115f80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac009fc0 tx=0x7fc8ac00b330 comp rx=0 tx=0).stop 2024-04-26T01:53:25.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.337+0000 7fc8b9e00700 1 -- 172.21.15.151:0/188795596 shutdown_connections 2024-04-26T01:53:25.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.337+0000 7fc8b9e00700 1 --2- 172.21.15.151:0/188795596 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8b410e3b0 0x7fc8b4115f80 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:25.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.337+0000 7fc8b9e00700 1 --2- 172.21.15.151:0/188795596 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8b41092d0 0x7fc8b41096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:25.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.337+0000 7fc8b9e00700 1 --2- 172.21.15.151:0/188795596 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8b4109c80 0x7fc8b410dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:25.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.337+0000 7fc8b9e00700 1 -- 172.21.15.151:0/188795596 >> 172.21.15.151:0/188795596 conn(0x7fc8b4076210 msgr2=0x7fc8b4107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:25.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.337+0000 7fc8b9e00700 1 -- 172.21.15.151:0/188795596 shutdown_connections 2024-04-26T01:53:25.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.337+0000 7fc8b9e00700 1 -- 172.21.15.151:0/188795596 wait complete. 2024-04-26T01:53:25.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.338+0000 7fc8b9e00700 1 Processor -- start 2024-04-26T01:53:25.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.338+0000 7fc8b9e00700 1 -- start start 2024-04-26T01:53:25.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.339+0000 7fc8b9e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8b41092d0 0x7fc8b419f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:25.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.339+0000 7fc8b9e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8b4109c80 0x7fc8b419fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:25.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.339+0000 7fc8b9e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8b410e3b0 0x7fc8b41a3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:25.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.339+0000 7fc8b9e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b41192a0 con 0x7fc8b410e3b0 2024-04-26T01:53:25.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.339+0000 7fc8b9e00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b4119120 con 0x7fc8b41092d0 2024-04-26T01:53:25.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.339+0000 7fc8b9e00700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b4119420 con 0x7fc8b4109c80 2024-04-26T01:53:25.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.339+0000 7fc8b3400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8b410e3b0 0x7fc8b41a3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:25.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.339+0000 7fc8b2000700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8b4109c80 0x7fc8b419fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:25.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.339+0000 7fc8b2a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8b41092d0 0x7fc8b419f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:25.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.339+0000 7fc8b3400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8b410e3b0 0x7fc8b41a3fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36828/0 (socket says 172.21.15.151:36828) 2024-04-26T01:53:25.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.339+0000 7fc8b3400700 1 -- 172.21.15.151:0/1215709431 learned_addr learned my addr 172.21.15.151:0/1215709431 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:53:25.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.340+0000 7fc8b3400700 1 -- 172.21.15.151:0/1215709431 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8b4109c80 msgr2=0x7fc8b419fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:25.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.340+0000 7fc8b3400700 1 --2- 172.21.15.151:0/1215709431 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8b4109c80 0x7fc8b419fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:25.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.340+0000 7fc8b3400700 1 -- 172.21.15.151:0/1215709431 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8b41092d0 msgr2=0x7fc8b419f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:25.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.340+0000 7fc8b3400700 1 --2- 172.21.15.151:0/1215709431 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8b41092d0 0x7fc8b419f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:25.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.340+0000 7fc8b3400700 1 -- 172.21.15.151:0/1215709431 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8b41a46e0 con 0x7fc8b410e3b0 2024-04-26T01:53:25.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.340+0000 7fc8b2000700 1 --2- 172.21.15.151:0/1215709431 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8b4109c80 0x7fc8b419fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:53:25.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.340+0000 7fc8b2a00700 1 --2- 172.21.15.151:0/1215709431 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8b41092d0 0x7fc8b419f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:53:25.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.340+0000 7fc8b3400700 1 --2- 172.21.15.151:0/1215709431 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8b410e3b0 0x7fc8b41a3fa0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac009790 tx=0x7fc8ac00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:25.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.341+0000 7fc8b8a00700 1 -- 172.21.15.151:0/1215709431 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac021070 con 0x7fc8b410e3b0 2024-04-26T01:53:25.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.341+0000 7fc8b8a00700 1 -- 172.21.15.151:0/1215709431 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8ac007ac0 con 0x7fc8b410e3b0 2024-04-26T01:53:25.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.341+0000 7fc8b8a00700 1 -- 172.21.15.151:0/1215709431 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac005040 con 0x7fc8b410e3b0 2024-04-26T01:53:25.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.341+0000 7fc8b9e00700 1 -- 172.21.15.151:0/1215709431 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8b41a4970 con 0x7fc8b410e3b0 2024-04-26T01:53:25.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.341+0000 7fc8b9e00700 1 -- 172.21.15.151:0/1215709431 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8b41ac250 con 0x7fc8b410e3b0 2024-04-26T01:53:25.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.344+0000 7fc8b8a00700 1 -- 172.21.15.151:0/1215709431 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7fc8ac024070 con 0x7fc8b410e3b0 2024-04-26T01:53:25.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.346+0000 7fc8c46b0b80 1 Processor -- start 2024-04-26T01:53:25.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.346+0000 7fc8c46b0b80 1 -- start start 2024-04-26T01:53:25.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.346+0000 7fc8c46b0b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5571eabcd780 0x5571eabcdb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:25.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.346+0000 7fc8c46b0b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5571eabce0a0 0x5571eabc79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:25.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.346+0000 7fc8c46b0b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571eabc7ef0 0x5571eabc8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:25.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.346+0000 7fc8c46b0b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5571eab19e40 con 0x5571eabc7ef0 2024-04-26T01:53:25.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.347+0000 7fc8c46b0b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5571eaac06e0 con 0x5571eabce0a0 2024-04-26T01:53:25.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.347+0000 7fc8c46b0b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5571eab15e00 con 0x5571eabcd780 2024-04-26T01:53:25.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.347+0000 7fc8b2000700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5571eabcd780 0x5571eabcdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:25.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.347+0000 7fc8b2a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5571eabce0a0 0x5571eabc79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:25.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.347+0000 7fc8b2a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5571eabce0a0 0x5571eabc79b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:43720/0 (socket says 172.21.15.151:43720) 2024-04-26T01:53:25.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.347+0000 7fc8b8a00700 1 --2- 172.21.15.151:0/1215709431 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fc8980a3cc0 0x7fc8980a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:25.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.347+0000 7fc8b2000700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5571eabcd780 0x5571eabcdb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:43224/0 (socket says 172.21.15.151:43224) 2024-04-26T01:53:25.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.347+0000 7fc8b2a00700 1 -- 172.21.15.151:0/717878522 learned_addr learned my addr 172.21.15.151:0/717878522 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:53:25.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.347+0000 7fc8b2000700 1 --2- 172.21.15.151:0/717878522 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571eabc7ef0 0x5571eabc8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:25.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.347+0000 7fc8b8a00700 1 -- 172.21.15.151:0/1215709431 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7fc8ac0d1620 con 0x7fc8b410e3b0 2024-04-26T01:53:25.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.347+0000 7fc8b3400700 1 --2- 172.21.15.151:0/1215709431 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fc8980a3cc0 0x7fc8980a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:25.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.347+0000 7fc8b2a00700 1 -- 172.21.15.151:0/717878522 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5571eabcd780 msgr2=0x5571eabcdb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:25.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.347+0000 7fc8b2a00700 1 --2- 172.21.15.151:0/717878522 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5571eabcd780 0x5571eabcdb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:25.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.347+0000 7fc8b2a00700 1 -- 172.21.15.151:0/717878522 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571eabc7ef0 msgr2=0x5571eabc8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:25.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.347+0000 7fc8b2a00700 1 --2- 172.21.15.151:0/717878522 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571eabc7ef0 0x5571eabc8330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:25.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.347+0000 7fc8b2a00700 1 -- 172.21.15.151:0/717878522 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571eaab6630 con 0x5571eabce0a0 2024-04-26T01:53:25.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.348+0000 7fc8b2a00700 1 --2- 172.21.15.151:0/717878522 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5571eabce0a0 0x5571eabc79b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc8a40115a0 tx=0x7fc8a40118b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7351aa8f9fe47a83 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:25.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.348+0000 7fc8b3400700 1 --2- 172.21.15.151:0/1215709431 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fc8980a3cc0 0x7fc8980a6180 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac004fd0 tx=0x7fc8ac00b330 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:25.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.348+0000 7fc8a8c00700 1 -- 172.21.15.151:0/717878522 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a4004230 con 0x5571eabce0a0 2024-04-26T01:53:25.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.348+0000 7fc8a8c00700 1 -- 172.21.15.151:0/717878522 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8a40043d0 con 0x5571eabce0a0 2024-04-26T01:53:25.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.348+0000 7fc8a8c00700 1 -- 172.21.15.151:0/717878522 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a4019960 con 0x5571eabce0a0 2024-04-26T01:53:25.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.349+0000 7fc8c46b0b80 1 -- 172.21.15.151:0/717878522 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5571eabce0a0 msgr2=0x5571eabc79b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:25.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.349+0000 7fc8c46b0b80 1 --2- 172.21.15.151:0/717878522 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5571eabce0a0 0x5571eabc79b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc8a40115a0 tx=0x7fc8a40118b0 comp rx=0 tx=0).stop 2024-04-26T01:53:25.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.349+0000 7fc8c46b0b80 1 -- 172.21.15.151:0/717878522 shutdown_connections 2024-04-26T01:53:25.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.349+0000 7fc8c46b0b80 1 --2- 172.21.15.151:0/717878522 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571eabc7ef0 0x5571eabc8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:25.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.349+0000 7fc8c46b0b80 1 --2- 172.21.15.151:0/717878522 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5571eabcd780 0x5571eabcdb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:25.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.349+0000 7fc8c46b0b80 1 --2- 172.21.15.151:0/717878522 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5571eabce0a0 0x5571eabc79b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:25.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.349+0000 7fc8c46b0b80 1 -- 172.21.15.151:0/717878522 >> 172.21.15.151:0/717878522 conn(0x5571eaaf5f00 msgr2=0x5571eabc28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:25.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.349+0000 7fc8c46b0b80 1 -- 172.21.15.151:0/717878522 shutdown_connections 2024-04-26T01:53:25.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.349+0000 7fc8c46b0b80 1 -- 172.21.15.151:0/717878522 wait complete. 2024-04-26T01:53:25.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.350+0000 7fc8c46b0b80 1 -- start start 2024-04-26T01:53:25.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.350+0000 7fc8c46b0b80 1 Processor -- start 2024-04-26T01:53:25.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.351+0000 7fc8c46b0b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571eabc7ef0 0x5571eabd3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:25.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.351+0000 7fc8c46b0b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5571eabcd780 0x5571eabd0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:25.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.351+0000 7fc8c46b0b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5571eabd14c0 0x5571eabd1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:25.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.351+0000 7fc8c46b0b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5571eabd4920 con 0x5571eabc7ef0 2024-04-26T01:53:25.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.351+0000 7fc8c46b0b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5571eaad3810 con 0x5571eabd14c0 2024-04-26T01:53:25.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.351+0000 7fc8c46b0b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5571eabd4aa0 con 0x5571eabcd780 2024-04-26T01:53:25.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.351+0000 7fc8c46b0b80 10 client.0 _ll_register_callbacks cb 0x5571eaab26d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:53:25.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.351+0000 7fc8b2000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571eabc7ef0 0x5571eabd3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:25.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.351+0000 7fc8b2a00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5571eabcd780 0x5571eabd0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:25.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.351+0000 7fc8b2000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571eabc7ef0 0x5571eabd3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36852/0 (socket says 172.21.15.151:36852) 2024-04-26T01:53:25.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.351+0000 7fc8b2000700 1 -- 172.21.15.151:0/1826313068 learned_addr learned my addr 172.21.15.151:0/1826313068 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:53:25.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.351+0000 7fc8b2000700 1 --2- 172.21.15.151:0/1826313068 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5571eabd14c0 0x5571eabd1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:25.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.351+0000 7fc8b2000700 1 -- 172.21.15.151:0/1826313068 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5571eabcd780 msgr2=0x5571eabd0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:25.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.351+0000 7fc8b2000700 1 --2- 172.21.15.151:0/1826313068 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5571eabcd780 0x5571eabd0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:25.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.351+0000 7fc8b2000700 1 -- 172.21.15.151:0/1826313068 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5571eabd14c0 msgr2=0x5571eabd1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:25.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.351+0000 7fc8b2000700 1 --2- 172.21.15.151:0/1826313068 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5571eabd14c0 0x5571eabd1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:25.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.351+0000 7fc8b2000700 1 -- 172.21.15.151:0/1826313068 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571eaa67880 con 0x5571eabc7ef0 2024-04-26T01:53:25.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.352+0000 7fc8b2000700 1 --2- 172.21.15.151:0/1826313068 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571eabc7ef0 0x5571eabd3c80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc8a0013fd0 tx=0x7fc8a0013730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7c96ef49404f011 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:25.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.352+0000 7fc89d600700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-04-26T01:53:25.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.352+0000 7fc89d600700 1 -- 172.21.15.151:0/1826313068 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a0004050 con 0x5571eabc7ef0 2024-04-26T01:53:25.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.352+0000 7fc89d600700 1 -- 172.21.15.151:0/1826313068 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7fc8a0009040 con 0x5571eabc7ef0 2024-04-26T01:53:25.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.352+0000 7fc8c46b0b80 1 -- 172.21.15.151:0/1826313068 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8a40043d0 con 0x5571eabc7ef0 2024-04-26T01:53:25.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.352+0000 7fc89d600700 1 -- 172.21.15.151:0/1826313068 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc8a0028050 con 0x5571eabc7ef0 2024-04-26T01:53:25.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.352+0000 7fc8c46b0b80 10 client.14904 fetch_fsmap learned FSMap version 25 2024-04-26T01:53:25.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.352+0000 7fc8c46b0b80 1 -- 172.21.15.151:0/1826313068 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5571eaab0aa0 con 0x5571eabc7ef0 2024-04-26T01:53:25.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.353+0000 7fc89d600700 1 -- 172.21.15.151:0/1826313068 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8a40043d0 con 0x5571eabc7ef0 2024-04-26T01:53:25.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.353+0000 7fc89d600700 1 -- 172.21.15.151:0/1826313068 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fc8a0010460 con 0x5571eabc7ef0 2024-04-26T01:53:25.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.353+0000 7fc8c46b0b80 10 client.14904 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T01:53:25.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.353+0000 7fc8c46b0b80 10 client.14904 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T01:53:25.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.353+0000 7fc8c46b0b80 20 client.14904 populate_metadata read hostname 'smithi151' 2024-04-26T01:53:25.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.353+0000 7fc8c46b0b80 1 --2- 172.21.15.151:0/1826313068 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5571eabe1a40 0x5571eabe2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:25.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.353+0000 7fc8c46b0b80 4 client.14904 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:53:25.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.353+0000 7fc8c46b0b80 1 -- 172.21.15.151:0/1826313068 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5571eab27630 con 0x5571eabe1a40 2024-04-26T01:53:25.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.353+0000 7fc8b2a00700 1 --2- 172.21.15.151:0/1826313068 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5571eabe1a40 0x5571eabe2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:25.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.354+0000 7fc8b2a00700 1 --2- 172.21.15.151:0/1826313068 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5571eabe1a40 0x5571eabe2eb0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3b9a049f0b0f8ef server_cookie=29cdb8ee5b1a2e9f in_seq=0 out_seq=0 2024-04-26T01:53:25.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.354+0000 7fc89d600700 10 client.14904 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:53:25.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.355+0000 7fc89d600700 1 -- 172.21.15.151:0/1826313068 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5571eab27630 con 0x5571eabe1a40 2024-04-26T01:53:25.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.355+0000 7fc89d600700 10 client.14904 handle_command_reply: tid=0 2024-04-26T01:53:25.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.355+0000 7fc8c46b0b80 1 client.14904 shutdown 2024-04-26T01:53:25.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.355+0000 7fc8c46b0b80 1 -- 172.21.15.151:0/1826313068 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571eabc7ef0 msgr2=0x5571eabd3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:25.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.355+0000 7fc8c46b0b80 1 --2- 172.21.15.151:0/1826313068 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571eabc7ef0 0x5571eabd3c80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc8a0013fd0 tx=0x7fc8a0013730 comp rx=0 tx=0).stop 2024-04-26T01:53:25.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.355+0000 7fc8c46b0b80 1 -- 172.21.15.151:0/1826313068 shutdown_connections 2024-04-26T01:53:25.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.355+0000 7fc8c46b0b80 1 --2- 172.21.15.151:0/1826313068 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5571eabe1a40 0x5571eabe2eb0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:25.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.355+0000 7fc8c46b0b80 1 --2- 172.21.15.151:0/1826313068 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5571eabd14c0 0x5571eabd1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:25.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.355+0000 7fc89d600700 0 client.14904 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:53:25.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.355+0000 7fc8c46b0b80 1 --2- 172.21.15.151:0/1826313068 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571eabc7ef0 0x5571eabd3c80 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:25.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.355+0000 7fc8c46b0b80 1 --2- 172.21.15.151:0/1826313068 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5571eabcd780 0x5571eabd0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:25.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.355+0000 7fc8c46b0b80 1 -- 172.21.15.151:0/1826313068 >> 172.21.15.151:0/1826313068 conn(0x5571eaaf5f00 msgr2=0x5571eabc1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:25.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.356+0000 7fc8c46b0b80 1 -- 172.21.15.151:0/1826313068 shutdown_connections 2024-04-26T01:53:25.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.356+0000 7fc8c46b0b80 1 -- 172.21.15.151:0/1826313068 wait complete. 2024-04-26T01:53:25.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.356+0000 7fc8c46b0b80 20 client.14904 trim_cache size 0 max 16384 2024-04-26T01:53:25.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.379+0000 7fc8c46b0b80 1 Processor -- start 2024-04-26T01:53:25.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.379+0000 7fc8c46b0b80 1 -- start start 2024-04-26T01:53:25.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.380+0000 7fc8c46b0b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5571eabc7ef0 0x5571eabd18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:25.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.380+0000 7fc8c46b0b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571eabcd780 0x5571eabdde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:25.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.380+0000 7fc8c46b0b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5571eabd14c0 0x5571eabc25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:25.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.380+0000 7fc8c46b0b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5571eabe11f0 con 0x5571eabcd780 2024-04-26T01:53:25.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.380+0000 7fc8c46b0b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5571eabe18c0 con 0x5571eabd14c0 2024-04-26T01:53:25.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.380+0000 7fc8c46b0b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5571eabe1f90 con 0x5571eabc7ef0 2024-04-26T01:53:25.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.380+0000 7fc8b2a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571eabcd780 0x5571eabdde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:25.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.380+0000 7fc8b2a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571eabcd780 0x5571eabdde70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36864/0 (socket says 172.21.15.151:36864) 2024-04-26T01:53:25.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.380+0000 7fc8b2a00700 1 -- 172.21.15.151:0/3525110941 learned_addr learned my addr 172.21.15.151:0/3525110941 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:53:25.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.380+0000 7fc8b2000700 1 --2- 172.21.15.151:0/3525110941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5571eabc7ef0 0x5571eabd18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:25.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.380+0000 7fc8b2000700 1 --2- 172.21.15.151:0/3525110941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5571eabd14c0 0x5571eabc25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:25.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.380+0000 7fc8b2a00700 1 -- 172.21.15.151:0/3525110941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5571eabc7ef0 msgr2=0x5571eabd18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:25.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.380+0000 7fc8b2a00700 1 --2- 172.21.15.151:0/3525110941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5571eabc7ef0 0x5571eabd18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:25.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.380+0000 7fc8b2a00700 1 -- 172.21.15.151:0/3525110941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5571eabd14c0 msgr2=0x5571eabc25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:25.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.380+0000 7fc8b2a00700 1 --2- 172.21.15.151:0/3525110941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5571eabd14c0 0x5571eabc25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:25.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.380+0000 7fc8b2a00700 1 -- 172.21.15.151:0/3525110941 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571eabcba40 con 0x5571eabcd780 2024-04-26T01:53:25.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.381+0000 7fc8b2000700 1 --2- 172.21.15.151:0/3525110941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5571eabc7ef0 0x5571eabd18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:53:25.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.381+0000 7fc8b2a00700 1 --2- 172.21.15.151:0/3525110941 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571eabcd780 0x5571eabdde70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4009350 tx=0x7fc8a4005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c981b36b9e678e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:25.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.381+0000 7fc89fe00700 1 -- 172.21.15.151:0/3525110941 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a4004140 con 0x5571eabcd780 2024-04-26T01:53:25.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.381+0000 7fc89fe00700 1 -- 172.21.15.151:0/3525110941 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8a4031070 con 0x5571eabcd780 2024-04-26T01:53:25.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.381+0000 7fc89fe00700 1 -- 172.21.15.151:0/3525110941 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a4005040 con 0x5571eabcd780 2024-04-26T01:53:25.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.382+0000 7fc8c46b0b80 1 -- 172.21.15.151:0/3525110941 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571eabcd780 msgr2=0x5571eabdde70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:25.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.382+0000 7fc8c46b0b80 1 --2- 172.21.15.151:0/3525110941 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571eabcd780 0x5571eabdde70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4009350 tx=0x7fc8a4005fb0 comp rx=0 tx=0).stop 2024-04-26T01:53:25.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.382+0000 7fc8c46b0b80 1 -- 172.21.15.151:0/3525110941 shutdown_connections 2024-04-26T01:53:25.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.382+0000 7fc8c46b0b80 1 --2- 172.21.15.151:0/3525110941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5571eabd14c0 0x5571eabc25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:25.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.382+0000 7fc8c46b0b80 1 --2- 172.21.15.151:0/3525110941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5571eabc7ef0 0x5571eabd18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:25.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.382+0000 7fc8c46b0b80 1 --2- 172.21.15.151:0/3525110941 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571eabcd780 0x5571eabdde70 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:25.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.382+0000 7fc8c46b0b80 1 -- 172.21.15.151:0/3525110941 >> 172.21.15.151:0/3525110941 conn(0x5571eaaf5f00 msgr2=0x5571eabcf890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:25.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.382+0000 7fc8c46b0b80 1 -- 172.21.15.151:0/3525110941 shutdown_connections 2024-04-26T01:53:25.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.382+0000 7fc8c46b0b80 1 -- 172.21.15.151:0/3525110941 wait complete. 2024-04-26T01:53:25.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.382+0000 7fc8c46b0b80 1 -- start start 2024-04-26T01:53:25.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.383+0000 7fc8c46b0b80 1 Processor -- start 2024-04-26T01:53:25.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.383+0000 7fc8c46b0b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5571eabc7ef0 0x5571eabec330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:25.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.383+0000 7fc8c46b0b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5571eabcd780 0x5571eabec870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:25.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.383+0000 7fc8c46b0b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571eabd14c0 0x5571eabe7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:25.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.383+0000 7fc8c46b0b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5571eabe2d30 con 0x5571eabd14c0 2024-04-26T01:53:25.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.383+0000 7fc8c46b0b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5571eabe2660 con 0x5571eabcd780 2024-04-26T01:53:25.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.383+0000 7fc8c46b0b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5571eabe3400 con 0x5571eabc7ef0 2024-04-26T01:53:25.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.383+0000 7fc8c46b0b80 10 client.0 _ll_register_callbacks cb 0x5571eaab26d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:53:25.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.383+0000 7fc8b2000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571eabd14c0 0x5571eabe7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:25.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.384+0000 7fc8b2a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5571eabcd780 0x5571eabec870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:25.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.384+0000 7fc8b2a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5571eabcd780 0x5571eabec870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:43746/0 (socket says 172.21.15.151:43746) 2024-04-26T01:53:25.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.384+0000 7fc8b2000700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5571eabc7ef0 0x5571eabec330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:25.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.384+0000 7fc8b2a00700 1 -- 172.21.15.151:0/77058635 learned_addr learned my addr 172.21.15.151:0/77058635 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:53:25.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.384+0000 7fc8b2a00700 1 -- 172.21.15.151:0/77058635 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5571eabc7ef0 msgr2=0x5571eabec330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:25.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.384+0000 7fc8b2a00700 1 --2- 172.21.15.151:0/77058635 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5571eabc7ef0 0x5571eabec330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:25.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.384+0000 7fc8b2a00700 1 -- 172.21.15.151:0/77058635 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571eabd14c0 msgr2=0x5571eabe7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:25.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.384+0000 7fc8b2a00700 1 --2- 172.21.15.151:0/77058635 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571eabd14c0 0x5571eabe7360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:25.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.384+0000 7fc8b2a00700 1 -- 172.21.15.151:0/77058635 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571eabe7ba0 con 0x5571eabcd780 2024-04-26T01:53:25.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.384+0000 7fc8b2a00700 1 --2- 172.21.15.151:0/77058635 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5571eabcd780 0x5571eabec870 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4011570 tx=0x7fc8a4004140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a195866eb9061a60 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:25.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.384+0000 7fc89d600700 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T01:53:25.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.384+0000 7fc89d600700 1 -- 172.21.15.151:0/77058635 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a4039020 con 0x5571eabcd780 2024-04-26T01:53:25.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.385+0000 7fc89d600700 1 -- 172.21.15.151:0/77058635 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7fc8a4005030 con 0x5571eabcd780 2024-04-26T01:53:25.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.385+0000 7fc8c46b0b80 1 -- 172.21.15.151:0/77058635 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8a4031070 con 0x5571eabcd780 2024-04-26T01:53:25.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.385+0000 7fc89d600700 1 -- 172.21.15.151:0/77058635 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc8a4005230 con 0x5571eabcd780 2024-04-26T01:53:25.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.385+0000 7fc8c46b0b80 10 client.25004 fetch_fsmap learned FSMap version 25 2024-04-26T01:53:25.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.385+0000 7fc8c46b0b80 1 -- 172.21.15.151:0/77058635 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5571eac49730 con 0x5571eabcd780 2024-04-26T01:53:25.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.385+0000 7fc89d600700 1 -- 172.21.15.151:0/77058635 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8a4031070 con 0x5571eabcd780 2024-04-26T01:53:25.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.385+0000 7fc89d600700 1 -- 172.21.15.151:0/77058635 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fc8a4004830 con 0x5571eabcd780 2024-04-26T01:53:25.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.385+0000 7fc8c46b0b80 10 client.25004 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T01:53:25.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.385+0000 7fc8c46b0b80 10 client.25004 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T01:53:25.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.385+0000 7fc8c46b0b80 20 client.25004 populate_metadata read hostname 'smithi151' 2024-04-26T01:53:25.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.385+0000 7fc8c46b0b80 1 --2- 172.21.15.151:0/77058635 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5571eac4dc90 0x5571eac4c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:25.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.385+0000 7fc8c46b0b80 4 client.25004 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T01:53:25.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.385+0000 7fc8c46b0b80 1 -- 172.21.15.151:0/77058635 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "status"}) -- 0x5571eab216d0 con 0x5571eac4dc90 2024-04-26T01:53:25.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.386+0000 7fc8b2000700 1 --2- 172.21.15.151:0/77058635 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5571eac4dc90 0x5571eac4c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:25.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.386+0000 7fc8b2000700 1 --2- 172.21.15.151:0/77058635 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5571eac4dc90 0x5571eac4c9a0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5d9a5f630da889a server_cookie=a6d7448862502ce in_seq=0 out_seq=0 2024-04-26T01:53:25.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.386+0000 7fc89d600700 10 client.25004 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:53:25.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.386+0000 7fc89d600700 1 -- 172.21.15.151:0/77058635 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5571eab216d0 con 0x5571eac4dc90 2024-04-26T01:53:25.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.386+0000 7fc89d600700 10 client.25004 handle_command_reply: tid=0 2024-04-26T01:53:25.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.386+0000 7fc8c46b0b80 1 client.25004 shutdown 2024-04-26T01:53:25.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.387+0000 7fc8c46b0b80 1 -- 172.21.15.151:0/77058635 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5571eabcd780 msgr2=0x5571eabec870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:25.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.387+0000 7fc8c46b0b80 1 --2- 172.21.15.151:0/77058635 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5571eabcd780 0x5571eabec870 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4011570 tx=0x7fc8a4004140 comp rx=0 tx=0).stop 2024-04-26T01:53:25.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.387+0000 7fc8c46b0b80 1 -- 172.21.15.151:0/77058635 shutdown_connections 2024-04-26T01:53:25.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.387+0000 7fc8c46b0b80 1 --2- 172.21.15.151:0/77058635 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5571eabd14c0 0x5571eabe7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:25.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.387+0000 7fc8c46b0b80 1 --2- 172.21.15.151:0/77058635 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5571eabc7ef0 0x5571eabec330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:25.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.387+0000 7fc8c46b0b80 1 --2- 172.21.15.151:0/77058635 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5571eac4dc90 0x5571eac4c9a0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:25.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.387+0000 7fc8c46b0b80 1 --2- 172.21.15.151:0/77058635 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5571eabcd780 0x5571eabec870 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:25.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.387+0000 7fc89d600700 0 client.25004 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:53:25.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.387+0000 7fc8c46b0b80 1 -- 172.21.15.151:0/77058635 >> 172.21.15.151:0/77058635 conn(0x5571eaaf5f00 msgr2=0x5571eabcf480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:25.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.387+0000 7fc8c46b0b80 1 -- 172.21.15.151:0/77058635 shutdown_connections 2024-04-26T01:53:25.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.387+0000 7fc8c46b0b80 1 -- 172.21.15.151:0/77058635 wait complete. 2024-04-26T01:53:25.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.387+0000 7fc8c46b0b80 20 client.25004 trim_cache size 0 max 16384 2024-04-26T01:53:25.389 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-26T01:53:25.390 INFO:teuthology.orchestra.run.smithi151.stdout: "cluster_fsid": "c0e49e64-036d-11ef-bc93-c7b262605968", 2024-04-26T01:53:25.390 INFO:teuthology.orchestra.run.smithi151.stdout: "whoami": 0, 2024-04-26T01:53:25.390 INFO:teuthology.orchestra.run.smithi151.stdout: "id": 24509, 2024-04-26T01:53:25.390 INFO:teuthology.orchestra.run.smithi151.stdout: "want_state": "up:active", 2024-04-26T01:53:25.390 INFO:teuthology.orchestra.run.smithi151.stdout: "state": "up:active", 2024-04-26T01:53:25.390 INFO:teuthology.orchestra.run.smithi151.stdout: "fs_name": "cephfs", 2024-04-26T01:53:25.390 INFO:teuthology.orchestra.run.smithi151.stdout: "rank_uptime": 223.71247067499999, 2024-04-26T01:53:25.390 INFO:teuthology.orchestra.run.smithi151.stdout: "mdsmap_epoch": 25, 2024-04-26T01:53:25.390 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch": 92, 2024-04-26T01:53:25.390 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch_barrier": 90, 2024-04-26T01:53:25.390 INFO:teuthology.orchestra.run.smithi151.stdout: "uptime": 247.27203549999999 2024-04-26T01:53:25.390 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-26T01:53:25.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.388+0000 7fc8b9e00700 1 -- 172.21.15.151:0/1215709431 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fc8980a3cc0 msgr2=0x7fc8980a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:25.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.388+0000 7fc8b9e00700 1 --2- 172.21.15.151:0/1215709431 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fc8980a3cc0 0x7fc8980a6180 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac004fd0 tx=0x7fc8ac00b330 comp rx=0 tx=0).stop 2024-04-26T01:53:25.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.388+0000 7fc8b9e00700 1 -- 172.21.15.151:0/1215709431 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8b410e3b0 msgr2=0x7fc8b41a3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:25.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.388+0000 7fc8b9e00700 1 --2- 172.21.15.151:0/1215709431 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8b410e3b0 0x7fc8b41a3fa0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac009790 tx=0x7fc8ac00cc30 comp rx=0 tx=0).stop 2024-04-26T01:53:25.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.388+0000 7fc8b9e00700 1 -- 172.21.15.151:0/1215709431 shutdown_connections 2024-04-26T01:53:25.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.388+0000 7fc8b9e00700 1 --2- 172.21.15.151:0/1215709431 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fc8980a3cc0 0x7fc8980a6180 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:25.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.388+0000 7fc8b9e00700 1 --2- 172.21.15.151:0/1215709431 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8b410e3b0 0x7fc8b41a3fa0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:25.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.388+0000 7fc8b9e00700 1 --2- 172.21.15.151:0/1215709431 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8b41092d0 0x7fc8b419f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:25.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.388+0000 7fc8b9e00700 1 --2- 172.21.15.151:0/1215709431 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8b4109c80 0x7fc8b419fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:25.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.388+0000 7fc8b9e00700 1 -- 172.21.15.151:0/1215709431 >> 172.21.15.151:0/1215709431 conn(0x7fc8b4076210 msgr2=0x7fc8b4076750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:25.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.388+0000 7fc8b9e00700 1 -- 172.21.15.151:0/1215709431 shutdown_connections 2024-04-26T01:53:25.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:25.388+0000 7fc8b9e00700 1 -- 172.21.15.151:0/1215709431 wait complete. 2024-04-26T01:53:25.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:25 smithi172 ceph-mon[32785]: pgmap v441: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:53:25.595 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:25 smithi165 ceph-mon[32648]: pgmap v441: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:53:25.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:25 smithi151 ceph-mon[28057]: pgmap v441: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:53:26.343 INFO:teuthology.run_tasks:Running task sequential... 2024-04-26T01:53:26.353 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-26T01:53:26.353 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-26T01:53:26.354 INFO:teuthology.task.print:Enabling mgr modules 2024-04-26T01:53:26.354 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-26T01:53:26.355 INFO:teuthology.task.exec:Executing custom commands... 2024-04-26T01:53:26.355 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi151.front.sepia.ceph.com 2024-04-26T01:53:26.355 DEBUG:teuthology.orchestra.run.smithi151:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-26T01:53:26.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:26.565+0000 7fcccd400640 1 -- 172.21.15.151:0/793678598 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fccc8109fb0 msgr2=0x7fccc8110840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:26.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:26.565+0000 7fcccd400640 1 --2- 172.21.15.151:0/793678598 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fccc8109fb0 0x7fccc8110840 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fccc0009ca0 tx=0x7fccc002a8c0 comp rx=0 tx=0).stop 2024-04-26T01:53:26.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:26.565+0000 7fcccd400640 1 -- 172.21.15.151:0/793678598 shutdown_connections 2024-04-26T01:53:26.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:26.565+0000 7fcccd400640 1 --2- 172.21.15.151:0/793678598 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fccc8109fb0 0x7fccc8110840 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:26.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:26.565+0000 7fcccd400640 1 --2- 172.21.15.151:0/793678598 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fccc81095f0 0x7fccc8109a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:26.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:26.565+0000 7fcccd400640 1 --2- 172.21.15.151:0/793678598 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fccc8108400 0x7fccc8108820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:26.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:26.565+0000 7fcccd400640 1 -- 172.21.15.151:0/793678598 >> 172.21.15.151:0/793678598 conn(0x7fccc80773e0 msgr2=0x7fccc8079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:26.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:26.565+0000 7fcccd400640 1 -- 172.21.15.151:0/793678598 shutdown_connections 2024-04-26T01:53:26.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:26.565+0000 7fcccd400640 1 -- 172.21.15.151:0/793678598 wait complete. 2024-04-26T01:53:26.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:26.566+0000 7fcccd400640 1 Processor -- start 2024-04-26T01:53:26.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:26.566+0000 7fcccd400640 1 -- start start 2024-04-26T01:53:26.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:26.567+0000 7fcccd400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fccc8108400 0x7fccc819e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:26.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:26.567+0000 7fcccd400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fccc81095f0 0x7fccc819ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:26.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:26.567+0000 7fcccd400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fccc8109fb0 0x7fccc81a5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:26.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:26.567+0000 7fcccd400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fccc8113f90 con 0x7fccc8109fb0 2024-04-26T01:53:26.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:26.567+0000 7fcccd400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fccc8113e10 con 0x7fccc8108400 2024-04-26T01:53:26.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:26.567+0000 7fcccd400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fccc8114110 con 0x7fccc81095f0 2024-04-26T01:53:26.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:26.567+0000 7fcccca00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fccc8109fb0 0x7fccc81a5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:26.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:26.567+0000 7fccbf400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fccc81095f0 0x7fccc819ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:26.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:26.567+0000 7fccc7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fccc8108400 0x7fccc819e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:26.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:26.567+0000 7fcccca00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fccc8109fb0 0x7fccc81a5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36892/0 (socket says 172.21.15.151:36892) 2024-04-26T01:53:26.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:26.567+0000 7fcccca00640 1 -- 172.21.15.151:0/488467325 learned_addr learned my addr 172.21.15.151:0/488467325 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:53:26.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:26.567+0000 7fccbf400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fccc81095f0 0x7fccc819ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:43268/0 (socket says 172.21.15.151:43268) 2024-04-26T01:53:26.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:26.567+0000 7fccbf400640 1 -- 172.21.15.151:0/488467325 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fccc8108400 msgr2=0x7fccc819e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:26.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:26.567+0000 7fccbf400640 1 --2- 172.21.15.151:0/488467325 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fccc8108400 0x7fccc819e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:26.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:26.568+0000 7fccbf400640 1 -- 172.21.15.151:0/488467325 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fccc8109fb0 msgr2=0x7fccc81a5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:26.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:26.568+0000 7fccbf400640 1 --2- 172.21.15.151:0/488467325 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fccc8109fb0 0x7fccc81a5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:26.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:26.568+0000 7fccbf400640 1 -- 172.21.15.151:0/488467325 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccc81a61b0 con 0x7fccc81095f0 2024-04-26T01:53:26.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:26.568+0000 7fccbf400640 1 --2- 172.21.15.151:0/488467325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fccc81095f0 0x7fccc819ead0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fccb000e9f0 tx=0x7fccb000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:26.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:26.568+0000 7fccc5600640 1 -- 172.21.15.151:0/488467325 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb000cd80 con 0x7fccc81095f0 2024-04-26T01:53:26.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:26.568+0000 7fccc5600640 1 -- 172.21.15.151:0/488467325 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccb00035f0 con 0x7fccc81095f0 2024-04-26T01:53:26.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:26.568+0000 7fcccd400640 1 -- 172.21.15.151:0/488467325 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccc81a64a0 con 0x7fccc81095f0 2024-04-26T01:53:26.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:26.568+0000 7fccc5600640 1 -- 172.21.15.151:0/488467325 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb0010670 con 0x7fccc81095f0 2024-04-26T01:53:26.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:26.568+0000 7fcccd400640 1 -- 172.21.15.151:0/488467325 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccc81a6a40 con 0x7fccc81095f0 2024-04-26T01:53:26.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:26.569+0000 7fcccd400640 1 -- 172.21.15.151:0/488467325 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccc810f7a0 con 0x7fccc81095f0 2024-04-26T01:53:26.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:26.571+0000 7fccc5600640 1 -- 172.21.15.151:0/488467325 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7fccb00037a0 con 0x7fccc81095f0 2024-04-26T01:53:26.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:26.572+0000 7fccc5600640 1 --2- 172.21.15.151:0/488467325 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fccac0a3c20 0x7fccac0a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:26.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:26.573+0000 7fccc5600640 1 -- 172.21.15.151:0/488467325 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7fccb00d05d0 con 0x7fccc81095f0 2024-04-26T01:53:26.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:26.573+0000 7fccc7e00640 1 --2- 172.21.15.151:0/488467325 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fccac0a3c20 0x7fccac0a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:26.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:26.573+0000 7fccc7e00640 1 --2- 172.21.15.151:0/488467325 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fccac0a3c20 0x7fccac0a60e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fccb80095f0 tx=0x7fccb8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:26.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:26.577+0000 7fccc5600640 1 -- 172.21.15.151:0/488467325 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fccb009cfb0 con 0x7fccc81095f0 2024-04-26T01:53:26.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:26.833+0000 7fcccd400640 1 -- 172.21.15.151:0/488467325 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7fccc8062b80 con 0x7fccc81095f0 2024-04-26T01:53:27.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.262+0000 7fccc5600640 1 -- 172.21.15.151:0/488467325 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mgrmap(e 19) ==== 138664+0+0 (secure 0 0 0) 0x7fccb0009310 con 0x7fccc81095f0 2024-04-26T01:53:27.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.263+0000 7fccc5600640 1 -- 172.21.15.151:0/488467325 <== mon.1 v2:172.21.15.172:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7fccb004d630 con 0x7fccc81095f0 2024-04-26T01:53:27.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.268+0000 7fcccd400640 1 -- 172.21.15.151:0/488467325 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fccac0a3c20 msgr2=0x7fccac0a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:27.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.268+0000 7fcccd400640 1 --2- 172.21.15.151:0/488467325 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fccac0a3c20 0x7fccac0a60e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fccb80095f0 tx=0x7fccb8005f90 comp rx=0 tx=0).stop 2024-04-26T01:53:27.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.268+0000 7fcccd400640 1 -- 172.21.15.151:0/488467325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fccc81095f0 msgr2=0x7fccc819ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:27.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.268+0000 7fcccd400640 1 --2- 172.21.15.151:0/488467325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fccc81095f0 0x7fccc819ead0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fccb000e9f0 tx=0x7fccb000eeb0 comp rx=0 tx=0).stop 2024-04-26T01:53:27.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.268+0000 7fcccd400640 1 -- 172.21.15.151:0/488467325 shutdown_connections 2024-04-26T01:53:27.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.268+0000 7fcccd400640 1 --2- 172.21.15.151:0/488467325 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fccc8109fb0 0x7fccc81a5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:27.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.268+0000 7fcccd400640 1 --2- 172.21.15.151:0/488467325 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fccac0a3c20 0x7fccac0a60e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:27.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.268+0000 7fcccd400640 1 --2- 172.21.15.151:0/488467325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fccc81095f0 0x7fccc819ead0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:27.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.268+0000 7fcccd400640 1 --2- 172.21.15.151:0/488467325 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fccc8108400 0x7fccc819e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:27.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.268+0000 7fcccd400640 1 -- 172.21.15.151:0/488467325 >> 172.21.15.151:0/488467325 conn(0x7fccc80773e0 msgr2=0x7fccc810c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:27.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.268+0000 7fcccd400640 1 -- 172.21.15.151:0/488467325 shutdown_connections 2024-04-26T01:53:27.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.268+0000 7fcccd400640 1 -- 172.21.15.151:0/488467325 wait complete. 2024-04-26T01:53:27.287 DEBUG:teuthology.orchestra.run.smithi151:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-26T01:53:27.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.466+0000 7ffa8f600640 1 -- 172.21.15.151:0/2138532306 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa88105d90 msgr2=0x7ffa8810c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:27.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.466+0000 7ffa8f600640 1 --2- 172.21.15.151:0/2138532306 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa88105d90 0x7ffa8810c620 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ffa80009ca0 tx=0x7ffa8002a8c0 comp rx=0 tx=0).stop 2024-04-26T01:53:27.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.466+0000 7ffa8f600640 1 -- 172.21.15.151:0/2138532306 shutdown_connections 2024-04-26T01:53:27.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.466+0000 7ffa8f600640 1 --2- 172.21.15.151:0/2138532306 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa88105d90 0x7ffa8810c620 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:27.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.466+0000 7ffa8f600640 1 --2- 172.21.15.151:0/2138532306 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa881053d0 0x7ffa88105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:27.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.466+0000 7ffa8f600640 1 --2- 172.21.15.151:0/2138532306 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa881041e0 0x7ffa88104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:27.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.466+0000 7ffa8f600640 1 -- 172.21.15.151:0/2138532306 >> 172.21.15.151:0/2138532306 conn(0x7ffa880ff770 msgr2=0x7ffa88101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:27.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.466+0000 7ffa8f600640 1 -- 172.21.15.151:0/2138532306 shutdown_connections 2024-04-26T01:53:27.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.467+0000 7ffa8f600640 1 -- 172.21.15.151:0/2138532306 wait complete. 2024-04-26T01:53:27.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.468+0000 7ffa8f600640 1 Processor -- start 2024-04-26T01:53:27.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.468+0000 7ffa8f600640 1 -- start start 2024-04-26T01:53:27.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.469+0000 7ffa8f600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa881041e0 0x7ffa8819a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:27.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.469+0000 7ffa8f600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa881053d0 0x7ffa8819a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:27.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.469+0000 7ffa8f600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa88105d90 0x7ffa881a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:27.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.469+0000 7ffa8f600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8810fd70 con 0x7ffa88105d90 2024-04-26T01:53:27.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.469+0000 7ffa8f600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8810fbf0 con 0x7ffa881053d0 2024-04-26T01:53:27.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.469+0000 7ffa8f600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8810fef0 con 0x7ffa881041e0 2024-04-26T01:53:27.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.469+0000 7ffa87e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa881053d0 0x7ffa8819a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:27.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.469+0000 7ffa8cc00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa881041e0 0x7ffa8819a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:27.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.469+0000 7ffa87e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa881053d0 0x7ffa8819a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:43792/0 (socket says 172.21.15.151:43792) 2024-04-26T01:53:27.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.469+0000 7ffa87e00640 1 -- 172.21.15.151:0/1119145815 learned_addr learned my addr 172.21.15.151:0/1119145815 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:53:27.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.469+0000 7ffa8d600640 1 --2- 172.21.15.151:0/1119145815 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa88105d90 0x7ffa881a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:27.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.469+0000 7ffa87e00640 1 -- 172.21.15.151:0/1119145815 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa881041e0 msgr2=0x7ffa8819a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:27.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.469+0000 7ffa87e00640 1 --2- 172.21.15.151:0/1119145815 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa881041e0 0x7ffa8819a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:27.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.469+0000 7ffa87e00640 1 -- 172.21.15.151:0/1119145815 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa88105d90 msgr2=0x7ffa881a1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:27.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.469+0000 7ffa87e00640 1 --2- 172.21.15.151:0/1119145815 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa88105d90 0x7ffa881a1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:27.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.469+0000 7ffa87e00640 1 -- 172.21.15.151:0/1119145815 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa881a1ed0 con 0x7ffa881053d0 2024-04-26T01:53:27.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.469+0000 7ffa8cc00640 1 --2- 172.21.15.151:0/1119145815 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa881041e0 0x7ffa8819a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:53:27.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.470+0000 7ffa87e00640 1 --2- 172.21.15.151:0/1119145815 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa881053d0 0x7ffa8819a7f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ffa7400e9f0 tx=0x7ffa7400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:27.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.470+0000 7ffa85600640 1 -- 172.21.15.151:0/1119145815 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7400cd80 con 0x7ffa881053d0 2024-04-26T01:53:27.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.470+0000 7ffa8f600640 1 -- 172.21.15.151:0/1119145815 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa881a21c0 con 0x7ffa881053d0 2024-04-26T01:53:27.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.470+0000 7ffa85600640 1 -- 172.21.15.151:0/1119145815 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa740054d0 con 0x7ffa881053d0 2024-04-26T01:53:27.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.470+0000 7ffa85600640 1 -- 172.21.15.151:0/1119145815 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa74010690 con 0x7ffa881053d0 2024-04-26T01:53:27.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.470+0000 7ffa8f600640 1 -- 172.21.15.151:0/1119145815 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa881a2710 con 0x7ffa881053d0 2024-04-26T01:53:27.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.471+0000 7ffa8f600640 1 -- 172.21.15.151:0/1119145815 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa8810b4a0 con 0x7ffa881053d0 2024-04-26T01:53:27.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.475+0000 7ffa85600640 1 -- 172.21.15.151:0/1119145815 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 19) ==== 138664+0+0 (secure 0 0 0) 0x7ffa74004e90 con 0x7ffa881053d0 2024-04-26T01:53:27.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.476+0000 7ffa85600640 1 --2- 172.21.15.151:0/1119145815 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7ffa700a3ca0 0x7ffa700a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:27.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.476+0000 7ffa8cc00640 1 -- 172.21.15.151:0/1119145815 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7ffa700a3ca0 msgr2=0x7ffa700a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.151:6800/2251256505 2024-04-26T01:53:27.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.476+0000 7ffa85600640 1 -- 172.21.15.151:0/1119145815 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7ffa740d0880 con 0x7ffa881053d0 2024-04-26T01:53:27.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.476+0000 7ffa8cc00640 1 --2- 172.21.15.151:0/1119145815 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7ffa700a3ca0 0x7ffa700a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T01:53:27.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.484+0000 7ffa85600640 1 -- 172.21.15.151:0/1119145815 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffa7409d2e0 con 0x7ffa881053d0 2024-04-26T01:53:27.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:27 smithi172 ceph-mon[32785]: pgmap v442: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:53:27.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:27 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/488467325' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-26T01:53:27.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:27 smithi172 ceph-mon[32785]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-26T01:53:27.537 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 01:53:27 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-z[34652]: ignoring --setuser ceph since I am not root 2024-04-26T01:53:27.537 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 01:53:27 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-z[34652]: ignoring --setgroup ceph since I am not root 2024-04-26T01:53:27.537 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 01:53:27 smithi172 ceph-mgr[34656]: -- 172.21.15.172:0/556944193 <== mon.1 v2:172.21.15.172:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56387cb805a0 con 0x56387cb54c00 2024-04-26T01:53:27.537 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 01:53:27 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-z[34652]: 2024-04-26T01:53:27.431+0000 7f62c8568200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T01:53:27.583 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:27 smithi151 ceph-mon[28057]: pgmap v442: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:53:27.584 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:27 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/488467325' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-26T01:53:27.584 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:27 smithi151 ceph-mon[28057]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-26T01:53:27.584 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:53:27 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: ignoring --setuser ceph since I am not root 2024-04-26T01:53:27.584 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:53:27 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: ignoring --setgroup ceph since I am not root 2024-04-26T01:53:27.584 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:53:27 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:53:27.584+0000 7fba6fdfe200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T01:53:27.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:27 smithi165 ceph-mon[32648]: pgmap v442: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:53:27.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:27 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/488467325' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-26T01:53:27.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:27 smithi165 ceph-mon[32648]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-26T01:53:27.596 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 01:53:27 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-y[34029]: ignoring --setuser ceph since I am not root 2024-04-26T01:53:27.596 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 01:53:27 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-y[34029]: ignoring --setgroup ceph since I am not root 2024-04-26T01:53:27.596 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 01:53:27 smithi165 ceph-mgr[34033]: -- 172.21.15.165:0/1411103629 <== mon.2 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562c424da5a0 con 0x562c424ae800 2024-04-26T01:53:27.596 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 01:53:27 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-y[34029]: 2024-04-26T01:53:27.433+0000 7fb5add29200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T01:53:27.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.677+0000 7ffa8cc00640 1 -- 172.21.15.151:0/1119145815 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7ffa700a3ca0 msgr2=0x7ffa700a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.151:6800/2251256505 2024-04-26T01:53:27.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.677+0000 7ffa8cc00640 1 --2- 172.21.15.151:0/1119145815 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7ffa700a3ca0 0x7ffa700a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T01:53:27.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.690+0000 7ffa8f600640 1 -- 172.21.15.151:0/1119145815 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7ffa88062b50 con 0x7ffa881053d0 2024-04-26T01:53:27.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.698+0000 7ffa85600640 1 -- 172.21.15.151:0/1119145815 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v30)=0 v30) ==== 138+0+0 (secure 0 0 0) 0x7ffa7409cbe0 con 0x7ffa881053d0 2024-04-26T01:53:27.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.701+0000 7ffa8f600640 1 -- 172.21.15.151:0/1119145815 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7ffa700a3ca0 msgr2=0x7ffa700a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T01:53:27.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.701+0000 7ffa8f600640 1 --2- 172.21.15.151:0/1119145815 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7ffa700a3ca0 0x7ffa700a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:27.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.701+0000 7ffa8f600640 1 -- 172.21.15.151:0/1119145815 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa881053d0 msgr2=0x7ffa8819a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:27.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.701+0000 7ffa8f600640 1 --2- 172.21.15.151:0/1119145815 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa881053d0 0x7ffa8819a7f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ffa7400e9f0 tx=0x7ffa7400edf0 comp rx=0 tx=0).stop 2024-04-26T01:53:27.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.701+0000 7ffa8f600640 1 -- 172.21.15.151:0/1119145815 shutdown_connections 2024-04-26T01:53:27.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.701+0000 7ffa8f600640 1 --2- 172.21.15.151:0/1119145815 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa88105d90 0x7ffa881a1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:27.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.701+0000 7ffa8f600640 1 --2- 172.21.15.151:0/1119145815 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa881053d0 0x7ffa8819a7f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:27.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.701+0000 7ffa8f600640 1 --2- 172.21.15.151:0/1119145815 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7ffa700a3ca0 0x7ffa700a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:27.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.701+0000 7ffa8f600640 1 --2- 172.21.15.151:0/1119145815 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa881041e0 0x7ffa8819a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:27.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.701+0000 7ffa8f600640 1 -- 172.21.15.151:0/1119145815 >> 172.21.15.151:0/1119145815 conn(0x7ffa880ff770 msgr2=0x7ffa88108670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:27.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.701+0000 7ffa8f600640 1 -- 172.21.15.151:0/1119145815 shutdown_connections 2024-04-26T01:53:27.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.701+0000 7ffa8f600640 1 -- 172.21.15.151:0/1119145815 wait complete. 2024-04-26T01:53:27.717 DEBUG:teuthology.orchestra.run.smithi151:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-04-26T01:53:27.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.894+0000 7fe1be600640 1 -- 172.21.15.151:0/2769603028 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe1b8105360 msgr2=0x7fe1b8107750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:27.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.894+0000 7fe1be600640 1 --2- 172.21.15.151:0/2769603028 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe1b8105360 0x7fe1b8107750 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe1a4009900 tx=0x7fe1a402a410 comp rx=0 tx=0).stop 2024-04-26T01:53:27.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.894+0000 7fe1be600640 1 -- 172.21.15.151:0/2769603028 shutdown_connections 2024-04-26T01:53:27.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.894+0000 7fe1be600640 1 --2- 172.21.15.151:0/2769603028 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe1b8107c90 0x7fe1b810a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:27.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.894+0000 7fe1be600640 1 --2- 172.21.15.151:0/2769603028 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe1b8105360 0x7fe1b8107750 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:27.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.894+0000 7fe1be600640 1 --2- 172.21.15.151:0/2769603028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1b8102a30 0x7fe1b8104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:27.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.894+0000 7fe1be600640 1 -- 172.21.15.151:0/2769603028 >> 172.21.15.151:0/2769603028 conn(0x7fe1b80fc440 msgr2=0x7fe1b80fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:27.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.894+0000 7fe1be600640 1 -- 172.21.15.151:0/2769603028 shutdown_connections 2024-04-26T01:53:27.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.894+0000 7fe1be600640 1 -- 172.21.15.151:0/2769603028 wait complete. 2024-04-26T01:53:27.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.895+0000 7fe1be600640 1 Processor -- start 2024-04-26T01:53:27.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.895+0000 7fe1be600640 1 -- start start 2024-04-26T01:53:27.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.895+0000 7fe1be600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe1b8102a30 0x7fe1b8198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:27.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.896+0000 7fe1be600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe1b8105360 0x7fe1b8198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:27.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.896+0000 7fe1be600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1b8107c90 0x7fe1b819f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:27.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.896+0000 7fe1be600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b810db40 con 0x7fe1b8102a30 2024-04-26T01:53:27.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.896+0000 7fe1be600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b810d9c0 con 0x7fe1b8105360 2024-04-26T01:53:27.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.896+0000 7fe1be600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b810dcc0 con 0x7fe1b8107c90 2024-04-26T01:53:27.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.896+0000 7fe1b6a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe1b8105360 0x7fe1b8198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:27.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.896+0000 7fe1b6a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe1b8105360 0x7fe1b8198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:43810/0 (socket says 172.21.15.151:43810) 2024-04-26T01:53:27.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.896+0000 7fe1b6a00640 1 -- 172.21.15.151:0/2222736594 learned_addr learned my addr 172.21.15.151:0/2222736594 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:53:27.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.896+0000 7fe1b7e00640 1 --2- 172.21.15.151:0/2222736594 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1b8107c90 0x7fe1b819f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:27.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.896+0000 7fe1b7400640 1 --2- 172.21.15.151:0/2222736594 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe1b8102a30 0x7fe1b8198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:27.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.896+0000 7fe1b6a00640 1 -- 172.21.15.151:0/2222736594 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1b8107c90 msgr2=0x7fe1b819f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:27.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.896+0000 7fe1b6a00640 1 --2- 172.21.15.151:0/2222736594 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1b8107c90 0x7fe1b819f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:27.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.896+0000 7fe1b6a00640 1 -- 172.21.15.151:0/2222736594 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe1b8102a30 msgr2=0x7fe1b8198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:27.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.896+0000 7fe1b6a00640 1 --2- 172.21.15.151:0/2222736594 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe1b8102a30 0x7fe1b8198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:27.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.896+0000 7fe1b6a00640 1 -- 172.21.15.151:0/2222736594 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1b819fd20 con 0x7fe1b8105360 2024-04-26T01:53:27.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.897+0000 7fe1b6a00640 1 --2- 172.21.15.151:0/2222736594 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe1b8105360 0x7fe1b8198640 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe1a4031500 tx=0x7fe1a4003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:27.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.897+0000 7fe1afe00640 1 -- 172.21.15.151:0/2222736594 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a403f070 con 0x7fe1b8105360 2024-04-26T01:53:27.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.897+0000 7fe1afe00640 1 -- 172.21.15.151:0/2222736594 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1a4031c50 con 0x7fe1b8105360 2024-04-26T01:53:27.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.897+0000 7fe1be600640 1 -- 172.21.15.151:0/2222736594 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1b819ffb0 con 0x7fe1b8105360 2024-04-26T01:53:27.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.897+0000 7fe1afe00640 1 -- 172.21.15.151:0/2222736594 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a4035030 con 0x7fe1b8105360 2024-04-26T01:53:27.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.897+0000 7fe1be600640 1 -- 172.21.15.151:0/2222736594 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1b81a0410 con 0x7fe1b8105360 2024-04-26T01:53:27.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.898+0000 7fe1be600640 1 -- 172.21.15.151:0/2222736594 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1b80fdf30 con 0x7fe1b8105360 2024-04-26T01:53:27.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.900+0000 7fe1afe00640 1 -- 172.21.15.151:0/2222736594 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 19) ==== 138664+0+0 (secure 0 0 0) 0x7fe1a4042070 con 0x7fe1b8105360 2024-04-26T01:53:27.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.901+0000 7fe1afe00640 1 --2- 172.21.15.151:0/2222736594 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fe19c0a3ca0 0x7fe19c0a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:27.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.901+0000 7fe1b7400640 1 -- 172.21.15.151:0/2222736594 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fe19c0a3ca0 msgr2=0x7fe19c0a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.151:6800/2251256505 2024-04-26T01:53:27.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.902+0000 7fe1afe00640 1 -- 172.21.15.151:0/2222736594 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7fe1a40f11e0 con 0x7fe1b8105360 2024-04-26T01:53:27.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.902+0000 7fe1b7400640 1 --2- 172.21.15.151:0/2222736594 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fe19c0a3ca0 0x7fe19c0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T01:53:27.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:27.908+0000 7fe1afe00640 1 -- 172.21.15.151:0/2222736594 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe1a40bd570 con 0x7fe1b8105360 2024-04-26T01:53:28.021 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 01:53:27 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-z[34652]: 2024-04-26T01:53:27.614+0000 7f62c8568200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T01:53:28.021 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 01:53:27 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-z[34652]: 2024-04-26T01:53:27.716+0000 7f62c8568200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T01:53:28.038 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 01:53:27 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-y[34029]: 2024-04-26T01:53:27.621+0000 7fb5add29200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T01:53:28.038 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 01:53:27 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-y[34029]: 2024-04-26T01:53:27.726+0000 7fb5add29200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T01:53:28.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:28.103+0000 7fe1b7400640 1 -- 172.21.15.151:0/2222736594 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fe19c0a3ca0 msgr2=0x7fe19c0a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.151:6800/2251256505 2024-04-26T01:53:28.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:28.103+0000 7fe1b7400640 1 --2- 172.21.15.151:0/2222736594 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fe19c0a3ca0 0x7fe19c0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T01:53:28.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:28.111+0000 7fe1be600640 1 -- 172.21.15.151:0/2222736594 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7fe1b8062c60 con 0x7fe1b8105360 2024-04-26T01:53:28.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:28.119+0000 7fe1afe00640 1 -- 172.21.15.151:0/2222736594 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v31)=0 v31) ==== 133+0+0 (secure 0 0 0) 0x7fe1a40c1870 con 0x7fe1b8105360 2024-04-26T01:53:28.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:28.122+0000 7fe1be600640 1 -- 172.21.15.151:0/2222736594 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fe19c0a3ca0 msgr2=0x7fe19c0a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T01:53:28.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:28.122+0000 7fe1be600640 1 --2- 172.21.15.151:0/2222736594 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fe19c0a3ca0 0x7fe19c0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:28.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:28.122+0000 7fe1be600640 1 -- 172.21.15.151:0/2222736594 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe1b8105360 msgr2=0x7fe1b8198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:28.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:28.122+0000 7fe1be600640 1 --2- 172.21.15.151:0/2222736594 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe1b8105360 0x7fe1b8198640 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe1a4031500 tx=0x7fe1a4003c10 comp rx=0 tx=0).stop 2024-04-26T01:53:28.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:28.122+0000 7fe1be600640 1 -- 172.21.15.151:0/2222736594 shutdown_connections 2024-04-26T01:53:28.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:28.122+0000 7fe1be600640 1 --2- 172.21.15.151:0/2222736594 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7fe19c0a3ca0 0x7fe19c0a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:28.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:28.122+0000 7fe1be600640 1 --2- 172.21.15.151:0/2222736594 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1b8107c90 0x7fe1b819f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:28.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:28.122+0000 7fe1be600640 1 --2- 172.21.15.151:0/2222736594 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe1b8105360 0x7fe1b8198640 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:28.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:28.122+0000 7fe1be600640 1 --2- 172.21.15.151:0/2222736594 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe1b8102a30 0x7fe1b8198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:28.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:28.122+0000 7fe1be600640 1 -- 172.21.15.151:0/2222736594 >> 172.21.15.151:0/2222736594 conn(0x7fe1b80fc440 msgr2=0x7fe1b8103470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:28.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:28.122+0000 7fe1be600640 1 -- 172.21.15.151:0/2222736594 shutdown_connections 2024-04-26T01:53:28.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:28.122+0000 7fe1be600640 1 -- 172.21.15.151:0/2222736594 wait complete. 2024-04-26T01:53:28.122 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:53:27 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:53:27.840+0000 7fba6fdfe200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T01:53:28.122 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:53:28 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:53:28.002+0000 7fba6fdfe200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T01:53:28.139 INFO:teuthology.run_tasks:Running task kclient... 2024-04-26T01:53:28.164 INFO:tasks.kclient:Mounting kernel clients... 2024-04-26T01:53:28.164 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-26T01:53:28.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T01:53:28.165 DEBUG:teuthology.orchestra.run.smithi151:> ip netns list 2024-04-26T01:53:28.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T01:53:28.210 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip link delete ceph-brx 2024-04-26T01:53:28.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:28 smithi172 ceph-mon[32785]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-26T01:53:28.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:28 smithi172 ceph-mon[32785]: mgrmap e19: x(active, since 12m), standbys: y, z 2024-04-26T01:53:28.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:28 smithi172 ceph-mon[32785]: from='client.? ' entity='client.admin' 2024-04-26T01:53:28.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:28 smithi172 ceph-mon[32785]: from='client.? ' entity='client.admin' 2024-04-26T01:53:28.287 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 01:53:28 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-z[34652]: 2024-04-26T01:53:28.022+0000 7f62c8568200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T01:53:28.287 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 01:53:28 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-z[34652]: 2024-04-26T01:53:28.107+0000 7f62c8568200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T01:53:28.322 INFO:teuthology.orchestra.run.smithi151.stderr:Cannot find device "ceph-brx" 2024-04-26T01:53:28.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T01:53:28.326 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-04-26T01:53:28.326 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T01:53:28.326 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T01:53:28.326 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-26T01:53:28.346 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 01:53:28 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-y[34029]: 2024-04-26T01:53:28.039+0000 7fb5add29200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T01:53:28.346 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 01:53:28 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-y[34029]: 2024-04-26T01:53:28.129+0000 7fb5add29200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T01:53:28.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:28 smithi165 ceph-mon[32648]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-26T01:53:28.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:28 smithi165 ceph-mon[32648]: mgrmap e19: x(active, since 12m), standbys: y, z 2024-04-26T01:53:28.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:28 smithi165 ceph-mon[32648]: from='client.? ' entity='client.admin' 2024-04-26T01:53:28.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:28 smithi165 ceph-mon[32648]: from='client.? ' entity='client.admin' 2024-04-26T01:53:28.379 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:28 smithi151 ceph-mon[28057]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-26T01:53:28.379 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:28 smithi151 ceph-mon[28057]: mgrmap e19: x(active, since 12m), standbys: y, z 2024-04-26T01:53:28.380 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:28 smithi151 ceph-mon[28057]: from='client.? ' entity='client.admin' 2024-04-26T01:53:28.380 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:28 smithi151 ceph-mon[28057]: from='client.? ' entity='client.admin' 2024-04-26T01:53:28.744 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:53:28.871 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:53:28 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:53:28.428+0000 7fba6fdfe200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T01:53:28.871 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:53:28 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:53:28.546+0000 7fba6fdfe200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T01:53:29.596 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 01:53:29 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-y[34029]: 2024-04-26T01:53:29.320+0000 7fb5add29200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T01:53:29.596 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 01:53:29 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-y[34029]: 2024-04-26T01:53:29.490+0000 7fb5add29200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T01:53:29.627 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 01:53:29 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-z[34652]: 2024-04-26T01:53:29.288+0000 7f62c8568200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T01:53:29.627 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 01:53:29 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-z[34652]: 2024-04-26T01:53:29.459+0000 7f62c8568200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T01:53:29.954 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 01:53:29 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-y[34029]: 2024-04-26T01:53:29.664+0000 7fb5add29200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T01:53:30.220 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 01:53:29 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-z[34652]: 2024-04-26T01:53:29.629+0000 7f62c8568200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T01:53:30.220 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 01:53:29 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-z[34652]: 2024-04-26T01:53:29.923+0000 7f62c8568200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T01:53:30.221 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 01:53:30 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-z[34652]: 2024-04-26T01:53:30.017+0000 7f62c8568200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T01:53:30.221 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 01:53:30 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-z[34652]: 2024-04-26T01:53:30.222+0000 7f62c8568200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T01:53:30.251 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 01:53:29 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-y[34029]: 2024-04-26T01:53:29.956+0000 7fb5add29200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T01:53:30.251 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 01:53:30 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-y[34029]: 2024-04-26T01:53:30.049+0000 7fb5add29200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T01:53:30.371 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:53:30 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:53:30.142+0000 7fba6fdfe200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T01:53:30.536 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 01:53:30 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-z[34652]: 2024-04-26T01:53:30.416+0000 7f62c8568200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T01:53:30.596 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 01:53:30 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-y[34029]: 2024-04-26T01:53:30.253+0000 7fb5add29200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T01:53:30.596 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 01:53:30 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-y[34029]: 2024-04-26T01:53:30.448+0000 7fb5add29200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T01:53:30.724 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:53:30 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:53:30.380+0000 7fba6fdfe200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T01:53:30.724 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:53:30 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:53:30.610+0000 7fba6fdfe200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T01:53:30.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:30.885+0000 7f5c64a00700 1 -- 172.21.15.151:0/1425917477 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c6010f1b0 msgr2=0x7f5c60111620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:30.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:30.885+0000 7f5c64a00700 1 --2- 172.21.15.151:0/1425917477 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c6010f1b0 0x7f5c60111620 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5c5800b0a0 tx=0x7f5c5800b3b0 comp rx=0 tx=0).stop 2024-04-26T01:53:30.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:30.886+0000 7f5c64a00700 1 -- 172.21.15.151:0/1425917477 shutdown_connections 2024-04-26T01:53:30.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:30.886+0000 7f5c64a00700 1 --2- 172.21.15.151:0/1425917477 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c6010f1b0 0x7f5c60111620 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:30.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:30.886+0000 7f5c64a00700 1 --2- 172.21.15.151:0/1425917477 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5c60074e70 0x7f5c60075250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:30.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:30.886+0000 7f5c64a00700 1 --2- 172.21.15.151:0/1425917477 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c60075790 0x7f5c6010ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:30.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:30.886+0000 7f5c64a00700 1 -- 172.21.15.151:0/1425917477 >> 172.21.15.151:0/1425917477 conn(0x7f5c600fe4c0 msgr2=0x7f5c601008e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:30.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:30.886+0000 7f5c64a00700 1 -- 172.21.15.151:0/1425917477 shutdown_connections 2024-04-26T01:53:30.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:30.886+0000 7f5c64a00700 1 -- 172.21.15.151:0/1425917477 wait complete. 2024-04-26T01:53:30.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:30.887+0000 7f5c64a00700 1 Processor -- start 2024-04-26T01:53:30.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:30.887+0000 7f5c64a00700 1 -- start start 2024-04-26T01:53:30.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:30.887+0000 7f5c64a00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c60074e70 0x7f5c6019b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:30.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:30.887+0000 7f5c64a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c60075790 0x7f5c6019b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:30.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:30.887+0000 7f5c64a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5c6010f1b0 0x7f5c6019faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:30.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:30.887+0000 7f5c64a00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5c60114d90 con 0x7f5c6010f1b0 2024-04-26T01:53:30.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:30.887+0000 7f5c64a00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5c60114c10 con 0x7f5c60075790 2024-04-26T01:53:30.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:30.887+0000 7f5c64a00700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5c60114f10 con 0x7f5c60074e70 2024-04-26T01:53:30.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:30.888+0000 7f5c5de00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c60075790 0x7f5c6019b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:30.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:30.888+0000 7f5c5e800700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c60074e70 0x7f5c6019b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:30.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:30.888+0000 7f5c5de00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c60075790 0x7f5c6019b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:46320/0 (socket says 172.21.15.151:46320) 2024-04-26T01:53:30.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:30.888+0000 7f5c5de00700 1 -- 172.21.15.151:0/2425971557 learned_addr learned my addr 172.21.15.151:0/2425971557 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:53:30.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:30.888+0000 7f5c5f200700 1 --2- 172.21.15.151:0/2425971557 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5c6010f1b0 0x7f5c6019faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:30.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:30.888+0000 7f5c5de00700 1 -- 172.21.15.151:0/2425971557 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c60074e70 msgr2=0x7f5c6019b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:30.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:30.888+0000 7f5c5de00700 1 --2- 172.21.15.151:0/2425971557 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c60074e70 0x7f5c6019b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:30.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:30.888+0000 7f5c5de00700 1 -- 172.21.15.151:0/2425971557 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5c6010f1b0 msgr2=0x7f5c6019faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:30.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:30.888+0000 7f5c5de00700 1 --2- 172.21.15.151:0/2425971557 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5c6010f1b0 0x7f5c6019faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:30.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:30.888+0000 7f5c5de00700 1 -- 172.21.15.151:0/2425971557 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c601a0230 con 0x7f5c60075790 2024-04-26T01:53:30.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:30.888+0000 7f5c5f200700 1 --2- 172.21.15.151:0/2425971557 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5c6010f1b0 0x7f5c6019faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T01:53:30.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:30.888+0000 7f5c5de00700 1 --2- 172.21.15.151:0/2425971557 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c60075790 0x7f5c6019b760 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5c4800e9a0 tx=0x7f5c4800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:30.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:30.889+0000 7f5c57400700 1 -- 172.21.15.151:0/2425971557 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4800cbd0 con 0x7f5c60075790 2024-04-26T01:53:30.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:30.889+0000 7f5c57400700 1 -- 172.21.15.151:0/2425971557 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c4800cd70 con 0x7f5c60075790 2024-04-26T01:53:30.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:30.889+0000 7f5c64a00700 1 -- 172.21.15.151:0/2425971557 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c601a0520 con 0x7f5c60075790 2024-04-26T01:53:30.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:30.889+0000 7f5c57400700 1 -- 172.21.15.151:0/2425971557 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c48018680 con 0x7f5c60075790 2024-04-26T01:53:30.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:30.889+0000 7f5c64a00700 1 -- 172.21.15.151:0/2425971557 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c601a7da0 con 0x7f5c60075790 2024-04-26T01:53:30.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:30.890+0000 7f5c64a00700 1 -- 172.21.15.151:0/2425971557 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c60069730 con 0x7f5c60075790 2024-04-26T01:53:30.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:30.892+0000 7f5c57400700 1 -- 172.21.15.151:0/2425971557 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 19) ==== 138664+0+0 (secure 0 0 0) 0x7f5c48010430 con 0x7f5c60075790 2024-04-26T01:53:30.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:30.893+0000 7f5c57400700 1 --2- 172.21.15.151:0/2425971557 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f5c440a3d10 0x7f5c440a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:30.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:30.893+0000 7f5c5e800700 1 -- 172.21.15.151:0/2425971557 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f5c440a3d10 msgr2=0x7f5c440a61d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.151:6800/2251256505 2024-04-26T01:53:30.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:30.893+0000 7f5c57400700 1 -- 172.21.15.151:0/2425971557 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f5c480d05e0 con 0x7f5c60075790 2024-04-26T01:53:30.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:30.893+0000 7f5c5e800700 1 --2- 172.21.15.151:0/2425971557 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f5c440a3d10 0x7f5c440a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T01:53:30.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:30.896+0000 7f5c57400700 1 -- 172.21.15.151:0/2425971557 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5c4809cfc0 con 0x7f5c60075790 2024-04-26T01:53:31.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:31.094+0000 7f5c5e800700 1 -- 172.21.15.151:0/2425971557 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f5c440a3d10 msgr2=0x7f5c440a61d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.151:6800/2251256505 2024-04-26T01:53:31.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:31.094+0000 7f5c5e800700 1 --2- 172.21.15.151:0/2425971557 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f5c440a3d10 0x7f5c440a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T01:53:31.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:31.171+0000 7f5c64a00700 1 -- 172.21.15.151:0/2425971557 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5c6019c580 con 0x7f5c60075790 2024-04-26T01:53:31.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:31.172+0000 7f5c57400700 1 -- 172.21.15.151:0/2425971557 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) ==== 53+0+73 (secure 0 0 0) 0x7f5c4809c8c0 con 0x7f5c60075790 2024-04-26T01:53:31.171 INFO:teuthology.orchestra.run.smithi151.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T01:53:31.174 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:31.175+0000 7f5c64a00700 1 -- 172.21.15.151:0/2425971557 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f5c440a3d10 msgr2=0x7f5c440a61d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T01:53:31.174 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:31.175+0000 7f5c64a00700 1 --2- 172.21.15.151:0/2425971557 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f5c440a3d10 0x7f5c440a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:31.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:31.175+0000 7f5c64a00700 1 -- 172.21.15.151:0/2425971557 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c60075790 msgr2=0x7f5c6019b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:31.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:31.175+0000 7f5c64a00700 1 --2- 172.21.15.151:0/2425971557 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c60075790 0x7f5c6019b760 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5c4800e9a0 tx=0x7f5c4800ecb0 comp rx=0 tx=0).stop 2024-04-26T01:53:31.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:31.175+0000 7f5c64a00700 1 -- 172.21.15.151:0/2425971557 shutdown_connections 2024-04-26T01:53:31.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:31.175+0000 7f5c64a00700 1 --2- 172.21.15.151:0/2425971557 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f5c440a3d10 0x7f5c440a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:31.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:31.175+0000 7f5c64a00700 1 --2- 172.21.15.151:0/2425971557 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5c6010f1b0 0x7f5c6019faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:31.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:31.175+0000 7f5c64a00700 1 --2- 172.21.15.151:0/2425971557 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c60074e70 0x7f5c6019b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:31.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:31.175+0000 7f5c64a00700 1 --2- 172.21.15.151:0/2425971557 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c60075790 0x7f5c6019b760 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:31.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:31.175+0000 7f5c64a00700 1 -- 172.21.15.151:0/2425971557 >> 172.21.15.151:0/2425971557 conn(0x7f5c600fe4c0 msgr2=0x7f5c600ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:31.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:31.176+0000 7f5c64a00700 1 -- 172.21.15.151:0/2425971557 shutdown_connections 2024-04-26T01:53:31.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:31.176+0000 7f5c64a00700 1 -- 172.21.15.151:0/2425971557 wait complete. 2024-04-26T01:53:31.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:31 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/2425971557' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T01:53:31.286 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 01:53:31 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-z[34652]: 2024-04-26T01:53:31.015+0000 7f62c8568200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T01:53:31.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:31 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/2425971557' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T01:53:31.346 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 01:53:31 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-y[34029]: 2024-04-26T01:53:31.054+0000 7fb5add29200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T01:53:31.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:31 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/2425971557' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T01:53:31.371 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:53:31 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:53:31.010+0000 7fba6fdfe200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T01:53:31.371 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:53:31 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:53:31.139+0000 7fba6fdfe200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T01:53:31.662 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:53:31 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:53:31.404+0000 7fba6fdfe200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T01:53:31.731 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 01:53:31 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-z[34652]: 2024-04-26T01:53:31.434+0000 7f62c8568200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T01:53:31.731 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 01:53:31 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-z[34652]: 2024-04-26T01:53:31.531+0000 7f62c8568200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T01:53:31.731 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 01:53:31 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-z[34652]: 2024-04-26T01:53:31.732+0000 7f62c8568200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T01:53:31.780 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 01:53:31 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-y[34029]: 2024-04-26T01:53:31.481+0000 7fb5add29200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T01:53:31.780 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 01:53:31 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-y[34029]: 2024-04-26T01:53:31.579+0000 7fb5add29200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T01:53:31.781 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 01:53:31 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-y[34029]: 2024-04-26T01:53:31.782+0000 7fb5add29200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T01:53:31.926 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:53:31 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:53:31.663+0000 7fba6fdfe200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T01:53:31.929 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T01:53:31.929 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T01:53:31.929 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi151.front.sepia.ceph.com 2024-04-26T01:53:31.929 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T01:53:31.929 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T01:53:31.929 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T01:53:31.929 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T01:53:31.929 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T01:53:31.929 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T01:53:31.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T01:53:31.929 DEBUG:teuthology.orchestra.run.smithi151:> ip addr 2024-04-26T01:53:31.979 INFO:teuthology.orchestra.run.smithi151.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T01:53:31.979 INFO:teuthology.orchestra.run.smithi151.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T01:53:31.979 INFO:teuthology.orchestra.run.smithi151.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T01:53:31.979 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-04-26T01:53:31.979 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 ::1/128 scope host 2024-04-26T01:53:31.979 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-04-26T01:53:31.980 INFO:teuthology.orchestra.run.smithi151.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T01:53:31.980 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:da:6e:0c brd ff:ff:ff:ff:ff:ff 2024-04-26T01:53:31.980 INFO:teuthology.orchestra.run.smithi151.stdout: altname enp7s0f0 2024-04-26T01:53:31.980 INFO:teuthology.orchestra.run.smithi151.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T01:53:31.980 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:da:6e:0d brd ff:ff:ff:ff:ff:ff 2024-04-26T01:53:31.980 INFO:teuthology.orchestra.run.smithi151.stdout: altname enp7s0f1 2024-04-26T01:53:31.980 INFO:teuthology.orchestra.run.smithi151.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T01:53:31.980 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:88:6f:62 brd ff:ff:ff:ff:ff:ff 2024-04-26T01:53:31.980 INFO:teuthology.orchestra.run.smithi151.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T01:53:31.980 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:88:6f:63 brd ff:ff:ff:ff:ff:ff 2024-04-26T01:53:31.980 INFO:teuthology.orchestra.run.smithi151.stdout: inet 172.21.15.151/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-26T01:53:31.980 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft 41829sec preferred_lft 41829sec 2024-04-26T01:53:31.980 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 fe80::ec4:7aff:fe88:6f63/64 scope link 2024-04-26T01:53:31.981 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-04-26T01:53:31.981 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-26T01:53:31.981 DEBUG:teuthology.orchestra.run.smithi151:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T01:53:31.981 DEBUG:teuthology.orchestra.run.smithi151:> set -e 2024-04-26T01:53:31.981 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip link add name ceph-brx type bridge 2024-04-26T01:53:31.981 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip addr flush dev ceph-brx 2024-04-26T01:53:31.981 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip link set ceph-brx up 2024-04-26T01:53:31.982 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-26T01:53:31.982 DEBUG:teuthology.orchestra.run.smithi151:> ') 2024-04-26T01:53:32.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T01:53:32.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T01:53:32.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T01:53:32.485 DEBUG:teuthology.orchestra.run.smithi151:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T01:53:32.536 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 01:53:32 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-z[34652]: 2024-04-26T01:53:32.252+0000 7f62c8568200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T01:53:32.536 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 01:53:32 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-z[34652]: 2024-04-26T01:53:32.343+0000 7f62c8568200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T01:53:32.551 INFO:teuthology.orchestra.run.smithi151.stdout:1 2024-04-26T01:53:32.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T01:53:32.555 DEBUG:teuthology.orchestra.run.smithi151:> route 2024-04-26T01:53:32.596 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 01:53:32 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-y[34029]: 2024-04-26T01:53:32.309+0000 7fb5add29200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T01:53:32.596 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 01:53:32 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-y[34029]: 2024-04-26T01:53:32.402+0000 7fb5add29200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T01:53:32.603 INFO:teuthology.orchestra.run.smithi151.stdout:Kernel IP routing table 2024-04-26T01:53:32.604 INFO:teuthology.orchestra.run.smithi151.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T01:53:32.604 INFO:teuthology.orchestra.run.smithi151.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T01:53:32.604 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T01:53:32.604 INFO:teuthology.orchestra.run.smithi151.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T01:53:32.605 DEBUG:teuthology.orchestra.run.smithi151:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T01:53:32.605 DEBUG:teuthology.orchestra.run.smithi151:> set -e 2024-04-26T01:53:32.605 DEBUG:teuthology.orchestra.run.smithi151:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T01:53:32.605 DEBUG:teuthology.orchestra.run.smithi151:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T01:53:32.605 DEBUG:teuthology.orchestra.run.smithi151:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T01:53:32.605 DEBUG:teuthology.orchestra.run.smithi151:> ') 2024-04-26T01:53:32.697 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T01:53:32.719 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:53:32 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:53:32.496+0000 7fba6fdfe200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T01:53:32.925 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 01:53:32 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-z[34652]: 2024-04-26T01:53:32.584+0000 7f62c8568200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T01:53:32.925 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 01:53:32 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-z[34652]: 2024-04-26T01:53:32.717+0000 7f62c8568200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T01:53:32.925 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 01:53:32 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-z[34652]: 2024-04-26T01:53:32.812+0000 7f62c8568200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T01:53:32.925 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 01:53:32 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-z[34652]: 2024-04-26T01:53:32.926+0000 7f62c8568200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T01:53:32.993 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 01:53:32 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-y[34029]: 2024-04-26T01:53:32.647+0000 7fb5add29200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T01:53:32.993 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 01:53:32 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-y[34029]: 2024-04-26T01:53:32.783+0000 7fb5add29200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T01:53:32.993 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 01:53:32 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-y[34029]: 2024-04-26T01:53:32.878+0000 7fb5add29200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T01:53:32.993 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 01:53:32 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-y[34029]: 2024-04-26T01:53:32.994+0000 7fb5add29200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T01:53:33.341 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:53:33 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:53:33.080+0000 7fba6fdfe200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T01:53:33.342 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:53:33 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:53:33.213+0000 7fba6fdfe200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T01:53:33.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T01:53:33.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T01:53:33.353 DEBUG:teuthology.orchestra.run.smithi151:> ip netns list 2024-04-26T01:53:33.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T01:53:33.400 DEBUG:teuthology.orchestra.run.smithi151:> ip netns list-id 2024-04-26T01:53:33.490 DEBUG:teuthology.orchestra.run.smithi151:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T01:53:33.490 DEBUG:teuthology.orchestra.run.smithi151:> set -e 2024-04-26T01:53:33.490 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T01:53:33.490 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-26T01:53:33.490 DEBUG:teuthology.orchestra.run.smithi151:> ') 2024-04-26T01:53:33.536 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 01:53:33 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-z[34652]: 2024-04-26T01:53:33.228+0000 7f62c8568200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T01:53:33.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T01:53:33.595 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 01:53:33 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-y[34029]: 2024-04-26T01:53:33.303+0000 7fb5add29200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T01:53:33.608 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:53:33 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:53:33.495+0000 7fba6fdfe200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T01:53:33.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T01:53:33.660 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-26T01:53:33.660 DEBUG:teuthology.orchestra.run.smithi151:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T01:53:33.661 DEBUG:teuthology.orchestra.run.smithi151:> set -e 2024-04-26T01:53:33.661 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-26T01:53:33.661 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-26T01:53:33.661 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-26T01:53:33.661 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-26T01:53:33.661 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-26T01:53:33.661 DEBUG:teuthology.orchestra.run.smithi151:> ') 2024-04-26T01:53:33.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T01:53:33.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T01:53:33.949 DEBUG:teuthology.orchestra.run.smithi151:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T01:53:33.949 DEBUG:teuthology.orchestra.run.smithi151:> set -e 2024-04-26T01:53:33.949 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip link set brx.0 up 2024-04-26T01:53:33.949 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip link set dev brx.0 master ceph-brx 2024-04-26T01:53:33.949 DEBUG:teuthology.orchestra.run.smithi151:> ') 2024-04-26T01:53:34.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T01:53:34.110 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 01:53:33 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-z[34652]: 2024-04-26T01:53:33.829+0000 7f62c8568200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T01:53:34.110 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 01:53:34 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-z[34652]: 2024-04-26T01:53:34.043+0000 7f62c8568200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T01:53:34.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:34 smithi151 ceph-mon[28057]: Standby manager daemon z restarted 2024-04-26T01:53:34.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:34 smithi151 ceph-mon[28057]: Standby manager daemon z started 2024-04-26T01:53:34.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:34 smithi151 ceph-mon[28057]: from='mgr.? 172.21.15.172:0/4048451480' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T01:53:34.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:34 smithi151 ceph-mon[28057]: from='mgr.? 172.21.15.172:0/4048451480' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T01:53:34.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:34 smithi151 ceph-mon[28057]: from='mgr.? 172.21.15.172:0/4048451480' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T01:53:34.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:34 smithi151 ceph-mon[28057]: from='mgr.? 172.21.15.172:0/4048451480' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T01:53:34.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T01:53:34.139 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T01:53:34.139 DEBUG:teuthology.orchestra.run.smithi151:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T01:53:34.180 INFO:teuthology.orchestra.run.smithi151.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T01:53:34.181 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T01:53:34.181 DEBUG:teuthology.orchestra.run.smithi151:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T01:53:34.266 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-26T01:53:34.266 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-26T01:53:34.266 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/87d7cc94-2a5d-4cec-9bbf-5cb1e42ad242 2024-04-26T01:53:34.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T01:53:34.266 DEBUG:teuthology.orchestra.run.smithi151:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/87d7cc94-2a5d-4cec-9bbf-5cb1e42ad242 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-04-26T01:53:34.346 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 01:53:33 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-y[34029]: 2024-04-26T01:53:33.912+0000 7fb5add29200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T01:53:34.346 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 01:53:34 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-y[34029]: 2024-04-26T01:53:34.134+0000 7fb5add29200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T01:53:34.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:34 smithi165 ceph-mon[32648]: Standby manager daemon z restarted 2024-04-26T01:53:34.347 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:34 smithi165 ceph-mon[32648]: Standby manager daemon z started 2024-04-26T01:53:34.347 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:34 smithi165 ceph-mon[32648]: from='mgr.? 172.21.15.172:0/4048451480' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T01:53:34.347 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:34 smithi165 ceph-mon[32648]: from='mgr.? 172.21.15.172:0/4048451480' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T01:53:34.347 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:34 smithi165 ceph-mon[32648]: from='mgr.? 172.21.15.172:0/4048451480' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T01:53:34.347 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:34 smithi165 ceph-mon[32648]: from='mgr.? 172.21.15.172:0/4048451480' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T01:53:34.437 INFO:teuthology.orchestra.run.smithi151.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-04-26T01:53:34.437 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync". 2024-04-26T01:53:34.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:34 smithi172 ceph-mon[32785]: Standby manager daemon z restarted 2024-04-26T01:53:34.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:34 smithi172 ceph-mon[32785]: Standby manager daemon z started 2024-04-26T01:53:34.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:34 smithi172 ceph-mon[32785]: from='mgr.? 172.21.15.172:0/4048451480' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T01:53:34.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:34 smithi172 ceph-mon[32785]: from='mgr.? 172.21.15.172:0/4048451480' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T01:53:34.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:34 smithi172 ceph-mon[32785]: from='mgr.? 172.21.15.172:0/4048451480' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T01:53:34.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:34 smithi172 ceph-mon[32785]: from='mgr.? 172.21.15.172:0/4048451480' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T01:53:34.561 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:53:34 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:53:34.202+0000 7fba6fdfe200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T01:53:34.562 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:53:34 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:53:34.332+0000 7fba6fdfe200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T01:53:34.860 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:53:34 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:53:34.672+0000 7fba6fdfe200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T01:53:34.860 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:53:34 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:53:34.860+0000 7fba6fdfe200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T01:53:35.112 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:53:34 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:53:34.991+0000 7fba6fdfe200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T01:53:35.315 INFO:teuthology.orchestra.run.smithi151.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-04-26T01:53:35.315 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync". 2024-04-26T01:53:35.316 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: resolved to: "172.21.15.151:3300,172.21.15.165:3300,172.21.15.172:3300" 2024-04-26T01:53:35.316 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: trying mount with new device syntax: 0@c0e49e64-036d-11ef-bc93-c7b262605968.cephfs=/volumes/qa/sv_1/87d7cc94-2a5d-4cec-9bbf-5cb1e42ad242 2024-04-26T01:53:35.316 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync,name=0,key=0,mon_addr=172.21.15.151:3300/172.21.15.165:3300/172.21.15.172:3300" will pass to kernel 2024-04-26T01:53:35.322 INFO:teuthology.orchestra.run.smithi151.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T01:53:35.322 INFO:teuthology.orchestra.run.smithi151.stdout: You just mounted a file system that supports labels which does not 2024-04-26T01:53:35.322 INFO:teuthology.orchestra.run.smithi151.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T01:53:35.322 INFO:teuthology.orchestra.run.smithi151.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T01:53:35.322 INFO:teuthology.orchestra.run.smithi151.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T01:53:35.327 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T01:53:35.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T01:53:35.328 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T01:53:35.371 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:53:35 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:53:35.154+0000 7fba6fdfe200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T01:53:35.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:35 smithi151 ceph-mon[28057]: mgrmap e20: x(active, since 12m), standbys: y, z 2024-04-26T01:53:35.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:35 smithi151 ceph-mon[28057]: Standby manager daemon y restarted 2024-04-26T01:53:35.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:35 smithi151 ceph-mon[28057]: Standby manager daemon y started 2024-04-26T01:53:35.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:35 smithi151 ceph-mon[28057]: from='mgr.? 172.21.15.165:0/3045682535' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T01:53:35.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:35 smithi151 ceph-mon[28057]: from='mgr.? 172.21.15.165:0/3045682535' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T01:53:35.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:35 smithi151 ceph-mon[28057]: from='mgr.? 172.21.15.165:0/3045682535' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T01:53:35.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:35 smithi151 ceph-mon[28057]: from='mgr.? 172.21.15.165:0/3045682535' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T01:53:35.400 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T01:53:35.438 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:35 smithi172 ceph-mon[32785]: mgrmap e20: x(active, since 12m), standbys: y, z 2024-04-26T01:53:35.438 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:35 smithi172 ceph-mon[32785]: Standby manager daemon y restarted 2024-04-26T01:53:35.438 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:35 smithi172 ceph-mon[32785]: Standby manager daemon y started 2024-04-26T01:53:35.438 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:35 smithi172 ceph-mon[32785]: from='mgr.? 172.21.15.165:0/3045682535' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T01:53:35.438 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:35 smithi172 ceph-mon[32785]: from='mgr.? 172.21.15.165:0/3045682535' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T01:53:35.439 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:35 smithi172 ceph-mon[32785]: from='mgr.? 172.21.15.165:0/3045682535' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T01:53:35.439 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:35 smithi172 ceph-mon[32785]: from='mgr.? 172.21.15.165:0/3045682535' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T01:53:35.548 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:35 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T01:53:35.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T01:53:35.564 INFO:teuthology.orchestra.run.smithi151.stdout:client25066 2024-04-26T01:53:35.564 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-26T01:53:35.564 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T01:53:35.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:35 smithi165 ceph-mon[32648]: mgrmap e20: x(active, since 12m), standbys: y, z 2024-04-26T01:53:35.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:35 smithi165 ceph-mon[32648]: Standby manager daemon y restarted 2024-04-26T01:53:35.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:35 smithi165 ceph-mon[32648]: Standby manager daemon y started 2024-04-26T01:53:35.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:35 smithi165 ceph-mon[32648]: from='mgr.? 172.21.15.165:0/3045682535' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T01:53:35.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:35 smithi165 ceph-mon[32648]: from='mgr.? 172.21.15.165:0/3045682535' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T01:53:35.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:35 smithi165 ceph-mon[32648]: from='mgr.? 172.21.15.165:0/3045682535' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T01:53:35.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:35 smithi165 ceph-mon[32648]: from='mgr.? 172.21.15.165:0/3045682535' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T01:53:35.613 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T01:53:35.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:35 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T01:53:35.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T01:53:35.769 INFO:teuthology.orchestra.run.smithi151.stdout:client25066 2024-04-26T01:53:35.769 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T01:53:35.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:35 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T01:53:35.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T01:53:35.871 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:53:35 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:53:35.544+0000 7fba6fdfe200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T01:53:35.877 INFO:teuthology.orchestra.run.smithi151.stdout:client25066 2024-04-26T01:53:35.878 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/c0e49e64-036d-11ef-bc93-c7b262605968.client25066/status') 2024-04-26T01:53:35.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T01:53:36.003 INFO:teuthology.orchestra.run.smithi151.stdout:instance: client.25066 (3)172.21.15.151:0/1473554155 2024-04-26T01:53:36.003 INFO:teuthology.orchestra.run.smithi151.stdout:blocklisted: false 2024-04-26T01:53:36.003 INFO:teuthology.orchestra.run.smithi151.stderr:0+1 records in 2024-04-26T01:53:36.003 INFO:teuthology.orchestra.run.smithi151.stderr:0+1 records out 2024-04-26T01:53:36.003 INFO:teuthology.orchestra.run.smithi151.stderr:72 bytes copied, 7.2833e-05 s, 989 kB/s 2024-04-26T01:53:36.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T01:53:36.015 INFO:teuthology.run_tasks:Running task exec... 2024-04-26T01:53:36.025 INFO:teuthology.task.exec:Executing custom commands... 2024-04-26T01:53:36.025 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi151.front.sepia.ceph.com 2024-04-26T01:53:36.025 DEBUG:teuthology.orchestra.run.smithi151:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-04-26T01:53:36.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:36.222+0000 7f7cf2800640 1 -- 172.21.15.151:0/1288622988 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7cec100fa0 msgr2=0x7f7cec101400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:36.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:36.222+0000 7f7cf2800640 1 --2- 172.21.15.151:0/1288622988 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7cec100fa0 0x7f7cec101400 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8009900 tx=0x7f7cd802a3f0 comp rx=0 tx=0).stop 2024-04-26T01:53:36.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:36.222+0000 7f7cf2800640 1 -- 172.21.15.151:0/1288622988 shutdown_connections 2024-04-26T01:53:36.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:36.222+0000 7f7cf2800640 1 --2- 172.21.15.151:0/1288622988 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7cec101940 0x7f7cec101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:36.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:36.222+0000 7f7cf2800640 1 --2- 172.21.15.151:0/1288622988 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7cec100fa0 0x7f7cec101400 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:36.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:36.222+0000 7f7cf2800640 1 --2- 172.21.15.151:0/1288622988 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7cec100640 0x7f7cec100a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:36.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:36.222+0000 7f7cf2800640 1 -- 172.21.15.151:0/1288622988 >> 172.21.15.151:0/1288622988 conn(0x7f7cec0fc240 msgr2=0x7f7cec0fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:36.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:36.222+0000 7f7cf2800640 1 -- 172.21.15.151:0/1288622988 shutdown_connections 2024-04-26T01:53:36.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:36.222+0000 7f7cf2800640 1 -- 172.21.15.151:0/1288622988 wait complete. 2024-04-26T01:53:36.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:36.223+0000 7f7cf2800640 1 Processor -- start 2024-04-26T01:53:36.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:36.223+0000 7f7cf2800640 1 -- start start 2024-04-26T01:53:36.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:36.224+0000 7f7cf2800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7cec100640 0x7f7cec19a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:36.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:36.224+0000 7f7cf2800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7cec100fa0 0x7f7cec19a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:36.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:36.224+0000 7f7cf2800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7cec101940 0x7f7cec1a17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:36.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:36.224+0000 7f7cf2800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7cec10fb20 con 0x7f7cec101940 2024-04-26T01:53:36.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:36.224+0000 7f7cf2800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7cec10f9a0 con 0x7f7cec100fa0 2024-04-26T01:53:36.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:36.224+0000 7f7cf2800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7cec10fca0 con 0x7f7cec100640 2024-04-26T01:53:36.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:36.224+0000 7f7cf1400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7cec100640 0x7f7cec19a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:36.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:36.224+0000 7f7cf1400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7cec100640 0x7f7cec19a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:37646/0 (socket says 172.21.15.151:37646) 2024-04-26T01:53:36.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:36.224+0000 7f7cf1400640 1 -- 172.21.15.151:0/1029961540 learned_addr learned my addr 172.21.15.151:0/1029961540 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:53:36.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:36.224+0000 7f7cf0a00640 1 --2- 172.21.15.151:0/1029961540 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7cec100fa0 0x7f7cec19a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:36.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:36.224+0000 7f7cf1e00640 1 --2- 172.21.15.151:0/1029961540 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7cec101940 0x7f7cec1a17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:36.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:36.224+0000 7f7cf1400640 1 -- 172.21.15.151:0/1029961540 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7cec100fa0 msgr2=0x7f7cec19a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:36.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:36.224+0000 7f7cf1400640 1 --2- 172.21.15.151:0/1029961540 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7cec100fa0 0x7f7cec19a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:36.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:36.224+0000 7f7cf1400640 1 -- 172.21.15.151:0/1029961540 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7cec101940 msgr2=0x7f7cec1a17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:36.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:36.224+0000 7f7cf1400640 1 --2- 172.21.15.151:0/1029961540 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7cec101940 0x7f7cec1a17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:36.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:36.224+0000 7f7cf1400640 1 -- 172.21.15.151:0/1029961540 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cec1a1e30 con 0x7f7cec100640 2024-04-26T01:53:36.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:36.224+0000 7f7cf1e00640 1 --2- 172.21.15.151:0/1029961540 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7cec101940 0x7f7cec1a17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:53:36.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:36.225+0000 7f7cf1400640 1 --2- 172.21.15.151:0/1029961540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7cec100640 0x7f7cec19a210 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7ce000e9f0 tx=0x7f7ce000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:36.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:36.225+0000 7f7cea000640 1 -- 172.21.15.151:0/1029961540 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce000cd80 con 0x7f7cec100640 2024-04-26T01:53:36.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:36.225+0000 7f7cf2800640 1 -- 172.21.15.151:0/1029961540 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cec1a2120 con 0x7f7cec100640 2024-04-26T01:53:36.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:36.225+0000 7f7cea000640 1 -- 172.21.15.151:0/1029961540 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ce00054d0 con 0x7f7cec100640 2024-04-26T01:53:36.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:36.225+0000 7f7cea000640 1 -- 172.21.15.151:0/1029961540 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce0010690 con 0x7f7cec100640 2024-04-26T01:53:36.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:36.225+0000 7f7cf2800640 1 -- 172.21.15.151:0/1029961540 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cec1a25f0 con 0x7f7cec100640 2024-04-26T01:53:36.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:36.226+0000 7f7cf2800640 1 -- 172.21.15.151:0/1029961540 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cec10b5e0 con 0x7f7cec100640 2024-04-26T01:53:36.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:36.230+0000 7f7cea000640 1 -- 172.21.15.151:0/1029961540 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 21) ==== 138664+0+0 (secure 0 0 0) 0x7f7ce0004e90 con 0x7f7cec100640 2024-04-26T01:53:36.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:36.231+0000 7f7cea000640 1 --2- 172.21.15.151:0/1029961540 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f7cd40a3bf0 0x7f7cd40a60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:36.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:36.232+0000 7f7cf0a00640 1 -- 172.21.15.151:0/1029961540 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f7cd40a3bf0 msgr2=0x7f7cd40a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.151:6800/2251256505 2024-04-26T01:53:36.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:36.232+0000 7f7cea000640 1 -- 172.21.15.151:0/1029961540 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f7ce00d0900 con 0x7f7cec100640 2024-04-26T01:53:36.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:36.232+0000 7f7cf0a00640 1 --2- 172.21.15.151:0/1029961540 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f7cd40a3bf0 0x7f7cd40a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T01:53:36.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:36.236+0000 7f7cea000640 1 -- 172.21.15.151:0/1029961540 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7ce009d2e0 con 0x7f7cec100640 2024-04-26T01:53:36.352 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:36 smithi151 ceph-mon[28057]: mgrmap e21: x(active, since 12m), standbys: y, z 2024-04-26T01:53:36.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:36.433+0000 7f7cf0a00640 1 -- 172.21.15.151:0/1029961540 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f7cd40a3bf0 msgr2=0x7f7cd40a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.151:6800/2251256505 2024-04-26T01:53:36.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:36.433+0000 7f7cf0a00640 1 --2- 172.21.15.151:0/1029961540 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f7cd40a3bf0 0x7f7cd40a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T01:53:36.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:36.480+0000 7f7cf2800640 1 -- 172.21.15.151:0/1029961540 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f7cec02d5c0 con 0x7f7cec100640 2024-04-26T01:53:36.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:36 smithi172 ceph-mon[32785]: mgrmap e21: x(active, since 12m), standbys: y, z 2024-04-26T01:53:36.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:36 smithi165 ceph-mon[32648]: mgrmap e21: x(active, since 12m), standbys: y, z 2024-04-26T01:53:36.622 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:53:36 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:53:36.353+0000 7fba6fdfe200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T01:53:36.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:36.674+0000 7f7cea000640 1 -- 172.21.15.151:0/1029961540 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mgrmap(e 22) ==== 138331+0+0 (secure 0 0 0) 0x7f7ce00bd0d0 con 0x7f7cec100640 2024-04-26T01:53:36.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:36.675+0000 7f7cea000640 1 -- 172.21.15.151:0/1029961540 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f7cd40a3bf0 msgr2=0x7f7cd40a60b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T01:53:36.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:36.675+0000 7f7cea000640 1 --2- 172.21.15.151:0/1029961540 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f7cd40a3bf0 0x7f7cd40a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:36.965 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:53:36 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:53:36.647+0000 7fba6fdfe200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T01:53:37.131 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.132+0000 7f7cea000640 1 -- 172.21.15.151:0/1029961540 <== mon.1 v2:172.21.15.172:3300/0 8 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v26) ==== 118+0+0 (secure 0 0 0) 0x7f7ce005c810 con 0x7f7cec100640 2024-04-26T01:53:37.133 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.135+0000 7f7cf2800640 1 -- 172.21.15.151:0/1029961540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7cec100640 msgr2=0x7f7cec19a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:37.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.135+0000 7f7cf2800640 1 --2- 172.21.15.151:0/1029961540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7cec100640 0x7f7cec19a210 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7ce000e9f0 tx=0x7f7ce000edf0 comp rx=0 tx=0).stop 2024-04-26T01:53:37.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.135+0000 7f7cf2800640 1 -- 172.21.15.151:0/1029961540 shutdown_connections 2024-04-26T01:53:37.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.135+0000 7f7cf2800640 1 --2- 172.21.15.151:0/1029961540 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7cec101940 0x7f7cec1a17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:37.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.135+0000 7f7cf2800640 1 --2- 172.21.15.151:0/1029961540 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7cec100fa0 0x7f7cec19a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:37.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.135+0000 7f7cf2800640 1 --2- 172.21.15.151:0/1029961540 >> [v2:172.21.15.151:6800/2251256505,v1:172.21.15.151:6801/2251256505] conn(0x7f7cd40a3bf0 0x7f7cd40a60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:37.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.135+0000 7f7cf2800640 1 --2- 172.21.15.151:0/1029961540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7cec100640 0x7f7cec19a210 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:37.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.135+0000 7f7cf2800640 1 -- 172.21.15.151:0/1029961540 >> 172.21.15.151:0/1029961540 conn(0x7f7cec0fc240 msgr2=0x7f7cec0fda20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:37.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.135+0000 7f7cf2800640 1 -- 172.21.15.151:0/1029961540 shutdown_connections 2024-04-26T01:53:37.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.135+0000 7f7cf2800640 1 -- 172.21.15.151:0/1029961540 wait complete. 2024-04-26T01:53:37.153 DEBUG:teuthology.orchestra.run.smithi151:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-04-26T01:53:37.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.332+0000 7fb118400640 1 -- 172.21.15.151:0/1745813015 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb1101097d0 msgr2=0x7fb110109c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:37.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.332+0000 7fb118400640 1 --2- 172.21.15.151:0/1745813015 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb1101097d0 0x7fb110109c50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc009900 tx=0x7fb0fc02a400 comp rx=0 tx=0).stop 2024-04-26T01:53:37.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.332+0000 7fb118400640 1 -- 172.21.15.151:0/1745813015 shutdown_connections 2024-04-26T01:53:37.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.332+0000 7fb118400640 1 --2- 172.21.15.151:0/1745813015 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb11010a190 0x7fb110110a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:37.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.332+0000 7fb118400640 1 --2- 172.21.15.151:0/1745813015 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb1101097d0 0x7fb110109c50 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:37.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.332+0000 7fb118400640 1 --2- 172.21.15.151:0/1745813015 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1101085e0 0x7fb110108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:37.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.332+0000 7fb118400640 1 -- 172.21.15.151:0/1745813015 >> 172.21.15.151:0/1745813015 conn(0x7fb1100775a0 msgr2=0x7fb1100799c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:37.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.332+0000 7fb118400640 1 -- 172.21.15.151:0/1745813015 shutdown_connections 2024-04-26T01:53:37.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.332+0000 7fb118400640 1 -- 172.21.15.151:0/1745813015 wait complete. 2024-04-26T01:53:37.333 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.333+0000 7fb118400640 1 Processor -- start 2024-04-26T01:53:37.333 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.333+0000 7fb118400640 1 -- start start 2024-04-26T01:53:37.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.333+0000 7fb118400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb1101085e0 0x7fb11019e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:37.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.333+0000 7fb118400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1101097d0 0x7fb11019eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:37.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.333+0000 7fb118400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb11010a190 0x7fb1101a5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:37.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.334+0000 7fb118400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb110114150 con 0x7fb1101097d0 2024-04-26T01:53:37.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.334+0000 7fb118400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb110113fd0 con 0x7fb11010a190 2024-04-26T01:53:37.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.334+0000 7fb118400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb1101142d0 con 0x7fb1101085e0 2024-04-26T01:53:37.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.334+0000 7fb115a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb1101085e0 0x7fb11019e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:37.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.334+0000 7fb115a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb1101085e0 0x7fb11019e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:37770/0 (socket says 172.21.15.151:37770) 2024-04-26T01:53:37.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.334+0000 7fb115a00640 1 -- 172.21.15.151:0/4060697182 learned_addr learned my addr 172.21.15.151:0/4060697182 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:53:37.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.334+0000 7fb116400640 1 --2- 172.21.15.151:0/4060697182 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb11010a190 0x7fb1101a5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:37.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.334+0000 7fb115000640 1 --2- 172.21.15.151:0/4060697182 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1101097d0 0x7fb11019eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:37.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.334+0000 7fb115a00640 1 -- 172.21.15.151:0/4060697182 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb11010a190 msgr2=0x7fb1101a5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:37.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.334+0000 7fb115a00640 1 --2- 172.21.15.151:0/4060697182 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb11010a190 0x7fb1101a5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:37.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.334+0000 7fb115a00640 1 -- 172.21.15.151:0/4060697182 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1101097d0 msgr2=0x7fb11019eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:37.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.334+0000 7fb115a00640 1 --2- 172.21.15.151:0/4060697182 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1101097d0 0x7fb11019eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:37.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.334+0000 7fb115a00640 1 -- 172.21.15.151:0/4060697182 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1101a6380 con 0x7fb1101085e0 2024-04-26T01:53:37.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.334+0000 7fb115a00640 1 --2- 172.21.15.151:0/4060697182 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb1101085e0 0x7fb11019e760 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb10400e9c0 tx=0x7fb10400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:37.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.335+0000 7fb10e000640 1 -- 172.21.15.151:0/4060697182 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb10400cd80 con 0x7fb1101085e0 2024-04-26T01:53:37.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.335+0000 7fb10e000640 1 -- 172.21.15.151:0/4060697182 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1040035f0 con 0x7fb1101085e0 2024-04-26T01:53:37.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.335+0000 7fb118400640 1 -- 172.21.15.151:0/4060697182 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1101a6670 con 0x7fb1101085e0 2024-04-26T01:53:37.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.335+0000 7fb10e000640 1 -- 172.21.15.151:0/4060697182 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb104010690 con 0x7fb1101085e0 2024-04-26T01:53:37.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.335+0000 7fb118400640 1 -- 172.21.15.151:0/4060697182 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1101a6b90 con 0x7fb1101085e0 2024-04-26T01:53:37.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.336+0000 7fb118400640 1 -- 172.21.15.151:0/4060697182 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0f00052c0 con 0x7fb1101085e0 2024-04-26T01:53:37.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.338+0000 7fb10e000640 1 -- 172.21.15.151:0/4060697182 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 22) ==== 138331+0+0 (secure 0 0 0) 0x7fb1040037a0 con 0x7fb1101085e0 2024-04-26T01:53:37.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.339+0000 7fb10e000640 1 -- 172.21.15.151:0/4060697182 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fb1040d0340 con 0x7fb1101085e0 2024-04-26T01:53:37.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.347+0000 7fb10e000640 1 -- 172.21.15.151:0/4060697182 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb10409d010 con 0x7fb1101085e0 2024-04-26T01:53:37.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:37 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/1029961540' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-26T01:53:37.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:37 smithi151 ceph-mon[28057]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-26T01:53:37.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:37 smithi151 ceph-mon[28057]: Active manager daemon x restarted 2024-04-26T01:53:37.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:37 smithi151 ceph-mon[28057]: Activating manager daemon x 2024-04-26T01:53:37.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:37 smithi151 ceph-mon[28057]: osdmap e93: 12 total, 12 up, 12 in 2024-04-26T01:53:37.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:37 smithi151 ceph-mon[28057]: mgrmap e22: x(active, starting, since 0.0251204s), standbys: y, z 2024-04-26T01:53:37.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:37 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T01:53:37.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:37 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:53:37.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:37 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:53:37.373 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:37 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T01:53:37.373 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:37 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T01:53:37.373 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:37 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T01:53:37.373 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:37 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T01:53:37.373 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:37 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T01:53:37.373 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:37 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T01:53:37.373 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:37 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T01:53:37.373 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:37 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T01:53:37.373 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:37 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T01:53:37.373 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:37 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T01:53:37.373 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:37 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T01:53:37.373 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:37 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T01:53:37.373 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:37 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T01:53:37.374 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:37 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T01:53:37.374 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:37 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T01:53:37.374 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:37 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:53:37.374 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:37 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:53:37.374 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:37 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T01:53:37.374 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:37 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:53:37.374 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:37 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T01:53:37.374 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:37 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T01:53:37.374 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:37 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:53:37.374 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:37 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T01:53:37.374 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:37 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T01:53:37.374 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:37 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:53:37.374 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:37 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T01:53:37.374 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:37 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T01:53:37.375 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:37 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T01:53:37.375 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:37 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T01:53:37.375 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:37 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T01:53:37.375 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:37 smithi151 ceph-mon[28057]: Manager daemon x is now available 2024-04-26T01:53:37.375 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:37 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:53:37.375 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:37 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T01:53:37.375 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:37 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T01:53:37.375 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:37 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T01:53:37.375 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:37 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T01:53:37.375 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:53:36 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:53:36.972+0000 7fb9c6200700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:53:37.375 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:53:36 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:53:36.972+0000 7fb9c6200700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:53:37.376 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:53:36 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:53:36.972+0000 7fb9c6200700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:53:37.376 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:53:36 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:53:36.972+0000 7fb9c6200700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:53:37.376 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:53:36 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:53:36.972+0000 7fb9c6200700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:53:37.376 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:53:36 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:53:36.978+0000 7fb9c2600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:53:37.376 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:53:36 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:53:36.978+0000 7fb9c2600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:53:37.376 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:53:36 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:53:36.978+0000 7fb9c2600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:53:37.376 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:53:36 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:53:36.978+0000 7fb9c2600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:53:37.376 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:53:36 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:53:36.978+0000 7fb9c2600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:53:37.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:37 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/1029961540' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-26T01:53:37.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:37 smithi172 ceph-mon[32785]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-26T01:53:37.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:37 smithi172 ceph-mon[32785]: Active manager daemon x restarted 2024-04-26T01:53:37.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:37 smithi172 ceph-mon[32785]: Activating manager daemon x 2024-04-26T01:53:37.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:37 smithi172 ceph-mon[32785]: osdmap e93: 12 total, 12 up, 12 in 2024-04-26T01:53:37.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:37 smithi172 ceph-mon[32785]: mgrmap e22: x(active, starting, since 0.0251204s), standbys: y, z 2024-04-26T01:53:37.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:37 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T01:53:37.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:37 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:53:37.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:37 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:53:37.538 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:37 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T01:53:37.538 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:37 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T01:53:37.538 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:37 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T01:53:37.538 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:37 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T01:53:37.538 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:37 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T01:53:37.538 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:37 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T01:53:37.538 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:37 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T01:53:37.538 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:37 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T01:53:37.538 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:37 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T01:53:37.538 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:37 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T01:53:37.538 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:37 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T01:53:37.538 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:37 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T01:53:37.538 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:37 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T01:53:37.539 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:37 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T01:53:37.539 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:37 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T01:53:37.539 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:37 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:53:37.539 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:37 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:53:37.539 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:37 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T01:53:37.539 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:37 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:53:37.539 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:37 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T01:53:37.539 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:37 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T01:53:37.539 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:37 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:53:37.539 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:37 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T01:53:37.539 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:37 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T01:53:37.539 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:37 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:53:37.539 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:37 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T01:53:37.540 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:37 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T01:53:37.540 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:37 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T01:53:37.540 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:37 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T01:53:37.540 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:37 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T01:53:37.540 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:37 smithi172 ceph-mon[32785]: Manager daemon x is now available 2024-04-26T01:53:37.540 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:37 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:53:37.540 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:37 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T01:53:37.540 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:37 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T01:53:37.540 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:37 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T01:53:37.540 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:37 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T01:53:37.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:37 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/1029961540' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-26T01:53:37.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:37 smithi165 ceph-mon[32648]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-26T01:53:37.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:37 smithi165 ceph-mon[32648]: Active manager daemon x restarted 2024-04-26T01:53:37.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:37 smithi165 ceph-mon[32648]: Activating manager daemon x 2024-04-26T01:53:37.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:37 smithi165 ceph-mon[32648]: osdmap e93: 12 total, 12 up, 12 in 2024-04-26T01:53:37.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:37 smithi165 ceph-mon[32648]: mgrmap e22: x(active, starting, since 0.0251204s), standbys: y, z 2024-04-26T01:53:37.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:37 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T01:53:37.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:37 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:53:37.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:37 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:53:37.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:37 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T01:53:37.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:37 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T01:53:37.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:37 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T01:53:37.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:37 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T01:53:37.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:37 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T01:53:37.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:37 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T01:53:37.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:37 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T01:53:37.598 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:37 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T01:53:37.598 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:37 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T01:53:37.598 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:37 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T01:53:37.598 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:37 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T01:53:37.598 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:37 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T01:53:37.598 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:37 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T01:53:37.598 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:37 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T01:53:37.598 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:37 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T01:53:37.598 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:37 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:53:37.598 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:37 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:53:37.598 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:37 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T01:53:37.598 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:37 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:53:37.598 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:37 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T01:53:37.599 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:37 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T01:53:37.599 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:37 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:53:37.599 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:37 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T01:53:37.599 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:37 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T01:53:37.599 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:37 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:53:37.599 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:37 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T01:53:37.599 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:37 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T01:53:37.599 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:37 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T01:53:37.599 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:37 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T01:53:37.599 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:37 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T01:53:37.599 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:37 smithi165 ceph-mon[32648]: Manager daemon x is now available 2024-04-26T01:53:37.599 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:37 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:53:37.599 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:37 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T01:53:37.599 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:37 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T01:53:37.600 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:37 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T01:53:37.600 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:37 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T01:53:37.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.680+0000 7fb10e000640 1 -- 172.21.15.151:0/4060697182 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mgrmap(e 23) ==== 138458+0+0 (secure 0 0 0) 0x7fb104098660 con 0x7fb1101085e0 2024-04-26T01:53:37.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.681+0000 7fb10e000640 1 --2- 172.21.15.151:0/4060697182 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fb0f80aa850 0x7fb0f80acc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:37.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.681+0000 7fb10e000640 1 -- 172.21.15.151:0/4060697182 --> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7fb10401fe50 con 0x7fb0f80aa850 2024-04-26T01:53:37.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.686+0000 7fb115000640 1 --2- 172.21.15.151:0/4060697182 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fb0f80aa850 0x7fb0f80acc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:37.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.692+0000 7fb115000640 1 --2- 172.21.15.151:0/4060697182 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fb0f80aa850 0x7fb0f80acc40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc000c00 tx=0x7fb0fc0314a0 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:37.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.712+0000 7fb10e000640 1 -- 172.21.15.151:0/4060697182 <== mgr.25025 v2:172.21.15.151:6800/2427564083 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7fb10401fe50 con 0x7fb0f80aa850 2024-04-26T01:53:37.710 INFO:teuthology.orchestra.run.smithi151.stdout:{} 2024-04-26T01:53:37.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.714+0000 7fb118400640 1 -- 172.21.15.151:0/4060697182 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fb0f80aa850 msgr2=0x7fb0f80acc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:37.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.714+0000 7fb118400640 1 --2- 172.21.15.151:0/4060697182 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fb0f80aa850 0x7fb0f80acc40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc000c00 tx=0x7fb0fc0314a0 comp rx=0 tx=0).stop 2024-04-26T01:53:37.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.714+0000 7fb118400640 1 -- 172.21.15.151:0/4060697182 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb1101085e0 msgr2=0x7fb11019e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:37.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.714+0000 7fb118400640 1 --2- 172.21.15.151:0/4060697182 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb1101085e0 0x7fb11019e760 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb10400e9c0 tx=0x7fb10400ee80 comp rx=0 tx=0).stop 2024-04-26T01:53:37.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.714+0000 7fb118400640 1 -- 172.21.15.151:0/4060697182 shutdown_connections 2024-04-26T01:53:37.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.714+0000 7fb118400640 1 --2- 172.21.15.151:0/4060697182 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fb0f80aa850 0x7fb0f80acc40 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:37.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.714+0000 7fb118400640 1 --2- 172.21.15.151:0/4060697182 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb11010a190 0x7fb1101a5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:37.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.714+0000 7fb118400640 1 --2- 172.21.15.151:0/4060697182 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1101097d0 0x7fb11019eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:37.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.714+0000 7fb118400640 1 --2- 172.21.15.151:0/4060697182 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb1101085e0 0x7fb11019e760 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:37.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.714+0000 7fb118400640 1 -- 172.21.15.151:0/4060697182 >> 172.21.15.151:0/4060697182 conn(0x7fb1100775a0 msgr2=0x7fb11010cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:37.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.715+0000 7fb118400640 1 -- 172.21.15.151:0/4060697182 shutdown_connections 2024-04-26T01:53:37.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.715+0000 7fb118400640 1 -- 172.21.15.151:0/4060697182 wait complete. 2024-04-26T01:53:37.731 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-26T01:53:37.741 INFO:teuthology.run_tasks:Running task exec... 2024-04-26T01:53:37.768 INFO:teuthology.task.exec:Executing custom commands... 2024-04-26T01:53:37.768 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi151.front.sepia.ceph.com 2024-04-26T01:53:37.768 DEBUG:teuthology.orchestra.run.smithi151:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-26T01:53:37.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.945+0000 7f50bac00640 1 -- 172.21.15.151:0/853429034 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50b4102850 msgr2=0x7f50b4104c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:37.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.945+0000 7f50bac00640 1 --2- 172.21.15.151:0/853429034 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50b4102850 0x7f50b4104c40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f50a8009900 tx=0x7f50a802a3f0 comp rx=0 tx=0).stop 2024-04-26T01:53:37.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.945+0000 7f50bac00640 1 -- 172.21.15.151:0/853429034 shutdown_connections 2024-04-26T01:53:37.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.946+0000 7f50bac00640 1 --2- 172.21.15.151:0/853429034 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50b4107ab0 0x7f50b4109f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:37.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.946+0000 7f50bac00640 1 --2- 172.21.15.151:0/853429034 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50b4105180 0x7f50b4107570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:37.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.946+0000 7f50bac00640 1 --2- 172.21.15.151:0/853429034 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50b4102850 0x7f50b4104c40 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:37.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.946+0000 7f50bac00640 1 -- 172.21.15.151:0/853429034 >> 172.21.15.151:0/853429034 conn(0x7f50b40fc240 msgr2=0x7f50b40fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:37.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.946+0000 7f50bac00640 1 -- 172.21.15.151:0/853429034 shutdown_connections 2024-04-26T01:53:37.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.946+0000 7f50bac00640 1 -- 172.21.15.151:0/853429034 wait complete. 2024-04-26T01:53:37.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.947+0000 7f50bac00640 1 Processor -- start 2024-04-26T01:53:37.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.947+0000 7f50bac00640 1 -- start start 2024-04-26T01:53:37.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.947+0000 7f50bac00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50b4102850 0x7f50b419a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:37.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.947+0000 7f50bac00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50b4105180 0x7f50b419a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:37.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.947+0000 7f50bac00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50b4107ab0 0x7f50b41a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:37.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.947+0000 7f50bac00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f50b410d920 con 0x7f50b4107ab0 2024-04-26T01:53:37.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.947+0000 7f50bac00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f50b410d7a0 con 0x7f50b4105180 2024-04-26T01:53:37.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.947+0000 7f50bac00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f50b410daa0 con 0x7f50b4102850 2024-04-26T01:53:37.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.947+0000 7f50ba200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50b4107ab0 0x7f50b41a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:37.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.947+0000 7f50b9800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50b4102850 0x7f50b419a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:37.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.947+0000 7f50ba200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50b4107ab0 0x7f50b41a1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:50218/0 (socket says 172.21.15.151:50218) 2024-04-26T01:53:37.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.947+0000 7f50ba200640 1 -- 172.21.15.151:0/3639888840 learned_addr learned my addr 172.21.15.151:0/3639888840 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:53:37.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.947+0000 7f50b8e00640 1 --2- 172.21.15.151:0/3639888840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50b4105180 0x7f50b419a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:37.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.948+0000 7f50ba200640 1 -- 172.21.15.151:0/3639888840 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50b4102850 msgr2=0x7f50b419a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:37.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.948+0000 7f50ba200640 1 --2- 172.21.15.151:0/3639888840 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50b4102850 0x7f50b419a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:37.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.948+0000 7f50ba200640 1 -- 172.21.15.151:0/3639888840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50b4105180 msgr2=0x7f50b419a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:37.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.948+0000 7f50ba200640 1 --2- 172.21.15.151:0/3639888840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50b4105180 0x7f50b419a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:37.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.948+0000 7f50ba200640 1 -- 172.21.15.151:0/3639888840 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50b41a1e80 con 0x7f50b4107ab0 2024-04-26T01:53:37.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.948+0000 7f50b9800640 1 --2- 172.21.15.151:0/3639888840 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50b4102850 0x7f50b419a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:53:37.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.948+0000 7f50ba200640 1 --2- 172.21.15.151:0/3639888840 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50b4107ab0 0x7f50b41a1810 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f50ac00eb10 tx=0x7f50ac00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:37.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.948+0000 7f50b2000640 1 -- 172.21.15.151:0/3639888840 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ac004b20 con 0x7f50b4107ab0 2024-04-26T01:53:37.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.948+0000 7f50bac00640 1 -- 172.21.15.151:0/3639888840 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50b41a2170 con 0x7f50b4107ab0 2024-04-26T01:53:37.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.948+0000 7f50b2000640 1 -- 172.21.15.151:0/3639888840 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50ac004cc0 con 0x7f50b4107ab0 2024-04-26T01:53:37.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.948+0000 7f50b2000640 1 -- 172.21.15.151:0/3639888840 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ac010670 con 0x7f50b4107ab0 2024-04-26T01:53:37.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.948+0000 7f50bac00640 1 -- 172.21.15.151:0/3639888840 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50b41a26c0 con 0x7f50b4107ab0 2024-04-26T01:53:37.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.950+0000 7f50bac00640 1 -- 172.21.15.151:0/3639888840 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50b4109600 con 0x7f50b4107ab0 2024-04-26T01:53:37.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.951+0000 7f50b2000640 1 -- 172.21.15.151:0/3639888840 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 23) ==== 138458+0+0 (secure 0 0 0) 0x7f50ac005020 con 0x7f50b4107ab0 2024-04-26T01:53:37.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.954+0000 7f50b2000640 1 --2- 172.21.15.151:0/3639888840 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f509c0a3990 0x7f509c0a5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:37.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.954+0000 7f50b2000640 1 -- 172.21.15.151:0/3639888840 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f50ac0d0610 con 0x7f50b4107ab0 2024-04-26T01:53:37.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.954+0000 7f50b9800640 1 --2- 172.21.15.151:0/3639888840 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f509c0a3990 0x7f509c0a5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:37.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.955+0000 7f50b9800640 1 --2- 172.21.15.151:0/3639888840 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f509c0a3990 0x7f509c0a5e50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f50a8000c00 tx=0x7f50a80314a0 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:37.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:37.957+0000 7f50b2000640 1 -- 172.21.15.151:0/3639888840 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f50ac09d1f0 con 0x7f50b4107ab0 2024-04-26T01:53:38.121 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:53:37 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:53:37.703+0000 7fb9bea00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:53:38.122 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:53:37 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:53:37.703+0000 7fb9bea00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:53:38.122 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:53:37 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:53:37.703+0000 7fb9bea00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:53:38.122 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:53:37 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:53:37.703+0000 7fb9bea00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:53:38.122 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:53:37 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:53:37.703+0000 7fb9bea00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:53:38.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.184+0000 7f50bac00640 1 -- 172.21.15.151:0/3639888840 --> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f50b4062390 con 0x7f509c0a3990 2024-04-26T01:53:38.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:38 smithi172 ceph-mon[32785]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-26T01:53:38.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:38 smithi172 ceph-mon[32785]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T01:53:38.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:38 smithi172 ceph-mon[32785]: mgrmap e23: x(active, since 1.03059s), standbys: y, z 2024-04-26T01:53:38.313 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.315+0000 7f50b2000640 1 -- 172.21.15.151:0/3639888840 <== mgr.25025 v2:172.21.15.151:6800/2427564083 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f50b4062390 con 0x7f509c0a3990 2024-04-26T01:53:38.313 INFO:teuthology.orchestra.run.smithi151.stdout:Schedule set for path / 2024-04-26T01:53:38.316 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.317+0000 7f50bac00640 1 -- 172.21.15.151:0/3639888840 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f509c0a3990 msgr2=0x7f509c0a5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:38.316 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.317+0000 7f50bac00640 1 --2- 172.21.15.151:0/3639888840 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f509c0a3990 0x7f509c0a5e50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f50a8000c00 tx=0x7f50a80314a0 comp rx=0 tx=0).stop 2024-04-26T01:53:38.316 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.317+0000 7f50bac00640 1 -- 172.21.15.151:0/3639888840 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50b4107ab0 msgr2=0x7f50b41a1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:38.317 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.317+0000 7f50bac00640 1 --2- 172.21.15.151:0/3639888840 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50b4107ab0 0x7f50b41a1810 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f50ac00eb10 tx=0x7f50ac00c2d0 comp rx=0 tx=0).stop 2024-04-26T01:53:38.317 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.318+0000 7f50bac00640 1 -- 172.21.15.151:0/3639888840 shutdown_connections 2024-04-26T01:53:38.317 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.318+0000 7f50bac00640 1 --2- 172.21.15.151:0/3639888840 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f509c0a3990 0x7f509c0a5e50 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:38.317 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.318+0000 7f50bac00640 1 --2- 172.21.15.151:0/3639888840 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50b4107ab0 0x7f50b41a1810 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:38.317 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.318+0000 7f50bac00640 1 --2- 172.21.15.151:0/3639888840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50b4105180 0x7f50b419a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:38.317 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.318+0000 7f50bac00640 1 --2- 172.21.15.151:0/3639888840 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50b4102850 0x7f50b419a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:38.317 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.318+0000 7f50bac00640 1 -- 172.21.15.151:0/3639888840 >> 172.21.15.151:0/3639888840 conn(0x7f50b40fc240 msgr2=0x7f50b41020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:38.317 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.318+0000 7f50bac00640 1 -- 172.21.15.151:0/3639888840 shutdown_connections 2024-04-26T01:53:38.317 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.318+0000 7f50bac00640 1 -- 172.21.15.151:0/3639888840 wait complete. 2024-04-26T01:53:38.336 DEBUG:teuthology.orchestra.run.smithi151:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-26T01:53:38.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:38 smithi165 ceph-mon[32648]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-26T01:53:38.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:38 smithi165 ceph-mon[32648]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T01:53:38.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:38 smithi165 ceph-mon[32648]: mgrmap e23: x(active, since 1.03059s), standbys: y, z 2024-04-26T01:53:38.392 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:38 smithi151 ceph-mon[28057]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-26T01:53:38.392 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:38 smithi151 ceph-mon[28057]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T01:53:38.392 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:38 smithi151 ceph-mon[28057]: mgrmap e23: x(active, since 1.03059s), standbys: y, z 2024-04-26T01:53:38.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.513+0000 7f8bfa800640 1 -- 172.21.15.151:0/295494704 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bf4105d90 msgr2=0x7f8bf410c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:38.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.513+0000 7f8bfa800640 1 --2- 172.21.15.151:0/295494704 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bf4105d90 0x7f8bf410c620 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8bec009900 tx=0x7f8bec02a3f0 comp rx=0 tx=0).stop 2024-04-26T01:53:38.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.513+0000 7f8bfa800640 1 -- 172.21.15.151:0/295494704 shutdown_connections 2024-04-26T01:53:38.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.513+0000 7f8bfa800640 1 --2- 172.21.15.151:0/295494704 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bf4105d90 0x7f8bf410c620 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:38.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.513+0000 7f8bfa800640 1 --2- 172.21.15.151:0/295494704 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8bf41053d0 0x7f8bf4105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:38.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.513+0000 7f8bfa800640 1 --2- 172.21.15.151:0/295494704 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8bf41041e0 0x7f8bf4104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:38.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.513+0000 7f8bfa800640 1 -- 172.21.15.151:0/295494704 >> 172.21.15.151:0/295494704 conn(0x7f8bf40ff770 msgr2=0x7f8bf4101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:38.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.513+0000 7f8bfa800640 1 -- 172.21.15.151:0/295494704 shutdown_connections 2024-04-26T01:53:38.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.514+0000 7f8bfa800640 1 -- 172.21.15.151:0/295494704 wait complete. 2024-04-26T01:53:38.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.514+0000 7f8bfa800640 1 Processor -- start 2024-04-26T01:53:38.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.515+0000 7f8bfa800640 1 -- start start 2024-04-26T01:53:38.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.515+0000 7f8bfa800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8bf41041e0 0x7f8bf419a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:38.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.515+0000 7f8bfa800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bf41053d0 0x7f8bf419a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:38.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.515+0000 7f8bfa800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8bf4105d90 0x7f8bf41a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:38.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.515+0000 7f8bfa800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf410fcf0 con 0x7f8bf41041e0 2024-04-26T01:53:38.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.515+0000 7f8bfa800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf410fb70 con 0x7f8bf4105d90 2024-04-26T01:53:38.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.515+0000 7f8bfa800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf410fe70 con 0x7f8bf41053d0 2024-04-26T01:53:38.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.515+0000 7f8bf3400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8bf41041e0 0x7f8bf419a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:38.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.515+0000 7f8beaa00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bf41053d0 0x7f8bf419a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:38.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.515+0000 7f8bf3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8bf4105d90 0x7f8bf41a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:38.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.515+0000 7f8beaa00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bf41053d0 0x7f8bf419a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:37834/0 (socket says 172.21.15.151:37834) 2024-04-26T01:53:38.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.515+0000 7f8bf3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8bf4105d90 0x7f8bf41a1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:46548/0 (socket says 172.21.15.151:46548) 2024-04-26T01:53:38.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.515+0000 7f8bf3400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8bf41041e0 0x7f8bf419a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:50234/0 (socket says 172.21.15.151:50234) 2024-04-26T01:53:38.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.515+0000 7f8bf3e00640 1 -- 172.21.15.151:0/954280463 learned_addr learned my addr 172.21.15.151:0/954280463 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:53:38.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.516+0000 7f8beaa00640 1 -- 172.21.15.151:0/954280463 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8bf4105d90 msgr2=0x7f8bf41a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:38.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.516+0000 7f8beaa00640 1 --2- 172.21.15.151:0/954280463 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8bf4105d90 0x7f8bf41a1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:38.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.516+0000 7f8beaa00640 1 -- 172.21.15.151:0/954280463 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8bf41041e0 msgr2=0x7f8bf419a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:38.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.516+0000 7f8beaa00640 1 --2- 172.21.15.151:0/954280463 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8bf41041e0 0x7f8bf419a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:38.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.516+0000 7f8beaa00640 1 -- 172.21.15.151:0/954280463 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bf41a1ef0 con 0x7f8bf41053d0 2024-04-26T01:53:38.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.516+0000 7f8beaa00640 1 --2- 172.21.15.151:0/954280463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bf41053d0 0x7f8bf419a810 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc00e9f0 tx=0x7f8bdc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:38.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.516+0000 7f8bf0c00640 1 -- 172.21.15.151:0/954280463 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bdc00cd80 con 0x7f8bf41053d0 2024-04-26T01:53:38.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.516+0000 7f8bf0c00640 1 -- 172.21.15.151:0/954280463 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bdc0035f0 con 0x7f8bf41053d0 2024-04-26T01:53:38.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.516+0000 7f8bf0c00640 1 -- 172.21.15.151:0/954280463 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bdc010670 con 0x7f8bf41053d0 2024-04-26T01:53:38.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.516+0000 7f8bfa800640 1 -- 172.21.15.151:0/954280463 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bf41a21e0 con 0x7f8bf41053d0 2024-04-26T01:53:38.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.516+0000 7f8bfa800640 1 -- 172.21.15.151:0/954280463 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bf41a26e0 con 0x7f8bf41053d0 2024-04-26T01:53:38.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.518+0000 7f8bfa800640 1 -- 172.21.15.151:0/954280463 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bf410b500 con 0x7f8bf41053d0 2024-04-26T01:53:38.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.519+0000 7f8bf0c00640 1 -- 172.21.15.151:0/954280463 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 23) ==== 138458+0+0 (secure 0 0 0) 0x7f8bdc0037a0 con 0x7f8bf41053d0 2024-04-26T01:53:38.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.521+0000 7f8bf0c00640 1 --2- 172.21.15.151:0/954280463 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f8bd80a3a60 0x7f8bd80a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:38.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.521+0000 7f8bf0c00640 1 -- 172.21.15.151:0/954280463 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f8bdc0d0360 con 0x7f8bf41053d0 2024-04-26T01:53:38.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.521+0000 7f8bf3400640 1 --2- 172.21.15.151:0/954280463 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f8bd80a3a60 0x7f8bd80a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:38.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.522+0000 7f8bf3400640 1 --2- 172.21.15.151:0/954280463 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f8bd80a3a60 0x7f8bd80a5f20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4105230 tx=0x7f8be4009500 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:38.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.525+0000 7f8bf0c00640 1 -- 172.21.15.151:0/954280463 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8bdc09cf40 con 0x7f8bf41053d0 2024-04-26T01:53:38.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.765+0000 7f8bfa800640 1 -- 172.21.15.151:0/954280463 --> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f8bf4062430 con 0x7f8bd80a3a60 2024-04-26T01:53:38.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.864+0000 7f8bf0c00640 1 -- 172.21.15.151:0/954280463 <== mgr.25025 v2:172.21.15.151:6800/2427564083 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f8bf4062430 con 0x7f8bd80a3a60 2024-04-26T01:53:38.863 INFO:teuthology.orchestra.run.smithi151.stdout:Retention added to path / 2024-04-26T01:53:38.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.867+0000 7f8bfa800640 1 -- 172.21.15.151:0/954280463 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f8bd80a3a60 msgr2=0x7f8bd80a5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:38.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.867+0000 7f8bfa800640 1 --2- 172.21.15.151:0/954280463 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f8bd80a3a60 0x7f8bd80a5f20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4105230 tx=0x7f8be4009500 comp rx=0 tx=0).stop 2024-04-26T01:53:38.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.867+0000 7f8bfa800640 1 -- 172.21.15.151:0/954280463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bf41053d0 msgr2=0x7f8bf419a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:38.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.867+0000 7f8bfa800640 1 --2- 172.21.15.151:0/954280463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bf41053d0 0x7f8bf419a810 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc00e9f0 tx=0x7f8bdc00eeb0 comp rx=0 tx=0).stop 2024-04-26T01:53:38.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.868+0000 7f8bfa800640 1 -- 172.21.15.151:0/954280463 shutdown_connections 2024-04-26T01:53:38.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.868+0000 7f8bfa800640 1 --2- 172.21.15.151:0/954280463 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f8bd80a3a60 0x7f8bd80a5f20 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:38.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.868+0000 7f8bfa800640 1 --2- 172.21.15.151:0/954280463 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8bf4105d90 0x7f8bf41a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:38.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.868+0000 7f8bfa800640 1 --2- 172.21.15.151:0/954280463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bf41053d0 0x7f8bf419a810 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:38.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.868+0000 7f8bfa800640 1 --2- 172.21.15.151:0/954280463 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8bf41041e0 0x7f8bf419a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:38.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.868+0000 7f8bfa800640 1 -- 172.21.15.151:0/954280463 >> 172.21.15.151:0/954280463 conn(0x7f8bf40ff770 msgr2=0x7f8bf41086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:38.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.868+0000 7f8bfa800640 1 -- 172.21.15.151:0/954280463 shutdown_connections 2024-04-26T01:53:38.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:38.868+0000 7f8bfa800640 1 -- 172.21.15.151:0/954280463 wait complete. 2024-04-26T01:53:38.884 DEBUG:teuthology.orchestra.run.smithi151:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-26T01:53:39.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.079+0000 7f9de6200640 1 -- 172.21.15.151:0/2434179124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9de01053d0 msgr2=0x7f9de0105850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:39.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.079+0000 7f9de6200640 1 --2- 172.21.15.151:0/2434179124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9de01053d0 0x7f9de0105850 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc009900 tx=0x7f9dcc02a410 comp rx=0 tx=0).stop 2024-04-26T01:53:39.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.079+0000 7f9de6200640 1 -- 172.21.15.151:0/2434179124 shutdown_connections 2024-04-26T01:53:39.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.079+0000 7f9de6200640 1 --2- 172.21.15.151:0/2434179124 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9de0105d90 0x7f9de010c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:39.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.079+0000 7f9de6200640 1 --2- 172.21.15.151:0/2434179124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9de01053d0 0x7f9de0105850 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:39.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.079+0000 7f9de6200640 1 --2- 172.21.15.151:0/2434179124 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9de01041e0 0x7f9de0104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:39.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.079+0000 7f9de6200640 1 -- 172.21.15.151:0/2434179124 >> 172.21.15.151:0/2434179124 conn(0x7f9de00ff770 msgr2=0x7f9de0101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:39.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.079+0000 7f9de6200640 1 -- 172.21.15.151:0/2434179124 shutdown_connections 2024-04-26T01:53:39.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.079+0000 7f9de6200640 1 -- 172.21.15.151:0/2434179124 wait complete. 2024-04-26T01:53:39.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.080+0000 7f9de6200640 1 Processor -- start 2024-04-26T01:53:39.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.080+0000 7f9de6200640 1 -- start start 2024-04-26T01:53:39.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.080+0000 7f9de6200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9de01041e0 0x7f9de019a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:39.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.080+0000 7f9de6200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9de01053d0 0x7f9de019a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:39.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.080+0000 7f9de6200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9de0105d90 0x7f9de01a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:39.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.080+0000 7f9de6200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9de010fd70 con 0x7f9de0105d90 2024-04-26T01:53:39.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.080+0000 7f9de6200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9de010fbf0 con 0x7f9de01041e0 2024-04-26T01:53:39.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.080+0000 7f9de6200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9de010fef0 con 0x7f9de01053d0 2024-04-26T01:53:39.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.080+0000 7f9ddf400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9de01041e0 0x7f9de019a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:39.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.081+0000 7f9ddf400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9de01041e0 0x7f9de019a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:46586/0 (socket says 172.21.15.151:46586) 2024-04-26T01:53:39.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.081+0000 7f9ddf400640 1 -- 172.21.15.151:0/1630440663 learned_addr learned my addr 172.21.15.151:0/1630440663 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:53:39.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.081+0000 7f9ddea00640 1 --2- 172.21.15.151:0/1630440663 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9de01053d0 0x7f9de019a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:39.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.081+0000 7f9ddfe00640 1 --2- 172.21.15.151:0/1630440663 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9de0105d90 0x7f9de01a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:39.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.081+0000 7f9ddea00640 1 -- 172.21.15.151:0/1630440663 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9de01041e0 msgr2=0x7f9de019a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:39.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.081+0000 7f9ddea00640 1 --2- 172.21.15.151:0/1630440663 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9de01041e0 0x7f9de019a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:39.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.081+0000 7f9ddea00640 1 -- 172.21.15.151:0/1630440663 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9de0105d90 msgr2=0x7f9de01a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:39.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.081+0000 7f9ddea00640 1 --2- 172.21.15.151:0/1630440663 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9de0105d90 0x7f9de01a18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:39.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.081+0000 7f9ddea00640 1 -- 172.21.15.151:0/1630440663 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9de01a1f60 con 0x7f9de01053d0 2024-04-26T01:53:39.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.081+0000 7f9ddfe00640 1 --2- 172.21.15.151:0/1630440663 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9de0105d90 0x7f9de01a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:53:39.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.081+0000 7f9ddf400640 1 --2- 172.21.15.151:0/1630440663 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9de01041e0 0x7f9de019a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:53:39.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.081+0000 7f9ddea00640 1 --2- 172.21.15.151:0/1630440663 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9de01053d0 0x7f9de019a880 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc02aa00 tx=0x7f9dcc003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:39.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.082+0000 7f9dd7e00640 1 -- 172.21.15.151:0/1630440663 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc03f070 con 0x7f9de01053d0 2024-04-26T01:53:39.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.082+0000 7f9dd7e00640 1 -- 172.21.15.151:0/1630440663 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dcc031c50 con 0x7f9de01053d0 2024-04-26T01:53:39.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.082+0000 7f9de6200640 1 -- 172.21.15.151:0/1630440663 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9de01a21f0 con 0x7f9de01053d0 2024-04-26T01:53:39.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.082+0000 7f9dd7e00640 1 -- 172.21.15.151:0/1630440663 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc035030 con 0x7f9de01053d0 2024-04-26T01:53:39.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.082+0000 7f9de6200640 1 -- 172.21.15.151:0/1630440663 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9de01a26b0 con 0x7f9de01053d0 2024-04-26T01:53:39.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.083+0000 7f9de6200640 1 -- 172.21.15.151:0/1630440663 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9de010b500 con 0x7f9de01053d0 2024-04-26T01:53:39.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.086+0000 7f9dd7e00640 1 -- 172.21.15.151:0/1630440663 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 23) ==== 138458+0+0 (secure 0 0 0) 0x7f9dcc042070 con 0x7f9de01053d0 2024-04-26T01:53:39.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.087+0000 7f9dd7e00640 1 --2- 172.21.15.151:0/1630440663 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f9dc40a3a60 0x7f9dc40a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:39.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.088+0000 7f9dd7e00640 1 -- 172.21.15.151:0/1630440663 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f9dcc0f0830 con 0x7f9de01053d0 2024-04-26T01:53:39.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.088+0000 7f9ddf400640 1 --2- 172.21.15.151:0/1630440663 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f9dc40a3a60 0x7f9dc40a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:39.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.088+0000 7f9ddf400640 1 --2- 172.21.15.151:0/1630440663 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f9dc40a3a60 0x7f9dc40a5f20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9dd00058c0 tx=0x7f9dd0005f90 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:39.090 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.092+0000 7f9dd7e00640 1 -- 172.21.15.151:0/1630440663 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9dcc0bd450 con 0x7f9de01053d0 2024-04-26T01:53:39.161 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.162+0000 7f9dd7e00640 1 -- 172.21.15.151:0/1630440663 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mgrmap(e 24) ==== 138818+0+0 (secure 0 0 0) 0x7f9dcc0b93c0 con 0x7f9de01053d0 2024-04-26T01:53:39.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.341+0000 7f9de6200640 1 -- 172.21.15.151:0/1630440663 --> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f9de0062390 con 0x7f9dc40a3a60 2024-04-26T01:53:39.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.351+0000 7f9dd7e00640 1 -- 172.21.15.151:0/1630440663 <== mgr.25025 v2:172.21.15.151:6800/2427564083 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f9de0062390 con 0x7f9dc40a3a60 2024-04-26T01:53:39.349 INFO:teuthology.orchestra.run.smithi151.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-26T00:00:00", "created": "2024-04-26T01:53:38", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-26T01:53:39.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.353+0000 7f9de6200640 1 -- 172.21.15.151:0/1630440663 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f9dc40a3a60 msgr2=0x7f9dc40a5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:39.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.353+0000 7f9de6200640 1 --2- 172.21.15.151:0/1630440663 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f9dc40a3a60 0x7f9dc40a5f20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9dd00058c0 tx=0x7f9dd0005f90 comp rx=0 tx=0).stop 2024-04-26T01:53:39.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.353+0000 7f9de6200640 1 -- 172.21.15.151:0/1630440663 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9de01053d0 msgr2=0x7f9de019a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:39.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.353+0000 7f9de6200640 1 --2- 172.21.15.151:0/1630440663 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9de01053d0 0x7f9de019a880 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc02aa00 tx=0x7f9dcc003c10 comp rx=0 tx=0).stop 2024-04-26T01:53:39.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.353+0000 7f9de6200640 1 -- 172.21.15.151:0/1630440663 shutdown_connections 2024-04-26T01:53:39.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.353+0000 7f9de6200640 1 --2- 172.21.15.151:0/1630440663 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f9dc40a3a60 0x7f9dc40a5f20 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:39.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.353+0000 7f9de6200640 1 --2- 172.21.15.151:0/1630440663 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9de0105d90 0x7f9de01a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:39.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.353+0000 7f9de6200640 1 --2- 172.21.15.151:0/1630440663 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9de01053d0 0x7f9de019a880 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:39.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.353+0000 7f9de6200640 1 --2- 172.21.15.151:0/1630440663 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9de01041e0 0x7f9de019a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:39.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.353+0000 7f9de6200640 1 -- 172.21.15.151:0/1630440663 >> 172.21.15.151:0/1630440663 conn(0x7f9de00ff770 msgr2=0x7f9de01086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:39.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.353+0000 7f9de6200640 1 -- 172.21.15.151:0/1630440663 shutdown_connections 2024-04-26T01:53:39.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.353+0000 7f9de6200640 1 -- 172.21.15.151:0/1630440663 wait complete. 2024-04-26T01:53:39.370 DEBUG:teuthology.orchestra.run.smithi151:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-26T01:53:39.433 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:39 smithi151 ceph-mon[28057]: [26/Apr/2024:01:53:37] ENGINE Bus STARTING 2024-04-26T01:53:39.433 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:39 smithi151 ceph-mon[28057]: [26/Apr/2024:01:53:37] ENGINE Serving on https://172.21.15.151:7150 2024-04-26T01:53:39.433 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:39 smithi151 ceph-mon[28057]: from='client.25111 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:53:39.433 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:39 smithi151 ceph-mon[28057]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:39.433 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:39 smithi151 ceph-mon[28057]: [26/Apr/2024:01:53:37] ENGINE Serving on http://172.21.15.151:8765 2024-04-26T01:53:39.433 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:39 smithi151 ceph-mon[28057]: [26/Apr/2024:01:53:37] ENGINE Bus STARTED 2024-04-26T01:53:39.434 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:39 smithi151 ceph-mon[28057]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:53:39.434 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:39 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:53:39.434 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:39 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:53:39.434 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:39 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:53:39.434 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:39 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:53:39.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:39 smithi172 ceph-mon[32785]: [26/Apr/2024:01:53:37] ENGINE Bus STARTING 2024-04-26T01:53:39.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:39 smithi172 ceph-mon[32785]: [26/Apr/2024:01:53:37] ENGINE Serving on https://172.21.15.151:7150 2024-04-26T01:53:39.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:39 smithi172 ceph-mon[32785]: from='client.25111 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:53:39.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:39 smithi172 ceph-mon[32785]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:39.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:39 smithi172 ceph-mon[32785]: [26/Apr/2024:01:53:37] ENGINE Serving on http://172.21.15.151:8765 2024-04-26T01:53:39.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:39 smithi172 ceph-mon[32785]: [26/Apr/2024:01:53:37] ENGINE Bus STARTED 2024-04-26T01:53:39.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:39 smithi172 ceph-mon[32785]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:53:39.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:39 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:53:39.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:39 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:53:39.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:39 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:53:39.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:39 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:53:39.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.556+0000 7fb043a00640 1 -- 172.21.15.151:0/1673236832 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb03c1031d0 msgr2=0x7fb03c103650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:39.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.556+0000 7fb043a00640 1 --2- 172.21.15.151:0/1673236832 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb03c1031d0 0x7fb03c103650 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb028009860 tx=0x7fb02802a340 comp rx=0 tx=0).stop 2024-04-26T01:53:39.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.556+0000 7fb043a00640 1 -- 172.21.15.151:0/1673236832 shutdown_connections 2024-04-26T01:53:39.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.556+0000 7fb043a00640 1 --2- 172.21.15.151:0/1673236832 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb03c103b90 0x7fb03c10a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:39.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.556+0000 7fb043a00640 1 --2- 172.21.15.151:0/1673236832 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb03c1031d0 0x7fb03c103650 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:39.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.556+0000 7fb043a00640 1 --2- 172.21.15.151:0/1673236832 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb03c101fe0 0x7fb03c102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:39.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.556+0000 7fb043a00640 1 -- 172.21.15.151:0/1673236832 >> 172.21.15.151:0/1673236832 conn(0x7fb03c0fd570 msgr2=0x7fb03c0ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:39.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.556+0000 7fb043a00640 1 -- 172.21.15.151:0/1673236832 shutdown_connections 2024-04-26T01:53:39.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.556+0000 7fb043a00640 1 -- 172.21.15.151:0/1673236832 wait complete. 2024-04-26T01:53:39.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.557+0000 7fb043a00640 1 Processor -- start 2024-04-26T01:53:39.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.558+0000 7fb043a00640 1 -- start start 2024-04-26T01:53:39.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.558+0000 7fb043a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb03c101fe0 0x7fb03c107d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:39.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.558+0000 7fb043a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb03c1031d0 0x7fb03c1082a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:39.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.558+0000 7fb043a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb03c103b90 0x7fb03c1048d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:39.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.558+0000 7fb043a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb03c10db20 con 0x7fb03c101fe0 2024-04-26T01:53:39.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.558+0000 7fb043a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb03c10d9a0 con 0x7fb03c103b90 2024-04-26T01:53:39.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.558+0000 7fb043a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb03c10dca0 con 0x7fb03c1031d0 2024-04-26T01:53:39.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.558+0000 7fb041000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb03c101fe0 0x7fb03c107d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:39.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.558+0000 7fb03be00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb03c1031d0 0x7fb03c1082a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:39.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.558+0000 7fb041000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb03c101fe0 0x7fb03c107d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:50296/0 (socket says 172.21.15.151:50296) 2024-04-26T01:53:39.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.558+0000 7fb041a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb03c103b90 0x7fb03c1048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:39.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.558+0000 7fb041000640 1 -- 172.21.15.151:0/4238020705 learned_addr learned my addr 172.21.15.151:0/4238020705 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:53:39.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.559+0000 7fb041000640 1 -- 172.21.15.151:0/4238020705 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb03c1031d0 msgr2=0x7fb03c1082a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:39.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.559+0000 7fb041000640 1 --2- 172.21.15.151:0/4238020705 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb03c1031d0 0x7fb03c1082a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:39.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.559+0000 7fb041000640 1 -- 172.21.15.151:0/4238020705 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb03c103b90 msgr2=0x7fb03c1048d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:39.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.559+0000 7fb041000640 1 --2- 172.21.15.151:0/4238020705 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb03c103b90 0x7fb03c1048d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:39.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.559+0000 7fb041000640 1 -- 172.21.15.151:0/4238020705 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb03c105190 con 0x7fb03c101fe0 2024-04-26T01:53:39.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.559+0000 7fb03be00640 1 --2- 172.21.15.151:0/4238020705 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb03c1031d0 0x7fb03c1082a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:53:39.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.559+0000 7fb041000640 1 --2- 172.21.15.151:0/4238020705 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb03c101fe0 0x7fb03c107d60 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb03000e9a0 tx=0x7fb03000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:39.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.559+0000 7fb039600640 1 -- 172.21.15.151:0/4238020705 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03000cd80 con 0x7fb03c101fe0 2024-04-26T01:53:39.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.559+0000 7fb043a00640 1 -- 172.21.15.151:0/4238020705 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb03c1a6780 con 0x7fb03c101fe0 2024-04-26T01:53:39.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.559+0000 7fb039600640 1 -- 172.21.15.151:0/4238020705 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0300108e0 con 0x7fb03c101fe0 2024-04-26T01:53:39.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.559+0000 7fb039600640 1 -- 172.21.15.151:0/4238020705 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb030018960 con 0x7fb03c101fe0 2024-04-26T01:53:39.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.559+0000 7fb043a00640 1 -- 172.21.15.151:0/4238020705 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb03c1a6be0 con 0x7fb03c101fe0 2024-04-26T01:53:39.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.560+0000 7fb043a00640 1 -- 172.21.15.151:0/4238020705 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb03c04fce0 con 0x7fb03c101fe0 2024-04-26T01:53:39.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.562+0000 7fb039600640 1 -- 172.21.15.151:0/4238020705 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 24) ==== 138818+0+0 (secure 0 0 0) 0x7fb030010a90 con 0x7fb03c101fe0 2024-04-26T01:53:39.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.565+0000 7fb039600640 1 --2- 172.21.15.151:0/4238020705 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fb0240a3e60 0x7fb0240a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:39.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.565+0000 7fb039600640 1 -- 172.21.15.151:0/4238020705 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fb0300d0cc0 con 0x7fb03c101fe0 2024-04-26T01:53:39.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.565+0000 7fb03be00640 1 --2- 172.21.15.151:0/4238020705 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fb0240a3e60 0x7fb0240a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:39.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.566+0000 7fb03be00640 1 --2- 172.21.15.151:0/4238020705 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fb0240a3e60 0x7fb0240a6320 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb02802a930 tx=0x7fb028030450 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:39.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.568+0000 7fb039600640 1 -- 172.21.15.151:0/4238020705 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb030053ff0 con 0x7fb03c101fe0 2024-04-26T01:53:39.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:39 smithi165 ceph-mon[32648]: [26/Apr/2024:01:53:37] ENGINE Bus STARTING 2024-04-26T01:53:39.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:39 smithi165 ceph-mon[32648]: [26/Apr/2024:01:53:37] ENGINE Serving on https://172.21.15.151:7150 2024-04-26T01:53:39.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:39 smithi165 ceph-mon[32648]: from='client.25111 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:53:39.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:39 smithi165 ceph-mon[32648]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:39.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:39 smithi165 ceph-mon[32648]: [26/Apr/2024:01:53:37] ENGINE Serving on http://172.21.15.151:8765 2024-04-26T01:53:39.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:39 smithi165 ceph-mon[32648]: [26/Apr/2024:01:53:37] ENGINE Bus STARTED 2024-04-26T01:53:39.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:39 smithi165 ceph-mon[32648]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:53:39.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:39 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:53:39.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:39 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:53:39.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:39 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:53:39.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:39 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:53:39.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.807+0000 7fb043a00640 1 -- 172.21.15.151:0/4238020705 --> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fb03c0758a0 con 0x7fb0240a3e60 2024-04-26T01:53:39.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.815+0000 7fb039600640 1 -- 172.21.15.151:0/4238020705 <== mgr.25025 v2:172.21.15.151:6800/2427564083 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fb03c0758a0 con 0x7fb0240a3e60 2024-04-26T01:53:39.813 INFO:teuthology.orchestra.run.smithi151.stdout:/ 1m 6m3h 2024-04-26T01:53:39.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.817+0000 7fb043a00640 1 -- 172.21.15.151:0/4238020705 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fb0240a3e60 msgr2=0x7fb0240a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:39.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.817+0000 7fb043a00640 1 --2- 172.21.15.151:0/4238020705 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fb0240a3e60 0x7fb0240a6320 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb02802a930 tx=0x7fb028030450 comp rx=0 tx=0).stop 2024-04-26T01:53:39.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.817+0000 7fb043a00640 1 -- 172.21.15.151:0/4238020705 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb03c101fe0 msgr2=0x7fb03c107d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:39.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.817+0000 7fb043a00640 1 --2- 172.21.15.151:0/4238020705 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb03c101fe0 0x7fb03c107d60 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb03000e9a0 tx=0x7fb03000ee60 comp rx=0 tx=0).stop 2024-04-26T01:53:39.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.818+0000 7fb043a00640 1 -- 172.21.15.151:0/4238020705 shutdown_connections 2024-04-26T01:53:39.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.818+0000 7fb043a00640 1 --2- 172.21.15.151:0/4238020705 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fb0240a3e60 0x7fb0240a6320 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:39.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.818+0000 7fb043a00640 1 --2- 172.21.15.151:0/4238020705 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb03c103b90 0x7fb03c1048d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:39.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.818+0000 7fb043a00640 1 --2- 172.21.15.151:0/4238020705 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb03c1031d0 0x7fb03c1082a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:39.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.818+0000 7fb043a00640 1 --2- 172.21.15.151:0/4238020705 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb03c101fe0 0x7fb03c107d60 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:39.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.818+0000 7fb043a00640 1 -- 172.21.15.151:0/4238020705 >> 172.21.15.151:0/4238020705 conn(0x7fb03c0fd570 msgr2=0x7fb03c074180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:39.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.818+0000 7fb043a00640 1 -- 172.21.15.151:0/4238020705 shutdown_connections 2024-04-26T01:53:39.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:39.818+0000 7fb043a00640 1 -- 172.21.15.151:0/4238020705 wait complete. 2024-04-26T01:53:39.834 DEBUG:teuthology.orchestra.run.smithi151:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-26T01:53:39.894 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-26T01:53:39.905 INFO:teuthology.run_tasks:Running task background_exec... 2024-04-26T01:53:39.924 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi151.front.sepia.ceph.com 2024-04-26T01:53:39.924 DEBUG:teuthology.orchestra.run.smithi151:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-04-26T01:53:39.931 INFO:teuthology.run_tasks:Running task quiescer... 2024-04-26T01:53:39.942 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-26T01:53:39.943 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T01:53:40.334 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:53:40.477 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:40 smithi151 ceph-mon[28057]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:40.477 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:40 smithi151 ceph-mon[28057]: from='client.25129 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:53:40.478 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:40 smithi151 ceph-mon[28057]: mgrmap e24: x(active, since 2s), standbys: y, z 2024-04-26T01:53:40.478 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:40 smithi151 ceph-mon[28057]: from='client.25141 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:53:40.478 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:40 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:53:40.478 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:40 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:53:40.478 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:40 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:53:40.478 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:40 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:53:40.478 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:40 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:40.478 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:40 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:40.478 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:40 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:40.478 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:40 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:40.479 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:40 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:40.479 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:40 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:40.479 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:40 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:40.479 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:40 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:40.479 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:40 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:53:40.479 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:40 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:53:40.479 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:40 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:40.479 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:40 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:40.479 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:40 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:40.479 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:40 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:40.479 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:40 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:40.479 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:40 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:40.479 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:40 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:40.480 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:40 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:40.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:40 smithi172 ceph-mon[32785]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:40.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:40 smithi172 ceph-mon[32785]: from='client.25129 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:53:40.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:40 smithi172 ceph-mon[32785]: mgrmap e24: x(active, since 2s), standbys: y, z 2024-04-26T01:53:40.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:40 smithi172 ceph-mon[32785]: from='client.25141 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:53:40.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:40 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:53:40.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:40 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:53:40.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:40 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:53:40.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:40 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:53:40.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:40 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:40.538 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:40 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:40.538 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:40 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:40.538 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:40 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:40.538 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:40 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:40.538 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:40 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:40.538 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:40 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:40.538 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:40 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:40.538 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:40 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:53:40.538 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:40 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:53:40.538 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:40 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:40.538 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:40 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:40.538 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:40 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:40.538 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:40 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:40.539 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:40 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:40.539 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:40 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:40.539 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:40 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:40.539 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:40 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:40.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:40 smithi165 ceph-mon[32648]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:40.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:40 smithi165 ceph-mon[32648]: from='client.25129 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:53:40.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:40 smithi165 ceph-mon[32648]: mgrmap e24: x(active, since 2s), standbys: y, z 2024-04-26T01:53:40.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:40 smithi165 ceph-mon[32648]: from='client.25141 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:53:40.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:40 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:53:40.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:40 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:53:40.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:40 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:53:40.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:40 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:53:40.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:40 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:40.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:40 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:40.598 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:40 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:40.598 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:40 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:40.598 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:40 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:40.598 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:40 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:40.598 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:40 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:40.598 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:40 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:40.598 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:40 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:53:40.598 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:40 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:53:40.598 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:40 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:40.598 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:40 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:40.598 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:40 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:40.598 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:40 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:40.599 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:40 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:40.599 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:40 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:40.599 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:40 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:40.599 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:40 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:41.512 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:41 smithi151 ceph-mon[28057]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:53:41.512 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:41 smithi151 ceph-mon[28057]: Adjusting osd_memory_target on smithi172 to 176.2M 2024-04-26T01:53:41.512 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:41 smithi151 ceph-mon[28057]: Unable to set osd_memory_target on smithi172 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-26T01:53:41.512 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:41 smithi151 ceph-mon[28057]: Adjusting osd_memory_target on smithi165 to 176.2M 2024-04-26T01:53:41.512 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:41 smithi151 ceph-mon[28057]: Unable to set osd_memory_target on smithi165 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-26T01:53:41.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:41 smithi172 ceph-mon[32785]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:53:41.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:41 smithi172 ceph-mon[32785]: Adjusting osd_memory_target on smithi172 to 176.2M 2024-04-26T01:53:41.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:41 smithi172 ceph-mon[32785]: Unable to set osd_memory_target on smithi172 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-26T01:53:41.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:41 smithi172 ceph-mon[32785]: Adjusting osd_memory_target on smithi165 to 176.2M 2024-04-26T01:53:41.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:41 smithi172 ceph-mon[32785]: Unable to set osd_memory_target on smithi165 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-26T01:53:41.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:41 smithi165 ceph-mon[32648]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:53:41.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:41 smithi165 ceph-mon[32648]: Adjusting osd_memory_target on smithi172 to 176.2M 2024-04-26T01:53:41.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:41 smithi165 ceph-mon[32648]: Unable to set osd_memory_target on smithi172 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-26T01:53:41.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:41 smithi165 ceph-mon[32648]: Adjusting osd_memory_target on smithi165 to 176.2M 2024-04-26T01:53:41.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:41 smithi165 ceph-mon[32648]: Unable to set osd_memory_target on smithi165 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-26T01:53:42.403 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:42 smithi151 ceph-mon[28057]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:42.403 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:42 smithi151 ceph-mon[28057]: mgrmap e25: x(active, since 4s), standbys: y, z 2024-04-26T01:53:42.403 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:42 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:53:42.403 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:42 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:53:42.403 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:42 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:42.403 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:42 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:42.403 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:42 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:42.403 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:42 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:42.403 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:42 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:42.403 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:42 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:42.403 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:42 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:42.404 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:42 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:42.404 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:42 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:53:42.404 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:42 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:53:42.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:42 smithi172 ceph-mon[32785]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:42.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:42 smithi172 ceph-mon[32785]: mgrmap e25: x(active, since 4s), standbys: y, z 2024-04-26T01:53:42.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:42 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:53:42.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:42 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:53:42.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:42 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:42.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:42 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:42.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:42 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:42.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:42 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:42.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:42 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:42.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:42 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:42.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:42 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:42.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:42 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:42.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:42 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:53:42.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:42 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:53:42.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:42 smithi165 ceph-mon[32648]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:42.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:42 smithi165 ceph-mon[32648]: mgrmap e25: x(active, since 4s), standbys: y, z 2024-04-26T01:53:42.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:42 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:53:42.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:42 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:53:42.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:42 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:42.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:42 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:42.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:42 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:42.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:42 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:42.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:42 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:42.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:42 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:42.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:42 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:42.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:42 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:53:42.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:42 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:53:42.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:42 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:53:42.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:42.741+0000 7f253d800700 1 -- 172.21.15.151:0/2946221089 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2538108a30 msgr2=0x7f2538108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:42.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:42.741+0000 7f253d800700 1 --2- 172.21.15.151:0/2946221089 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2538108a30 0x7f2538108e10 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f25280098f0 tx=0x7f2528009c00 comp rx=0 tx=0).stop 2024-04-26T01:53:42.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:42.741+0000 7f253d800700 1 -- 172.21.15.151:0/2946221089 shutdown_connections 2024-04-26T01:53:42.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:42.741+0000 7f253d800700 1 --2- 172.21.15.151:0/2946221089 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f253806b0f0 0x7f253810f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:42.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:42.741+0000 7f253d800700 1 --2- 172.21.15.151:0/2946221089 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2538108a30 0x7f2538108e10 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:42.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:42.742+0000 7f253d800700 1 --2- 172.21.15.151:0/2946221089 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f253806a750 0x7f253806abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:42.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:42.742+0000 7f253d800700 1 -- 172.21.15.151:0/2946221089 >> 172.21.15.151:0/2946221089 conn(0x7f25380788f0 msgr2=0x7f2538078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:42.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:42.742+0000 7f253d800700 1 -- 172.21.15.151:0/2946221089 shutdown_connections 2024-04-26T01:53:42.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:42.742+0000 7f253d800700 1 -- 172.21.15.151:0/2946221089 wait complete. 2024-04-26T01:53:42.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:42.743+0000 7f253d800700 1 Processor -- start 2024-04-26T01:53:42.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:42.743+0000 7f253d800700 1 -- start start 2024-04-26T01:53:42.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:42.744+0000 7f253d800700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f253806a750 0x7f253819b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:42.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:42.744+0000 7f253d800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f253806b0f0 0x7f253819b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:42.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:42.744+0000 7f253d800700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2538108a30 0x7f253819fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:42.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:42.744+0000 7f253d800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2538112c00 con 0x7f253806b0f0 2024-04-26T01:53:42.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:42.744+0000 7f253d800700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2538112a80 con 0x7f2538108a30 2024-04-26T01:53:42.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:42.744+0000 7f253d800700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2538112d80 con 0x7f253806a750 2024-04-26T01:53:42.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:42.744+0000 7f2536000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f253806b0f0 0x7f253819b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:42.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:42.744+0000 7f2536a00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f253806a750 0x7f253819b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:42.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:42.744+0000 7f2536a00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f253806a750 0x7f253819b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:58010/0 (socket says 172.21.15.151:58010) 2024-04-26T01:53:42.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:42.744+0000 7f2537400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2538108a30 0x7f253819fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:42.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:42.744+0000 7f2536a00700 1 -- 172.21.15.151:0/2426366281 learned_addr learned my addr 172.21.15.151:0/2426366281 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:53:42.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:42.745+0000 7f2536000700 1 -- 172.21.15.151:0/2426366281 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f253806a750 msgr2=0x7f253819b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:42.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:42.745+0000 7f2536000700 1 --2- 172.21.15.151:0/2426366281 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f253806a750 0x7f253819b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:42.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:42.745+0000 7f2536000700 1 -- 172.21.15.151:0/2426366281 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2538108a30 msgr2=0x7f253819fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:42.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:42.745+0000 7f2536000700 1 --2- 172.21.15.151:0/2426366281 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2538108a30 0x7f253819fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:42.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:42.745+0000 7f2536000700 1 -- 172.21.15.151:0/2426366281 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25381a03e0 con 0x7f253806b0f0 2024-04-26T01:53:42.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:42.745+0000 7f2536a00700 1 --2- 172.21.15.151:0/2426366281 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f253806a750 0x7f253819b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:53:42.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:42.745+0000 7f2537400700 1 --2- 172.21.15.151:0/2426366281 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2538108a30 0x7f253819fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:53:42.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:42.745+0000 7f2536000700 1 --2- 172.21.15.151:0/2426366281 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f253806b0f0 0x7f253819b910 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f252400e9c0 tx=0x7f252400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:42.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:42.746+0000 7f252f400700 1 -- 172.21.15.151:0/2426366281 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252400c9c0 con 0x7f253806b0f0 2024-04-26T01:53:42.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:42.746+0000 7f252f400700 1 -- 172.21.15.151:0/2426366281 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f252400cb60 con 0x7f253806b0f0 2024-04-26T01:53:42.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:42.746+0000 7f253d800700 1 -- 172.21.15.151:0/2426366281 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25381a06d0 con 0x7f253806b0f0 2024-04-26T01:53:42.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:42.746+0000 7f252f400700 1 -- 172.21.15.151:0/2426366281 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2524018420 con 0x7f253806b0f0 2024-04-26T01:53:42.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:42.746+0000 7f253d800700 1 -- 172.21.15.151:0/2426366281 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25381a7f50 con 0x7f253806b0f0 2024-04-26T01:53:42.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:42.747+0000 7f253d800700 1 -- 172.21.15.151:0/2426366281 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2538069730 con 0x7f253806b0f0 2024-04-26T01:53:42.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:42.749+0000 7f252f400700 1 -- 172.21.15.151:0/2426366281 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 25) ==== 138874+0+0 (secure 0 0 0) 0x7f2524010430 con 0x7f253806b0f0 2024-04-26T01:53:42.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:42.751+0000 7f252f400700 1 --2- 172.21.15.151:0/2426366281 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f251c0a3ed0 0x7f251c0a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:42.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:42.751+0000 7f252f400700 1 -- 172.21.15.151:0/2426366281 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f25240d0f70 con 0x7f253806b0f0 2024-04-26T01:53:42.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:42.751+0000 7f2536a00700 1 --2- 172.21.15.151:0/2426366281 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f251c0a3ed0 0x7f251c0a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:42.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:42.752+0000 7f2536a00700 1 --2- 172.21.15.151:0/2426366281 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f251c0a3ed0 0x7f251c0a6390 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2528009fd0 tx=0x7f2528019040 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:42.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:42.754+0000 7f252f400700 1 -- 172.21.15.151:0/2426366281 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f252409d3d0 con 0x7f253806b0f0 2024-04-26T01:53:42.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:42.997+0000 7f253d800700 1 -- 172.21.15.151:0/2426366281 --> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2538062e80 con 0x7f251c0a3ed0 2024-04-26T01:53:43.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:43.008+0000 7f252f400700 1 -- 172.21.15.151:0/2426366281 <== mgr.25025 v2:172.21.15.151:6800/2427564083 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257764 (secure 0 0 0) 0x7f2538062e80 con 0x7f251c0a3ed0 2024-04-26T01:53:43.009 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:53:43.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:43.013+0000 7f253d800700 1 -- 172.21.15.151:0/2426366281 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f251c0a3ed0 msgr2=0x7f251c0a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:43.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:43.013+0000 7f253d800700 1 --2- 172.21.15.151:0/2426366281 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f251c0a3ed0 0x7f251c0a6390 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2528009fd0 tx=0x7f2528019040 comp rx=0 tx=0).stop 2024-04-26T01:53:43.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:43.013+0000 7f253d800700 1 -- 172.21.15.151:0/2426366281 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f253806b0f0 msgr2=0x7f253819b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:43.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:43.013+0000 7f253d800700 1 --2- 172.21.15.151:0/2426366281 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f253806b0f0 0x7f253819b910 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f252400e9c0 tx=0x7f252400ecd0 comp rx=0 tx=0).stop 2024-04-26T01:53:43.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:43.013+0000 7f253d800700 1 -- 172.21.15.151:0/2426366281 shutdown_connections 2024-04-26T01:53:43.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:43.013+0000 7f253d800700 1 --2- 172.21.15.151:0/2426366281 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f251c0a3ed0 0x7f251c0a6390 secure :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2528009fd0 tx=0x7f2528019040 comp rx=0 tx=0).stop 2024-04-26T01:53:43.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:43.013+0000 7f253d800700 1 --2- 172.21.15.151:0/2426366281 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2538108a30 0x7f253819fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:43.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:43.013+0000 7f253d800700 1 --2- 172.21.15.151:0/2426366281 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f253806a750 0x7f253819b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:43.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:43.013+0000 7f253d800700 1 --2- 172.21.15.151:0/2426366281 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f253806b0f0 0x7f253819b910 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:43.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:43.013+0000 7f253d800700 1 -- 172.21.15.151:0/2426366281 >> 172.21.15.151:0/2426366281 conn(0x7f25380788f0 msgr2=0x7f253810df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:43.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:43.014+0000 7f253d800700 1 -- 172.21.15.151:0/2426366281 shutdown_connections 2024-04-26T01:53:43.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:43.014+0000 7f253d800700 1 -- 172.21.15.151:0/2426366281 wait complete. 2024-04-26T01:53:43.015 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-04-26T01:53:43.369 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:43 smithi151 ceph-mon[28057]: Adjusting osd_memory_target on smithi151 to 176.4M 2024-04-26T01:53:43.369 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:43 smithi151 ceph-mon[28057]: Unable to set osd_memory_target on smithi151 to 185064857: error parsing value: Value '185064857' is below minimum 939524096 2024-04-26T01:53:43.370 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:43 smithi151 ceph-mon[28057]: Updating smithi151:/etc/ceph/ceph.conf 2024-04-26T01:53:43.370 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:43 smithi151 ceph-mon[28057]: Updating smithi165:/etc/ceph/ceph.conf 2024-04-26T01:53:43.370 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:43 smithi151 ceph-mon[28057]: Updating smithi172:/etc/ceph/ceph.conf 2024-04-26T01:53:43.370 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:43 smithi151 ceph-mon[28057]: Updating smithi165:/var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:53:43.370 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:43 smithi151 ceph-mon[28057]: Updating smithi172:/var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:53:43.370 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:43 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:53:43.370 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:43 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:53:43.370 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:43 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:53:43.370 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:43 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:53:43.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:43 smithi172 ceph-mon[32785]: Adjusting osd_memory_target on smithi151 to 176.4M 2024-04-26T01:53:43.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:43 smithi172 ceph-mon[32785]: Unable to set osd_memory_target on smithi151 to 185064857: error parsing value: Value '185064857' is below minimum 939524096 2024-04-26T01:53:43.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:43 smithi172 ceph-mon[32785]: Updating smithi151:/etc/ceph/ceph.conf 2024-04-26T01:53:43.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:43 smithi172 ceph-mon[32785]: Updating smithi165:/etc/ceph/ceph.conf 2024-04-26T01:53:43.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:43 smithi172 ceph-mon[32785]: Updating smithi172:/etc/ceph/ceph.conf 2024-04-26T01:53:43.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:43 smithi172 ceph-mon[32785]: Updating smithi165:/var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:53:43.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:43 smithi172 ceph-mon[32785]: Updating smithi172:/var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:53:43.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:43 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:53:43.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:43 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:53:43.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:43 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:53:43.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:43 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:53:43.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:43 smithi165 ceph-mon[32648]: Adjusting osd_memory_target on smithi151 to 176.4M 2024-04-26T01:53:43.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:43 smithi165 ceph-mon[32648]: Unable to set osd_memory_target on smithi151 to 185064857: error parsing value: Value '185064857' is below minimum 939524096 2024-04-26T01:53:43.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:43 smithi165 ceph-mon[32648]: Updating smithi151:/etc/ceph/ceph.conf 2024-04-26T01:53:43.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:43 smithi165 ceph-mon[32648]: Updating smithi165:/etc/ceph/ceph.conf 2024-04-26T01:53:43.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:43 smithi165 ceph-mon[32648]: Updating smithi172:/etc/ceph/ceph.conf 2024-04-26T01:53:43.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:43 smithi165 ceph-mon[32648]: Updating smithi165:/var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:53:43.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:43 smithi165 ceph-mon[32648]: Updating smithi172:/var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:53:43.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:43 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:53:43.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:43 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:53:43.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:43 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:53:43.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:43 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:53:43.922 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-04-26T01:53:42.692481+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1187854,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1560,"num_read_kb":942,"num_write":473,"num_write_kb":4377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":866,"ondisk_log_size":866,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":389548,"kb_used_data":15296,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124552276,"statfs":{"total":1151940427776,"available":1151541530624,"internally_reserved":0,"allocated":15663104,"data_stored":9270026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.992231"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.159469+0000","last_change":"2024-04-26T01:49:37.641666+0000","last_active":"2024-04-26T01:53:37.159469+0000","last_peered":"2024-04-26T01:53:37.159469+0000","last_clean":"2024-04-26T01:53:37.159469+0000","last_became_active":"2024-04-26T01:49:37.641310+0000","last_became_peered":"2024-04-26T01:49:37.641310+0000","last_unstale":"2024-04-26T01:53:37.159469+0000","last_undegraded":"2024-04-26T01:53:37.159469+0000","last_fullsized":"2024-04-26T01:53:37.159469+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:28:02.952851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.161188+0000","last_change":"2024-04-26T01:49:37.637607+0000","last_active":"2024-04-26T01:53:37.161188+0000","last_peered":"2024-04-26T01:53:37.161188+0000","last_clean":"2024-04-26T01:53:37.161188+0000","last_became_active":"2024-04-26T01:49:37.637283+0000","last_became_peered":"2024-04-26T01:49:37.637283+0000","last_unstale":"2024-04-26T01:53:37.161188+0000","last_undegraded":"2024-04-26T01:53:37.161188+0000","last_fullsized":"2024-04-26T01:53:37.161188+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:16:46.519381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.161303+0000","last_change":"2024-04-26T01:49:37.647769+0000","last_active":"2024-04-26T01:53:37.161303+0000","last_peered":"2024-04-26T01:53:37.161303+0000","last_clean":"2024-04-26T01:53:37.161303+0000","last_became_active":"2024-04-26T01:49:37.643750+0000","last_became_peered":"2024-04-26T01:49:37.643750+0000","last_unstale":"2024-04-26T01:53:37.161303+0000","last_undegraded":"2024-04-26T01:53:37.161303+0000","last_fullsized":"2024-04-26T01:53:37.161303+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:34:18.801267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.669043+0000","last_change":"2024-04-26T01:49:37.643291+0000","last_active":"2024-04-26T01:53:36.669043+0000","last_peered":"2024-04-26T01:53:36.669043+0000","last_clean":"2024-04-26T01:53:36.669043+0000","last_became_active":"2024-04-26T01:49:37.642947+0000","last_became_peered":"2024-04-26T01:49:37.642947+0000","last_unstale":"2024-04-26T01:53:36.669043+0000","last_undegraded":"2024-04-26T01:53:36.669043+0000","last_fullsized":"2024-04-26T01:53:36.669043+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:31:21.600533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.789294+0000","last_change":"2024-04-26T01:49:37.665311+0000","last_active":"2024-04-26T01:53:36.789294+0000","last_peered":"2024-04-26T01:53:36.789294+0000","last_clean":"2024-04-26T01:53:36.789294+0000","last_became_active":"2024-04-26T01:49:37.664944+0000","last_became_peered":"2024-04-26T01:49:37.664944+0000","last_unstale":"2024-04-26T01:53:36.789294+0000","last_undegraded":"2024-04-26T01:53:36.789294+0000","last_fullsized":"2024-04-26T01:53:36.789294+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:49:50.744630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.799444+0000","last_change":"2024-04-26T01:49:37.638667+0000","last_active":"2024-04-26T01:53:36.799444+0000","last_peered":"2024-04-26T01:53:36.799444+0000","last_clean":"2024-04-26T01:53:36.799444+0000","last_became_active":"2024-04-26T01:49:37.638226+0000","last_became_peered":"2024-04-26T01:49:37.638226+0000","last_unstale":"2024-04-26T01:53:36.799444+0000","last_undegraded":"2024-04-26T01:53:36.799444+0000","last_fullsized":"2024-04-26T01:53:36.799444+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:46:17.786883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.985342+0000","last_change":"2024-04-26T01:49:37.650082+0000","last_active":"2024-04-26T01:53:36.985342+0000","last_peered":"2024-04-26T01:53:36.985342+0000","last_clean":"2024-04-26T01:53:36.985342+0000","last_became_active":"2024-04-26T01:49:37.649637+0000","last_became_peered":"2024-04-26T01:49:37.649637+0000","last_unstale":"2024-04-26T01:53:36.985342+0000","last_undegraded":"2024-04-26T01:53:36.985342+0000","last_fullsized":"2024-04-26T01:53:36.985342+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:21:04.686398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.789412+0000","last_change":"2024-04-26T01:49:37.661255+0000","last_active":"2024-04-26T01:53:36.789412+0000","last_peered":"2024-04-26T01:53:36.789412+0000","last_clean":"2024-04-26T01:53:36.789412+0000","last_became_active":"2024-04-26T01:49:37.658882+0000","last_became_peered":"2024-04-26T01:49:37.658882+0000","last_unstale":"2024-04-26T01:53:36.789412+0000","last_undegraded":"2024-04-26T01:53:36.789412+0000","last_fullsized":"2024-04-26T01:53:36.789412+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:36:27.874191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.982393+0000","last_change":"2024-04-26T01:49:37.648874+0000","last_active":"2024-04-26T01:53:36.982393+0000","last_peered":"2024-04-26T01:53:36.982393+0000","last_clean":"2024-04-26T01:53:36.982393+0000","last_became_active":"2024-04-26T01:49:37.648490+0000","last_became_peered":"2024-04-26T01:49:37.648490+0000","last_unstale":"2024-04-26T01:53:36.982393+0000","last_undegraded":"2024-04-26T01:53:36.982393+0000","last_fullsized":"2024-04-26T01:53:36.982393+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:46:12.552861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.799768+0000","last_change":"2024-04-26T01:49:37.643842+0000","last_active":"2024-04-26T01:53:36.799768+0000","last_peered":"2024-04-26T01:53:36.799768+0000","last_clean":"2024-04-26T01:53:36.799768+0000","last_became_active":"2024-04-26T01:49:37.643084+0000","last_became_peered":"2024-04-26T01:49:37.643084+0000","last_unstale":"2024-04-26T01:53:36.799768+0000","last_undegraded":"2024-04-26T01:53:36.799768+0000","last_fullsized":"2024-04-26T01:53:36.799768+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:16:53.102318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.985635+0000","last_change":"2024-04-26T01:49:37.644209+0000","last_active":"2024-04-26T01:53:36.985635+0000","last_peered":"2024-04-26T01:53:36.985635+0000","last_clean":"2024-04-26T01:53:36.985635+0000","last_became_active":"2024-04-26T01:49:37.643659+0000","last_became_peered":"2024-04-26T01:49:37.643659+0000","last_unstale":"2024-04-26T01:53:36.985635+0000","last_undegraded":"2024-04-26T01:53:36.985635+0000","last_fullsized":"2024-04-26T01:53:36.985635+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:24:46.363873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.665727+0000","last_change":"2024-04-26T01:50:27.001615+0000","last_active":"2024-04-26T01:53:36.665727+0000","last_peered":"2024-04-26T01:53:36.665727+0000","last_clean":"2024-04-26T01:53:36.665727+0000","last_became_active":"2024-04-26T01:50:27.001039+0000","last_became_peered":"2024-04-26T01:50:27.001039+0000","last_unstale":"2024-04-26T01:53:36.665727+0000","last_undegraded":"2024-04-26T01:53:36.665727+0000","last_fullsized":"2024-04-26T01:53:36.665727+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:42:46.148500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.162831+0000","last_change":"2024-04-26T01:49:37.651085+0000","last_active":"2024-04-26T01:53:37.162831+0000","last_peered":"2024-04-26T01:53:37.162831+0000","last_clean":"2024-04-26T01:53:37.162831+0000","last_became_active":"2024-04-26T01:49:37.649662+0000","last_became_peered":"2024-04-26T01:49:37.649662+0000","last_unstale":"2024-04-26T01:53:37.162831+0000","last_undegraded":"2024-04-26T01:53:37.162831+0000","last_fullsized":"2024-04-26T01:53:37.162831+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:41:34.537533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.160412+0000","last_change":"2024-04-26T01:49:37.648432+0000","last_active":"2024-04-26T01:53:37.160412+0000","last_peered":"2024-04-26T01:53:37.160412+0000","last_clean":"2024-04-26T01:53:37.160412+0000","last_became_active":"2024-04-26T01:49:37.648047+0000","last_became_peered":"2024-04-26T01:49:37.648047+0000","last_unstale":"2024-04-26T01:53:37.160412+0000","last_undegraded":"2024-04-26T01:53:37.160412+0000","last_fullsized":"2024-04-26T01:53:37.160412+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:00:06.843384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.160320+0000","last_change":"2024-04-26T01:49:37.644203+0000","last_active":"2024-04-26T01:53:37.160320+0000","last_peered":"2024-04-26T01:53:37.160320+0000","last_clean":"2024-04-26T01:53:37.160320+0000","last_became_active":"2024-04-26T01:49:37.643772+0000","last_became_peered":"2024-04-26T01:49:37.643772+0000","last_unstale":"2024-04-26T01:53:37.160320+0000","last_undegraded":"2024-04-26T01:53:37.160320+0000","last_fullsized":"2024-04-26T01:53:37.160320+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:52:20.707845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.981645+0000","last_change":"2024-04-26T01:49:37.647364+0000","last_active":"2024-04-26T01:53:36.981645+0000","last_peered":"2024-04-26T01:53:36.981645+0000","last_clean":"2024-04-26T01:53:36.981645+0000","last_became_active":"2024-04-26T01:49:37.647054+0000","last_became_peered":"2024-04-26T01:49:37.647054+0000","last_unstale":"2024-04-26T01:53:36.981645+0000","last_undegraded":"2024-04-26T01:53:36.981645+0000","last_fullsized":"2024-04-26T01:53:36.981645+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:53:09.631463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.159581+0000","last_change":"2024-04-26T01:49:37.641843+0000","last_active":"2024-04-26T01:53:37.159581+0000","last_peered":"2024-04-26T01:53:37.159581+0000","last_clean":"2024-04-26T01:53:37.159581+0000","last_became_active":"2024-04-26T01:49:37.641514+0000","last_became_peered":"2024-04-26T01:49:37.641514+0000","last_unstale":"2024-04-26T01:53:37.159581+0000","last_undegraded":"2024-04-26T01:53:37.159581+0000","last_fullsized":"2024-04-26T01:53:37.159581+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:24:32.261834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.665881+0000","last_change":"2024-04-26T01:49:37.644750+0000","last_active":"2024-04-26T01:53:36.665881+0000","last_peered":"2024-04-26T01:53:36.665881+0000","last_clean":"2024-04-26T01:53:36.665881+0000","last_became_active":"2024-04-26T01:49:37.644086+0000","last_became_peered":"2024-04-26T01:49:37.644086+0000","last_unstale":"2024-04-26T01:53:36.665881+0000","last_undegraded":"2024-04-26T01:53:36.665881+0000","last_fullsized":"2024-04-26T01:53:36.665881+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:59:42.614846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.982091+0000","last_change":"2024-04-26T01:49:37.664204+0000","last_active":"2024-04-26T01:53:36.982091+0000","last_peered":"2024-04-26T01:53:36.982091+0000","last_clean":"2024-04-26T01:53:36.982091+0000","last_became_active":"2024-04-26T01:49:37.663878+0000","last_became_peered":"2024-04-26T01:49:37.663878+0000","last_unstale":"2024-04-26T01:53:36.982091+0000","last_undegraded":"2024-04-26T01:53:36.982091+0000","last_fullsized":"2024-04-26T01:53:36.982091+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:43:49.272308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.665575+0000","last_change":"2024-04-26T01:49:33.592017+0000","last_active":"2024-04-26T01:53:36.665575+0000","last_peered":"2024-04-26T01:53:36.665575+0000","last_clean":"2024-04-26T01:53:36.665575+0000","last_became_active":"2024-04-26T01:49:33.587928+0000","last_became_peered":"2024-04-26T01:49:33.587928+0000","last_unstale":"2024-04-26T01:53:36.665575+0000","last_undegraded":"2024-04-26T01:53:36.665575+0000","last_fullsized":"2024-04-26T01:53:36.665575+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:02:04.313785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.791898+0000","last_change":"2024-04-26T01:49:37.662010+0000","last_active":"2024-04-26T01:53:36.791898+0000","last_peered":"2024-04-26T01:53:36.791898+0000","last_clean":"2024-04-26T01:53:36.791898+0000","last_became_active":"2024-04-26T01:49:37.661673+0000","last_became_peered":"2024-04-26T01:49:37.661673+0000","last_unstale":"2024-04-26T01:53:36.791898+0000","last_undegraded":"2024-04-26T01:53:36.791898+0000","last_fullsized":"2024-04-26T01:53:36.791898+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:37:34.155998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.792067+0000","last_change":"2024-04-26T01:49:34.185758+0000","last_active":"2024-04-26T01:53:36.792067+0000","last_peered":"2024-04-26T01:53:36.792067+0000","last_clean":"2024-04-26T01:53:36.792067+0000","last_became_active":"2024-04-26T01:49:34.184669+0000","last_became_peered":"2024-04-26T01:49:34.184669+0000","last_unstale":"2024-04-26T01:53:36.792067+0000","last_undegraded":"2024-04-26T01:53:36.792067+0000","last_fullsized":"2024-04-26T01:53:36.792067+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:23:23.742032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.161793+0000","last_change":"2024-04-26T01:49:37.641307+0000","last_active":"2024-04-26T01:53:37.161793+0000","last_peered":"2024-04-26T01:53:37.161793+0000","last_clean":"2024-04-26T01:53:37.161793+0000","last_became_active":"2024-04-26T01:49:37.640677+0000","last_became_peered":"2024-04-26T01:49:37.640677+0000","last_unstale":"2024-04-26T01:53:37.161793+0000","last_undegraded":"2024-04-26T01:53:37.161793+0000","last_fullsized":"2024-04-26T01:53:37.161793+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:35:56.859045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.159340+0000","last_change":"2024-04-26T01:50:27.007034+0000","last_active":"2024-04-26T01:53:37.159340+0000","last_peered":"2024-04-26T01:53:37.159340+0000","last_clean":"2024-04-26T01:53:37.159340+0000","last_became_active":"2024-04-26T01:50:27.000945+0000","last_became_peered":"2024-04-26T01:50:27.000945+0000","last_unstale":"2024-04-26T01:53:37.159340+0000","last_undegraded":"2024-04-26T01:53:37.159340+0000","last_fullsized":"2024-04-26T01:53:37.159340+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:39:02.484719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.981778+0000","last_change":"2024-04-26T01:49:37.645936+0000","last_active":"2024-04-26T01:53:36.981778+0000","last_peered":"2024-04-26T01:53:36.981778+0000","last_clean":"2024-04-26T01:53:36.981778+0000","last_became_active":"2024-04-26T01:49:37.645535+0000","last_became_peered":"2024-04-26T01:49:37.645535+0000","last_unstale":"2024-04-26T01:53:36.981778+0000","last_undegraded":"2024-04-26T01:53:36.981778+0000","last_fullsized":"2024-04-26T01:53:36.981778+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:45:04.434415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"90'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.789508+0000","last_change":"2024-04-26T01:49:33.589904+0000","last_active":"2024-04-26T01:53:36.789508+0000","last_peered":"2024-04-26T01:53:36.789508+0000","last_clean":"2024-04-26T01:53:36.789508+0000","last_became_active":"2024-04-26T01:49:33.589332+0000","last_became_peered":"2024-04-26T01:49:33.589332+0000","last_unstale":"2024-04-26T01:53:36.789508+0000","last_undegraded":"2024-04-26T01:53:36.789508+0000","last_fullsized":"2024-04-26T01:53:36.789508+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:28:07.972583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.666639+0000","last_change":"2024-04-26T01:49:37.652093+0000","last_active":"2024-04-26T01:53:36.666639+0000","last_peered":"2024-04-26T01:53:36.666639+0000","last_clean":"2024-04-26T01:53:36.666639+0000","last_became_active":"2024-04-26T01:49:37.651506+0000","last_became_peered":"2024-04-26T01:49:37.651506+0000","last_unstale":"2024-04-26T01:53:36.666639+0000","last_undegraded":"2024-04-26T01:53:36.666639+0000","last_fullsized":"2024-04-26T01:53:36.666639+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:01:31.421498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"93'9","reported_seq":260,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.114483+0000","last_change":"2024-04-26T01:49:33.587652+0000","last_active":"2024-04-26T01:53:37.114483+0000","last_peered":"2024-04-26T01:53:37.114483+0000","last_clean":"2024-04-26T01:53:37.114483+0000","last_became_active":"2024-04-26T01:49:33.587294+0000","last_became_peered":"2024-04-26T01:49:33.587294+0000","last_unstale":"2024-04-26T01:53:37.114483+0000","last_undegraded":"2024-04-26T01:53:37.114483+0000","last_fullsized":"2024-04-26T01:53:37.114483+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:32:46.803415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1855,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":222,"num_read_kb":6,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.789135+0000","last_change":"2024-04-26T01:49:37.647899+0000","last_active":"2024-04-26T01:53:36.789135+0000","last_peered":"2024-04-26T01:53:36.789135+0000","last_clean":"2024-04-26T01:53:36.789135+0000","last_became_active":"2024-04-26T01:49:37.647420+0000","last_became_peered":"2024-04-26T01:49:37.647420+0000","last_unstale":"2024-04-26T01:53:36.789135+0000","last_undegraded":"2024-04-26T01:53:36.789135+0000","last_fullsized":"2024-04-26T01:53:36.789135+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:16:39.262842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.799464+0000","last_change":"2024-04-26T01:49:33.596096+0000","last_active":"2024-04-26T01:53:36.799464+0000","last_peered":"2024-04-26T01:53:36.799464+0000","last_clean":"2024-04-26T01:53:36.799464+0000","last_became_active":"2024-04-26T01:49:33.595604+0000","last_became_peered":"2024-04-26T01:49:33.595604+0000","last_unstale":"2024-04-26T01:53:36.799464+0000","last_undegraded":"2024-04-26T01:53:36.799464+0000","last_fullsized":"2024-04-26T01:53:36.799464+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:44:26.497448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.666518+0000","last_change":"2024-04-26T01:49:37.665001+0000","last_active":"2024-04-26T01:53:36.666518+0000","last_peered":"2024-04-26T01:53:36.666518+0000","last_clean":"2024-04-26T01:53:36.666518+0000","last_became_active":"2024-04-26T01:49:37.664598+0000","last_became_peered":"2024-04-26T01:49:37.664598+0000","last_unstale":"2024-04-26T01:53:36.666518+0000","last_undegraded":"2024-04-26T01:53:36.666518+0000","last_fullsized":"2024-04-26T01:53:36.666518+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:54:28.299606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.162316+0000","last_change":"2024-04-26T01:49:33.602593+0000","last_active":"2024-04-26T01:53:37.162316+0000","last_peered":"2024-04-26T01:53:37.162316+0000","last_clean":"2024-04-26T01:53:37.162316+0000","last_became_active":"2024-04-26T01:49:33.602223+0000","last_became_peered":"2024-04-26T01:49:33.602223+0000","last_unstale":"2024-04-26T01:53:37.162316+0000","last_undegraded":"2024-04-26T01:53:37.162316+0000","last_fullsized":"2024-04-26T01:53:37.162316+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:43:29.404719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.789481+0000","last_change":"2024-04-26T01:49:37.662351+0000","last_active":"2024-04-26T01:53:36.789481+0000","last_peered":"2024-04-26T01:53:36.789481+0000","last_clean":"2024-04-26T01:53:36.789481+0000","last_became_active":"2024-04-26T01:49:37.661900+0000","last_became_peered":"2024-04-26T01:49:37.661900+0000","last_unstale":"2024-04-26T01:53:36.789481+0000","last_undegraded":"2024-04-26T01:53:36.789481+0000","last_fullsized":"2024-04-26T01:53:36.789481+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:49:10.878580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.666086+0000","last_change":"2024-04-26T01:49:33.586538+0000","last_active":"2024-04-26T01:53:36.666086+0000","last_peered":"2024-04-26T01:53:36.666086+0000","last_clean":"2024-04-26T01:53:36.666086+0000","last_became_active":"2024-04-26T01:49:33.586127+0000","last_became_peered":"2024-04-26T01:49:33.586127+0000","last_unstale":"2024-04-26T01:53:36.666086+0000","last_undegraded":"2024-04-26T01:53:36.666086+0000","last_fullsized":"2024-04-26T01:53:36.666086+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:05.599479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.665204+0000","last_change":"2024-04-26T01:49:37.657154+0000","last_active":"2024-04-26T01:53:36.665204+0000","last_peered":"2024-04-26T01:53:36.665204+0000","last_clean":"2024-04-26T01:53:36.665204+0000","last_became_active":"2024-04-26T01:49:37.656814+0000","last_became_peered":"2024-04-26T01:49:37.656814+0000","last_unstale":"2024-04-26T01:53:36.665204+0000","last_undegraded":"2024-04-26T01:53:36.665204+0000","last_fullsized":"2024-04-26T01:53:36.665204+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:26:54.538267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"92'5","reported_seq":250,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.799794+0000","last_change":"2024-04-26T01:49:33.602441+0000","last_active":"2024-04-26T01:53:36.799794+0000","last_peered":"2024-04-26T01:53:36.799794+0000","last_clean":"2024-04-26T01:53:36.799794+0000","last_became_active":"2024-04-26T01:49:33.602116+0000","last_became_peered":"2024-04-26T01:49:33.602116+0000","last_unstale":"2024-04-26T01:53:36.799794+0000","last_undegraded":"2024-04-26T01:53:36.799794+0000","last_fullsized":"2024-04-26T01:53:36.799794+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:45:52.541043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":216,"num_read_kb":216,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.982335+0000","last_change":"2024-04-26T01:49:37.645921+0000","last_active":"2024-04-26T01:53:36.982335+0000","last_peered":"2024-04-26T01:53:36.982335+0000","last_clean":"2024-04-26T01:53:36.982335+0000","last_became_active":"2024-04-26T01:49:37.645590+0000","last_became_peered":"2024-04-26T01:49:37.645590+0000","last_unstale":"2024-04-26T01:53:36.982335+0000","last_undegraded":"2024-04-26T01:53:36.982335+0000","last_fullsized":"2024-04-26T01:53:36.982335+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:00:08.931051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.985630+0000","last_change":"2024-04-26T01:49:33.589350+0000","last_active":"2024-04-26T01:53:36.985630+0000","last_peered":"2024-04-26T01:53:36.985630+0000","last_clean":"2024-04-26T01:53:36.985630+0000","last_became_active":"2024-04-26T01:49:33.588943+0000","last_became_peered":"2024-04-26T01:49:33.588943+0000","last_unstale":"2024-04-26T01:53:36.985630+0000","last_undegraded":"2024-04-26T01:53:36.985630+0000","last_fullsized":"2024-04-26T01:53:36.985630+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:36:36.104012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.161647+0000","last_change":"2024-04-26T01:49:37.648458+0000","last_active":"2024-04-26T01:53:37.161647+0000","last_peered":"2024-04-26T01:53:37.161647+0000","last_clean":"2024-04-26T01:53:37.161647+0000","last_became_active":"2024-04-26T01:49:37.648037+0000","last_became_peered":"2024-04-26T01:49:37.648037+0000","last_unstale":"2024-04-26T01:53:37.161647+0000","last_undegraded":"2024-04-26T01:53:37.161647+0000","last_fullsized":"2024-04-26T01:53:37.161647+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:28:05.711795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.666743+0000","last_change":"2024-04-26T01:49:34.189193+0000","last_active":"2024-04-26T01:53:36.666743+0000","last_peered":"2024-04-26T01:53:36.666743+0000","last_clean":"2024-04-26T01:53:36.666743+0000","last_became_active":"2024-04-26T01:49:34.188827+0000","last_became_peered":"2024-04-26T01:49:34.188827+0000","last_unstale":"2024-04-26T01:53:36.666743+0000","last_undegraded":"2024-04-26T01:53:36.666743+0000","last_fullsized":"2024-04-26T01:53:36.666743+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:02:04.583215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.981825+0000","last_change":"2024-04-26T01:49:37.644208+0000","last_active":"2024-04-26T01:53:36.981825+0000","last_peered":"2024-04-26T01:53:36.981825+0000","last_clean":"2024-04-26T01:53:36.981825+0000","last_became_active":"2024-04-26T01:49:37.643787+0000","last_became_peered":"2024-04-26T01:49:37.643787+0000","last_unstale":"2024-04-26T01:53:36.981825+0000","last_undegraded":"2024-04-26T01:53:36.981825+0000","last_fullsized":"2024-04-26T01:53:36.981825+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:42:21.243628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"93'35","reported_seq":508,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.114024+0000","last_change":"2024-04-26T01:49:33.590965+0000","last_active":"2024-04-26T01:53:37.114024+0000","last_peered":"2024-04-26T01:53:37.114024+0000","last_clean":"2024-04-26T01:53:37.114024+0000","last_became_active":"2024-04-26T01:49:33.590543+0000","last_became_peered":"2024-04-26T01:49:33.590543+0000","last_unstale":"2024-04-26T01:53:37.114024+0000","last_undegraded":"2024-04-26T01:53:37.114024+0000","last_fullsized":"2024-04-26T01:53:37.114024+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:26.627549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67061,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":444,"num_read_kb":288,"num_write":36,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.789460+0000","last_change":"2024-04-26T01:49:37.641172+0000","last_active":"2024-04-26T01:53:36.789460+0000","last_peered":"2024-04-26T01:53:36.789460+0000","last_clean":"2024-04-26T01:53:36.789460+0000","last_became_active":"2024-04-26T01:49:37.640866+0000","last_became_peered":"2024-04-26T01:49:37.640866+0000","last_unstale":"2024-04-26T01:53:36.789460+0000","last_undegraded":"2024-04-26T01:53:36.789460+0000","last_fullsized":"2024-04-26T01:53:36.789460+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:29:12.461005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.981907+0000","last_change":"2024-04-26T01:49:33.598466+0000","last_active":"2024-04-26T01:53:36.981907+0000","last_peered":"2024-04-26T01:53:36.981907+0000","last_clean":"2024-04-26T01:53:36.981907+0000","last_became_active":"2024-04-26T01:49:33.598042+0000","last_became_peered":"2024-04-26T01:49:33.598042+0000","last_unstale":"2024-04-26T01:53:36.981907+0000","last_undegraded":"2024-04-26T01:53:36.981907+0000","last_fullsized":"2024-04-26T01:53:36.981907+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:55:54.901016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.162529+0000","last_change":"2024-04-26T01:49:37.665772+0000","last_active":"2024-04-26T01:53:37.162529+0000","last_peered":"2024-04-26T01:53:37.162529+0000","last_clean":"2024-04-26T01:53:37.162529+0000","last_became_active":"2024-04-26T01:49:37.665524+0000","last_became_peered":"2024-04-26T01:49:37.665524+0000","last_unstale":"2024-04-26T01:53:37.162529+0000","last_undegraded":"2024-04-26T01:53:37.162529+0000","last_fullsized":"2024-04-26T01:53:37.162529+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:00:19.729098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.666223+0000","last_change":"2024-04-26T01:49:33.599456+0000","last_active":"2024-04-26T01:53:36.666223+0000","last_peered":"2024-04-26T01:53:36.666223+0000","last_clean":"2024-04-26T01:53:36.666223+0000","last_became_active":"2024-04-26T01:49:33.599101+0000","last_became_peered":"2024-04-26T01:49:33.599101+0000","last_unstale":"2024-04-26T01:53:36.666223+0000","last_undegraded":"2024-04-26T01:53:36.666223+0000","last_fullsized":"2024-04-26T01:53:36.666223+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:14:35.923011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.985701+0000","last_change":"2024-04-26T01:49:37.647193+0000","last_active":"2024-04-26T01:53:36.985701+0000","last_peered":"2024-04-26T01:53:36.985701+0000","last_clean":"2024-04-26T01:53:36.985701+0000","last_became_active":"2024-04-26T01:49:37.646881+0000","last_became_peered":"2024-04-26T01:49:37.646881+0000","last_unstale":"2024-04-26T01:53:36.985701+0000","last_undegraded":"2024-04-26T01:53:36.985701+0000","last_fullsized":"2024-04-26T01:53:36.985701+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:50:10.006986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"92'5","reported_seq":254,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.668634+0000","last_change":"2024-04-26T01:49:33.590398+0000","last_active":"2024-04-26T01:53:36.668634+0000","last_peered":"2024-04-26T01:53:36.668634+0000","last_clean":"2024-04-26T01:53:36.668634+0000","last_became_active":"2024-04-26T01:49:33.590033+0000","last_became_peered":"2024-04-26T01:49:33.590033+0000","last_unstale":"2024-04-26T01:53:36.668634+0000","last_undegraded":"2024-04-26T01:53:36.668634+0000","last_fullsized":"2024-04-26T01:53:36.668634+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:27:43.203781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":221,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.985282+0000","last_change":"2024-04-26T01:49:37.649784+0000","last_active":"2024-04-26T01:53:36.985282+0000","last_peered":"2024-04-26T01:53:36.985282+0000","last_clean":"2024-04-26T01:53:36.985282+0000","last_became_active":"2024-04-26T01:49:37.649431+0000","last_became_peered":"2024-04-26T01:49:37.649431+0000","last_unstale":"2024-04-26T01:53:36.985282+0000","last_undegraded":"2024-04-26T01:53:36.985282+0000","last_fullsized":"2024-04-26T01:53:36.985282+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:45:19.045983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.789114+0000","last_change":"2024-04-26T01:49:34.182987+0000","last_active":"2024-04-26T01:53:36.789114+0000","last_peered":"2024-04-26T01:53:36.789114+0000","last_clean":"2024-04-26T01:53:36.789114+0000","last_became_active":"2024-04-26T01:49:34.182600+0000","last_became_peered":"2024-04-26T01:49:34.182600+0000","last_unstale":"2024-04-26T01:53:36.789114+0000","last_undegraded":"2024-04-26T01:53:36.789114+0000","last_fullsized":"2024-04-26T01:53:36.789114+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:05:52.774170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.666131+0000","last_change":"2024-04-26T01:49:37.646097+0000","last_active":"2024-04-26T01:53:36.666131+0000","last_peered":"2024-04-26T01:53:36.666131+0000","last_clean":"2024-04-26T01:53:36.666131+0000","last_became_active":"2024-04-26T01:49:37.645763+0000","last_became_peered":"2024-04-26T01:49:37.645763+0000","last_unstale":"2024-04-26T01:53:36.666131+0000","last_undegraded":"2024-04-26T01:53:36.666131+0000","last_fullsized":"2024-04-26T01:53:36.666131+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:42:37.159950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.789434+0000","last_change":"2024-04-26T01:49:33.591457+0000","last_active":"2024-04-26T01:53:36.789434+0000","last_peered":"2024-04-26T01:53:36.789434+0000","last_clean":"2024-04-26T01:53:36.789434+0000","last_became_active":"2024-04-26T01:49:33.590655+0000","last_became_peered":"2024-04-26T01:49:33.590655+0000","last_unstale":"2024-04-26T01:53:36.789434+0000","last_undegraded":"2024-04-26T01:53:36.789434+0000","last_fullsized":"2024-04-26T01:53:36.789434+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:33:18.803796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.159392+0000","last_change":"2024-04-26T01:49:37.646423+0000","last_active":"2024-04-26T01:53:37.159392+0000","last_peered":"2024-04-26T01:53:37.159392+0000","last_clean":"2024-04-26T01:53:37.159392+0000","last_became_active":"2024-04-26T01:49:37.646046+0000","last_became_peered":"2024-04-26T01:49:37.646046+0000","last_unstale":"2024-04-26T01:53:37.159392+0000","last_undegraded":"2024-04-26T01:53:37.159392+0000","last_fullsized":"2024-04-26T01:53:37.159392+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:26:03.570479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.665726+0000","last_change":"2024-04-26T01:49:34.192848+0000","last_active":"2024-04-26T01:53:36.665726+0000","last_peered":"2024-04-26T01:53:36.665726+0000","last_clean":"2024-04-26T01:53:36.665726+0000","last_became_active":"2024-04-26T01:49:34.190449+0000","last_became_peered":"2024-04-26T01:49:34.190449+0000","last_unstale":"2024-04-26T01:53:36.665726+0000","last_undegraded":"2024-04-26T01:53:36.665726+0000","last_fullsized":"2024-04-26T01:53:36.665726+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:05:22.094991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'5","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.799444+0000","last_change":"2024-04-26T01:49:37.667147+0000","last_active":"2024-04-26T01:53:36.799444+0000","last_peered":"2024-04-26T01:53:36.799444+0000","last_clean":"2024-04-26T01:53:36.799444+0000","last_became_active":"2024-04-26T01:49:37.666752+0000","last_became_peered":"2024-04-26T01:49:37.666752+0000","last_unstale":"2024-04-26T01:53:36.799444+0000","last_undegraded":"2024-04-26T01:53:36.799444+0000","last_fullsized":"2024-04-26T01:53:36.799444+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:18:31.778916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.985450+0000","last_change":"2024-04-26T01:49:33.596155+0000","last_active":"2024-04-26T01:53:36.985450+0000","last_peered":"2024-04-26T01:53:36.985450+0000","last_clean":"2024-04-26T01:53:36.985450+0000","last_became_active":"2024-04-26T01:49:33.595760+0000","last_became_peered":"2024-04-26T01:49:33.595760+0000","last_unstale":"2024-04-26T01:53:36.985450+0000","last_undegraded":"2024-04-26T01:53:36.985450+0000","last_fullsized":"2024-04-26T01:53:36.985450+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:07:57.208116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.799446+0000","last_change":"2024-04-26T01:49:37.641660+0000","last_active":"2024-04-26T01:53:36.799446+0000","last_peered":"2024-04-26T01:53:36.799446+0000","last_clean":"2024-04-26T01:53:36.799446+0000","last_became_active":"2024-04-26T01:49:37.641302+0000","last_became_peered":"2024-04-26T01:49:37.641302+0000","last_unstale":"2024-04-26T01:53:36.799446+0000","last_undegraded":"2024-04-26T01:53:36.799446+0000","last_fullsized":"2024-04-26T01:53:36.799446+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:32:59.889189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.666400+0000","last_change":"2024-04-26T01:49:37.643083+0000","last_active":"2024-04-26T01:53:36.666400+0000","last_peered":"2024-04-26T01:53:36.666400+0000","last_clean":"2024-04-26T01:53:36.666400+0000","last_became_active":"2024-04-26T01:49:37.642548+0000","last_became_peered":"2024-04-26T01:49:37.642548+0000","last_unstale":"2024-04-26T01:53:36.666400+0000","last_undegraded":"2024-04-26T01:53:36.666400+0000","last_fullsized":"2024-04-26T01:53:36.666400+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:09:06.136510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'5","reported_seq":250,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.162195+0000","last_change":"2024-04-26T01:49:34.185069+0000","last_active":"2024-04-26T01:53:37.162195+0000","last_peered":"2024-04-26T01:53:37.162195+0000","last_clean":"2024-04-26T01:53:37.162195+0000","last_became_active":"2024-04-26T01:49:34.183989+0000","last_became_peered":"2024-04-26T01:49:34.183989+0000","last_unstale":"2024-04-26T01:53:37.162195+0000","last_undegraded":"2024-04-26T01:53:37.162195+0000","last_fullsized":"2024-04-26T01:53:37.162195+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:04:26.327960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":216,"num_read_kb":216,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.791363+0000","last_change":"2024-04-26T01:49:37.655512+0000","last_active":"2024-04-26T01:53:36.791363+0000","last_peered":"2024-04-26T01:53:36.791363+0000","last_clean":"2024-04-26T01:53:36.791363+0000","last_became_active":"2024-04-26T01:49:37.655142+0000","last_became_peered":"2024-04-26T01:49:37.655142+0000","last_unstale":"2024-04-26T01:53:36.791363+0000","last_undegraded":"2024-04-26T01:53:36.791363+0000","last_fullsized":"2024-04-26T01:53:36.791363+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:02:58.090885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.668788+0000","last_change":"2024-04-26T01:49:33.599578+0000","last_active":"2024-04-26T01:53:36.668788+0000","last_peered":"2024-04-26T01:53:36.668788+0000","last_clean":"2024-04-26T01:53:36.668788+0000","last_became_active":"2024-04-26T01:49:33.599010+0000","last_became_peered":"2024-04-26T01:49:33.599010+0000","last_unstale":"2024-04-26T01:53:36.668788+0000","last_undegraded":"2024-04-26T01:53:36.668788+0000","last_fullsized":"2024-04-26T01:53:36.668788+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:16:09.925304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":363,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.985395+0000","last_change":"2024-04-26T01:47:59.584819+0000","last_active":"2024-04-26T01:53:36.985395+0000","last_peered":"2024-04-26T01:53:36.985395+0000","last_clean":"2024-04-26T01:53:36.985395+0000","last_became_active":"2024-04-26T01:47:59.584356+0000","last_became_peered":"2024-04-26T01:47:59.584356+0000","last_unstale":"2024-04-26T01:53:36.985395+0000","last_undegraded":"2024-04-26T01:53:36.985395+0000","last_fullsized":"2024-04-26T01:53:36.985395+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:44:11.653568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:44:11.653568+0000","last_clean_scrub_stamp":"2024-04-26T01:44:11.653568+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:25:11.168583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.161855+0000","last_change":"2024-04-26T01:50:27.008911+0000","last_active":"2024-04-26T01:53:37.161855+0000","last_peered":"2024-04-26T01:53:37.161855+0000","last_clean":"2024-04-26T01:53:37.161855+0000","last_became_active":"2024-04-26T01:50:27.008507+0000","last_became_peered":"2024-04-26T01:50:27.008507+0000","last_unstale":"2024-04-26T01:53:37.161855+0000","last_undegraded":"2024-04-26T01:53:37.161855+0000","last_fullsized":"2024-04-26T01:53:37.161855+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:21:14.985935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.985710+0000","last_change":"2024-04-26T01:49:33.590171+0000","last_active":"2024-04-26T01:53:36.985710+0000","last_peered":"2024-04-26T01:53:36.985710+0000","last_clean":"2024-04-26T01:53:36.985710+0000","last_became_active":"2024-04-26T01:49:33.589312+0000","last_became_peered":"2024-04-26T01:49:33.589312+0000","last_unstale":"2024-04-26T01:53:36.985710+0000","last_undegraded":"2024-04-26T01:53:36.985710+0000","last_fullsized":"2024-04-26T01:53:36.985710+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:01:25.441527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.789485+0000","last_change":"2024-04-26T01:49:34.190338+0000","last_active":"2024-04-26T01:53:36.789485+0000","last_peered":"2024-04-26T01:53:36.789485+0000","last_clean":"2024-04-26T01:53:36.789485+0000","last_became_active":"2024-04-26T01:49:34.189984+0000","last_became_peered":"2024-04-26T01:49:34.189984+0000","last_unstale":"2024-04-26T01:53:36.789485+0000","last_undegraded":"2024-04-26T01:53:36.789485+0000","last_fullsized":"2024-04-26T01:53:36.789485+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:58.588012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.163165+0000","last_change":"2024-04-26T01:49:37.649368+0000","last_active":"2024-04-26T01:53:37.163165+0000","last_peered":"2024-04-26T01:53:37.163165+0000","last_clean":"2024-04-26T01:53:37.163165+0000","last_became_active":"2024-04-26T01:49:37.648987+0000","last_became_peered":"2024-04-26T01:49:37.648987+0000","last_unstale":"2024-04-26T01:53:37.163165+0000","last_undegraded":"2024-04-26T01:53:37.163165+0000","last_fullsized":"2024-04-26T01:53:37.163165+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:04:59.080703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"93'47","reported_seq":99,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.935485+0000","last_change":"2024-04-26T01:49:33.600813+0000","last_active":"2024-04-26T01:53:36.935485+0000","last_peered":"2024-04-26T01:53:36.935485+0000","last_clean":"2024-04-26T01:53:36.935485+0000","last_became_active":"2024-04-26T01:49:33.600361+0000","last_became_peered":"2024-04-26T01:49:33.600361+0000","last_unstale":"2024-04-26T01:53:36.935485+0000","last_undegraded":"2024-04-26T01:53:36.935485+0000","last_fullsized":"2024-04-26T01:53:36.935485+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:02:58.638449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":85,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.791866+0000","last_change":"2024-04-26T01:49:37.659381+0000","last_active":"2024-04-26T01:53:36.791866+0000","last_peered":"2024-04-26T01:53:36.791866+0000","last_clean":"2024-04-26T01:53:36.791866+0000","last_became_active":"2024-04-26T01:49:37.656205+0000","last_became_peered":"2024-04-26T01:49:37.656205+0000","last_unstale":"2024-04-26T01:53:36.791866+0000","last_undegraded":"2024-04-26T01:53:36.791866+0000","last_fullsized":"2024-04-26T01:53:36.791866+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:02:10.103330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.668898+0000","last_change":"2024-04-26T01:49:33.854732+0000","last_active":"2024-04-26T01:53:36.668898+0000","last_peered":"2024-04-26T01:53:36.668898+0000","last_clean":"2024-04-26T01:53:36.668898+0000","last_became_active":"2024-04-26T01:49:33.854396+0000","last_became_peered":"2024-04-26T01:49:33.854396+0000","last_unstale":"2024-04-26T01:53:36.668898+0000","last_undegraded":"2024-04-26T01:53:36.668898+0000","last_fullsized":"2024-04-26T01:53:36.668898+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:12:37.183725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.161320+0000","last_change":"2024-04-26T01:49:37.656366+0000","last_active":"2024-04-26T01:53:37.161320+0000","last_peered":"2024-04-26T01:53:37.161320+0000","last_clean":"2024-04-26T01:53:37.161320+0000","last_became_active":"2024-04-26T01:49:37.655869+0000","last_became_peered":"2024-04-26T01:49:37.655869+0000","last_unstale":"2024-04-26T01:53:37.161320+0000","last_undegraded":"2024-04-26T01:53:37.161320+0000","last_fullsized":"2024-04-26T01:53:37.161320+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:01:57.083315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.981654+0000","last_change":"2024-04-26T01:49:34.186070+0000","last_active":"2024-04-26T01:53:36.981654+0000","last_peered":"2024-04-26T01:53:36.981654+0000","last_clean":"2024-04-26T01:53:36.981654+0000","last_became_active":"2024-04-26T01:49:34.185742+0000","last_became_peered":"2024-04-26T01:49:34.185742+0000","last_unstale":"2024-04-26T01:53:36.981654+0000","last_undegraded":"2024-04-26T01:53:36.981654+0000","last_fullsized":"2024-04-26T01:53:36.981654+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:05.870056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.789114+0000","last_change":"2024-04-26T01:49:37.640938+0000","last_active":"2024-04-26T01:53:36.789114+0000","last_peered":"2024-04-26T01:53:36.789114+0000","last_clean":"2024-04-26T01:53:36.789114+0000","last_became_active":"2024-04-26T01:49:37.640608+0000","last_became_peered":"2024-04-26T01:49:37.640608+0000","last_unstale":"2024-04-26T01:53:36.789114+0000","last_undegraded":"2024-04-26T01:53:36.789114+0000","last_fullsized":"2024-04-26T01:53:36.789114+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:30:30.460641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.791217+0000","last_change":"2024-04-26T01:49:33.604152+0000","last_active":"2024-04-26T01:53:36.791217+0000","last_peered":"2024-04-26T01:53:36.791217+0000","last_clean":"2024-04-26T01:53:36.791217+0000","last_became_active":"2024-04-26T01:49:33.603479+0000","last_became_peered":"2024-04-26T01:49:33.603479+0000","last_unstale":"2024-04-26T01:53:36.791217+0000","last_undegraded":"2024-04-26T01:53:36.791217+0000","last_fullsized":"2024-04-26T01:53:36.791217+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:25:13.685392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.159096+0000","last_change":"2024-04-26T01:49:37.645152+0000","last_active":"2024-04-26T01:53:37.159096+0000","last_peered":"2024-04-26T01:53:37.159096+0000","last_clean":"2024-04-26T01:53:37.159096+0000","last_became_active":"2024-04-26T01:49:37.644789+0000","last_became_peered":"2024-04-26T01:49:37.644789+0000","last_unstale":"2024-04-26T01:53:37.159096+0000","last_undegraded":"2024-04-26T01:53:37.159096+0000","last_fullsized":"2024-04-26T01:53:37.159096+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:20:01.494222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.159155+0000","last_change":"2024-04-26T01:50:27.009973+0000","last_active":"2024-04-26T01:53:37.159155+0000","last_peered":"2024-04-26T01:53:37.159155+0000","last_clean":"2024-04-26T01:53:37.159155+0000","last_became_active":"2024-04-26T01:50:27.001678+0000","last_became_peered":"2024-04-26T01:50:27.001678+0000","last_unstale":"2024-04-26T01:53:37.159155+0000","last_undegraded":"2024-04-26T01:53:37.159155+0000","last_fullsized":"2024-04-26T01:53:37.159155+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:59:54.807792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.791338+0000","last_change":"2024-04-26T01:49:37.662665+0000","last_active":"2024-04-26T01:53:36.791338+0000","last_peered":"2024-04-26T01:53:36.791338+0000","last_clean":"2024-04-26T01:53:36.791338+0000","last_became_active":"2024-04-26T01:49:37.662216+0000","last_became_peered":"2024-04-26T01:49:37.662216+0000","last_unstale":"2024-04-26T01:53:36.791338+0000","last_undegraded":"2024-04-26T01:53:36.791338+0000","last_fullsized":"2024-04-26T01:53:36.791338+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:10:54.724858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.789242+0000","last_change":"2024-04-26T01:49:33.592078+0000","last_active":"2024-04-26T01:53:36.789242+0000","last_peered":"2024-04-26T01:53:36.789242+0000","last_clean":"2024-04-26T01:53:36.789242+0000","last_became_active":"2024-04-26T01:49:33.590265+0000","last_became_peered":"2024-04-26T01:49:33.590265+0000","last_unstale":"2024-04-26T01:53:36.789242+0000","last_undegraded":"2024-04-26T01:53:36.789242+0000","last_fullsized":"2024-04-26T01:53:36.789242+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:44:27.163705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.985525+0000","last_change":"2024-04-26T01:49:37.645943+0000","last_active":"2024-04-26T01:53:36.985525+0000","last_peered":"2024-04-26T01:53:36.985525+0000","last_clean":"2024-04-26T01:53:36.985525+0000","last_became_active":"2024-04-26T01:49:37.645586+0000","last_became_peered":"2024-04-26T01:49:37.645586+0000","last_unstale":"2024-04-26T01:53:36.985525+0000","last_undegraded":"2024-04-26T01:53:36.985525+0000","last_fullsized":"2024-04-26T01:53:36.985525+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:40.682344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.982081+0000","last_change":"2024-04-26T01:49:34.187115+0000","last_active":"2024-04-26T01:53:36.982081+0000","last_peered":"2024-04-26T01:53:36.982081+0000","last_clean":"2024-04-26T01:53:36.982081+0000","last_became_active":"2024-04-26T01:49:34.186660+0000","last_became_peered":"2024-04-26T01:49:34.186660+0000","last_unstale":"2024-04-26T01:53:36.982081+0000","last_undegraded":"2024-04-26T01:53:36.982081+0000","last_fullsized":"2024-04-26T01:53:36.982081+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:56:32.274752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.985756+0000","last_change":"2024-04-26T01:49:37.650382+0000","last_active":"2024-04-26T01:53:36.985756+0000","last_peered":"2024-04-26T01:53:36.985756+0000","last_clean":"2024-04-26T01:53:36.985756+0000","last_became_active":"2024-04-26T01:49:37.649545+0000","last_became_peered":"2024-04-26T01:49:37.649545+0000","last_unstale":"2024-04-26T01:53:36.985756+0000","last_undegraded":"2024-04-26T01:53:36.985756+0000","last_fullsized":"2024-04-26T01:53:36.985756+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:43:16.291311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"93'39","reported_seq":76,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.932472+0000","last_change":"2024-04-26T01:49:33.595881+0000","last_active":"2024-04-26T01:53:36.932472+0000","last_peered":"2024-04-26T01:53:36.932472+0000","last_clean":"2024-04-26T01:53:36.932472+0000","last_became_active":"2024-04-26T01:49:33.595480+0000","last_became_peered":"2024-04-26T01:49:33.595480+0000","last_unstale":"2024-04-26T01:53:36.932472+0000","last_undegraded":"2024-04-26T01:53:36.932472+0000","last_fullsized":"2024-04-26T01:53:36.932472+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:30:58.028230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.985506+0000","last_change":"2024-04-26T01:49:37.660765+0000","last_active":"2024-04-26T01:53:36.985506+0000","last_peered":"2024-04-26T01:53:36.985506+0000","last_clean":"2024-04-26T01:53:36.985506+0000","last_became_active":"2024-04-26T01:49:37.660475+0000","last_became_peered":"2024-04-26T01:49:37.660475+0000","last_unstale":"2024-04-26T01:53:36.985506+0000","last_undegraded":"2024-04-26T01:53:36.985506+0000","last_fullsized":"2024-04-26T01:53:36.985506+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:36:00.483005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"90'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.162985+0000","last_change":"2024-04-26T01:49:33.598247+0000","last_active":"2024-04-26T01:53:37.162985+0000","last_peered":"2024-04-26T01:53:37.162985+0000","last_clean":"2024-04-26T01:53:37.162985+0000","last_became_active":"2024-04-26T01:49:33.597796+0000","last_became_peered":"2024-04-26T01:49:33.597796+0000","last_unstale":"2024-04-26T01:53:37.162985+0000","last_undegraded":"2024-04-26T01:53:37.162985+0000","last_fullsized":"2024-04-26T01:53:37.162985+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:53:38.994798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.985835+0000","last_change":"2024-04-26T01:49:37.648903+0000","last_active":"2024-04-26T01:53:36.985835+0000","last_peered":"2024-04-26T01:53:36.985835+0000","last_clean":"2024-04-26T01:53:36.985835+0000","last_became_active":"2024-04-26T01:49:37.648570+0000","last_became_peered":"2024-04-26T01:49:37.648570+0000","last_unstale":"2024-04-26T01:53:36.985835+0000","last_undegraded":"2024-04-26T01:53:36.985835+0000","last_fullsized":"2024-04-26T01:53:36.985835+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:33:12.678513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.159153+0000","last_change":"2024-04-26T01:49:37.648485+0000","last_active":"2024-04-26T01:53:37.159153+0000","last_peered":"2024-04-26T01:53:37.159153+0000","last_clean":"2024-04-26T01:53:37.159153+0000","last_became_active":"2024-04-26T01:49:37.648129+0000","last_became_peered":"2024-04-26T01:49:37.648129+0000","last_unstale":"2024-04-26T01:53:37.159153+0000","last_undegraded":"2024-04-26T01:53:37.159153+0000","last_fullsized":"2024-04-26T01:53:37.159153+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:56:12.047821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.981838+0000","last_change":"2024-04-26T01:49:37.645929+0000","last_active":"2024-04-26T01:53:36.981838+0000","last_peered":"2024-04-26T01:53:36.981838+0000","last_clean":"2024-04-26T01:53:36.981838+0000","last_became_active":"2024-04-26T01:49:37.645473+0000","last_became_peered":"2024-04-26T01:49:37.645473+0000","last_unstale":"2024-04-26T01:53:36.981838+0000","last_undegraded":"2024-04-26T01:53:36.981838+0000","last_fullsized":"2024-04-26T01:53:36.981838+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:39:56.687973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.981790+0000","last_change":"2024-04-26T01:49:37.657494+0000","last_active":"2024-04-26T01:53:36.981790+0000","last_peered":"2024-04-26T01:53:36.981790+0000","last_clean":"2024-04-26T01:53:36.981790+0000","last_became_active":"2024-04-26T01:49:37.657121+0000","last_became_peered":"2024-04-26T01:49:37.657121+0000","last_unstale":"2024-04-26T01:53:36.981790+0000","last_undegraded":"2024-04-26T01:53:36.981790+0000","last_fullsized":"2024-04-26T01:53:36.981790+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:50:44.691179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.160724+0000","last_change":"2024-04-26T01:49:37.646012+0000","last_active":"2024-04-26T01:53:37.160724+0000","last_peered":"2024-04-26T01:53:37.160724+0000","last_clean":"2024-04-26T01:53:37.160724+0000","last_became_active":"2024-04-26T01:49:37.645695+0000","last_became_peered":"2024-04-26T01:49:37.645695+0000","last_unstale":"2024-04-26T01:53:37.160724+0000","last_undegraded":"2024-04-26T01:53:37.160724+0000","last_fullsized":"2024-04-26T01:53:37.160724+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:40:51.117523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.163306+0000","last_change":"2024-04-26T01:49:37.646503+0000","last_active":"2024-04-26T01:53:37.163306+0000","last_peered":"2024-04-26T01:53:37.163306+0000","last_clean":"2024-04-26T01:53:37.163306+0000","last_became_active":"2024-04-26T01:49:37.645200+0000","last_became_peered":"2024-04-26T01:49:37.645200+0000","last_unstale":"2024-04-26T01:53:37.163306+0000","last_undegraded":"2024-04-26T01:53:37.163306+0000","last_fullsized":"2024-04-26T01:53:37.163306+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:31:10.957199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.159402+0000","last_change":"2024-04-26T01:50:27.002908+0000","last_active":"2024-04-26T01:53:37.159402+0000","last_peered":"2024-04-26T01:53:37.159402+0000","last_clean":"2024-04-26T01:53:37.159402+0000","last_became_active":"2024-04-26T01:50:27.002454+0000","last_became_peered":"2024-04-26T01:50:27.002454+0000","last_unstale":"2024-04-26T01:53:37.159402+0000","last_undegraded":"2024-04-26T01:53:37.159402+0000","last_fullsized":"2024-04-26T01:53:37.159402+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:18:01.848427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.982171+0000","last_change":"2024-04-26T01:49:37.642676+0000","last_active":"2024-04-26T01:53:36.982171+0000","last_peered":"2024-04-26T01:53:36.982171+0000","last_clean":"2024-04-26T01:53:36.982171+0000","last_became_active":"2024-04-26T01:49:37.642302+0000","last_became_peered":"2024-04-26T01:49:37.642302+0000","last_unstale":"2024-04-26T01:53:36.982171+0000","last_undegraded":"2024-04-26T01:53:36.982171+0000","last_fullsized":"2024-04-26T01:53:36.982171+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:51:43.454649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.791673+0000","last_change":"2024-04-26T01:50:27.003914+0000","last_active":"2024-04-26T01:53:36.791673+0000","last_peered":"2024-04-26T01:53:36.791673+0000","last_clean":"2024-04-26T01:53:36.791673+0000","last_became_active":"2024-04-26T01:50:27.002344+0000","last_became_peered":"2024-04-26T01:50:27.002344+0000","last_unstale":"2024-04-26T01:53:36.791673+0000","last_undegraded":"2024-04-26T01:53:36.791673+0000","last_fullsized":"2024-04-26T01:53:36.791673+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:46:02.101508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.666138+0000","last_change":"2024-04-26T01:49:37.663306+0000","last_active":"2024-04-26T01:53:36.666138+0000","last_peered":"2024-04-26T01:53:36.666138+0000","last_clean":"2024-04-26T01:53:36.666138+0000","last_became_active":"2024-04-26T01:49:37.662999+0000","last_became_peered":"2024-04-26T01:49:37.662999+0000","last_unstale":"2024-04-26T01:53:36.666138+0000","last_undegraded":"2024-04-26T01:53:36.666138+0000","last_fullsized":"2024-04-26T01:53:36.666138+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:55.155096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.789468+0000","last_change":"2024-04-26T01:49:37.655157+0000","last_active":"2024-04-26T01:53:36.789468+0000","last_peered":"2024-04-26T01:53:36.789468+0000","last_clean":"2024-04-26T01:53:36.789468+0000","last_became_active":"2024-04-26T01:49:37.654810+0000","last_became_peered":"2024-04-26T01:49:37.654810+0000","last_unstale":"2024-04-26T01:53:36.789468+0000","last_undegraded":"2024-04-26T01:53:36.789468+0000","last_fullsized":"2024-04-26T01:53:36.789468+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:28:04.012382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.981829+0000","last_change":"2024-04-26T01:49:37.667228+0000","last_active":"2024-04-26T01:53:36.981829+0000","last_peered":"2024-04-26T01:53:36.981829+0000","last_clean":"2024-04-26T01:53:36.981829+0000","last_became_active":"2024-04-26T01:49:37.666835+0000","last_became_peered":"2024-04-26T01:49:37.666835+0000","last_unstale":"2024-04-26T01:53:36.981829+0000","last_undegraded":"2024-04-26T01:53:36.981829+0000","last_fullsized":"2024-04-26T01:53:36.981829+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:54:16.902836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.162824+0000","last_change":"2024-04-26T01:49:37.658231+0000","last_active":"2024-04-26T01:53:37.162824+0000","last_peered":"2024-04-26T01:53:37.162824+0000","last_clean":"2024-04-26T01:53:37.162824+0000","last_became_active":"2024-04-26T01:49:37.657842+0000","last_became_peered":"2024-04-26T01:49:37.657842+0000","last_unstale":"2024-04-26T01:53:37.162824+0000","last_undegraded":"2024-04-26T01:53:37.162824+0000","last_fullsized":"2024-04-26T01:53:37.162824+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:02:01.445262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.159180+0000","last_change":"2024-04-26T01:49:37.645221+0000","last_active":"2024-04-26T01:53:37.159180+0000","last_peered":"2024-04-26T01:53:37.159180+0000","last_clean":"2024-04-26T01:53:37.159180+0000","last_became_active":"2024-04-26T01:49:37.644814+0000","last_became_peered":"2024-04-26T01:49:37.644814+0000","last_unstale":"2024-04-26T01:53:37.159180+0000","last_undegraded":"2024-04-26T01:53:37.159180+0000","last_fullsized":"2024-04-26T01:53:37.159180+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:03:17.850633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":597252,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1424,"num_read_kb":823,"num_write":290,"num_write_kb":1437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2224128,"data_stored":1991299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318339,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":995328,"data_stored":486380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579973,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28824,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716328,"statfs":{"total":95995035648,"available":95965519872,"internally_reserved":0,"allocated":1851392,"data_stored":1285923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743499,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36728,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708424,"statfs":{"total":95995035648,"available":95957426176,"internally_reserved":0,"allocated":1421312,"data_stored":888701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005138,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":486536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266774,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28584,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716568,"statfs":{"total":95995035648,"available":95965765632,"internally_reserved":0,"allocated":1605632,"data_stored":1079449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430301,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":486005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574970,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36400,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708752,"statfs":{"total":95995035648,"available":95957762048,"internally_reserved":0,"allocated":1085440,"data_stored":564639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738494,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1036288,"data_stored":496145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411444,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36424,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708728,"statfs":{"total":95995035648,"available":95957737472,"internally_reserved":0,"allocated":1110016,"data_stored":564311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117102,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37172,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707980,"statfs":{"total":95995035648,"available":95956971520,"internally_reserved":0,"allocated":1757184,"data_stored":1216633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855465,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1019904,"data_stored":495843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691939,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28744,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716408,"statfs":{"total":95995035648,"available":95965601792,"internally_reserved":0,"allocated":1769472,"data_stored":1219461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":69644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":741376,"data_stored":721732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":134309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":393954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":200771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T01:53:43.927 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T01:53:44.384 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:53:44.534 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:44 smithi151 ceph-mon[28057]: Updating smithi165:/etc/ceph/ceph.client.admin.keyring 2024-04-26T01:53:44.534 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:44 smithi151 ceph-mon[28057]: Updating smithi172:/etc/ceph/ceph.client.admin.keyring 2024-04-26T01:53:44.534 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:44 smithi151 ceph-mon[28057]: Updating smithi151:/var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:53:44.534 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:44 smithi151 ceph-mon[28057]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:44.534 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:44 smithi151 ceph-mon[28057]: Updating smithi172:/var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T01:53:44.534 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:44 smithi151 ceph-mon[28057]: Updating smithi165:/var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T01:53:44.534 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:44 smithi151 ceph-mon[28057]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:53:44.534 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:44 smithi151 ceph-mon[28057]: Updating smithi151:/etc/ceph/ceph.client.admin.keyring 2024-04-26T01:53:44.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:44 smithi172 ceph-mon[32785]: Updating smithi165:/etc/ceph/ceph.client.admin.keyring 2024-04-26T01:53:44.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:44 smithi172 ceph-mon[32785]: Updating smithi172:/etc/ceph/ceph.client.admin.keyring 2024-04-26T01:53:44.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:44 smithi172 ceph-mon[32785]: Updating smithi151:/var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:53:44.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:44 smithi172 ceph-mon[32785]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:44.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:44 smithi172 ceph-mon[32785]: Updating smithi172:/var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T01:53:44.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:44 smithi172 ceph-mon[32785]: Updating smithi165:/var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T01:53:44.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:44 smithi172 ceph-mon[32785]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:53:44.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:44 smithi172 ceph-mon[32785]: Updating smithi151:/etc/ceph/ceph.client.admin.keyring 2024-04-26T01:53:44.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:44 smithi165 ceph-mon[32648]: Updating smithi165:/etc/ceph/ceph.client.admin.keyring 2024-04-26T01:53:44.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:44 smithi165 ceph-mon[32648]: Updating smithi172:/etc/ceph/ceph.client.admin.keyring 2024-04-26T01:53:44.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:44 smithi165 ceph-mon[32648]: Updating smithi151:/var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:53:44.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:44 smithi165 ceph-mon[32648]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:44.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:44 smithi165 ceph-mon[32648]: Updating smithi172:/var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T01:53:44.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:44 smithi165 ceph-mon[32648]: Updating smithi165:/var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T01:53:44.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:44 smithi165 ceph-mon[32648]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:53:44.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:44 smithi165 ceph-mon[32648]: Updating smithi151:/etc/ceph/ceph.client.admin.keyring 2024-04-26T01:53:45.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:45 smithi172 ceph-mon[32785]: Updating smithi151:/var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T01:53:45.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:45 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:53:45.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:45 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:53:45.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:45 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:53:45.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:45 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:53:45.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:45 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:53:45.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:45 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:53:45.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:45 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:53:45.832 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:45 smithi151 ceph-mon[28057]: Updating smithi151:/var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T01:53:45.832 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:45 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:53:45.832 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:45 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:53:45.832 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:45 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:53:45.832 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:45 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:53:45.832 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:45 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:53:45.833 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:45 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:53:45.833 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:45 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:53:45.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:45 smithi165 ceph-mon[32648]: Updating smithi151:/var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T01:53:45.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:45 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:53:45.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:45 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:53:45.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:45 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:53:45.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:45 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:53:45.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:45 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:53:45.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:45 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:53:45.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:45 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:53:46.660 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:46 smithi151 ceph-mon[28057]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 132 KiB/s wr, 77 op/s 2024-04-26T01:53:46.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:46.722+0000 7fda8c400700 1 -- 172.21.15.151:0/3996003629 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda84105470 msgr2=0x7fda84105900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:46.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:46.722+0000 7fda8c400700 1 --2- 172.21.15.151:0/3996003629 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda84105470 0x7fda84105900 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fda7c00b0a0 tx=0x7fda7c00b3b0 comp rx=0 tx=0).stop 2024-04-26T01:53:46.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:46.722+0000 7fda8c400700 1 -- 172.21.15.151:0/3996003629 shutdown_connections 2024-04-26T01:53:46.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:46.722+0000 7fda8c400700 1 --2- 172.21.15.151:0/3996003629 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda84105470 0x7fda84105900 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:46.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:46.722+0000 7fda8c400700 1 --2- 172.21.15.151:0/3996003629 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda84107960 0x7fda8406afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:46.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:46.722+0000 7fda8c400700 1 --2- 172.21.15.151:0/3996003629 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda8406b520 0x7fda8406b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:46.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:46.722+0000 7fda8c400700 1 -- 172.21.15.151:0/3996003629 >> 172.21.15.151:0/3996003629 conn(0x7fda840788b0 msgr2=0x7fda84078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:46.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:46.722+0000 7fda8c400700 1 -- 172.21.15.151:0/3996003629 shutdown_connections 2024-04-26T01:53:46.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:46.722+0000 7fda8c400700 1 -- 172.21.15.151:0/3996003629 wait complete. 2024-04-26T01:53:46.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:46.724+0000 7fda8c400700 1 Processor -- start 2024-04-26T01:53:46.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:46.724+0000 7fda8c400700 1 -- start start 2024-04-26T01:53:46.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:46.724+0000 7fda8c400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda8406b520 0x7fda8419f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:46.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:46.724+0000 7fda8c400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda84105470 0x7fda8419fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:46.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:46.724+0000 7fda8c400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda84107960 0x7fda841a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:46.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:46.724+0000 7fda8c400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fda84119220 con 0x7fda84107960 2024-04-26T01:53:46.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:46.724+0000 7fda8c400700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fda841190a0 con 0x7fda8406b520 2024-04-26T01:53:46.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:46.724+0000 7fda8c400700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fda841193a0 con 0x7fda84105470 2024-04-26T01:53:46.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:46.725+0000 7fda89a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda8406b520 0x7fda8419f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:46.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:46.725+0000 7fda8a400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda84107960 0x7fda841a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:46.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:46.725+0000 7fda89000700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda84105470 0x7fda8419fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:46.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:46.725+0000 7fda89a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda8406b520 0x7fda8419f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:33448/0 (socket says 172.21.15.151:33448) 2024-04-26T01:53:46.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:46.725+0000 7fda89a00700 1 -- 172.21.15.151:0/679143971 learned_addr learned my addr 172.21.15.151:0/679143971 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:53:46.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:46.725+0000 7fda89000700 1 -- 172.21.15.151:0/679143971 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda8406b520 msgr2=0x7fda8419f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:46.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:46.725+0000 7fda89000700 1 --2- 172.21.15.151:0/679143971 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda8406b520 0x7fda8419f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:46.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:46.725+0000 7fda89000700 1 -- 172.21.15.151:0/679143971 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda84107960 msgr2=0x7fda841a3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:46.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:46.725+0000 7fda89000700 1 --2- 172.21.15.151:0/679143971 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda84107960 0x7fda841a3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:46.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:46.725+0000 7fda89000700 1 -- 172.21.15.151:0/679143971 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda841a4710 con 0x7fda84105470 2024-04-26T01:53:46.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:46.725+0000 7fda89a00700 1 --2- 172.21.15.151:0/679143971 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda8406b520 0x7fda8419f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:53:46.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:46.726+0000 7fda89000700 1 --2- 172.21.15.151:0/679143971 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda84105470 0x7fda8419fc90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fda7000e9f0 tx=0x7fda7000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:46.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:46.726+0000 7fda82000700 1 -- 172.21.15.151:0/679143971 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda70004220 con 0x7fda84105470 2024-04-26T01:53:46.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:46.726+0000 7fda82000700 1 -- 172.21.15.151:0/679143971 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda700043c0 con 0x7fda84105470 2024-04-26T01:53:46.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:46.726+0000 7fda8c400700 1 -- 172.21.15.151:0/679143971 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda841a4a00 con 0x7fda84105470 2024-04-26T01:53:46.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:46.726+0000 7fda82000700 1 -- 172.21.15.151:0/679143971 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda70005020 con 0x7fda84105470 2024-04-26T01:53:46.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:46.726+0000 7fda8c400700 1 -- 172.21.15.151:0/679143971 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda841ac280 con 0x7fda84105470 2024-04-26T01:53:46.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:46.727+0000 7fda8c400700 1 -- 172.21.15.151:0/679143971 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda84069730 con 0x7fda84105470 2024-04-26T01:53:46.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:46.729+0000 7fda82000700 1 -- 172.21.15.151:0/679143971 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 25) ==== 138874+0+0 (secure 0 0 0) 0x7fda70004750 con 0x7fda84105470 2024-04-26T01:53:46.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:46.730+0000 7fda82000700 1 --2- 172.21.15.151:0/679143971 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fda6c0a3ed0 0x7fda6c0a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:46.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:46.731+0000 7fda82000700 1 -- 172.21.15.151:0/679143971 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fda700d0960 con 0x7fda84105470 2024-04-26T01:53:46.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:46.731+0000 7fda89a00700 1 --2- 172.21.15.151:0/679143971 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fda6c0a3ed0 0x7fda6c0a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:46.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:46.731+0000 7fda89a00700 1 --2- 172.21.15.151:0/679143971 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fda6c0a3ed0 0x7fda6c0a6390 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fda78005950 tx=0x7fda7800b500 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:46.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:46.735+0000 7fda82000700 1 -- 172.21.15.151:0/679143971 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fda7009ce90 con 0x7fda84105470 2024-04-26T01:53:46.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:46 smithi172 ceph-mon[32785]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 132 KiB/s wr, 77 op/s 2024-04-26T01:53:46.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:46 smithi165 ceph-mon[32648]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 132 KiB/s wr, 77 op/s 2024-04-26T01:53:46.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:46.956+0000 7fda8c400700 1 -- 172.21.15.151:0/679143971 --> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fda84104b60 con 0x7fda6c0a3ed0 2024-04-26T01:53:46.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:46.968+0000 7fda82000700 1 -- 172.21.15.151:0/679143971 <== mgr.25025 v2:172.21.15.151:6800/2427564083 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257768 (secure 0 0 0) 0x7fda84104b60 con 0x7fda6c0a3ed0 2024-04-26T01:53:46.972 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:53:46.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:46.974+0000 7fda8c400700 1 -- 172.21.15.151:0/679143971 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fda6c0a3ed0 msgr2=0x7fda6c0a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:46.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:46.974+0000 7fda8c400700 1 --2- 172.21.15.151:0/679143971 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fda6c0a3ed0 0x7fda6c0a6390 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fda78005950 tx=0x7fda7800b500 comp rx=0 tx=0).stop 2024-04-26T01:53:46.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:46.974+0000 7fda8c400700 1 -- 172.21.15.151:0/679143971 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda84105470 msgr2=0x7fda8419fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:46.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:46.974+0000 7fda8c400700 1 --2- 172.21.15.151:0/679143971 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda84105470 0x7fda8419fc90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fda7000e9f0 tx=0x7fda7000ed00 comp rx=0 tx=0).stop 2024-04-26T01:53:46.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:46.974+0000 7fda8c400700 1 -- 172.21.15.151:0/679143971 shutdown_connections 2024-04-26T01:53:46.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:46.974+0000 7fda8c400700 1 --2- 172.21.15.151:0/679143971 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fda6c0a3ed0 0x7fda6c0a6390 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:46.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:46.974+0000 7fda8c400700 1 --2- 172.21.15.151:0/679143971 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda84107960 0x7fda841a3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:46.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:46.974+0000 7fda8c400700 1 --2- 172.21.15.151:0/679143971 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda8406b520 0x7fda8419f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:46.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:46.974+0000 7fda8c400700 1 --2- 172.21.15.151:0/679143971 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda84105470 0x7fda8419fc90 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:46.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:46.974+0000 7fda8c400700 1 -- 172.21.15.151:0/679143971 >> 172.21.15.151:0/679143971 conn(0x7fda840788b0 msgr2=0x7fda84104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:46.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:46.975+0000 7fda8c400700 1 -- 172.21.15.151:0/679143971 shutdown_connections 2024-04-26T01:53:46.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:46.975+0000 7fda8c400700 1 -- 172.21.15.151:0/679143971 wait complete. 2024-04-26T01:53:46.976 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-04-26T01:53:47.747 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":8,"stamp":"2024-04-26T01:53:46.693940+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1387397,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1946,"num_read_kb":1402,"num_write":627,"num_write_kb":5298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":961,"ondisk_log_size":961,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":390624,"kb_used_data":16128,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124551200,"statfs":{"total":1151940427776,"available":1151540428800,"internally_reserved":0,"allocated":16515072,"data_stored":10098195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":199543,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":386,"num_read_kb":460,"num_write":154,"num_write_kb":921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.993690"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.159469+0000","last_change":"2024-04-26T01:49:37.641666+0000","last_active":"2024-04-26T01:53:37.159469+0000","last_peered":"2024-04-26T01:53:37.159469+0000","last_clean":"2024-04-26T01:53:37.159469+0000","last_became_active":"2024-04-26T01:49:37.641310+0000","last_became_peered":"2024-04-26T01:49:37.641310+0000","last_unstale":"2024-04-26T01:53:37.159469+0000","last_undegraded":"2024-04-26T01:53:37.159469+0000","last_fullsized":"2024-04-26T01:53:37.159469+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:28:02.952851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.161188+0000","last_change":"2024-04-26T01:49:37.637607+0000","last_active":"2024-04-26T01:53:37.161188+0000","last_peered":"2024-04-26T01:53:37.161188+0000","last_clean":"2024-04-26T01:53:37.161188+0000","last_became_active":"2024-04-26T01:49:37.637283+0000","last_became_peered":"2024-04-26T01:49:37.637283+0000","last_unstale":"2024-04-26T01:53:37.161188+0000","last_undegraded":"2024-04-26T01:53:37.161188+0000","last_fullsized":"2024-04-26T01:53:37.161188+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:16:46.519381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.161303+0000","last_change":"2024-04-26T01:49:37.647769+0000","last_active":"2024-04-26T01:53:37.161303+0000","last_peered":"2024-04-26T01:53:37.161303+0000","last_clean":"2024-04-26T01:53:37.161303+0000","last_became_active":"2024-04-26T01:49:37.643750+0000","last_became_peered":"2024-04-26T01:49:37.643750+0000","last_unstale":"2024-04-26T01:53:37.161303+0000","last_undegraded":"2024-04-26T01:53:37.161303+0000","last_fullsized":"2024-04-26T01:53:37.161303+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:34:18.801267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.669085+0000","last_change":"2024-04-26T01:49:37.643291+0000","last_active":"2024-04-26T01:53:36.669085+0000","last_peered":"2024-04-26T01:53:36.669085+0000","last_clean":"2024-04-26T01:53:36.669085+0000","last_became_active":"2024-04-26T01:49:37.642947+0000","last_became_peered":"2024-04-26T01:49:37.642947+0000","last_unstale":"2024-04-26T01:53:36.669085+0000","last_undegraded":"2024-04-26T01:53:36.669085+0000","last_fullsized":"2024-04-26T01:53:36.669085+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:31:21.600533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.789294+0000","last_change":"2024-04-26T01:49:37.665311+0000","last_active":"2024-04-26T01:53:36.789294+0000","last_peered":"2024-04-26T01:53:36.789294+0000","last_clean":"2024-04-26T01:53:36.789294+0000","last_became_active":"2024-04-26T01:49:37.664944+0000","last_became_peered":"2024-04-26T01:49:37.664944+0000","last_unstale":"2024-04-26T01:53:36.789294+0000","last_undegraded":"2024-04-26T01:53:36.789294+0000","last_fullsized":"2024-04-26T01:53:36.789294+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:49:50.744630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.799444+0000","last_change":"2024-04-26T01:49:37.638667+0000","last_active":"2024-04-26T01:53:36.799444+0000","last_peered":"2024-04-26T01:53:36.799444+0000","last_clean":"2024-04-26T01:53:36.799444+0000","last_became_active":"2024-04-26T01:49:37.638226+0000","last_became_peered":"2024-04-26T01:49:37.638226+0000","last_unstale":"2024-04-26T01:53:36.799444+0000","last_undegraded":"2024-04-26T01:53:36.799444+0000","last_fullsized":"2024-04-26T01:53:36.799444+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:46:17.786883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.985342+0000","last_change":"2024-04-26T01:49:37.650082+0000","last_active":"2024-04-26T01:53:36.985342+0000","last_peered":"2024-04-26T01:53:36.985342+0000","last_clean":"2024-04-26T01:53:36.985342+0000","last_became_active":"2024-04-26T01:49:37.649637+0000","last_became_peered":"2024-04-26T01:49:37.649637+0000","last_unstale":"2024-04-26T01:53:36.985342+0000","last_undegraded":"2024-04-26T01:53:36.985342+0000","last_fullsized":"2024-04-26T01:53:36.985342+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:21:04.686398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.789412+0000","last_change":"2024-04-26T01:49:37.661255+0000","last_active":"2024-04-26T01:53:36.789412+0000","last_peered":"2024-04-26T01:53:36.789412+0000","last_clean":"2024-04-26T01:53:36.789412+0000","last_became_active":"2024-04-26T01:49:37.658882+0000","last_became_peered":"2024-04-26T01:49:37.658882+0000","last_unstale":"2024-04-26T01:53:36.789412+0000","last_undegraded":"2024-04-26T01:53:36.789412+0000","last_fullsized":"2024-04-26T01:53:36.789412+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:36:27.874191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.982393+0000","last_change":"2024-04-26T01:49:37.648874+0000","last_active":"2024-04-26T01:53:36.982393+0000","last_peered":"2024-04-26T01:53:36.982393+0000","last_clean":"2024-04-26T01:53:36.982393+0000","last_became_active":"2024-04-26T01:49:37.648490+0000","last_became_peered":"2024-04-26T01:49:37.648490+0000","last_unstale":"2024-04-26T01:53:36.982393+0000","last_undegraded":"2024-04-26T01:53:36.982393+0000","last_fullsized":"2024-04-26T01:53:36.982393+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:46:12.552861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.799768+0000","last_change":"2024-04-26T01:49:37.643842+0000","last_active":"2024-04-26T01:53:36.799768+0000","last_peered":"2024-04-26T01:53:36.799768+0000","last_clean":"2024-04-26T01:53:36.799768+0000","last_became_active":"2024-04-26T01:49:37.643084+0000","last_became_peered":"2024-04-26T01:49:37.643084+0000","last_unstale":"2024-04-26T01:53:36.799768+0000","last_undegraded":"2024-04-26T01:53:36.799768+0000","last_fullsized":"2024-04-26T01:53:36.799768+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:16:53.102318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.985635+0000","last_change":"2024-04-26T01:49:37.644209+0000","last_active":"2024-04-26T01:53:36.985635+0000","last_peered":"2024-04-26T01:53:36.985635+0000","last_clean":"2024-04-26T01:53:36.985635+0000","last_became_active":"2024-04-26T01:49:37.643659+0000","last_became_peered":"2024-04-26T01:49:37.643659+0000","last_unstale":"2024-04-26T01:53:36.985635+0000","last_undegraded":"2024-04-26T01:53:36.985635+0000","last_fullsized":"2024-04-26T01:53:36.985635+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:24:46.363873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.665727+0000","last_change":"2024-04-26T01:50:27.001615+0000","last_active":"2024-04-26T01:53:36.665727+0000","last_peered":"2024-04-26T01:53:36.665727+0000","last_clean":"2024-04-26T01:53:36.665727+0000","last_became_active":"2024-04-26T01:50:27.001039+0000","last_became_peered":"2024-04-26T01:50:27.001039+0000","last_unstale":"2024-04-26T01:53:36.665727+0000","last_undegraded":"2024-04-26T01:53:36.665727+0000","last_fullsized":"2024-04-26T01:53:36.665727+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:42:46.148500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.162831+0000","last_change":"2024-04-26T01:49:37.651085+0000","last_active":"2024-04-26T01:53:37.162831+0000","last_peered":"2024-04-26T01:53:37.162831+0000","last_clean":"2024-04-26T01:53:37.162831+0000","last_became_active":"2024-04-26T01:49:37.649662+0000","last_became_peered":"2024-04-26T01:49:37.649662+0000","last_unstale":"2024-04-26T01:53:37.162831+0000","last_undegraded":"2024-04-26T01:53:37.162831+0000","last_fullsized":"2024-04-26T01:53:37.162831+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:41:34.537533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.160412+0000","last_change":"2024-04-26T01:49:37.648432+0000","last_active":"2024-04-26T01:53:37.160412+0000","last_peered":"2024-04-26T01:53:37.160412+0000","last_clean":"2024-04-26T01:53:37.160412+0000","last_became_active":"2024-04-26T01:49:37.648047+0000","last_became_peered":"2024-04-26T01:49:37.648047+0000","last_unstale":"2024-04-26T01:53:37.160412+0000","last_undegraded":"2024-04-26T01:53:37.160412+0000","last_fullsized":"2024-04-26T01:53:37.160412+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:00:06.843384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.160320+0000","last_change":"2024-04-26T01:49:37.644203+0000","last_active":"2024-04-26T01:53:37.160320+0000","last_peered":"2024-04-26T01:53:37.160320+0000","last_clean":"2024-04-26T01:53:37.160320+0000","last_became_active":"2024-04-26T01:49:37.643772+0000","last_became_peered":"2024-04-26T01:49:37.643772+0000","last_unstale":"2024-04-26T01:53:37.160320+0000","last_undegraded":"2024-04-26T01:53:37.160320+0000","last_fullsized":"2024-04-26T01:53:37.160320+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:52:20.707845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.981645+0000","last_change":"2024-04-26T01:49:37.647364+0000","last_active":"2024-04-26T01:53:36.981645+0000","last_peered":"2024-04-26T01:53:36.981645+0000","last_clean":"2024-04-26T01:53:36.981645+0000","last_became_active":"2024-04-26T01:49:37.647054+0000","last_became_peered":"2024-04-26T01:49:37.647054+0000","last_unstale":"2024-04-26T01:53:36.981645+0000","last_undegraded":"2024-04-26T01:53:36.981645+0000","last_fullsized":"2024-04-26T01:53:36.981645+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:53:09.631463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.159581+0000","last_change":"2024-04-26T01:49:37.641843+0000","last_active":"2024-04-26T01:53:37.159581+0000","last_peered":"2024-04-26T01:53:37.159581+0000","last_clean":"2024-04-26T01:53:37.159581+0000","last_became_active":"2024-04-26T01:49:37.641514+0000","last_became_peered":"2024-04-26T01:49:37.641514+0000","last_unstale":"2024-04-26T01:53:37.159581+0000","last_undegraded":"2024-04-26T01:53:37.159581+0000","last_fullsized":"2024-04-26T01:53:37.159581+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:24:32.261834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.665881+0000","last_change":"2024-04-26T01:49:37.644750+0000","last_active":"2024-04-26T01:53:36.665881+0000","last_peered":"2024-04-26T01:53:36.665881+0000","last_clean":"2024-04-26T01:53:36.665881+0000","last_became_active":"2024-04-26T01:49:37.644086+0000","last_became_peered":"2024-04-26T01:49:37.644086+0000","last_unstale":"2024-04-26T01:53:36.665881+0000","last_undegraded":"2024-04-26T01:53:36.665881+0000","last_fullsized":"2024-04-26T01:53:36.665881+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:59:42.614846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.982091+0000","last_change":"2024-04-26T01:49:37.664204+0000","last_active":"2024-04-26T01:53:36.982091+0000","last_peered":"2024-04-26T01:53:36.982091+0000","last_clean":"2024-04-26T01:53:36.982091+0000","last_became_active":"2024-04-26T01:49:37.663878+0000","last_became_peered":"2024-04-26T01:49:37.663878+0000","last_unstale":"2024-04-26T01:53:36.982091+0000","last_undegraded":"2024-04-26T01:53:36.982091+0000","last_fullsized":"2024-04-26T01:53:36.982091+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:43:49.272308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.665626+0000","last_change":"2024-04-26T01:49:33.592017+0000","last_active":"2024-04-26T01:53:36.665626+0000","last_peered":"2024-04-26T01:53:36.665626+0000","last_clean":"2024-04-26T01:53:36.665626+0000","last_became_active":"2024-04-26T01:49:33.587928+0000","last_became_peered":"2024-04-26T01:49:33.587928+0000","last_unstale":"2024-04-26T01:53:36.665626+0000","last_undegraded":"2024-04-26T01:53:36.665626+0000","last_fullsized":"2024-04-26T01:53:36.665626+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:02:04.313785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.791898+0000","last_change":"2024-04-26T01:49:37.662010+0000","last_active":"2024-04-26T01:53:36.791898+0000","last_peered":"2024-04-26T01:53:36.791898+0000","last_clean":"2024-04-26T01:53:36.791898+0000","last_became_active":"2024-04-26T01:49:37.661673+0000","last_became_peered":"2024-04-26T01:49:37.661673+0000","last_unstale":"2024-04-26T01:53:36.791898+0000","last_undegraded":"2024-04-26T01:53:36.791898+0000","last_fullsized":"2024-04-26T01:53:36.791898+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:37:34.155998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.792067+0000","last_change":"2024-04-26T01:49:34.185758+0000","last_active":"2024-04-26T01:53:36.792067+0000","last_peered":"2024-04-26T01:53:36.792067+0000","last_clean":"2024-04-26T01:53:36.792067+0000","last_became_active":"2024-04-26T01:49:34.184669+0000","last_became_peered":"2024-04-26T01:49:34.184669+0000","last_unstale":"2024-04-26T01:53:36.792067+0000","last_undegraded":"2024-04-26T01:53:36.792067+0000","last_fullsized":"2024-04-26T01:53:36.792067+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:23:23.742032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.161793+0000","last_change":"2024-04-26T01:49:37.641307+0000","last_active":"2024-04-26T01:53:37.161793+0000","last_peered":"2024-04-26T01:53:37.161793+0000","last_clean":"2024-04-26T01:53:37.161793+0000","last_became_active":"2024-04-26T01:49:37.640677+0000","last_became_peered":"2024-04-26T01:49:37.640677+0000","last_unstale":"2024-04-26T01:53:37.161793+0000","last_undegraded":"2024-04-26T01:53:37.161793+0000","last_fullsized":"2024-04-26T01:53:37.161793+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:35:56.859045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.159340+0000","last_change":"2024-04-26T01:50:27.007034+0000","last_active":"2024-04-26T01:53:37.159340+0000","last_peered":"2024-04-26T01:53:37.159340+0000","last_clean":"2024-04-26T01:53:37.159340+0000","last_became_active":"2024-04-26T01:50:27.000945+0000","last_became_peered":"2024-04-26T01:50:27.000945+0000","last_unstale":"2024-04-26T01:53:37.159340+0000","last_undegraded":"2024-04-26T01:53:37.159340+0000","last_fullsized":"2024-04-26T01:53:37.159340+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:39:02.484719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.981778+0000","last_change":"2024-04-26T01:49:37.645936+0000","last_active":"2024-04-26T01:53:36.981778+0000","last_peered":"2024-04-26T01:53:36.981778+0000","last_clean":"2024-04-26T01:53:36.981778+0000","last_became_active":"2024-04-26T01:49:37.645535+0000","last_became_peered":"2024-04-26T01:49:37.645535+0000","last_unstale":"2024-04-26T01:53:36.981778+0000","last_undegraded":"2024-04-26T01:53:36.981778+0000","last_fullsized":"2024-04-26T01:53:36.981778+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:45:04.434415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"90'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.789508+0000","last_change":"2024-04-26T01:49:33.589904+0000","last_active":"2024-04-26T01:53:36.789508+0000","last_peered":"2024-04-26T01:53:36.789508+0000","last_clean":"2024-04-26T01:53:36.789508+0000","last_became_active":"2024-04-26T01:49:33.589332+0000","last_became_peered":"2024-04-26T01:49:33.589332+0000","last_unstale":"2024-04-26T01:53:36.789508+0000","last_undegraded":"2024-04-26T01:53:36.789508+0000","last_fullsized":"2024-04-26T01:53:36.789508+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:28:07.972583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.666639+0000","last_change":"2024-04-26T01:49:37.652093+0000","last_active":"2024-04-26T01:53:36.666639+0000","last_peered":"2024-04-26T01:53:36.666639+0000","last_clean":"2024-04-26T01:53:36.666639+0000","last_became_active":"2024-04-26T01:49:37.651506+0000","last_became_peered":"2024-04-26T01:49:37.651506+0000","last_unstale":"2024-04-26T01:53:36.666639+0000","last_undegraded":"2024-04-26T01:53:36.666639+0000","last_fullsized":"2024-04-26T01:53:36.666639+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:01:31.421498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"93'9","reported_seq":265,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:41.120643+0000","last_change":"2024-04-26T01:49:33.587652+0000","last_active":"2024-04-26T01:53:41.120643+0000","last_peered":"2024-04-26T01:53:41.120643+0000","last_clean":"2024-04-26T01:53:41.120643+0000","last_became_active":"2024-04-26T01:49:33.587294+0000","last_became_peered":"2024-04-26T01:49:33.587294+0000","last_unstale":"2024-04-26T01:53:41.120643+0000","last_undegraded":"2024-04-26T01:53:41.120643+0000","last_fullsized":"2024-04-26T01:53:41.120643+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:32:46.803415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1855,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":227,"num_read_kb":6,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.789135+0000","last_change":"2024-04-26T01:49:37.647899+0000","last_active":"2024-04-26T01:53:36.789135+0000","last_peered":"2024-04-26T01:53:36.789135+0000","last_clean":"2024-04-26T01:53:36.789135+0000","last_became_active":"2024-04-26T01:49:37.647420+0000","last_became_peered":"2024-04-26T01:49:37.647420+0000","last_unstale":"2024-04-26T01:53:36.789135+0000","last_undegraded":"2024-04-26T01:53:36.789135+0000","last_fullsized":"2024-04-26T01:53:36.789135+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:16:39.262842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.799464+0000","last_change":"2024-04-26T01:49:33.596096+0000","last_active":"2024-04-26T01:53:36.799464+0000","last_peered":"2024-04-26T01:53:36.799464+0000","last_clean":"2024-04-26T01:53:36.799464+0000","last_became_active":"2024-04-26T01:49:33.595604+0000","last_became_peered":"2024-04-26T01:49:33.595604+0000","last_unstale":"2024-04-26T01:53:36.799464+0000","last_undegraded":"2024-04-26T01:53:36.799464+0000","last_fullsized":"2024-04-26T01:53:36.799464+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:44:26.497448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.666518+0000","last_change":"2024-04-26T01:49:37.665001+0000","last_active":"2024-04-26T01:53:36.666518+0000","last_peered":"2024-04-26T01:53:36.666518+0000","last_clean":"2024-04-26T01:53:36.666518+0000","last_became_active":"2024-04-26T01:49:37.664598+0000","last_became_peered":"2024-04-26T01:49:37.664598+0000","last_unstale":"2024-04-26T01:53:36.666518+0000","last_undegraded":"2024-04-26T01:53:36.666518+0000","last_fullsized":"2024-04-26T01:53:36.666518+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:54:28.299606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.162316+0000","last_change":"2024-04-26T01:49:33.602593+0000","last_active":"2024-04-26T01:53:37.162316+0000","last_peered":"2024-04-26T01:53:37.162316+0000","last_clean":"2024-04-26T01:53:37.162316+0000","last_became_active":"2024-04-26T01:49:33.602223+0000","last_became_peered":"2024-04-26T01:49:33.602223+0000","last_unstale":"2024-04-26T01:53:37.162316+0000","last_undegraded":"2024-04-26T01:53:37.162316+0000","last_fullsized":"2024-04-26T01:53:37.162316+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:43:29.404719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.789481+0000","last_change":"2024-04-26T01:49:37.662351+0000","last_active":"2024-04-26T01:53:36.789481+0000","last_peered":"2024-04-26T01:53:36.789481+0000","last_clean":"2024-04-26T01:53:36.789481+0000","last_became_active":"2024-04-26T01:49:37.661900+0000","last_became_peered":"2024-04-26T01:49:37.661900+0000","last_unstale":"2024-04-26T01:53:36.789481+0000","last_undegraded":"2024-04-26T01:53:36.789481+0000","last_fullsized":"2024-04-26T01:53:36.789481+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:49:10.878580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.666086+0000","last_change":"2024-04-26T01:49:33.586538+0000","last_active":"2024-04-26T01:53:36.666086+0000","last_peered":"2024-04-26T01:53:36.666086+0000","last_clean":"2024-04-26T01:53:36.666086+0000","last_became_active":"2024-04-26T01:49:33.586127+0000","last_became_peered":"2024-04-26T01:49:33.586127+0000","last_unstale":"2024-04-26T01:53:36.666086+0000","last_undegraded":"2024-04-26T01:53:36.666086+0000","last_fullsized":"2024-04-26T01:53:36.666086+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:05.599479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.665204+0000","last_change":"2024-04-26T01:49:37.657154+0000","last_active":"2024-04-26T01:53:36.665204+0000","last_peered":"2024-04-26T01:53:36.665204+0000","last_clean":"2024-04-26T01:53:36.665204+0000","last_became_active":"2024-04-26T01:49:37.656814+0000","last_became_peered":"2024-04-26T01:49:37.656814+0000","last_unstale":"2024-04-26T01:53:36.665204+0000","last_undegraded":"2024-04-26T01:53:36.665204+0000","last_fullsized":"2024-04-26T01:53:36.665204+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:26:54.538267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"92'5","reported_seq":255,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:41.119940+0000","last_change":"2024-04-26T01:49:33.602441+0000","last_active":"2024-04-26T01:53:41.119940+0000","last_peered":"2024-04-26T01:53:41.119940+0000","last_clean":"2024-04-26T01:53:41.119940+0000","last_became_active":"2024-04-26T01:49:33.602116+0000","last_became_peered":"2024-04-26T01:49:33.602116+0000","last_unstale":"2024-04-26T01:53:41.119940+0000","last_undegraded":"2024-04-26T01:53:41.119940+0000","last_fullsized":"2024-04-26T01:53:41.119940+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:45:52.541043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":221,"num_read_kb":221,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.982335+0000","last_change":"2024-04-26T01:49:37.645921+0000","last_active":"2024-04-26T01:53:36.982335+0000","last_peered":"2024-04-26T01:53:36.982335+0000","last_clean":"2024-04-26T01:53:36.982335+0000","last_became_active":"2024-04-26T01:49:37.645590+0000","last_became_peered":"2024-04-26T01:49:37.645590+0000","last_unstale":"2024-04-26T01:53:36.982335+0000","last_undegraded":"2024-04-26T01:53:36.982335+0000","last_fullsized":"2024-04-26T01:53:36.982335+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:00:08.931051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.985630+0000","last_change":"2024-04-26T01:49:33.589350+0000","last_active":"2024-04-26T01:53:36.985630+0000","last_peered":"2024-04-26T01:53:36.985630+0000","last_clean":"2024-04-26T01:53:36.985630+0000","last_became_active":"2024-04-26T01:49:33.588943+0000","last_became_peered":"2024-04-26T01:49:33.588943+0000","last_unstale":"2024-04-26T01:53:36.985630+0000","last_undegraded":"2024-04-26T01:53:36.985630+0000","last_fullsized":"2024-04-26T01:53:36.985630+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:36:36.104012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.161647+0000","last_change":"2024-04-26T01:49:37.648458+0000","last_active":"2024-04-26T01:53:37.161647+0000","last_peered":"2024-04-26T01:53:37.161647+0000","last_clean":"2024-04-26T01:53:37.161647+0000","last_became_active":"2024-04-26T01:49:37.648037+0000","last_became_peered":"2024-04-26T01:49:37.648037+0000","last_unstale":"2024-04-26T01:53:37.161647+0000","last_undegraded":"2024-04-26T01:53:37.161647+0000","last_fullsized":"2024-04-26T01:53:37.161647+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:28:05.711795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.666743+0000","last_change":"2024-04-26T01:49:34.189193+0000","last_active":"2024-04-26T01:53:36.666743+0000","last_peered":"2024-04-26T01:53:36.666743+0000","last_clean":"2024-04-26T01:53:36.666743+0000","last_became_active":"2024-04-26T01:49:34.188827+0000","last_became_peered":"2024-04-26T01:49:34.188827+0000","last_unstale":"2024-04-26T01:53:36.666743+0000","last_undegraded":"2024-04-26T01:53:36.666743+0000","last_fullsized":"2024-04-26T01:53:36.666743+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:02:04.583215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.981825+0000","last_change":"2024-04-26T01:49:37.644208+0000","last_active":"2024-04-26T01:53:36.981825+0000","last_peered":"2024-04-26T01:53:36.981825+0000","last_clean":"2024-04-26T01:53:36.981825+0000","last_became_active":"2024-04-26T01:49:37.643787+0000","last_became_peered":"2024-04-26T01:49:37.643787+0000","last_unstale":"2024-04-26T01:53:36.981825+0000","last_undegraded":"2024-04-26T01:53:36.981825+0000","last_fullsized":"2024-04-26T01:53:36.981825+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:42:21.243628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"93'37","reported_seq":522,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:42.121432+0000","last_change":"2024-04-26T01:49:33.590965+0000","last_active":"2024-04-26T01:53:42.121432+0000","last_peered":"2024-04-26T01:53:42.121432+0000","last_clean":"2024-04-26T01:53:42.121432+0000","last_became_active":"2024-04-26T01:49:33.590543+0000","last_became_peered":"2024-04-26T01:49:33.590543+0000","last_unstale":"2024-04-26T01:53:42.121432+0000","last_undegraded":"2024-04-26T01:53:42.121432+0000","last_fullsized":"2024-04-26T01:53:42.121432+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:26.627549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69972,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":456,"num_read_kb":297,"num_write":38,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.789460+0000","last_change":"2024-04-26T01:49:37.641172+0000","last_active":"2024-04-26T01:53:36.789460+0000","last_peered":"2024-04-26T01:53:36.789460+0000","last_clean":"2024-04-26T01:53:36.789460+0000","last_became_active":"2024-04-26T01:49:37.640866+0000","last_became_peered":"2024-04-26T01:49:37.640866+0000","last_unstale":"2024-04-26T01:53:36.789460+0000","last_undegraded":"2024-04-26T01:53:36.789460+0000","last_fullsized":"2024-04-26T01:53:36.789460+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:29:12.461005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.981907+0000","last_change":"2024-04-26T01:49:33.598466+0000","last_active":"2024-04-26T01:53:36.981907+0000","last_peered":"2024-04-26T01:53:36.981907+0000","last_clean":"2024-04-26T01:53:36.981907+0000","last_became_active":"2024-04-26T01:49:33.598042+0000","last_became_peered":"2024-04-26T01:49:33.598042+0000","last_unstale":"2024-04-26T01:53:36.981907+0000","last_undegraded":"2024-04-26T01:53:36.981907+0000","last_fullsized":"2024-04-26T01:53:36.981907+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:55:54.901016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.162529+0000","last_change":"2024-04-26T01:49:37.665772+0000","last_active":"2024-04-26T01:53:37.162529+0000","last_peered":"2024-04-26T01:53:37.162529+0000","last_clean":"2024-04-26T01:53:37.162529+0000","last_became_active":"2024-04-26T01:49:37.665524+0000","last_became_peered":"2024-04-26T01:49:37.665524+0000","last_unstale":"2024-04-26T01:53:37.162529+0000","last_undegraded":"2024-04-26T01:53:37.162529+0000","last_fullsized":"2024-04-26T01:53:37.162529+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:00:19.729098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.666223+0000","last_change":"2024-04-26T01:49:33.599456+0000","last_active":"2024-04-26T01:53:36.666223+0000","last_peered":"2024-04-26T01:53:36.666223+0000","last_clean":"2024-04-26T01:53:36.666223+0000","last_became_active":"2024-04-26T01:49:33.599101+0000","last_became_peered":"2024-04-26T01:49:33.599101+0000","last_unstale":"2024-04-26T01:53:36.666223+0000","last_undegraded":"2024-04-26T01:53:36.666223+0000","last_fullsized":"2024-04-26T01:53:36.666223+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:14:35.923011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.985701+0000","last_change":"2024-04-26T01:49:37.647193+0000","last_active":"2024-04-26T01:53:36.985701+0000","last_peered":"2024-04-26T01:53:36.985701+0000","last_clean":"2024-04-26T01:53:36.985701+0000","last_became_active":"2024-04-26T01:49:37.646881+0000","last_became_peered":"2024-04-26T01:49:37.646881+0000","last_unstale":"2024-04-26T01:53:36.985701+0000","last_undegraded":"2024-04-26T01:53:36.985701+0000","last_fullsized":"2024-04-26T01:53:36.985701+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:50:10.006986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"92'5","reported_seq":259,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:41.261442+0000","last_change":"2024-04-26T01:49:33.590398+0000","last_active":"2024-04-26T01:53:41.261442+0000","last_peered":"2024-04-26T01:53:41.261442+0000","last_clean":"2024-04-26T01:53:41.261442+0000","last_became_active":"2024-04-26T01:49:33.590033+0000","last_became_peered":"2024-04-26T01:49:33.590033+0000","last_unstale":"2024-04-26T01:53:41.261442+0000","last_undegraded":"2024-04-26T01:53:41.261442+0000","last_fullsized":"2024-04-26T01:53:41.261442+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:27:43.203781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":226,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.985282+0000","last_change":"2024-04-26T01:49:37.649784+0000","last_active":"2024-04-26T01:53:36.985282+0000","last_peered":"2024-04-26T01:53:36.985282+0000","last_clean":"2024-04-26T01:53:36.985282+0000","last_became_active":"2024-04-26T01:49:37.649431+0000","last_became_peered":"2024-04-26T01:49:37.649431+0000","last_unstale":"2024-04-26T01:53:36.985282+0000","last_undegraded":"2024-04-26T01:53:36.985282+0000","last_fullsized":"2024-04-26T01:53:36.985282+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:45:19.045983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.789114+0000","last_change":"2024-04-26T01:49:34.182987+0000","last_active":"2024-04-26T01:53:36.789114+0000","last_peered":"2024-04-26T01:53:36.789114+0000","last_clean":"2024-04-26T01:53:36.789114+0000","last_became_active":"2024-04-26T01:49:34.182600+0000","last_became_peered":"2024-04-26T01:49:34.182600+0000","last_unstale":"2024-04-26T01:53:36.789114+0000","last_undegraded":"2024-04-26T01:53:36.789114+0000","last_fullsized":"2024-04-26T01:53:36.789114+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:05:52.774170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.666131+0000","last_change":"2024-04-26T01:49:37.646097+0000","last_active":"2024-04-26T01:53:36.666131+0000","last_peered":"2024-04-26T01:53:36.666131+0000","last_clean":"2024-04-26T01:53:36.666131+0000","last_became_active":"2024-04-26T01:49:37.645763+0000","last_became_peered":"2024-04-26T01:49:37.645763+0000","last_unstale":"2024-04-26T01:53:36.666131+0000","last_undegraded":"2024-04-26T01:53:36.666131+0000","last_fullsized":"2024-04-26T01:53:36.666131+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:42:37.159950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.789434+0000","last_change":"2024-04-26T01:49:33.591457+0000","last_active":"2024-04-26T01:53:36.789434+0000","last_peered":"2024-04-26T01:53:36.789434+0000","last_clean":"2024-04-26T01:53:36.789434+0000","last_became_active":"2024-04-26T01:49:33.590655+0000","last_became_peered":"2024-04-26T01:49:33.590655+0000","last_unstale":"2024-04-26T01:53:36.789434+0000","last_undegraded":"2024-04-26T01:53:36.789434+0000","last_fullsized":"2024-04-26T01:53:36.789434+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:33:18.803796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.159392+0000","last_change":"2024-04-26T01:49:37.646423+0000","last_active":"2024-04-26T01:53:37.159392+0000","last_peered":"2024-04-26T01:53:37.159392+0000","last_clean":"2024-04-26T01:53:37.159392+0000","last_became_active":"2024-04-26T01:49:37.646046+0000","last_became_peered":"2024-04-26T01:49:37.646046+0000","last_unstale":"2024-04-26T01:53:37.159392+0000","last_undegraded":"2024-04-26T01:53:37.159392+0000","last_fullsized":"2024-04-26T01:53:37.159392+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:26:03.570479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.665726+0000","last_change":"2024-04-26T01:49:34.192848+0000","last_active":"2024-04-26T01:53:36.665726+0000","last_peered":"2024-04-26T01:53:36.665726+0000","last_clean":"2024-04-26T01:53:36.665726+0000","last_became_active":"2024-04-26T01:49:34.190449+0000","last_became_peered":"2024-04-26T01:49:34.190449+0000","last_unstale":"2024-04-26T01:53:36.665726+0000","last_undegraded":"2024-04-26T01:53:36.665726+0000","last_fullsized":"2024-04-26T01:53:36.665726+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:05:22.094991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'5","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.799444+0000","last_change":"2024-04-26T01:49:37.667147+0000","last_active":"2024-04-26T01:53:36.799444+0000","last_peered":"2024-04-26T01:53:36.799444+0000","last_clean":"2024-04-26T01:53:36.799444+0000","last_became_active":"2024-04-26T01:49:37.666752+0000","last_became_peered":"2024-04-26T01:49:37.666752+0000","last_unstale":"2024-04-26T01:53:36.799444+0000","last_undegraded":"2024-04-26T01:53:36.799444+0000","last_fullsized":"2024-04-26T01:53:36.799444+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:18:31.778916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.985505+0000","last_change":"2024-04-26T01:49:33.596155+0000","last_active":"2024-04-26T01:53:36.985505+0000","last_peered":"2024-04-26T01:53:36.985505+0000","last_clean":"2024-04-26T01:53:36.985505+0000","last_became_active":"2024-04-26T01:49:33.595760+0000","last_became_peered":"2024-04-26T01:49:33.595760+0000","last_unstale":"2024-04-26T01:53:36.985505+0000","last_undegraded":"2024-04-26T01:53:36.985505+0000","last_fullsized":"2024-04-26T01:53:36.985505+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:07:57.208116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.799446+0000","last_change":"2024-04-26T01:49:37.641660+0000","last_active":"2024-04-26T01:53:36.799446+0000","last_peered":"2024-04-26T01:53:36.799446+0000","last_clean":"2024-04-26T01:53:36.799446+0000","last_became_active":"2024-04-26T01:49:37.641302+0000","last_became_peered":"2024-04-26T01:49:37.641302+0000","last_unstale":"2024-04-26T01:53:36.799446+0000","last_undegraded":"2024-04-26T01:53:36.799446+0000","last_fullsized":"2024-04-26T01:53:36.799446+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:32:59.889189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.666400+0000","last_change":"2024-04-26T01:49:37.643083+0000","last_active":"2024-04-26T01:53:36.666400+0000","last_peered":"2024-04-26T01:53:36.666400+0000","last_clean":"2024-04-26T01:53:36.666400+0000","last_became_active":"2024-04-26T01:49:37.642548+0000","last_became_peered":"2024-04-26T01:49:37.642548+0000","last_unstale":"2024-04-26T01:53:36.666400+0000","last_undegraded":"2024-04-26T01:53:36.666400+0000","last_fullsized":"2024-04-26T01:53:36.666400+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:09:06.136510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'5","reported_seq":256,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:42.262655+0000","last_change":"2024-04-26T01:49:34.185069+0000","last_active":"2024-04-26T01:53:42.262655+0000","last_peered":"2024-04-26T01:53:42.262655+0000","last_clean":"2024-04-26T01:53:42.262655+0000","last_became_active":"2024-04-26T01:49:34.183989+0000","last_became_peered":"2024-04-26T01:49:34.183989+0000","last_unstale":"2024-04-26T01:53:42.262655+0000","last_undegraded":"2024-04-26T01:53:42.262655+0000","last_fullsized":"2024-04-26T01:53:42.262655+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:04:26.327960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":221,"num_read_kb":221,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.791363+0000","last_change":"2024-04-26T01:49:37.655512+0000","last_active":"2024-04-26T01:53:36.791363+0000","last_peered":"2024-04-26T01:53:36.791363+0000","last_clean":"2024-04-26T01:53:36.791363+0000","last_became_active":"2024-04-26T01:49:37.655142+0000","last_became_peered":"2024-04-26T01:49:37.655142+0000","last_unstale":"2024-04-26T01:53:36.791363+0000","last_undegraded":"2024-04-26T01:53:36.791363+0000","last_fullsized":"2024-04-26T01:53:36.791363+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:02:58.090885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.668849+0000","last_change":"2024-04-26T01:49:33.599578+0000","last_active":"2024-04-26T01:53:36.668849+0000","last_peered":"2024-04-26T01:53:36.668849+0000","last_clean":"2024-04-26T01:53:36.668849+0000","last_became_active":"2024-04-26T01:49:33.599010+0000","last_became_peered":"2024-04-26T01:49:33.599010+0000","last_unstale":"2024-04-26T01:53:36.668849+0000","last_undegraded":"2024-04-26T01:53:36.668849+0000","last_fullsized":"2024-04-26T01:53:36.668849+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:16:09.925304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"93'148","reported_seq":407,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:38.805713+0000","last_change":"2024-04-26T01:47:59.584819+0000","last_active":"2024-04-26T01:53:38.805713+0000","last_peered":"2024-04-26T01:53:38.805713+0000","last_clean":"2024-04-26T01:53:38.805713+0000","last_became_active":"2024-04-26T01:47:59.584356+0000","last_became_peered":"2024-04-26T01:47:59.584356+0000","last_unstale":"2024-04-26T01:53:38.805713+0000","last_undegraded":"2024-04-26T01:53:38.805713+0000","last_fullsized":"2024-04-26T01:53:38.805713+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:44:11.653568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:44:11.653568+0000","last_clean_scrub_stamp":"2024-04-26T01:44:11.653568+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:25:11.168583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.161855+0000","last_change":"2024-04-26T01:50:27.008911+0000","last_active":"2024-04-26T01:53:37.161855+0000","last_peered":"2024-04-26T01:53:37.161855+0000","last_clean":"2024-04-26T01:53:37.161855+0000","last_became_active":"2024-04-26T01:50:27.008507+0000","last_became_peered":"2024-04-26T01:50:27.008507+0000","last_unstale":"2024-04-26T01:53:37.161855+0000","last_undegraded":"2024-04-26T01:53:37.161855+0000","last_fullsized":"2024-04-26T01:53:37.161855+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:21:14.985935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.985710+0000","last_change":"2024-04-26T01:49:33.590171+0000","last_active":"2024-04-26T01:53:36.985710+0000","last_peered":"2024-04-26T01:53:36.985710+0000","last_clean":"2024-04-26T01:53:36.985710+0000","last_became_active":"2024-04-26T01:49:33.589312+0000","last_became_peered":"2024-04-26T01:49:33.589312+0000","last_unstale":"2024-04-26T01:53:36.985710+0000","last_undegraded":"2024-04-26T01:53:36.985710+0000","last_fullsized":"2024-04-26T01:53:36.985710+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:01:25.441527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.789485+0000","last_change":"2024-04-26T01:49:34.190338+0000","last_active":"2024-04-26T01:53:36.789485+0000","last_peered":"2024-04-26T01:53:36.789485+0000","last_clean":"2024-04-26T01:53:36.789485+0000","last_became_active":"2024-04-26T01:49:34.189984+0000","last_became_peered":"2024-04-26T01:49:34.189984+0000","last_unstale":"2024-04-26T01:53:36.789485+0000","last_undegraded":"2024-04-26T01:53:36.789485+0000","last_fullsized":"2024-04-26T01:53:36.789485+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:58.588012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.163165+0000","last_change":"2024-04-26T01:49:37.649368+0000","last_active":"2024-04-26T01:53:37.163165+0000","last_peered":"2024-04-26T01:53:37.163165+0000","last_clean":"2024-04-26T01:53:37.163165+0000","last_became_active":"2024-04-26T01:49:37.648987+0000","last_became_peered":"2024-04-26T01:49:37.648987+0000","last_unstale":"2024-04-26T01:53:37.163165+0000","last_undegraded":"2024-04-26T01:53:37.163165+0000","last_fullsized":"2024-04-26T01:53:37.163165+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:04:59.080703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"93'102","reported_seq":223,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:39.813697+0000","last_change":"2024-04-26T01:49:33.600813+0000","last_active":"2024-04-26T01:53:39.813697+0000","last_peered":"2024-04-26T01:53:39.813697+0000","last_clean":"2024-04-26T01:53:39.813697+0000","last_became_active":"2024-04-26T01:49:33.600361+0000","last_became_peered":"2024-04-26T01:49:33.600361+0000","last_unstale":"2024-04-26T01:53:39.813697+0000","last_undegraded":"2024-04-26T01:53:39.813697+0000","last_fullsized":"2024-04-26T01:53:39.813697+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:02:58.638449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":188,"num_write_kb":1288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.791866+0000","last_change":"2024-04-26T01:49:37.659381+0000","last_active":"2024-04-26T01:53:36.791866+0000","last_peered":"2024-04-26T01:53:36.791866+0000","last_clean":"2024-04-26T01:53:36.791866+0000","last_became_active":"2024-04-26T01:49:37.656205+0000","last_became_peered":"2024-04-26T01:49:37.656205+0000","last_unstale":"2024-04-26T01:53:36.791866+0000","last_undegraded":"2024-04-26T01:53:36.791866+0000","last_fullsized":"2024-04-26T01:53:36.791866+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:02:10.103330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.668898+0000","last_change":"2024-04-26T01:49:33.854732+0000","last_active":"2024-04-26T01:53:36.668898+0000","last_peered":"2024-04-26T01:53:36.668898+0000","last_clean":"2024-04-26T01:53:36.668898+0000","last_became_active":"2024-04-26T01:49:33.854396+0000","last_became_peered":"2024-04-26T01:49:33.854396+0000","last_unstale":"2024-04-26T01:53:36.668898+0000","last_undegraded":"2024-04-26T01:53:36.668898+0000","last_fullsized":"2024-04-26T01:53:36.668898+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:12:37.183725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.161320+0000","last_change":"2024-04-26T01:49:37.656366+0000","last_active":"2024-04-26T01:53:37.161320+0000","last_peered":"2024-04-26T01:53:37.161320+0000","last_clean":"2024-04-26T01:53:37.161320+0000","last_became_active":"2024-04-26T01:49:37.655869+0000","last_became_peered":"2024-04-26T01:49:37.655869+0000","last_unstale":"2024-04-26T01:53:37.161320+0000","last_undegraded":"2024-04-26T01:53:37.161320+0000","last_fullsized":"2024-04-26T01:53:37.161320+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:01:57.083315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.981654+0000","last_change":"2024-04-26T01:49:34.186070+0000","last_active":"2024-04-26T01:53:36.981654+0000","last_peered":"2024-04-26T01:53:36.981654+0000","last_clean":"2024-04-26T01:53:36.981654+0000","last_became_active":"2024-04-26T01:49:34.185742+0000","last_became_peered":"2024-04-26T01:49:34.185742+0000","last_unstale":"2024-04-26T01:53:36.981654+0000","last_undegraded":"2024-04-26T01:53:36.981654+0000","last_fullsized":"2024-04-26T01:53:36.981654+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:05.870056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.789114+0000","last_change":"2024-04-26T01:49:37.640938+0000","last_active":"2024-04-26T01:53:36.789114+0000","last_peered":"2024-04-26T01:53:36.789114+0000","last_clean":"2024-04-26T01:53:36.789114+0000","last_became_active":"2024-04-26T01:49:37.640608+0000","last_became_peered":"2024-04-26T01:49:37.640608+0000","last_unstale":"2024-04-26T01:53:36.789114+0000","last_undegraded":"2024-04-26T01:53:36.789114+0000","last_fullsized":"2024-04-26T01:53:36.789114+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:30:30.460641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.791217+0000","last_change":"2024-04-26T01:49:33.604152+0000","last_active":"2024-04-26T01:53:36.791217+0000","last_peered":"2024-04-26T01:53:36.791217+0000","last_clean":"2024-04-26T01:53:36.791217+0000","last_became_active":"2024-04-26T01:49:33.603479+0000","last_became_peered":"2024-04-26T01:49:33.603479+0000","last_unstale":"2024-04-26T01:53:36.791217+0000","last_undegraded":"2024-04-26T01:53:36.791217+0000","last_fullsized":"2024-04-26T01:53:36.791217+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:25:13.685392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.159096+0000","last_change":"2024-04-26T01:49:37.645152+0000","last_active":"2024-04-26T01:53:37.159096+0000","last_peered":"2024-04-26T01:53:37.159096+0000","last_clean":"2024-04-26T01:53:37.159096+0000","last_became_active":"2024-04-26T01:49:37.644789+0000","last_became_peered":"2024-04-26T01:49:37.644789+0000","last_unstale":"2024-04-26T01:53:37.159096+0000","last_undegraded":"2024-04-26T01:53:37.159096+0000","last_fullsized":"2024-04-26T01:53:37.159096+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:20:01.494222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"93'2","reported_seq":19,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.159226+0000","last_change":"2024-04-26T01:50:27.009973+0000","last_active":"2024-04-26T01:53:37.159226+0000","last_peered":"2024-04-26T01:53:37.159226+0000","last_clean":"2024-04-26T01:53:37.159226+0000","last_became_active":"2024-04-26T01:50:27.001678+0000","last_became_peered":"2024-04-26T01:50:27.001678+0000","last_unstale":"2024-04-26T01:53:37.159226+0000","last_undegraded":"2024-04-26T01:53:37.159226+0000","last_fullsized":"2024-04-26T01:53:37.159226+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:59:54.807792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.791338+0000","last_change":"2024-04-26T01:49:37.662665+0000","last_active":"2024-04-26T01:53:36.791338+0000","last_peered":"2024-04-26T01:53:36.791338+0000","last_clean":"2024-04-26T01:53:36.791338+0000","last_became_active":"2024-04-26T01:49:37.662216+0000","last_became_peered":"2024-04-26T01:49:37.662216+0000","last_unstale":"2024-04-26T01:53:36.791338+0000","last_undegraded":"2024-04-26T01:53:36.791338+0000","last_fullsized":"2024-04-26T01:53:36.791338+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:10:54.724858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.789242+0000","last_change":"2024-04-26T01:49:33.592078+0000","last_active":"2024-04-26T01:53:36.789242+0000","last_peered":"2024-04-26T01:53:36.789242+0000","last_clean":"2024-04-26T01:53:36.789242+0000","last_became_active":"2024-04-26T01:49:33.590265+0000","last_became_peered":"2024-04-26T01:49:33.590265+0000","last_unstale":"2024-04-26T01:53:36.789242+0000","last_undegraded":"2024-04-26T01:53:36.789242+0000","last_fullsized":"2024-04-26T01:53:36.789242+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:44:27.163705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.985525+0000","last_change":"2024-04-26T01:49:37.645943+0000","last_active":"2024-04-26T01:53:36.985525+0000","last_peered":"2024-04-26T01:53:36.985525+0000","last_clean":"2024-04-26T01:53:36.985525+0000","last_became_active":"2024-04-26T01:49:37.645586+0000","last_became_peered":"2024-04-26T01:49:37.645586+0000","last_unstale":"2024-04-26T01:53:36.985525+0000","last_undegraded":"2024-04-26T01:53:36.985525+0000","last_fullsized":"2024-04-26T01:53:36.985525+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:40.682344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.982081+0000","last_change":"2024-04-26T01:49:34.187115+0000","last_active":"2024-04-26T01:53:36.982081+0000","last_peered":"2024-04-26T01:53:36.982081+0000","last_clean":"2024-04-26T01:53:36.982081+0000","last_became_active":"2024-04-26T01:49:34.186660+0000","last_became_peered":"2024-04-26T01:49:34.186660+0000","last_unstale":"2024-04-26T01:53:36.982081+0000","last_undegraded":"2024-04-26T01:53:36.982081+0000","last_fullsized":"2024-04-26T01:53:36.982081+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:56:32.274752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.985756+0000","last_change":"2024-04-26T01:49:37.650382+0000","last_active":"2024-04-26T01:53:36.985756+0000","last_peered":"2024-04-26T01:53:36.985756+0000","last_clean":"2024-04-26T01:53:36.985756+0000","last_became_active":"2024-04-26T01:49:37.649545+0000","last_became_peered":"2024-04-26T01:49:37.649545+0000","last_unstale":"2024-04-26T01:53:36.985756+0000","last_undegraded":"2024-04-26T01:53:36.985756+0000","last_fullsized":"2024-04-26T01:53:36.985756+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:43:16.291311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"93'65","reported_seq":125,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:39.347187+0000","last_change":"2024-04-26T01:49:33.595881+0000","last_active":"2024-04-26T01:53:39.347187+0000","last_peered":"2024-04-26T01:53:39.347187+0000","last_clean":"2024-04-26T01:53:39.347187+0000","last_became_active":"2024-04-26T01:49:33.595480+0000","last_became_peered":"2024-04-26T01:49:33.595480+0000","last_unstale":"2024-04-26T01:53:39.347187+0000","last_undegraded":"2024-04-26T01:53:39.347187+0000","last_fullsized":"2024-04-26T01:53:39.347187+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:30:58.028230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.985506+0000","last_change":"2024-04-26T01:49:37.660765+0000","last_active":"2024-04-26T01:53:36.985506+0000","last_peered":"2024-04-26T01:53:36.985506+0000","last_clean":"2024-04-26T01:53:36.985506+0000","last_became_active":"2024-04-26T01:49:37.660475+0000","last_became_peered":"2024-04-26T01:49:37.660475+0000","last_unstale":"2024-04-26T01:53:36.985506+0000","last_undegraded":"2024-04-26T01:53:36.985506+0000","last_fullsized":"2024-04-26T01:53:36.985506+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:36:00.483005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"90'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.162985+0000","last_change":"2024-04-26T01:49:33.598247+0000","last_active":"2024-04-26T01:53:37.162985+0000","last_peered":"2024-04-26T01:53:37.162985+0000","last_clean":"2024-04-26T01:53:37.162985+0000","last_became_active":"2024-04-26T01:49:33.597796+0000","last_became_peered":"2024-04-26T01:49:33.597796+0000","last_unstale":"2024-04-26T01:53:37.162985+0000","last_undegraded":"2024-04-26T01:53:37.162985+0000","last_fullsized":"2024-04-26T01:53:37.162985+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:53:38.994798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.985835+0000","last_change":"2024-04-26T01:49:37.648903+0000","last_active":"2024-04-26T01:53:36.985835+0000","last_peered":"2024-04-26T01:53:36.985835+0000","last_clean":"2024-04-26T01:53:36.985835+0000","last_became_active":"2024-04-26T01:49:37.648570+0000","last_became_peered":"2024-04-26T01:49:37.648570+0000","last_unstale":"2024-04-26T01:53:36.985835+0000","last_undegraded":"2024-04-26T01:53:36.985835+0000","last_fullsized":"2024-04-26T01:53:36.985835+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:33:12.678513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.159153+0000","last_change":"2024-04-26T01:49:37.648485+0000","last_active":"2024-04-26T01:53:37.159153+0000","last_peered":"2024-04-26T01:53:37.159153+0000","last_clean":"2024-04-26T01:53:37.159153+0000","last_became_active":"2024-04-26T01:49:37.648129+0000","last_became_peered":"2024-04-26T01:49:37.648129+0000","last_unstale":"2024-04-26T01:53:37.159153+0000","last_undegraded":"2024-04-26T01:53:37.159153+0000","last_fullsized":"2024-04-26T01:53:37.159153+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:56:12.047821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.981838+0000","last_change":"2024-04-26T01:49:37.645929+0000","last_active":"2024-04-26T01:53:36.981838+0000","last_peered":"2024-04-26T01:53:36.981838+0000","last_clean":"2024-04-26T01:53:36.981838+0000","last_became_active":"2024-04-26T01:49:37.645473+0000","last_became_peered":"2024-04-26T01:49:37.645473+0000","last_unstale":"2024-04-26T01:53:36.981838+0000","last_undegraded":"2024-04-26T01:53:36.981838+0000","last_fullsized":"2024-04-26T01:53:36.981838+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:39:56.687973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.981790+0000","last_change":"2024-04-26T01:49:37.657494+0000","last_active":"2024-04-26T01:53:36.981790+0000","last_peered":"2024-04-26T01:53:36.981790+0000","last_clean":"2024-04-26T01:53:36.981790+0000","last_became_active":"2024-04-26T01:49:37.657121+0000","last_became_peered":"2024-04-26T01:49:37.657121+0000","last_unstale":"2024-04-26T01:53:36.981790+0000","last_undegraded":"2024-04-26T01:53:36.981790+0000","last_fullsized":"2024-04-26T01:53:36.981790+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:50:44.691179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.160724+0000","last_change":"2024-04-26T01:49:37.646012+0000","last_active":"2024-04-26T01:53:37.160724+0000","last_peered":"2024-04-26T01:53:37.160724+0000","last_clean":"2024-04-26T01:53:37.160724+0000","last_became_active":"2024-04-26T01:49:37.645695+0000","last_became_peered":"2024-04-26T01:49:37.645695+0000","last_unstale":"2024-04-26T01:53:37.160724+0000","last_undegraded":"2024-04-26T01:53:37.160724+0000","last_fullsized":"2024-04-26T01:53:37.160724+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:40:51.117523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.163306+0000","last_change":"2024-04-26T01:49:37.646503+0000","last_active":"2024-04-26T01:53:37.163306+0000","last_peered":"2024-04-26T01:53:37.163306+0000","last_clean":"2024-04-26T01:53:37.163306+0000","last_became_active":"2024-04-26T01:49:37.645200+0000","last_became_peered":"2024-04-26T01:49:37.645200+0000","last_unstale":"2024-04-26T01:53:37.163306+0000","last_undegraded":"2024-04-26T01:53:37.163306+0000","last_fullsized":"2024-04-26T01:53:37.163306+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:31:10.957199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.159402+0000","last_change":"2024-04-26T01:50:27.002908+0000","last_active":"2024-04-26T01:53:37.159402+0000","last_peered":"2024-04-26T01:53:37.159402+0000","last_clean":"2024-04-26T01:53:37.159402+0000","last_became_active":"2024-04-26T01:50:27.002454+0000","last_became_peered":"2024-04-26T01:50:27.002454+0000","last_unstale":"2024-04-26T01:53:37.159402+0000","last_undegraded":"2024-04-26T01:53:37.159402+0000","last_fullsized":"2024-04-26T01:53:37.159402+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:18:01.848427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.982171+0000","last_change":"2024-04-26T01:49:37.642676+0000","last_active":"2024-04-26T01:53:36.982171+0000","last_peered":"2024-04-26T01:53:36.982171+0000","last_clean":"2024-04-26T01:53:36.982171+0000","last_became_active":"2024-04-26T01:49:37.642302+0000","last_became_peered":"2024-04-26T01:49:37.642302+0000","last_unstale":"2024-04-26T01:53:36.982171+0000","last_undegraded":"2024-04-26T01:53:36.982171+0000","last_fullsized":"2024-04-26T01:53:36.982171+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:51:43.454649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.791673+0000","last_change":"2024-04-26T01:50:27.003914+0000","last_active":"2024-04-26T01:53:36.791673+0000","last_peered":"2024-04-26T01:53:36.791673+0000","last_clean":"2024-04-26T01:53:36.791673+0000","last_became_active":"2024-04-26T01:50:27.002344+0000","last_became_peered":"2024-04-26T01:50:27.002344+0000","last_unstale":"2024-04-26T01:53:36.791673+0000","last_undegraded":"2024-04-26T01:53:36.791673+0000","last_fullsized":"2024-04-26T01:53:36.791673+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:46:02.101508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.666138+0000","last_change":"2024-04-26T01:49:37.663306+0000","last_active":"2024-04-26T01:53:36.666138+0000","last_peered":"2024-04-26T01:53:36.666138+0000","last_clean":"2024-04-26T01:53:36.666138+0000","last_became_active":"2024-04-26T01:49:37.662999+0000","last_became_peered":"2024-04-26T01:49:37.662999+0000","last_unstale":"2024-04-26T01:53:36.666138+0000","last_undegraded":"2024-04-26T01:53:36.666138+0000","last_fullsized":"2024-04-26T01:53:36.666138+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:55.155096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.789468+0000","last_change":"2024-04-26T01:49:37.655157+0000","last_active":"2024-04-26T01:53:36.789468+0000","last_peered":"2024-04-26T01:53:36.789468+0000","last_clean":"2024-04-26T01:53:36.789468+0000","last_became_active":"2024-04-26T01:49:37.654810+0000","last_became_peered":"2024-04-26T01:49:37.654810+0000","last_unstale":"2024-04-26T01:53:36.789468+0000","last_undegraded":"2024-04-26T01:53:36.789468+0000","last_fullsized":"2024-04-26T01:53:36.789468+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:28:04.012382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.981829+0000","last_change":"2024-04-26T01:49:37.667228+0000","last_active":"2024-04-26T01:53:36.981829+0000","last_peered":"2024-04-26T01:53:36.981829+0000","last_clean":"2024-04-26T01:53:36.981829+0000","last_became_active":"2024-04-26T01:49:37.666835+0000","last_became_peered":"2024-04-26T01:49:37.666835+0000","last_unstale":"2024-04-26T01:53:36.981829+0000","last_undegraded":"2024-04-26T01:53:36.981829+0000","last_fullsized":"2024-04-26T01:53:36.981829+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:54:16.902836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.162824+0000","last_change":"2024-04-26T01:49:37.658231+0000","last_active":"2024-04-26T01:53:37.162824+0000","last_peered":"2024-04-26T01:53:37.162824+0000","last_clean":"2024-04-26T01:53:37.162824+0000","last_became_active":"2024-04-26T01:49:37.657842+0000","last_became_peered":"2024-04-26T01:49:37.657842+0000","last_unstale":"2024-04-26T01:53:37.162824+0000","last_undegraded":"2024-04-26T01:53:37.162824+0000","last_fullsized":"2024-04-26T01:53:37.162824+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:02:01.445262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.159180+0000","last_change":"2024-04-26T01:49:37.645221+0000","last_active":"2024-04-26T01:53:37.159180+0000","last_peered":"2024-04-26T01:53:37.159180+0000","last_clean":"2024-04-26T01:53:37.159180+0000","last_became_active":"2024-04-26T01:49:37.644814+0000","last_became_peered":"2024-04-26T01:49:37.644814+0000","last_unstale":"2024-04-26T01:53:37.159180+0000","last_undegraded":"2024-04-26T01:53:37.159180+0000","last_fullsized":"2024-04-26T01:53:37.159180+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:03:17.850633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":796795,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1709,"num_read_kb":1072,"num_write":424,"num_write_kb":2358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2617344,"data_stored":2390385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":311,"ondisk_log_size":311,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318340,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1011712,"data_stored":495249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579974,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29136,"kb_used_data":2000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716016,"statfs":{"total":95995035648,"available":95965200384,"internally_reserved":0,"allocated":2048000,"data_stored":1485466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743500,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36724,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708428,"statfs":{"total":95995035648,"available":95957430272,"internally_reserved":0,"allocated":1421312,"data_stored":888701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005139,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28404,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716748,"statfs":{"total":95995035648,"available":95965949952,"internally_reserved":0,"allocated":1421312,"data_stored":888621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266775,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28600,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716552,"statfs":{"total":95995035648,"available":95965749248,"internally_reserved":0,"allocated":1622016,"data_stored":1088709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430302,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1015808,"data_stored":494874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574970,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36400,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708752,"statfs":{"total":95995035648,"available":95957762048,"internally_reserved":0,"allocated":1085440,"data_stored":564639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738494,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1036288,"data_stored":496145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411445,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36424,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708728,"statfs":{"total":95995035648,"available":95957737472,"internally_reserved":0,"allocated":1110016,"data_stored":567222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117102,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37172,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707980,"statfs":{"total":95995035648,"available":95956971520,"internally_reserved":0,"allocated":1757184,"data_stored":1216633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855466,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1019904,"data_stored":495843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691940,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29056,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716096,"statfs":{"total":95995035648,"available":95965282304,"internally_reserved":0,"allocated":1966080,"data_stored":1416093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":72555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":741376,"data_stored":721732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":360448,"data_stored":330941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":393954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":400314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T01:53:47.751 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-26T01:53:47.751 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T01:53:48.234 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:53:48.720 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:48 smithi151 ceph-mon[28057]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 102 KiB/s wr, 60 op/s 2024-04-26T01:53:48.721 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:48 smithi151 ceph-mon[28057]: from='client.25159 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:53:48.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:48 smithi172 ceph-mon[32785]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 102 KiB/s wr, 60 op/s 2024-04-26T01:53:48.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:48 smithi172 ceph-mon[32785]: from='client.25159 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:53:48.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:48 smithi165 ceph-mon[32648]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 102 KiB/s wr, 60 op/s 2024-04-26T01:53:48.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:48 smithi165 ceph-mon[32648]: from='client.25159 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:53:49.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:49 smithi172 ceph-mon[32785]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-04-26T01:53:49.824 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:49 smithi151 ceph-mon[28057]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-04-26T01:53:49.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:49 smithi165 ceph-mon[32648]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-04-26T01:53:50.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:50.407+0000 7fab29800700 1 -- 172.21.15.151:0/888197843 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab24109fc0 msgr2=0x7fab24111b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:50.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:50.407+0000 7fab29800700 1 --2- 172.21.15.151:0/888197843 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab24109fc0 0x7fab24111b90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fab1c00b0d0 tx=0x7fab1c00b3e0 comp rx=0 tx=0).stop 2024-04-26T01:53:50.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:50.407+0000 7fab29800700 1 -- 172.21.15.151:0/888197843 shutdown_connections 2024-04-26T01:53:50.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:50.407+0000 7fab29800700 1 --2- 172.21.15.151:0/888197843 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab24109fc0 0x7fab24111b90 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:50.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:50.407+0000 7fab29800700 1 --2- 172.21.15.151:0/888197843 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab24104ee0 0x7fab241052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:50.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:50.407+0000 7fab29800700 1 --2- 172.21.15.151:0/888197843 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fab24105890 0x7fab241098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:50.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:50.407+0000 7fab29800700 1 -- 172.21.15.151:0/888197843 >> 172.21.15.151:0/888197843 conn(0x7fab24100770 msgr2=0x7fab24102b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:50.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:50.407+0000 7fab29800700 1 -- 172.21.15.151:0/888197843 shutdown_connections 2024-04-26T01:53:50.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:50.408+0000 7fab29800700 1 -- 172.21.15.151:0/888197843 wait complete. 2024-04-26T01:53:50.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:50.409+0000 7fab29800700 1 Processor -- start 2024-04-26T01:53:50.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:50.409+0000 7fab29800700 1 -- start start 2024-04-26T01:53:50.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:50.409+0000 7fab29800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab24104ee0 0x7fab2419b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:50.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:50.409+0000 7fab29800700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fab24105890 0x7fab2419b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:50.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:50.409+0000 7fab29800700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab24109fc0 0x7fab2419fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:50.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:50.409+0000 7fab29800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fab24114ec0 con 0x7fab24104ee0 2024-04-26T01:53:50.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:50.409+0000 7fab29800700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fab24114d40 con 0x7fab24105890 2024-04-26T01:53:50.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:50.409+0000 7fab29800700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fab24115040 con 0x7fab24109fc0 2024-04-26T01:53:50.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:50.409+0000 7fab22a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab24104ee0 0x7fab2419b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:50.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:50.410+0000 7fab22a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab24104ee0 0x7fab2419b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:39318/0 (socket says 172.21.15.151:39318) 2024-04-26T01:53:50.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:50.410+0000 7fab22a00700 1 -- 172.21.15.151:0/1764668703 learned_addr learned my addr 172.21.15.151:0/1764668703 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:53:50.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:50.410+0000 7fab22000700 1 --2- 172.21.15.151:0/1764668703 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fab24105890 0x7fab2419b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:50.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:50.410+0000 7fab23400700 1 --2- 172.21.15.151:0/1764668703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab24109fc0 0x7fab2419fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:50.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:50.410+0000 7fab22a00700 1 -- 172.21.15.151:0/1764668703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab24109fc0 msgr2=0x7fab2419fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:50.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:50.410+0000 7fab22a00700 1 --2- 172.21.15.151:0/1764668703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab24109fc0 0x7fab2419fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:50.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:50.410+0000 7fab22a00700 1 -- 172.21.15.151:0/1764668703 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fab24105890 msgr2=0x7fab2419b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:50.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:50.410+0000 7fab22a00700 1 --2- 172.21.15.151:0/1764668703 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fab24105890 0x7fab2419b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:50.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:50.410+0000 7fab22a00700 1 -- 172.21.15.151:0/1764668703 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab241a0300 con 0x7fab24104ee0 2024-04-26T01:53:50.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:50.410+0000 7fab23400700 1 --2- 172.21.15.151:0/1764668703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab24109fc0 0x7fab2419fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:53:50.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:50.410+0000 7fab22a00700 1 --2- 172.21.15.151:0/1764668703 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab24104ee0 0x7fab2419b2f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fab1400e9c0 tx=0x7fab1400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:50.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:50.411+0000 7fab1b400700 1 -- 172.21.15.151:0/1764668703 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1400cbd0 con 0x7fab24104ee0 2024-04-26T01:53:50.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:50.411+0000 7fab1b400700 1 -- 172.21.15.151:0/1764668703 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab1400cd70 con 0x7fab24104ee0 2024-04-26T01:53:50.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:50.411+0000 7fab29800700 1 -- 172.21.15.151:0/1764668703 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab241a05f0 con 0x7fab24104ee0 2024-04-26T01:53:50.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:50.411+0000 7fab1b400700 1 -- 172.21.15.151:0/1764668703 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab14005610 con 0x7fab24104ee0 2024-04-26T01:53:50.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:50.411+0000 7fab29800700 1 -- 172.21.15.151:0/1764668703 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab241a7e70 con 0x7fab24104ee0 2024-04-26T01:53:50.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:50.412+0000 7fab29800700 1 -- 172.21.15.151:0/1764668703 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab24069730 con 0x7fab24104ee0 2024-04-26T01:53:50.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:50.414+0000 7fab1b400700 1 -- 172.21.15.151:0/1764668703 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 25) ==== 138874+0+0 (secure 0 0 0) 0x7fab14004750 con 0x7fab24104ee0 2024-04-26T01:53:50.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:50.415+0000 7fab1b400700 1 --2- 172.21.15.151:0/1764668703 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fab080a3ed0 0x7fab080a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:50.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:50.416+0000 7fab1b400700 1 -- 172.21.15.151:0/1764668703 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fab140d0de0 con 0x7fab24104ee0 2024-04-26T01:53:50.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:50.416+0000 7fab22000700 1 --2- 172.21.15.151:0/1764668703 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fab080a3ed0 0x7fab080a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:50.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:50.416+0000 7fab22000700 1 --2- 172.21.15.151:0/1764668703 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fab080a3ed0 0x7fab080a6390 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fab2410b510 tx=0x7fab10009260 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:50.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:50.420+0000 7fab1b400700 1 -- 172.21.15.151:0/1764668703 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fab1409d240 con 0x7fab24104ee0 2024-04-26T01:53:50.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:50.644+0000 7fab29800700 1 -- 172.21.15.151:0/1764668703 --> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fab24062e80 con 0x7fab080a3ed0 2024-04-26T01:53:50.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:50.654+0000 7fab1b400700 1 -- 172.21.15.151:0/1764668703 <== mgr.25025 v2:172.21.15.151:6800/2427564083 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257769 (secure 0 0 0) 0x7fab24062e80 con 0x7fab080a3ed0 2024-04-26T01:53:50.660 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:53:50.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:50.661+0000 7fab29800700 1 -- 172.21.15.151:0/1764668703 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fab080a3ed0 msgr2=0x7fab080a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:50.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:50.661+0000 7fab29800700 1 --2- 172.21.15.151:0/1764668703 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fab080a3ed0 0x7fab080a6390 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fab2410b510 tx=0x7fab10009260 comp rx=0 tx=0).stop 2024-04-26T01:53:50.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:50.661+0000 7fab29800700 1 -- 172.21.15.151:0/1764668703 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab24104ee0 msgr2=0x7fab2419b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:50.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:50.661+0000 7fab29800700 1 --2- 172.21.15.151:0/1764668703 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab24104ee0 0x7fab2419b2f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fab1400e9c0 tx=0x7fab1400ecd0 comp rx=0 tx=0).stop 2024-04-26T01:53:50.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:50.661+0000 7fab29800700 1 -- 172.21.15.151:0/1764668703 shutdown_connections 2024-04-26T01:53:50.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:50.661+0000 7fab29800700 1 --2- 172.21.15.151:0/1764668703 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fab080a3ed0 0x7fab080a6390 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:50.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:50.661+0000 7fab29800700 1 --2- 172.21.15.151:0/1764668703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab24109fc0 0x7fab2419fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:50.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:50.661+0000 7fab29800700 1 --2- 172.21.15.151:0/1764668703 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab24104ee0 0x7fab2419b2f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:50.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:50.661+0000 7fab29800700 1 --2- 172.21.15.151:0/1764668703 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fab24105890 0x7fab2419b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:50.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:50.661+0000 7fab29800700 1 -- 172.21.15.151:0/1764668703 >> 172.21.15.151:0/1764668703 conn(0x7fab24100770 msgr2=0x7fab24101260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:50.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:50.661+0000 7fab29800700 1 -- 172.21.15.151:0/1764668703 shutdown_connections 2024-04-26T01:53:50.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:50.662+0000 7fab29800700 1 -- 172.21.15.151:0/1764668703 wait complete. 2024-04-26T01:53:50.664 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-04-26T01:53:51.571 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":9,"stamp":"2024-04-26T01:53:48.694658+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1387397,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1976,"num_read_kb":1417,"num_write":627,"num_write_kb":5298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":961,"ondisk_log_size":961,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":390624,"kb_used_data":16128,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124551200,"statfs":{"total":1151940427776,"available":1151540428800,"internally_reserved":0,"allocated":16515072,"data_stored":10098195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":199543,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":416,"num_read_kb":475,"num_write":154,"num_write_kb":921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.994408"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.159496+0000","last_change":"2024-04-26T01:49:37.641666+0000","last_active":"2024-04-26T01:53:37.159496+0000","last_peered":"2024-04-26T01:53:37.159496+0000","last_clean":"2024-04-26T01:53:37.159496+0000","last_became_active":"2024-04-26T01:49:37.641310+0000","last_became_peered":"2024-04-26T01:49:37.641310+0000","last_unstale":"2024-04-26T01:53:37.159496+0000","last_undegraded":"2024-04-26T01:53:37.159496+0000","last_fullsized":"2024-04-26T01:53:37.159496+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:28:02.952851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.161240+0000","last_change":"2024-04-26T01:49:37.637607+0000","last_active":"2024-04-26T01:53:37.161240+0000","last_peered":"2024-04-26T01:53:37.161240+0000","last_clean":"2024-04-26T01:53:37.161240+0000","last_became_active":"2024-04-26T01:49:37.637283+0000","last_became_peered":"2024-04-26T01:49:37.637283+0000","last_unstale":"2024-04-26T01:53:37.161240+0000","last_undegraded":"2024-04-26T01:53:37.161240+0000","last_fullsized":"2024-04-26T01:53:37.161240+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:16:46.519381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.161424+0000","last_change":"2024-04-26T01:49:37.647769+0000","last_active":"2024-04-26T01:53:37.161424+0000","last_peered":"2024-04-26T01:53:37.161424+0000","last_clean":"2024-04-26T01:53:37.161424+0000","last_became_active":"2024-04-26T01:49:37.643750+0000","last_became_peered":"2024-04-26T01:49:37.643750+0000","last_unstale":"2024-04-26T01:53:37.161424+0000","last_undegraded":"2024-04-26T01:53:37.161424+0000","last_fullsized":"2024-04-26T01:53:37.161424+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:34:18.801267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.669085+0000","last_change":"2024-04-26T01:49:37.643291+0000","last_active":"2024-04-26T01:53:36.669085+0000","last_peered":"2024-04-26T01:53:36.669085+0000","last_clean":"2024-04-26T01:53:36.669085+0000","last_became_active":"2024-04-26T01:49:37.642947+0000","last_became_peered":"2024-04-26T01:49:37.642947+0000","last_unstale":"2024-04-26T01:53:36.669085+0000","last_undegraded":"2024-04-26T01:53:36.669085+0000","last_fullsized":"2024-04-26T01:53:36.669085+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:31:21.600533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.789294+0000","last_change":"2024-04-26T01:49:37.665311+0000","last_active":"2024-04-26T01:53:36.789294+0000","last_peered":"2024-04-26T01:53:36.789294+0000","last_clean":"2024-04-26T01:53:36.789294+0000","last_became_active":"2024-04-26T01:49:37.664944+0000","last_became_peered":"2024-04-26T01:49:37.664944+0000","last_unstale":"2024-04-26T01:53:36.789294+0000","last_undegraded":"2024-04-26T01:53:36.789294+0000","last_fullsized":"2024-04-26T01:53:36.789294+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:49:50.744630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.799444+0000","last_change":"2024-04-26T01:49:37.638667+0000","last_active":"2024-04-26T01:53:36.799444+0000","last_peered":"2024-04-26T01:53:36.799444+0000","last_clean":"2024-04-26T01:53:36.799444+0000","last_became_active":"2024-04-26T01:49:37.638226+0000","last_became_peered":"2024-04-26T01:49:37.638226+0000","last_unstale":"2024-04-26T01:53:36.799444+0000","last_undegraded":"2024-04-26T01:53:36.799444+0000","last_fullsized":"2024-04-26T01:53:36.799444+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:46:17.786883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.985342+0000","last_change":"2024-04-26T01:49:37.650082+0000","last_active":"2024-04-26T01:53:36.985342+0000","last_peered":"2024-04-26T01:53:36.985342+0000","last_clean":"2024-04-26T01:53:36.985342+0000","last_became_active":"2024-04-26T01:49:37.649637+0000","last_became_peered":"2024-04-26T01:49:37.649637+0000","last_unstale":"2024-04-26T01:53:36.985342+0000","last_undegraded":"2024-04-26T01:53:36.985342+0000","last_fullsized":"2024-04-26T01:53:36.985342+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:21:04.686398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.789412+0000","last_change":"2024-04-26T01:49:37.661255+0000","last_active":"2024-04-26T01:53:36.789412+0000","last_peered":"2024-04-26T01:53:36.789412+0000","last_clean":"2024-04-26T01:53:36.789412+0000","last_became_active":"2024-04-26T01:49:37.658882+0000","last_became_peered":"2024-04-26T01:49:37.658882+0000","last_unstale":"2024-04-26T01:53:36.789412+0000","last_undegraded":"2024-04-26T01:53:36.789412+0000","last_fullsized":"2024-04-26T01:53:36.789412+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:36:27.874191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.982393+0000","last_change":"2024-04-26T01:49:37.648874+0000","last_active":"2024-04-26T01:53:36.982393+0000","last_peered":"2024-04-26T01:53:36.982393+0000","last_clean":"2024-04-26T01:53:36.982393+0000","last_became_active":"2024-04-26T01:49:37.648490+0000","last_became_peered":"2024-04-26T01:49:37.648490+0000","last_unstale":"2024-04-26T01:53:36.982393+0000","last_undegraded":"2024-04-26T01:53:36.982393+0000","last_fullsized":"2024-04-26T01:53:36.982393+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:46:12.552861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.799768+0000","last_change":"2024-04-26T01:49:37.643842+0000","last_active":"2024-04-26T01:53:36.799768+0000","last_peered":"2024-04-26T01:53:36.799768+0000","last_clean":"2024-04-26T01:53:36.799768+0000","last_became_active":"2024-04-26T01:49:37.643084+0000","last_became_peered":"2024-04-26T01:49:37.643084+0000","last_unstale":"2024-04-26T01:53:36.799768+0000","last_undegraded":"2024-04-26T01:53:36.799768+0000","last_fullsized":"2024-04-26T01:53:36.799768+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:16:53.102318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.985635+0000","last_change":"2024-04-26T01:49:37.644209+0000","last_active":"2024-04-26T01:53:36.985635+0000","last_peered":"2024-04-26T01:53:36.985635+0000","last_clean":"2024-04-26T01:53:36.985635+0000","last_became_active":"2024-04-26T01:49:37.643659+0000","last_became_peered":"2024-04-26T01:49:37.643659+0000","last_unstale":"2024-04-26T01:53:36.985635+0000","last_undegraded":"2024-04-26T01:53:36.985635+0000","last_fullsized":"2024-04-26T01:53:36.985635+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:24:46.363873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.665727+0000","last_change":"2024-04-26T01:50:27.001615+0000","last_active":"2024-04-26T01:53:36.665727+0000","last_peered":"2024-04-26T01:53:36.665727+0000","last_clean":"2024-04-26T01:53:36.665727+0000","last_became_active":"2024-04-26T01:50:27.001039+0000","last_became_peered":"2024-04-26T01:50:27.001039+0000","last_unstale":"2024-04-26T01:53:36.665727+0000","last_undegraded":"2024-04-26T01:53:36.665727+0000","last_fullsized":"2024-04-26T01:53:36.665727+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:42:46.148500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.162831+0000","last_change":"2024-04-26T01:49:37.651085+0000","last_active":"2024-04-26T01:53:37.162831+0000","last_peered":"2024-04-26T01:53:37.162831+0000","last_clean":"2024-04-26T01:53:37.162831+0000","last_became_active":"2024-04-26T01:49:37.649662+0000","last_became_peered":"2024-04-26T01:49:37.649662+0000","last_unstale":"2024-04-26T01:53:37.162831+0000","last_undegraded":"2024-04-26T01:53:37.162831+0000","last_fullsized":"2024-04-26T01:53:37.162831+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:41:34.537533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.160412+0000","last_change":"2024-04-26T01:49:37.648432+0000","last_active":"2024-04-26T01:53:37.160412+0000","last_peered":"2024-04-26T01:53:37.160412+0000","last_clean":"2024-04-26T01:53:37.160412+0000","last_became_active":"2024-04-26T01:49:37.648047+0000","last_became_peered":"2024-04-26T01:49:37.648047+0000","last_unstale":"2024-04-26T01:53:37.160412+0000","last_undegraded":"2024-04-26T01:53:37.160412+0000","last_fullsized":"2024-04-26T01:53:37.160412+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:00:06.843384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.160320+0000","last_change":"2024-04-26T01:49:37.644203+0000","last_active":"2024-04-26T01:53:37.160320+0000","last_peered":"2024-04-26T01:53:37.160320+0000","last_clean":"2024-04-26T01:53:37.160320+0000","last_became_active":"2024-04-26T01:49:37.643772+0000","last_became_peered":"2024-04-26T01:49:37.643772+0000","last_unstale":"2024-04-26T01:53:37.160320+0000","last_undegraded":"2024-04-26T01:53:37.160320+0000","last_fullsized":"2024-04-26T01:53:37.160320+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:52:20.707845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.981645+0000","last_change":"2024-04-26T01:49:37.647364+0000","last_active":"2024-04-26T01:53:36.981645+0000","last_peered":"2024-04-26T01:53:36.981645+0000","last_clean":"2024-04-26T01:53:36.981645+0000","last_became_active":"2024-04-26T01:49:37.647054+0000","last_became_peered":"2024-04-26T01:49:37.647054+0000","last_unstale":"2024-04-26T01:53:36.981645+0000","last_undegraded":"2024-04-26T01:53:36.981645+0000","last_fullsized":"2024-04-26T01:53:36.981645+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:53:09.631463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.159581+0000","last_change":"2024-04-26T01:49:37.641843+0000","last_active":"2024-04-26T01:53:37.159581+0000","last_peered":"2024-04-26T01:53:37.159581+0000","last_clean":"2024-04-26T01:53:37.159581+0000","last_became_active":"2024-04-26T01:49:37.641514+0000","last_became_peered":"2024-04-26T01:49:37.641514+0000","last_unstale":"2024-04-26T01:53:37.159581+0000","last_undegraded":"2024-04-26T01:53:37.159581+0000","last_fullsized":"2024-04-26T01:53:37.159581+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:24:32.261834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.665881+0000","last_change":"2024-04-26T01:49:37.644750+0000","last_active":"2024-04-26T01:53:36.665881+0000","last_peered":"2024-04-26T01:53:36.665881+0000","last_clean":"2024-04-26T01:53:36.665881+0000","last_became_active":"2024-04-26T01:49:37.644086+0000","last_became_peered":"2024-04-26T01:49:37.644086+0000","last_unstale":"2024-04-26T01:53:36.665881+0000","last_undegraded":"2024-04-26T01:53:36.665881+0000","last_fullsized":"2024-04-26T01:53:36.665881+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:59:42.614846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.982091+0000","last_change":"2024-04-26T01:49:37.664204+0000","last_active":"2024-04-26T01:53:36.982091+0000","last_peered":"2024-04-26T01:53:36.982091+0000","last_clean":"2024-04-26T01:53:36.982091+0000","last_became_active":"2024-04-26T01:49:37.663878+0000","last_became_peered":"2024-04-26T01:49:37.663878+0000","last_unstale":"2024-04-26T01:53:36.982091+0000","last_undegraded":"2024-04-26T01:53:36.982091+0000","last_fullsized":"2024-04-26T01:53:36.982091+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:43:49.272308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.665626+0000","last_change":"2024-04-26T01:49:33.592017+0000","last_active":"2024-04-26T01:53:36.665626+0000","last_peered":"2024-04-26T01:53:36.665626+0000","last_clean":"2024-04-26T01:53:36.665626+0000","last_became_active":"2024-04-26T01:49:33.587928+0000","last_became_peered":"2024-04-26T01:49:33.587928+0000","last_unstale":"2024-04-26T01:53:36.665626+0000","last_undegraded":"2024-04-26T01:53:36.665626+0000","last_fullsized":"2024-04-26T01:53:36.665626+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:02:04.313785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.791898+0000","last_change":"2024-04-26T01:49:37.662010+0000","last_active":"2024-04-26T01:53:36.791898+0000","last_peered":"2024-04-26T01:53:36.791898+0000","last_clean":"2024-04-26T01:53:36.791898+0000","last_became_active":"2024-04-26T01:49:37.661673+0000","last_became_peered":"2024-04-26T01:49:37.661673+0000","last_unstale":"2024-04-26T01:53:36.791898+0000","last_undegraded":"2024-04-26T01:53:36.791898+0000","last_fullsized":"2024-04-26T01:53:36.791898+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:37:34.155998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.792067+0000","last_change":"2024-04-26T01:49:34.185758+0000","last_active":"2024-04-26T01:53:36.792067+0000","last_peered":"2024-04-26T01:53:36.792067+0000","last_clean":"2024-04-26T01:53:36.792067+0000","last_became_active":"2024-04-26T01:49:34.184669+0000","last_became_peered":"2024-04-26T01:49:34.184669+0000","last_unstale":"2024-04-26T01:53:36.792067+0000","last_undegraded":"2024-04-26T01:53:36.792067+0000","last_fullsized":"2024-04-26T01:53:36.792067+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:23:23.742032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.161839+0000","last_change":"2024-04-26T01:49:37.641307+0000","last_active":"2024-04-26T01:53:37.161839+0000","last_peered":"2024-04-26T01:53:37.161839+0000","last_clean":"2024-04-26T01:53:37.161839+0000","last_became_active":"2024-04-26T01:49:37.640677+0000","last_became_peered":"2024-04-26T01:49:37.640677+0000","last_unstale":"2024-04-26T01:53:37.161839+0000","last_undegraded":"2024-04-26T01:53:37.161839+0000","last_fullsized":"2024-04-26T01:53:37.161839+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:35:56.859045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.159340+0000","last_change":"2024-04-26T01:50:27.007034+0000","last_active":"2024-04-26T01:53:37.159340+0000","last_peered":"2024-04-26T01:53:37.159340+0000","last_clean":"2024-04-26T01:53:37.159340+0000","last_became_active":"2024-04-26T01:50:27.000945+0000","last_became_peered":"2024-04-26T01:50:27.000945+0000","last_unstale":"2024-04-26T01:53:37.159340+0000","last_undegraded":"2024-04-26T01:53:37.159340+0000","last_fullsized":"2024-04-26T01:53:37.159340+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:39:02.484719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.981778+0000","last_change":"2024-04-26T01:49:37.645936+0000","last_active":"2024-04-26T01:53:36.981778+0000","last_peered":"2024-04-26T01:53:36.981778+0000","last_clean":"2024-04-26T01:53:36.981778+0000","last_became_active":"2024-04-26T01:49:37.645535+0000","last_became_peered":"2024-04-26T01:49:37.645535+0000","last_unstale":"2024-04-26T01:53:36.981778+0000","last_undegraded":"2024-04-26T01:53:36.981778+0000","last_fullsized":"2024-04-26T01:53:36.981778+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:45:04.434415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"90'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.789508+0000","last_change":"2024-04-26T01:49:33.589904+0000","last_active":"2024-04-26T01:53:36.789508+0000","last_peered":"2024-04-26T01:53:36.789508+0000","last_clean":"2024-04-26T01:53:36.789508+0000","last_became_active":"2024-04-26T01:49:33.589332+0000","last_became_peered":"2024-04-26T01:49:33.589332+0000","last_unstale":"2024-04-26T01:53:36.789508+0000","last_undegraded":"2024-04-26T01:53:36.789508+0000","last_fullsized":"2024-04-26T01:53:36.789508+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:28:07.972583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.666639+0000","last_change":"2024-04-26T01:49:37.652093+0000","last_active":"2024-04-26T01:53:36.666639+0000","last_peered":"2024-04-26T01:53:36.666639+0000","last_clean":"2024-04-26T01:53:36.666639+0000","last_became_active":"2024-04-26T01:49:37.651506+0000","last_became_peered":"2024-04-26T01:49:37.651506+0000","last_unstale":"2024-04-26T01:53:36.666639+0000","last_undegraded":"2024-04-26T01:53:36.666639+0000","last_fullsized":"2024-04-26T01:53:36.666639+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:01:31.421498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"93'9","reported_seq":270,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:46.128091+0000","last_change":"2024-04-26T01:49:33.587652+0000","last_active":"2024-04-26T01:53:46.128091+0000","last_peered":"2024-04-26T01:53:46.128091+0000","last_clean":"2024-04-26T01:53:46.128091+0000","last_became_active":"2024-04-26T01:49:33.587294+0000","last_became_peered":"2024-04-26T01:49:33.587294+0000","last_unstale":"2024-04-26T01:53:46.128091+0000","last_undegraded":"2024-04-26T01:53:46.128091+0000","last_fullsized":"2024-04-26T01:53:46.128091+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:32:46.803415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1855,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":232,"num_read_kb":6,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.789135+0000","last_change":"2024-04-26T01:49:37.647899+0000","last_active":"2024-04-26T01:53:36.789135+0000","last_peered":"2024-04-26T01:53:36.789135+0000","last_clean":"2024-04-26T01:53:36.789135+0000","last_became_active":"2024-04-26T01:49:37.647420+0000","last_became_peered":"2024-04-26T01:49:37.647420+0000","last_unstale":"2024-04-26T01:53:36.789135+0000","last_undegraded":"2024-04-26T01:53:36.789135+0000","last_fullsized":"2024-04-26T01:53:36.789135+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:16:39.262842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.799464+0000","last_change":"2024-04-26T01:49:33.596096+0000","last_active":"2024-04-26T01:53:36.799464+0000","last_peered":"2024-04-26T01:53:36.799464+0000","last_clean":"2024-04-26T01:53:36.799464+0000","last_became_active":"2024-04-26T01:49:33.595604+0000","last_became_peered":"2024-04-26T01:49:33.595604+0000","last_unstale":"2024-04-26T01:53:36.799464+0000","last_undegraded":"2024-04-26T01:53:36.799464+0000","last_fullsized":"2024-04-26T01:53:36.799464+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:44:26.497448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.666518+0000","last_change":"2024-04-26T01:49:37.665001+0000","last_active":"2024-04-26T01:53:36.666518+0000","last_peered":"2024-04-26T01:53:36.666518+0000","last_clean":"2024-04-26T01:53:36.666518+0000","last_became_active":"2024-04-26T01:49:37.664598+0000","last_became_peered":"2024-04-26T01:49:37.664598+0000","last_unstale":"2024-04-26T01:53:36.666518+0000","last_undegraded":"2024-04-26T01:53:36.666518+0000","last_fullsized":"2024-04-26T01:53:36.666518+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:54:28.299606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.162316+0000","last_change":"2024-04-26T01:49:33.602593+0000","last_active":"2024-04-26T01:53:37.162316+0000","last_peered":"2024-04-26T01:53:37.162316+0000","last_clean":"2024-04-26T01:53:37.162316+0000","last_became_active":"2024-04-26T01:49:33.602223+0000","last_became_peered":"2024-04-26T01:49:33.602223+0000","last_unstale":"2024-04-26T01:53:37.162316+0000","last_undegraded":"2024-04-26T01:53:37.162316+0000","last_fullsized":"2024-04-26T01:53:37.162316+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:43:29.404719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.789481+0000","last_change":"2024-04-26T01:49:37.662351+0000","last_active":"2024-04-26T01:53:36.789481+0000","last_peered":"2024-04-26T01:53:36.789481+0000","last_clean":"2024-04-26T01:53:36.789481+0000","last_became_active":"2024-04-26T01:49:37.661900+0000","last_became_peered":"2024-04-26T01:49:37.661900+0000","last_unstale":"2024-04-26T01:53:36.789481+0000","last_undegraded":"2024-04-26T01:53:36.789481+0000","last_fullsized":"2024-04-26T01:53:36.789481+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:49:10.878580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.666086+0000","last_change":"2024-04-26T01:49:33.586538+0000","last_active":"2024-04-26T01:53:36.666086+0000","last_peered":"2024-04-26T01:53:36.666086+0000","last_clean":"2024-04-26T01:53:36.666086+0000","last_became_active":"2024-04-26T01:49:33.586127+0000","last_became_peered":"2024-04-26T01:49:33.586127+0000","last_unstale":"2024-04-26T01:53:36.666086+0000","last_undegraded":"2024-04-26T01:53:36.666086+0000","last_fullsized":"2024-04-26T01:53:36.666086+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:05.599479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.665204+0000","last_change":"2024-04-26T01:49:37.657154+0000","last_active":"2024-04-26T01:53:36.665204+0000","last_peered":"2024-04-26T01:53:36.665204+0000","last_clean":"2024-04-26T01:53:36.665204+0000","last_became_active":"2024-04-26T01:49:37.656814+0000","last_became_peered":"2024-04-26T01:49:37.656814+0000","last_unstale":"2024-04-26T01:53:36.665204+0000","last_undegraded":"2024-04-26T01:53:36.665204+0000","last_fullsized":"2024-04-26T01:53:36.665204+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:26:54.538267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"92'5","reported_seq":260,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:46.127221+0000","last_change":"2024-04-26T01:49:33.602441+0000","last_active":"2024-04-26T01:53:46.127221+0000","last_peered":"2024-04-26T01:53:46.127221+0000","last_clean":"2024-04-26T01:53:46.127221+0000","last_became_active":"2024-04-26T01:49:33.602116+0000","last_became_peered":"2024-04-26T01:49:33.602116+0000","last_unstale":"2024-04-26T01:53:46.127221+0000","last_undegraded":"2024-04-26T01:53:46.127221+0000","last_fullsized":"2024-04-26T01:53:46.127221+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:45:52.541043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":226,"num_read_kb":226,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.982335+0000","last_change":"2024-04-26T01:49:37.645921+0000","last_active":"2024-04-26T01:53:36.982335+0000","last_peered":"2024-04-26T01:53:36.982335+0000","last_clean":"2024-04-26T01:53:36.982335+0000","last_became_active":"2024-04-26T01:49:37.645590+0000","last_became_peered":"2024-04-26T01:49:37.645590+0000","last_unstale":"2024-04-26T01:53:36.982335+0000","last_undegraded":"2024-04-26T01:53:36.982335+0000","last_fullsized":"2024-04-26T01:53:36.982335+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:00:08.931051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.985630+0000","last_change":"2024-04-26T01:49:33.589350+0000","last_active":"2024-04-26T01:53:36.985630+0000","last_peered":"2024-04-26T01:53:36.985630+0000","last_clean":"2024-04-26T01:53:36.985630+0000","last_became_active":"2024-04-26T01:49:33.588943+0000","last_became_peered":"2024-04-26T01:49:33.588943+0000","last_unstale":"2024-04-26T01:53:36.985630+0000","last_undegraded":"2024-04-26T01:53:36.985630+0000","last_fullsized":"2024-04-26T01:53:36.985630+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:36:36.104012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.161647+0000","last_change":"2024-04-26T01:49:37.648458+0000","last_active":"2024-04-26T01:53:37.161647+0000","last_peered":"2024-04-26T01:53:37.161647+0000","last_clean":"2024-04-26T01:53:37.161647+0000","last_became_active":"2024-04-26T01:49:37.648037+0000","last_became_peered":"2024-04-26T01:49:37.648037+0000","last_unstale":"2024-04-26T01:53:37.161647+0000","last_undegraded":"2024-04-26T01:53:37.161647+0000","last_fullsized":"2024-04-26T01:53:37.161647+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:28:05.711795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.666743+0000","last_change":"2024-04-26T01:49:34.189193+0000","last_active":"2024-04-26T01:53:36.666743+0000","last_peered":"2024-04-26T01:53:36.666743+0000","last_clean":"2024-04-26T01:53:36.666743+0000","last_became_active":"2024-04-26T01:49:34.188827+0000","last_became_peered":"2024-04-26T01:49:34.188827+0000","last_unstale":"2024-04-26T01:53:36.666743+0000","last_undegraded":"2024-04-26T01:53:36.666743+0000","last_fullsized":"2024-04-26T01:53:36.666743+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:02:04.583215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.981825+0000","last_change":"2024-04-26T01:49:37.644208+0000","last_active":"2024-04-26T01:53:36.981825+0000","last_peered":"2024-04-26T01:53:36.981825+0000","last_clean":"2024-04-26T01:53:36.981825+0000","last_became_active":"2024-04-26T01:49:37.643787+0000","last_became_peered":"2024-04-26T01:49:37.643787+0000","last_unstale":"2024-04-26T01:53:36.981825+0000","last_undegraded":"2024-04-26T01:53:36.981825+0000","last_fullsized":"2024-04-26T01:53:36.981825+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:42:21.243628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"93'37","reported_seq":532,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:47.128863+0000","last_change":"2024-04-26T01:49:33.590965+0000","last_active":"2024-04-26T01:53:47.128863+0000","last_peered":"2024-04-26T01:53:47.128863+0000","last_clean":"2024-04-26T01:53:47.128863+0000","last_became_active":"2024-04-26T01:49:33.590543+0000","last_became_peered":"2024-04-26T01:49:33.590543+0000","last_unstale":"2024-04-26T01:53:47.128863+0000","last_undegraded":"2024-04-26T01:53:47.128863+0000","last_fullsized":"2024-04-26T01:53:47.128863+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:26.627549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69972,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":466,"num_read_kb":302,"num_write":38,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.789460+0000","last_change":"2024-04-26T01:49:37.641172+0000","last_active":"2024-04-26T01:53:36.789460+0000","last_peered":"2024-04-26T01:53:36.789460+0000","last_clean":"2024-04-26T01:53:36.789460+0000","last_became_active":"2024-04-26T01:49:37.640866+0000","last_became_peered":"2024-04-26T01:49:37.640866+0000","last_unstale":"2024-04-26T01:53:36.789460+0000","last_undegraded":"2024-04-26T01:53:36.789460+0000","last_fullsized":"2024-04-26T01:53:36.789460+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:29:12.461005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.981907+0000","last_change":"2024-04-26T01:49:33.598466+0000","last_active":"2024-04-26T01:53:36.981907+0000","last_peered":"2024-04-26T01:53:36.981907+0000","last_clean":"2024-04-26T01:53:36.981907+0000","last_became_active":"2024-04-26T01:49:33.598042+0000","last_became_peered":"2024-04-26T01:49:33.598042+0000","last_unstale":"2024-04-26T01:53:36.981907+0000","last_undegraded":"2024-04-26T01:53:36.981907+0000","last_fullsized":"2024-04-26T01:53:36.981907+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:55:54.901016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.162529+0000","last_change":"2024-04-26T01:49:37.665772+0000","last_active":"2024-04-26T01:53:37.162529+0000","last_peered":"2024-04-26T01:53:37.162529+0000","last_clean":"2024-04-26T01:53:37.162529+0000","last_became_active":"2024-04-26T01:49:37.665524+0000","last_became_peered":"2024-04-26T01:49:37.665524+0000","last_unstale":"2024-04-26T01:53:37.162529+0000","last_undegraded":"2024-04-26T01:53:37.162529+0000","last_fullsized":"2024-04-26T01:53:37.162529+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:00:19.729098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.666223+0000","last_change":"2024-04-26T01:49:33.599456+0000","last_active":"2024-04-26T01:53:36.666223+0000","last_peered":"2024-04-26T01:53:36.666223+0000","last_clean":"2024-04-26T01:53:36.666223+0000","last_became_active":"2024-04-26T01:49:33.599101+0000","last_became_peered":"2024-04-26T01:49:33.599101+0000","last_unstale":"2024-04-26T01:53:36.666223+0000","last_undegraded":"2024-04-26T01:53:36.666223+0000","last_fullsized":"2024-04-26T01:53:36.666223+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:14:35.923011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.985701+0000","last_change":"2024-04-26T01:49:37.647193+0000","last_active":"2024-04-26T01:53:36.985701+0000","last_peered":"2024-04-26T01:53:36.985701+0000","last_clean":"2024-04-26T01:53:36.985701+0000","last_became_active":"2024-04-26T01:49:37.646881+0000","last_became_peered":"2024-04-26T01:49:37.646881+0000","last_unstale":"2024-04-26T01:53:36.985701+0000","last_undegraded":"2024-04-26T01:53:36.985701+0000","last_fullsized":"2024-04-26T01:53:36.985701+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:50:10.006986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"92'5","reported_seq":264,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:46.270706+0000","last_change":"2024-04-26T01:49:33.590398+0000","last_active":"2024-04-26T01:53:46.270706+0000","last_peered":"2024-04-26T01:53:46.270706+0000","last_clean":"2024-04-26T01:53:46.270706+0000","last_became_active":"2024-04-26T01:49:33.590033+0000","last_became_peered":"2024-04-26T01:49:33.590033+0000","last_unstale":"2024-04-26T01:53:46.270706+0000","last_undegraded":"2024-04-26T01:53:46.270706+0000","last_fullsized":"2024-04-26T01:53:46.270706+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:27:43.203781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":231,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.985282+0000","last_change":"2024-04-26T01:49:37.649784+0000","last_active":"2024-04-26T01:53:36.985282+0000","last_peered":"2024-04-26T01:53:36.985282+0000","last_clean":"2024-04-26T01:53:36.985282+0000","last_became_active":"2024-04-26T01:49:37.649431+0000","last_became_peered":"2024-04-26T01:49:37.649431+0000","last_unstale":"2024-04-26T01:53:36.985282+0000","last_undegraded":"2024-04-26T01:53:36.985282+0000","last_fullsized":"2024-04-26T01:53:36.985282+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:45:19.045983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.789114+0000","last_change":"2024-04-26T01:49:34.182987+0000","last_active":"2024-04-26T01:53:36.789114+0000","last_peered":"2024-04-26T01:53:36.789114+0000","last_clean":"2024-04-26T01:53:36.789114+0000","last_became_active":"2024-04-26T01:49:34.182600+0000","last_became_peered":"2024-04-26T01:49:34.182600+0000","last_unstale":"2024-04-26T01:53:36.789114+0000","last_undegraded":"2024-04-26T01:53:36.789114+0000","last_fullsized":"2024-04-26T01:53:36.789114+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:05:52.774170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.666131+0000","last_change":"2024-04-26T01:49:37.646097+0000","last_active":"2024-04-26T01:53:36.666131+0000","last_peered":"2024-04-26T01:53:36.666131+0000","last_clean":"2024-04-26T01:53:36.666131+0000","last_became_active":"2024-04-26T01:49:37.645763+0000","last_became_peered":"2024-04-26T01:49:37.645763+0000","last_unstale":"2024-04-26T01:53:36.666131+0000","last_undegraded":"2024-04-26T01:53:36.666131+0000","last_fullsized":"2024-04-26T01:53:36.666131+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:42:37.159950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.789434+0000","last_change":"2024-04-26T01:49:33.591457+0000","last_active":"2024-04-26T01:53:36.789434+0000","last_peered":"2024-04-26T01:53:36.789434+0000","last_clean":"2024-04-26T01:53:36.789434+0000","last_became_active":"2024-04-26T01:49:33.590655+0000","last_became_peered":"2024-04-26T01:49:33.590655+0000","last_unstale":"2024-04-26T01:53:36.789434+0000","last_undegraded":"2024-04-26T01:53:36.789434+0000","last_fullsized":"2024-04-26T01:53:36.789434+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:33:18.803796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.159392+0000","last_change":"2024-04-26T01:49:37.646423+0000","last_active":"2024-04-26T01:53:37.159392+0000","last_peered":"2024-04-26T01:53:37.159392+0000","last_clean":"2024-04-26T01:53:37.159392+0000","last_became_active":"2024-04-26T01:49:37.646046+0000","last_became_peered":"2024-04-26T01:49:37.646046+0000","last_unstale":"2024-04-26T01:53:37.159392+0000","last_undegraded":"2024-04-26T01:53:37.159392+0000","last_fullsized":"2024-04-26T01:53:37.159392+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:26:03.570479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.665901+0000","last_change":"2024-04-26T01:49:34.192848+0000","last_active":"2024-04-26T01:53:36.665901+0000","last_peered":"2024-04-26T01:53:36.665901+0000","last_clean":"2024-04-26T01:53:36.665901+0000","last_became_active":"2024-04-26T01:49:34.190449+0000","last_became_peered":"2024-04-26T01:49:34.190449+0000","last_unstale":"2024-04-26T01:53:36.665901+0000","last_undegraded":"2024-04-26T01:53:36.665901+0000","last_fullsized":"2024-04-26T01:53:36.665901+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:05:22.094991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'5","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.799444+0000","last_change":"2024-04-26T01:49:37.667147+0000","last_active":"2024-04-26T01:53:36.799444+0000","last_peered":"2024-04-26T01:53:36.799444+0000","last_clean":"2024-04-26T01:53:36.799444+0000","last_became_active":"2024-04-26T01:49:37.666752+0000","last_became_peered":"2024-04-26T01:49:37.666752+0000","last_unstale":"2024-04-26T01:53:36.799444+0000","last_undegraded":"2024-04-26T01:53:36.799444+0000","last_fullsized":"2024-04-26T01:53:36.799444+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:18:31.778916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.985505+0000","last_change":"2024-04-26T01:49:33.596155+0000","last_active":"2024-04-26T01:53:36.985505+0000","last_peered":"2024-04-26T01:53:36.985505+0000","last_clean":"2024-04-26T01:53:36.985505+0000","last_became_active":"2024-04-26T01:49:33.595760+0000","last_became_peered":"2024-04-26T01:49:33.595760+0000","last_unstale":"2024-04-26T01:53:36.985505+0000","last_undegraded":"2024-04-26T01:53:36.985505+0000","last_fullsized":"2024-04-26T01:53:36.985505+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:07:57.208116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.799446+0000","last_change":"2024-04-26T01:49:37.641660+0000","last_active":"2024-04-26T01:53:36.799446+0000","last_peered":"2024-04-26T01:53:36.799446+0000","last_clean":"2024-04-26T01:53:36.799446+0000","last_became_active":"2024-04-26T01:49:37.641302+0000","last_became_peered":"2024-04-26T01:49:37.641302+0000","last_unstale":"2024-04-26T01:53:36.799446+0000","last_undegraded":"2024-04-26T01:53:36.799446+0000","last_fullsized":"2024-04-26T01:53:36.799446+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:32:59.889189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.666400+0000","last_change":"2024-04-26T01:49:37.643083+0000","last_active":"2024-04-26T01:53:36.666400+0000","last_peered":"2024-04-26T01:53:36.666400+0000","last_clean":"2024-04-26T01:53:36.666400+0000","last_became_active":"2024-04-26T01:49:37.642548+0000","last_became_peered":"2024-04-26T01:49:37.642548+0000","last_unstale":"2024-04-26T01:53:36.666400+0000","last_undegraded":"2024-04-26T01:53:36.666400+0000","last_fullsized":"2024-04-26T01:53:36.666400+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:09:06.136510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'5","reported_seq":261,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:46.270233+0000","last_change":"2024-04-26T01:49:34.185069+0000","last_active":"2024-04-26T01:53:46.270233+0000","last_peered":"2024-04-26T01:53:46.270233+0000","last_clean":"2024-04-26T01:53:46.270233+0000","last_became_active":"2024-04-26T01:49:34.183989+0000","last_became_peered":"2024-04-26T01:49:34.183989+0000","last_unstale":"2024-04-26T01:53:46.270233+0000","last_undegraded":"2024-04-26T01:53:46.270233+0000","last_fullsized":"2024-04-26T01:53:46.270233+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:04:26.327960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":226,"num_read_kb":226,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.791392+0000","last_change":"2024-04-26T01:49:37.655512+0000","last_active":"2024-04-26T01:53:36.791392+0000","last_peered":"2024-04-26T01:53:36.791392+0000","last_clean":"2024-04-26T01:53:36.791392+0000","last_became_active":"2024-04-26T01:49:37.655142+0000","last_became_peered":"2024-04-26T01:49:37.655142+0000","last_unstale":"2024-04-26T01:53:36.791392+0000","last_undegraded":"2024-04-26T01:53:36.791392+0000","last_fullsized":"2024-04-26T01:53:36.791392+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:02:58.090885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.668849+0000","last_change":"2024-04-26T01:49:33.599578+0000","last_active":"2024-04-26T01:53:36.668849+0000","last_peered":"2024-04-26T01:53:36.668849+0000","last_clean":"2024-04-26T01:53:36.668849+0000","last_became_active":"2024-04-26T01:49:33.599010+0000","last_became_peered":"2024-04-26T01:49:33.599010+0000","last_unstale":"2024-04-26T01:53:36.668849+0000","last_undegraded":"2024-04-26T01:53:36.668849+0000","last_fullsized":"2024-04-26T01:53:36.668849+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:16:09.925304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"93'148","reported_seq":407,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:38.805713+0000","last_change":"2024-04-26T01:47:59.584819+0000","last_active":"2024-04-26T01:53:38.805713+0000","last_peered":"2024-04-26T01:53:38.805713+0000","last_clean":"2024-04-26T01:53:38.805713+0000","last_became_active":"2024-04-26T01:47:59.584356+0000","last_became_peered":"2024-04-26T01:47:59.584356+0000","last_unstale":"2024-04-26T01:53:38.805713+0000","last_undegraded":"2024-04-26T01:53:38.805713+0000","last_fullsized":"2024-04-26T01:53:38.805713+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:44:11.653568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:44:11.653568+0000","last_clean_scrub_stamp":"2024-04-26T01:44:11.653568+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:25:11.168583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.161948+0000","last_change":"2024-04-26T01:50:27.008911+0000","last_active":"2024-04-26T01:53:37.161948+0000","last_peered":"2024-04-26T01:53:37.161948+0000","last_clean":"2024-04-26T01:53:37.161948+0000","last_became_active":"2024-04-26T01:50:27.008507+0000","last_became_peered":"2024-04-26T01:50:27.008507+0000","last_unstale":"2024-04-26T01:53:37.161948+0000","last_undegraded":"2024-04-26T01:53:37.161948+0000","last_fullsized":"2024-04-26T01:53:37.161948+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:21:14.985935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.985710+0000","last_change":"2024-04-26T01:49:33.590171+0000","last_active":"2024-04-26T01:53:36.985710+0000","last_peered":"2024-04-26T01:53:36.985710+0000","last_clean":"2024-04-26T01:53:36.985710+0000","last_became_active":"2024-04-26T01:49:33.589312+0000","last_became_peered":"2024-04-26T01:49:33.589312+0000","last_unstale":"2024-04-26T01:53:36.985710+0000","last_undegraded":"2024-04-26T01:53:36.985710+0000","last_fullsized":"2024-04-26T01:53:36.985710+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:01:25.441527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.789552+0000","last_change":"2024-04-26T01:49:34.190338+0000","last_active":"2024-04-26T01:53:36.789552+0000","last_peered":"2024-04-26T01:53:36.789552+0000","last_clean":"2024-04-26T01:53:36.789552+0000","last_became_active":"2024-04-26T01:49:34.189984+0000","last_became_peered":"2024-04-26T01:49:34.189984+0000","last_unstale":"2024-04-26T01:53:36.789552+0000","last_undegraded":"2024-04-26T01:53:36.789552+0000","last_fullsized":"2024-04-26T01:53:36.789552+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:58.588012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.163165+0000","last_change":"2024-04-26T01:49:37.649368+0000","last_active":"2024-04-26T01:53:37.163165+0000","last_peered":"2024-04-26T01:53:37.163165+0000","last_clean":"2024-04-26T01:53:37.163165+0000","last_became_active":"2024-04-26T01:49:37.648987+0000","last_became_peered":"2024-04-26T01:49:37.648987+0000","last_unstale":"2024-04-26T01:53:37.163165+0000","last_undegraded":"2024-04-26T01:53:37.163165+0000","last_fullsized":"2024-04-26T01:53:37.163165+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:04:59.080703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"93'102","reported_seq":223,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:39.813697+0000","last_change":"2024-04-26T01:49:33.600813+0000","last_active":"2024-04-26T01:53:39.813697+0000","last_peered":"2024-04-26T01:53:39.813697+0000","last_clean":"2024-04-26T01:53:39.813697+0000","last_became_active":"2024-04-26T01:49:33.600361+0000","last_became_peered":"2024-04-26T01:49:33.600361+0000","last_unstale":"2024-04-26T01:53:39.813697+0000","last_undegraded":"2024-04-26T01:53:39.813697+0000","last_fullsized":"2024-04-26T01:53:39.813697+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:02:58.638449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":188,"num_write_kb":1288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.791866+0000","last_change":"2024-04-26T01:49:37.659381+0000","last_active":"2024-04-26T01:53:36.791866+0000","last_peered":"2024-04-26T01:53:36.791866+0000","last_clean":"2024-04-26T01:53:36.791866+0000","last_became_active":"2024-04-26T01:49:37.656205+0000","last_became_peered":"2024-04-26T01:49:37.656205+0000","last_unstale":"2024-04-26T01:53:36.791866+0000","last_undegraded":"2024-04-26T01:53:36.791866+0000","last_fullsized":"2024-04-26T01:53:36.791866+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:02:10.103330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.668898+0000","last_change":"2024-04-26T01:49:33.854732+0000","last_active":"2024-04-26T01:53:36.668898+0000","last_peered":"2024-04-26T01:53:36.668898+0000","last_clean":"2024-04-26T01:53:36.668898+0000","last_became_active":"2024-04-26T01:49:33.854396+0000","last_became_peered":"2024-04-26T01:49:33.854396+0000","last_unstale":"2024-04-26T01:53:36.668898+0000","last_undegraded":"2024-04-26T01:53:36.668898+0000","last_fullsized":"2024-04-26T01:53:36.668898+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:12:37.183725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.161320+0000","last_change":"2024-04-26T01:49:37.656366+0000","last_active":"2024-04-26T01:53:37.161320+0000","last_peered":"2024-04-26T01:53:37.161320+0000","last_clean":"2024-04-26T01:53:37.161320+0000","last_became_active":"2024-04-26T01:49:37.655869+0000","last_became_peered":"2024-04-26T01:49:37.655869+0000","last_unstale":"2024-04-26T01:53:37.161320+0000","last_undegraded":"2024-04-26T01:53:37.161320+0000","last_fullsized":"2024-04-26T01:53:37.161320+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:01:57.083315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.981654+0000","last_change":"2024-04-26T01:49:34.186070+0000","last_active":"2024-04-26T01:53:36.981654+0000","last_peered":"2024-04-26T01:53:36.981654+0000","last_clean":"2024-04-26T01:53:36.981654+0000","last_became_active":"2024-04-26T01:49:34.185742+0000","last_became_peered":"2024-04-26T01:49:34.185742+0000","last_unstale":"2024-04-26T01:53:36.981654+0000","last_undegraded":"2024-04-26T01:53:36.981654+0000","last_fullsized":"2024-04-26T01:53:36.981654+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:05.870056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.789114+0000","last_change":"2024-04-26T01:49:37.640938+0000","last_active":"2024-04-26T01:53:36.789114+0000","last_peered":"2024-04-26T01:53:36.789114+0000","last_clean":"2024-04-26T01:53:36.789114+0000","last_became_active":"2024-04-26T01:49:37.640608+0000","last_became_peered":"2024-04-26T01:49:37.640608+0000","last_unstale":"2024-04-26T01:53:36.789114+0000","last_undegraded":"2024-04-26T01:53:36.789114+0000","last_fullsized":"2024-04-26T01:53:36.789114+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:30:30.460641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.791217+0000","last_change":"2024-04-26T01:49:33.604152+0000","last_active":"2024-04-26T01:53:36.791217+0000","last_peered":"2024-04-26T01:53:36.791217+0000","last_clean":"2024-04-26T01:53:36.791217+0000","last_became_active":"2024-04-26T01:49:33.603479+0000","last_became_peered":"2024-04-26T01:49:33.603479+0000","last_unstale":"2024-04-26T01:53:36.791217+0000","last_undegraded":"2024-04-26T01:53:36.791217+0000","last_fullsized":"2024-04-26T01:53:36.791217+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:25:13.685392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.159096+0000","last_change":"2024-04-26T01:49:37.645152+0000","last_active":"2024-04-26T01:53:37.159096+0000","last_peered":"2024-04-26T01:53:37.159096+0000","last_clean":"2024-04-26T01:53:37.159096+0000","last_became_active":"2024-04-26T01:49:37.644789+0000","last_became_peered":"2024-04-26T01:49:37.644789+0000","last_unstale":"2024-04-26T01:53:37.159096+0000","last_undegraded":"2024-04-26T01:53:37.159096+0000","last_fullsized":"2024-04-26T01:53:37.159096+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:20:01.494222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"93'2","reported_seq":19,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.159226+0000","last_change":"2024-04-26T01:50:27.009973+0000","last_active":"2024-04-26T01:53:37.159226+0000","last_peered":"2024-04-26T01:53:37.159226+0000","last_clean":"2024-04-26T01:53:37.159226+0000","last_became_active":"2024-04-26T01:50:27.001678+0000","last_became_peered":"2024-04-26T01:50:27.001678+0000","last_unstale":"2024-04-26T01:53:37.159226+0000","last_undegraded":"2024-04-26T01:53:37.159226+0000","last_fullsized":"2024-04-26T01:53:37.159226+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:59:54.807792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.791338+0000","last_change":"2024-04-26T01:49:37.662665+0000","last_active":"2024-04-26T01:53:36.791338+0000","last_peered":"2024-04-26T01:53:36.791338+0000","last_clean":"2024-04-26T01:53:36.791338+0000","last_became_active":"2024-04-26T01:49:37.662216+0000","last_became_peered":"2024-04-26T01:49:37.662216+0000","last_unstale":"2024-04-26T01:53:36.791338+0000","last_undegraded":"2024-04-26T01:53:36.791338+0000","last_fullsized":"2024-04-26T01:53:36.791338+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:10:54.724858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.789242+0000","last_change":"2024-04-26T01:49:33.592078+0000","last_active":"2024-04-26T01:53:36.789242+0000","last_peered":"2024-04-26T01:53:36.789242+0000","last_clean":"2024-04-26T01:53:36.789242+0000","last_became_active":"2024-04-26T01:49:33.590265+0000","last_became_peered":"2024-04-26T01:49:33.590265+0000","last_unstale":"2024-04-26T01:53:36.789242+0000","last_undegraded":"2024-04-26T01:53:36.789242+0000","last_fullsized":"2024-04-26T01:53:36.789242+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:44:27.163705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.985525+0000","last_change":"2024-04-26T01:49:37.645943+0000","last_active":"2024-04-26T01:53:36.985525+0000","last_peered":"2024-04-26T01:53:36.985525+0000","last_clean":"2024-04-26T01:53:36.985525+0000","last_became_active":"2024-04-26T01:49:37.645586+0000","last_became_peered":"2024-04-26T01:49:37.645586+0000","last_unstale":"2024-04-26T01:53:36.985525+0000","last_undegraded":"2024-04-26T01:53:36.985525+0000","last_fullsized":"2024-04-26T01:53:36.985525+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:40.682344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.982081+0000","last_change":"2024-04-26T01:49:34.187115+0000","last_active":"2024-04-26T01:53:36.982081+0000","last_peered":"2024-04-26T01:53:36.982081+0000","last_clean":"2024-04-26T01:53:36.982081+0000","last_became_active":"2024-04-26T01:49:34.186660+0000","last_became_peered":"2024-04-26T01:49:34.186660+0000","last_unstale":"2024-04-26T01:53:36.982081+0000","last_undegraded":"2024-04-26T01:53:36.982081+0000","last_fullsized":"2024-04-26T01:53:36.982081+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:56:32.274752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.985756+0000","last_change":"2024-04-26T01:49:37.650382+0000","last_active":"2024-04-26T01:53:36.985756+0000","last_peered":"2024-04-26T01:53:36.985756+0000","last_clean":"2024-04-26T01:53:36.985756+0000","last_became_active":"2024-04-26T01:49:37.649545+0000","last_became_peered":"2024-04-26T01:49:37.649545+0000","last_unstale":"2024-04-26T01:53:36.985756+0000","last_undegraded":"2024-04-26T01:53:36.985756+0000","last_fullsized":"2024-04-26T01:53:36.985756+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:43:16.291311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"93'65","reported_seq":125,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:39.347187+0000","last_change":"2024-04-26T01:49:33.595881+0000","last_active":"2024-04-26T01:53:39.347187+0000","last_peered":"2024-04-26T01:53:39.347187+0000","last_clean":"2024-04-26T01:53:39.347187+0000","last_became_active":"2024-04-26T01:49:33.595480+0000","last_became_peered":"2024-04-26T01:49:33.595480+0000","last_unstale":"2024-04-26T01:53:39.347187+0000","last_undegraded":"2024-04-26T01:53:39.347187+0000","last_fullsized":"2024-04-26T01:53:39.347187+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:30:58.028230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.985506+0000","last_change":"2024-04-26T01:49:37.660765+0000","last_active":"2024-04-26T01:53:36.985506+0000","last_peered":"2024-04-26T01:53:36.985506+0000","last_clean":"2024-04-26T01:53:36.985506+0000","last_became_active":"2024-04-26T01:49:37.660475+0000","last_became_peered":"2024-04-26T01:49:37.660475+0000","last_unstale":"2024-04-26T01:53:36.985506+0000","last_undegraded":"2024-04-26T01:53:36.985506+0000","last_fullsized":"2024-04-26T01:53:36.985506+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:36:00.483005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"90'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.162985+0000","last_change":"2024-04-26T01:49:33.598247+0000","last_active":"2024-04-26T01:53:37.162985+0000","last_peered":"2024-04-26T01:53:37.162985+0000","last_clean":"2024-04-26T01:53:37.162985+0000","last_became_active":"2024-04-26T01:49:33.597796+0000","last_became_peered":"2024-04-26T01:49:33.597796+0000","last_unstale":"2024-04-26T01:53:37.162985+0000","last_undegraded":"2024-04-26T01:53:37.162985+0000","last_fullsized":"2024-04-26T01:53:37.162985+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:32.566518+0000","last_clean_scrub_stamp":"2024-04-26T01:49:32.566518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:53:38.994798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.985835+0000","last_change":"2024-04-26T01:49:37.648903+0000","last_active":"2024-04-26T01:53:36.985835+0000","last_peered":"2024-04-26T01:53:36.985835+0000","last_clean":"2024-04-26T01:53:36.985835+0000","last_became_active":"2024-04-26T01:49:37.648570+0000","last_became_peered":"2024-04-26T01:49:37.648570+0000","last_unstale":"2024-04-26T01:53:36.985835+0000","last_undegraded":"2024-04-26T01:53:36.985835+0000","last_fullsized":"2024-04-26T01:53:36.985835+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:33:12.678513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.159220+0000","last_change":"2024-04-26T01:49:37.648485+0000","last_active":"2024-04-26T01:53:37.159220+0000","last_peered":"2024-04-26T01:53:37.159220+0000","last_clean":"2024-04-26T01:53:37.159220+0000","last_became_active":"2024-04-26T01:49:37.648129+0000","last_became_peered":"2024-04-26T01:49:37.648129+0000","last_unstale":"2024-04-26T01:53:37.159220+0000","last_undegraded":"2024-04-26T01:53:37.159220+0000","last_fullsized":"2024-04-26T01:53:37.159220+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:56:12.047821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.981838+0000","last_change":"2024-04-26T01:49:37.645929+0000","last_active":"2024-04-26T01:53:36.981838+0000","last_peered":"2024-04-26T01:53:36.981838+0000","last_clean":"2024-04-26T01:53:36.981838+0000","last_became_active":"2024-04-26T01:49:37.645473+0000","last_became_peered":"2024-04-26T01:49:37.645473+0000","last_unstale":"2024-04-26T01:53:36.981838+0000","last_undegraded":"2024-04-26T01:53:36.981838+0000","last_fullsized":"2024-04-26T01:53:36.981838+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:39:56.687973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.981790+0000","last_change":"2024-04-26T01:49:37.657494+0000","last_active":"2024-04-26T01:53:36.981790+0000","last_peered":"2024-04-26T01:53:36.981790+0000","last_clean":"2024-04-26T01:53:36.981790+0000","last_became_active":"2024-04-26T01:49:37.657121+0000","last_became_peered":"2024-04-26T01:49:37.657121+0000","last_unstale":"2024-04-26T01:53:36.981790+0000","last_undegraded":"2024-04-26T01:53:36.981790+0000","last_fullsized":"2024-04-26T01:53:36.981790+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:50:44.691179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.160773+0000","last_change":"2024-04-26T01:49:37.646012+0000","last_active":"2024-04-26T01:53:37.160773+0000","last_peered":"2024-04-26T01:53:37.160773+0000","last_clean":"2024-04-26T01:53:37.160773+0000","last_became_active":"2024-04-26T01:49:37.645695+0000","last_became_peered":"2024-04-26T01:49:37.645695+0000","last_unstale":"2024-04-26T01:53:37.160773+0000","last_undegraded":"2024-04-26T01:53:37.160773+0000","last_fullsized":"2024-04-26T01:53:37.160773+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:40:51.117523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.163306+0000","last_change":"2024-04-26T01:49:37.646503+0000","last_active":"2024-04-26T01:53:37.163306+0000","last_peered":"2024-04-26T01:53:37.163306+0000","last_clean":"2024-04-26T01:53:37.163306+0000","last_became_active":"2024-04-26T01:49:37.645200+0000","last_became_peered":"2024-04-26T01:49:37.645200+0000","last_unstale":"2024-04-26T01:53:37.163306+0000","last_undegraded":"2024-04-26T01:53:37.163306+0000","last_fullsized":"2024-04-26T01:53:37.163306+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:31:10.957199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.159402+0000","last_change":"2024-04-26T01:50:27.002908+0000","last_active":"2024-04-26T01:53:37.159402+0000","last_peered":"2024-04-26T01:53:37.159402+0000","last_clean":"2024-04-26T01:53:37.159402+0000","last_became_active":"2024-04-26T01:50:27.002454+0000","last_became_peered":"2024-04-26T01:50:27.002454+0000","last_unstale":"2024-04-26T01:53:37.159402+0000","last_undegraded":"2024-04-26T01:53:37.159402+0000","last_fullsized":"2024-04-26T01:53:37.159402+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:18:01.848427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.982171+0000","last_change":"2024-04-26T01:49:37.642676+0000","last_active":"2024-04-26T01:53:36.982171+0000","last_peered":"2024-04-26T01:53:36.982171+0000","last_clean":"2024-04-26T01:53:36.982171+0000","last_became_active":"2024-04-26T01:49:37.642302+0000","last_became_peered":"2024-04-26T01:49:37.642302+0000","last_unstale":"2024-04-26T01:53:36.982171+0000","last_undegraded":"2024-04-26T01:53:36.982171+0000","last_fullsized":"2024-04-26T01:53:36.982171+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:51:43.454649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.791673+0000","last_change":"2024-04-26T01:50:27.003914+0000","last_active":"2024-04-26T01:53:36.791673+0000","last_peered":"2024-04-26T01:53:36.791673+0000","last_clean":"2024-04-26T01:53:36.791673+0000","last_became_active":"2024-04-26T01:50:27.002344+0000","last_became_peered":"2024-04-26T01:50:27.002344+0000","last_unstale":"2024-04-26T01:53:36.791673+0000","last_undegraded":"2024-04-26T01:53:36.791673+0000","last_fullsized":"2024-04-26T01:53:36.791673+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:46:02.101508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.666138+0000","last_change":"2024-04-26T01:49:37.663306+0000","last_active":"2024-04-26T01:53:36.666138+0000","last_peered":"2024-04-26T01:53:36.666138+0000","last_clean":"2024-04-26T01:53:36.666138+0000","last_became_active":"2024-04-26T01:49:37.662999+0000","last_became_peered":"2024-04-26T01:49:37.662999+0000","last_unstale":"2024-04-26T01:53:36.666138+0000","last_undegraded":"2024-04-26T01:53:36.666138+0000","last_fullsized":"2024-04-26T01:53:36.666138+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:55.155096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.789468+0000","last_change":"2024-04-26T01:49:37.655157+0000","last_active":"2024-04-26T01:53:36.789468+0000","last_peered":"2024-04-26T01:53:36.789468+0000","last_clean":"2024-04-26T01:53:36.789468+0000","last_became_active":"2024-04-26T01:49:37.654810+0000","last_became_peered":"2024-04-26T01:49:37.654810+0000","last_unstale":"2024-04-26T01:53:36.789468+0000","last_undegraded":"2024-04-26T01:53:36.789468+0000","last_fullsized":"2024-04-26T01:53:36.789468+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:28:04.012382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:36.981829+0000","last_change":"2024-04-26T01:49:37.667228+0000","last_active":"2024-04-26T01:53:36.981829+0000","last_peered":"2024-04-26T01:53:36.981829+0000","last_clean":"2024-04-26T01:53:36.981829+0000","last_became_active":"2024-04-26T01:49:37.666835+0000","last_became_peered":"2024-04-26T01:49:37.666835+0000","last_unstale":"2024-04-26T01:53:36.981829+0000","last_undegraded":"2024-04-26T01:53:36.981829+0000","last_fullsized":"2024-04-26T01:53:36.981829+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:54:16.902836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.162824+0000","last_change":"2024-04-26T01:49:37.658231+0000","last_active":"2024-04-26T01:53:37.162824+0000","last_peered":"2024-04-26T01:53:37.162824+0000","last_clean":"2024-04-26T01:53:37.162824+0000","last_became_active":"2024-04-26T01:49:37.657842+0000","last_became_peered":"2024-04-26T01:49:37.657842+0000","last_unstale":"2024-04-26T01:53:37.162824+0000","last_undegraded":"2024-04-26T01:53:37.162824+0000","last_fullsized":"2024-04-26T01:53:37.162824+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:02:01.445262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:53:37.159180+0000","last_change":"2024-04-26T01:49:37.645221+0000","last_active":"2024-04-26T01:53:37.159180+0000","last_peered":"2024-04-26T01:53:37.159180+0000","last_clean":"2024-04-26T01:53:37.159180+0000","last_became_active":"2024-04-26T01:49:37.644814+0000","last_became_peered":"2024-04-26T01:49:37.644814+0000","last_unstale":"2024-04-26T01:53:37.159180+0000","last_undegraded":"2024-04-26T01:53:37.159180+0000","last_fullsized":"2024-04-26T01:53:37.159180+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:49:36.600606+0000","last_clean_scrub_stamp":"2024-04-26T01:49:36.600606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:03:17.850633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":796795,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1739,"num_read_kb":1087,"num_write":424,"num_write_kb":2358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2617344,"data_stored":2390385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":311,"ondisk_log_size":311,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318341,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1011712,"data_stored":495249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579975,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29136,"kb_used_data":2000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716016,"statfs":{"total":95995035648,"available":95965200384,"internally_reserved":0,"allocated":2048000,"data_stored":1485466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743501,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36724,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708428,"statfs":{"total":95995035648,"available":95957430272,"internally_reserved":0,"allocated":1421312,"data_stored":888701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005140,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28404,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716748,"statfs":{"total":95995035648,"available":95965949952,"internally_reserved":0,"allocated":1421312,"data_stored":888621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266776,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28600,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716552,"statfs":{"total":95995035648,"available":95965749248,"internally_reserved":0,"allocated":1622016,"data_stored":1088709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430303,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1015808,"data_stored":494874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574971,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36400,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708752,"statfs":{"total":95995035648,"available":95957762048,"internally_reserved":0,"allocated":1085440,"data_stored":564639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738495,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1036288,"data_stored":496145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411446,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36424,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708728,"statfs":{"total":95995035648,"available":95957737472,"internally_reserved":0,"allocated":1110016,"data_stored":567222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117103,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37172,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707980,"statfs":{"total":95995035648,"available":95956971520,"internally_reserved":0,"allocated":1757184,"data_stored":1216633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855467,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1019904,"data_stored":495843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691941,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29056,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716096,"statfs":{"total":95995035648,"available":95965282304,"internally_reserved":0,"allocated":1966080,"data_stored":1416093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":72555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":741376,"data_stored":721732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":360448,"data_stored":330941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":393954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":400314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T01:53:51.578 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:53:51.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:51 smithi151 ceph-mon[28057]: from='client.15045 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:53:51.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:51 smithi151 ceph-mon[28057]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-26T01:53:52.022 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:53:52.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:51 smithi172 ceph-mon[32785]: from='client.15045 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:53:52.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:51 smithi172 ceph-mon[32785]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-26T01:53:52.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:51 smithi165 ceph-mon[32648]: from='client.15045 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:53:52.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:51 smithi165 ceph-mon[32648]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-26T01:53:53.163 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.165+0000 7f9542200640 1 -- 172.21.15.151:0/2397527300 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f953c104d50 msgr2=0x7f953c108d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:53.164 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.165+0000 7f9542200640 1 --2- 172.21.15.151:0/2397527300 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f953c104d50 0x7f953c108d90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9524009900 tx=0x7f952402a410 comp rx=0 tx=0).stop 2024-04-26T01:53:53.164 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.165+0000 7f9542200640 1 -- 172.21.15.151:0/2397527300 shutdown_connections 2024-04-26T01:53:53.164 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.165+0000 7f9542200640 1 --2- 172.21.15.151:0/2397527300 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f953c109470 0x7f953c111040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:53.164 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.165+0000 7f9542200640 1 --2- 172.21.15.151:0/2397527300 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f953c104d50 0x7f953c108d90 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:53.165 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.165+0000 7f9542200640 1 --2- 172.21.15.151:0/2397527300 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f953c1043a0 0x7f953c104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:53.165 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.166+0000 7f9542200640 1 -- 172.21.15.151:0/2397527300 >> 172.21.15.151:0/2397527300 conn(0x7f953c0ffc40 msgr2=0x7f953c102060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:53.165 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.166+0000 7f9542200640 1 -- 172.21.15.151:0/2397527300 shutdown_connections 2024-04-26T01:53:53.165 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.166+0000 7f9542200640 1 -- 172.21.15.151:0/2397527300 wait complete. 2024-04-26T01:53:53.166 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.167+0000 7f9542200640 1 Processor -- start 2024-04-26T01:53:53.167 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.167+0000 7f9542200640 1 -- start start 2024-04-26T01:53:53.167 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.167+0000 7f9542200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f953c1043a0 0x7f953c1a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:53.167 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.167+0000 7f9542200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f953c109470 0x7f953c1a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:53.167 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.167+0000 7f9542200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f953c19a720 0x7f953c19aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:53.167 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.167+0000 7f9542200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f953c114400 con 0x7f953c1043a0 2024-04-26T01:53:53.167 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.167+0000 7f9542200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f953c114280 con 0x7f953c109470 2024-04-26T01:53:53.167 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.167+0000 7f9542200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f953c114580 con 0x7f953c19a720 2024-04-26T01:53:53.167 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.168+0000 7f953b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f953c1043a0 0x7f953c1a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:53.167 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.168+0000 7f953b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f953c1043a0 0x7f953c1a0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:39328/0 (socket says 172.21.15.151:39328) 2024-04-26T01:53:53.167 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.168+0000 7f953b400640 1 -- 172.21.15.151:0/3695974240 learned_addr learned my addr 172.21.15.151:0/3695974240 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:53:53.167 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.168+0000 7f9532a00640 1 --2- 172.21.15.151:0/3695974240 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f953c109470 0x7f953c1a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:53.167 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.168+0000 7f953be00640 1 --2- 172.21.15.151:0/3695974240 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f953c19a720 0x7f953c19aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:53.167 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.168+0000 7f953b400640 1 -- 172.21.15.151:0/3695974240 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f953c19a720 msgr2=0x7f953c19aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:53.168 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.168+0000 7f953b400640 1 --2- 172.21.15.151:0/3695974240 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f953c19a720 0x7f953c19aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:53.168 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.168+0000 7f953b400640 1 -- 172.21.15.151:0/3695974240 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f953c109470 msgr2=0x7f953c1a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:53.168 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.168+0000 7f953b400640 1 --2- 172.21.15.151:0/3695974240 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f953c109470 0x7f953c1a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:53.168 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.168+0000 7f953b400640 1 -- 172.21.15.151:0/3695974240 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f953c19b320 con 0x7f953c1043a0 2024-04-26T01:53:53.168 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.168+0000 7f953b400640 1 --2- 172.21.15.151:0/3695974240 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f953c1043a0 0x7f953c1a0610 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f952c00eb90 tx=0x7f952c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:53.168 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.169+0000 7f9538c00640 1 -- 172.21.15.151:0/3695974240 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952c004d20 con 0x7f953c1043a0 2024-04-26T01:53:53.169 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.169+0000 7f9542200640 1 -- 172.21.15.151:0/3695974240 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f953c1a73b0 con 0x7f953c1043a0 2024-04-26T01:53:53.169 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.169+0000 7f9538c00640 1 -- 172.21.15.151:0/3695974240 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f952c00ee90 con 0x7f953c1043a0 2024-04-26T01:53:53.169 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.169+0000 7f9538c00640 1 -- 172.21.15.151:0/3695974240 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952c010670 con 0x7f953c1043a0 2024-04-26T01:53:53.169 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.169+0000 7f9542200640 1 -- 172.21.15.151:0/3695974240 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f953c1a7760 con 0x7f953c1043a0 2024-04-26T01:53:53.171 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.172+0000 7f9544fcf740 1 Processor -- start 2024-04-26T01:53:53.171 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.172+0000 7f9538c00640 1 -- 172.21.15.151:0/3695974240 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 25) ==== 138874+0+0 (secure 0 0 0) 0x7f952c005020 con 0x7f953c1043a0 2024-04-26T01:53:53.171 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.172+0000 7f9544fcf740 1 -- start start 2024-04-26T01:53:53.172 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.173+0000 7f9544fcf740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac2d72e820 0x55ac2d72ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:53.172 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.173+0000 7f9544fcf740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac2d72f140 0x55ac2d7394c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:53.172 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.173+0000 7f9544fcf740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac2d739a00 0x55ac2d73bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:53.172 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.173+0000 7f9544fcf740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ac2d694900 con 0x55ac2d72e820 2024-04-26T01:53:53.172 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.173+0000 7f9544fcf740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ac2d63aef0 con 0x55ac2d72f140 2024-04-26T01:53:53.172 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.173+0000 7f9544fcf740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ac2d6908c0 con 0x55ac2d739a00 2024-04-26T01:53:53.172 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.173+0000 7f953be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac2d72e820 0x55ac2d72ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:53.172 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.173+0000 7f953b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac2d72f140 0x55ac2d7394c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:53.172 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.173+0000 7f9532a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac2d739a00 0x55ac2d73bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:53.172 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.173+0000 7f953be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac2d72e820 0x55ac2d72ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:39334/0 (socket says 172.21.15.151:39334) 2024-04-26T01:53:53.172 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.173+0000 7f9532a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac2d739a00 0x55ac2d73bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:43300/0 (socket says 172.21.15.151:43300) 2024-04-26T01:53:53.172 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.173+0000 7f9532a00640 1 -- 172.21.15.151:0/3812100230 learned_addr learned my addr 172.21.15.151:0/3812100230 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:53:53.172 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.173+0000 7f953b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac2d72f140 0x55ac2d7394c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:42428/0 (socket says 172.21.15.151:42428) 2024-04-26T01:53:53.174 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.174+0000 7f9532a00640 1 -- 172.21.15.151:0/3812100230 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac2d72f140 msgr2=0x55ac2d7394c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:53.174 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.174+0000 7f9532a00640 1 --2- 172.21.15.151:0/3812100230 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac2d72f140 0x55ac2d7394c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:53.174 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.174+0000 7f9532a00640 1 -- 172.21.15.151:0/3812100230 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac2d72e820 msgr2=0x55ac2d72ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:53.175 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.174+0000 7f9532a00640 1 --2- 172.21.15.151:0/3812100230 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac2d72e820 0x55ac2d72ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:53.175 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.174+0000 7f9532a00640 1 -- 172.21.15.151:0/3812100230 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac2d3a7210 con 0x55ac2d739a00 2024-04-26T01:53:53.175 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.174+0000 7f9532a00640 1 --2- 172.21.15.151:0/3812100230 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac2d739a00 0x55ac2d73bdf0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9524006010 tx=0x7f9524004170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2924b2fdac342347 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:53.175 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.174+0000 7f9538c00640 1 --2- 172.21.15.151:0/3695974240 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f95200a3ee0 0x7f95200a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:53.175 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.174+0000 7f9538c00640 1 -- 172.21.15.151:0/3695974240 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f952c0d10a0 con 0x7f953c1043a0 2024-04-26T01:53:53.175 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.174+0000 7f952be00640 1 -- 172.21.15.151:0/3812100230 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9524035040 con 0x55ac2d739a00 2024-04-26T01:53:53.175 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.174+0000 7f953be00640 1 --2- 172.21.15.151:0/3695974240 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f95200a3ee0 0x7f95200a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:53.175 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.174+0000 7f952be00640 1 -- 172.21.15.151:0/3812100230 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95240054f0 con 0x55ac2d739a00 2024-04-26T01:53:53.175 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.174+0000 7f952be00640 1 -- 172.21.15.151:0/3812100230 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95240035f0 con 0x55ac2d739a00 2024-04-26T01:53:53.175 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.175+0000 7f9544fcf740 1 -- 172.21.15.151:0/3812100230 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac2d739a00 msgr2=0x55ac2d73bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:53.175 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.175+0000 7f953be00640 1 --2- 172.21.15.151:0/3695974240 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f95200a3ee0 0x7f95200a63a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f953400b6c0 tx=0x7f953400b2e0 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:53.175 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.175+0000 7f9544fcf740 1 --2- 172.21.15.151:0/3812100230 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac2d739a00 0x55ac2d73bdf0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9524006010 tx=0x7f9524004170 comp rx=0 tx=0).stop 2024-04-26T01:53:53.175 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.175+0000 7f9544fcf740 1 -- 172.21.15.151:0/3812100230 shutdown_connections 2024-04-26T01:53:53.175 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.175+0000 7f9544fcf740 1 --2- 172.21.15.151:0/3812100230 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac2d739a00 0x55ac2d73bdf0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:53.176 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.175+0000 7f9544fcf740 1 --2- 172.21.15.151:0/3812100230 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac2d72f140 0x55ac2d7394c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:53.176 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.175+0000 7f9544fcf740 1 --2- 172.21.15.151:0/3812100230 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac2d72e820 0x55ac2d72ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:53.176 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.175+0000 7f9544fcf740 1 -- 172.21.15.151:0/3812100230 >> 172.21.15.151:0/3812100230 conn(0x55ac2d72ad30 msgr2=0x55ac2d72b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:53.176 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.175+0000 7f9544fcf740 1 -- 172.21.15.151:0/3812100230 shutdown_connections 2024-04-26T01:53:53.176 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.175+0000 7f9544fcf740 1 -- 172.21.15.151:0/3812100230 wait complete. 2024-04-26T01:53:53.176 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.176+0000 7f9544fcf740 1 -- start start 2024-04-26T01:53:53.176 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.176+0000 7f9544fcf740 1 Processor -- start 2024-04-26T01:53:53.180 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.177+0000 7f9544fcf740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac2d72e820 0x55ac2d7368e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:53.180 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.177+0000 7f9544fcf740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac2d72f140 0x55ac2d736e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:53.180 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.177+0000 7f9544fcf740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac2d739a00 0x55ac2d737360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:53.180 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.177+0000 7f9544fcf740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ac2d73c5d0 con 0x55ac2d72e820 2024-04-26T01:53:53.180 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.177+0000 7f9544fcf740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ac2d64e310 con 0x55ac2d72f140 2024-04-26T01:53:53.180 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.177+0000 7f9544fcf740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ac2d73c750 con 0x55ac2d739a00 2024-04-26T01:53:53.180 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.177+0000 7f9544fcf740 10 client.0 _ll_register_callbacks cb 0x55ac2d61ccb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:53:53.180 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.177+0000 7f953b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac2d72e820 0x55ac2d7368e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:53.180 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.177+0000 7f953be00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac2d739a00 0x55ac2d737360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:53.180 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.177+0000 7f953b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac2d72e820 0x55ac2d7368e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:39340/0 (socket says 172.21.15.151:39340) 2024-04-26T01:53:53.180 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.177+0000 7f953b400640 1 -- 172.21.15.151:0/2710326885 learned_addr learned my addr 172.21.15.151:0/2710326885 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:53:53.180 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.177+0000 7f9532a00640 1 --2- 172.21.15.151:0/2710326885 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac2d72f140 0x55ac2d736e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:53.180 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.177+0000 7f953b400640 1 -- 172.21.15.151:0/2710326885 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac2d739a00 msgr2=0x55ac2d737360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:53.181 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.177+0000 7f953b400640 1 --2- 172.21.15.151:0/2710326885 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac2d739a00 0x55ac2d737360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:53.181 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.177+0000 7f953b400640 1 -- 172.21.15.151:0/2710326885 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac2d72f140 msgr2=0x55ac2d736e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:53.181 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.177+0000 7f953b400640 1 --2- 172.21.15.151:0/2710326885 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac2d72f140 0x55ac2d736e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:53.181 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.177+0000 7f953b400640 1 -- 172.21.15.151:0/2710326885 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac2d473620 con 0x55ac2d72e820 2024-04-26T01:53:53.181 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.178+0000 7f953b400640 1 --2- 172.21.15.151:0/2710326885 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac2d72e820 0x55ac2d7368e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f952c09cb00 tx=0x7f952c004f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=712fc509a9a48d97 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:53.181 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.178+0000 7f9528c00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-04-26T01:53:53.181 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.178+0000 7f9528c00640 1 -- 172.21.15.151:0/2710326885 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952c09c940 con 0x55ac2d72e820 2024-04-26T01:53:53.181 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.178+0000 7f9528c00640 1 -- 172.21.15.151:0/2710326885 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f952c0d3110 con 0x55ac2d72e820 2024-04-26T01:53:53.181 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.178+0000 7f9544fcf740 1 -- 172.21.15.151:0/2710326885 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95240054f0 con 0x55ac2d72e820 2024-04-26T01:53:53.181 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.178+0000 7f9528c00640 1 -- 172.21.15.151:0/2710326885 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f952c09c760 con 0x55ac2d72e820 2024-04-26T01:53:53.181 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.178+0000 7f9544fcf740 10 client.15057 fetch_fsmap learned FSMap version 26 2024-04-26T01:53:53.181 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.178+0000 7f9544fcf740 1 -- 172.21.15.151:0/2710326885 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ac2d7487d0 con 0x55ac2d72e820 2024-04-26T01:53:53.181 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.178+0000 7f9528c00640 1 -- 172.21.15.151:0/2710326885 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95240054f0 con 0x55ac2d72e820 2024-04-26T01:53:53.181 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.179+0000 7f9528c00640 1 -- 172.21.15.151:0/2710326885 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f952c0a55e0 con 0x55ac2d72e820 2024-04-26T01:53:53.182 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.179+0000 7f9544fcf740 10 client.15057 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T01:53:53.182 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.179+0000 7f9544fcf740 10 client.15057 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:53:53.182 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.179+0000 7f9544fcf740 20 client.15057 populate_metadata read hostname 'smithi151' 2024-04-26T01:53:53.182 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.179+0000 7f9544fcf740 1 --2- 172.21.15.151:0/2710326885 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55ac2d749f60 0x55ac2d74c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:53.182 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.179+0000 7f9544fcf740 4 client.15057 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:53:53.182 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.179+0000 7f9544fcf740 1 -- 172.21.15.151:0/2710326885 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac2d6a88b0 con 0x55ac2d749f60 2024-04-26T01:53:53.182 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.179+0000 7f9532a00640 1 --2- 172.21.15.151:0/2710326885 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55ac2d749f60 0x55ac2d74c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:53.182 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.180+0000 7f9532a00640 1 --2- 172.21.15.151:0/2710326885 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55ac2d749f60 0x55ac2d74c350 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7611cb033c093ade server_cookie=347fffc72ce37dac in_seq=0 out_seq=0 2024-04-26T01:53:53.182 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.180+0000 7f9528c00640 10 client.15057 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:53:53.182 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.180+0000 7f9528c00640 1 -- 172.21.15.151:0/2710326885 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ac2d6a88b0 con 0x55ac2d749f60 2024-04-26T01:53:53.182 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.180+0000 7f9528c00640 10 client.15057 handle_command_reply: tid=0 2024-04-26T01:53:53.182 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.181+0000 7f9544fcf740 1 client.15057 shutdown 2024-04-26T01:53:53.182 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.181+0000 7f9544fcf740 1 -- 172.21.15.151:0/2710326885 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac2d72e820 msgr2=0x55ac2d7368e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:53.183 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.181+0000 7f9544fcf740 1 --2- 172.21.15.151:0/2710326885 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac2d72e820 0x55ac2d7368e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f952c09cb00 tx=0x7f952c004f40 comp rx=0 tx=0).stop 2024-04-26T01:53:53.183 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.181+0000 7f9544fcf740 1 -- 172.21.15.151:0/2710326885 shutdown_connections 2024-04-26T01:53:53.183 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.181+0000 7f9544fcf740 1 --2- 172.21.15.151:0/2710326885 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55ac2d749f60 0x55ac2d74c350 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:53.183 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.181+0000 7f9544fcf740 1 --2- 172.21.15.151:0/2710326885 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac2d739a00 0x55ac2d737360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:53.183 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.181+0000 7f9544fcf740 1 --2- 172.21.15.151:0/2710326885 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac2d72f140 0x55ac2d736e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:53.183 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.181+0000 7f9528c00640 0 client.15057 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:53:53.183 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.181+0000 7f9544fcf740 1 --2- 172.21.15.151:0/2710326885 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac2d72e820 0x55ac2d7368e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:53.183 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.181+0000 7f9544fcf740 1 -- 172.21.15.151:0/2710326885 >> 172.21.15.151:0/2710326885 conn(0x55ac2d72ad30 msgr2=0x55ac2d730ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:53.183 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.181+0000 7f9544fcf740 1 -- 172.21.15.151:0/2710326885 shutdown_connections 2024-04-26T01:53:53.183 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.181+0000 7f9544fcf740 1 -- 172.21.15.151:0/2710326885 wait complete. 2024-04-26T01:53:53.183 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.181+0000 7f9544fcf740 20 client.15057 trim_cache size 0 max 16384 2024-04-26T01:53:53.196 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.197+0000 7f9544fcf740 1 Processor -- start 2024-04-26T01:53:53.196 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.197+0000 7f9544fcf740 1 -- start start 2024-04-26T01:53:53.196 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.198+0000 7f9544fcf740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac2d72e820 0x55ac2d745490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:53.196 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.198+0000 7f9544fcf740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac2d72f140 0x55ac2d729d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:53.196 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.198+0000 7f9544fcf740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac2d739a00 0x55ac2d751920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:53.196 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.198+0000 7f9544fcf740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f953c1b2430 con 0x55ac2d72f140 2024-04-26T01:53:53.196 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.198+0000 7f9544fcf740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f953c1afd00 con 0x55ac2d739a00 2024-04-26T01:53:53.196 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.198+0000 7f9544fcf740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f953c1adbb0 con 0x55ac2d72e820 2024-04-26T01:53:53.198 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.198+0000 7f953b400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac2d72e820 0x55ac2d745490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:53.198 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.198+0000 7f9532a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac2d72f140 0x55ac2d729d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:53.198 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.198+0000 7f953b400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac2d72e820 0x55ac2d745490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:43314/0 (socket says 172.21.15.151:43314) 2024-04-26T01:53:53.198 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.198+0000 7f953b400640 1 -- 172.21.15.151:0/592725688 learned_addr learned my addr 172.21.15.151:0/592725688 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:53:53.198 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.198+0000 7f953be00640 1 --2- 172.21.15.151:0/592725688 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac2d739a00 0x55ac2d751920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:53.198 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.199+0000 7f9532a00640 1 -- 172.21.15.151:0/592725688 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac2d72e820 msgr2=0x55ac2d745490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:53.199 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.199+0000 7f9532a00640 1 --2- 172.21.15.151:0/592725688 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac2d72e820 0x55ac2d745490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:53.199 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.199+0000 7f9532a00640 1 -- 172.21.15.151:0/592725688 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac2d739a00 msgr2=0x55ac2d751920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:53.199 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.199+0000 7f9532a00640 1 --2- 172.21.15.151:0/592725688 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac2d739a00 0x55ac2d751920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:53.199 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.199+0000 7f9532a00640 1 -- 172.21.15.151:0/592725688 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac2d72a890 con 0x55ac2d72f140 2024-04-26T01:53:53.199 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.199+0000 7f953b400640 1 --2- 172.21.15.151:0/592725688 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac2d72e820 0x55ac2d745490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:53:53.199 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.199+0000 7f9532a00640 1 --2- 172.21.15.151:0/592725688 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac2d72f140 0x55ac2d729d10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x55ac2d72c150 tx=0x7f9524005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8c7efb899796f26 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:53.199 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.199+0000 7f952b400640 1 -- 172.21.15.151:0/592725688 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9524035040 con 0x55ac2d72f140 2024-04-26T01:53:53.199 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.199+0000 7f952b400640 1 -- 172.21.15.151:0/592725688 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95240048c0 con 0x55ac2d72f140 2024-04-26T01:53:53.199 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.199+0000 7f952b400640 1 -- 172.21.15.151:0/592725688 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9524004eb0 con 0x55ac2d72f140 2024-04-26T01:53:53.199 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.200+0000 7f9544fcf740 1 -- 172.21.15.151:0/592725688 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac2d72f140 msgr2=0x55ac2d729d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:53.199 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.200+0000 7f9544fcf740 1 --2- 172.21.15.151:0/592725688 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac2d72f140 0x55ac2d729d10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x55ac2d72c150 tx=0x7f9524005be0 comp rx=0 tx=0).stop 2024-04-26T01:53:53.199 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.200+0000 7f9544fcf740 1 -- 172.21.15.151:0/592725688 shutdown_connections 2024-04-26T01:53:53.199 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.200+0000 7f9544fcf740 1 --2- 172.21.15.151:0/592725688 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac2d739a00 0x55ac2d751920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:53.200 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.200+0000 7f9544fcf740 1 --2- 172.21.15.151:0/592725688 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac2d72f140 0x55ac2d729d10 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:53.200 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.200+0000 7f9544fcf740 1 --2- 172.21.15.151:0/592725688 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac2d72e820 0x55ac2d745490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:53.200 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.200+0000 7f9544fcf740 1 -- 172.21.15.151:0/592725688 >> 172.21.15.151:0/592725688 conn(0x55ac2d72ad30 msgr2=0x55ac2d729600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:53.200 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.200+0000 7f9544fcf740 1 -- 172.21.15.151:0/592725688 shutdown_connections 2024-04-26T01:53:53.200 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.200+0000 7f9544fcf740 1 -- 172.21.15.151:0/592725688 wait complete. 2024-04-26T01:53:53.203 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.200+0000 7f9544fcf740 1 -- start start 2024-04-26T01:53:53.203 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.201+0000 7f9544fcf740 1 Processor -- start 2024-04-26T01:53:53.203 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.201+0000 7f9544fcf740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac2d72e820 0x55ac2d741d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:53.203 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.201+0000 7f9544fcf740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac2d72f140 0x55ac2d73f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:53.203 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.201+0000 7f9544fcf740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac2d739a00 0x55ac2d73f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:53.203 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.201+0000 7f9544fcf740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f953c1ae200 con 0x55ac2d72e820 2024-04-26T01:53:53.203 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.201+0000 7f9544fcf740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f953c1af630 con 0x55ac2d72f140 2024-04-26T01:53:53.203 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.201+0000 7f9544fcf740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f953c1ae890 con 0x55ac2d739a00 2024-04-26T01:53:53.203 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.201+0000 7f9544fcf740 10 client.0 _ll_register_callbacks cb 0x7f952c0d4820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:53:53.203 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.201+0000 7f953b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac2d72e820 0x55ac2d741d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:53.203 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.201+0000 7f953b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac2d72e820 0x55ac2d741d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:39352/0 (socket says 172.21.15.151:39352) 2024-04-26T01:53:53.204 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.201+0000 7f953b400640 1 -- 172.21.15.151:0/3552077210 learned_addr learned my addr 172.21.15.151:0/3552077210 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:53:53.204 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.201+0000 7f9532a00640 1 --2- 172.21.15.151:0/3552077210 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac2d72f140 0x55ac2d73f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:53.204 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.201+0000 7f953be00640 1 --2- 172.21.15.151:0/3552077210 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac2d739a00 0x55ac2d73f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:53.204 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.202+0000 7f953b400640 1 -- 172.21.15.151:0/3552077210 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac2d739a00 msgr2=0x55ac2d73f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:53.204 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.202+0000 7f953b400640 1 --2- 172.21.15.151:0/3552077210 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac2d739a00 0x55ac2d73f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:53.204 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.202+0000 7f953b400640 1 -- 172.21.15.151:0/3552077210 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac2d72f140 msgr2=0x55ac2d73f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:53.204 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.202+0000 7f953b400640 1 --2- 172.21.15.151:0/3552077210 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac2d72f140 0x55ac2d73f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:53.204 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.202+0000 7f953b400640 1 -- 172.21.15.151:0/3552077210 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac2d73fe50 con 0x55ac2d72e820 2024-04-26T01:53:53.204 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.202+0000 7f9532a00640 1 --2- 172.21.15.151:0/3552077210 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac2d72f140 0x55ac2d73f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:53:53.204 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.202+0000 7f953b400640 1 --2- 172.21.15.151:0/3552077210 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac2d72e820 0x55ac2d741d80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f952c099780 tx=0x7f952c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43f4e900f8082998 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:53.204 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.202+0000 7f9528c00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-04-26T01:53:53.204 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.202+0000 7f9528c00640 1 -- 172.21.15.151:0/3552077210 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952c09c940 con 0x55ac2d72e820 2024-04-26T01:53:53.204 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.202+0000 7f9528c00640 1 -- 172.21.15.151:0/3552077210 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f952c0ce960 con 0x55ac2d72e820 2024-04-26T01:53:53.204 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.202+0000 7f9544fcf740 1 -- 172.21.15.151:0/3552077210 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95240048c0 con 0x55ac2d72e820 2024-04-26T01:53:53.205 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.202+0000 7f9528c00640 1 -- 172.21.15.151:0/3552077210 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f952c0ce5c0 con 0x55ac2d72e820 2024-04-26T01:53:53.205 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.202+0000 7f9544fcf740 10 client.15063 fetch_fsmap learned FSMap version 26 2024-04-26T01:53:53.205 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.202+0000 7f9544fcf740 1 -- 172.21.15.151:0/3552077210 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ac2d7b89f0 con 0x55ac2d72e820 2024-04-26T01:53:53.205 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.203+0000 7f9528c00640 1 -- 172.21.15.151:0/3552077210 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95240048c0 con 0x55ac2d72e820 2024-04-26T01:53:53.205 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.203+0000 7f9528c00640 1 -- 172.21.15.151:0/3552077210 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f952c005310 con 0x55ac2d72e820 2024-04-26T01:53:53.205 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.203+0000 7f9544fcf740 10 client.15063 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T01:53:53.205 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.203+0000 7f9544fcf740 10 client.15063 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:53:53.205 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.203+0000 7f9544fcf740 20 client.15063 populate_metadata read hostname 'smithi151' 2024-04-26T01:53:53.205 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.203+0000 7f9544fcf740 1 --2- 172.21.15.151:0/3552077210 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55ac2d7b9140 0x55ac2d7bb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:53.205 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.203+0000 7f9544fcf740 4 client.15063 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:53:53.205 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.203+0000 7f9544fcf740 1 -- 172.21.15.151:0/3552077210 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ac2d6a2320 con 0x55ac2d7b9140 2024-04-26T01:53:53.205 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.203+0000 7f9532a00640 1 --2- 172.21.15.151:0/3552077210 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55ac2d7b9140 0x55ac2d7bb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:53.205 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.204+0000 7f9532a00640 1 --2- 172.21.15.151:0/3552077210 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55ac2d7b9140 0x55ac2d7bb560 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f700557311d65bbd server_cookie=3ffd86413afd185f in_seq=0 out_seq=0 2024-04-26T01:53:53.206 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.204+0000 7f9528c00640 10 client.15063 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:53:53.215 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.216+0000 7f9528c00640 1 -- 172.21.15.151:0/3552077210 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ac2d6a2320 con 0x55ac2d7b9140 2024-04-26T01:53:53.216 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.216+0000 7f9528c00640 10 client.15063 handle_command_reply: tid=0 2024-04-26T01:53:53.216 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.216+0000 7f9544fcf740 1 client.15063 shutdown 2024-04-26T01:53:53.216 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.216+0000 7f9544fcf740 1 -- 172.21.15.151:0/3552077210 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac2d72e820 msgr2=0x55ac2d741d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:53.216 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.216+0000 7f9544fcf740 1 --2- 172.21.15.151:0/3552077210 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac2d72e820 0x55ac2d741d80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f952c099780 tx=0x7f952c00cf90 comp rx=0 tx=0).stop 2024-04-26T01:53:53.216 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.216+0000 7f9544fcf740 1 -- 172.21.15.151:0/3552077210 shutdown_connections 2024-04-26T01:53:53.216 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.216+0000 7f9544fcf740 1 --2- 172.21.15.151:0/3552077210 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55ac2d7b9140 0x55ac2d7bb560 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:53.216 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.216+0000 7f9544fcf740 1 --2- 172.21.15.151:0/3552077210 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac2d739a00 0x55ac2d73f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:53.216 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.216+0000 7f9544fcf740 1 --2- 172.21.15.151:0/3552077210 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac2d72f140 0x55ac2d73f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:53.216 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.216+0000 7f9544fcf740 1 --2- 172.21.15.151:0/3552077210 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ac2d72e820 0x55ac2d741d80 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:53.216 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.216+0000 7f9528c00640 0 client.15063 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:53:53.216 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.216+0000 7f9544fcf740 1 -- 172.21.15.151:0/3552077210 >> 172.21.15.151:0/3552077210 conn(0x55ac2d72ad30 msgr2=0x55ac2d729e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:53.216 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.217+0000 7f9544fcf740 1 -- 172.21.15.151:0/3552077210 shutdown_connections 2024-04-26T01:53:53.216 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.217+0000 7f9544fcf740 1 -- 172.21.15.151:0/3552077210 wait complete. 2024-04-26T01:53:53.217 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.217+0000 7f9544fcf740 20 client.15063 trim_cache size 0 max 16384 2024-04-26T01:53:53.217 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:{ 2024-04-26T01:53:53.217 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "message": "", 2024-04-26T01:53:53.217 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "return_code": 0 2024-04-26T01:53:53.217 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:} 2024-04-26T01:53:53.218 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.217+0000 7f9542200640 1 -- 172.21.15.151:0/3695974240 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f95200a3ee0 msgr2=0x7f95200a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:53.218 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.217+0000 7f9542200640 1 --2- 172.21.15.151:0/3695974240 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f95200a3ee0 0x7f95200a63a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f953400b6c0 tx=0x7f953400b2e0 comp rx=0 tx=0).stop 2024-04-26T01:53:53.218 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.217+0000 7f9542200640 1 -- 172.21.15.151:0/3695974240 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f953c1043a0 msgr2=0x7f953c1a0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:53.218 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.217+0000 7f9542200640 1 --2- 172.21.15.151:0/3695974240 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f953c1043a0 0x7f953c1a0610 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f952c00eb90 tx=0x7f952c00c6a0 comp rx=0 tx=0).stop 2024-04-26T01:53:53.218 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.218+0000 7f9542200640 1 -- 172.21.15.151:0/3695974240 shutdown_connections 2024-04-26T01:53:53.218 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.218+0000 7f9542200640 1 --2- 172.21.15.151:0/3695974240 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f95200a3ee0 0x7f95200a63a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:53.218 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.218+0000 7f9542200640 1 --2- 172.21.15.151:0/3695974240 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f953c19a720 0x7f953c19aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:53.219 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.218+0000 7f9542200640 1 --2- 172.21.15.151:0/3695974240 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f953c109470 0x7f953c1a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:53.219 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.218+0000 7f9542200640 1 --2- 172.21.15.151:0/3695974240 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f953c1043a0 0x7f953c1a0610 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:53.219 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.218+0000 7f9542200640 1 -- 172.21.15.151:0/3695974240 >> 172.21.15.151:0/3695974240 conn(0x7f953c0ffc40 msgr2=0x7f953c101fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:53.219 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.218+0000 7f9542200640 1 -- 172.21.15.151:0/3695974240 shutdown_connections 2024-04-26T01:53:53.219 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:53:53.218+0000 7f9542200640 1 -- 172.21.15.151:0/3695974240 wait complete. 2024-04-26T01:53:53.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:53 smithi151 ceph-mon[28057]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-26T01:53:54.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:53 smithi172 ceph-mon[32785]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-26T01:53:54.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:53 smithi165 ceph-mon[32648]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-26T01:53:54.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:54.394+0000 7fd511c00700 1 -- 172.21.15.151:0/4291696884 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd50c109cc0 msgr2=0x7fd50c10dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:54.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:54.394+0000 7fd511c00700 1 --2- 172.21.15.151:0/4291696884 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd50c109cc0 0x7fd50c10dcc0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd4f80099c0 tx=0x7fd4f8009cd0 comp rx=0 tx=0).stop 2024-04-26T01:53:54.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:54.395+0000 7fd511c00700 1 -- 172.21.15.151:0/4291696884 shutdown_connections 2024-04-26T01:53:54.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:54.395+0000 7fd511c00700 1 --2- 172.21.15.151:0/4291696884 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd50c10e3a0 0x7fd50c115f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:54.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:54.395+0000 7fd511c00700 1 --2- 172.21.15.151:0/4291696884 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd50c109310 0x7fd50c1096f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:54.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:54.395+0000 7fd511c00700 1 --2- 172.21.15.151:0/4291696884 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd50c109cc0 0x7fd50c10dcc0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:54.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:54.395+0000 7fd511c00700 1 -- 172.21.15.151:0/4291696884 >> 172.21.15.151:0/4291696884 conn(0x7fd50c076210 msgr2=0x7fd50c078630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:54.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:54.395+0000 7fd511c00700 1 -- 172.21.15.151:0/4291696884 shutdown_connections 2024-04-26T01:53:54.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:54.395+0000 7fd511c00700 1 -- 172.21.15.151:0/4291696884 wait complete. 2024-04-26T01:53:54.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:54.396+0000 7fd511c00700 1 Processor -- start 2024-04-26T01:53:54.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:54.396+0000 7fd511c00700 1 -- start start 2024-04-26T01:53:54.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:54.396+0000 7fd511c00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd50c109310 0x7fd50c1a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:54.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:54.396+0000 7fd511c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd50c109cc0 0x7fd50c1a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:54.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:54.396+0000 7fd511c00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd50c10e3a0 0x7fd50c19f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:54.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:54.396+0000 7fd511c00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd50c119260 con 0x7fd50c109cc0 2024-04-26T01:53:54.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:54.396+0000 7fd511c00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd50c1190e0 con 0x7fd50c10e3a0 2024-04-26T01:53:54.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:54.396+0000 7fd511c00700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd50c1193e0 con 0x7fd50c109310 2024-04-26T01:53:54.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:54.397+0000 7fd50a000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd50c109cc0 0x7fd50c1a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:54.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:54.397+0000 7fd50a000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd50c109cc0 0x7fd50c1a5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:39370/0 (socket says 172.21.15.151:39370) 2024-04-26T01:53:54.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:54.397+0000 7fd50a000700 1 -- 172.21.15.151:0/1521610256 learned_addr learned my addr 172.21.15.151:0/1521610256 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:53:54.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:54.397+0000 7fd50b400700 1 --2- 172.21.15.151:0/1521610256 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd50c10e3a0 0x7fd50c19f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:54.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:54.397+0000 7fd50aa00700 1 --2- 172.21.15.151:0/1521610256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd50c109310 0x7fd50c1a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:54.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:54.397+0000 7fd50a000700 1 -- 172.21.15.151:0/1521610256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd50c109310 msgr2=0x7fd50c1a53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:54.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:54.397+0000 7fd50a000700 1 --2- 172.21.15.151:0/1521610256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd50c109310 0x7fd50c1a53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:54.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:54.397+0000 7fd50a000700 1 -- 172.21.15.151:0/1521610256 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd50c10e3a0 msgr2=0x7fd50c19f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:54.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:54.397+0000 7fd50a000700 1 --2- 172.21.15.151:0/1521610256 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd50c10e3a0 0x7fd50c19f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:54.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:54.397+0000 7fd50a000700 1 -- 172.21.15.151:0/1521610256 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd50c19fd00 con 0x7fd50c109cc0 2024-04-26T01:53:54.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:54.398+0000 7fd50a000700 1 --2- 172.21.15.151:0/1521610256 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd50c109cc0 0x7fd50c1a5920 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8005f50 tx=0x7fd4f8005280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:54.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:54.398+0000 7fd503400700 1 -- 172.21.15.151:0/1521610256 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f8026070 con 0x7fd50c109cc0 2024-04-26T01:53:54.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:54.398+0000 7fd503400700 1 -- 172.21.15.151:0/1521610256 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4f8003c70 con 0x7fd50c109cc0 2024-04-26T01:53:54.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:54.398+0000 7fd511c00700 1 -- 172.21.15.151:0/1521610256 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd50c19ff90 con 0x7fd50c109cc0 2024-04-26T01:53:54.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:54.398+0000 7fd503400700 1 -- 172.21.15.151:0/1521610256 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f800f9a0 con 0x7fd50c109cc0 2024-04-26T01:53:54.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:54.399+0000 7fd511c00700 1 -- 172.21.15.151:0/1521610256 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd50c1ac2b0 con 0x7fd50c109cc0 2024-04-26T01:53:54.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:54.400+0000 7fd511c00700 1 -- 172.21.15.151:0/1521610256 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd50c069730 con 0x7fd50c109cc0 2024-04-26T01:53:54.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:54.401+0000 7fd503400700 1 -- 172.21.15.151:0/1521610256 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 25) ==== 138874+0+0 (secure 0 0 0) 0x7fd4f8029070 con 0x7fd50c109cc0 2024-04-26T01:53:54.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:54.402+0000 7fd503400700 1 --2- 172.21.15.151:0/1521610256 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fd4f00a3f50 0x7fd4f00a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:54.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:54.403+0000 7fd503400700 1 -- 172.21.15.151:0/1521610256 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fd4f80d0ff0 con 0x7fd50c109cc0 2024-04-26T01:53:54.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:54.403+0000 7fd50aa00700 1 --2- 172.21.15.151:0/1521610256 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fd4f00a3f50 0x7fd4f00a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:54.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:54.403+0000 7fd50aa00700 1 --2- 172.21.15.151:0/1521610256 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fd4f00a3f50 0x7fd4f00a6410 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc009680 tx=0x7fd4fc009260 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:54.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:54.407+0000 7fd503400700 1 -- 172.21.15.151:0/1521610256 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd4f8013070 con 0x7fd50c109cc0 2024-04-26T01:53:54.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:54.686+0000 7fd511c00700 1 -- 172.21.15.151:0/1521610256 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd50c02df80 con 0x7fd50c109cc0 2024-04-26T01:53:54.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:54.687+0000 7fd503400700 1 -- 172.21.15.151:0/1521610256 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11889 (secure 0 0 0) 0x7fd4f809d4e0 con 0x7fd50c109cc0 2024-04-26T01:53:54.687 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:53:54.687 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2937864097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2937864097},{"type":"v1","addr":"172.21.15.151:6835","nonce":2937864097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14373,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/2947090802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2947090802},{"type":"v1","addr":"172.21.15.151:6837","nonce":2947090802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/1356563280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1356563280},{"type":"v1","addr":"172.21.15.151:6839","nonce":1356563280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6841/1075240552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1075240552},{"type":"v1","addr":"172.21.15.151:6841","nonce":1075240552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/504675912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":504675912},{"type":"v1","addr":"172.21.15.165:6833","nonce":504675912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1207580015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1207580015},{"type":"v1","addr":"172.21.15.172:6833","nonce":1207580015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:49:40.746876+0000","modified":"2024-04-26T01:53:36.483847+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24505,"mds_2":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/3598138294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3598138294},{"type":"v1","addr":"172.21.15.165:6835","nonce":3598138294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/1847859054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":1847859054},{"type":"v1","addr":"172.21.15.165:6837","nonce":1847859054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/2309751171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2309751171},{"type":"v1","addr":"172.21.15.172:6835","nonce":2309751171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.172:6837/55972537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":55972537},{"type":"v1","addr":"172.21.15.172:6837","nonce":55972537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.172:6839/1242302810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1242302810},{"type":"v1","addr":"172.21.15.172:6839","nonce":1242302810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/1969716254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1969716254},{"type":"v1","addr":"172.21.15.165:6839","nonce":1969716254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24493,24505,24509]},"id":1}]} 2024-04-26T01:53:54.690 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:54.690+0000 7fd511c00700 1 -- 172.21.15.151:0/1521610256 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fd4f00a3f50 msgr2=0x7fd4f00a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:54.690 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:54.690+0000 7fd511c00700 1 --2- 172.21.15.151:0/1521610256 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fd4f00a3f50 0x7fd4f00a6410 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc009680 tx=0x7fd4fc009260 comp rx=0 tx=0).stop 2024-04-26T01:53:54.690 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:54.690+0000 7fd511c00700 1 -- 172.21.15.151:0/1521610256 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd50c109cc0 msgr2=0x7fd50c1a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:54.690 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:54.690+0000 7fd511c00700 1 --2- 172.21.15.151:0/1521610256 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd50c109cc0 0x7fd50c1a5920 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8005f50 tx=0x7fd4f8005280 comp rx=0 tx=0).stop 2024-04-26T01:53:54.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:54.690+0000 7fd511c00700 1 -- 172.21.15.151:0/1521610256 shutdown_connections 2024-04-26T01:53:54.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:54.690+0000 7fd511c00700 1 --2- 172.21.15.151:0/1521610256 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fd4f00a3f50 0x7fd4f00a6410 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:54.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:54.690+0000 7fd511c00700 1 --2- 172.21.15.151:0/1521610256 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd50c10e3a0 0x7fd50c19f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:54.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:54.690+0000 7fd511c00700 1 --2- 172.21.15.151:0/1521610256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd50c109310 0x7fd50c1a53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:54.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:54.690+0000 7fd511c00700 1 --2- 172.21.15.151:0/1521610256 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd50c109cc0 0x7fd50c1a5920 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:54.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:54.690+0000 7fd511c00700 1 -- 172.21.15.151:0/1521610256 >> 172.21.15.151:0/1521610256 conn(0x7fd50c076210 msgr2=0x7fd50c077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:54.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:54.691+0000 7fd511c00700 1 -- 172.21.15.151:0/1521610256 shutdown_connections 2024-04-26T01:53:54.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:54.691+0000 7fd511c00700 1 -- 172.21.15.151:0/1521610256 wait complete. 2024-04-26T01:53:54.692 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 26 2024-04-26T01:53:55.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:54 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/1521610256' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:53:55.085 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:54 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/1521610256' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:53:55.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:54 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/1521610256' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:53:55.456 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:53:55.908 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:53:56.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:55 smithi172 ceph-mon[32785]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-04-26T01:53:56.059 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:55 smithi151 ceph-mon[28057]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-04-26T01:53:56.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:55 smithi165 ceph-mon[32648]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-04-26T01:53:57.986 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:57 smithi151 ceph-mon[28057]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T01:53:58.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:57 smithi172 ceph-mon[32785]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T01:53:58.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:57 smithi165 ceph-mon[32648]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T01:53:58.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:58.161+0000 7fc417a00700 1 -- 172.21.15.151:0/3317918693 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc41006afe0 msgr2=0x7fc41006b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:58.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:58.161+0000 7fc417a00700 1 --2- 172.21.15.151:0/3317918693 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc41006afe0 0x7fc41006b440 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc009990 tx=0x7fc3fc009ca0 comp rx=0 tx=0).stop 2024-04-26T01:53:58.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:58.161+0000 7fc417a00700 1 -- 172.21.15.151:0/3317918693 shutdown_connections 2024-04-26T01:53:58.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:58.161+0000 7fc417a00700 1 --2- 172.21.15.151:0/3317918693 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc41006b980 0x7fc41010f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:58.160 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:58.161+0000 7fc417a00700 1 --2- 172.21.15.151:0/3317918693 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc410108a20 0x7fc410108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:58.160 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:58.161+0000 7fc417a00700 1 --2- 172.21.15.151:0/3317918693 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc41006afe0 0x7fc41006b440 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:58.160 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:58.161+0000 7fc417a00700 1 -- 172.21.15.151:0/3317918693 >> 172.21.15.151:0/3317918693 conn(0x7fc4100788b0 msgr2=0x7fc410078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:58.160 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:58.161+0000 7fc417a00700 1 -- 172.21.15.151:0/3317918693 shutdown_connections 2024-04-26T01:53:58.160 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:58.161+0000 7fc417a00700 1 -- 172.21.15.151:0/3317918693 wait complete. 2024-04-26T01:53:58.161 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:58.163+0000 7fc417a00700 1 Processor -- start 2024-04-26T01:53:58.161 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:58.163+0000 7fc417a00700 1 -- start start 2024-04-26T01:53:58.161 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:58.163+0000 7fc417a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc41006afe0 0x7fc410101a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:58.161 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:58.163+0000 7fc417a00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc41006b980 0x7fc410101f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:58.161 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:58.163+0000 7fc417a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc410108a20 0x7fc410102480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:58.161 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:58.163+0000 7fc417a00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc410112bc0 con 0x7fc410108a20 2024-04-26T01:53:58.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:58.163+0000 7fc417a00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc410112a40 con 0x7fc41006afe0 2024-04-26T01:53:58.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:58.163+0000 7fc417a00700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc410112d40 con 0x7fc41006b980 2024-04-26T01:53:58.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:58.163+0000 7fc415a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc410108a20 0x7fc410102480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:58.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:58.164+0000 7fc415a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc410108a20 0x7fc410102480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:39386/0 (socket says 172.21.15.151:39386) 2024-04-26T01:53:58.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:58.164+0000 7fc415000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc41006afe0 0x7fc410101a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:58.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:58.164+0000 7fc415a00700 1 -- 172.21.15.151:0/3651007462 learned_addr learned my addr 172.21.15.151:0/3651007462 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:53:58.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:58.164+0000 7fc40fe00700 1 --2- 172.21.15.151:0/3651007462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc41006b980 0x7fc410101f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:58.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:58.164+0000 7fc415000700 1 -- 172.21.15.151:0/3651007462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc41006b980 msgr2=0x7fc410101f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:58.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:58.164+0000 7fc415000700 1 --2- 172.21.15.151:0/3651007462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc41006b980 0x7fc410101f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:58.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:58.164+0000 7fc415000700 1 -- 172.21.15.151:0/3651007462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc410108a20 msgr2=0x7fc410102480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:58.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:58.164+0000 7fc415000700 1 --2- 172.21.15.151:0/3651007462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc410108a20 0x7fc410102480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:58.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:58.164+0000 7fc415000700 1 -- 172.21.15.151:0/3651007462 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4101a7680 con 0x7fc41006afe0 2024-04-26T01:53:58.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:58.164+0000 7fc415a00700 1 --2- 172.21.15.151:0/3651007462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc410108a20 0x7fc410102480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:53:58.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:58.164+0000 7fc415000700 1 --2- 172.21.15.151:0/3651007462 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc41006afe0 0x7fc410101a00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc404007f10 tx=0x7fc40400d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:58.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:58.164+0000 7fc40d600700 1 -- 172.21.15.151:0/3651007462 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc404004db0 con 0x7fc41006afe0 2024-04-26T01:53:58.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:58.165+0000 7fc40d600700 1 -- 172.21.15.151:0/3651007462 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc404005020 con 0x7fc41006afe0 2024-04-26T01:53:58.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:58.165+0000 7fc40d600700 1 -- 172.21.15.151:0/3651007462 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc404015620 con 0x7fc41006afe0 2024-04-26T01:53:58.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:58.165+0000 7fc417a00700 1 -- 172.21.15.151:0/3651007462 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4101a7970 con 0x7fc41006afe0 2024-04-26T01:53:58.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:58.165+0000 7fc417a00700 1 -- 172.21.15.151:0/3651007462 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4101a7e30 con 0x7fc41006afe0 2024-04-26T01:53:58.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:58.166+0000 7fc417a00700 1 -- 172.21.15.151:0/3651007462 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc410069730 con 0x7fc41006afe0 2024-04-26T01:53:58.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:58.168+0000 7fc40d600700 1 -- 172.21.15.151:0/3651007462 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 25) ==== 138874+0+0 (secure 0 0 0) 0x7fc40400dac0 con 0x7fc41006afe0 2024-04-26T01:53:58.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:58.169+0000 7fc40d600700 1 --2- 172.21.15.151:0/3651007462 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fc3f80a3ed0 0x7fc3f80a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:58.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:58.169+0000 7fc40d600700 1 -- 172.21.15.151:0/3651007462 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fc4040d0390 con 0x7fc41006afe0 2024-04-26T01:53:58.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:58.170+0000 7fc40fe00700 1 --2- 172.21.15.151:0/3651007462 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fc3f80a3ed0 0x7fc3f80a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:58.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:58.170+0000 7fc40fe00700 1 --2- 172.21.15.151:0/3651007462 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fc3f80a3ed0 0x7fc3f80a6390 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc0098e0 tx=0x7fc3fc005960 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:58.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:58.173+0000 7fc40d600700 1 -- 172.21.15.151:0/3651007462 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc404010070 con 0x7fc41006afe0 2024-04-26T01:53:58.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:58.450+0000 7fc417a00700 1 -- 172.21.15.151:0/3651007462 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc41002e140 con 0x7fc41006afe0 2024-04-26T01:53:58.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:58.452+0000 7fc40d600700 1 -- 172.21.15.151:0/3651007462 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11889 (secure 0 0 0) 0x7fc40409c7f0 con 0x7fc41006afe0 2024-04-26T01:53:58.450 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:53:58.450 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2937864097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2937864097},{"type":"v1","addr":"172.21.15.151:6835","nonce":2937864097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14373,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/2947090802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2947090802},{"type":"v1","addr":"172.21.15.151:6837","nonce":2947090802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/1356563280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1356563280},{"type":"v1","addr":"172.21.15.151:6839","nonce":1356563280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6841/1075240552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1075240552},{"type":"v1","addr":"172.21.15.151:6841","nonce":1075240552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/504675912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":504675912},{"type":"v1","addr":"172.21.15.165:6833","nonce":504675912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1207580015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1207580015},{"type":"v1","addr":"172.21.15.172:6833","nonce":1207580015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:49:40.746876+0000","modified":"2024-04-26T01:53:36.483847+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24505,"mds_2":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/3598138294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3598138294},{"type":"v1","addr":"172.21.15.165:6835","nonce":3598138294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/1847859054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":1847859054},{"type":"v1","addr":"172.21.15.165:6837","nonce":1847859054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/2309751171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2309751171},{"type":"v1","addr":"172.21.15.172:6835","nonce":2309751171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.172:6837/55972537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":55972537},{"type":"v1","addr":"172.21.15.172:6837","nonce":55972537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.172:6839/1242302810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1242302810},{"type":"v1","addr":"172.21.15.172:6839","nonce":1242302810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/1969716254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1969716254},{"type":"v1","addr":"172.21.15.165:6839","nonce":1969716254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24493,24505,24509]},"id":1}]} 2024-04-26T01:53:58.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:58.455+0000 7fc417a00700 1 -- 172.21.15.151:0/3651007462 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fc3f80a3ed0 msgr2=0x7fc3f80a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:58.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:58.455+0000 7fc417a00700 1 --2- 172.21.15.151:0/3651007462 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fc3f80a3ed0 0x7fc3f80a6390 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc0098e0 tx=0x7fc3fc005960 comp rx=0 tx=0).stop 2024-04-26T01:53:58.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:58.455+0000 7fc417a00700 1 -- 172.21.15.151:0/3651007462 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc41006afe0 msgr2=0x7fc410101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:58.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:58.455+0000 7fc417a00700 1 --2- 172.21.15.151:0/3651007462 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc41006afe0 0x7fc410101a00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc404007f10 tx=0x7fc40400d3b0 comp rx=0 tx=0).stop 2024-04-26T01:53:58.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:58.455+0000 7fc417a00700 1 -- 172.21.15.151:0/3651007462 shutdown_connections 2024-04-26T01:53:58.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:58.455+0000 7fc417a00700 1 --2- 172.21.15.151:0/3651007462 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fc3f80a3ed0 0x7fc3f80a6390 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:58.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:58.455+0000 7fc417a00700 1 --2- 172.21.15.151:0/3651007462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc410108a20 0x7fc410102480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:58.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:58.455+0000 7fc417a00700 1 --2- 172.21.15.151:0/3651007462 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc41006afe0 0x7fc410101a00 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:58.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:58.455+0000 7fc417a00700 1 --2- 172.21.15.151:0/3651007462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc41006b980 0x7fc410101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:58.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:58.455+0000 7fc417a00700 1 -- 172.21.15.151:0/3651007462 >> 172.21.15.151:0/3651007462 conn(0x7fc4100788b0 msgr2=0x7fc4100ffae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:58.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:58.455+0000 7fc417a00700 1 -- 172.21.15.151:0/3651007462 shutdown_connections 2024-04-26T01:53:58.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:53:58.455+0000 7fc417a00700 1 -- 172.21.15.151:0/3651007462 wait complete. 2024-04-26T01:53:58.454 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 26 2024-04-26T01:53:59.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:58 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/3651007462' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:53:59.077 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:58 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/3651007462' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:53:59.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:58 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/3651007462' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:53:59.303 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T01:53:59.770 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:54:00.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:53:59 smithi172 ceph-mon[32785]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-04-26T01:54:00.053 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:53:59 smithi151 ceph-mon[28057]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-04-26T01:54:00.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:53:59 smithi165 ceph-mon[32648]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-04-26T01:54:01.122 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:54:00 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:54:00.893+0000 7fb991000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:54:01.122 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:54:00 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:54:00.893+0000 7fb991000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:54:01.122 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:54:00 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:54:00.893+0000 7fb991000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:54:01.122 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:54:00 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:54:00.893+0000 7fb991000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:54:01.122 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 01:54:00 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T01:54:00.893+0000 7fb991000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:54:01.938 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:54:01 smithi151 ceph-mon[28057]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-04-26T01:54:02.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:02.019+0000 7ff8aa600700 1 -- 172.21.15.151:0/282867778 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff8a4075590 msgr2=0x7ff8a410ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:02.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:02.019+0000 7ff8aa600700 1 --2- 172.21.15.151:0/282867778 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff8a4075590 0x7ff8a410ef00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff88c009990 tx=0x7ff88c009ca0 comp rx=0 tx=0).stop 2024-04-26T01:54:02.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:02.020+0000 7ff8aa600700 1 -- 172.21.15.151:0/282867778 shutdown_connections 2024-04-26T01:54:02.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:02.020+0000 7ff8aa600700 1 --2- 172.21.15.151:0/282867778 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8a410f440 0x7ff8a4111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:02.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:02.020+0000 7ff8aa600700 1 --2- 172.21.15.151:0/282867778 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff8a4074c70 0x7ff8a4075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:02.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:02.020+0000 7ff8aa600700 1 --2- 172.21.15.151:0/282867778 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff8a4075590 0x7ff8a410ef00 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:02.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:02.020+0000 7ff8aa600700 1 -- 172.21.15.151:0/282867778 >> 172.21.15.151:0/282867778 conn(0x7ff8a4079ba0 msgr2=0x7ff8a4079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:02.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:02.020+0000 7ff8aa600700 1 -- 172.21.15.151:0/282867778 shutdown_connections 2024-04-26T01:54:02.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:02.020+0000 7ff8aa600700 1 -- 172.21.15.151:0/282867778 wait complete. 2024-04-26T01:54:02.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:02.021+0000 7ff8aa600700 1 Processor -- start 2024-04-26T01:54:02.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:02.021+0000 7ff8aa600700 1 -- start start 2024-04-26T01:54:02.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:02.021+0000 7ff8aa600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff8a4074c70 0x7ff8a419b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:02.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:02.021+0000 7ff8aa600700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff8a4075590 0x7ff8a419b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:02.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:02.021+0000 7ff8aa600700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8a410f440 0x7ff8a419fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:02.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:02.021+0000 7ff8aa600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a4114e70 con 0x7ff8a4074c70 2024-04-26T01:54:02.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:02.021+0000 7ff8aa600700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a4114cf0 con 0x7ff8a4075590 2024-04-26T01:54:02.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:02.021+0000 7ff8aa600700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a4114ff0 con 0x7ff8a410f440 2024-04-26T01:54:02.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:02.022+0000 7ff8a3400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff8a4074c70 0x7ff8a419b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:02.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:02.022+0000 7ff89aa00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff8a4075590 0x7ff8a419b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:02.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:02.022+0000 7ff8a3400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff8a4074c70 0x7ff8a419b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:39734/0 (socket says 172.21.15.151:39734) 2024-04-26T01:54:02.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:02.022+0000 7ff8a3400700 1 -- 172.21.15.151:0/2197031439 learned_addr learned my addr 172.21.15.151:0/2197031439 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:54:02.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:02.022+0000 7ff8a3e00700 1 --2- 172.21.15.151:0/2197031439 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8a410f440 0x7ff8a419fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:02.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:02.022+0000 7ff8a3400700 1 -- 172.21.15.151:0/2197031439 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8a410f440 msgr2=0x7ff8a419fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:02.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:02.022+0000 7ff8a3400700 1 --2- 172.21.15.151:0/2197031439 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8a410f440 0x7ff8a419fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:02.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:02.022+0000 7ff8a3400700 1 -- 172.21.15.151:0/2197031439 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff8a4075590 msgr2=0x7ff8a419b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:02.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:02.022+0000 7ff8a3400700 1 --2- 172.21.15.151:0/2197031439 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff8a4075590 0x7ff8a419b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:02.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:02.022+0000 7ff8a3400700 1 -- 172.21.15.151:0/2197031439 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8a41a02c0 con 0x7ff8a4074c70 2024-04-26T01:54:02.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:02.022+0000 7ff8a3e00700 1 --2- 172.21.15.151:0/2197031439 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8a410f440 0x7ff8a419fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:54:02.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:02.022+0000 7ff89aa00700 1 --2- 172.21.15.151:0/2197031439 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff8a4075590 0x7ff8a419b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:54:02.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:02.023+0000 7ff8a3400700 1 --2- 172.21.15.151:0/2197031439 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff8a4074c70 0x7ff8a419b300 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff89400d980 tx=0x7ff89400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:02.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:02.023+0000 7ff8a0c00700 1 -- 172.21.15.151:0/2197031439 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff894016070 con 0x7ff8a4074c70 2024-04-26T01:54:02.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:02.023+0000 7ff8aa600700 1 -- 172.21.15.151:0/2197031439 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8a41a05b0 con 0x7ff8a4074c70 2024-04-26T01:54:02.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:02.023+0000 7ff8a0c00700 1 -- 172.21.15.151:0/2197031439 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff894004830 con 0x7ff8a4074c70 2024-04-26T01:54:02.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:02.023+0000 7ff8a0c00700 1 -- 172.21.15.151:0/2197031439 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff894005540 con 0x7ff8a4074c70 2024-04-26T01:54:02.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:02.023+0000 7ff8aa600700 1 -- 172.21.15.151:0/2197031439 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8a41a7e30 con 0x7ff8a4074c70 2024-04-26T01:54:02.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:02.024+0000 7ff8aa600700 1 -- 172.21.15.151:0/2197031439 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8a4069730 con 0x7ff8a4074c70 2024-04-26T01:54:02.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:02.026+0000 7ff8a0c00700 1 -- 172.21.15.151:0/2197031439 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 25) ==== 138874+0+0 (secure 0 0 0) 0x7ff894004220 con 0x7ff8a4074c70 2024-04-26T01:54:02.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:02.027+0000 7ff8a0c00700 1 --2- 172.21.15.151:0/2197031439 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7ff8880a3ed0 0x7ff8880a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:02.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:02.028+0000 7ff8a0c00700 1 -- 172.21.15.151:0/2197031439 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7ff8940d08a0 con 0x7ff8a4074c70 2024-04-26T01:54:02.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:02.028+0000 7ff89aa00700 1 --2- 172.21.15.151:0/2197031439 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7ff8880a3ed0 0x7ff8880a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:02.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:02.029+0000 7ff89aa00700 1 --2- 172.21.15.151:0/2197031439 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7ff8880a3ed0 0x7ff8880a6390 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4111160 tx=0x7ff88c01a040 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:02.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:02.033+0000 7ff8a0c00700 1 -- 172.21.15.151:0/2197031439 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff89401f020 con 0x7ff8a4074c70 2024-04-26T01:54:02.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:54:01 smithi172 ceph-mon[32785]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-04-26T01:54:02.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:54:01 smithi165 ceph-mon[32648]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-04-26T01:54:02.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:02.259+0000 7ff8aa600700 1 -- 172.21.15.151:0/2197031439 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff8a402df80 con 0x7ff8a4074c70 2024-04-26T01:54:02.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:02.261+0000 7ff8a0c00700 1 -- 172.21.15.151:0/2197031439 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+22982 (secure 0 0 0) 0x7ff89409cd00 con 0x7ff8a4074c70 2024-04-26T01:54:02.259 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:54:02.259 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":93,"fsid":"c0e49e64-036d-11ef-bc93-c7b262605968","created":"2024-04-26T01:40:07.483197+0000","modified":"2024-04-26T01:53:36.653639+0000","last_up_change":"2024-04-26T01:48:21.167691+0000","last_in_change":"2024-04-26T01:48:00.556990+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T01:44:11.411722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T01:49:31.756032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T01:49:36.529819+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ca27e814-a982-482c-ae1a-e596aac5c59a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":669937998},{"type":"v1","addr":"172.21.15.151:6803","nonce":669937998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":669937998},{"type":"v1","addr":"172.21.15.151:6805","nonce":669937998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":669937998},{"type":"v1","addr":"172.21.15.151:6809","nonce":669937998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":669937998},{"type":"v1","addr":"172.21.15.151:6807","nonce":669937998}]},"public_addr":"172.21.15.151:6803/669937998","cluster_addr":"172.21.15.151:6805/669937998","heartbeat_back_addr":"172.21.15.151:6809/669937998","heartbeat_front_addr":"172.21.15.151:6807/669937998","state":["exists","up"]},{"osd":1,"uuid":"e7d192c3-9624-4625-9ccb-a76c114d8aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":1356168171},{"type":"v1","addr":"172.21.15.165:6801","nonce":1356168171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":1356168171},{"type":"v1","addr":"172.21.15.165:6803","nonce":1356168171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":1356168171},{"type":"v1","addr":"172.21.15.165:6807","nonce":1356168171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":1356168171},{"type":"v1","addr":"172.21.15.165:6805","nonce":1356168171}]},"public_addr":"172.21.15.165:6801/1356168171","cluster_addr":"172.21.15.165:6803/1356168171","heartbeat_back_addr":"172.21.15.165:6807/1356168171","heartbeat_front_addr":"172.21.15.165:6805/1356168171","state":["exists","up"]},{"osd":2,"uuid":"0631df59-1ecc-48b4-b00a-48b0118dfc24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":237502635},{"type":"v1","addr":"172.21.15.172:6801","nonce":237502635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":237502635},{"type":"v1","addr":"172.21.15.172:6803","nonce":237502635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":237502635},{"type":"v1","addr":"172.21.15.172:6807","nonce":237502635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":237502635},{"type":"v1","addr":"172.21.15.172:6805","nonce":237502635}]},"public_addr":"172.21.15.172:6801/237502635","cluster_addr":"172.21.15.172:6803/237502635","heartbeat_back_addr":"172.21.15.172:6807/237502635","heartbeat_front_addr":"172.21.15.172:6805/237502635","state":["exists","up"]},{"osd":3,"uuid":"73076e49-2f8a-4a3d-99eb-7b50aff448aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3442279476},{"type":"v1","addr":"172.21.15.151:6811","nonce":3442279476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3442279476},{"type":"v1","addr":"172.21.15.151:6813","nonce":3442279476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":3442279476},{"type":"v1","addr":"172.21.15.151:6817","nonce":3442279476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3442279476},{"type":"v1","addr":"172.21.15.151:6815","nonce":3442279476}]},"public_addr":"172.21.15.151:6811/3442279476","cluster_addr":"172.21.15.151:6813/3442279476","heartbeat_back_addr":"172.21.15.151:6817/3442279476","heartbeat_front_addr":"172.21.15.151:6815/3442279476","state":["exists","up"]},{"osd":4,"uuid":"0230d615-548c-43f0-b8aa-7130f4cbbc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4029980723},{"type":"v1","addr":"172.21.15.165:6809","nonce":4029980723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4029980723},{"type":"v1","addr":"172.21.15.165:6811","nonce":4029980723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4029980723},{"type":"v1","addr":"172.21.15.165:6815","nonce":4029980723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4029980723},{"type":"v1","addr":"172.21.15.165:6813","nonce":4029980723}]},"public_addr":"172.21.15.165:6809/4029980723","cluster_addr":"172.21.15.165:6811/4029980723","heartbeat_back_addr":"172.21.15.165:6815/4029980723","heartbeat_front_addr":"172.21.15.165:6813/4029980723","state":["exists","up"]},{"osd":5,"uuid":"9739f44b-2605-47dd-82fb-d30034742bf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2216578687},{"type":"v1","addr":"172.21.15.172:6809","nonce":2216578687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2216578687},{"type":"v1","addr":"172.21.15.172:6811","nonce":2216578687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2216578687},{"type":"v1","addr":"172.21.15.172:6815","nonce":2216578687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2216578687},{"type":"v1","addr":"172.21.15.172:6813","nonce":2216578687}]},"public_addr":"172.21.15.172:6809/2216578687","cluster_addr":"172.21.15.172:6811/2216578687","heartbeat_back_addr":"172.21.15.172:6815/2216578687","heartbeat_front_addr":"172.21.15.172:6813/2216578687","state":["exists","up"]},{"osd":6,"uuid":"4290baaa-6160-49cb-a4e0-c96f070f2ef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1723252315},{"type":"v1","addr":"172.21.15.151:6819","nonce":1723252315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1723252315},{"type":"v1","addr":"172.21.15.151:6821","nonce":1723252315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":1723252315},{"type":"v1","addr":"172.21.15.151:6825","nonce":1723252315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1723252315},{"type":"v1","addr":"172.21.15.151:6823","nonce":1723252315}]},"public_addr":"172.21.15.151:6819/1723252315","cluster_addr":"172.21.15.151:6821/1723252315","heartbeat_back_addr":"172.21.15.151:6825/1723252315","heartbeat_front_addr":"172.21.15.151:6823/1723252315","state":["exists","up"]},{"osd":7,"uuid":"9f5315f2-036c-4e2f-92ed-3321a9e3c860","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":1977803781},{"type":"v1","addr":"172.21.15.165:6817","nonce":1977803781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":1977803781},{"type":"v1","addr":"172.21.15.165:6819","nonce":1977803781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":1977803781},{"type":"v1","addr":"172.21.15.165:6823","nonce":1977803781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":1977803781},{"type":"v1","addr":"172.21.15.165:6821","nonce":1977803781}]},"public_addr":"172.21.15.165:6817/1977803781","cluster_addr":"172.21.15.165:6819/1977803781","heartbeat_back_addr":"172.21.15.165:6823/1977803781","heartbeat_front_addr":"172.21.15.165:6821/1977803781","state":["exists","up"]},{"osd":8,"uuid":"8ddbe36d-cea7-45ef-a3ca-044a0e64ff85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":491199512},{"type":"v1","addr":"172.21.15.172:6817","nonce":491199512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":491199512},{"type":"v1","addr":"172.21.15.172:6819","nonce":491199512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":491199512},{"type":"v1","addr":"172.21.15.172:6823","nonce":491199512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":491199512},{"type":"v1","addr":"172.21.15.172:6821","nonce":491199512}]},"public_addr":"172.21.15.172:6817/491199512","cluster_addr":"172.21.15.172:6819/491199512","heartbeat_back_addr":"172.21.15.172:6823/491199512","heartbeat_front_addr":"172.21.15.172:6821/491199512","state":["exists","up"]},{"osd":9,"uuid":"40654ba0-1bd9-4b54-97ae-f1ed6c4785d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":1818923828},{"type":"v1","addr":"172.21.15.151:6827","nonce":1818923828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":1818923828},{"type":"v1","addr":"172.21.15.151:6829","nonce":1818923828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1818923828},{"type":"v1","addr":"172.21.15.151:6833","nonce":1818923828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":1818923828},{"type":"v1","addr":"172.21.15.151:6831","nonce":1818923828}]},"public_addr":"172.21.15.151:6827/1818923828","cluster_addr":"172.21.15.151:6829/1818923828","heartbeat_back_addr":"172.21.15.151:6833/1818923828","heartbeat_front_addr":"172.21.15.151:6831/1818923828","state":["exists","up"]},{"osd":10,"uuid":"19e75a1a-7bd0-41d9-b0cc-0fa4d1475dda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":2632621809},{"type":"v1","addr":"172.21.15.165:6825","nonce":2632621809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":2632621809},{"type":"v1","addr":"172.21.15.165:6827","nonce":2632621809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":2632621809},{"type":"v1","addr":"172.21.15.165:6831","nonce":2632621809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":2632621809},{"type":"v1","addr":"172.21.15.165:6829","nonce":2632621809}]},"public_addr":"172.21.15.165:6825/2632621809","cluster_addr":"172.21.15.165:6827/2632621809","heartbeat_back_addr":"172.21.15.165:6831/2632621809","heartbeat_front_addr":"172.21.15.165:6829/2632621809","state":["exists","up"]},{"osd":11,"uuid":"aa84e5f5-93a3-4375-8544-a1175f0eb06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":886604365},{"type":"v1","addr":"172.21.15.172:6825","nonce":886604365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":886604365},{"type":"v1","addr":"172.21.15.172:6827","nonce":886604365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":886604365},{"type":"v1","addr":"172.21.15.172:6831","nonce":886604365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":886604365},{"type":"v1","addr":"172.21.15.172:6829","nonce":886604365}]},"public_addr":"172.21.15.172:6825/886604365","cluster_addr":"172.21.15.172:6827/886604365","heartbeat_back_addr":"172.21.15.172:6831/886604365","heartbeat_front_addr":"172.21.15.172:6829/886604365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:43:18.296278+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:43:42.860620+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:44:07.073601+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:44:39.631760+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:45:04.771214+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:45:29.606674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:46:03.101851+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:46:28.351995+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:46:53.445435+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:47:27.391179+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:47:53.543638+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:48:18.687811+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.151:6800/2251256505":"2024-04-27T01:53:36.653535+0000","172.21.15.151:0/2357384001":"2024-04-27T01:53:36.653535+0000","172.21.15.151:6801/796314699":"2024-04-27T01:41:25.234322+0000","172.21.15.151:6800/796314699":"2024-04-27T01:41:25.234322+0000","172.21.15.151:0/4066781930":"2024-04-27T01:53:36.653535+0000","172.21.15.151:0/1173525554":"2024-04-27T01:41:25.234322+0000","172.21.15.151:0/1342496801":"2024-04-27T01:40:50.492554+0000","172.21.15.151:0/755039608":"2024-04-27T01:53:36.653535+0000","172.21.15.151:0/11665926":"2024-04-27T01:40:50.492554+0000","172.21.15.151:0/1765448952":"2024-04-27T01:40:50.492554+0000","172.21.15.151:0/3463483606":"2024-04-27T01:53:36.653535+0000","172.21.15.151:6801/2251256505":"2024-04-27T01:53:36.653535+0000","172.21.15.151:0/46007430":"2024-04-27T01:41:25.234322+0000","172.21.15.151:6800/244725060":"2024-04-27T01:40:50.492554+0000","172.21.15.151:0/3703862543":"2024-04-27T01:41:25.234322+0000","172.21.15.151:6801/244725060":"2024-04-27T01:40:50.492554+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T01:54:02.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:02.264+0000 7ff8aa600700 1 -- 172.21.15.151:0/2197031439 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7ff8880a3ed0 msgr2=0x7ff8880a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:02.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:02.264+0000 7ff8aa600700 1 --2- 172.21.15.151:0/2197031439 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7ff8880a3ed0 0x7ff8880a6390 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4111160 tx=0x7ff88c01a040 comp rx=0 tx=0).stop 2024-04-26T01:54:02.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:02.264+0000 7ff8aa600700 1 -- 172.21.15.151:0/2197031439 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff8a4074c70 msgr2=0x7ff8a419b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:02.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:02.264+0000 7ff8aa600700 1 --2- 172.21.15.151:0/2197031439 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff8a4074c70 0x7ff8a419b300 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff89400d980 tx=0x7ff89400dc90 comp rx=0 tx=0).stop 2024-04-26T01:54:02.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:02.264+0000 7ff8aa600700 1 -- 172.21.15.151:0/2197031439 shutdown_connections 2024-04-26T01:54:02.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:02.264+0000 7ff8aa600700 1 --2- 172.21.15.151:0/2197031439 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7ff8880a3ed0 0x7ff8880a6390 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:02.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:02.264+0000 7ff8aa600700 1 --2- 172.21.15.151:0/2197031439 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8a410f440 0x7ff8a419fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:02.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:02.264+0000 7ff8aa600700 1 --2- 172.21.15.151:0/2197031439 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff8a4074c70 0x7ff8a419b300 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:02.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:02.264+0000 7ff8aa600700 1 --2- 172.21.15.151:0/2197031439 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff8a4075590 0x7ff8a419b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:02.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:02.264+0000 7ff8aa600700 1 -- 172.21.15.151:0/2197031439 >> 172.21.15.151:0/2197031439 conn(0x7ff8a4079ba0 msgr2=0x7ff8a40ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:02.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:02.264+0000 7ff8aa600700 1 -- 172.21.15.151:0/2197031439 shutdown_connections 2024-04-26T01:54:02.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:02.264+0000 7ff8aa600700 1 -- 172.21.15.151:0/2197031439 wait complete. 2024-04-26T01:54:02.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:54:02 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/2197031439' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:54:03.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:54:02 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/2197031439' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:54:03.155 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-04-26T01:54:03.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:54:02 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/2197031439' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:54:03.613 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:54:04.058 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:54:03 smithi151 ceph-mon[28057]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 27 op/s 2024-04-26T01:54:04.059 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:54:03 smithi151 ceph-mon[28057]: mgrmap e26: x(active, since 26s), standbys: y, z 2024-04-26T01:54:04.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:54:03 smithi165 ceph-mon[32648]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 27 op/s 2024-04-26T01:54:04.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:54:03 smithi165 ceph-mon[32648]: mgrmap e26: x(active, since 26s), standbys: y, z 2024-04-26T01:54:04.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:54:03 smithi172 ceph-mon[32785]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 27 op/s 2024-04-26T01:54:04.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:54:03 smithi172 ceph-mon[32785]: mgrmap e26: x(active, since 26s), standbys: y, z 2024-04-26T01:54:06.005 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:54:05 smithi151 ceph-mon[28057]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 27 op/s 2024-04-26T01:54:06.088 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:54:05 smithi172 ceph-mon[32785]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 27 op/s 2024-04-26T01:54:06.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:54:05 smithi165 ceph-mon[32648]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 27 op/s 2024-04-26T01:54:06.366 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.368+0000 7fc58f200640 1 -- 172.21.15.151:0/112436161 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc588101fd0 msgr2=0x7fc5881023b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.368+0000 7fc58f200640 1 --2- 172.21.15.151:0/112436161 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc588101fd0 0x7fc5881023b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc57c009900 tx=0x7fc57c02a410 comp rx=0 tx=0).stop 2024-04-26T01:54:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.368+0000 7fc58f200640 1 -- 172.21.15.151:0/112436161 shutdown_connections 2024-04-26T01:54:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.368+0000 7fc58f200640 1 --2- 172.21.15.151:0/112436161 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc5881070a0 0x7fc58810ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.368+0000 7fc58f200640 1 --2- 172.21.15.151:0/112436161 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc588102980 0x7fc5881069c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.368+0000 7fc58f200640 1 --2- 172.21.15.151:0/112436161 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc588101fd0 0x7fc5881023b0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.368+0000 7fc58f200640 1 -- 172.21.15.151:0/112436161 >> 172.21.15.151:0/112436161 conn(0x7fc5880fd890 msgr2=0x7fc5880ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.368+0000 7fc58f200640 1 -- 172.21.15.151:0/112436161 shutdown_connections 2024-04-26T01:54:06.370 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.368+0000 7fc58f200640 1 -- 172.21.15.151:0/112436161 wait complete. 2024-04-26T01:54:06.372 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.369+0000 7fc58f200640 1 Processor -- start 2024-04-26T01:54:06.372 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.369+0000 7fc58f200640 1 -- start start 2024-04-26T01:54:06.372 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.370+0000 7fc58f200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc588101fd0 0x7fc588198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:06.372 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.370+0000 7fc58f200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc5881070a0 0x7fc588198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:06.372 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.370+0000 7fc58f200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc58819cef0 0x7fc58819d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:06.372 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.370+0000 7fc58f200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc5881120b0 con 0x7fc588101fd0 2024-04-26T01:54:06.372 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.370+0000 7fc58f200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc588111f30 con 0x7fc5881070a0 2024-04-26T01:54:06.372 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.370+0000 7fc58f200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc588112230 con 0x7fc58819cef0 2024-04-26T01:54:06.372 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.370+0000 7fc58d400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc5881070a0 0x7fc588198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:06.373 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.370+0000 7fc58de00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc588101fd0 0x7fc588198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:06.373 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.370+0000 7fc58d400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc5881070a0 0x7fc588198b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:45510/0 (socket says 172.21.15.151:45510) 2024-04-26T01:54:06.373 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.370+0000 7fc58d400640 1 -- 172.21.15.151:0/4198815437 learned_addr learned my addr 172.21.15.151:0/4198815437 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:54:06.373 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.371+0000 7fc58d400640 1 -- 172.21.15.151:0/4198815437 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc58819cef0 msgr2=0x7fc58819d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:06.373 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.371+0000 7fc58d400640 1 --2- 172.21.15.151:0/4198815437 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc58819cef0 0x7fc58819d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:06.373 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.371+0000 7fc58d400640 1 -- 172.21.15.151:0/4198815437 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc588101fd0 msgr2=0x7fc588198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:06.373 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.371+0000 7fc58d400640 1 --2- 172.21.15.151:0/4198815437 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc588101fd0 0x7fc588198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:06.373 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.371+0000 7fc58d400640 1 -- 172.21.15.151:0/4198815437 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc58819d9e0 con 0x7fc5881070a0 2024-04-26T01:54:06.376 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.378+0000 7fc58d400640 1 --2- 172.21.15.151:0/4198815437 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc5881070a0 0x7fc588198b70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc57800b470 tx=0x7fc57800b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:06.376 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.378+0000 7fc582a00640 1 -- 172.21.15.151:0/4198815437 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc578015020 con 0x7fc5881070a0 2024-04-26T01:54:06.376 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.378+0000 7fc582a00640 1 -- 172.21.15.151:0/4198815437 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc578011d40 con 0x7fc5881070a0 2024-04-26T01:54:06.377 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.378+0000 7fc58f200640 1 -- 172.21.15.151:0/4198815437 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc58819dcd0 con 0x7fc5881070a0 2024-04-26T01:54:06.377 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.378+0000 7fc582a00640 1 -- 172.21.15.151:0/4198815437 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57800f340 con 0x7fc5881070a0 2024-04-26T01:54:06.377 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.378+0000 7fc58f200640 1 -- 172.21.15.151:0/4198815437 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5881a54c0 con 0x7fc5881070a0 2024-04-26T01:54:06.381 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.381+0000 7fc593788740 1 Processor -- start 2024-04-26T01:54:06.381 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.381+0000 7fc593788740 1 -- start start 2024-04-26T01:54:06.381 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.382+0000 7fc593788740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5602808e89b0 0x5602808e8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:06.382 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.382+0000 7fc593788740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602808e92d0 0x5602808f3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:06.382 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.382+0000 7fc593788740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602808f3b90 0x5602808f5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:06.382 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.382+0000 7fc593788740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56028084e900 con 0x5602808e89b0 2024-04-26T01:54:06.382 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.382+0000 7fc593788740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5602807f4ef0 con 0x5602808f3b90 2024-04-26T01:54:06.382 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.382+0000 7fc593788740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56028084a8c0 con 0x5602808e92d0 2024-04-26T01:54:06.382 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.382+0000 7fc58e800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5602808e89b0 0x5602808e8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:06.382 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.382+0000 7fc58de00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602808e92d0 0x5602808f3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:06.382 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.382+0000 7fc58e800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5602808e89b0 0x5602808e8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:39766/0 (socket says 172.21.15.151:39766) 2024-04-26T01:54:06.382 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.382+0000 7fc58e800640 1 -- 172.21.15.151:0/1243000444 learned_addr learned my addr 172.21.15.151:0/1243000444 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:54:06.382 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.382+0000 7fc58e800640 1 -- 172.21.15.151:0/1243000444 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602808e92d0 msgr2=0x5602808f3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:06.382 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.382+0000 7fc58e800640 1 --2- 172.21.15.151:0/1243000444 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602808e92d0 0x5602808f3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:06.383 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.382+0000 7fc58e800640 1 -- 172.21.15.151:0/1243000444 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602808f3b90 msgr2=0x5602808f5f80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T01:54:06.383 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.382+0000 7fc58e800640 1 --2- 172.21.15.151:0/1243000444 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602808f3b90 0x5602808f5f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:06.383 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.383+0000 7fc58e800640 1 -- 172.21.15.151:0/1243000444 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560280561210 con 0x5602808e89b0 2024-04-26T01:54:06.383 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.383+0000 7fc58e800640 1 --2- 172.21.15.151:0/1243000444 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5602808e89b0 0x5602808e8d90 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc584009ea0 tx=0x7fc584009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84532fe75191610c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:06.383 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.383+0000 7fc582a00640 1 -- 172.21.15.151:0/4198815437 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 26) ==== 138931+0+0 (secure 0 0 0) 0x7fc578018020 con 0x7fc5881070a0 2024-04-26T01:54:06.383 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.383+0000 7fc577000640 1 -- 172.21.15.151:0/1243000444 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc584004d20 con 0x5602808e89b0 2024-04-26T01:54:06.383 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.383+0000 7fc577000640 1 -- 172.21.15.151:0/1243000444 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc58400ee60 con 0x5602808e89b0 2024-04-26T01:54:06.383 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.383+0000 7fc577000640 1 -- 172.21.15.151:0/1243000444 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5840107d0 con 0x5602808e89b0 2024-04-26T01:54:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.384+0000 7fc593788740 1 -- 172.21.15.151:0/1243000444 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5602808e89b0 msgr2=0x5602808e8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.384+0000 7fc593788740 1 --2- 172.21.15.151:0/1243000444 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5602808e89b0 0x5602808e8d90 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc584009ea0 tx=0x7fc584009f90 comp rx=0 tx=0).stop 2024-04-26T01:54:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.384+0000 7fc593788740 1 -- 172.21.15.151:0/1243000444 shutdown_connections 2024-04-26T01:54:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.384+0000 7fc593788740 1 --2- 172.21.15.151:0/1243000444 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602808f3b90 0x5602808f5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.384+0000 7fc593788740 1 --2- 172.21.15.151:0/1243000444 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602808e92d0 0x5602808f3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.384+0000 7fc593788740 1 --2- 172.21.15.151:0/1243000444 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5602808e89b0 0x5602808e8d90 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.384+0000 7fc593788740 1 -- 172.21.15.151:0/1243000444 >> 172.21.15.151:0/1243000444 conn(0x5602808e4ec0 msgr2=0x5602808e52a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.384+0000 7fc593788740 1 -- 172.21.15.151:0/1243000444 shutdown_connections 2024-04-26T01:54:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.384+0000 7fc593788740 1 -- 172.21.15.151:0/1243000444 wait complete. 2024-04-26T01:54:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.385+0000 7fc593788740 1 -- start start 2024-04-26T01:54:06.388 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.385+0000 7fc593788740 1 Processor -- start 2024-04-26T01:54:06.388 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.385+0000 7fc582a00640 1 --2- 172.21.15.151:0/4198815437 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fc5700a3e50 0x7fc5700a6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:06.388 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.385+0000 7fc582a00640 1 -- 172.21.15.151:0/4198815437 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fc5780d03a0 con 0x7fc5881070a0 2024-04-26T01:54:06.388 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.385+0000 7fc58e800640 1 --2- 172.21.15.151:0/4198815437 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fc5700a3e50 0x7fc5700a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:06.388 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.386+0000 7fc593788740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602808e89b0 0x5602808f0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:06.388 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.386+0000 7fc593788740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602808e92d0 0x5602808f10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:06.388 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.386+0000 7fc593788740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5602808f3b90 0x5602808f1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:06.388 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.386+0000 7fc593788740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5602808f67c0 con 0x5602808f3b90 2024-04-26T01:54:06.388 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.386+0000 7fc593788740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560280808310 con 0x5602808e92d0 2024-04-26T01:54:06.388 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.386+0000 7fc593788740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5602808f6940 con 0x5602808e89b0 2024-04-26T01:54:06.388 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.386+0000 7fc593788740 10 client.0 _ll_register_callbacks cb 0x5602807d6cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:54:06.388 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.386+0000 7fc58de00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602808e89b0 0x5602808f0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:06.389 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.386+0000 7fc58de00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602808e89b0 0x5602808f0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:39642/0 (socket says 172.21.15.151:39642) 2024-04-26T01:54:06.389 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.386+0000 7fc58de00640 1 -- 172.21.15.151:0/475848099 learned_addr learned my addr 172.21.15.151:0/475848099 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:54:06.389 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.386+0000 7fc58d400640 1 --2- 172.21.15.151:0/475848099 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602808e92d0 0x5602808f10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:06.389 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.386+0000 7fc58e800640 1 --2- 172.21.15.151:0/475848099 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5602808f3b90 0x5602808f1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:06.389 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.387+0000 7fc58de00640 1 -- 172.21.15.151:0/475848099 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602808e92d0 msgr2=0x5602808f10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:06.389 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.387+0000 7fc58de00640 1 --2- 172.21.15.151:0/475848099 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602808e92d0 0x5602808f10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:06.389 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.387+0000 7fc58de00640 1 -- 172.21.15.151:0/475848099 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5602808f3b90 msgr2=0x5602808f1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:06.389 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.387+0000 7fc58de00640 1 --2- 172.21.15.151:0/475848099 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5602808f3b90 0x5602808f1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:06.389 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.387+0000 7fc58de00640 1 -- 172.21.15.151:0/475848099 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56028062d620 con 0x5602808e89b0 2024-04-26T01:54:06.389 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.387+0000 7fc58e800640 1 --2- 172.21.15.151:0/4198815437 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fc5700a3e50 0x7fc5700a6310 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc584009540 tx=0x7fc584005fd0 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:06.389 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.387+0000 7fc58d400640 1 --2- 172.21.15.151:0/475848099 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602808e92d0 0x5602808f10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:54:06.389 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.387+0000 7fc58de00640 1 --2- 172.21.15.151:0/475848099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602808e89b0 0x5602808f0bb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc57c009230 tx=0x7fc57c004580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ba517ca78e6e4a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:06.390 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.387+0000 7fc56b400640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-26T01:54:06.390 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.387+0000 7fc56b400640 1 -- 172.21.15.151:0/475848099 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57c035040 con 0x5602808e89b0 2024-04-26T01:54:06.390 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.387+0000 7fc56b400640 1 -- 172.21.15.151:0/475848099 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fc57c005590 con 0x5602808e89b0 2024-04-26T01:54:06.390 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.387+0000 7fc593788740 1 -- 172.21.15.151:0/475848099 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc58400ee60 con 0x5602808e89b0 2024-04-26T01:54:06.390 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.388+0000 7fc56b400640 1 -- 172.21.15.151:0/475848099 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc57c004770 con 0x5602808e89b0 2024-04-26T01:54:06.390 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.388+0000 7fc593788740 10 client.25213 fetch_fsmap learned FSMap version 26 2024-04-26T01:54:06.390 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.388+0000 7fc593788740 1 -- 172.21.15.151:0/475848099 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560280902a80 con 0x5602808e89b0 2024-04-26T01:54:06.390 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.388+0000 7fc56b400640 1 -- 172.21.15.151:0/475848099 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc58400ee60 con 0x5602808e89b0 2024-04-26T01:54:06.390 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.388+0000 7fc56b400640 1 -- 172.21.15.151:0/475848099 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fc57c04e050 con 0x5602808e89b0 2024-04-26T01:54:06.390 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.388+0000 7fc593788740 10 client.25213 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T01:54:06.390 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.388+0000 7fc593788740 10 client.25213 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:54:06.390 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.388+0000 7fc593788740 20 client.25213 populate_metadata read hostname 'smithi151' 2024-04-26T01:54:06.390 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.388+0000 7fc593788740 1 --2- 172.21.15.151:0/475848099 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x560280903ef0 0x5602809062e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:06.391 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.388+0000 7fc593788740 4 client.25213 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:54:06.391 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.388+0000 7fc593788740 1 -- 172.21.15.151:0/475848099 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5602808628b0 con 0x560280903ef0 2024-04-26T01:54:06.392 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.391+0000 7fc58d400640 1 --2- 172.21.15.151:0/475848099 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x560280903ef0 0x5602809062e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:06.392 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.392+0000 7fc58d400640 1 --2- 172.21.15.151:0/475848099 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x560280903ef0 0x5602809062e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83c633d983d59c15 server_cookie=10403ca7119b4f98 in_seq=0 out_seq=0 2024-04-26T01:54:06.392 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.392+0000 7fc56b400640 10 client.25213 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:54:06.392 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.392+0000 7fc56b400640 1 -- 172.21.15.151:0/475848099 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5602808628b0 con 0x560280903ef0 2024-04-26T01:54:06.392 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.392+0000 7fc56b400640 10 client.25213 handle_command_reply: tid=0 2024-04-26T01:54:06.392 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.392+0000 7fc593788740 1 client.25213 shutdown 2024-04-26T01:54:06.392 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.393+0000 7fc593788740 1 -- 172.21.15.151:0/475848099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602808e89b0 msgr2=0x5602808f0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:06.392 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.393+0000 7fc593788740 1 --2- 172.21.15.151:0/475848099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602808e89b0 0x5602808f0bb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc57c009230 tx=0x7fc57c004580 comp rx=0 tx=0).stop 2024-04-26T01:54:06.392 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.393+0000 7fc593788740 1 -- 172.21.15.151:0/475848099 shutdown_connections 2024-04-26T01:54:06.393 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.393+0000 7fc593788740 1 --2- 172.21.15.151:0/475848099 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x560280903ef0 0x5602809062e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:06.393 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.393+0000 7fc593788740 1 --2- 172.21.15.151:0/475848099 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5602808f3b90 0x5602808f1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:06.393 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.393+0000 7fc593788740 1 --2- 172.21.15.151:0/475848099 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602808e92d0 0x5602808f10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:06.393 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.393+0000 7fc56b400640 0 client.25213 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:54:06.393 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.393+0000 7fc593788740 1 --2- 172.21.15.151:0/475848099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602808e89b0 0x5602808f0bb0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:06.393 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.393+0000 7fc593788740 1 -- 172.21.15.151:0/475848099 >> 172.21.15.151:0/475848099 conn(0x5602808e4ec0 msgr2=0x5602808eabf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:06.393 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.393+0000 7fc593788740 1 -- 172.21.15.151:0/475848099 shutdown_connections 2024-04-26T01:54:06.393 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.393+0000 7fc593788740 1 -- 172.21.15.151:0/475848099 wait complete. 2024-04-26T01:54:06.393 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.393+0000 7fc593788740 20 client.25213 trim_cache size 0 max 16384 2024-04-26T01:54:06.410 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.410+0000 7fc593788740 1 Processor -- start 2024-04-26T01:54:06.410 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.410+0000 7fc593788740 1 -- start start 2024-04-26T01:54:06.410 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.410+0000 7fc593788740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602808e89b0 0x5602808ff680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:06.410 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.410+0000 7fc593788740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5602808e92d0 0x5602808f1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:06.410 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.410+0000 7fc593788740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602808f3b90 0x56028090bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:06.410 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.410+0000 7fc593788740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc5881b01c0 con 0x5602808e92d0 2024-04-26T01:54:06.410 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.410+0000 7fc593788740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc5881ada90 con 0x5602808f3b90 2024-04-26T01:54:06.411 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.410+0000 7fc593788740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc5881ab900 con 0x5602808e89b0 2024-04-26T01:54:06.411 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.410+0000 7fc58d400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5602808e92d0 0x5602808f1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:06.411 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.411+0000 7fc58d400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5602808e92d0 0x5602808f1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:39794/0 (socket says 172.21.15.151:39794) 2024-04-26T01:54:06.411 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.411+0000 7fc58d400640 1 -- 172.21.15.151:0/1727414444 learned_addr learned my addr 172.21.15.151:0/1727414444 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:54:06.411 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.411+0000 7fc58de00640 1 --2- 172.21.15.151:0/1727414444 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602808e89b0 0x5602808ff680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:06.411 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.411+0000 7fc58e800640 1 --2- 172.21.15.151:0/1727414444 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602808f3b90 0x56028090bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:06.411 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.411+0000 7fc58de00640 1 -- 172.21.15.151:0/1727414444 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602808f3b90 msgr2=0x56028090bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:06.411 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.411+0000 7fc58de00640 1 --2- 172.21.15.151:0/1727414444 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602808f3b90 0x56028090bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:06.411 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.411+0000 7fc58de00640 1 -- 172.21.15.151:0/1727414444 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5602808e92d0 msgr2=0x5602808f1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:06.411 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.411+0000 7fc58de00640 1 --2- 172.21.15.151:0/1727414444 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5602808e92d0 0x5602808f1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:06.411 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.411+0000 7fc58de00640 1 -- 172.21.15.151:0/1727414444 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602808e9100 con 0x5602808e89b0 2024-04-26T01:54:06.411 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.411+0000 7fc58d400640 1 --2- 172.21.15.151:0/1727414444 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5602808e92d0 0x5602808f1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:54:06.412 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.411+0000 7fc58de00640 1 --2- 172.21.15.151:0/1727414444 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602808e89b0 0x5602808ff680 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc57c02aa00 tx=0x7fc57c031400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b01ac25519306d66 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:06.412 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.411+0000 7fc576600640 1 -- 172.21.15.151:0/1727414444 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57c000e60 con 0x5602808e89b0 2024-04-26T01:54:06.412 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.412+0000 7fc576600640 1 -- 172.21.15.151:0/1727414444 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc57c041070 con 0x5602808e89b0 2024-04-26T01:54:06.412 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.412+0000 7fc576600640 1 -- 172.21.15.151:0/1727414444 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57c035070 con 0x5602808e89b0 2024-04-26T01:54:06.415 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.412+0000 7fc593788740 1 -- 172.21.15.151:0/1727414444 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602808e89b0 msgr2=0x5602808ff680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:06.415 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.412+0000 7fc593788740 1 --2- 172.21.15.151:0/1727414444 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602808e89b0 0x5602808ff680 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc57c02aa00 tx=0x7fc57c031400 comp rx=0 tx=0).stop 2024-04-26T01:54:06.415 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.412+0000 7fc593788740 1 -- 172.21.15.151:0/1727414444 shutdown_connections 2024-04-26T01:54:06.415 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.412+0000 7fc593788740 1 --2- 172.21.15.151:0/1727414444 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602808f3b90 0x56028090bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:06.415 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.412+0000 7fc593788740 1 --2- 172.21.15.151:0/1727414444 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5602808e92d0 0x5602808f1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.412+0000 7fc593788740 1 --2- 172.21.15.151:0/1727414444 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602808e89b0 0x5602808ff680 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.412+0000 7fc593788740 1 -- 172.21.15.151:0/1727414444 >> 172.21.15.151:0/1727414444 conn(0x5602808e4ec0 msgr2=0x5602808eabf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.412+0000 7fc593788740 1 -- 172.21.15.151:0/1727414444 shutdown_connections 2024-04-26T01:54:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.412+0000 7fc593788740 1 -- 172.21.15.151:0/1727414444 wait complete. 2024-04-26T01:54:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.413+0000 7fc593788740 1 -- start start 2024-04-26T01:54:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.413+0000 7fc593788740 1 Processor -- start 2024-04-26T01:54:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.413+0000 7fc593788740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602808e89b0 0x5602808fb900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.413+0000 7fc593788740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602808e92d0 0x5602808fbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.413+0000 7fc593788740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5602808f3b90 0x5602808f8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.413+0000 7fc593788740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc5881abf50 con 0x5602808f3b90 2024-04-26T01:54:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.413+0000 7fc593788740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc5881ad3c0 con 0x5602808e89b0 2024-04-26T01:54:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.413+0000 7fc593788740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc5881ac620 con 0x5602808e92d0 2024-04-26T01:54:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.413+0000 7fc593788740 10 client.0 _ll_register_callbacks cb 0x7fc57c004210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:54:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.414+0000 7fc58de00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602808e89b0 0x5602808fb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.414+0000 7fc58de00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602808e89b0 0x5602808fb900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:45534/0 (socket says 172.21.15.151:45534) 2024-04-26T01:54:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.414+0000 7fc58de00640 1 -- 172.21.15.151:0/2601280537 learned_addr learned my addr 172.21.15.151:0/2601280537 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:54:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.414+0000 7fc58e800640 1 --2- 172.21.15.151:0/2601280537 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5602808f3b90 0x5602808f8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.414+0000 7fc58d400640 1 --2- 172.21.15.151:0/2601280537 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602808e92d0 0x5602808fbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.414+0000 7fc58de00640 1 -- 172.21.15.151:0/2601280537 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602808e92d0 msgr2=0x5602808fbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.414+0000 7fc58de00640 1 --2- 172.21.15.151:0/2601280537 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602808e92d0 0x5602808fbe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.414+0000 7fc58de00640 1 -- 172.21.15.151:0/2601280537 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5602808f3b90 msgr2=0x5602808f8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.414+0000 7fc58de00640 1 --2- 172.21.15.151:0/2601280537 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5602808f3b90 0x5602808f8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.414+0000 7fc58de00640 1 -- 172.21.15.151:0/2601280537 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602808f9640 con 0x5602808e89b0 2024-04-26T01:54:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.414+0000 7fc58e800640 1 --2- 172.21.15.151:0/2601280537 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5602808f3b90 0x5602808f8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:54:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.414+0000 7fc58de00640 1 --2- 172.21.15.151:0/2601280537 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602808e89b0 0x5602808fb900 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc57c02a450 tx=0x7fc57c0049b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=628c0d7ea7654f65 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.414+0000 7fc56b400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T01:54:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.415+0000 7fc56b400640 1 -- 172.21.15.151:0/2601280537 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57c035070 con 0x5602808e89b0 2024-04-26T01:54:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.415+0000 7fc593788740 1 -- 172.21.15.151:0/2601280537 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc57c041070 con 0x5602808e89b0 2024-04-26T01:54:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.415+0000 7fc56b400640 1 -- 172.21.15.151:0/2601280537 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fc57c004cb0 con 0x5602808e89b0 2024-04-26T01:54:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.415+0000 7fc56b400640 1 -- 172.21.15.151:0/2601280537 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc57c066050 con 0x5602808e89b0 2024-04-26T01:54:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.415+0000 7fc593788740 10 client.25202 fetch_fsmap learned FSMap version 26 2024-04-26T01:54:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.415+0000 7fc593788740 1 -- 172.21.15.151:0/2601280537 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560280969f40 con 0x5602808e89b0 2024-04-26T01:54:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.415+0000 7fc56b400640 1 -- 172.21.15.151:0/2601280537 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc57c041070 con 0x5602808e89b0 2024-04-26T01:54:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.415+0000 7fc56b400640 1 -- 172.21.15.151:0/2601280537 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fc57c04e040 con 0x5602808e89b0 2024-04-26T01:54:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.415+0000 7fc593788740 10 client.25202 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T01:54:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.415+0000 7fc593788740 10 client.25202 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:54:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.415+0000 7fc593788740 20 client.25202 populate_metadata read hostname 'smithi151' 2024-04-26T01:54:06.419 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.415+0000 7fc593788740 1 --2- 172.21.15.151:0/2601280537 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x56028096a5a0 0x56028096c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:06.419 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.415+0000 7fc593788740 4 client.25202 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:54:06.419 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.415+0000 7fc593788740 1 -- 172.21.15.151:0/2601280537 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56028085c320 con 0x56028096a5a0 2024-04-26T01:54:06.419 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.416+0000 7fc58d400640 1 --2- 172.21.15.151:0/2601280537 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x56028096a5a0 0x56028096c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:06.419 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.416+0000 7fc58d400640 1 --2- 172.21.15.151:0/2601280537 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x56028096a5a0 0x56028096c990 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b0b2084c2dc354f server_cookie=2737f34077a6fdac in_seq=0 out_seq=0 2024-04-26T01:54:06.419 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.416+0000 7fc56b400640 10 client.25202 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:54:06.422 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.423+0000 7fc56b400640 1 -- 172.21.15.151:0/2601280537 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56028085c320 con 0x56028096a5a0 2024-04-26T01:54:06.422 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.423+0000 7fc56b400640 10 client.25202 handle_command_reply: tid=0 2024-04-26T01:54:06.422 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.423+0000 7fc593788740 1 client.25202 shutdown 2024-04-26T01:54:06.422 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.423+0000 7fc593788740 1 -- 172.21.15.151:0/2601280537 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602808e89b0 msgr2=0x5602808fb900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:06.422 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.423+0000 7fc593788740 1 --2- 172.21.15.151:0/2601280537 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602808e89b0 0x5602808fb900 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc57c02a450 tx=0x7fc57c0049b0 comp rx=0 tx=0).stop 2024-04-26T01:54:06.422 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.423+0000 7fc593788740 1 -- 172.21.15.151:0/2601280537 shutdown_connections 2024-04-26T01:54:06.423 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.423+0000 7fc593788740 1 --2- 172.21.15.151:0/2601280537 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x56028096a5a0 0x56028096c990 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:06.423 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.423+0000 7fc593788740 1 --2- 172.21.15.151:0/2601280537 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5602808f3b90 0x5602808f8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:06.423 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.423+0000 7fc593788740 1 --2- 172.21.15.151:0/2601280537 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602808e92d0 0x5602808fbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:06.423 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.423+0000 7fc593788740 1 --2- 172.21.15.151:0/2601280537 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602808e89b0 0x5602808fb900 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:06.423 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.423+0000 7fc56b400640 0 client.25202 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:54:06.423 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.423+0000 7fc593788740 1 -- 172.21.15.151:0/2601280537 >> 172.21.15.151:0/2601280537 conn(0x5602808e4ec0 msgr2=0x5602808e3790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:06.423 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.423+0000 7fc593788740 1 -- 172.21.15.151:0/2601280537 shutdown_connections 2024-04-26T01:54:06.423 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.423+0000 7fc593788740 1 -- 172.21.15.151:0/2601280537 wait complete. 2024-04-26T01:54:06.423 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.424+0000 7fc593788740 20 client.25202 trim_cache size 0 max 16384 2024-04-26T01:54:06.424 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:{ 2024-04-26T01:54:06.424 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "message": "", 2024-04-26T01:54:06.424 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "return_code": 0 2024-04-26T01:54:06.424 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:} 2024-04-26T01:54:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.424+0000 7fc58f200640 1 -- 172.21.15.151:0/4198815437 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fc5700a3e50 msgr2=0x7fc5700a6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.424+0000 7fc58f200640 1 --2- 172.21.15.151:0/4198815437 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fc5700a3e50 0x7fc5700a6310 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc584009540 tx=0x7fc584005fd0 comp rx=0 tx=0).stop 2024-04-26T01:54:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.424+0000 7fc58f200640 1 -- 172.21.15.151:0/4198815437 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc5881070a0 msgr2=0x7fc588198b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.424+0000 7fc58f200640 1 --2- 172.21.15.151:0/4198815437 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc5881070a0 0x7fc588198b70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc57800b470 tx=0x7fc57800b930 comp rx=0 tx=0).stop 2024-04-26T01:54:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.424+0000 7fc58f200640 1 -- 172.21.15.151:0/4198815437 shutdown_connections 2024-04-26T01:54:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.424+0000 7fc58f200640 1 --2- 172.21.15.151:0/4198815437 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fc5700a3e50 0x7fc5700a6310 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.424+0000 7fc58f200640 1 --2- 172.21.15.151:0/4198815437 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc58819cef0 0x7fc58819d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.424+0000 7fc58f200640 1 --2- 172.21.15.151:0/4198815437 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc5881070a0 0x7fc588198b70 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.424+0000 7fc58f200640 1 --2- 172.21.15.151:0/4198815437 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc588101fd0 0x7fc588198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.424+0000 7fc58f200640 1 -- 172.21.15.151:0/4198815437 >> 172.21.15.151:0/4198815437 conn(0x7fc5880fd890 msgr2=0x7fc5880fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:06.426 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.425+0000 7fc58f200640 1 -- 172.21.15.151:0/4198815437 shutdown_connections 2024-04-26T01:54:06.426 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:06.425+0000 7fc58f200640 1 -- 172.21.15.151:0/4198815437 wait complete. 2024-04-26T01:54:06.745 INFO:teuthology.orchestra.run.smithi151.stdout:{"run_dir":"/var/run/ceph"} 2024-04-26T01:54:06.746 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-04-26T01:54:06.746 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 3448 2024-04-26T01:54:06.746 DEBUG:tasks.quiescer:Yielding 2024-04-26T01:54:06.747 INFO:teuthology.run_tasks:Running task workunit... 2024-04-26T01:54:06.761 INFO:tasks.workunit:Pulling workunits from ref afa1933b0fdbb6c99c947d1eda34d661d23cd327 2024-04-26T01:54:06.762 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-26T01:54:06.762 INFO:tasks.workunit:timeout=3h 2024-04-26T01:54:06.762 INFO:tasks.workunit:cleanup=True 2024-04-26T01:54:06.762 DEBUG:teuthology.orchestra.run.smithi151:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-26T01:54:06.763 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:54:06.816 INFO:teuthology.orchestra.run.smithi151.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-26T01:54:06.816 INFO:teuthology.orchestra.run.smithi151.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-26T01:54:06.816 INFO:teuthology.orchestra.run.smithi151.stdout:Device: 72h/114d Inode: 1099511628283 Links: 2 2024-04-26T01:54:06.816 INFO:teuthology.orchestra.run.smithi151.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T01:54:06.816 INFO:teuthology.orchestra.run.smithi151.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-26T01:54:06.817 INFO:teuthology.orchestra.run.smithi151.stdout:Access: 2024-04-26 01:50:21.314695252 +0000 2024-04-26T01:54:06.817 INFO:teuthology.orchestra.run.smithi151.stdout:Modify: 2024-04-26 01:50:21.314695252 +0000 2024-04-26T01:54:06.817 INFO:teuthology.orchestra.run.smithi151.stdout:Change: 2024-04-26 01:53:35.396362772 +0000 2024-04-26T01:54:06.817 INFO:teuthology.orchestra.run.smithi151.stdout: Birth: 2024-04-26 01:50:21.314695252 +0000 2024-04-26T01:54:06.817 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-26T01:54:06.817 DEBUG:teuthology.orchestra.run.smithi151:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-26T01:54:06.941 DEBUG:teuthology.orchestra.run.smithi151:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout afa1933b0fdbb6c99c947d1eda34d661d23cd327 2024-04-26T01:54:06.991 INFO:tasks.workunit.client.0.smithi151.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-26T01:54:07.209 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:54:08.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:54:07 smithi165 ceph-mon[32648]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-04-26T01:54:08.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:54:07 smithi151 ceph-mon[28057]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-04-26T01:54:08.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:54:07 smithi172 ceph-mon[32785]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-04-26T01:54:09.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:09.689+0000 7fe7ea600700 1 -- 172.21.15.151:0/2598074185 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7e4102a90 msgr2=0x7fe7e4102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:09.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:09.689+0000 7fe7ea600700 1 --2- 172.21.15.151:0/2598074185 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7e4102a90 0x7fe7e4102f10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0009990 tx=0x7fe7d0009ca0 comp rx=0 tx=0).stop 2024-04-26T01:54:09.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:09.689+0000 7fe7ea600700 1 -- 172.21.15.151:0/2598074185 shutdown_connections 2024-04-26T01:54:09.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:09.689+0000 7fe7ea600700 1 --2- 172.21.15.151:0/2598074185 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe7e4103450 0x7fe7e410f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:09.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:09.689+0000 7fe7ea600700 1 --2- 172.21.15.151:0/2598074185 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7e4108a50 0x7fe7e4108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:09.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:09.689+0000 7fe7ea600700 1 --2- 172.21.15.151:0/2598074185 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7e4102a90 0x7fe7e4102f10 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:09.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:09.689+0000 7fe7ea600700 1 -- 172.21.15.151:0/2598074185 >> 172.21.15.151:0/2598074185 conn(0x7fe7e4079ba0 msgr2=0x7fe7e4079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:09.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:09.689+0000 7fe7ea600700 1 -- 172.21.15.151:0/2598074185 shutdown_connections 2024-04-26T01:54:09.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:09.689+0000 7fe7ea600700 1 -- 172.21.15.151:0/2598074185 wait complete. 2024-04-26T01:54:09.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:09.691+0000 7fe7ea600700 1 Processor -- start 2024-04-26T01:54:09.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:09.691+0000 7fe7ea600700 1 -- start start 2024-04-26T01:54:09.690 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:09.691+0000 7fe7ea600700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7e4102a90 0x7fe7e41991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:09.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:09.691+0000 7fe7ea600700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7e4103450 0x7fe7e41996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:09.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:09.691+0000 7fe7ea600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe7e4108a50 0x7fe7e419da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:09.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:09.691+0000 7fe7ea600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e4112c50 con 0x7fe7e4108a50 2024-04-26T01:54:09.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:09.691+0000 7fe7ea600700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e4112ad0 con 0x7fe7e4102a90 2024-04-26T01:54:09.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:09.691+0000 7fe7ea600700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e4112dd0 con 0x7fe7e4103450 2024-04-26T01:54:09.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:09.692+0000 7fe7e3400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7e4102a90 0x7fe7e41991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:09.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:09.692+0000 7fe7e2a00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7e4103450 0x7fe7e41996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:09.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:09.692+0000 7fe7e3400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7e4102a90 0x7fe7e41991a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:45548/0 (socket says 172.21.15.151:45548) 2024-04-26T01:54:09.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:09.692+0000 7fe7e3400700 1 -- 172.21.15.151:0/2860157926 learned_addr learned my addr 172.21.15.151:0/2860157926 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:54:09.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:09.692+0000 7fe7e2a00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7e4103450 0x7fe7e41996e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:39672/0 (socket says 172.21.15.151:39672) 2024-04-26T01:54:09.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:09.692+0000 7fe7e3e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe7e4108a50 0x7fe7e419da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:09.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:09.692+0000 7fe7e3400700 1 -- 172.21.15.151:0/2860157926 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7e4103450 msgr2=0x7fe7e41996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:09.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:09.692+0000 7fe7e3400700 1 --2- 172.21.15.151:0/2860157926 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7e4103450 0x7fe7e41996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:09.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:09.692+0000 7fe7e3400700 1 -- 172.21.15.151:0/2860157926 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe7e4108a50 msgr2=0x7fe7e419da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:09.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:09.692+0000 7fe7e3400700 1 --2- 172.21.15.151:0/2860157926 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe7e4108a50 0x7fe7e419da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:09.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:09.692+0000 7fe7e3400700 1 -- 172.21.15.151:0/2860157926 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7e419e1b0 con 0x7fe7e4102a90 2024-04-26T01:54:09.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:09.692+0000 7fe7e3e00700 1 --2- 172.21.15.151:0/2860157926 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe7e4108a50 0x7fe7e419da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:54:09.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:09.692+0000 7fe7e2a00700 1 --2- 172.21.15.151:0/2860157926 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7e4103450 0x7fe7e41996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:54:09.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:09.693+0000 7fe7e3400700 1 --2- 172.21.15.151:0/2860157926 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7e4102a90 0x7fe7e41991a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe7d400c600 tx=0x7fe7d400c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:09.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:09.693+0000 7fe7dbe00700 1 -- 172.21.15.151:0/2860157926 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d400ce70 con 0x7fe7e4102a90 2024-04-26T01:54:09.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:09.693+0000 7fe7ea600700 1 -- 172.21.15.151:0/2860157926 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7e419e4a0 con 0x7fe7e4102a90 2024-04-26T01:54:09.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:09.693+0000 7fe7dbe00700 1 -- 172.21.15.151:0/2860157926 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7d4015d60 con 0x7fe7e4102a90 2024-04-26T01:54:09.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:09.693+0000 7fe7dbe00700 1 -- 172.21.15.151:0/2860157926 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d401e410 con 0x7fe7e4102a90 2024-04-26T01:54:09.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:09.693+0000 7fe7ea600700 1 -- 172.21.15.151:0/2860157926 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7e4074c70 con 0x7fe7e4102a90 2024-04-26T01:54:09.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:09.694+0000 7fe7ea600700 1 -- 172.21.15.151:0/2860157926 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7e4069730 con 0x7fe7e4102a90 2024-04-26T01:54:09.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:09.698+0000 7fe7dbe00700 1 -- 172.21.15.151:0/2860157926 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 26) ==== 138931+0+0 (secure 0 0 0) 0x7fe7d4005020 con 0x7fe7e4102a90 2024-04-26T01:54:09.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:09.699+0000 7fe7dbe00700 1 --2- 172.21.15.151:0/2860157926 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fe7c80a3f60 0x7fe7c80a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:09.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:09.699+0000 7fe7dbe00700 1 -- 172.21.15.151:0/2860157926 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fe7d40d8a20 con 0x7fe7e4102a90 2024-04-26T01:54:09.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:09.700+0000 7fe7e2a00700 1 --2- 172.21.15.151:0/2860157926 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fe7c80a3f60 0x7fe7c80a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:09.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:09.700+0000 7fe7e2a00700 1 --2- 172.21.15.151:0/2860157926 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fe7c80a3f60 0x7fe7c80a6420 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe7e407a2e0 tx=0x7fe7d001a040 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:09.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:09.704+0000 7fe7dbe00700 1 -- 172.21.15.151:0/2860157926 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe7d40a4ef0 con 0x7fe7e4102a90 2024-04-26T01:54:09.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:54:09 smithi151 ceph-mon[28057]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 26 KiB/s wr, 28 op/s 2024-04-26T01:54:10.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:10.007+0000 7fe7ea600700 1 -- 172.21.15.151:0/2860157926 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe7e402df80 con 0x7fe7e4102a90 2024-04-26T01:54:10.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:10.009+0000 7fe7dbe00700 1 -- 172.21.15.151:0/2860157926 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11889 (secure 0 0 0) 0x7fe7d40a47f0 con 0x7fe7e4102a90 2024-04-26T01:54:10.007 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:54:10.008 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2937864097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2937864097},{"type":"v1","addr":"172.21.15.151:6835","nonce":2937864097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14373,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/2947090802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2947090802},{"type":"v1","addr":"172.21.15.151:6837","nonce":2947090802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/1356563280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1356563280},{"type":"v1","addr":"172.21.15.151:6839","nonce":1356563280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6841/1075240552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1075240552},{"type":"v1","addr":"172.21.15.151:6841","nonce":1075240552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/504675912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":504675912},{"type":"v1","addr":"172.21.15.165:6833","nonce":504675912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1207580015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1207580015},{"type":"v1","addr":"172.21.15.172:6833","nonce":1207580015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:49:40.746876+0000","modified":"2024-04-26T01:53:36.483847+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24505,"mds_2":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/3598138294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3598138294},{"type":"v1","addr":"172.21.15.165:6835","nonce":3598138294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/1847859054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":1847859054},{"type":"v1","addr":"172.21.15.165:6837","nonce":1847859054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/2309751171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2309751171},{"type":"v1","addr":"172.21.15.172:6835","nonce":2309751171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.172:6837/55972537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":55972537},{"type":"v1","addr":"172.21.15.172:6837","nonce":55972537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.172:6839/1242302810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1242302810},{"type":"v1","addr":"172.21.15.172:6839","nonce":1242302810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/1969716254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1969716254},{"type":"v1","addr":"172.21.15.165:6839","nonce":1969716254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24493,24505,24509]},"id":1}]} 2024-04-26T01:54:10.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:10.012+0000 7fe7ea600700 1 -- 172.21.15.151:0/2860157926 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fe7c80a3f60 msgr2=0x7fe7c80a6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:10.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:10.012+0000 7fe7ea600700 1 --2- 172.21.15.151:0/2860157926 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fe7c80a3f60 0x7fe7c80a6420 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe7e407a2e0 tx=0x7fe7d001a040 comp rx=0 tx=0).stop 2024-04-26T01:54:10.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:10.012+0000 7fe7ea600700 1 -- 172.21.15.151:0/2860157926 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7e4102a90 msgr2=0x7fe7e41991a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:10.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:10.012+0000 7fe7ea600700 1 --2- 172.21.15.151:0/2860157926 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7e4102a90 0x7fe7e41991a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe7d400c600 tx=0x7fe7d400c910 comp rx=0 tx=0).stop 2024-04-26T01:54:10.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:10.012+0000 7fe7ea600700 1 -- 172.21.15.151:0/2860157926 shutdown_connections 2024-04-26T01:54:10.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:10.012+0000 7fe7ea600700 1 --2- 172.21.15.151:0/2860157926 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fe7c80a3f60 0x7fe7c80a6420 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:10.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:10.012+0000 7fe7ea600700 1 --2- 172.21.15.151:0/2860157926 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe7e4108a50 0x7fe7e419da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:10.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:10.012+0000 7fe7ea600700 1 --2- 172.21.15.151:0/2860157926 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7e4102a90 0x7fe7e41991a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:10.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:10.012+0000 7fe7ea600700 1 --2- 172.21.15.151:0/2860157926 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7e4103450 0x7fe7e41996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:10.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:10.012+0000 7fe7ea600700 1 -- 172.21.15.151:0/2860157926 >> 172.21.15.151:0/2860157926 conn(0x7fe7e4079ba0 msgr2=0x7fe7e4101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:10.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:10.012+0000 7fe7ea600700 1 -- 172.21.15.151:0/2860157926 shutdown_connections 2024-04-26T01:54:10.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:10.012+0000 7fe7ea600700 1 -- 172.21.15.151:0/2860157926 wait complete. 2024-04-26T01:54:10.012 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 26 2024-04-26T01:54:10.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:54:09 smithi165 ceph-mon[32648]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 26 KiB/s wr, 28 op/s 2024-04-26T01:54:10.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:54:09 smithi172 ceph-mon[32785]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 26 KiB/s wr, 28 op/s 2024-04-26T01:54:10.898 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-26T01:49:40.746876+0000', 'modified': '2024-04-26T01:53:36.483847+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24509, 'mds_1': 24505, 'mds_2': 24493}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24458': {'gid': 24458, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.165:6835/3598138294', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6834', 'nonce': 3598138294}, {'type': 'v1', 'addr': '172.21.15.165:6835', 'nonce': 3598138294}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24470': {'gid': 24470, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.165:6837/1847859054', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6836', 'nonce': 1847859054}, {'type': 'v1', 'addr': '172.21.15.165:6837', 'nonce': 1847859054}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24481': {'gid': 24481, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.172:6835/2309751171', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6834', 'nonce': 2309751171}, {'type': 'v1', 'addr': '172.21.15.172:6835', 'nonce': 2309751171}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24493': {'gid': 24493, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.172:6837/55972537', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6836', 'nonce': 55972537}, {'type': 'v1', 'addr': '172.21.15.172:6837', 'nonce': 55972537}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24505': {'gid': 24505, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.172:6839/1242302810', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6838', 'nonce': 1242302810}, {'type': 'v1', 'addr': '172.21.15.172:6839', 'nonce': 1242302810}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24509': {'gid': 24509, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.165:6839/1969716254', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6838', 'nonce': 1969716254}, {'type': 'v1', 'addr': '172.21.15.165:6839', 'nonce': 1969716254}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24509, 'qdb_cluster': [24493, 24505, 24509]} 2024-04-26T01:54:10.898 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-26T01:54:10.899 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph tell mds.c status 2024-04-26T01:54:11.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:54:10 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/2860157926' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:54:11.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:54:10 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/2860157926' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:54:11.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:54:10 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/2860157926' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:54:11.378 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:54:12.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:54:11 smithi165 ceph-mon[32648]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-04-26T01:54:12.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:54:11 smithi151 ceph-mon[28057]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-04-26T01:54:12.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:54:11 smithi172 ceph-mon[32785]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-04-26T01:54:13.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.794+0000 7f0d03200700 1 -- 172.21.15.151:0/462399078 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d04076f70 msgr2=0x7f0d04077350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:13.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.794+0000 7f0d03200700 1 --2- 172.21.15.151:0/462399078 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d04076f70 0x7f0d04077350 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4009990 tx=0x7f0cf4009ca0 comp rx=0 tx=0).stop 2024-04-26T01:54:13.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.794+0000 7f0d03200700 1 -- 172.21.15.151:0/462399078 shutdown_connections 2024-04-26T01:54:13.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.794+0000 7f0d03200700 1 --2- 172.21.15.151:0/462399078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d041137f0 0x7f0d04115be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:13.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.794+0000 7f0d03200700 1 --2- 172.21.15.151:0/462399078 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d04076f70 0x7f0d04077350 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:13.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.794+0000 7f0d03200700 1 --2- 172.21.15.151:0/462399078 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d04077890 0x7f0d041132b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:13.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.794+0000 7f0d03200700 1 -- 172.21.15.151:0/462399078 >> 172.21.15.151:0/462399078 conn(0x7f0d04100620 msgr2=0x7f0d04102a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:13.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.794+0000 7f0d03200700 1 -- 172.21.15.151:0/462399078 shutdown_connections 2024-04-26T01:54:13.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.795+0000 7f0d03200700 1 -- 172.21.15.151:0/462399078 wait complete. 2024-04-26T01:54:13.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.795+0000 7f0d03200700 1 Processor -- start 2024-04-26T01:54:13.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.796+0000 7f0d03200700 1 -- start start 2024-04-26T01:54:13.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.796+0000 7f0d03200700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d04076f70 0x7f0d0419f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:13.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.796+0000 7f0d03200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d04077890 0x7f0d0419fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:13.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.796+0000 7f0d03200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d041137f0 0x7f0d041a3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:13.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.796+0000 7f0d03200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0d04119160 con 0x7f0d04077890 2024-04-26T01:54:13.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.796+0000 7f0d03200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0d04118fe0 con 0x7f0d041137f0 2024-04-26T01:54:13.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.796+0000 7f0d03200700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0d041192e0 con 0x7f0d04076f70 2024-04-26T01:54:13.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.796+0000 7f0d01400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d04077890 0x7f0d0419fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:13.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.796+0000 7f0d01400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d04077890 0x7f0d0419fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36392/0 (socket says 172.21.15.151:36392) 2024-04-26T01:54:13.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.796+0000 7f0d01400700 1 -- 172.21.15.151:0/1371433506 learned_addr learned my addr 172.21.15.151:0/1371433506 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:54:13.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.797+0000 7f0d01e00700 1 --2- 172.21.15.151:0/1371433506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d04076f70 0x7f0d0419f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:13.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.797+0000 7f0d02800700 1 --2- 172.21.15.151:0/1371433506 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d041137f0 0x7f0d041a3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:13.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.797+0000 7f0d01400700 1 -- 172.21.15.151:0/1371433506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d04076f70 msgr2=0x7f0d0419f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:13.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.797+0000 7f0d01400700 1 --2- 172.21.15.151:0/1371433506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d04076f70 0x7f0d0419f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:13.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.797+0000 7f0d01400700 1 -- 172.21.15.151:0/1371433506 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d041137f0 msgr2=0x7f0d041a3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:13.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.797+0000 7f0d01400700 1 --2- 172.21.15.151:0/1371433506 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d041137f0 0x7f0d041a3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:13.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.797+0000 7f0d01400700 1 -- 172.21.15.151:0/1371433506 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d041a4610 con 0x7f0d04077890 2024-04-26T01:54:13.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.797+0000 7f0d01400700 1 --2- 172.21.15.151:0/1371433506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d04077890 0x7f0d0419fb90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0cf000e9a0 tx=0x7f0cf000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:13.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.798+0000 7f0cfaa00700 1 -- 172.21.15.151:0/1371433506 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf000ca10 con 0x7f0d04077890 2024-04-26T01:54:13.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.798+0000 7f0cfaa00700 1 -- 172.21.15.151:0/1371433506 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cf000cbb0 con 0x7f0d04077890 2024-04-26T01:54:13.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.798+0000 7f0cfaa00700 1 -- 172.21.15.151:0/1371433506 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf0018680 con 0x7f0d04077890 2024-04-26T01:54:13.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.798+0000 7f0d03200700 1 -- 172.21.15.151:0/1371433506 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d041a4900 con 0x7f0d04077890 2024-04-26T01:54:13.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.798+0000 7f0d03200700 1 -- 172.21.15.151:0/1371433506 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d041ac180 con 0x7f0d04077890 2024-04-26T01:54:13.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.801+0000 7f0cfaa00700 1 -- 172.21.15.151:0/1371433506 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 138931+0+0 (secure 0 0 0) 0x7f0cf0010430 con 0x7f0d04077890 2024-04-26T01:54:13.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.803+0000 7f0cfaa00700 1 --2- 172.21.15.151:0/1371433506 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f0ce80a3f60 0x7f0ce80a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:13.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.803+0000 7f0cfaa00700 1 -- 172.21.15.151:0/1371433506 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f0cf00d0be0 con 0x7f0d04077890 2024-04-26T01:54:13.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.803+0000 7f0d01e00700 1 --2- 172.21.15.151:0/1371433506 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f0ce80a3f60 0x7f0ce80a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:13.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.804+0000 7f0d01e00700 1 --2- 172.21.15.151:0/1371433506 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f0ce80a3f60 0x7f0ce80a6420 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0cf40098e0 tx=0x7f0cf4005960 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:13.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.804+0000 7f0d13a12b80 1 Processor -- start 2024-04-26T01:54:13.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.805+0000 7f0d13a12b80 1 -- start start 2024-04-26T01:54:13.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.805+0000 7f0d13a12b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cc2acb3e0 0x556cc2acb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:13.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.805+0000 7f0d13a12b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556cc2acbd00 0x556cc2ad6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:13.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.805+0000 7f0d13a12b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556cc2ad6c40 0x556cc2ad9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:13.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.805+0000 7f0d13a12b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x556cc2a1ee40 con 0x556cc2acbd00 2024-04-26T01:54:13.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.805+0000 7f0d13a12b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556cc29c56e0 con 0x556cc2ad6c40 2024-04-26T01:54:13.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.805+0000 7f0d13a12b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556cc2a1ae00 con 0x556cc2acb3e0 2024-04-26T01:54:13.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.806+0000 7f0d01400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556cc2acbd00 0x556cc2ad6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:13.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.806+0000 7f0d01e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cc2acb3e0 0x556cc2acb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:13.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.806+0000 7f0d01400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556cc2acbd00 0x556cc2ad6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36408/0 (socket says 172.21.15.151:36408) 2024-04-26T01:54:13.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.806+0000 7f0d01400700 1 -- 172.21.15.151:0/1388113209 learned_addr learned my addr 172.21.15.151:0/1388113209 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:54:13.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.806+0000 7f0d02800700 1 --2- 172.21.15.151:0/1388113209 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556cc2ad6c40 0x556cc2ad9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:13.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.806+0000 7f0d01e00700 1 -- 172.21.15.151:0/1388113209 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556cc2ad6c40 msgr2=0x556cc2ad9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:13.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.806+0000 7f0d01e00700 1 --2- 172.21.15.151:0/1388113209 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556cc2ad6c40 0x556cc2ad9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:13.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.806+0000 7f0d01e00700 1 -- 172.21.15.151:0/1388113209 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556cc2acbd00 msgr2=0x556cc2ad6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:13.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.806+0000 7f0d01e00700 1 --2- 172.21.15.151:0/1388113209 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556cc2acbd00 0x556cc2ad6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:13.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.806+0000 7f0d01e00700 1 -- 172.21.15.151:0/1388113209 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cc29bb630 con 0x556cc2acb3e0 2024-04-26T01:54:13.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.806+0000 7f0d01400700 1 --2- 172.21.15.151:0/1388113209 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556cc2acbd00 0x556cc2ad6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:54:13.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.806+0000 7f0d01e00700 1 --2- 172.21.15.151:0/1388113209 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cc2acb3e0 0x556cc2acb7c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4006010 tx=0x7f0cf401f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4049e91467dc4f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:13.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.807+0000 7f0cef400700 1 -- 172.21.15.151:0/1388113209 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf400bd80 con 0x556cc2acb3e0 2024-04-26T01:54:13.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.807+0000 7f0cef400700 1 -- 172.21.15.151:0/1388113209 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cf400f910 con 0x556cc2acb3e0 2024-04-26T01:54:13.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.807+0000 7f0cef400700 1 -- 172.21.15.151:0/1388113209 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf4026790 con 0x556cc2acb3e0 2024-04-26T01:54:13.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.808+0000 7f0d13a12b80 1 -- 172.21.15.151:0/1388113209 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cc2acb3e0 msgr2=0x556cc2acb7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:13.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.808+0000 7f0d13a12b80 1 --2- 172.21.15.151:0/1388113209 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cc2acb3e0 0x556cc2acb7c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4006010 tx=0x7f0cf401f000 comp rx=0 tx=0).stop 2024-04-26T01:54:13.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.808+0000 7f0d13a12b80 1 -- 172.21.15.151:0/1388113209 shutdown_connections 2024-04-26T01:54:13.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.808+0000 7f0d13a12b80 1 --2- 172.21.15.151:0/1388113209 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556cc2ad6c40 0x556cc2ad9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:13.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.808+0000 7f0d13a12b80 1 --2- 172.21.15.151:0/1388113209 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cc2acb3e0 0x556cc2acb7c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:13.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.808+0000 7f0d13a12b80 1 --2- 172.21.15.151:0/1388113209 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556cc2acbd00 0x556cc2ad6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:13.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.808+0000 7f0d13a12b80 1 -- 172.21.15.151:0/1388113209 >> 172.21.15.151:0/1388113209 conn(0x556cc29fa550 msgr2=0x556cc2ac7a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:13.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.808+0000 7f0d13a12b80 1 -- 172.21.15.151:0/1388113209 shutdown_connections 2024-04-26T01:54:13.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.808+0000 7f0d13a12b80 1 -- 172.21.15.151:0/1388113209 wait complete. 2024-04-26T01:54:13.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.808+0000 7f0d13a12b80 1 -- start start 2024-04-26T01:54:13.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.809+0000 7f0d13a12b80 1 Processor -- start 2024-04-26T01:54:13.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.809+0000 7f0d13a12b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556cc2acb3e0 0x556cc2ad6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:13.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.809+0000 7f0d13a12b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556cc2acbd00 0x556cc2ad3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:13.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.809+0000 7f0d13a12b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cc2ad6c40 0x556cc2ad3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:13.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.809+0000 7f0d13a12b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x556cc2ad97c0 con 0x556cc2acb3e0 2024-04-26T01:54:13.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.809+0000 7f0d13a12b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556cc29d8810 con 0x556cc2acbd00 2024-04-26T01:54:13.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.809+0000 7f0d13a12b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556cc2ad9940 con 0x556cc2ad6c40 2024-04-26T01:54:13.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.809+0000 7f0d13a12b80 10 client.0 _ll_register_callbacks cb 0x556cc29b76d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:54:13.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.810+0000 7f0d02800700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cc2ad6c40 0x556cc2ad3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:13.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.810+0000 7f0d02800700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cc2ad6c40 0x556cc2ad3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:47926/0 (socket says 172.21.15.151:47926) 2024-04-26T01:54:13.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.810+0000 7f0d02800700 1 -- 172.21.15.151:0/2395581895 learned_addr learned my addr 172.21.15.151:0/2395581895 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:54:13.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.810+0000 7f0d01400700 1 --2- 172.21.15.151:0/2395581895 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556cc2acbd00 0x556cc2ad3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:13.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.810+0000 7f0d02800700 1 -- 172.21.15.151:0/2395581895 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556cc2acbd00 msgr2=0x556cc2ad3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:13.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.810+0000 7f0d02800700 1 --2- 172.21.15.151:0/2395581895 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556cc2acbd00 0x556cc2ad3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:13.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.810+0000 7f0d02800700 1 -- 172.21.15.151:0/2395581895 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556cc2acb3e0 msgr2=0x556cc2ad6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:13.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.810+0000 7f0d02800700 1 --2- 172.21.15.151:0/2395581895 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556cc2acb3e0 0x556cc2ad6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:13.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.810+0000 7f0d02800700 1 -- 172.21.15.151:0/2395581895 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cc296c880 con 0x556cc2ad6c40 2024-04-26T01:54:13.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.810+0000 7f0d01400700 1 --2- 172.21.15.151:0/2395581895 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556cc2acbd00 0x556cc2ad3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:54:13.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.810+0000 7f0d02800700 1 --2- 172.21.15.151:0/2395581895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cc2ad6c40 0x556cc2ad3bc0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc004070 tx=0x7f0cfc004150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebce1ba9a1bf7f82 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:13.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.810+0000 7f0ce3e00700 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-26T01:54:13.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.811+0000 7f0ce3e00700 1 -- 172.21.15.151:0/2395581895 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc01a070 con 0x556cc2ad6c40 2024-04-26T01:54:13.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.811+0000 7f0ce3e00700 1 -- 172.21.15.151:0/2395581895 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f0cfc004db0 con 0x556cc2ad6c40 2024-04-26T01:54:13.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.811+0000 7f0d13a12b80 1 -- 172.21.15.151:0/2395581895 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cf400f910 con 0x556cc2ad6c40 2024-04-26T01:54:13.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.811+0000 7f0ce3e00700 1 -- 172.21.15.151:0/2395581895 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0cfc023050 con 0x556cc2ad6c40 2024-04-26T01:54:13.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.811+0000 7f0d13a12b80 10 client.25237 fetch_fsmap learned FSMap version 26 2024-04-26T01:54:13.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.811+0000 7f0d13a12b80 1 -- 172.21.15.151:0/2395581895 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556cc29b5aa0 con 0x556cc2ad6c40 2024-04-26T01:54:13.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.812+0000 7f0ce3e00700 1 -- 172.21.15.151:0/2395581895 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cf400f910 con 0x556cc2ad6c40 2024-04-26T01:54:13.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.812+0000 7f0ce3e00700 1 -- 172.21.15.151:0/2395581895 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f0cfc005260 con 0x556cc2ad6c40 2024-04-26T01:54:13.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.812+0000 7f0d13a12b80 10 client.25237 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T01:54:13.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.812+0000 7f0d13a12b80 10 client.25237 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T01:54:13.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.812+0000 7f0d13a12b80 20 client.25237 populate_metadata read hostname 'smithi151' 2024-04-26T01:54:13.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.812+0000 7f0d13a12b80 1 --2- 172.21.15.151:0/2395581895 >> [v2:172.21.15.172:6836/55972537,v1:172.21.15.172:6837/55972537] conn(0x556cc2ae6600 0x556cc2ae7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:13.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.812+0000 7f0d13a12b80 4 client.25237 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:54:13.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.812+0000 7f0d13a12b80 1 -- 172.21.15.151:0/2395581895 --> [v2:172.21.15.172:6836/55972537,v1:172.21.15.172:6837/55972537] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556cc2a2c630 con 0x556cc2ae6600 2024-04-26T01:54:13.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.813+0000 7f0d01e00700 1 --2- 172.21.15.151:0/2395581895 >> [v2:172.21.15.172:6836/55972537,v1:172.21.15.172:6837/55972537] conn(0x556cc2ae6600 0x556cc2ae7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:13.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.813+0000 7f0d01e00700 1 --2- 172.21.15.151:0/2395581895 >> [v2:172.21.15.172:6836/55972537,v1:172.21.15.172:6837/55972537] conn(0x556cc2ae6600 0x556cc2ae7a70 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1c57b50712fd8965 server_cookie=84e59697e66f6227 in_seq=0 out_seq=0 2024-04-26T01:54:13.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.813+0000 7f0ce3e00700 10 client.25237 ms_handle_connect on v2:172.21.15.172:6836/55972537 2024-04-26T01:54:13.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.814+0000 7f0ce3e00700 1 -- 172.21.15.151:0/2395581895 <== mds.2 v2:172.21.15.172:6836/55972537 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556cc2a2c630 con 0x556cc2ae6600 2024-04-26T01:54:13.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.814+0000 7f0ce3e00700 10 client.25237 handle_command_reply: tid=0 2024-04-26T01:54:13.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.814+0000 7f0d13a12b80 1 client.25237 shutdown 2024-04-26T01:54:13.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.814+0000 7f0d13a12b80 1 -- 172.21.15.151:0/2395581895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cc2ad6c40 msgr2=0x556cc2ad3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:13.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.814+0000 7f0d13a12b80 1 --2- 172.21.15.151:0/2395581895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cc2ad6c40 0x556cc2ad3bc0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc004070 tx=0x7f0cfc004150 comp rx=0 tx=0).stop 2024-04-26T01:54:13.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.814+0000 7f0d13a12b80 1 -- 172.21.15.151:0/2395581895 shutdown_connections 2024-04-26T01:54:13.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.814+0000 7f0d13a12b80 1 --2- 172.21.15.151:0/2395581895 >> [v2:172.21.15.172:6836/55972537,v1:172.21.15.172:6837/55972537] conn(0x556cc2ae6600 0x556cc2ae7a70 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:13.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.814+0000 7f0d13a12b80 1 --2- 172.21.15.151:0/2395581895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cc2ad6c40 0x556cc2ad3bc0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:13.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.814+0000 7f0d13a12b80 1 --2- 172.21.15.151:0/2395581895 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556cc2acb3e0 0x556cc2ad6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:13.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.814+0000 7f0d13a12b80 1 --2- 172.21.15.151:0/2395581895 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556cc2acbd00 0x556cc2ad3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:13.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.814+0000 7f0ce3e00700 0 client.25237 ms_handle_reset on v2:172.21.15.172:6836/55972537 2024-04-26T01:54:13.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.814+0000 7f0d13a12b80 1 -- 172.21.15.151:0/2395581895 >> 172.21.15.151:0/2395581895 conn(0x556cc29fa550 msgr2=0x556cc2ac7680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:13.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.815+0000 7f0d13a12b80 1 -- 172.21.15.151:0/2395581895 shutdown_connections 2024-04-26T01:54:13.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.815+0000 7f0d13a12b80 1 -- 172.21.15.151:0/2395581895 wait complete. 2024-04-26T01:54:13.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.815+0000 7f0d13a12b80 20 client.25237 trim_cache size 0 max 16384 2024-04-26T01:54:13.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.844+0000 7f0d13a12b80 1 Processor -- start 2024-04-26T01:54:13.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.845+0000 7f0d13a12b80 1 -- start start 2024-04-26T01:54:13.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.845+0000 7f0d13a12b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556cc2acb3e0 0x556cc2ac7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:13.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.845+0000 7f0d13a12b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556cc2acbd00 0x556cc2ad3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:13.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.845+0000 7f0d13a12b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cc2ad6c40 0x556cc2ad39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:13.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.845+0000 7f0d13a12b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x556cc2ae5db0 con 0x556cc2acb3e0 2024-04-26T01:54:13.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.845+0000 7f0d13a12b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556cc2ae6480 con 0x556cc2acbd00 2024-04-26T01:54:13.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.845+0000 7f0d13a12b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556cc2ae6b50 con 0x556cc2ad6c40 2024-04-26T01:54:13.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.845+0000 7f0d02800700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cc2ad6c40 0x556cc2ad39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:13.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.845+0000 7f0d02800700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cc2ad6c40 0x556cc2ad39a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:47934/0 (socket says 172.21.15.151:47934) 2024-04-26T01:54:13.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.845+0000 7f0d02800700 1 -- 172.21.15.151:0/2372652935 learned_addr learned my addr 172.21.15.151:0/2372652935 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:54:13.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.845+0000 7f0d01e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556cc2acb3e0 0x556cc2ac7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:13.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.846+0000 7f0d01400700 1 --2- 172.21.15.151:0/2372652935 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556cc2acbd00 0x556cc2ad3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:13.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.846+0000 7f0d01400700 1 -- 172.21.15.151:0/2372652935 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cc2ad6c40 msgr2=0x556cc2ad39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:13.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.846+0000 7f0d01400700 1 --2- 172.21.15.151:0/2372652935 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cc2ad6c40 0x556cc2ad39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:13.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.846+0000 7f0d01400700 1 -- 172.21.15.151:0/2372652935 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556cc2acb3e0 msgr2=0x556cc2ac7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:13.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.846+0000 7f0d01400700 1 --2- 172.21.15.151:0/2372652935 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556cc2acb3e0 0x556cc2ac7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:13.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.846+0000 7f0d01400700 1 -- 172.21.15.151:0/2372652935 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cc2ad5fc0 con 0x556cc2acbd00 2024-04-26T01:54:13.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.846+0000 7f0d01e00700 1 --2- 172.21.15.151:0/2372652935 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556cc2acb3e0 0x556cc2ac7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:54:13.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.846+0000 7f0d01400700 1 --2- 172.21.15.151:0/2372652935 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556cc2acbd00 0x556cc2ad3460 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0cf009c6b0 tx=0x7f0cf00bd040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96eb8c2a28d2aa23 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:13.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.846+0000 7f0ceea00700 1 -- 172.21.15.151:0/2372652935 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf00a0d60 con 0x556cc2acbd00 2024-04-26T01:54:13.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.847+0000 7f0ceea00700 1 -- 172.21.15.151:0/2372652935 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cf00a5620 con 0x556cc2acbd00 2024-04-26T01:54:13.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.847+0000 7f0ceea00700 1 -- 172.21.15.151:0/2372652935 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf00870b0 con 0x556cc2acbd00 2024-04-26T01:54:13.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.847+0000 7f0d13a12b80 1 -- 172.21.15.151:0/2372652935 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556cc2acbd00 msgr2=0x556cc2ad3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:13.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.847+0000 7f0d13a12b80 1 --2- 172.21.15.151:0/2372652935 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556cc2acbd00 0x556cc2ad3460 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0cf009c6b0 tx=0x7f0cf00bd040 comp rx=0 tx=0).stop 2024-04-26T01:54:13.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.847+0000 7f0d13a12b80 1 -- 172.21.15.151:0/2372652935 shutdown_connections 2024-04-26T01:54:13.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.847+0000 7f0d13a12b80 1 --2- 172.21.15.151:0/2372652935 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cc2ad6c40 0x556cc2ad39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:13.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.847+0000 7f0d13a12b80 1 --2- 172.21.15.151:0/2372652935 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556cc2acb3e0 0x556cc2ac7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:13.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.847+0000 7f0d13a12b80 1 --2- 172.21.15.151:0/2372652935 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556cc2acbd00 0x556cc2ad3460 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:13.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.847+0000 7f0d13a12b80 1 -- 172.21.15.151:0/2372652935 >> 172.21.15.151:0/2372652935 conn(0x556cc29fa550 msgr2=0x556cc2ad1f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:13.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.848+0000 7f0d13a12b80 1 -- 172.21.15.151:0/2372652935 shutdown_connections 2024-04-26T01:54:13.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.848+0000 7f0d13a12b80 1 -- 172.21.15.151:0/2372652935 wait complete. 2024-04-26T01:54:13.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.848+0000 7f0d13a12b80 1 -- start start 2024-04-26T01:54:13.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.848+0000 7f0d13a12b80 1 Processor -- start 2024-04-26T01:54:13.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.849+0000 7f0d13a12b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556cc2acb3e0 0x556cc2aecf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:13.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.849+0000 7f0d13a12b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cc2acbd00 0x556cc2aed4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:13.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.849+0000 7f0d13a12b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556cc2ad6c40 0x556cc2b4c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:13.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.849+0000 7f0d13a12b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x556cc2ae78f0 con 0x556cc2acb3e0 2024-04-26T01:54:13.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.849+0000 7f0d13a12b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556cc2ae7220 con 0x556cc2ad6c40 2024-04-26T01:54:13.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.849+0000 7f0d13a12b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556cc2ae7fc0 con 0x556cc2acbd00 2024-04-26T01:54:13.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.849+0000 7f0d13a12b80 10 client.0 _ll_register_callbacks cb 0x556cc29b76d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:54:13.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.849+0000 7f0d01e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556cc2acb3e0 0x556cc2aecf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:13.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.849+0000 7f0d01e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556cc2acb3e0 0x556cc2aecf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36420/0 (socket says 172.21.15.151:36420) 2024-04-26T01:54:13.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.849+0000 7f0d01e00700 1 -- 172.21.15.151:0/2305509243 learned_addr learned my addr 172.21.15.151:0/2305509243 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:54:13.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.849+0000 7f0d02800700 1 --2- 172.21.15.151:0/2305509243 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556cc2ad6c40 0x556cc2b4c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:13.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.850+0000 7f0d01400700 1 --2- 172.21.15.151:0/2305509243 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cc2acbd00 0x556cc2aed4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:13.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.850+0000 7f0d01e00700 1 -- 172.21.15.151:0/2305509243 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cc2acbd00 msgr2=0x556cc2aed4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:13.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.850+0000 7f0d01e00700 1 --2- 172.21.15.151:0/2305509243 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cc2acbd00 0x556cc2aed4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:13.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.850+0000 7f0d01e00700 1 -- 172.21.15.151:0/2305509243 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556cc2ad6c40 msgr2=0x556cc2b4c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:13.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.850+0000 7f0d01e00700 1 --2- 172.21.15.151:0/2305509243 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556cc2ad6c40 0x556cc2b4c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:13.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.850+0000 7f0d01e00700 1 -- 172.21.15.151:0/2305509243 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cc2b4cc70 con 0x556cc2acb3e0 2024-04-26T01:54:13.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.850+0000 7f0d01e00700 1 --2- 172.21.15.151:0/2305509243 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556cc2acb3e0 0x556cc2aecf70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4000c00 tx=0x7f0cf4009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df099fab65805f58 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:13.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.850+0000 7f0ce3e00700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-04-26T01:54:13.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.850+0000 7f0ce3e00700 1 -- 172.21.15.151:0/2305509243 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf403c020 con 0x556cc2acb3e0 2024-04-26T01:54:13.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.850+0000 7f0ce3e00700 1 -- 172.21.15.151:0/2305509243 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f0cf4038bc0 con 0x556cc2acb3e0 2024-04-26T01:54:13.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.850+0000 7f0d13a12b80 1 -- 172.21.15.151:0/2305509243 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cf00a5620 con 0x556cc2acb3e0 2024-04-26T01:54:13.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.851+0000 7f0ce3e00700 1 -- 172.21.15.151:0/2305509243 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0cf4038dc0 con 0x556cc2acb3e0 2024-04-26T01:54:13.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.851+0000 7f0d13a12b80 10 client.15126 fetch_fsmap learned FSMap version 26 2024-04-26T01:54:13.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.851+0000 7f0d13a12b80 1 -- 172.21.15.151:0/2305509243 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556cc2b4df10 con 0x556cc2acb3e0 2024-04-26T01:54:13.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.851+0000 7f0ce3e00700 1 -- 172.21.15.151:0/2305509243 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cf00a5620 con 0x556cc2acb3e0 2024-04-26T01:54:13.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.851+0000 7f0ce3e00700 1 -- 172.21.15.151:0/2305509243 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f0cf400f9d0 con 0x556cc2acb3e0 2024-04-26T01:54:13.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.851+0000 7f0d13a12b80 10 client.15126 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T01:54:13.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.851+0000 7f0d13a12b80 10 client.15126 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T01:54:13.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.851+0000 7f0d13a12b80 20 client.15126 populate_metadata read hostname 'smithi151' 2024-04-26T01:54:13.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.851+0000 7f0d13a12b80 1 --2- 172.21.15.151:0/2305509243 >> [v2:172.21.15.172:6836/55972537,v1:172.21.15.172:6837/55972537] conn(0x556cc2b525f0 0x556cc2b51300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:13.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.851+0000 7f0d13a12b80 4 client.15126 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T01:54:13.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.851+0000 7f0d13a12b80 1 -- 172.21.15.151:0/2305509243 --> [v2:172.21.15.172:6836/55972537,v1:172.21.15.172:6837/55972537] -- command(tid 0: {"prefix": "status"}) -- 0x556cc2a266d0 con 0x556cc2b525f0 2024-04-26T01:54:13.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.852+0000 7f0d01400700 1 --2- 172.21.15.151:0/2305509243 >> [v2:172.21.15.172:6836/55972537,v1:172.21.15.172:6837/55972537] conn(0x556cc2b525f0 0x556cc2b51300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:13.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.852+0000 7f0d01400700 1 --2- 172.21.15.151:0/2305509243 >> [v2:172.21.15.172:6836/55972537,v1:172.21.15.172:6837/55972537] conn(0x556cc2b525f0 0x556cc2b51300 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=af844b946eed97a7 server_cookie=1504b20a7a772dbf in_seq=0 out_seq=0 2024-04-26T01:54:13.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.852+0000 7f0ce3e00700 10 client.15126 ms_handle_connect on v2:172.21.15.172:6836/55972537 2024-04-26T01:54:13.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.852+0000 7f0ce3e00700 1 -- 172.21.15.151:0/2305509243 <== mds.2 v2:172.21.15.172:6836/55972537 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x556cc2a266d0 con 0x556cc2b525f0 2024-04-26T01:54:13.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.852+0000 7f0ce3e00700 10 client.15126 handle_command_reply: tid=0 2024-04-26T01:54:13.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.852+0000 7f0d13a12b80 1 client.15126 shutdown 2024-04-26T01:54:13.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.853+0000 7f0d13a12b80 1 -- 172.21.15.151:0/2305509243 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556cc2acb3e0 msgr2=0x556cc2aecf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:13.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.853+0000 7f0d13a12b80 1 --2- 172.21.15.151:0/2305509243 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556cc2acb3e0 0x556cc2aecf70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4000c00 tx=0x7f0cf4009560 comp rx=0 tx=0).stop 2024-04-26T01:54:13.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.853+0000 7f0d13a12b80 1 -- 172.21.15.151:0/2305509243 shutdown_connections 2024-04-26T01:54:13.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.853+0000 7f0d13a12b80 1 --2- 172.21.15.151:0/2305509243 >> [v2:172.21.15.172:6836/55972537,v1:172.21.15.172:6837/55972537] conn(0x556cc2b525f0 0x556cc2b51300 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:13.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.853+0000 7f0d13a12b80 1 --2- 172.21.15.151:0/2305509243 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556cc2ad6c40 0x556cc2b4c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:13.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.853+0000 7f0d13a12b80 1 --2- 172.21.15.151:0/2305509243 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556cc2acb3e0 0x556cc2aecf70 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:13.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.853+0000 7f0d13a12b80 1 --2- 172.21.15.151:0/2305509243 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cc2acbd00 0x556cc2aed4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:13.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.853+0000 7f0d13a12b80 1 -- 172.21.15.151:0/2305509243 >> 172.21.15.151:0/2305509243 conn(0x556cc29fa550 msgr2=0x556cc2ad3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:13.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.853+0000 7f0ce3e00700 0 client.15126 ms_handle_reset on v2:172.21.15.172:6836/55972537 2024-04-26T01:54:13.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.853+0000 7f0d13a12b80 1 -- 172.21.15.151:0/2305509243 shutdown_connections 2024-04-26T01:54:13.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.853+0000 7f0d13a12b80 1 -- 172.21.15.151:0/2305509243 wait complete. 2024-04-26T01:54:13.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.853+0000 7f0d13a12b80 20 client.15126 trim_cache size 0 max 16384 2024-04-26T01:54:13.855 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-26T01:54:13.855 INFO:teuthology.orchestra.run.smithi151.stdout: "cluster_fsid": "c0e49e64-036d-11ef-bc93-c7b262605968", 2024-04-26T01:54:13.855 INFO:teuthology.orchestra.run.smithi151.stdout: "whoami": 2, 2024-04-26T01:54:13.855 INFO:teuthology.orchestra.run.smithi151.stdout: "id": 24493, 2024-04-26T01:54:13.855 INFO:teuthology.orchestra.run.smithi151.stdout: "want_state": "up:active", 2024-04-26T01:54:13.855 INFO:teuthology.orchestra.run.smithi151.stdout: "state": "up:active", 2024-04-26T01:54:13.855 INFO:teuthology.orchestra.run.smithi151.stdout: "fs_name": "cephfs", 2024-04-26T01:54:13.855 INFO:teuthology.orchestra.run.smithi151.stdout: "rank_uptime": 255.08276059799999, 2024-04-26T01:54:13.855 INFO:teuthology.orchestra.run.smithi151.stdout: "mdsmap_epoch": 26, 2024-04-26T01:54:13.855 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch": 93, 2024-04-26T01:54:13.855 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch_barrier": 93, 2024-04-26T01:54:13.856 INFO:teuthology.orchestra.run.smithi151.stdout: "uptime": 313.59500256199999 2024-04-26T01:54:13.856 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-26T01:54:13.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.854+0000 7f0d03200700 1 -- 172.21.15.151:0/1371433506 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f0ce80a3f60 msgr2=0x7f0ce80a6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:13.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.854+0000 7f0d03200700 1 --2- 172.21.15.151:0/1371433506 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f0ce80a3f60 0x7f0ce80a6420 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0cf40098e0 tx=0x7f0cf4005960 comp rx=0 tx=0).stop 2024-04-26T01:54:13.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.854+0000 7f0d03200700 1 -- 172.21.15.151:0/1371433506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d04077890 msgr2=0x7f0d0419fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:13.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.854+0000 7f0d03200700 1 --2- 172.21.15.151:0/1371433506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d04077890 0x7f0d0419fb90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0cf000e9a0 tx=0x7f0cf000ecb0 comp rx=0 tx=0).stop 2024-04-26T01:54:13.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.854+0000 7f0d03200700 1 -- 172.21.15.151:0/1371433506 shutdown_connections 2024-04-26T01:54:13.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.854+0000 7f0d03200700 1 --2- 172.21.15.151:0/1371433506 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f0ce80a3f60 0x7f0ce80a6420 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:13.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.854+0000 7f0d03200700 1 --2- 172.21.15.151:0/1371433506 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d041137f0 0x7f0d041a3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:13.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.854+0000 7f0d03200700 1 --2- 172.21.15.151:0/1371433506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d04076f70 0x7f0d0419f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:13.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.854+0000 7f0d03200700 1 --2- 172.21.15.151:0/1371433506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d04077890 0x7f0d0419fb90 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:13.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.854+0000 7f0d03200700 1 -- 172.21.15.151:0/1371433506 >> 172.21.15.151:0/1371433506 conn(0x7f0d04100620 msgr2=0x7f0d04102a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:13.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.854+0000 7f0d03200700 1 -- 172.21.15.151:0/1371433506 shutdown_connections 2024-04-26T01:54:13.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:13.854+0000 7f0d03200700 1 -- 172.21.15.151:0/1371433506 wait complete. 2024-04-26T01:54:14.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:54:13 smithi165 ceph-mon[32648]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-04-26T01:54:14.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:54:13 smithi151 ceph-mon[28057]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-04-26T01:54:14.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:54:13 smithi172 ceph-mon[32785]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-04-26T01:54:14.776 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph tell mds.i status 2024-04-26T01:54:15.211 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:54:16.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:54:15 smithi165 ceph-mon[32648]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-04-26T01:54:16.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:54:15 smithi151 ceph-mon[28057]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-04-26T01:54:16.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:54:15 smithi172 ceph-mon[32785]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-04-26T01:54:17.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.517+0000 7f2df8000700 1 -- 172.21.15.151:0/2160930851 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2df0108a50 msgr2=0x7f2df0108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:17.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.517+0000 7f2df8000700 1 --2- 172.21.15.151:0/2160930851 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2df0108a50 0x7f2df0108e30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2de40098f0 tx=0x7f2de4009c00 comp rx=0 tx=0).stop 2024-04-26T01:54:17.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.517+0000 7f2df8000700 1 -- 172.21.15.151:0/2160930851 shutdown_connections 2024-04-26T01:54:17.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.517+0000 7f2df8000700 1 --2- 172.21.15.151:0/2160930851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2df0103450 0x7f2df010f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:17.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.517+0000 7f2df8000700 1 --2- 172.21.15.151:0/2160930851 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2df0108a50 0x7f2df0108e30 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:17.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.517+0000 7f2df8000700 1 --2- 172.21.15.151:0/2160930851 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2df0102a90 0x7f2df0102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:17.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.517+0000 7f2df8000700 1 -- 172.21.15.151:0/2160930851 >> 172.21.15.151:0/2160930851 conn(0x7f2df0079ba0 msgr2=0x7f2df0079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:17.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.517+0000 7f2df8000700 1 -- 172.21.15.151:0/2160930851 shutdown_connections 2024-04-26T01:54:17.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.517+0000 7f2df8000700 1 -- 172.21.15.151:0/2160930851 wait complete. 2024-04-26T01:54:17.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.518+0000 7f2df8000700 1 Processor -- start 2024-04-26T01:54:17.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.519+0000 7f2df8000700 1 -- start start 2024-04-26T01:54:17.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.519+0000 7f2df8000700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2df0102a90 0x7f2df0074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:17.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.519+0000 7f2df8000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2df0103450 0x7f2df0075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:17.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.519+0000 7f2df8000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2df0108a50 0x7f2df00757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:17.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.519+0000 7f2df8000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2df0112c60 con 0x7f2df0103450 2024-04-26T01:54:17.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.519+0000 7f2df8000700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2df0112ae0 con 0x7f2df0108a50 2024-04-26T01:54:17.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.519+0000 7f2df8000700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2df0112de0 con 0x7f2df0102a90 2024-04-26T01:54:17.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.519+0000 7f2df4c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2df0103450 0x7f2df0075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:17.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.519+0000 7f2df4c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2df0103450 0x7f2df0075280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36442/0 (socket says 172.21.15.151:36442) 2024-04-26T01:54:17.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.519+0000 7f2df4c00700 1 -- 172.21.15.151:0/274225416 learned_addr learned my addr 172.21.15.151:0/274225416 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:54:17.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.519+0000 7f2df6000700 1 --2- 172.21.15.151:0/274225416 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2df0108a50 0x7f2df00757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:17.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.519+0000 7f2df5600700 1 --2- 172.21.15.151:0/274225416 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2df0102a90 0x7f2df0074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:17.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.520+0000 7f2df6000700 1 -- 172.21.15.151:0/274225416 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2df0102a90 msgr2=0x7f2df0074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:17.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.520+0000 7f2df6000700 1 --2- 172.21.15.151:0/274225416 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2df0102a90 0x7f2df0074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:17.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.520+0000 7f2df6000700 1 -- 172.21.15.151:0/274225416 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2df0103450 msgr2=0x7f2df0075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:17.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.520+0000 7f2df6000700 1 --2- 172.21.15.151:0/274225416 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2df0103450 0x7f2df0075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:17.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.520+0000 7f2df6000700 1 -- 172.21.15.151:0/274225416 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2df01a7720 con 0x7f2df0108a50 2024-04-26T01:54:17.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.520+0000 7f2df6000700 1 --2- 172.21.15.151:0/274225416 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2df0108a50 0x7f2df00757c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2de800efc0 tx=0x7f2de800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:17.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.520+0000 7f2dee000700 1 -- 172.21.15.151:0/274225416 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de80092e0 con 0x7f2df0108a50 2024-04-26T01:54:17.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.521+0000 7f2dee000700 1 -- 172.21.15.151:0/274225416 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2de8004bb0 con 0x7f2df0108a50 2024-04-26T01:54:17.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.521+0000 7f2dee000700 1 -- 172.21.15.151:0/274225416 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de8004020 con 0x7f2df0108a50 2024-04-26T01:54:17.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.521+0000 7f2df8000700 1 -- 172.21.15.151:0/274225416 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2df0198f30 con 0x7f2df0108a50 2024-04-26T01:54:17.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.521+0000 7f2df8000700 1 -- 172.21.15.151:0/274225416 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2df0199420 con 0x7f2df0108a50 2024-04-26T01:54:17.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.525+0000 7f2e029c0b80 1 Processor -- start 2024-04-26T01:54:17.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.526+0000 7f2dee000700 1 -- 172.21.15.151:0/274225416 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 26) ==== 138931+0+0 (secure 0 0 0) 0x7f2de801b030 con 0x7f2df0108a50 2024-04-26T01:54:17.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.526+0000 7f2e029c0b80 1 -- start start 2024-04-26T01:54:17.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.526+0000 7f2e029c0b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d52480e780 0x55d52480eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:17.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.526+0000 7f2e029c0b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d52480f0a0 0x55d5248089b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:17.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.526+0000 7f2e029c0b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d524808ef0 0x55d524809330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:17.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.526+0000 7f2e029c0b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d52475ae40 con 0x55d52480f0a0 2024-04-26T01:54:17.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.526+0000 7f2e029c0b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d5247016e0 con 0x55d52480e780 2024-04-26T01:54:17.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.526+0000 7f2e029c0b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d524756e00 con 0x55d524808ef0 2024-04-26T01:54:17.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.527+0000 7f2df4c00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d52480e780 0x55d52480eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:17.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.527+0000 7f2df4c00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d52480e780 0x55d52480eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:45820/0 (socket says 172.21.15.151:45820) 2024-04-26T01:54:17.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.527+0000 7f2df4c00700 1 -- 172.21.15.151:0/3601854903 learned_addr learned my addr 172.21.15.151:0/3601854903 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:54:17.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.527+0000 7f2df5600700 1 --2- 172.21.15.151:0/3601854903 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d52480f0a0 0x55d5248089b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:17.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.527+0000 7f2df4c00700 1 --2- 172.21.15.151:0/3601854903 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d524808ef0 0x55d524809330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:17.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.528+0000 7f2df5600700 1 -- 172.21.15.151:0/3601854903 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d524808ef0 msgr2=0x55d524809330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:17.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.528+0000 7f2df5600700 1 --2- 172.21.15.151:0/3601854903 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d524808ef0 0x55d524809330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:17.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.528+0000 7f2df5600700 1 -- 172.21.15.151:0/3601854903 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d52480e780 msgr2=0x55d52480eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:17.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.528+0000 7f2df5600700 1 --2- 172.21.15.151:0/3601854903 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d52480e780 0x55d52480eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:17.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.528+0000 7f2df5600700 1 -- 172.21.15.151:0/3601854903 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5246f7630 con 0x55d52480f0a0 2024-04-26T01:54:17.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.528+0000 7f2df4c00700 1 --2- 172.21.15.151:0/3601854903 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d524808ef0 0x55d524809330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:54:17.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.528+0000 7f2dee000700 1 --2- 172.21.15.151:0/274225416 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f2dd80a3f60 0x7f2dd80a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:17.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.528+0000 7f2dee000700 1 -- 172.21.15.151:0/274225416 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f2de80d04b0 con 0x7f2df0108a50 2024-04-26T01:54:17.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.528+0000 7f2df5600700 1 --2- 172.21.15.151:0/3601854903 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d52480f0a0 0x55d5248089b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2de400c010 tx=0x7f2de400bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19ef1c2fda8b1221 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:17.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.528+0000 7f2df4c00700 1 --2- 172.21.15.151:0/274225416 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f2dd80a3f60 0x7f2dd80a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:17.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.528+0000 7f2de2a00700 1 -- 172.21.15.151:0/3601854903 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de400f460 con 0x55d52480f0a0 2024-04-26T01:54:17.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.528+0000 7f2de2a00700 1 -- 172.21.15.151:0/3601854903 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2de4005040 con 0x55d52480f0a0 2024-04-26T01:54:17.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.528+0000 7f2de2a00700 1 -- 172.21.15.151:0/3601854903 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de40038c0 con 0x55d52480f0a0 2024-04-26T01:54:17.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.529+0000 7f2e029c0b80 1 -- 172.21.15.151:0/3601854903 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d52480f0a0 msgr2=0x55d5248089b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:17.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.529+0000 7f2e029c0b80 1 --2- 172.21.15.151:0/3601854903 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d52480f0a0 0x55d5248089b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2de400c010 tx=0x7f2de400bf60 comp rx=0 tx=0).stop 2024-04-26T01:54:17.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.529+0000 7f2df4c00700 1 --2- 172.21.15.151:0/274225416 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f2dd80a3f60 0x7f2dd80a6420 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc009750 tx=0x7f2ddc0096a0 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:17.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.530+0000 7f2e029c0b80 1 -- 172.21.15.151:0/3601854903 shutdown_connections 2024-04-26T01:54:17.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.530+0000 7f2e029c0b80 1 --2- 172.21.15.151:0/3601854903 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d524808ef0 0x55d524809330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:17.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.530+0000 7f2e029c0b80 1 --2- 172.21.15.151:0/3601854903 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d52480e780 0x55d52480eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:17.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.530+0000 7f2e029c0b80 1 --2- 172.21.15.151:0/3601854903 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d52480f0a0 0x55d5248089b0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:17.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.530+0000 7f2e029c0b80 1 -- 172.21.15.151:0/3601854903 >> 172.21.15.151:0/3601854903 conn(0x55d524736f00 msgr2=0x55d5248038a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:17.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.530+0000 7f2e029c0b80 1 -- 172.21.15.151:0/3601854903 shutdown_connections 2024-04-26T01:54:17.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.530+0000 7f2e029c0b80 1 -- 172.21.15.151:0/3601854903 wait complete. 2024-04-26T01:54:17.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.530+0000 7f2e029c0b80 1 -- start start 2024-04-26T01:54:17.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.531+0000 7f2e029c0b80 1 Processor -- start 2024-04-26T01:54:17.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.531+0000 7f2e029c0b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d524808ef0 0x55d524814ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:17.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.531+0000 7f2e029c0b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d52480e780 0x55d524811fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:17.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.531+0000 7f2e029c0b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d5248124e0 0x55d524812960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:17.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.531+0000 7f2e029c0b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d5248158c0 con 0x55d5248124e0 2024-04-26T01:54:17.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.531+0000 7f2e029c0b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d524714810 con 0x55d52480e780 2024-04-26T01:54:17.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.531+0000 7f2e029c0b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d524815a40 con 0x55d524808ef0 2024-04-26T01:54:17.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.531+0000 7f2e029c0b80 10 client.0 _ll_register_callbacks cb 0x55d5246f36d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:54:17.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.532+0000 7f2df5600700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d524808ef0 0x55d524814ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:17.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.532+0000 7f2df4c00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d52480e780 0x55d524811fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:17.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.532+0000 7f2df5600700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d524808ef0 0x55d524814ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:47984/0 (socket says 172.21.15.151:47984) 2024-04-26T01:54:17.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.532+0000 7f2df5600700 1 -- 172.21.15.151:0/2511303043 learned_addr learned my addr 172.21.15.151:0/2511303043 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:54:17.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.532+0000 7f2df6000700 1 --2- 172.21.15.151:0/2511303043 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d5248124e0 0x55d524812960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:17.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.532+0000 7f2df5600700 1 -- 172.21.15.151:0/2511303043 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d52480e780 msgr2=0x55d524811fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:17.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.532+0000 7f2df5600700 1 --2- 172.21.15.151:0/2511303043 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d52480e780 0x55d524811fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:17.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.532+0000 7f2df5600700 1 -- 172.21.15.151:0/2511303043 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d5248124e0 msgr2=0x55d524812960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:17.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.532+0000 7f2df5600700 1 --2- 172.21.15.151:0/2511303043 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d5248124e0 0x55d524812960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:17.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.532+0000 7f2df5600700 1 -- 172.21.15.151:0/2511303043 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5246a8880 con 0x55d524808ef0 2024-04-26T01:54:17.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.532+0000 7f2df5600700 1 --2- 172.21.15.151:0/2511303043 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d524808ef0 0x55d524814ca0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2de4021010 tx=0x7f2de400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dda00490f29af8bc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:17.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.532+0000 7f2dd7400700 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-26T01:54:17.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.533+0000 7f2dd7400700 1 -- 172.21.15.151:0/2511303043 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de4003880 con 0x55d524808ef0 2024-04-26T01:54:17.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.533+0000 7f2dd7400700 1 -- 172.21.15.151:0/2511303043 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f2de4003a20 con 0x55d524808ef0 2024-04-26T01:54:17.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.533+0000 7f2e029c0b80 1 -- 172.21.15.151:0/2511303043 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2de4005040 con 0x55d524808ef0 2024-04-26T01:54:17.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.533+0000 7f2dd7400700 1 -- 172.21.15.151:0/2511303043 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2de4041050 con 0x55d524808ef0 2024-04-26T01:54:17.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.533+0000 7f2e029c0b80 10 client.25255 fetch_fsmap learned FSMap version 26 2024-04-26T01:54:17.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.533+0000 7f2e029c0b80 1 -- 172.21.15.151:0/2511303043 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d5246f1aa0 con 0x55d524808ef0 2024-04-26T01:54:17.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.534+0000 7f2dd7400700 1 -- 172.21.15.151:0/2511303043 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2de4005040 con 0x55d524808ef0 2024-04-26T01:54:17.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.534+0000 7f2dd7400700 1 -- 172.21.15.151:0/2511303043 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f2de4028070 con 0x55d524808ef0 2024-04-26T01:54:17.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.534+0000 7f2e029c0b80 10 client.25255 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T01:54:17.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.534+0000 7f2e029c0b80 10 client.25255 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-26T01:54:17.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.534+0000 7f2e029c0b80 20 client.25255 populate_metadata read hostname 'smithi151' 2024-04-26T01:54:17.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.534+0000 7f2e029c0b80 1 --2- 172.21.15.151:0/2511303043 >> [v2:172.21.15.172:6838/1242302810,v1:172.21.15.172:6839/1242302810] conn(0x55d524822520 0x55d524823990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:17.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.534+0000 7f2e029c0b80 4 client.25255 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:54:17.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.534+0000 7f2e029c0b80 1 -- 172.21.15.151:0/2511303043 --> [v2:172.21.15.172:6838/1242302810,v1:172.21.15.172:6839/1242302810] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d524768630 con 0x55d524822520 2024-04-26T01:54:17.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.534+0000 7f2df4c00700 1 --2- 172.21.15.151:0/2511303043 >> [v2:172.21.15.172:6838/1242302810,v1:172.21.15.172:6839/1242302810] conn(0x55d524822520 0x55d524823990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:17.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.535+0000 7f2df4c00700 1 --2- 172.21.15.151:0/2511303043 >> [v2:172.21.15.172:6838/1242302810,v1:172.21.15.172:6839/1242302810] conn(0x55d524822520 0x55d524823990 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5bf6fed5f9ce90ca server_cookie=b83e0fb203fb69a1 in_seq=0 out_seq=0 2024-04-26T01:54:17.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.535+0000 7f2dd7400700 10 client.25255 ms_handle_connect on v2:172.21.15.172:6838/1242302810 2024-04-26T01:54:17.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.536+0000 7f2dd7400700 1 -- 172.21.15.151:0/2511303043 <== mds.1 v2:172.21.15.172:6838/1242302810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d524768630 con 0x55d524822520 2024-04-26T01:54:17.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.536+0000 7f2dd7400700 10 client.25255 handle_command_reply: tid=0 2024-04-26T01:54:17.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.536+0000 7f2e029c0b80 1 client.25255 shutdown 2024-04-26T01:54:17.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.536+0000 7f2e029c0b80 1 -- 172.21.15.151:0/2511303043 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d524808ef0 msgr2=0x55d524814ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:17.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.536+0000 7f2e029c0b80 1 --2- 172.21.15.151:0/2511303043 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d524808ef0 0x55d524814ca0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2de4021010 tx=0x7f2de400f690 comp rx=0 tx=0).stop 2024-04-26T01:54:17.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.536+0000 7f2e029c0b80 1 -- 172.21.15.151:0/2511303043 shutdown_connections 2024-04-26T01:54:17.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.536+0000 7f2e029c0b80 1 --2- 172.21.15.151:0/2511303043 >> [v2:172.21.15.172:6838/1242302810,v1:172.21.15.172:6839/1242302810] conn(0x55d524822520 0x55d524823990 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:17.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.536+0000 7f2e029c0b80 1 --2- 172.21.15.151:0/2511303043 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d5248124e0 0x55d524812960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:17.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.536+0000 7f2e029c0b80 1 --2- 172.21.15.151:0/2511303043 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d524808ef0 0x55d524814ca0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:17.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.536+0000 7f2e029c0b80 1 --2- 172.21.15.151:0/2511303043 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d52480e780 0x55d524811fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:17.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.536+0000 7f2e029c0b80 1 -- 172.21.15.151:0/2511303043 >> 172.21.15.151:0/2511303043 conn(0x55d524736f00 msgr2=0x55d524802e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:17.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.536+0000 7f2dd7400700 0 client.25255 ms_handle_reset on v2:172.21.15.172:6838/1242302810 2024-04-26T01:54:17.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.536+0000 7f2e029c0b80 1 -- 172.21.15.151:0/2511303043 shutdown_connections 2024-04-26T01:54:17.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.536+0000 7f2e029c0b80 1 -- 172.21.15.151:0/2511303043 wait complete. 2024-04-26T01:54:17.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.537+0000 7f2e029c0b80 20 client.25255 trim_cache size 0 max 16384 2024-04-26T01:54:17.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.561+0000 7f2e029c0b80 1 Processor -- start 2024-04-26T01:54:17.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.561+0000 7f2e029c0b80 1 -- start start 2024-04-26T01:54:17.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.561+0000 7f2e029c0b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d524808ef0 0x55d5248089b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:17.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.561+0000 7f2e029c0b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d52480e780 0x55d5248115c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:17.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.561+0000 7f2e029c0b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d5248124e0 0x55d524804ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:17.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.561+0000 7f2e029c0b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d524821cd0 con 0x55d5248124e0 2024-04-26T01:54:17.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.561+0000 7f2e029c0b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d5248223a0 con 0x55d52480e780 2024-04-26T01:54:17.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.561+0000 7f2e029c0b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d524822a70 con 0x55d524808ef0 2024-04-26T01:54:17.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.561+0000 7f2df6000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d5248124e0 0x55d524804ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:17.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.561+0000 7f2df4c00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d52480e780 0x55d5248115c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:17.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.562+0000 7f2df4c00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d52480e780 0x55d5248115c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:45842/0 (socket says 172.21.15.151:45842) 2024-04-26T01:54:17.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.562+0000 7f2df6000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d5248124e0 0x55d524804ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36464/0 (socket says 172.21.15.151:36464) 2024-04-26T01:54:17.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.562+0000 7f2df4c00700 1 -- 172.21.15.151:0/910901874 learned_addr learned my addr 172.21.15.151:0/910901874 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:54:17.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.562+0000 7f2df5600700 1 --2- 172.21.15.151:0/910901874 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d524808ef0 0x55d5248089b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:17.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.562+0000 7f2df5600700 1 -- 172.21.15.151:0/910901874 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d52480e780 msgr2=0x55d5248115c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:17.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.562+0000 7f2df5600700 1 --2- 172.21.15.151:0/910901874 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d52480e780 0x55d5248115c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:17.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.562+0000 7f2df5600700 1 -- 172.21.15.151:0/910901874 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d5248124e0 msgr2=0x55d524804ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:17.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.562+0000 7f2df5600700 1 --2- 172.21.15.151:0/910901874 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d5248124e0 0x55d524804ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:17.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.562+0000 7f2df5600700 1 -- 172.21.15.151:0/910901874 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5248148e0 con 0x55d524808ef0 2024-04-26T01:54:17.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.562+0000 7f2df4c00700 1 --2- 172.21.15.151:0/910901874 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d52480e780 0x55d5248115c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:54:17.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.562+0000 7f2df6000700 1 --2- 172.21.15.151:0/910901874 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d5248124e0 0x55d524804ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:54:17.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.563+0000 7f2df5600700 1 --2- 172.21.15.151:0/910901874 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d524808ef0 0x55d5248089b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2de4009c40 tx=0x7f2de40095f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2efa4ba5a91041d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:17.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.563+0000 7f2de2000700 1 -- 172.21.15.151:0/910901874 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de404a070 con 0x55d524808ef0 2024-04-26T01:54:17.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.563+0000 7f2de2000700 1 -- 172.21.15.151:0/910901874 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2de40056d0 con 0x55d524808ef0 2024-04-26T01:54:17.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.563+0000 7f2de2000700 1 -- 172.21.15.151:0/910901874 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de400bba0 con 0x55d524808ef0 2024-04-26T01:54:17.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.564+0000 7f2e029c0b80 1 -- 172.21.15.151:0/910901874 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d524808ef0 msgr2=0x55d5248089b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:17.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.564+0000 7f2e029c0b80 1 --2- 172.21.15.151:0/910901874 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d524808ef0 0x55d5248089b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2de4009c40 tx=0x7f2de40095f0 comp rx=0 tx=0).stop 2024-04-26T01:54:17.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.564+0000 7f2e029c0b80 1 -- 172.21.15.151:0/910901874 shutdown_connections 2024-04-26T01:54:17.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.564+0000 7f2e029c0b80 1 --2- 172.21.15.151:0/910901874 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d5248124e0 0x55d524804ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:17.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.564+0000 7f2e029c0b80 1 --2- 172.21.15.151:0/910901874 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d524808ef0 0x55d5248089b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:17.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.564+0000 7f2e029c0b80 1 --2- 172.21.15.151:0/910901874 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d52480e780 0x55d5248115c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:17.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.564+0000 7f2e029c0b80 1 -- 172.21.15.151:0/910901874 >> 172.21.15.151:0/910901874 conn(0x55d524736f00 msgr2=0x55d5247ffac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:17.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.564+0000 7f2e029c0b80 1 -- 172.21.15.151:0/910901874 shutdown_connections 2024-04-26T01:54:17.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.564+0000 7f2e029c0b80 1 -- 172.21.15.151:0/910901874 wait complete. 2024-04-26T01:54:17.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.564+0000 7f2e029c0b80 1 -- start start 2024-04-26T01:54:17.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.565+0000 7f2e029c0b80 1 Processor -- start 2024-04-26T01:54:17.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.565+0000 7f2e029c0b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d524808ef0 0x55d52482ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:17.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.565+0000 7f2e029c0b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d52480e780 0x55d52482d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:17.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.565+0000 7f2e029c0b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d5248124e0 0x55d524827dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:17.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.565+0000 7f2e029c0b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d524823810 con 0x55d5248124e0 2024-04-26T01:54:17.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.565+0000 7f2e029c0b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d524823140 con 0x55d52480e780 2024-04-26T01:54:17.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.565+0000 7f2e029c0b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d524823ee0 con 0x55d524808ef0 2024-04-26T01:54:17.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.565+0000 7f2e029c0b80 10 client.0 _ll_register_callbacks cb 0x55d5246f36d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:54:17.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.565+0000 7f2df5600700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d524808ef0 0x55d52482ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:17.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.565+0000 7f2df5600700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d524808ef0 0x55d52482ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:48008/0 (socket says 172.21.15.151:48008) 2024-04-26T01:54:17.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.565+0000 7f2df5600700 1 -- 172.21.15.151:0/1032121272 learned_addr learned my addr 172.21.15.151:0/1032121272 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:54:17.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.566+0000 7f2df6000700 1 --2- 172.21.15.151:0/1032121272 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d5248124e0 0x55d524827dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:17.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.566+0000 7f2df5600700 1 -- 172.21.15.151:0/1032121272 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d52480e780 msgr2=0x55d52482d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:17.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.566+0000 7f2df5600700 1 --2- 172.21.15.151:0/1032121272 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d52480e780 0x55d52482d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:17.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.566+0000 7f2df5600700 1 -- 172.21.15.151:0/1032121272 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d5248124e0 msgr2=0x55d524827dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:17.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.566+0000 7f2df5600700 1 --2- 172.21.15.151:0/1032121272 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d5248124e0 0x55d524827dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:17.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.566+0000 7f2df5600700 1 -- 172.21.15.151:0/1032121272 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d524828450 con 0x55d524808ef0 2024-04-26T01:54:17.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.566+0000 7f2df5600700 1 --2- 172.21.15.151:0/1032121272 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d524808ef0 0x55d52482ce10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2de40324f0 tx=0x7f2de4032800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e2727a4c09c4bdf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:17.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.566+0000 7f2dd7400700 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-26T01:54:17.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.566+0000 7f2dd7400700 1 -- 172.21.15.151:0/1032121272 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de404a070 con 0x55d524808ef0 2024-04-26T01:54:17.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.567+0000 7f2e029c0b80 1 -- 172.21.15.151:0/1032121272 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2de40056d0 con 0x55d524808ef0 2024-04-26T01:54:17.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.567+0000 7f2dd7400700 1 -- 172.21.15.151:0/1032121272 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f2de400b560 con 0x55d524808ef0 2024-04-26T01:54:17.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.567+0000 7f2dd7400700 1 -- 172.21.15.151:0/1032121272 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2de400b760 con 0x55d524808ef0 2024-04-26T01:54:17.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.567+0000 7f2e029c0b80 10 client.25261 fetch_fsmap learned FSMap version 26 2024-04-26T01:54:17.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.567+0000 7f2e029c0b80 1 -- 172.21.15.151:0/1032121272 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d524889fd0 con 0x55d524808ef0 2024-04-26T01:54:17.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.567+0000 7f2dd7400700 1 -- 172.21.15.151:0/1032121272 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2de40056d0 con 0x55d524808ef0 2024-04-26T01:54:17.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.567+0000 7f2dd7400700 1 -- 172.21.15.151:0/1032121272 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f2de4028040 con 0x55d524808ef0 2024-04-26T01:54:17.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.568+0000 7f2e029c0b80 10 client.25261 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T01:54:17.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.568+0000 7f2e029c0b80 10 client.25261 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-26T01:54:17.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.568+0000 7f2e029c0b80 20 client.25261 populate_metadata read hostname 'smithi151' 2024-04-26T01:54:17.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.568+0000 7f2e029c0b80 1 --2- 172.21.15.151:0/1032121272 >> [v2:172.21.15.172:6838/1242302810,v1:172.21.15.172:6839/1242302810] conn(0x55d52488e870 0x55d52488d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:17.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.568+0000 7f2e029c0b80 4 client.25261 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T01:54:17.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.568+0000 7f2e029c0b80 1 -- 172.21.15.151:0/1032121272 --> [v2:172.21.15.172:6838/1242302810,v1:172.21.15.172:6839/1242302810] -- command(tid 0: {"prefix": "status"}) -- 0x55d5247626d0 con 0x55d52488e870 2024-04-26T01:54:17.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.568+0000 7f2df4c00700 1 --2- 172.21.15.151:0/1032121272 >> [v2:172.21.15.172:6838/1242302810,v1:172.21.15.172:6839/1242302810] conn(0x55d52488e870 0x55d52488d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:17.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.568+0000 7f2df4c00700 1 --2- 172.21.15.151:0/1032121272 >> [v2:172.21.15.172:6838/1242302810,v1:172.21.15.172:6839/1242302810] conn(0x55d52488e870 0x55d52488d580 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9b2dfd3ee3e4dbee server_cookie=a57a869088498e33 in_seq=0 out_seq=0 2024-04-26T01:54:17.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.568+0000 7f2dd7400700 10 client.25261 ms_handle_connect on v2:172.21.15.172:6838/1242302810 2024-04-26T01:54:17.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.569+0000 7f2dd7400700 1 -- 172.21.15.151:0/1032121272 <== mds.1 v2:172.21.15.172:6838/1242302810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55d5247626d0 con 0x55d52488e870 2024-04-26T01:54:17.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.569+0000 7f2dd7400700 10 client.25261 handle_command_reply: tid=0 2024-04-26T01:54:17.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.569+0000 7f2e029c0b80 1 client.25261 shutdown 2024-04-26T01:54:17.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.569+0000 7f2e029c0b80 1 -- 172.21.15.151:0/1032121272 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d524808ef0 msgr2=0x55d52482ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:17.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.569+0000 7f2e029c0b80 1 --2- 172.21.15.151:0/1032121272 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d524808ef0 0x55d52482ce10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2de40324f0 tx=0x7f2de4032800 comp rx=0 tx=0).stop 2024-04-26T01:54:17.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.569+0000 7f2e029c0b80 1 -- 172.21.15.151:0/1032121272 shutdown_connections 2024-04-26T01:54:17.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.569+0000 7f2e029c0b80 1 --2- 172.21.15.151:0/1032121272 >> [v2:172.21.15.172:6838/1242302810,v1:172.21.15.172:6839/1242302810] conn(0x55d52488e870 0x55d52488d580 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:17.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.569+0000 7f2e029c0b80 1 --2- 172.21.15.151:0/1032121272 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d5248124e0 0x55d524827dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:17.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.569+0000 7f2e029c0b80 1 --2- 172.21.15.151:0/1032121272 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d524808ef0 0x55d52482ce10 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:17.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.569+0000 7f2e029c0b80 1 --2- 172.21.15.151:0/1032121272 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d52480e780 0x55d52482d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:17.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.569+0000 7f2e029c0b80 1 -- 172.21.15.151:0/1032121272 >> 172.21.15.151:0/1032121272 conn(0x55d524736f00 msgr2=0x55d524804ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:17.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.569+0000 7f2dd7400700 0 client.25261 ms_handle_reset on v2:172.21.15.172:6838/1242302810 2024-04-26T01:54:17.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.569+0000 7f2e029c0b80 1 -- 172.21.15.151:0/1032121272 shutdown_connections 2024-04-26T01:54:17.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.569+0000 7f2e029c0b80 1 -- 172.21.15.151:0/1032121272 wait complete. 2024-04-26T01:54:17.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.569+0000 7f2e029c0b80 20 client.25261 trim_cache size 0 max 16384 2024-04-26T01:54:17.571 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-26T01:54:17.571 INFO:teuthology.orchestra.run.smithi151.stdout: "cluster_fsid": "c0e49e64-036d-11ef-bc93-c7b262605968", 2024-04-26T01:54:17.571 INFO:teuthology.orchestra.run.smithi151.stdout: "whoami": 1, 2024-04-26T01:54:17.571 INFO:teuthology.orchestra.run.smithi151.stdout: "id": 24505, 2024-04-26T01:54:17.572 INFO:teuthology.orchestra.run.smithi151.stdout: "want_state": "up:active", 2024-04-26T01:54:17.572 INFO:teuthology.orchestra.run.smithi151.stdout: "state": "up:active", 2024-04-26T01:54:17.572 INFO:teuthology.orchestra.run.smithi151.stdout: "fs_name": "cephfs", 2024-04-26T01:54:17.572 INFO:teuthology.orchestra.run.smithi151.stdout: "rank_uptime": 259.811084032, 2024-04-26T01:54:17.572 INFO:teuthology.orchestra.run.smithi151.stdout: "mdsmap_epoch": 26, 2024-04-26T01:54:17.572 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch": 93, 2024-04-26T01:54:17.572 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch_barrier": 93, 2024-04-26T01:54:17.572 INFO:teuthology.orchestra.run.smithi151.stdout: "uptime": 311.78381169300002 2024-04-26T01:54:17.572 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-26T01:54:17.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.570+0000 7f2df8000700 1 -- 172.21.15.151:0/274225416 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f2dd80a3f60 msgr2=0x7f2dd80a6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:17.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.570+0000 7f2df8000700 1 --2- 172.21.15.151:0/274225416 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f2dd80a3f60 0x7f2dd80a6420 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc009750 tx=0x7f2ddc0096a0 comp rx=0 tx=0).stop 2024-04-26T01:54:17.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.570+0000 7f2df8000700 1 -- 172.21.15.151:0/274225416 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2df0108a50 msgr2=0x7f2df00757c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:17.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.570+0000 7f2df8000700 1 --2- 172.21.15.151:0/274225416 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2df0108a50 0x7f2df00757c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2de800efc0 tx=0x7f2de800c3a0 comp rx=0 tx=0).stop 2024-04-26T01:54:17.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.570+0000 7f2df8000700 1 -- 172.21.15.151:0/274225416 shutdown_connections 2024-04-26T01:54:17.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.570+0000 7f2df8000700 1 --2- 172.21.15.151:0/274225416 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f2dd80a3f60 0x7f2dd80a6420 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:17.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.570+0000 7f2df8000700 1 --2- 172.21.15.151:0/274225416 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2df0108a50 0x7f2df00757c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:17.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.570+0000 7f2df8000700 1 --2- 172.21.15.151:0/274225416 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2df0102a90 0x7f2df0074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:17.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.570+0000 7f2df8000700 1 --2- 172.21.15.151:0/274225416 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2df0103450 0x7f2df0075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:17.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.570+0000 7f2df8000700 1 -- 172.21.15.151:0/274225416 >> 172.21.15.151:0/274225416 conn(0x7f2df0079ba0 msgr2=0x7f2df0101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:17.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.571+0000 7f2df8000700 1 -- 172.21.15.151:0/274225416 shutdown_connections 2024-04-26T01:54:17.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:17.571+0000 7f2df8000700 1 -- 172.21.15.151:0/274225416 wait complete. 2024-04-26T01:54:18.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:54:17 smithi165 ceph-mon[32648]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-04-26T01:54:18.098 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:54:17 smithi151 ceph-mon[28057]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-04-26T01:54:18.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:54:17 smithi172 ceph-mon[32785]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-04-26T01:54:18.590 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph tell mds.b status 2024-04-26T01:54:19.083 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:54:19.573 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.574+0000 7f2228e00640 1 -- 172.21.15.151:0/1848117914 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2224101fc0 msgr2=0x7f22241023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:19.573 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.574+0000 7f2228e00640 1 --2- 172.21.15.151:0/1848117914 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2224101fc0 0x7f22241023a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2214009900 tx=0x7f221402a3f0 comp rx=0 tx=0).stop 2024-04-26T01:54:19.573 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.574+0000 7f2228e00640 1 -- 172.21.15.151:0/1848117914 shutdown_connections 2024-04-26T01:54:19.573 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.574+0000 7f2228e00640 1 --2- 172.21.15.151:0/1848117914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2224107090 0x7f222410ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:19.573 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.574+0000 7f2228e00640 1 --2- 172.21.15.151:0/1848117914 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2224102970 0x7f22241069b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:19.573 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.574+0000 7f2228e00640 1 --2- 172.21.15.151:0/1848117914 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2224101fc0 0x7f22241023a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:19.573 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.574+0000 7f2228e00640 1 -- 172.21.15.151:0/1848117914 >> 172.21.15.151:0/1848117914 conn(0x7f22240fd8a0 msgr2=0x7f22240ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:19.573 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.574+0000 7f2228e00640 1 -- 172.21.15.151:0/1848117914 shutdown_connections 2024-04-26T01:54:19.574 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.574+0000 7f2228e00640 1 -- 172.21.15.151:0/1848117914 wait complete. 2024-04-26T01:54:19.575 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.575+0000 7f2228e00640 1 Processor -- start 2024-04-26T01:54:19.575 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.576+0000 7f2228e00640 1 -- start start 2024-04-26T01:54:19.575 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.576+0000 7f2228e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2224101fc0 0x7f2224074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:19.576 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.576+0000 7f2228e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2224107090 0x7f2224074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:19.576 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.576+0000 7f2228e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2224075ef0 0x7f2224074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:19.576 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.576+0000 7f2228e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2224111fd0 con 0x7f2224075ef0 2024-04-26T01:54:19.576 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.576+0000 7f2228e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2224111e50 con 0x7f2224107090 2024-04-26T01:54:19.576 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.576+0000 7f2228e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2224112150 con 0x7f2224101fc0 2024-04-26T01:54:19.576 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.576+0000 7f2222a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2224107090 0x7f2224074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:19.576 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.576+0000 7f2223400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2224101fc0 0x7f2224074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:19.576 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.576+0000 7f2223400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2224101fc0 0x7f2224074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:48022/0 (socket says 172.21.15.151:48022) 2024-04-26T01:54:19.576 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.576+0000 7f2222a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2224107090 0x7f2224074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:45868/0 (socket says 172.21.15.151:45868) 2024-04-26T01:54:19.576 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.576+0000 7f2223400640 1 -- 172.21.15.151:0/2901619924 learned_addr learned my addr 172.21.15.151:0/2901619924 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:54:19.576 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.577+0000 7f2223e00640 1 --2- 172.21.15.151:0/2901619924 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2224075ef0 0x7f2224074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:19.576 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.577+0000 7f2223400640 1 -- 172.21.15.151:0/2901619924 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2224107090 msgr2=0x7f2224074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:19.576 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.577+0000 7f2223400640 1 --2- 172.21.15.151:0/2901619924 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2224107090 0x7f2224074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:19.577 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.577+0000 7f2223400640 1 -- 172.21.15.151:0/2901619924 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2224075ef0 msgr2=0x7f2224074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:19.577 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.577+0000 7f2223400640 1 --2- 172.21.15.151:0/2901619924 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2224075ef0 0x7f2224074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:19.577 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.577+0000 7f2223400640 1 -- 172.21.15.151:0/2901619924 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22241ab2a0 con 0x7f2224101fc0 2024-04-26T01:54:19.577 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.577+0000 7f2223400640 1 --2- 172.21.15.151:0/2901619924 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2224101fc0 0x7f2224074250 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2214005940 tx=0x7f22140040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:19.577 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.577+0000 7f221be00640 1 -- 172.21.15.151:0/2901619924 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221403f030 con 0x7f2224101fc0 2024-04-26T01:54:19.578 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.578+0000 7f221be00640 1 -- 172.21.15.151:0/2901619924 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22140044c0 con 0x7f2224101fc0 2024-04-26T01:54:19.578 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.578+0000 7f2228e00640 1 -- 172.21.15.151:0/2901619924 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22241ab530 con 0x7f2224101fc0 2024-04-26T01:54:19.578 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.578+0000 7f221be00640 1 -- 172.21.15.151:0/2901619924 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2214035070 con 0x7f2224101fc0 2024-04-26T01:54:19.578 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.578+0000 7f2228e00640 1 -- 172.21.15.151:0/2901619924 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22241aba70 con 0x7f2224101fc0 2024-04-26T01:54:19.579 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.581+0000 7f221be00640 1 -- 172.21.15.151:0/2901619924 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 26) ==== 138931+0+0 (secure 0 0 0) 0x7f2214042030 con 0x7f2224101fc0 2024-04-26T01:54:19.580 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.582+0000 7f222d1ea740 1 Processor -- start 2024-04-26T01:54:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.582+0000 7f222d1ea740 1 -- start start 2024-04-26T01:54:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.583+0000 7f221be00640 1 --2- 172.21.15.151:0/2901619924 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f22080a3f40 0x7f22080a6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.583+0000 7f222d1ea740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2e1b979b0 0x55e2e1b97d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.583+0000 7f222d1ea740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2e1b982d0 0x55e2e1ba2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.583+0000 7f222d1ea740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e2e1ba2b90 0x55e2e1ba4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.583+0000 7f222d1ea740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e2e1afd900 con 0x55e2e1b979b0 2024-04-26T01:54:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.583+0000 7f222d1ea740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e2e1aa3ef0 con 0x55e2e1ba2b90 2024-04-26T01:54:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.583+0000 7f222d1ea740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e2e1af98c0 con 0x55e2e1b982d0 2024-04-26T01:54:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.583+0000 7f221be00640 1 -- 172.21.15.151:0/2901619924 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f22140f0970 con 0x7f2224101fc0 2024-04-26T01:54:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.583+0000 7f2223e00640 1 --2- 172.21.15.151:0/2901619924 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f22080a3f40 0x7f22080a6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.583+0000 7f2223400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2e1b979b0 0x55e2e1b97d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.583+0000 7f2223400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2e1b979b0 0x55e2e1b97d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36486/0 (socket says 172.21.15.151:36486) 2024-04-26T01:54:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.583+0000 7f2223400640 1 -- 172.21.15.151:0/1397362254 learned_addr learned my addr 172.21.15.151:0/1397362254 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:54:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.583+0000 7f2222a00640 1 --2- 172.21.15.151:0/1397362254 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2e1b982d0 0x55e2e1ba2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.584+0000 7f2223e00640 1 --2- 172.21.15.151:0/1397362254 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e2e1ba2b90 0x55e2e1ba4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.584+0000 7f2223400640 1 -- 172.21.15.151:0/1397362254 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2e1b982d0 msgr2=0x55e2e1ba2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.584+0000 7f2223400640 1 --2- 172.21.15.151:0/1397362254 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2e1b982d0 0x55e2e1ba2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.584+0000 7f2223400640 1 -- 172.21.15.151:0/1397362254 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e2e1ba2b90 msgr2=0x55e2e1ba4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.584+0000 7f2223400640 1 --2- 172.21.15.151:0/1397362254 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e2e1ba2b90 0x55e2e1ba4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.584+0000 7f2223400640 1 -- 172.21.15.151:0/1397362254 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2e1810210 con 0x55e2e1b979b0 2024-04-26T01:54:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.584+0000 7f2223400640 1 --2- 172.21.15.151:0/1397362254 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2e1b979b0 0x55e2e1b97d90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f22140bc4f0 tx=0x7f221402a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=accf77989e2e5a08 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.584+0000 7f2223e00640 1 --2- 172.21.15.151:0/2901619924 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f22080a3f40 0x7f22080a6400 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f221c009fd0 tx=0x7f221c013b00 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.584+0000 7f2218c00640 1 -- 172.21.15.151:0/1397362254 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2214049070 con 0x55e2e1b979b0 2024-04-26T01:54:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.584+0000 7f2218c00640 1 -- 172.21.15.151:0/1397362254 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22140c0af0 con 0x55e2e1b979b0 2024-04-26T01:54:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.584+0000 7f2218c00640 1 -- 172.21.15.151:0/1397362254 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22140a7070 con 0x55e2e1b979b0 2024-04-26T01:54:19.587 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.585+0000 7f222d1ea740 1 -- 172.21.15.151:0/1397362254 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2e1b979b0 msgr2=0x55e2e1b97d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.585+0000 7f222d1ea740 1 --2- 172.21.15.151:0/1397362254 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2e1b979b0 0x55e2e1b97d90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f22140bc4f0 tx=0x7f221402a3f0 comp rx=0 tx=0).stop 2024-04-26T01:54:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.585+0000 7f222d1ea740 1 -- 172.21.15.151:0/1397362254 shutdown_connections 2024-04-26T01:54:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.585+0000 7f222d1ea740 1 --2- 172.21.15.151:0/1397362254 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e2e1ba2b90 0x55e2e1ba4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.585+0000 7f222d1ea740 1 --2- 172.21.15.151:0/1397362254 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2e1b982d0 0x55e2e1ba2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.585+0000 7f222d1ea740 1 --2- 172.21.15.151:0/1397362254 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2e1b979b0 0x55e2e1b97d90 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.585+0000 7f222d1ea740 1 -- 172.21.15.151:0/1397362254 >> 172.21.15.151:0/1397362254 conn(0x55e2e1b93ec0 msgr2=0x55e2e1b942a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.585+0000 7f222d1ea740 1 -- 172.21.15.151:0/1397362254 shutdown_connections 2024-04-26T01:54:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.585+0000 7f222d1ea740 1 -- 172.21.15.151:0/1397362254 wait complete. 2024-04-26T01:54:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.586+0000 7f222d1ea740 1 -- start start 2024-04-26T01:54:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.586+0000 7f222d1ea740 1 Processor -- start 2024-04-26T01:54:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.586+0000 7f222d1ea740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2e1b979b0 0x55e2e1b9fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.586+0000 7f222d1ea740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e2e1b982d0 0x55e2e1ba00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.586+0000 7f222d1ea740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2e1ba2b90 0x55e2e1ba0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.586+0000 7f222d1ea740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e2e1afd900 con 0x55e2e1b979b0 2024-04-26T01:54:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.586+0000 7f222d1ea740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e2e1ab7310 con 0x55e2e1b982d0 2024-04-26T01:54:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.586+0000 7f222d1ea740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e2e1ba5580 con 0x55e2e1ba2b90 2024-04-26T01:54:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.587+0000 7f222d1ea740 10 client.0 _ll_register_callbacks cb 0x55e2e1a85cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:54:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.587+0000 7f2223400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2e1b979b0 0x55e2e1b9fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.587+0000 7f2223400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2e1b979b0 0x55e2e1b9fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36496/0 (socket says 172.21.15.151:36496) 2024-04-26T01:54:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.587+0000 7f2223400640 1 -- 172.21.15.151:0/1386495364 learned_addr learned my addr 172.21.15.151:0/1386495364 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:54:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.587+0000 7f2222a00640 1 --2- 172.21.15.151:0/1386495364 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e2e1b982d0 0x55e2e1ba00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.587+0000 7f2223e00640 1 --2- 172.21.15.151:0/1386495364 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2e1ba2b90 0x55e2e1ba0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.587+0000 7f2223400640 1 -- 172.21.15.151:0/1386495364 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2e1ba2b90 msgr2=0x55e2e1ba0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:19.590 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.587+0000 7f2223400640 1 --2- 172.21.15.151:0/1386495364 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2e1ba2b90 0x55e2e1ba0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:19.590 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.587+0000 7f2223400640 1 -- 172.21.15.151:0/1386495364 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e2e1b982d0 msgr2=0x55e2e1ba00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:19.590 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.587+0000 7f2223400640 1 --2- 172.21.15.151:0/1386495364 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e2e1b982d0 0x55e2e1ba00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:19.590 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.587+0000 7f2223400640 1 -- 172.21.15.151:0/1386495364 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2e18dc620 con 0x55e2e1b979b0 2024-04-26T01:54:19.590 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.588+0000 7f2223400640 1 --2- 172.21.15.151:0/1386495364 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2e1b979b0 0x55e2e1b9fbb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f22140ee030 tx=0x7f22140bc8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf725a81db618125 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:19.590 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.588+0000 7f220d600640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-04-26T01:54:19.590 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.588+0000 7f220d600640 1 -- 172.21.15.151:0/1386495364 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2214049030 con 0x55e2e1b979b0 2024-04-26T01:54:19.590 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.588+0000 7f220d600640 1 -- 172.21.15.151:0/1386495364 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f22140b8e60 con 0x55e2e1b979b0 2024-04-26T01:54:19.590 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.588+0000 7f222d1ea740 1 -- 172.21.15.151:0/1386495364 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22140c0af0 con 0x55e2e1b979b0 2024-04-26T01:54:19.590 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.588+0000 7f220d600640 1 -- 172.21.15.151:0/1386495364 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f22140c53b0 con 0x55e2e1b979b0 2024-04-26T01:54:19.590 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.588+0000 7f222d1ea740 10 client.15156 fetch_fsmap learned FSMap version 26 2024-04-26T01:54:19.590 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.588+0000 7f222d1ea740 1 -- 172.21.15.151:0/1386495364 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e2e1bb16f0 con 0x55e2e1b979b0 2024-04-26T01:54:19.591 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.588+0000 7f220d600640 1 -- 172.21.15.151:0/1386495364 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22140c0af0 con 0x55e2e1b979b0 2024-04-26T01:54:19.591 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.589+0000 7f220d600640 1 -- 172.21.15.151:0/1386495364 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f22140eb0e0 con 0x55e2e1b979b0 2024-04-26T01:54:19.591 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.589+0000 7f222d1ea740 10 client.15156 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T01:54:19.591 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.589+0000 7f222d1ea740 10 client.15156 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:54:19.591 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.589+0000 7f222d1ea740 20 client.15156 populate_metadata read hostname 'smithi151' 2024-04-26T01:54:19.591 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.589+0000 7f222d1ea740 1 --2- 172.21.15.151:0/1386495364 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55e2e1bb2a40 0x55e2e1bb4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:19.591 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.589+0000 7f222d1ea740 4 client.15156 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:54:19.591 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.589+0000 7f222d1ea740 1 -- 172.21.15.151:0/1386495364 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e2e1b118b0 con 0x55e2e1bb2a40 2024-04-26T01:54:19.591 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.589+0000 7f2222a00640 1 --2- 172.21.15.151:0/1386495364 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55e2e1bb2a40 0x55e2e1bb4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:19.592 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.590+0000 7f2222a00640 1 --2- 172.21.15.151:0/1386495364 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55e2e1bb2a40 0x55e2e1bb4e30 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa2945e7ceb66726 server_cookie=6751f1e74c7ad3ff in_seq=0 out_seq=0 2024-04-26T01:54:19.592 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.590+0000 7f220d600640 10 client.15156 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:54:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.591+0000 7f220d600640 1 -- 172.21.15.151:0/1386495364 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e2e1b118b0 con 0x55e2e1bb2a40 2024-04-26T01:54:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.591+0000 7f220d600640 10 client.15156 handle_command_reply: tid=0 2024-04-26T01:54:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.591+0000 7f222d1ea740 1 client.15156 shutdown 2024-04-26T01:54:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.591+0000 7f222d1ea740 1 -- 172.21.15.151:0/1386495364 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2e1b979b0 msgr2=0x55e2e1b9fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.591+0000 7f222d1ea740 1 --2- 172.21.15.151:0/1386495364 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2e1b979b0 0x55e2e1b9fbb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f22140ee030 tx=0x7f22140bc8b0 comp rx=0 tx=0).stop 2024-04-26T01:54:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.591+0000 7f222d1ea740 1 -- 172.21.15.151:0/1386495364 shutdown_connections 2024-04-26T01:54:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.591+0000 7f222d1ea740 1 --2- 172.21.15.151:0/1386495364 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55e2e1bb2a40 0x55e2e1bb4e30 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.591+0000 7f222d1ea740 1 --2- 172.21.15.151:0/1386495364 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2e1ba2b90 0x55e2e1ba0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.591+0000 7f222d1ea740 1 --2- 172.21.15.151:0/1386495364 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e2e1b982d0 0x55e2e1ba00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.591+0000 7f222d1ea740 1 --2- 172.21.15.151:0/1386495364 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2e1b979b0 0x55e2e1b9fbb0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.591+0000 7f222d1ea740 1 -- 172.21.15.151:0/1386495364 >> 172.21.15.151:0/1386495364 conn(0x55e2e1b93ec0 msgr2=0x55e2e1b99c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.591+0000 7f220d600640 0 client.15156 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:54:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.591+0000 7f222d1ea740 1 -- 172.21.15.151:0/1386495364 shutdown_connections 2024-04-26T01:54:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.591+0000 7f222d1ea740 1 -- 172.21.15.151:0/1386495364 wait complete. 2024-04-26T01:54:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.591+0000 7f222d1ea740 20 client.15156 trim_cache size 0 max 16384 2024-04-26T01:54:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.607+0000 7f222d1ea740 1 Processor -- start 2024-04-26T01:54:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.607+0000 7f222d1ea740 1 -- start start 2024-04-26T01:54:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.608+0000 7f222d1ea740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2e1b979b0 0x55e2e1bae2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.608+0000 7f222d1ea740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e2e1b982d0 0x55e2e1b99a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.608+0000 7f222d1ea740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2e1ba2b90 0x55e2e1bba670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.608+0000 7f222d1ea740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f22241b6700 con 0x55e2e1b979b0 2024-04-26T01:54:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.608+0000 7f222d1ea740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f22241b3fd0 con 0x55e2e1b982d0 2024-04-26T01:54:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.608+0000 7f222d1ea740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f22241b1e40 con 0x55e2e1ba2b90 2024-04-26T01:54:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.608+0000 7f2223400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2e1b979b0 0x55e2e1bae2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.608+0000 7f2222a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e2e1b982d0 0x55e2e1b99a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.608+0000 7f2222a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e2e1b982d0 0x55e2e1b99a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:45888/0 (socket says 172.21.15.151:45888) 2024-04-26T01:54:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.608+0000 7f2223400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2e1b979b0 0x55e2e1bae2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36500/0 (socket says 172.21.15.151:36500) 2024-04-26T01:54:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.608+0000 7f2222a00640 1 -- 172.21.15.151:0/1216456207 learned_addr learned my addr 172.21.15.151:0/1216456207 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:54:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.608+0000 7f2223e00640 1 --2- 172.21.15.151:0/1216456207 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2e1ba2b90 0x55e2e1bba670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.608+0000 7f2223400640 1 -- 172.21.15.151:0/1216456207 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2e1ba2b90 msgr2=0x55e2e1bba670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.608+0000 7f2223400640 1 --2- 172.21.15.151:0/1216456207 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2e1ba2b90 0x55e2e1bba670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.608+0000 7f2223400640 1 -- 172.21.15.151:0/1216456207 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e2e1b982d0 msgr2=0x55e2e1b99a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.608+0000 7f2223400640 1 --2- 172.21.15.151:0/1216456207 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e2e1b982d0 0x55e2e1b99a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.608+0000 7f2223400640 1 -- 172.21.15.151:0/1216456207 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2e1ba4f80 con 0x55e2e1b979b0 2024-04-26T01:54:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.608+0000 7f2222a00640 1 --2- 172.21.15.151:0/1216456207 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e2e1b982d0 0x55e2e1b99a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:54:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.609+0000 7f2223400640 1 --2- 172.21.15.151:0/1216456207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2e1b979b0 0x55e2e1bae2c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f22140bc4f0 tx=0x7f221403b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=237c1912c2b45e8b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.609+0000 7f220fe00640 1 -- 172.21.15.151:0/1216456207 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2214049050 con 0x55e2e1b979b0 2024-04-26T01:54:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.609+0000 7f220fe00640 1 -- 172.21.15.151:0/1216456207 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22140c0af0 con 0x55e2e1b979b0 2024-04-26T01:54:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.609+0000 7f220fe00640 1 -- 172.21.15.151:0/1216456207 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22140c5180 con 0x55e2e1b979b0 2024-04-26T01:54:19.613 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.610+0000 7f222d1ea740 1 -- 172.21.15.151:0/1216456207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2e1b979b0 msgr2=0x55e2e1bae2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:19.613 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.610+0000 7f222d1ea740 1 --2- 172.21.15.151:0/1216456207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2e1b979b0 0x55e2e1bae2c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f22140bc4f0 tx=0x7f221403b4f0 comp rx=0 tx=0).stop 2024-04-26T01:54:19.613 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.610+0000 7f222d1ea740 1 -- 172.21.15.151:0/1216456207 shutdown_connections 2024-04-26T01:54:19.613 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.610+0000 7f222d1ea740 1 --2- 172.21.15.151:0/1216456207 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2e1ba2b90 0x55e2e1bba670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:19.613 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.610+0000 7f222d1ea740 1 --2- 172.21.15.151:0/1216456207 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e2e1b982d0 0x55e2e1b99a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:19.614 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.610+0000 7f222d1ea740 1 --2- 172.21.15.151:0/1216456207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2e1b979b0 0x55e2e1bae2c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:19.614 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.610+0000 7f222d1ea740 1 -- 172.21.15.151:0/1216456207 >> 172.21.15.151:0/1216456207 conn(0x55e2e1b93ec0 msgr2=0x55e2e1b99360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:19.614 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.610+0000 7f222d1ea740 1 -- 172.21.15.151:0/1216456207 shutdown_connections 2024-04-26T01:54:19.614 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.610+0000 7f222d1ea740 1 -- 172.21.15.151:0/1216456207 wait complete. 2024-04-26T01:54:19.614 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.611+0000 7f222d1ea740 1 -- start start 2024-04-26T01:54:19.614 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.611+0000 7f222d1ea740 1 Processor -- start 2024-04-26T01:54:19.614 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.611+0000 7f222d1ea740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2e1b979b0 0x55e2e1baaa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:19.614 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.611+0000 7f222d1ea740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2e1b982d0 0x55e2e1ba7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:19.614 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.611+0000 7f222d1ea740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e2e1ba2b90 0x55e2e1ba82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:19.614 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.611+0000 7f222d1ea740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f22241b2490 con 0x55e2e1b979b0 2024-04-26T01:54:19.614 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.611+0000 7f222d1ea740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f22241b3900 con 0x55e2e1ba2b90 2024-04-26T01:54:19.614 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.611+0000 7f222d1ea740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f22241b2b60 con 0x55e2e1b982d0 2024-04-26T01:54:19.614 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.611+0000 7f222d1ea740 10 client.0 _ll_register_callbacks cb 0x7f2214005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:54:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.612+0000 7f2223400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2e1b979b0 0x55e2e1baaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.612+0000 7f2223400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2e1b979b0 0x55e2e1baaa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36508/0 (socket says 172.21.15.151:36508) 2024-04-26T01:54:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.612+0000 7f2223400640 1 -- 172.21.15.151:0/1391372960 learned_addr learned my addr 172.21.15.151:0/1391372960 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:54:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.612+0000 7f2222a00640 1 --2- 172.21.15.151:0/1391372960 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2e1b982d0 0x55e2e1ba7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.612+0000 7f2223e00640 1 --2- 172.21.15.151:0/1391372960 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e2e1ba2b90 0x55e2e1ba82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.612+0000 7f2223400640 1 -- 172.21.15.151:0/1391372960 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2e1b982d0 msgr2=0x55e2e1ba7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.612+0000 7f2223400640 1 --2- 172.21.15.151:0/1391372960 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2e1b982d0 0x55e2e1ba7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.612+0000 7f2223400640 1 -- 172.21.15.151:0/1391372960 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e2e1ba2b90 msgr2=0x55e2e1ba82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.612+0000 7f2223400640 1 --2- 172.21.15.151:0/1391372960 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e2e1ba2b90 0x55e2e1ba82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.612+0000 7f2223400640 1 -- 172.21.15.151:0/1391372960 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2e1ba8950 con 0x55e2e1b979b0 2024-04-26T01:54:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.612+0000 7f2222a00640 1 --2- 172.21.15.151:0/1391372960 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2e1b982d0 0x55e2e1ba7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:54:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.612+0000 7f2223400640 1 --2- 172.21.15.151:0/1391372960 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2e1b979b0 0x55e2e1baaa10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f22140bc780 tx=0x7f221402a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ee0b05e6733b824 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.612+0000 7f220d600640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-04-26T01:54:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.613+0000 7f220d600640 1 -- 172.21.15.151:0/1391372960 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2214049020 con 0x55e2e1b979b0 2024-04-26T01:54:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.613+0000 7f220d600640 1 -- 172.21.15.151:0/1391372960 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f22140c4f20 con 0x55e2e1b979b0 2024-04-26T01:54:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.613+0000 7f222d1ea740 1 -- 172.21.15.151:0/1391372960 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22140c0af0 con 0x55e2e1b979b0 2024-04-26T01:54:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.613+0000 7f220d600640 1 -- 172.21.15.151:0/1391372960 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f22140eb0f0 con 0x55e2e1b979b0 2024-04-26T01:54:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.613+0000 7f222d1ea740 10 client.15162 fetch_fsmap learned FSMap version 26 2024-04-26T01:54:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.613+0000 7f222d1ea740 1 -- 172.21.15.151:0/1391372960 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e2e1ba8950 con 0x55e2e1b979b0 2024-04-26T01:54:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.613+0000 7f220d600640 1 -- 172.21.15.151:0/1391372960 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22140c0af0 con 0x55e2e1b979b0 2024-04-26T01:54:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.613+0000 7f220d600640 1 -- 172.21.15.151:0/1391372960 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f22140b8700 con 0x55e2e1b979b0 2024-04-26T01:54:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.613+0000 7f222d1ea740 10 client.15162 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T01:54:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.613+0000 7f222d1ea740 10 client.15162 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:54:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.613+0000 7f222d1ea740 20 client.15162 populate_metadata read hostname 'smithi151' 2024-04-26T01:54:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.613+0000 7f222d1ea740 1 --2- 172.21.15.151:0/1391372960 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55e2e1c190e0 0x55e2e1c1b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.613+0000 7f222d1ea740 4 client.15162 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:54:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.613+0000 7f222d1ea740 1 -- 172.21.15.151:0/1391372960 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e2e1b0b320 con 0x55e2e1c190e0 2024-04-26T01:54:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.614+0000 7f2222a00640 1 --2- 172.21.15.151:0/1391372960 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55e2e1c190e0 0x55e2e1c1b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.614+0000 7f220d600640 10 client.15162 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:54:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.614+0000 7f2222a00640 1 --2- 172.21.15.151:0/1391372960 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55e2e1c190e0 0x55e2e1c1b4d0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c703453c337921c server_cookie=d647ed0f3f1ce145 in_seq=0 out_seq=0 2024-04-26T01:54:19.623 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.624+0000 7f220d600640 1 -- 172.21.15.151:0/1391372960 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e2e1b0b320 con 0x55e2e1c190e0 2024-04-26T01:54:19.624 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.624+0000 7f220d600640 10 client.15162 handle_command_reply: tid=0 2024-04-26T01:54:19.624 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.625+0000 7f222d1ea740 1 client.15162 shutdown 2024-04-26T01:54:19.624 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.625+0000 7f222d1ea740 1 -- 172.21.15.151:0/1391372960 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2e1b979b0 msgr2=0x55e2e1baaa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:19.624 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.625+0000 7f222d1ea740 1 --2- 172.21.15.151:0/1391372960 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2e1b979b0 0x55e2e1baaa10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f22140bc780 tx=0x7f221402a3f0 comp rx=0 tx=0).stop 2024-04-26T01:54:19.624 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.625+0000 7f222d1ea740 1 -- 172.21.15.151:0/1391372960 shutdown_connections 2024-04-26T01:54:19.624 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.625+0000 7f222d1ea740 1 --2- 172.21.15.151:0/1391372960 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55e2e1c190e0 0x55e2e1c1b4d0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:19.624 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.625+0000 7f222d1ea740 1 --2- 172.21.15.151:0/1391372960 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e2e1ba2b90 0x55e2e1ba82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:19.624 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.625+0000 7f222d1ea740 1 --2- 172.21.15.151:0/1391372960 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2e1b982d0 0x55e2e1ba7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:19.624 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.625+0000 7f222d1ea740 1 --2- 172.21.15.151:0/1391372960 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2e1b979b0 0x55e2e1baaa10 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:19.624 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.625+0000 7f222d1ea740 1 -- 172.21.15.151:0/1391372960 >> 172.21.15.151:0/1391372960 conn(0x55e2e1b93ec0 msgr2=0x55e2e1b93020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:19.624 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.625+0000 7f220d600640 0 client.15162 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:54:19.624 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.625+0000 7f222d1ea740 1 -- 172.21.15.151:0/1391372960 shutdown_connections 2024-04-26T01:54:19.624 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.625+0000 7f222d1ea740 1 -- 172.21.15.151:0/1391372960 wait complete. 2024-04-26T01:54:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.625+0000 7f222d1ea740 20 client.15162 trim_cache size 0 max 16384 2024-04-26T01:54:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:{ 2024-04-26T01:54:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "message": "", 2024-04-26T01:54:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "return_code": 0 2024-04-26T01:54:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:} 2024-04-26T01:54:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.626+0000 7f2228e00640 1 -- 172.21.15.151:0/2901619924 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f22080a3f40 msgr2=0x7f22080a6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.626+0000 7f2228e00640 1 --2- 172.21.15.151:0/2901619924 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f22080a3f40 0x7f22080a6400 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f221c009fd0 tx=0x7f221c013b00 comp rx=0 tx=0).stop 2024-04-26T01:54:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.626+0000 7f2228e00640 1 -- 172.21.15.151:0/2901619924 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2224101fc0 msgr2=0x7f2224074250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.626+0000 7f2228e00640 1 --2- 172.21.15.151:0/2901619924 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2224101fc0 0x7f2224074250 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2214005940 tx=0x7f22140040c0 comp rx=0 tx=0).stop 2024-04-26T01:54:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.626+0000 7f2228e00640 1 -- 172.21.15.151:0/2901619924 shutdown_connections 2024-04-26T01:54:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.626+0000 7f2228e00640 1 --2- 172.21.15.151:0/2901619924 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f22080a3f40 0x7f22080a6400 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.626+0000 7f2228e00640 1 --2- 172.21.15.151:0/2901619924 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2224075ef0 0x7f2224074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.626+0000 7f2228e00640 1 --2- 172.21.15.151:0/2901619924 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2224107090 0x7f2224074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.626+0000 7f2228e00640 1 --2- 172.21.15.151:0/2901619924 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2224101fc0 0x7f2224074250 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.626+0000 7f2228e00640 1 -- 172.21.15.151:0/2901619924 >> 172.21.15.151:0/2901619924 conn(0x7f22240fd8a0 msgr2=0x7f22240fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.626+0000 7f2228e00640 1 -- 172.21.15.151:0/2901619924 shutdown_connections 2024-04-26T01:54:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:19.626+0000 7f2228e00640 1 -- 172.21.15.151:0/2901619924 wait complete. 2024-04-26T01:54:20.055 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:54:19 smithi151 ceph-mon[28057]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-04-26T01:54:20.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:54:19 smithi172 ceph-mon[32785]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-04-26T01:54:20.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:54:19 smithi165 ceph-mon[32648]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-04-26T01:54:21.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.338+0000 7fd98ec00700 1 -- 172.21.15.151:0/3312752023 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd988108a30 msgr2=0x7fd988108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:21.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.338+0000 7fd98ec00700 1 --2- 172.21.15.151:0/3312752023 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd988108a30 0x7fd988108e10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd9780098f0 tx=0x7fd978009c00 comp rx=0 tx=0).stop 2024-04-26T01:54:21.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.338+0000 7fd98ec00700 1 -- 172.21.15.151:0/3312752023 shutdown_connections 2024-04-26T01:54:21.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.338+0000 7fd98ec00700 1 --2- 172.21.15.151:0/3312752023 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd98806b0f0 0x7fd98810f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:21.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.338+0000 7fd98ec00700 1 --2- 172.21.15.151:0/3312752023 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd988108a30 0x7fd988108e10 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:21.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.338+0000 7fd98ec00700 1 --2- 172.21.15.151:0/3312752023 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd98806a750 0x7fd98806abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:21.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.338+0000 7fd98ec00700 1 -- 172.21.15.151:0/3312752023 >> 172.21.15.151:0/3312752023 conn(0x7fd9880788f0 msgr2=0x7fd988078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:21.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.339+0000 7fd98ec00700 1 -- 172.21.15.151:0/3312752023 shutdown_connections 2024-04-26T01:54:21.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.339+0000 7fd98ec00700 1 -- 172.21.15.151:0/3312752023 wait complete. 2024-04-26T01:54:21.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.340+0000 7fd98ec00700 1 Processor -- start 2024-04-26T01:54:21.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.340+0000 7fd98ec00700 1 -- start start 2024-04-26T01:54:21.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.341+0000 7fd98ec00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd98806a750 0x7fd988101a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:21.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.341+0000 7fd98ec00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd98806b0f0 0x7fd988101f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:21.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.341+0000 7fd98ec00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd988108a30 0x7fd988102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:21.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.341+0000 7fd98ec00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd988112c00 con 0x7fd98806b0f0 2024-04-26T01:54:21.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.341+0000 7fd98ec00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd988112a80 con 0x7fd98806a750 2024-04-26T01:54:21.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.341+0000 7fd98ec00700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd988112d80 con 0x7fd988108a30 2024-04-26T01:54:21.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.341+0000 7fd987400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd98806b0f0 0x7fd988101f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:21.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.341+0000 7fd987e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd98806a750 0x7fd988101a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:21.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.341+0000 7fd987e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd98806a750 0x7fd988101a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:50884/0 (socket says 172.21.15.151:50884) 2024-04-26T01:54:21.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.341+0000 7fd987e00700 1 -- 172.21.15.151:0/241996955 learned_addr learned my addr 172.21.15.151:0/241996955 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:54:21.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.341+0000 7fd987e00700 1 -- 172.21.15.151:0/241996955 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd988108a30 msgr2=0x7fd988102490 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T01:54:21.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.341+0000 7fd987e00700 1 --2- 172.21.15.151:0/241996955 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd988108a30 0x7fd988102490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:21.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.341+0000 7fd987e00700 1 -- 172.21.15.151:0/241996955 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd98806b0f0 msgr2=0x7fd988101f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:21.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.341+0000 7fd987e00700 1 --2- 172.21.15.151:0/241996955 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd98806b0f0 0x7fd988101f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:21.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.341+0000 7fd987e00700 1 -- 172.21.15.151:0/241996955 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9881a76e0 con 0x7fd98806a750 2024-04-26T01:54:21.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.342+0000 7fd987400700 1 --2- 172.21.15.151:0/241996955 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd98806b0f0 0x7fd988101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:54:21.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.342+0000 7fd987e00700 1 --2- 172.21.15.151:0/241996955 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd98806a750 0x7fd988101a10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd978009230 tx=0x7fd97800fa70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:21.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.342+0000 7fd984c00700 1 -- 172.21.15.151:0/241996955 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd978005200 con 0x7fd98806a750 2024-04-26T01:54:21.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.342+0000 7fd984c00700 1 -- 172.21.15.151:0/241996955 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd97800b560 con 0x7fd98806a750 2024-04-26T01:54:21.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.342+0000 7fd984c00700 1 -- 172.21.15.151:0/241996955 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd978003b10 con 0x7fd98806a750 2024-04-26T01:54:21.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.342+0000 7fd98ec00700 1 -- 172.21.15.151:0/241996955 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9881a7970 con 0x7fd98806a750 2024-04-26T01:54:21.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.342+0000 7fd98ec00700 1 -- 172.21.15.151:0/241996955 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9881a7d70 con 0x7fd98806a750 2024-04-26T01:54:21.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.346+0000 7fd984c00700 1 -- 172.21.15.151:0/241996955 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 26) ==== 138931+0+0 (secure 0 0 0) 0x7fd97800ba50 con 0x7fd98806a750 2024-04-26T01:54:21.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.347+0000 7fd984c00700 1 --2- 172.21.15.151:0/241996955 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fd96c0a3fe0 0x7fd96c0a64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:21.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.347+0000 7fd984c00700 1 -- 172.21.15.151:0/241996955 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fd9780d1980 con 0x7fd98806a750 2024-04-26T01:54:21.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.347+0000 7fd987400700 1 --2- 172.21.15.151:0/241996955 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fd96c0a3fe0 0x7fd96c0a64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:21.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.348+0000 7fd9994b1b80 1 Processor -- start 2024-04-26T01:54:21.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.348+0000 7fd9994b1b80 1 -- start start 2024-04-26T01:54:21.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.349+0000 7fd9994b1b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564b40be9b40 0x564b40be9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:21.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.349+0000 7fd9994b1b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564b40bea460 0x564b40bf3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:21.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.349+0000 7fd9994b1b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564b40bf4380 0x564b40bf6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:21.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.349+0000 7fd9994b1b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564b40b33e40 con 0x564b40bea460 2024-04-26T01:54:21.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.349+0000 7fd9994b1b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564b40ada6e0 con 0x564b40bf4380 2024-04-26T01:54:21.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.349+0000 7fd9994b1b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564b40b2fe00 con 0x564b40be9b40 2024-04-26T01:54:21.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.349+0000 7fd987400700 1 --2- 172.21.15.151:0/241996955 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fd96c0a3fe0 0x7fd96c0a64a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd974009750 tx=0x7fd974009310 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:21.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.349+0000 7fd98cc00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564b40bf4380 0x564b40bf6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:21.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.349+0000 7fd987400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564b40bea460 0x564b40bf3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:21.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.349+0000 7fd98cc00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564b40bf4380 0x564b40bf6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:50898/0 (socket says 172.21.15.151:50898) 2024-04-26T01:54:21.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.349+0000 7fd98cc00700 1 -- 172.21.15.151:0/509400344 learned_addr learned my addr 172.21.15.151:0/509400344 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:54:21.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.349+0000 7fd987e00700 1 --2- 172.21.15.151:0/509400344 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564b40be9b40 0x564b40be9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:21.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.349+0000 7fd98cc00700 1 -- 172.21.15.151:0/509400344 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564b40be9b40 msgr2=0x564b40be9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:21.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.349+0000 7fd98cc00700 1 --2- 172.21.15.151:0/509400344 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564b40be9b40 0x564b40be9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:21.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.349+0000 7fd98cc00700 1 -- 172.21.15.151:0/509400344 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564b40bea460 msgr2=0x564b40bf3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:21.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.349+0000 7fd98cc00700 1 --2- 172.21.15.151:0/509400344 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564b40bea460 0x564b40bf3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:21.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.349+0000 7fd98cc00700 1 -- 172.21.15.151:0/509400344 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b40ad0630 con 0x564b40bf4380 2024-04-26T01:54:21.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.350+0000 7fd98cc00700 1 --2- 172.21.15.151:0/509400344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564b40bf4380 0x564b40bf6770 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd98000e380 tx=0x7fd98000b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7f3c90c8c5ce0b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:21.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.350+0000 7fd97d600700 1 -- 172.21.15.151:0/509400344 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98000ee30 con 0x564b40bf4380 2024-04-26T01:54:21.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.350+0000 7fd97d600700 1 -- 172.21.15.151:0/509400344 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd980004bb0 con 0x564b40bf4380 2024-04-26T01:54:21.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.350+0000 7fd97d600700 1 -- 172.21.15.151:0/509400344 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd980004300 con 0x564b40bf4380 2024-04-26T01:54:21.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.351+0000 7fd9994b1b80 1 -- 172.21.15.151:0/509400344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564b40bf4380 msgr2=0x564b40bf6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:21.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.351+0000 7fd9994b1b80 1 --2- 172.21.15.151:0/509400344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564b40bf4380 0x564b40bf6770 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd98000e380 tx=0x7fd98000b040 comp rx=0 tx=0).stop 2024-04-26T01:54:21.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.351+0000 7fd9994b1b80 1 -- 172.21.15.151:0/509400344 shutdown_connections 2024-04-26T01:54:21.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.351+0000 7fd9994b1b80 1 --2- 172.21.15.151:0/509400344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564b40bf4380 0x564b40bf6770 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:21.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.351+0000 7fd9994b1b80 1 --2- 172.21.15.151:0/509400344 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564b40be9b40 0x564b40be9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:21.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.351+0000 7fd9994b1b80 1 --2- 172.21.15.151:0/509400344 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564b40bea460 0x564b40bf3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:21.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.351+0000 7fd9994b1b80 1 -- 172.21.15.151:0/509400344 >> 172.21.15.151:0/509400344 conn(0x564b40b0f550 msgr2=0x564b40bdc750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:21.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.351+0000 7fd9994b1b80 1 -- 172.21.15.151:0/509400344 shutdown_connections 2024-04-26T01:54:21.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.351+0000 7fd9994b1b80 1 -- 172.21.15.151:0/509400344 wait complete. 2024-04-26T01:54:21.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.352+0000 7fd9994b1b80 1 -- start start 2024-04-26T01:54:21.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.352+0000 7fd9994b1b80 1 Processor -- start 2024-04-26T01:54:21.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.353+0000 7fd9994b1b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564b40be9b40 0x564b40bf3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:21.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.353+0000 7fd9994b1b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564b40bea460 0x564b40bf0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:21.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.353+0000 7fd9994b1b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564b40bf4380 0x564b40bf1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:21.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.353+0000 7fd9994b1b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564b40bf6e40 con 0x564b40bea460 2024-04-26T01:54:21.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.353+0000 7fd9994b1b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564b40aed810 con 0x564b40bf4380 2024-04-26T01:54:21.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.353+0000 7fd9994b1b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564b40bf6fc0 con 0x564b40be9b40 2024-04-26T01:54:21.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.353+0000 7fd9994b1b80 10 client.0 _ll_register_callbacks cb 0x564b40acc6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:54:21.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.353+0000 7fd987e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564b40be9b40 0x564b40bf3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:21.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.353+0000 7fd987e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564b40be9b40 0x564b40bf3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:45034/0 (socket says 172.21.15.151:45034) 2024-04-26T01:54:21.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.353+0000 7fd987e00700 1 -- 172.21.15.151:0/3313932605 learned_addr learned my addr 172.21.15.151:0/3313932605 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:54:21.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.353+0000 7fd98cc00700 1 --2- 172.21.15.151:0/3313932605 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564b40bf4380 0x564b40bf1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:21.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.354+0000 7fd987e00700 1 -- 172.21.15.151:0/3313932605 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564b40bf4380 msgr2=0x564b40bf1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:21.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.354+0000 7fd987e00700 1 --2- 172.21.15.151:0/3313932605 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564b40bf4380 0x564b40bf1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:21.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.354+0000 7fd987400700 1 --2- 172.21.15.151:0/3313932605 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564b40bea460 0x564b40bf0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:21.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.354+0000 7fd987e00700 1 -- 172.21.15.151:0/3313932605 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564b40bea460 msgr2=0x564b40bf0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:21.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.354+0000 7fd987e00700 1 --2- 172.21.15.151:0/3313932605 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564b40bea460 0x564b40bf0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:21.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.354+0000 7fd987e00700 1 -- 172.21.15.151:0/3313932605 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b40a81880 con 0x564b40be9b40 2024-04-26T01:54:21.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.354+0000 7fd987400700 1 --2- 172.21.15.151:0/3313932605 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564b40bea460 0x564b40bf0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T01:54:21.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.354+0000 7fd987e00700 1 --2- 172.21.15.151:0/3313932605 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564b40be9b40 0x564b40bf3b20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd978009fd0 tx=0x7fd97809d140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8248ce636aaaa2cc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:21.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.355+0000 7fd972000700 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-26T01:54:21.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.355+0000 7fd972000700 1 -- 172.21.15.151:0/3313932605 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9780a1b90 con 0x564b40be9b40 2024-04-26T01:54:21.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.355+0000 7fd972000700 1 -- 172.21.15.151:0/3313932605 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fd9780880c0 con 0x564b40be9b40 2024-04-26T01:54:21.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.355+0000 7fd9994b1b80 1 -- 172.21.15.151:0/3313932605 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd980004bb0 con 0x564b40be9b40 2024-04-26T01:54:21.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.355+0000 7fd972000700 1 -- 172.21.15.151:0/3313932605 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd9780bb050 con 0x564b40be9b40 2024-04-26T01:54:21.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.355+0000 7fd9994b1b80 10 client.25288 fetch_fsmap learned FSMap version 26 2024-04-26T01:54:21.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.355+0000 7fd9994b1b80 1 -- 172.21.15.151:0/3313932605 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564b40acaaa0 con 0x564b40be9b40 2024-04-26T01:54:21.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.356+0000 7fd972000700 1 -- 172.21.15.151:0/3313932605 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd980004bb0 con 0x564b40be9b40 2024-04-26T01:54:21.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.356+0000 7fd972000700 1 -- 172.21.15.151:0/3313932605 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fd9780b5020 con 0x564b40be9b40 2024-04-26T01:54:21.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.356+0000 7fd9994b1b80 10 client.25288 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T01:54:21.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.356+0000 7fd9994b1b80 10 client.25288 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T01:54:21.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.356+0000 7fd9994b1b80 20 client.25288 populate_metadata read hostname 'smithi151' 2024-04-26T01:54:21.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.356+0000 7fd9994b1b80 1 --2- 172.21.15.151:0/3313932605 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x564b40c03c20 0x564b40c05090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:21.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.356+0000 7fd9994b1b80 4 client.25288 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:54:21.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.356+0000 7fd9994b1b80 1 -- 172.21.15.151:0/3313932605 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b40b41630 con 0x564b40c03c20 2024-04-26T01:54:21.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.356+0000 7fd987400700 1 --2- 172.21.15.151:0/3313932605 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x564b40c03c20 0x564b40c05090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:21.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.357+0000 7fd987400700 1 --2- 172.21.15.151:0/3313932605 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x564b40c03c20 0x564b40c05090 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a54a310dbe8ecd23 server_cookie=41d2aa5fd4156838 in_seq=0 out_seq=0 2024-04-26T01:54:21.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.357+0000 7fd972000700 10 client.25288 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:54:21.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.358+0000 7fd972000700 1 -- 172.21.15.151:0/3313932605 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564b40b41630 con 0x564b40c03c20 2024-04-26T01:54:21.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.358+0000 7fd972000700 10 client.25288 handle_command_reply: tid=0 2024-04-26T01:54:21.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.358+0000 7fd9994b1b80 1 client.25288 shutdown 2024-04-26T01:54:21.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.358+0000 7fd9994b1b80 1 -- 172.21.15.151:0/3313932605 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564b40be9b40 msgr2=0x564b40bf3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:21.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.358+0000 7fd9994b1b80 1 --2- 172.21.15.151:0/3313932605 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564b40be9b40 0x564b40bf3b20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd978009fd0 tx=0x7fd97809d140 comp rx=0 tx=0).stop 2024-04-26T01:54:21.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.358+0000 7fd9994b1b80 1 -- 172.21.15.151:0/3313932605 shutdown_connections 2024-04-26T01:54:21.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.358+0000 7fd9994b1b80 1 --2- 172.21.15.151:0/3313932605 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x564b40c03c20 0x564b40c05090 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:21.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.358+0000 7fd9994b1b80 1 --2- 172.21.15.151:0/3313932605 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564b40bf4380 0x564b40bf1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:21.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.358+0000 7fd9994b1b80 1 --2- 172.21.15.151:0/3313932605 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564b40be9b40 0x564b40bf3b20 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:21.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.358+0000 7fd9994b1b80 1 --2- 172.21.15.151:0/3313932605 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564b40bea460 0x564b40bf0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:21.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.358+0000 7fd972000700 0 client.25288 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:54:21.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.358+0000 7fd9994b1b80 1 -- 172.21.15.151:0/3313932605 >> 172.21.15.151:0/3313932605 conn(0x564b40b0f550 msgr2=0x564b40bdb950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:21.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.359+0000 7fd9994b1b80 1 -- 172.21.15.151:0/3313932605 shutdown_connections 2024-04-26T01:54:21.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.359+0000 7fd9994b1b80 1 -- 172.21.15.151:0/3313932605 wait complete. 2024-04-26T01:54:21.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.359+0000 7fd9994b1b80 20 client.25288 trim_cache size 0 max 16384 2024-04-26T01:54:21.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.382+0000 7fd9994b1b80 1 Processor -- start 2024-04-26T01:54:21.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.382+0000 7fd9994b1b80 1 -- start start 2024-04-26T01:54:21.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.382+0000 7fd9994b1b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564b40be9b40 0x564b40c03550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:21.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.382+0000 7fd9994b1b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564b40bea460 0x564b40c05090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:21.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.382+0000 7fd9994b1b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564b40bf4380 0x564b40bf1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:21.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.382+0000 7fd9994b1b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564b40c033d0 con 0x564b40be9b40 2024-04-26T01:54:21.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.382+0000 7fd9994b1b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564b40c03aa0 con 0x564b40bf4380 2024-04-26T01:54:21.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.382+0000 7fd9994b1b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564b40c04170 con 0x564b40bea460 2024-04-26T01:54:21.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.383+0000 7fd987400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564b40bea460 0x564b40c05090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:21.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.383+0000 7fd987e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564b40be9b40 0x564b40c03550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:21.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.383+0000 7fd987400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564b40bea460 0x564b40c05090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:45038/0 (socket says 172.21.15.151:45038) 2024-04-26T01:54:21.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.383+0000 7fd987400700 1 -- 172.21.15.151:0/858072077 learned_addr learned my addr 172.21.15.151:0/858072077 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:54:21.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.383+0000 7fd98cc00700 1 --2- 172.21.15.151:0/858072077 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564b40bf4380 0x564b40bf1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:21.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.383+0000 7fd987400700 1 -- 172.21.15.151:0/858072077 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564b40bf4380 msgr2=0x564b40bf1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:21.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.383+0000 7fd987400700 1 --2- 172.21.15.151:0/858072077 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564b40bf4380 0x564b40bf1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:21.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.383+0000 7fd987400700 1 -- 172.21.15.151:0/858072077 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564b40be9b40 msgr2=0x564b40c03550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:21.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.383+0000 7fd987400700 1 --2- 172.21.15.151:0/858072077 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564b40be9b40 0x564b40c03550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:21.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.383+0000 7fd987400700 1 -- 172.21.15.151:0/858072077 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b40c05ff0 con 0x564b40bea460 2024-04-26T01:54:21.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.383+0000 7fd98cc00700 1 --2- 172.21.15.151:0/858072077 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564b40bf4380 0x564b40bf1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:54:21.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.383+0000 7fd987400700 1 --2- 172.21.15.151:0/858072077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564b40bea460 0x564b40c05090 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd988078fd0 tx=0x7fd974014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8459516276794146 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:21.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.384+0000 7fd97cc00700 1 -- 172.21.15.151:0/858072077 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd97402d070 con 0x564b40bea460 2024-04-26T01:54:21.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.384+0000 7fd97cc00700 1 -- 172.21.15.151:0/858072077 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd974004100 con 0x564b40bea460 2024-04-26T01:54:21.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.384+0000 7fd97cc00700 1 -- 172.21.15.151:0/858072077 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd97401f690 con 0x564b40bea460 2024-04-26T01:54:21.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.385+0000 7fd9994b1b80 1 -- 172.21.15.151:0/858072077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564b40bea460 msgr2=0x564b40c05090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:21.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.385+0000 7fd9994b1b80 1 --2- 172.21.15.151:0/858072077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564b40bea460 0x564b40c05090 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd988078fd0 tx=0x7fd974014000 comp rx=0 tx=0).stop 2024-04-26T01:54:21.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.385+0000 7fd9994b1b80 1 -- 172.21.15.151:0/858072077 shutdown_connections 2024-04-26T01:54:21.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.385+0000 7fd9994b1b80 1 --2- 172.21.15.151:0/858072077 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564b40bf4380 0x564b40bf1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:21.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.385+0000 7fd9994b1b80 1 --2- 172.21.15.151:0/858072077 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564b40be9b40 0x564b40c03550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:21.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.385+0000 7fd9994b1b80 1 --2- 172.21.15.151:0/858072077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564b40bea460 0x564b40c05090 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:21.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.385+0000 7fd9994b1b80 1 -- 172.21.15.151:0/858072077 >> 172.21.15.151:0/858072077 conn(0x564b40b0f550 msgr2=0x564b40bf0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:21.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.385+0000 7fd9994b1b80 1 -- 172.21.15.151:0/858072077 shutdown_connections 2024-04-26T01:54:21.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.385+0000 7fd9994b1b80 1 -- 172.21.15.151:0/858072077 wait complete. 2024-04-26T01:54:21.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.385+0000 7fd9994b1b80 1 -- start start 2024-04-26T01:54:21.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.386+0000 7fd9994b1b80 1 Processor -- start 2024-04-26T01:54:21.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.386+0000 7fd9994b1b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564b40be9b40 0x564b40c082d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:21.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.386+0000 7fd9994b1b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564b40bea460 0x564b40c08810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:21.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.386+0000 7fd9994b1b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564b40bf4380 0x564b40c0ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:21.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.386+0000 7fd9994b1b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564b40c04f10 con 0x564b40bf4380 2024-04-26T01:54:21.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.386+0000 7fd9994b1b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564b40c04840 con 0x564b40be9b40 2024-04-26T01:54:21.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.386+0000 7fd9994b1b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564b40c055e0 con 0x564b40bea460 2024-04-26T01:54:21.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.386+0000 7fd9994b1b80 10 client.0 _ll_register_callbacks cb 0x564b40acc6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:54:21.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.387+0000 7fd98cc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564b40bf4380 0x564b40c0ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:21.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.387+0000 7fd98cc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564b40bf4380 0x564b40c0ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:40766/0 (socket says 172.21.15.151:40766) 2024-04-26T01:54:21.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.387+0000 7fd98cc00700 1 -- 172.21.15.151:0/971824490 learned_addr learned my addr 172.21.15.151:0/971824490 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:54:21.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.387+0000 7fd987e00700 1 --2- 172.21.15.151:0/971824490 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564b40be9b40 0x564b40c082d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:21.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.387+0000 7fd987400700 1 --2- 172.21.15.151:0/971824490 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564b40bea460 0x564b40c08810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:21.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.387+0000 7fd987e00700 1 -- 172.21.15.151:0/971824490 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564b40bea460 msgr2=0x564b40c08810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:21.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.387+0000 7fd987e00700 1 --2- 172.21.15.151:0/971824490 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564b40bea460 0x564b40c08810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:21.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.387+0000 7fd987e00700 1 -- 172.21.15.151:0/971824490 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564b40bf4380 msgr2=0x564b40c0ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:21.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.387+0000 7fd987e00700 1 --2- 172.21.15.151:0/971824490 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564b40bf4380 0x564b40c0ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:21.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.387+0000 7fd987e00700 1 -- 172.21.15.151:0/971824490 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b40c0cfd0 con 0x564b40be9b40 2024-04-26T01:54:21.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.387+0000 7fd98cc00700 1 --2- 172.21.15.151:0/971824490 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564b40bf4380 0x564b40c0ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:54:21.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.387+0000 7fd987e00700 1 --2- 172.21.15.151:0/971824490 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564b40be9b40 0x564b40c082d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd97809a1b0 tx=0x7fd9780882c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e30d47ad6d87fa0a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:21.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.387+0000 7fd972000700 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T01:54:21.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.388+0000 7fd972000700 1 -- 172.21.15.151:0/971824490 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9780a1b90 con 0x564b40be9b40 2024-04-26T01:54:21.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.388+0000 7fd972000700 1 -- 172.21.15.151:0/971824490 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fd978021380 con 0x564b40be9b40 2024-04-26T01:54:21.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.388+0000 7fd9994b1b80 1 -- 172.21.15.151:0/971824490 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd974004100 con 0x564b40be9b40 2024-04-26T01:54:21.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.388+0000 7fd972000700 1 -- 172.21.15.151:0/971824490 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd978021580 con 0x564b40be9b40 2024-04-26T01:54:21.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.388+0000 7fd9994b1b80 10 client.25277 fetch_fsmap learned FSMap version 26 2024-04-26T01:54:21.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.388+0000 7fd9994b1b80 1 -- 172.21.15.151:0/971824490 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564b40c6b4b0 con 0x564b40be9b40 2024-04-26T01:54:21.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.388+0000 7fd972000700 1 -- 172.21.15.151:0/971824490 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd974004100 con 0x564b40be9b40 2024-04-26T01:54:21.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.389+0000 7fd972000700 1 -- 172.21.15.151:0/971824490 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fd9780b5040 con 0x564b40be9b40 2024-04-26T01:54:21.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.389+0000 7fd9994b1b80 10 client.25277 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T01:54:21.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.389+0000 7fd9994b1b80 10 client.25277 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T01:54:21.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.389+0000 7fd9994b1b80 20 client.25277 populate_metadata read hostname 'smithi151' 2024-04-26T01:54:21.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.389+0000 7fd9994b1b80 1 --2- 172.21.15.151:0/971824490 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x564b40c6fed0 0x564b40c6ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:21.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.389+0000 7fd9994b1b80 4 client.25277 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T01:54:21.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.389+0000 7fd9994b1b80 1 -- 172.21.15.151:0/971824490 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "status"}) -- 0x564b40b3b6d0 con 0x564b40c6fed0 2024-04-26T01:54:21.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.389+0000 7fd987400700 1 --2- 172.21.15.151:0/971824490 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x564b40c6fed0 0x564b40c6ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:21.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.390+0000 7fd972000700 10 client.25277 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:54:21.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.390+0000 7fd987400700 1 --2- 172.21.15.151:0/971824490 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x564b40c6fed0 0x564b40c6ebe0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=227713e4659a128e server_cookie=237ae280236da0b1 in_seq=0 out_seq=0 2024-04-26T01:54:21.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.390+0000 7fd972000700 1 -- 172.21.15.151:0/971824490 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x564b40b3b6d0 con 0x564b40c6fed0 2024-04-26T01:54:21.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.390+0000 7fd972000700 10 client.25277 handle_command_reply: tid=0 2024-04-26T01:54:21.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.390+0000 7fd9994b1b80 1 client.25277 shutdown 2024-04-26T01:54:21.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.391+0000 7fd9994b1b80 1 -- 172.21.15.151:0/971824490 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564b40be9b40 msgr2=0x564b40c082d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:21.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.391+0000 7fd9994b1b80 1 --2- 172.21.15.151:0/971824490 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564b40be9b40 0x564b40c082d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd97809a1b0 tx=0x7fd9780882c0 comp rx=0 tx=0).stop 2024-04-26T01:54:21.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.391+0000 7fd9994b1b80 1 -- 172.21.15.151:0/971824490 shutdown_connections 2024-04-26T01:54:21.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.391+0000 7fd9994b1b80 1 --2- 172.21.15.151:0/971824490 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564b40bf4380 0x564b40c0ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:21.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.391+0000 7fd9994b1b80 1 --2- 172.21.15.151:0/971824490 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x564b40c6fed0 0x564b40c6ebe0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:21.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.391+0000 7fd9994b1b80 1 --2- 172.21.15.151:0/971824490 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564b40be9b40 0x564b40c082d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:21.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.391+0000 7fd9994b1b80 1 --2- 172.21.15.151:0/971824490 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564b40bea460 0x564b40c08810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:21.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.391+0000 7fd9994b1b80 1 -- 172.21.15.151:0/971824490 >> 172.21.15.151:0/971824490 conn(0x564b40b0f550 msgr2=0x564b40bebcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:21.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.391+0000 7fd972000700 0 client.25277 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:54:21.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.392+0000 7fd9994b1b80 1 -- 172.21.15.151:0/971824490 shutdown_connections 2024-04-26T01:54:21.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.392+0000 7fd9994b1b80 1 -- 172.21.15.151:0/971824490 wait complete. 2024-04-26T01:54:21.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.392+0000 7fd9994b1b80 20 client.25277 trim_cache size 0 max 16384 2024-04-26T01:54:21.393 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-26T01:54:21.393 INFO:teuthology.orchestra.run.smithi151.stdout: "cluster_fsid": "c0e49e64-036d-11ef-bc93-c7b262605968", 2024-04-26T01:54:21.393 INFO:teuthology.orchestra.run.smithi151.stdout: "whoami": 0, 2024-04-26T01:54:21.393 INFO:teuthology.orchestra.run.smithi151.stdout: "id": 24509, 2024-04-26T01:54:21.393 INFO:teuthology.orchestra.run.smithi151.stdout: "want_state": "up:active", 2024-04-26T01:54:21.393 INFO:teuthology.orchestra.run.smithi151.stdout: "state": "up:active", 2024-04-26T01:54:21.393 INFO:teuthology.orchestra.run.smithi151.stdout: "fs_name": "cephfs", 2024-04-26T01:54:21.393 INFO:teuthology.orchestra.run.smithi151.stdout: "rank_uptime": 279.71636532799999, 2024-04-26T01:54:21.394 INFO:teuthology.orchestra.run.smithi151.stdout: "mdsmap_epoch": 26, 2024-04-26T01:54:21.394 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch": 93, 2024-04-26T01:54:21.394 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch_barrier": 93, 2024-04-26T01:54:21.394 INFO:teuthology.orchestra.run.smithi151.stdout: "uptime": 303.27592991699998 2024-04-26T01:54:21.394 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-26T01:54:21.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.393+0000 7fd98ec00700 1 -- 172.21.15.151:0/241996955 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fd96c0a3fe0 msgr2=0x7fd96c0a64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:21.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.393+0000 7fd98ec00700 1 --2- 172.21.15.151:0/241996955 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fd96c0a3fe0 0x7fd96c0a64a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd974009750 tx=0x7fd974009310 comp rx=0 tx=0).stop 2024-04-26T01:54:21.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.393+0000 7fd98ec00700 1 -- 172.21.15.151:0/241996955 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd98806a750 msgr2=0x7fd988101a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:21.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.393+0000 7fd98ec00700 1 --2- 172.21.15.151:0/241996955 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd98806a750 0x7fd988101a10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd978009230 tx=0x7fd97800fa70 comp rx=0 tx=0).stop 2024-04-26T01:54:21.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.393+0000 7fd98ec00700 1 -- 172.21.15.151:0/241996955 shutdown_connections 2024-04-26T01:54:21.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.393+0000 7fd98ec00700 1 --2- 172.21.15.151:0/241996955 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fd96c0a3fe0 0x7fd96c0a64a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:21.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.393+0000 7fd98ec00700 1 --2- 172.21.15.151:0/241996955 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd988108a30 0x7fd988102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:21.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.393+0000 7fd98ec00700 1 --2- 172.21.15.151:0/241996955 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd98806a750 0x7fd988101a10 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:21.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.393+0000 7fd98ec00700 1 --2- 172.21.15.151:0/241996955 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd98806b0f0 0x7fd988101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:21.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.393+0000 7fd98ec00700 1 -- 172.21.15.151:0/241996955 >> 172.21.15.151:0/241996955 conn(0x7fd9880788f0 msgr2=0x7fd9880ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:21.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.394+0000 7fd98ec00700 1 -- 172.21.15.151:0/241996955 shutdown_connections 2024-04-26T01:54:21.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:21.394+0000 7fd98ec00700 1 -- 172.21.15.151:0/241996955 wait complete. 2024-04-26T01:54:22.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:54:21 smithi151 ceph-mon[28057]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T01:54:22.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:54:21 smithi172 ceph-mon[32785]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T01:54:22.312 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-04-26T01:54:22.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:54:21 smithi165 ceph-mon[32648]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T01:54:24.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:54:23 smithi151 ceph-mon[28057]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-04-26T01:54:24.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:54:23 smithi172 ceph-mon[32785]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-04-26T01:54:24.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:54:23 smithi165 ceph-mon[32648]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-04-26T01:54:26.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:54:25 smithi151 ceph-mon[28057]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-04-26T01:54:26.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:54:25 smithi172 ceph-mon[32785]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-04-26T01:54:26.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:54:25 smithi165 ceph-mon[32648]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-04-26T01:54:28.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:54:27 smithi151 ceph-mon[28057]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-04-26T01:54:28.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:54:27 smithi172 ceph-mon[32785]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-04-26T01:54:28.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:54:27 smithi165 ceph-mon[32648]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-04-26T01:54:30.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:54:29 smithi172 ceph-mon[32785]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-04-26T01:54:30.330 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:54:29 smithi151 ceph-mon[28057]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-04-26T01:54:30.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:54:29 smithi165 ceph-mon[32648]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-04-26T01:54:32.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:54:31 smithi151 ceph-mon[28057]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-26T01:54:32.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:54:31 smithi172 ceph-mon[32785]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-26T01:54:32.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:54:31 smithi165 ceph-mon[32648]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-26T01:54:32.776 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.777+0000 7fe522000640 1 -- 172.21.15.151:0/489528164 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe51c10c330 msgr2=0x7fe51c10e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:32.776 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.777+0000 7fe522000640 1 --2- 172.21.15.151:0/489528164 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe51c10c330 0x7fe51c10e7d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe514009ca0 tx=0x7fe51402a950 comp rx=0 tx=0).stop 2024-04-26T01:54:32.776 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.778+0000 7fe522000640 1 -- 172.21.15.151:0/489528164 shutdown_connections 2024-04-26T01:54:32.777 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.778+0000 7fe522000640 1 --2- 172.21.15.151:0/489528164 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe51c10c330 0x7fe51c10e7d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:32.777 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.778+0000 7fe522000640 1 --2- 172.21.15.151:0/489528164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe51c103830 0x7fe51c10bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:32.777 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.778+0000 7fe522000640 1 --2- 172.21.15.151:0/489528164 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe51c102f10 0x7fe51c1032f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:32.777 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.778+0000 7fe522000640 1 -- 172.21.15.151:0/489528164 >> 172.21.15.151:0/489528164 conn(0x7fe51c0fc770 msgr2=0x7fe51c0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:32.777 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.778+0000 7fe522000640 1 -- 172.21.15.151:0/489528164 shutdown_connections 2024-04-26T01:54:32.777 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.778+0000 7fe522000640 1 -- 172.21.15.151:0/489528164 wait complete. 2024-04-26T01:54:32.777 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.779+0000 7fe522000640 1 Processor -- start 2024-04-26T01:54:32.778 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.779+0000 7fe522000640 1 -- start start 2024-04-26T01:54:32.778 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.780+0000 7fe522000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe51c102f10 0x7fe51c196320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:32.778 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.780+0000 7fe522000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe51c103830 0x7fe51c196860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:32.779 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.780+0000 7fe522000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe51c10c330 0x7fe51c19ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:32.779 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.780+0000 7fe522000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c10fe20 con 0x7fe51c103830 2024-04-26T01:54:32.779 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.780+0000 7fe522000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c10fca0 con 0x7fe51c10c330 2024-04-26T01:54:32.779 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.780+0000 7fe522000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c10ffa0 con 0x7fe51c102f10 2024-04-26T01:54:32.780 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.780+0000 7fe51be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe51c103830 0x7fe51c196860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:32.780 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.780+0000 7fe51be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe51c103830 0x7fe51c196860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52198/0 (socket says 172.21.15.151:52198) 2024-04-26T01:54:32.780 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.780+0000 7fe51be00640 1 -- 172.21.15.151:0/606267205 learned_addr learned my addr 172.21.15.151:0/606267205 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:54:32.780 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.780+0000 7fe520c00640 1 --2- 172.21.15.151:0/606267205 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe51c102f10 0x7fe51c196320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:32.780 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.780+0000 7fe521600640 1 --2- 172.21.15.151:0/606267205 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe51c10c330 0x7fe51c19ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:32.780 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.781+0000 7fe51be00640 1 -- 172.21.15.151:0/606267205 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe51c102f10 msgr2=0x7fe51c196320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:32.780 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.781+0000 7fe51be00640 1 --2- 172.21.15.151:0/606267205 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe51c102f10 0x7fe51c196320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:32.780 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.781+0000 7fe51be00640 1 -- 172.21.15.151:0/606267205 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe51c10c330 msgr2=0x7fe51c19ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:32.781 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.781+0000 7fe51be00640 1 --2- 172.21.15.151:0/606267205 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe51c10c330 0x7fe51c19ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:32.781 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.781+0000 7fe51be00640 1 -- 172.21.15.151:0/606267205 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe51c19b2c0 con 0x7fe51c103830 2024-04-26T01:54:32.781 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.781+0000 7fe51be00640 1 --2- 172.21.15.151:0/606267205 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe51c103830 0x7fe51c196860 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe50800e9f0 tx=0x7fe50800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:32.781 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.782+0000 7fe519600640 1 -- 172.21.15.151:0/606267205 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50800ccc0 con 0x7fe51c103830 2024-04-26T01:54:32.781 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.782+0000 7fe519600640 1 -- 172.21.15.151:0/606267205 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5080108e0 con 0x7fe51c103830 2024-04-26T01:54:32.781 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.782+0000 7fe519600640 1 -- 172.21.15.151:0/606267205 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe508018880 con 0x7fe51c103830 2024-04-26T01:54:32.781 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.782+0000 7fe522000640 1 -- 172.21.15.151:0/606267205 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe51c19b5b0 con 0x7fe51c103830 2024-04-26T01:54:32.781 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.782+0000 7fe522000640 1 -- 172.21.15.151:0/606267205 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe51c1a2e40 con 0x7fe51c103830 2024-04-26T01:54:32.784 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.786+0000 7fe519600640 1 -- 172.21.15.151:0/606267205 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 138931+0+0 (secure 0 0 0) 0x7fe508018a20 con 0x7fe51c103830 2024-04-26T01:54:32.786 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.787+0000 7fe52641f740 1 Processor -- start 2024-04-26T01:54:32.786 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.787+0000 7fe52641f740 1 -- start start 2024-04-26T01:54:32.786 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.787+0000 7fe519600640 1 --2- 172.21.15.151:0/606267205 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fe5040a3f30 0x7fe5040a63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:32.786 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.787+0000 7fe519600640 1 -- 172.21.15.151:0/606267205 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fe5080d1ba0 con 0x7fe51c103830 2024-04-26T01:54:32.787 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.787+0000 7fe52641f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561e0e3949b0 0x561e0e394d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:32.787 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.787+0000 7fe52641f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e0e3952d0 0x561e0e39f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:32.787 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.787+0000 7fe52641f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561e0e39fb90 0x561e0e3a1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:32.787 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.787+0000 7fe52641f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x561e0e2fa900 con 0x561e0e39fb90 2024-04-26T01:54:32.787 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.787+0000 7fe52641f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561e0e2a0ef0 con 0x561e0e3952d0 2024-04-26T01:54:32.787 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.787+0000 7fe52641f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561e0e2f68c0 con 0x561e0e3949b0 2024-04-26T01:54:32.787 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.788+0000 7fe521600640 1 --2- 172.21.15.151:0/606267205 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fe5040a3f30 0x7fe5040a63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:32.787 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.788+0000 7fe520c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561e0e3949b0 0x561e0e394d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:32.787 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.788+0000 7fe51be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e0e3952d0 0x561e0e39f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:32.787 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.788+0000 7fe520c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561e0e3949b0 0x561e0e394d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:48582/0 (socket says 172.21.15.151:48582) 2024-04-26T01:54:32.787 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.788+0000 7fe520c00640 1 -- 172.21.15.151:0/1565457634 learned_addr learned my addr 172.21.15.151:0/1565457634 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:54:32.787 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.788+0000 7fe51be00640 1 --2- 172.21.15.151:0/1565457634 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e0e3952d0 0x561e0e39f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:42654/0 (socket says 172.21.15.151:42654) 2024-04-26T01:54:32.789 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.788+0000 7fe521600640 1 --2- 172.21.15.151:0/1565457634 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561e0e39fb90 0x561e0e3a1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:32.789 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.788+0000 7fe520c00640 1 -- 172.21.15.151:0/1565457634 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e0e3952d0 msgr2=0x561e0e39f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:32.789 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.788+0000 7fe520c00640 1 --2- 172.21.15.151:0/1565457634 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e0e3952d0 0x561e0e39f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:32.789 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.788+0000 7fe520c00640 1 -- 172.21.15.151:0/1565457634 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561e0e39fb90 msgr2=0x561e0e3a1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:32.789 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.788+0000 7fe520c00640 1 --2- 172.21.15.151:0/1565457634 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561e0e39fb90 0x561e0e3a1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.788+0000 7fe520c00640 1 -- 172.21.15.151:0/1565457634 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e0e00d210 con 0x561e0e3949b0 2024-04-26T01:54:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.788+0000 7fe51be00640 1 --2- 172.21.15.151:0/1565457634 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e0e3952d0 0x561e0e39f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:54:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.788+0000 7fe521600640 1 --2- 172.21.15.151:0/1565457634 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561e0e39fb90 0x561e0e3a1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T01:54:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.789+0000 7fe520c00640 1 --2- 172.21.15.151:0/1565457634 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561e0e3949b0 0x561e0e394d90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe510009cf0 tx=0x7fe510011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=264f7c343f892da0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.789+0000 7fe521600640 1 --2- 172.21.15.151:0/606267205 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fe5040a3f30 0x7fe5040a63f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe514006010 tx=0x7fe514009820 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.789+0000 7fe50dc00640 1 -- 172.21.15.151:0/1565457634 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe510004230 con 0x561e0e3949b0 2024-04-26T01:54:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.789+0000 7fe50dc00640 1 -- 172.21.15.151:0/1565457634 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5100043d0 con 0x561e0e3949b0 2024-04-26T01:54:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.789+0000 7fe50dc00640 1 -- 172.21.15.151:0/1565457634 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe510019880 con 0x561e0e3949b0 2024-04-26T01:54:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.790+0000 7fe52641f740 1 -- 172.21.15.151:0/1565457634 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561e0e3949b0 msgr2=0x561e0e394d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.790+0000 7fe52641f740 1 --2- 172.21.15.151:0/1565457634 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561e0e3949b0 0x561e0e394d90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe510009cf0 tx=0x7fe510011810 comp rx=0 tx=0).stop 2024-04-26T01:54:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.790+0000 7fe52641f740 1 -- 172.21.15.151:0/1565457634 shutdown_connections 2024-04-26T01:54:32.791 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.790+0000 7fe52641f740 1 --2- 172.21.15.151:0/1565457634 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561e0e39fb90 0x561e0e3a1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:32.791 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.790+0000 7fe52641f740 1 --2- 172.21.15.151:0/1565457634 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e0e3952d0 0x561e0e39f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:32.791 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.790+0000 7fe52641f740 1 --2- 172.21.15.151:0/1565457634 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561e0e3949b0 0x561e0e394d90 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:32.791 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.790+0000 7fe52641f740 1 -- 172.21.15.151:0/1565457634 >> 172.21.15.151:0/1565457634 conn(0x561e0e390ec0 msgr2=0x561e0e3912a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:32.791 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.790+0000 7fe52641f740 1 -- 172.21.15.151:0/1565457634 shutdown_connections 2024-04-26T01:54:32.791 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.790+0000 7fe52641f740 1 -- 172.21.15.151:0/1565457634 wait complete. 2024-04-26T01:54:32.791 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.791+0000 7fe52641f740 1 -- start start 2024-04-26T01:54:32.791 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.791+0000 7fe52641f740 1 Processor -- start 2024-04-26T01:54:32.793 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.791+0000 7fe52641f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561e0e3949b0 0x561e0e39ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:32.794 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.791+0000 7fe52641f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561e0e3952d0 0x561e0e39cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:32.794 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.791+0000 7fe52641f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e0e39fb90 0x561e0e39d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:32.794 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.791+0000 7fe52641f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x561e0e3a27c0 con 0x561e0e3949b0 2024-04-26T01:54:32.794 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.791+0000 7fe52641f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561e0e2b4310 con 0x561e0e39fb90 2024-04-26T01:54:32.794 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.791+0000 7fe52641f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561e0e3a2940 con 0x561e0e3952d0 2024-04-26T01:54:32.794 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.791+0000 7fe52641f740 10 client.0 _ll_register_callbacks cb 0x561e0e282cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:54:32.794 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.792+0000 7fe520c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561e0e3949b0 0x561e0e39ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:32.794 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.792+0000 7fe520c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561e0e3949b0 0x561e0e39ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52216/0 (socket says 172.21.15.151:52216) 2024-04-26T01:54:32.794 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.792+0000 7fe520c00640 1 -- 172.21.15.151:0/736092742 learned_addr learned my addr 172.21.15.151:0/736092742 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:54:32.794 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.792+0000 7fe51be00640 1 --2- 172.21.15.151:0/736092742 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561e0e3952d0 0x561e0e39cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:32.794 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.792+0000 7fe521600640 1 --2- 172.21.15.151:0/736092742 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e0e39fb90 0x561e0e39d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:32.794 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.792+0000 7fe51be00640 1 -- 172.21.15.151:0/736092742 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e0e39fb90 msgr2=0x561e0e39d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:32.795 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.792+0000 7fe51be00640 1 --2- 172.21.15.151:0/736092742 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e0e39fb90 0x561e0e39d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:32.795 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.792+0000 7fe51be00640 1 -- 172.21.15.151:0/736092742 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561e0e3949b0 msgr2=0x561e0e39ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:32.795 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.792+0000 7fe51be00640 1 --2- 172.21.15.151:0/736092742 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561e0e3949b0 0x561e0e39ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:32.795 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.792+0000 7fe51be00640 1 -- 172.21.15.151:0/736092742 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e0e0d9620 con 0x561e0e3952d0 2024-04-26T01:54:32.795 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.792+0000 7fe520c00640 1 --2- 172.21.15.151:0/736092742 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561e0e3949b0 0x561e0e39ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:54:32.795 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.792+0000 7fe51be00640 1 --2- 172.21.15.151:0/736092742 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561e0e3952d0 0x561e0e39cf60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe5080cf120 tx=0x7fe50809d440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae9dc39134b07b37 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:32.795 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.792+0000 7fe502000640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-26T01:54:32.795 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.793+0000 7fe502000640 1 -- 172.21.15.151:0/736092742 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5080a65e0 con 0x561e0e3952d0 2024-04-26T01:54:32.795 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.793+0000 7fe502000640 1 -- 172.21.15.151:0/736092742 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fe5080bcb30 con 0x561e0e3952d0 2024-04-26T01:54:32.795 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.793+0000 7fe52641f740 1 -- 172.21.15.151:0/736092742 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5100043d0 con 0x561e0e3952d0 2024-04-26T01:54:32.795 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.793+0000 7fe502000640 1 -- 172.21.15.151:0/736092742 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe5080bcd30 con 0x561e0e3952d0 2024-04-26T01:54:32.795 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.793+0000 7fe52641f740 10 client.25306 fetch_fsmap learned FSMap version 26 2024-04-26T01:54:32.795 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.793+0000 7fe52641f740 1 -- 172.21.15.151:0/736092742 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561e0e3bf900 con 0x561e0e3952d0 2024-04-26T01:54:32.795 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.794+0000 7fe502000640 1 -- 172.21.15.151:0/736092742 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5100043d0 con 0x561e0e3952d0 2024-04-26T01:54:32.796 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.794+0000 7fe502000640 1 -- 172.21.15.151:0/736092742 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fe508099770 con 0x561e0e3952d0 2024-04-26T01:54:32.796 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.794+0000 7fe52641f740 10 client.25306 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T01:54:32.796 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.794+0000 7fe52641f740 10 client.25306 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:54:32.796 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.794+0000 7fe52641f740 20 client.25306 populate_metadata read hostname 'smithi151' 2024-04-26T01:54:32.796 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.794+0000 7fe52641f740 1 --2- 172.21.15.151:0/736092742 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x561e0e3c0e10 0x561e0e3c32b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:32.796 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.794+0000 7fe52641f740 4 client.25306 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:54:32.796 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.794+0000 7fe52641f740 1 -- 172.21.15.151:0/736092742 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e0e30e8b0 con 0x561e0e3c0e10 2024-04-26T01:54:32.796 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.795+0000 7fe520c00640 1 --2- 172.21.15.151:0/736092742 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x561e0e3c0e10 0x561e0e3c32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:32.796 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.795+0000 7fe520c00640 1 --2- 172.21.15.151:0/736092742 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x561e0e3c0e10 0x561e0e3c32b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51a60e391348a1d3 server_cookie=c85ebecc67092aa2 in_seq=0 out_seq=0 2024-04-26T01:54:32.796 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.795+0000 7fe502000640 10 client.25306 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:54:32.797 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.796+0000 7fe502000640 1 -- 172.21.15.151:0/736092742 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561e0e30e8b0 con 0x561e0e3c0e10 2024-04-26T01:54:32.797 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.796+0000 7fe502000640 10 client.25306 handle_command_reply: tid=0 2024-04-26T01:54:32.797 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.796+0000 7fe52641f740 1 client.25306 shutdown 2024-04-26T01:54:32.797 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.796+0000 7fe52641f740 1 -- 172.21.15.151:0/736092742 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561e0e3952d0 msgr2=0x561e0e39cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:32.797 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.796+0000 7fe52641f740 1 --2- 172.21.15.151:0/736092742 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561e0e3952d0 0x561e0e39cf60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe5080cf120 tx=0x7fe50809d440 comp rx=0 tx=0).stop 2024-04-26T01:54:32.797 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.796+0000 7fe52641f740 1 -- 172.21.15.151:0/736092742 shutdown_connections 2024-04-26T01:54:32.798 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.796+0000 7fe52641f740 1 --2- 172.21.15.151:0/736092742 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x561e0e3c0e10 0x561e0e3c32b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:32.798 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.796+0000 7fe52641f740 1 --2- 172.21.15.151:0/736092742 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e0e39fb90 0x561e0e39d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:32.798 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.796+0000 7fe52641f740 1 --2- 172.21.15.151:0/736092742 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561e0e3952d0 0x561e0e39cf60 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:32.798 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.796+0000 7fe52641f740 1 --2- 172.21.15.151:0/736092742 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561e0e3949b0 0x561e0e39ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:32.798 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.796+0000 7fe52641f740 1 -- 172.21.15.151:0/736092742 >> 172.21.15.151:0/736092742 conn(0x561e0e390ec0 msgr2=0x561e0e396bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:32.798 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.797+0000 7fe502000640 0 client.25306 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:54:32.798 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.797+0000 7fe52641f740 1 -- 172.21.15.151:0/736092742 shutdown_connections 2024-04-26T01:54:32.798 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.797+0000 7fe52641f740 1 -- 172.21.15.151:0/736092742 wait complete. 2024-04-26T01:54:32.798 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.797+0000 7fe52641f740 20 client.25306 trim_cache size 0 max 16384 2024-04-26T01:54:32.811 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.813+0000 7fe52641f740 1 Processor -- start 2024-04-26T01:54:32.811 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.813+0000 7fe52641f740 1 -- start start 2024-04-26T01:54:32.812 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.813+0000 7fe52641f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e0e3949b0 0x561e0e3ab680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:32.812 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.813+0000 7fe52641f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561e0e3952d0 0x561e0e39ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:32.812 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.813+0000 7fe52641f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561e0e39fb90 0x561e0e3c8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:32.812 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.813+0000 7fe52641f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe4fc010e90 con 0x561e0e3952d0 2024-04-26T01:54:32.812 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.813+0000 7fe52641f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe4fc00e760 con 0x561e0e3949b0 2024-04-26T01:54:32.812 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.813+0000 7fe52641f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe4fc00c5b0 con 0x561e0e39fb90 2024-04-26T01:54:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.813+0000 7fe51be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561e0e3952d0 0x561e0e39ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.814+0000 7fe520c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e0e3949b0 0x561e0e3ab680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.814+0000 7fe520c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e0e3949b0 0x561e0e3ab680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:42674/0 (socket says 172.21.15.151:42674) 2024-04-26T01:54:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.814+0000 7fe521600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561e0e39fb90 0x561e0e3c8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.814+0000 7fe520c00640 1 -- 172.21.15.151:0/3664270377 learned_addr learned my addr 172.21.15.151:0/3664270377 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:54:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.814+0000 7fe520c00640 1 -- 172.21.15.151:0/3664270377 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561e0e39fb90 msgr2=0x561e0e3c8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.814+0000 7fe520c00640 1 --2- 172.21.15.151:0/3664270377 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561e0e39fb90 0x561e0e3c8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.814+0000 7fe520c00640 1 -- 172.21.15.151:0/3664270377 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561e0e3952d0 msgr2=0x561e0e39ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.814+0000 7fe520c00640 1 --2- 172.21.15.151:0/3664270377 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561e0e3952d0 0x561e0e39ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.814+0000 7fe520c00640 1 -- 172.21.15.151:0/3664270377 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e0e395100 con 0x561e0e3949b0 2024-04-26T01:54:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.814+0000 7fe520c00640 1 --2- 172.21.15.151:0/3664270377 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e0e3949b0 0x561e0e3ab680 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe5100099b0 tx=0x7fe510011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd76dc8857390c9d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.815+0000 7fe50d200640 1 -- 172.21.15.151:0/3664270377 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe510031070 con 0x561e0e3949b0 2024-04-26T01:54:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.815+0000 7fe50d200640 1 -- 172.21.15.151:0/3664270377 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe51002b360 con 0x561e0e3949b0 2024-04-26T01:54:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.815+0000 7fe50d200640 1 -- 172.21.15.151:0/3664270377 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe510003850 con 0x561e0e3949b0 2024-04-26T01:54:32.815 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.815+0000 7fe52641f740 1 -- 172.21.15.151:0/3664270377 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e0e3949b0 msgr2=0x561e0e3ab680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.815+0000 7fe52641f740 1 --2- 172.21.15.151:0/3664270377 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e0e3949b0 0x561e0e3ab680 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe5100099b0 tx=0x7fe510011810 comp rx=0 tx=0).stop 2024-04-26T01:54:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.815+0000 7fe52641f740 1 -- 172.21.15.151:0/3664270377 shutdown_connections 2024-04-26T01:54:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.815+0000 7fe52641f740 1 --2- 172.21.15.151:0/3664270377 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561e0e39fb90 0x561e0e3c8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.815+0000 7fe52641f740 1 --2- 172.21.15.151:0/3664270377 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561e0e3952d0 0x561e0e39ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.815+0000 7fe52641f740 1 --2- 172.21.15.151:0/3664270377 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e0e3949b0 0x561e0e3ab680 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.815+0000 7fe52641f740 1 -- 172.21.15.151:0/3664270377 >> 172.21.15.151:0/3664270377 conn(0x561e0e390ec0 msgr2=0x561e0e39b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.816+0000 7fe52641f740 1 -- 172.21.15.151:0/3664270377 shutdown_connections 2024-04-26T01:54:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.816+0000 7fe52641f740 1 -- 172.21.15.151:0/3664270377 wait complete. 2024-04-26T01:54:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.816+0000 7fe52641f740 1 -- start start 2024-04-26T01:54:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.816+0000 7fe52641f740 1 Processor -- start 2024-04-26T01:54:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.817+0000 7fe52641f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561e0e3949b0 0x561e0e3a4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.817+0000 7fe52641f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561e0e3952d0 0x561e0e3a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.817+0000 7fe52641f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e0e39fb90 0x561e0e425430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.817+0000 7fe52641f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe4fc00cc20 con 0x561e0e3952d0 2024-04-26T01:54:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.817+0000 7fe52641f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe4fc00e090 con 0x561e0e39fb90 2024-04-26T01:54:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.817+0000 7fe52641f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe4fc00d2f0 con 0x561e0e3949b0 2024-04-26T01:54:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.817+0000 7fe520c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561e0e3949b0 0x561e0e3a4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.817+0000 7fe52641f740 10 client.0 _ll_register_callbacks cb 0x7fe50809d630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:54:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.817+0000 7fe520c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561e0e3949b0 0x561e0e3a4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:48606/0 (socket says 172.21.15.151:48606) 2024-04-26T01:54:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.817+0000 7fe520c00640 1 -- 172.21.15.151:0/3769987011 learned_addr learned my addr 172.21.15.151:0/3769987011 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:54:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.817+0000 7fe51be00640 1 --2- 172.21.15.151:0/3769987011 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561e0e3952d0 0x561e0e3a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.817+0000 7fe520c00640 1 -- 172.21.15.151:0/3769987011 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e0e39fb90 msgr2=0x561e0e425430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.817+0000 7fe520c00640 1 --2- 172.21.15.151:0/3769987011 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e0e39fb90 0x561e0e425430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.817+0000 7fe520c00640 1 -- 172.21.15.151:0/3769987011 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561e0e3952d0 msgr2=0x561e0e3a50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.818+0000 7fe520c00640 1 --2- 172.21.15.151:0/3769987011 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561e0e3952d0 0x561e0e3a50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.818+0000 7fe520c00640 1 -- 172.21.15.151:0/3769987011 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e0e425970 con 0x561e0e3949b0 2024-04-26T01:54:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.818+0000 7fe51be00640 1 --2- 172.21.15.151:0/3769987011 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561e0e3952d0 0x561e0e3a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:54:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.818+0000 7fe520c00640 1 --2- 172.21.15.151:0/3769987011 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561e0e3949b0 0x561e0e3a4b60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe510005fd0 tx=0x7fe510005a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1858d5577ed9cc6b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.818+0000 7fe502000640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-26T01:54:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.818+0000 7fe502000640 1 -- 172.21.15.151:0/3769987011 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51003a060 con 0x561e0e3949b0 2024-04-26T01:54:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.818+0000 7fe52641f740 1 -- 172.21.15.151:0/3769987011 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe51002b360 con 0x561e0e3949b0 2024-04-26T01:54:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.818+0000 7fe502000640 1 -- 172.21.15.151:0/3769987011 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fe51002bae0 con 0x561e0e3949b0 2024-04-26T01:54:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.818+0000 7fe502000640 1 -- 172.21.15.151:0/3769987011 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe510031040 con 0x561e0e3949b0 2024-04-26T01:54:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.818+0000 7fe52641f740 10 client.25312 fetch_fsmap learned FSMap version 26 2024-04-26T01:54:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.818+0000 7fe52641f740 1 -- 172.21.15.151:0/3769987011 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561e0e426a80 con 0x561e0e3949b0 2024-04-26T01:54:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.819+0000 7fe502000640 1 -- 172.21.15.151:0/3769987011 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe51002b360 con 0x561e0e3949b0 2024-04-26T01:54:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.819+0000 7fe502000640 1 -- 172.21.15.151:0/3769987011 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fe51003d020 con 0x561e0e3949b0 2024-04-26T01:54:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.819+0000 7fe52641f740 10 client.25312 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T01:54:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.819+0000 7fe52641f740 10 client.25312 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:54:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.819+0000 7fe52641f740 20 client.25312 populate_metadata read hostname 'smithi151' 2024-04-26T01:54:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.819+0000 7fe52641f740 1 --2- 172.21.15.151:0/3769987011 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x561e0e426f50 0x561e0e429340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.819+0000 7fe52641f740 4 client.25312 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:54:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.819+0000 7fe52641f740 1 -- 172.21.15.151:0/3769987011 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561e0e308320 con 0x561e0e426f50 2024-04-26T01:54:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.820+0000 7fe51be00640 1 --2- 172.21.15.151:0/3769987011 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x561e0e426f50 0x561e0e429340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.820+0000 7fe51be00640 1 --2- 172.21.15.151:0/3769987011 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x561e0e426f50 0x561e0e429340 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b878b0931551fac server_cookie=4fad96b8dc4f94ce in_seq=0 out_seq=0 2024-04-26T01:54:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.820+0000 7fe502000640 10 client.25312 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:54:32.823 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.824+0000 7fe502000640 1 -- 172.21.15.151:0/3769987011 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561e0e308320 con 0x561e0e426f50 2024-04-26T01:54:32.823 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.824+0000 7fe502000640 10 client.25312 handle_command_reply: tid=0 2024-04-26T01:54:32.823 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.824+0000 7fe52641f740 1 client.25312 shutdown 2024-04-26T01:54:32.823 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.824+0000 7fe52641f740 1 -- 172.21.15.151:0/3769987011 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561e0e3949b0 msgr2=0x561e0e3a4b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:32.823 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.824+0000 7fe52641f740 1 --2- 172.21.15.151:0/3769987011 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561e0e3949b0 0x561e0e3a4b60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe510005fd0 tx=0x7fe510005a70 comp rx=0 tx=0).stop 2024-04-26T01:54:32.823 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.824+0000 7fe52641f740 1 -- 172.21.15.151:0/3769987011 shutdown_connections 2024-04-26T01:54:32.823 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.824+0000 7fe52641f740 1 --2- 172.21.15.151:0/3769987011 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x561e0e426f50 0x561e0e429340 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:32.824 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.824+0000 7fe52641f740 1 --2- 172.21.15.151:0/3769987011 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e0e39fb90 0x561e0e425430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:32.824 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.824+0000 7fe52641f740 1 --2- 172.21.15.151:0/3769987011 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561e0e3952d0 0x561e0e3a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:32.824 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.824+0000 7fe52641f740 1 --2- 172.21.15.151:0/3769987011 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561e0e3949b0 0x561e0e3a4b60 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:32.824 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.824+0000 7fe502000640 0 client.25312 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:54:32.824 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.824+0000 7fe52641f740 1 -- 172.21.15.151:0/3769987011 >> 172.21.15.151:0/3769987011 conn(0x561e0e390ec0 msgr2=0x561e0e39c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:32.824 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.825+0000 7fe52641f740 1 -- 172.21.15.151:0/3769987011 shutdown_connections 2024-04-26T01:54:32.824 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.825+0000 7fe52641f740 1 -- 172.21.15.151:0/3769987011 wait complete. 2024-04-26T01:54:32.824 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.825+0000 7fe52641f740 20 client.25312 trim_cache size 0 max 16384 2024-04-26T01:54:32.824 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:{ 2024-04-26T01:54:32.824 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "message": "", 2024-04-26T01:54:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "return_code": 0 2024-04-26T01:54:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:} 2024-04-26T01:54:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.826+0000 7fe522000640 1 -- 172.21.15.151:0/606267205 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fe5040a3f30 msgr2=0x7fe5040a63f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.826+0000 7fe522000640 1 --2- 172.21.15.151:0/606267205 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fe5040a3f30 0x7fe5040a63f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe514006010 tx=0x7fe514009820 comp rx=0 tx=0).stop 2024-04-26T01:54:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.826+0000 7fe522000640 1 -- 172.21.15.151:0/606267205 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe51c103830 msgr2=0x7fe51c196860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.826+0000 7fe522000640 1 --2- 172.21.15.151:0/606267205 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe51c103830 0x7fe51c196860 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe50800e9f0 tx=0x7fe50800eeb0 comp rx=0 tx=0).stop 2024-04-26T01:54:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.826+0000 7fe522000640 1 -- 172.21.15.151:0/606267205 shutdown_connections 2024-04-26T01:54:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.826+0000 7fe522000640 1 --2- 172.21.15.151:0/606267205 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fe5040a3f30 0x7fe5040a63f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.826+0000 7fe522000640 1 --2- 172.21.15.151:0/606267205 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe51c10c330 0x7fe51c19ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.826+0000 7fe522000640 1 --2- 172.21.15.151:0/606267205 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe51c103830 0x7fe51c196860 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.826+0000 7fe522000640 1 --2- 172.21.15.151:0/606267205 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe51c102f10 0x7fe51c196320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.826+0000 7fe522000640 1 -- 172.21.15.151:0/606267205 >> 172.21.15.151:0/606267205 conn(0x7fe51c0fc770 msgr2=0x7fe51c0fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.826+0000 7fe522000640 1 -- 172.21.15.151:0/606267205 shutdown_connections 2024-04-26T01:54:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:32.826+0000 7fe522000640 1 -- 172.21.15.151:0/606267205 wait complete. 2024-04-26T01:54:34.148 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:54:33 smithi165 ceph-mon[32648]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-26T01:54:34.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:54:33 smithi172 ceph-mon[32785]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-26T01:54:34.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:54:33 smithi151 ceph-mon[28057]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-26T01:54:36.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:54:35 smithi172 ceph-mon[32785]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T01:54:36.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:54:35 smithi165 ceph-mon[32648]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T01:54:36.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:54:35 smithi151 ceph-mon[28057]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T01:54:37.314 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.6 2024-04-26T01:54:37.314 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:54:37.842 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:54:38.179 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:54:37 smithi151 ceph-mon[28057]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T01:54:38.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:54:37 smithi172 ceph-mon[32785]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T01:54:38.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:54:37 smithi165 ceph-mon[32648]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T01:54:39.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:39.987+0000 7f7697200700 1 -- 172.21.15.151:0/3353607827 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7698102070 msgr2=0x7f769810ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:39.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:39.987+0000 7f7697200700 1 --2- 172.21.15.151:0/3353607827 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7698102070 0x7f769810ed30 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f76840098f0 tx=0x7f7684009c00 comp rx=0 tx=0).stop 2024-04-26T01:54:39.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:39.987+0000 7f7697200700 1 -- 172.21.15.151:0/3353607827 shutdown_connections 2024-04-26T01:54:39.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:39.987+0000 7f7697200700 1 --2- 172.21.15.151:0/3353607827 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f769810f270 0x7f7698111660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:39.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:39.987+0000 7f7697200700 1 --2- 172.21.15.151:0/3353607827 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76981016c0 0x7f7698101aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:39.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:39.987+0000 7f7697200700 1 --2- 172.21.15.151:0/3353607827 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7698102070 0x7f769810ed30 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:39.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:39.987+0000 7f7697200700 1 -- 172.21.15.151:0/3353607827 >> 172.21.15.151:0/3353607827 conn(0x7f76980fd360 msgr2=0x7f76980ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:39.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:39.988+0000 7f7697200700 1 -- 172.21.15.151:0/3353607827 shutdown_connections 2024-04-26T01:54:39.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:39.989+0000 7f7697200700 1 -- 172.21.15.151:0/3353607827 wait complete. 2024-04-26T01:54:39.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:39.990+0000 7f7697200700 1 Processor -- start 2024-04-26T01:54:39.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:39.991+0000 7f7697200700 1 -- start start 2024-04-26T01:54:39.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:39.991+0000 7f7697200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76981016c0 0x7f769819b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:39.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:39.991+0000 7f7697200700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7698102070 0x7f769819b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:39.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:39.991+0000 7f7697200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f769810f270 0x7f769819fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:39.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:39.991+0000 7f7697200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7698114d80 con 0x7f76981016c0 2024-04-26T01:54:39.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:39.991+0000 7f7697200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7698114c00 con 0x7f769810f270 2024-04-26T01:54:39.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:39.991+0000 7f7697200700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7698114f00 con 0x7f7698102070 2024-04-26T01:54:39.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:39.992+0000 7f7696800700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f769810f270 0x7f769819fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:39.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:39.992+0000 7f7696800700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f769810f270 0x7f769819fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:45368/0 (socket says 172.21.15.151:45368) 2024-04-26T01:54:39.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:39.992+0000 7f7696800700 1 -- 172.21.15.151:0/1317908488 learned_addr learned my addr 172.21.15.151:0/1317908488 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:54:39.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:39.992+0000 7f7695e00700 1 --2- 172.21.15.151:0/1317908488 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76981016c0 0x7f769819b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:39.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:39.992+0000 7f7695400700 1 --2- 172.21.15.151:0/1317908488 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7698102070 0x7f769819b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:39.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:39.992+0000 7f7695e00700 1 -- 172.21.15.151:0/1317908488 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7698102070 msgr2=0x7f769819b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:39.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:39.992+0000 7f7695e00700 1 --2- 172.21.15.151:0/1317908488 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7698102070 0x7f769819b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:39.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:39.992+0000 7f7695e00700 1 -- 172.21.15.151:0/1317908488 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f769810f270 msgr2=0x7f769819fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:39.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:39.992+0000 7f7695e00700 1 --2- 172.21.15.151:0/1317908488 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f769810f270 0x7f769819fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:39.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:39.992+0000 7f7695e00700 1 -- 172.21.15.151:0/1317908488 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76981a0300 con 0x7f76981016c0 2024-04-26T01:54:39.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:39.993+0000 7f7695e00700 1 --2- 172.21.15.151:0/1317908488 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76981016c0 0x7f769819b2f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f768800e9f0 tx=0x7f768800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:39.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:39.993+0000 7f768ea00700 1 -- 172.21.15.151:0/1317908488 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7688004220 con 0x7f76981016c0 2024-04-26T01:54:39.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:39.994+0000 7f7697200700 1 -- 172.21.15.151:0/1317908488 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76981a05f0 con 0x7f76981016c0 2024-04-26T01:54:39.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:39.994+0000 7f768ea00700 1 -- 172.21.15.151:0/1317908488 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76880043c0 con 0x7f76981016c0 2024-04-26T01:54:39.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:39.995+0000 7f768ea00700 1 -- 172.21.15.151:0/1317908488 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7688005020 con 0x7f76981016c0 2024-04-26T01:54:39.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:39.996+0000 7f7697200700 1 -- 172.21.15.151:0/1317908488 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76981a7e70 con 0x7f76981016c0 2024-04-26T01:54:39.996 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:39.997+0000 7f7697200700 1 -- 172.21.15.151:0/1317908488 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7698069730 con 0x7f76981016c0 2024-04-26T01:54:39.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:39.998+0000 7f768ea00700 1 -- 172.21.15.151:0/1317908488 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 138931+0+0 (secure 0 0 0) 0x7f76880051c0 con 0x7f76981016c0 2024-04-26T01:54:40.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:40.002+0000 7f768ea00700 1 --2- 172.21.15.151:0/1317908488 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f767c0a3fe0 0x7f767c0a64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:40.001 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:40.002+0000 7f768ea00700 1 -- 172.21.15.151:0/1317908488 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f76880d4060 con 0x7f76981016c0 2024-04-26T01:54:40.001 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:40.002+0000 7f7695400700 1 --2- 172.21.15.151:0/1317908488 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f767c0a3fe0 0x7f767c0a64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:40.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:40.004+0000 7f7695400700 1 --2- 172.21.15.151:0/1317908488 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f767c0a3fe0 0x7f767c0a64a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7684009fd0 tx=0x7f7684019040 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:40.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:40.008+0000 7f768ea00700 1 -- 172.21.15.151:0/1317908488 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f768800b2c0 con 0x7f76981016c0 2024-04-26T01:54:40.124 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:54:39 smithi151 ceph-mon[28057]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T01:54:40.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:40.280+0000 7f7697200700 1 -- 172.21.15.151:0/1317908488 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f769802df80 con 0x7f76981016c0 2024-04-26T01:54:40.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:40.282+0000 7f768ea00700 1 -- 172.21.15.151:0/1317908488 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11889 (secure 0 0 0) 0x7f768809c4a0 con 0x7f76981016c0 2024-04-26T01:54:40.281 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:54:40.281 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2937864097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2937864097},{"type":"v1","addr":"172.21.15.151:6835","nonce":2937864097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14373,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/2947090802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2947090802},{"type":"v1","addr":"172.21.15.151:6837","nonce":2947090802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/1356563280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1356563280},{"type":"v1","addr":"172.21.15.151:6839","nonce":1356563280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6841/1075240552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1075240552},{"type":"v1","addr":"172.21.15.151:6841","nonce":1075240552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/504675912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":504675912},{"type":"v1","addr":"172.21.15.165:6833","nonce":504675912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1207580015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1207580015},{"type":"v1","addr":"172.21.15.172:6833","nonce":1207580015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:49:40.746876+0000","modified":"2024-04-26T01:53:36.483847+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24505,"mds_2":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/3598138294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3598138294},{"type":"v1","addr":"172.21.15.165:6835","nonce":3598138294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/1847859054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":1847859054},{"type":"v1","addr":"172.21.15.165:6837","nonce":1847859054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/2309751171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2309751171},{"type":"v1","addr":"172.21.15.172:6835","nonce":2309751171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.172:6837/55972537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":55972537},{"type":"v1","addr":"172.21.15.172:6837","nonce":55972537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.172:6839/1242302810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1242302810},{"type":"v1","addr":"172.21.15.172:6839","nonce":1242302810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/1969716254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1969716254},{"type":"v1","addr":"172.21.15.165:6839","nonce":1969716254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24493,24505,24509]},"id":1}]} 2024-04-26T01:54:40.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:40.284+0000 7f7697200700 1 -- 172.21.15.151:0/1317908488 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f767c0a3fe0 msgr2=0x7f767c0a64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:40.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:40.284+0000 7f7697200700 1 --2- 172.21.15.151:0/1317908488 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f767c0a3fe0 0x7f767c0a64a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7684009fd0 tx=0x7f7684019040 comp rx=0 tx=0).stop 2024-04-26T01:54:40.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:40.284+0000 7f7697200700 1 -- 172.21.15.151:0/1317908488 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76981016c0 msgr2=0x7f769819b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:40.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:40.284+0000 7f7697200700 1 --2- 172.21.15.151:0/1317908488 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76981016c0 0x7f769819b2f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f768800e9f0 tx=0x7f768800ed00 comp rx=0 tx=0).stop 2024-04-26T01:54:40.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:40.284+0000 7f7697200700 1 -- 172.21.15.151:0/1317908488 shutdown_connections 2024-04-26T01:54:40.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:40.284+0000 7f7697200700 1 --2- 172.21.15.151:0/1317908488 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f767c0a3fe0 0x7f767c0a64a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:40.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:40.284+0000 7f7697200700 1 --2- 172.21.15.151:0/1317908488 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f769810f270 0x7f769819fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:40.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:40.284+0000 7f7697200700 1 --2- 172.21.15.151:0/1317908488 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76981016c0 0x7f769819b2f0 secure :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f768800e9f0 tx=0x7f768800ed00 comp rx=0 tx=0).stop 2024-04-26T01:54:40.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:40.284+0000 7f7697200700 1 --2- 172.21.15.151:0/1317908488 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7698102070 0x7f769819b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:40.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:40.284+0000 7f7697200700 1 -- 172.21.15.151:0/1317908488 >> 172.21.15.151:0/1317908488 conn(0x7f76980fd360 msgr2=0x7f76980ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:40.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:40.285+0000 7f7697200700 1 -- 172.21.15.151:0/1317908488 shutdown_connections 2024-04-26T01:54:40.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:40.286+0000 7f7697200700 1 -- 172.21.15.151:0/1317908488 wait complete. 2024-04-26T01:54:40.285 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 26 2024-04-26T01:54:40.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:54:39 smithi172 ceph-mon[32785]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T01:54:40.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:54:39 smithi165 ceph-mon[32648]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T01:54:40.907 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:54:40 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/1317908488' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:54:41.072 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24509 quiesce db / --timeout 60 --expiration 85.6 --await-for 42.0' 2024-04-26T01:54:41.073 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24509 quiesce db / --timeout 60 --expiration 85.6 --await-for 42.0 2024-04-26T01:54:41.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:54:40 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/1317908488' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:54:41.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:54:40 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/1317908488' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:54:41.463 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:54:42.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:54:41 smithi172 ceph-mon[32785]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T01:54:42.304 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:54:41 smithi151 ceph-mon[28057]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T01:54:42.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:54:41 smithi165 ceph-mon[32648]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T01:54:43.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.694+0000 7f2ae2e00700 1 -- 172.21.15.151:0/1043718630 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2adc1034b0 msgr2=0x7f2adc10f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:43.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.695+0000 7f2ae2e00700 1 --2- 172.21.15.151:0/1043718630 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2adc1034b0 0x7f2adc10f8e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4009fc0 tx=0x7f2ad400b330 comp rx=0 tx=0).stop 2024-04-26T01:54:43.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.695+0000 7f2ae2e00700 1 -- 172.21.15.151:0/1043718630 shutdown_connections 2024-04-26T01:54:43.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.695+0000 7f2ae2e00700 1 --2- 172.21.15.151:0/1043718630 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2adc1034b0 0x7f2adc10f8e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:43.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.695+0000 7f2ae2e00700 1 --2- 172.21.15.151:0/1043718630 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2adc108a30 0x7f2adc108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:43.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.695+0000 7f2ae2e00700 1 --2- 172.21.15.151:0/1043718630 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2adc102af0 0x7f2adc102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:43.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.695+0000 7f2ae2e00700 1 -- 172.21.15.151:0/1043718630 >> 172.21.15.151:0/1043718630 conn(0x7f2adc079b10 msgr2=0x7f2adc079f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:43.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.695+0000 7f2ae2e00700 1 -- 172.21.15.151:0/1043718630 shutdown_connections 2024-04-26T01:54:43.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.696+0000 7f2ae2e00700 1 -- 172.21.15.151:0/1043718630 wait complete. 2024-04-26T01:54:43.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.697+0000 7f2ae2e00700 1 Processor -- start 2024-04-26T01:54:43.697 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.697+0000 7f2ae2e00700 1 -- start start 2024-04-26T01:54:43.697 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.698+0000 7f2ae2e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2adc102af0 0x7f2adc19b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:43.697 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.698+0000 7f2ae2e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2adc1034b0 0x7f2adc19b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:43.697 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.698+0000 7f2ae2e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2adc108a30 0x7f2adc19fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:43.697 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.698+0000 7f2ae2e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2adc112c00 con 0x7f2adc1034b0 2024-04-26T01:54:43.697 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.698+0000 7f2ae2e00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2adc112a80 con 0x7f2adc108a30 2024-04-26T01:54:43.697 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.698+0000 7f2ae2e00700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2adc112d80 con 0x7f2adc102af0 2024-04-26T01:54:43.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.699+0000 7f2adbe00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2adc102af0 0x7f2adc19b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:43.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.699+0000 7f2adbe00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2adc102af0 0x7f2adc19b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:46786/0 (socket says 172.21.15.151:46786) 2024-04-26T01:54:43.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.699+0000 7f2adbe00700 1 -- 172.21.15.151:0/1888152424 learned_addr learned my addr 172.21.15.151:0/1888152424 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:54:43.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.699+0000 7f2ae0e00700 1 --2- 172.21.15.151:0/1888152424 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2adc108a30 0x7f2adc19fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:43.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.699+0000 7f2ae0e00700 1 -- 172.21.15.151:0/1888152424 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2adc102af0 msgr2=0x7f2adc19b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:43.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.699+0000 7f2ae0e00700 1 --2- 172.21.15.151:0/1888152424 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2adc102af0 0x7f2adc19b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:43.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.699+0000 7f2ae0e00700 1 -- 172.21.15.151:0/1888152424 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2adc1034b0 msgr2=0x7f2adc19b700 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T01:54:43.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.699+0000 7f2ae0e00700 1 --2- 172.21.15.151:0/1888152424 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2adc1034b0 0x7f2adc19b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:43.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.699+0000 7f2ae0e00700 1 -- 172.21.15.151:0/1888152424 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2adc1a01d0 con 0x7f2adc108a30 2024-04-26T01:54:43.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.700+0000 7f2ae0e00700 1 --2- 172.21.15.151:0/1888152424 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2adc108a30 0x7f2adc19fa90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4015000 tx=0x7f2ad400ccb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:43.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.700+0000 7f2ad8c00700 1 -- 172.21.15.151:0/1888152424 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad4021070 con 0x7f2adc108a30 2024-04-26T01:54:43.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.701+0000 7f2ad8c00700 1 -- 172.21.15.151:0/1888152424 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ad4007ac0 con 0x7f2adc108a30 2024-04-26T01:54:43.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.701+0000 7f2ae2e00700 1 -- 172.21.15.151:0/1888152424 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2adc1a0460 con 0x7f2adc108a30 2024-04-26T01:54:43.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.701+0000 7f2ad8c00700 1 -- 172.21.15.151:0/1888152424 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad4005040 con 0x7f2adc108a30 2024-04-26T01:54:43.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.701+0000 7f2ae2e00700 1 -- 172.21.15.151:0/1888152424 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2adc1a7d40 con 0x7f2adc108a30 2024-04-26T01:54:43.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.706+0000 7f2ad8c00700 1 -- 172.21.15.151:0/1888152424 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 26) ==== 138931+0+0 (secure 0 0 0) 0x7f2ad4024070 con 0x7f2adc108a30 2024-04-26T01:54:43.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.708+0000 7f2aed6fbb80 1 Processor -- start 2024-04-26T01:54:43.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.708+0000 7f2aed6fbb80 1 -- start start 2024-04-26T01:54:43.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.709+0000 7f2aed6fbb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559c4883dd0 0x5559c48841b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:43.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.709+0000 7f2ad8c00700 1 --2- 172.21.15.151:0/1888152424 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f2ac00a4080 0x7f2ac00a6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:43.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.709+0000 7f2aed6fbb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5559c48846f0 0x5559c488f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:43.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.709+0000 7f2aed6fbb80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559c488f560 0x5559c4891950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:43.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.709+0000 7f2aed6fbb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5559c47d7020 con 0x5559c48846f0 2024-04-26T01:54:43.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.709+0000 7f2aed6fbb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5559c477d8c0 con 0x5559c4883dd0 2024-04-26T01:54:43.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.709+0000 7f2aed6fbb80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5559c47d2fe0 con 0x5559c488f560 2024-04-26T01:54:43.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.709+0000 7f2ad8c00700 1 -- 172.21.15.151:0/1888152424 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f2ad400f020 con 0x7f2adc108a30 2024-04-26T01:54:43.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.709+0000 7f2adb400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559c4883dd0 0x5559c48841b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:43.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.709+0000 7f2ae0e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559c488f560 0x5559c4891950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:43.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.709+0000 7f2ae0e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559c488f560 0x5559c4891950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:46802/0 (socket says 172.21.15.151:46802) 2024-04-26T01:54:43.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.709+0000 7f2ae0e00700 1 -- 172.21.15.151:0/1624459091 learned_addr learned my addr 172.21.15.151:0/1624459091 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:54:43.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.710+0000 7f2adb400700 1 --2- 172.21.15.151:0/1624459091 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5559c48846f0 0x5559c488f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:43.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.711+0000 7f2ae0e00700 1 -- 172.21.15.151:0/1624459091 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559c4883dd0 msgr2=0x5559c48841b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:43.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.711+0000 7f2ae0e00700 1 --2- 172.21.15.151:0/1624459091 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559c4883dd0 0x5559c48841b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:43.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.711+0000 7f2ae0e00700 1 -- 172.21.15.151:0/1624459091 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5559c48846f0 msgr2=0x5559c488f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:43.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.711+0000 7f2ae0e00700 1 --2- 172.21.15.151:0/1624459091 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5559c48846f0 0x5559c488f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:43.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.711+0000 7f2ae0e00700 1 -- 172.21.15.151:0/1624459091 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559c4773810 con 0x5559c488f560 2024-04-26T01:54:43.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.711+0000 7f2adb400700 1 --2- 172.21.15.151:0/1624459091 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559c4883dd0 0x5559c48841b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:54:43.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.711+0000 7f2adbe00700 1 --2- 172.21.15.151:0/1888152424 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f2ac00a4080 0x7f2ac00a6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:43.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.711+0000 7f2ae0e00700 1 --2- 172.21.15.151:0/1624459091 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559c488f560 0x5559c4891950 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4004fd0 tx=0x7f2ad401da70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce406dc696d201d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:43.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.712+0000 7f2ad1600700 1 -- 172.21.15.151:0/1624459091 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad409e270 con 0x5559c488f560 2024-04-26T01:54:43.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.712+0000 7f2ad1600700 1 -- 172.21.15.151:0/1624459091 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ad409db70 con 0x5559c488f560 2024-04-26T01:54:43.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.712+0000 7f2ad1600700 1 -- 172.21.15.151:0/1624459091 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad409ddd0 con 0x5559c488f560 2024-04-26T01:54:43.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.712+0000 7f2adbe00700 1 --2- 172.21.15.151:0/1888152424 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f2ac00a4080 0x7f2ac00a6540 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2acc005950 tx=0x7f2acc009310 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:43.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.713+0000 7f2aed6fbb80 1 -- 172.21.15.151:0/1624459091 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559c488f560 msgr2=0x5559c4891950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:43.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.713+0000 7f2aed6fbb80 1 --2- 172.21.15.151:0/1624459091 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559c488f560 0x5559c4891950 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4004fd0 tx=0x7f2ad401da70 comp rx=0 tx=0).stop 2024-04-26T01:54:43.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.714+0000 7f2aed6fbb80 1 -- 172.21.15.151:0/1624459091 shutdown_connections 2024-04-26T01:54:43.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.714+0000 7f2aed6fbb80 1 --2- 172.21.15.151:0/1624459091 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559c488f560 0x5559c4891950 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:43.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.714+0000 7f2aed6fbb80 1 --2- 172.21.15.151:0/1624459091 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559c4883dd0 0x5559c48841b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:43.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.714+0000 7f2aed6fbb80 1 --2- 172.21.15.151:0/1624459091 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5559c48846f0 0x5559c488f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:43.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.714+0000 7f2aed6fbb80 1 -- 172.21.15.151:0/1624459091 >> 172.21.15.151:0/1624459091 conn(0x5559c47b30e0 msgr2=0x5559c48802e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:43.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.714+0000 7f2aed6fbb80 1 -- 172.21.15.151:0/1624459091 shutdown_connections 2024-04-26T01:54:43.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.714+0000 7f2aed6fbb80 1 -- 172.21.15.151:0/1624459091 wait complete. 2024-04-26T01:54:43.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.715+0000 7f2aed6fbb80 1 -- start start 2024-04-26T01:54:43.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.715+0000 7f2aed6fbb80 1 Processor -- start 2024-04-26T01:54:43.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.716+0000 7f2aed6fbb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559c4883dd0 0x5559c488eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:43.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.716+0000 7f2aed6fbb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5559c48846f0 0x5559c488bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:43.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.716+0000 7f2aed6fbb80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559c488f560 0x5559c488c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:43.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.716+0000 7f2aed6fbb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5559c4891e90 con 0x5559c48846f0 2024-04-26T01:54:43.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.716+0000 7f2aed6fbb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5559c47909f0 con 0x5559c4883dd0 2024-04-26T01:54:43.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.716+0000 7f2aed6fbb80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5559c4892010 con 0x5559c488f560 2024-04-26T01:54:43.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.716+0000 7f2aed6fbb80 10 client.0 _ll_register_callbacks cb 0x5559c476f8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:54:43.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.717+0000 7f2adbe00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559c4883dd0 0x5559c488eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:43.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.717+0000 7f2adbe00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559c4883dd0 0x5559c488eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:45412/0 (socket says 172.21.15.151:45412) 2024-04-26T01:54:43.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.717+0000 7f2adbe00700 1 -- 172.21.15.151:0/2345267925 learned_addr learned my addr 172.21.15.151:0/2345267925 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:54:43.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.717+0000 7f2ae0e00700 1 --2- 172.21.15.151:0/2345267925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559c488f560 0x5559c488c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:43.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.717+0000 7f2adbe00700 1 -- 172.21.15.151:0/2345267925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559c488f560 msgr2=0x5559c488c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:43.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.717+0000 7f2adbe00700 1 --2- 172.21.15.151:0/2345267925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559c488f560 0x5559c488c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:43.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.717+0000 7f2adbe00700 1 -- 172.21.15.151:0/2345267925 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5559c48846f0 msgr2=0x5559c488bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:43.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.717+0000 7f2adbe00700 1 --2- 172.21.15.151:0/2345267925 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5559c48846f0 0x5559c488bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:43.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.717+0000 7f2adbe00700 1 -- 172.21.15.151:0/2345267925 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559c4724a60 con 0x5559c4883dd0 2024-04-26T01:54:43.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.717+0000 7f2ae0e00700 1 --2- 172.21.15.151:0/2345267925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559c488f560 0x5559c488c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T01:54:43.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.717+0000 7f2adbe00700 1 --2- 172.21.15.151:0/2345267925 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559c4883dd0 0x5559c488eca0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2acc003900 tx=0x7f2acc0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83c52da77d5e3ba0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:43.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.718+0000 7f2ac6000700 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T01:54:43.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.718+0000 7f2ac6000700 1 -- 172.21.15.151:0/2345267925 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2acc00e9c0 con 0x5559c4883dd0 2024-04-26T01:54:43.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.718+0000 7f2ac6000700 1 -- 172.21.15.151:0/2345267925 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f2acc00eb60 con 0x5559c4883dd0 2024-04-26T01:54:43.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.718+0000 7f2aed6fbb80 1 -- 172.21.15.151:0/2345267925 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ad409db70 con 0x5559c4883dd0 2024-04-26T01:54:43.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.718+0000 7f2ac6000700 1 -- 172.21.15.151:0/2345267925 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2acc00ed60 con 0x5559c4883dd0 2024-04-26T01:54:43.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.719+0000 7f2aed6fbb80 10 client.25310 fetch_fsmap learned FSMap version 26 2024-04-26T01:54:43.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.719+0000 7f2aed6fbb80 1 -- 172.21.15.151:0/2345267925 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5559c476dc80 con 0x5559c4883dd0 2024-04-26T01:54:43.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.719+0000 7f2ac6000700 1 -- 172.21.15.151:0/2345267925 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ad409db70 con 0x5559c4883dd0 2024-04-26T01:54:43.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.719+0000 7f2ac6000700 1 -- 172.21.15.151:0/2345267925 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f2acc02e020 con 0x5559c4883dd0 2024-04-26T01:54:43.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.720+0000 7f2aed6fbb80 10 client.25310 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T01:54:43.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.720+0000 7f2aed6fbb80 10 client.25310 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-26T01:54:43.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.720+0000 7f2aed6fbb80 20 client.25310 populate_metadata read hostname 'smithi151' 2024-04-26T01:54:43.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.720+0000 7f2aed6fbb80 1 --2- 172.21.15.151:0/2345267925 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5559c489ea10 0x5559c48a0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:43.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.720+0000 7f2aed6fbb80 4 client.25310 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:54:43.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.720+0000 7f2aed6fbb80 1 -- 172.21.15.151:0/2345267925 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5559c47e4810 con 0x5559c489ea10 2024-04-26T01:54:43.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.721+0000 7f2adb400700 1 --2- 172.21.15.151:0/2345267925 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5559c489ea10 0x5559c48a0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:43.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.722+0000 7f2adb400700 1 --2- 172.21.15.151:0/2345267925 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5559c489ea10 0x5559c48a0e40 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83cf8786cc595320 server_cookie=474bef52f76bf6b9 in_seq=0 out_seq=0 2024-04-26T01:54:43.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.722+0000 7f2ac6000700 10 client.25310 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:54:43.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.723+0000 7f2ac6000700 1 -- 172.21.15.151:0/2345267925 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5559c47e4810 con 0x5559c489ea10 2024-04-26T01:54:43.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.723+0000 7f2ac6000700 10 client.25310 handle_command_reply: tid=0 2024-04-26T01:54:43.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.723+0000 7f2aed6fbb80 1 client.25310 shutdown 2024-04-26T01:54:43.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.723+0000 7f2aed6fbb80 1 -- 172.21.15.151:0/2345267925 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559c4883dd0 msgr2=0x5559c488eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:43.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.723+0000 7f2aed6fbb80 1 --2- 172.21.15.151:0/2345267925 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559c4883dd0 0x5559c488eca0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2acc003900 tx=0x7f2acc0047c0 comp rx=0 tx=0).stop 2024-04-26T01:54:43.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.724+0000 7f2aed6fbb80 1 -- 172.21.15.151:0/2345267925 shutdown_connections 2024-04-26T01:54:43.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.724+0000 7f2aed6fbb80 1 --2- 172.21.15.151:0/2345267925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559c488f560 0x5559c488c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:43.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.724+0000 7f2aed6fbb80 1 --2- 172.21.15.151:0/2345267925 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5559c489ea10 0x5559c48a0e40 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:43.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.724+0000 7f2ac6000700 0 client.25310 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:54:43.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.724+0000 7f2aed6fbb80 1 --2- 172.21.15.151:0/2345267925 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559c4883dd0 0x5559c488eca0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:43.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.724+0000 7f2aed6fbb80 1 --2- 172.21.15.151:0/2345267925 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5559c48846f0 0x5559c488bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:43.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.724+0000 7f2aed6fbb80 1 -- 172.21.15.151:0/2345267925 >> 172.21.15.151:0/2345267925 conn(0x5559c47b30e0 msgr2=0x5559c487fed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:43.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.725+0000 7f2aed6fbb80 1 -- 172.21.15.151:0/2345267925 shutdown_connections 2024-04-26T01:54:43.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.725+0000 7f2aed6fbb80 1 -- 172.21.15.151:0/2345267925 wait complete. 2024-04-26T01:54:43.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.725+0000 7f2aed6fbb80 20 client.25310 trim_cache size 0 max 16384 2024-04-26T01:54:43.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.765+0000 7f2aed6fbb80 1 Processor -- start 2024-04-26T01:54:43.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.765+0000 7f2aed6fbb80 1 -- start start 2024-04-26T01:54:43.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.766+0000 7f2aed6fbb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559c4883dd0 0x5559c489acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:43.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.766+0000 7f2aed6fbb80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559c48846f0 0x5559c48a6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:43.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.766+0000 7f2aed6fbb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5559c488f560 0x5559c48a6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:43.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.766+0000 7f2aed6fbb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2adc1a92a0 con 0x5559c488f560 2024-04-26T01:54:43.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.766+0000 7f2aed6fbb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2adc073830 con 0x5559c4883dd0 2024-04-26T01:54:43.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.766+0000 7f2aed6fbb80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2adc071700 con 0x5559c48846f0 2024-04-26T01:54:43.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.766+0000 7f2adb400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559c48846f0 0x5559c48a6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:43.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.766+0000 7f2adb400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559c48846f0 0x5559c48a6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:46822/0 (socket says 172.21.15.151:46822) 2024-04-26T01:54:43.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.766+0000 7f2adb400700 1 -- 172.21.15.151:0/3370840863 learned_addr learned my addr 172.21.15.151:0/3370840863 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:54:43.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.766+0000 7f2ae0e00700 1 --2- 172.21.15.151:0/3370840863 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5559c488f560 0x5559c48a6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:43.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.766+0000 7f2adbe00700 1 --2- 172.21.15.151:0/3370840863 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559c4883dd0 0x5559c489acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:43.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.766+0000 7f2adb400700 1 -- 172.21.15.151:0/3370840863 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559c4883dd0 msgr2=0x5559c489acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:43.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.767+0000 7f2adb400700 1 --2- 172.21.15.151:0/3370840863 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559c4883dd0 0x5559c489acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:43.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.767+0000 7f2adb400700 1 -- 172.21.15.151:0/3370840863 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5559c488f560 msgr2=0x5559c48a6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:43.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.767+0000 7f2adb400700 1 --2- 172.21.15.151:0/3370840863 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5559c488f560 0x5559c48a6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:43.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.767+0000 7f2adb400700 1 -- 172.21.15.151:0/3370840863 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559c488a510 con 0x5559c48846f0 2024-04-26T01:54:43.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.767+0000 7f2adbe00700 1 --2- 172.21.15.151:0/3370840863 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559c4883dd0 0x5559c489acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T01:54:43.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.767+0000 7f2ae0e00700 1 --2- 172.21.15.151:0/3370840863 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5559c488f560 0x5559c48a6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:54:43.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.767+0000 7f2adb400700 1 --2- 172.21.15.151:0/3370840863 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559c48846f0 0x5559c48a6710 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x5559c488d570 tx=0x7f2ac8009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5a5dc75a1c5461a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:43.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.768+0000 7f2ad0c00700 1 -- 172.21.15.151:0/3370840863 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac8004050 con 0x5559c48846f0 2024-04-26T01:54:43.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.768+0000 7f2ad0c00700 1 -- 172.21.15.151:0/3370840863 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ac8024910 con 0x5559c48846f0 2024-04-26T01:54:43.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.768+0000 7f2ad0c00700 1 -- 172.21.15.151:0/3370840863 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac802b750 con 0x5559c48846f0 2024-04-26T01:54:43.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.769+0000 7f2aed6fbb80 1 -- 172.21.15.151:0/3370840863 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559c48846f0 msgr2=0x5559c48a6710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:43.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.769+0000 7f2aed6fbb80 1 --2- 172.21.15.151:0/3370840863 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559c48846f0 0x5559c48a6710 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x5559c488d570 tx=0x7f2ac8009260 comp rx=0 tx=0).stop 2024-04-26T01:54:43.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.769+0000 7f2aed6fbb80 1 -- 172.21.15.151:0/3370840863 shutdown_connections 2024-04-26T01:54:43.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.769+0000 7f2aed6fbb80 1 --2- 172.21.15.151:0/3370840863 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5559c488f560 0x5559c48a6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:43.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.769+0000 7f2aed6fbb80 1 --2- 172.21.15.151:0/3370840863 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559c4883dd0 0x5559c489acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:43.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.769+0000 7f2aed6fbb80 1 --2- 172.21.15.151:0/3370840863 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559c48846f0 0x5559c48a6710 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:43.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.769+0000 7f2aed6fbb80 1 -- 172.21.15.151:0/3370840863 >> 172.21.15.151:0/3370840863 conn(0x5559c47b30e0 msgr2=0x5559c4886700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:43.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.769+0000 7f2aed6fbb80 1 -- 172.21.15.151:0/3370840863 shutdown_connections 2024-04-26T01:54:43.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.769+0000 7f2aed6fbb80 1 -- 172.21.15.151:0/3370840863 wait complete. 2024-04-26T01:54:43.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.770+0000 7f2aed6fbb80 1 -- start start 2024-04-26T01:54:43.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.770+0000 7f2aed6fbb80 1 Processor -- start 2024-04-26T01:54:43.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.771+0000 7f2aed6fbb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559c4883dd0 0x5559c4894380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:43.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.771+0000 7f2aed6fbb80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559c48846f0 0x5559c48948c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:43.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.771+0000 7f2aed6fbb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5559c488f560 0x5559c4904f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:43.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.771+0000 7f2aed6fbb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2adc071d50 con 0x5559c488f560 2024-04-26T01:54:43.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.771+0000 7f2aed6fbb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2adc073160 con 0x5559c4883dd0 2024-04-26T01:54:43.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.771+0000 7f2aed6fbb80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2adc0723c0 con 0x5559c48846f0 2024-04-26T01:54:43.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.771+0000 7f2aed6fbb80 10 client.0 _ll_register_callbacks cb 0x5559c476f8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:54:43.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.772+0000 7f2adbe00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559c4883dd0 0x5559c4894380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:43.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.772+0000 7f2adb400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559c48846f0 0x5559c48948c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:43.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.772+0000 7f2ae0e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5559c488f560 0x5559c4904f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:43.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.772+0000 7f2adb400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559c48846f0 0x5559c48948c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:46826/0 (socket says 172.21.15.151:46826) 2024-04-26T01:54:43.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.772+0000 7f2adb400700 1 -- 172.21.15.151:0/570401085 learned_addr learned my addr 172.21.15.151:0/570401085 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:54:43.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.772+0000 7f2adbe00700 1 -- 172.21.15.151:0/570401085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559c48846f0 msgr2=0x5559c48948c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:43.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.772+0000 7f2adbe00700 1 --2- 172.21.15.151:0/570401085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559c48846f0 0x5559c48948c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:43.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.773+0000 7f2adbe00700 1 -- 172.21.15.151:0/570401085 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5559c488f560 msgr2=0x5559c4904f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:43.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.773+0000 7f2adbe00700 1 --2- 172.21.15.151:0/570401085 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5559c488f560 0x5559c4904f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:43.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.773+0000 7f2adbe00700 1 -- 172.21.15.151:0/570401085 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559c4905550 con 0x5559c4883dd0 2024-04-26T01:54:43.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.773+0000 7f2ae0e00700 1 --2- 172.21.15.151:0/570401085 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5559c488f560 0x5559c4904f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:54:43.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.773+0000 7f2adbe00700 1 --2- 172.21.15.151:0/570401085 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559c4883dd0 0x5559c4894380 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x5559c488a4a0 tx=0x7f2acc011b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7dcb7c09fd4cc22b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:43.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.774+0000 7f2ac6000700 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T01:54:43.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.774+0000 7f2ac6000700 1 -- 172.21.15.151:0/570401085 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2acc01ed10 con 0x5559c4883dd0 2024-04-26T01:54:43.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.774+0000 7f2ac6000700 1 -- 172.21.15.151:0/570401085 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f2acc00e690 con 0x5559c4883dd0 2024-04-26T01:54:43.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.774+0000 7f2aed6fbb80 1 -- 172.21.15.151:0/570401085 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ac8024910 con 0x5559c4883dd0 2024-04-26T01:54:43.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.774+0000 7f2ac6000700 1 -- 172.21.15.151:0/570401085 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2acc00e890 con 0x5559c4883dd0 2024-04-26T01:54:43.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.774+0000 7f2aed6fbb80 10 client.25313 fetch_fsmap learned FSMap version 26 2024-04-26T01:54:43.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.774+0000 7f2aed6fbb80 1 -- 172.21.15.151:0/570401085 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5559c4906810 con 0x5559c4883dd0 2024-04-26T01:54:43.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.775+0000 7f2ac6000700 1 -- 172.21.15.151:0/570401085 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2acc035070 con 0x5559c4883dd0 2024-04-26T01:54:43.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.775+0000 7f2ac6000700 1 -- 172.21.15.151:0/570401085 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f2acc02e050 con 0x5559c4883dd0 2024-04-26T01:54:43.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.775+0000 7f2aed6fbb80 10 client.25313 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T01:54:43.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.775+0000 7f2aed6fbb80 10 client.25313 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-26T01:54:43.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.775+0000 7f2aed6fbb80 20 client.25313 populate_metadata read hostname 'smithi151' 2024-04-26T01:54:43.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.775+0000 7f2aed6fbb80 1 --2- 172.21.15.151:0/570401085 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5559c4907010 0x5559c4909430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:43.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.775+0000 7f2aed6fbb80 4 client.25313 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 85.6, "await_for": 42.0}] 2024-04-26T01:54:43.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.775+0000 7f2aed6fbb80 1 -- 172.21.15.151:0/570401085 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 85.6, "await_for": 42.0}) -- 0x7f2ac8024910 con 0x5559c4907010 2024-04-26T01:54:43.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.776+0000 7f2adb400700 1 --2- 172.21.15.151:0/570401085 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5559c4907010 0x5559c4909430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:43.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.776+0000 7f2adb400700 1 --2- 172.21.15.151:0/570401085 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5559c4907010 0x5559c4909430 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36baaad45760da5d server_cookie=85e6549391de4207 in_seq=0 out_seq=0 2024-04-26T01:54:43.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.776+0000 7f2ac6000700 10 client.25313 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:54:43.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.785+0000 7f2ac6000700 1 -- 172.21.15.151:0/570401085 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x7f2ac8024910 con 0x5559c4907010 2024-04-26T01:54:43.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.785+0000 7f2ac6000700 10 client.25313 handle_command_reply: tid=0 2024-04-26T01:54:43.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.785+0000 7f2aed6fbb80 1 client.25313 shutdown 2024-04-26T01:54:43.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.786+0000 7f2aed6fbb80 1 -- 172.21.15.151:0/570401085 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559c4883dd0 msgr2=0x5559c4894380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:43.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.786+0000 7f2aed6fbb80 1 --2- 172.21.15.151:0/570401085 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559c4883dd0 0x5559c4894380 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x5559c488a4a0 tx=0x7f2acc011b30 comp rx=0 tx=0).stop 2024-04-26T01:54:43.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.786+0000 7f2aed6fbb80 1 -- 172.21.15.151:0/570401085 shutdown_connections 2024-04-26T01:54:43.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.786+0000 7f2aed6fbb80 1 --2- 172.21.15.151:0/570401085 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5559c488f560 0x5559c4904f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:43.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.786+0000 7f2aed6fbb80 1 --2- 172.21.15.151:0/570401085 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5559c4907010 0x5559c4909430 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:43.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.786+0000 7f2aed6fbb80 1 --2- 172.21.15.151:0/570401085 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559c4883dd0 0x5559c4894380 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:43.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.786+0000 7f2aed6fbb80 1 --2- 172.21.15.151:0/570401085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559c48846f0 0x5559c48948c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:43.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.786+0000 7f2aed6fbb80 1 -- 172.21.15.151:0/570401085 >> 172.21.15.151:0/570401085 conn(0x5559c47b30e0 msgr2=0x5559c488d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:43.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.786+0000 7f2ac6000700 0 client.25313 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:54:43.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.786+0000 7f2aed6fbb80 1 -- 172.21.15.151:0/570401085 shutdown_connections 2024-04-26T01:54:43.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.786+0000 7f2aed6fbb80 1 -- 172.21.15.151:0/570401085 wait complete. 2024-04-26T01:54:43.786 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-26T01:54:43.786 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 26, 2024-04-26T01:54:43.786 INFO:teuthology.orchestra.run.smithi151.stdout: "leader": 24509, 2024-04-26T01:54:43.786 INFO:teuthology.orchestra.run.smithi151.stdout: "set_version": 2, 2024-04-26T01:54:43.786 INFO:teuthology.orchestra.run.smithi151.stdout: "sets": { 2024-04-26T01:54:43.786 INFO:teuthology.orchestra.run.smithi151.stdout: "7b96ec7c": { 2024-04-26T01:54:43.786 INFO:teuthology.orchestra.run.smithi151.stdout: "version": 2, 2024-04-26T01:54:43.787 INFO:teuthology.orchestra.run.smithi151.stdout: "age_ref": 0.0, 2024-04-26T01:54:43.787 INFO:teuthology.orchestra.run.smithi151.stdout: "state": { 2024-04-26T01:54:43.787 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "QUIESCED", 2024-04-26T01:54:43.787 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 0.0 2024-04-26T01:54:43.787 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T01:54:43.787 INFO:teuthology.orchestra.run.smithi151.stdout: "timeout": 60.0, 2024-04-26T01:54:43.787 INFO:teuthology.orchestra.run.smithi151.stdout: "expiration": 85.6, 2024-04-26T01:54:43.787 INFO:teuthology.orchestra.run.smithi151.stdout: "members": { 2024-04-26T01:54:43.787 INFO:teuthology.orchestra.run.smithi151.stdout: "file:/": { 2024-04-26T01:54:43.787 INFO:teuthology.orchestra.run.smithi151.stdout: "excluded": false, 2024-04-26T01:54:43.787 INFO:teuthology.orchestra.run.smithi151.stdout: "state": { 2024-04-26T01:54:43.787 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "QUIESCED", 2024-04-26T01:54:43.787 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 0.0 2024-04-26T01:54:43.787 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T01:54:43.787 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T01:54:43.788 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T01:54:43.788 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T01:54:43.788 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T01:54:43.788 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-26T01:54:43.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.787+0000 7f2aed6fbb80 20 client.25313 trim_cache size 0 max 16384 2024-04-26T01:54:43.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.787+0000 7f2ae2e00700 1 -- 172.21.15.151:0/1888152424 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f2ac00a4080 msgr2=0x7f2ac00a6540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:43.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.787+0000 7f2ae2e00700 1 --2- 172.21.15.151:0/1888152424 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f2ac00a4080 0x7f2ac00a6540 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2acc005950 tx=0x7f2acc009310 comp rx=0 tx=0).stop 2024-04-26T01:54:43.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.787+0000 7f2ae2e00700 1 -- 172.21.15.151:0/1888152424 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2adc108a30 msgr2=0x7f2adc19fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:43.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.787+0000 7f2ae2e00700 1 --2- 172.21.15.151:0/1888152424 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2adc108a30 0x7f2adc19fa90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4015000 tx=0x7f2ad400ccb0 comp rx=0 tx=0).stop 2024-04-26T01:54:43.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.789+0000 7f2ae2e00700 1 -- 172.21.15.151:0/1888152424 shutdown_connections 2024-04-26T01:54:43.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.789+0000 7f2ae2e00700 1 --2- 172.21.15.151:0/1888152424 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f2ac00a4080 0x7f2ac00a6540 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:43.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.789+0000 7f2ae2e00700 1 --2- 172.21.15.151:0/1888152424 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2adc108a30 0x7f2adc19fa90 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:43.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.789+0000 7f2ae2e00700 1 --2- 172.21.15.151:0/1888152424 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2adc102af0 0x7f2adc19b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:43.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.789+0000 7f2ae2e00700 1 --2- 172.21.15.151:0/1888152424 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2adc1034b0 0x7f2adc19b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:43.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.789+0000 7f2ae2e00700 1 -- 172.21.15.151:0/1888152424 >> 172.21.15.151:0/1888152424 conn(0x7f2adc079b10 msgr2=0x7f2adc10dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:43.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.789+0000 7f2ae2e00700 1 -- 172.21.15.151:0/1888152424 shutdown_connections 2024-04-26T01:54:43.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:54:43.789+0000 7f2ae2e00700 1 -- 172.21.15.151:0/1888152424 wait complete. 2024-04-26T01:54:44.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:54:43 smithi151 ceph-mon[28057]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-04-26T01:54:44.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:54:43 smithi172 ceph-mon[32785]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-04-26T01:54:44.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:54:43 smithi165 ceph-mon[32648]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-04-26T01:54:44.679 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '7b96ec7c', quiesce took 7.4 seconds. Will release after: 25.6 2024-04-26T01:54:45.975 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.976+0000 7f04e3200640 1 -- 172.21.15.151:0/1059356243 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04e4101860 msgr2=0x7f04e41058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:45.975 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.976+0000 7f04e3200640 1 --2- 172.21.15.151:0/1059356243 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04e4101860 0x7f04e41058a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f04d0009900 tx=0x7f04d002a3f0 comp rx=0 tx=0).stop 2024-04-26T01:54:45.975 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.976+0000 7f04e3200640 1 -- 172.21.15.151:0/1059356243 shutdown_connections 2024-04-26T01:54:45.975 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.976+0000 7f04e3200640 1 --2- 172.21.15.151:0/1059356243 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04e4105f80 0x7f04e410db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:45.975 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.976+0000 7f04e3200640 1 --2- 172.21.15.151:0/1059356243 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04e4101860 0x7f04e41058a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:45.976 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.976+0000 7f04e3200640 1 --2- 172.21.15.151:0/1059356243 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f04e4100eb0 0x7f04e4101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:45.976 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.976+0000 7f04e3200640 1 -- 172.21.15.151:0/1059356243 >> 172.21.15.151:0/1059356243 conn(0x7f04e40fc770 msgr2=0x7f04e40feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:45.976 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.977+0000 7f04e3200640 1 -- 172.21.15.151:0/1059356243 shutdown_connections 2024-04-26T01:54:45.976 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.977+0000 7f04e3200640 1 -- 172.21.15.151:0/1059356243 wait complete. 2024-04-26T01:54:45.976 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.978+0000 7f04e3200640 1 Processor -- start 2024-04-26T01:54:45.977 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.978+0000 7f04e3200640 1 -- start start 2024-04-26T01:54:45.977 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.978+0000 7f04e3200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f04e4100eb0 0x7f04e41a4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:45.977 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.978+0000 7f04e3200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04e4105f80 0x7f04e41a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:45.977 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.978+0000 7f04e3200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04e419ea20 0x7f04e419eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:45.977 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.978+0000 7f04e3200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f04e410fde0 con 0x7f04e4105f80 2024-04-26T01:54:45.977 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.978+0000 7f04e3200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f04e410fc60 con 0x7f04e419ea20 2024-04-26T01:54:45.977 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.978+0000 7f04e3200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f04e410ff60 con 0x7f04e4100eb0 2024-04-26T01:54:45.977 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.979+0000 7f04e2800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04e419ea20 0x7f04e419eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:45.978 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.979+0000 7f04e2800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04e419ea20 0x7f04e419eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:45444/0 (socket says 172.21.15.151:45444) 2024-04-26T01:54:45.978 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.979+0000 7f04e2800640 1 -- 172.21.15.151:0/1573805738 learned_addr learned my addr 172.21.15.151:0/1573805738 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:54:45.978 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.979+0000 7f04e1e00640 1 --2- 172.21.15.151:0/1573805738 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f04e4100eb0 0x7f04e41a4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:45.978 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.979+0000 7f04e1400640 1 --2- 172.21.15.151:0/1573805738 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04e4105f80 0x7f04e41a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:45.979 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.979+0000 7f04e2800640 1 -- 172.21.15.151:0/1573805738 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f04e4100eb0 msgr2=0x7f04e41a4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:45.979 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.979+0000 7f04e2800640 1 --2- 172.21.15.151:0/1573805738 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f04e4100eb0 0x7f04e41a4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:45.979 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.979+0000 7f04e2800640 1 -- 172.21.15.151:0/1573805738 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04e4105f80 msgr2=0x7f04e41a4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:45.979 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.979+0000 7f04e2800640 1 --2- 172.21.15.151:0/1573805738 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04e4105f80 0x7f04e41a4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:45.979 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.979+0000 7f04e2800640 1 -- 172.21.15.151:0/1573805738 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e419f540 con 0x7f04e419ea20 2024-04-26T01:54:45.979 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.979+0000 7f04e2800640 1 --2- 172.21.15.151:0/1573805738 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04e419ea20 0x7f04e419eed0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f04dc00eaa0 tx=0x7f04dc00ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:45.979 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.979+0000 7f04daa00640 1 -- 172.21.15.151:0/1573805738 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04dc00cdb0 con 0x7f04e419ea20 2024-04-26T01:54:45.979 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.980+0000 7f04daa00640 1 -- 172.21.15.151:0/1573805738 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04dc005020 con 0x7f04e419ea20 2024-04-26T01:54:45.979 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.980+0000 7f04daa00640 1 -- 172.21.15.151:0/1573805738 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04dc0116d0 con 0x7f04e419ea20 2024-04-26T01:54:45.979 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.980+0000 7f04e3200640 1 -- 172.21.15.151:0/1573805738 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04e419f830 con 0x7f04e419ea20 2024-04-26T01:54:45.979 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.980+0000 7f04e3200640 1 -- 172.21.15.151:0/1573805738 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04e41963e0 con 0x7f04e419ea20 2024-04-26T01:54:45.983 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.983+0000 7f04ebd72740 1 Processor -- start 2024-04-26T01:54:45.984 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.985+0000 7f04daa00640 1 -- 172.21.15.151:0/1573805738 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 26) ==== 138931+0+0 (secure 0 0 0) 0x7f04dc0051d0 con 0x7f04e419ea20 2024-04-26T01:54:45.984 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.986+0000 7f04ebd72740 1 -- start start 2024-04-26T01:54:45.985 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.986+0000 7f04ebd72740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56386fdc39b0 0x56386fdc3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:45.985 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.987+0000 7f04ebd72740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56386fdc42d0 0x56386fdce650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:45.985 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.987+0000 7f04daa00640 1 --2- 172.21.15.151:0/1573805738 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f04c80a3f30 0x7f04c80a63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:45.985 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.987+0000 7f04ebd72740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56386fdceb90 0x56386fdd0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:45.986 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.987+0000 7f04ebd72740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56386fd29900 con 0x56386fdc39b0 2024-04-26T01:54:45.986 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.987+0000 7f04ebd72740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56386fccfef0 con 0x56386fdc42d0 2024-04-26T01:54:45.986 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.987+0000 7f04ebd72740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56386fd258c0 con 0x56386fdceb90 2024-04-26T01:54:45.987 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.987+0000 7f04daa00640 1 -- 172.21.15.151:0/1573805738 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f04dc0d0b50 con 0x7f04e419ea20 2024-04-26T01:54:45.987 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.987+0000 7f04e1400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56386fdc39b0 0x56386fdc3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:45.987 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.988+0000 7f04e1400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56386fdc39b0 0x56386fdc3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49460/0 (socket says 172.21.15.151:49460) 2024-04-26T01:54:45.987 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.988+0000 7f04e1400640 1 -- 172.21.15.151:0/2488873868 learned_addr learned my addr 172.21.15.151:0/2488873868 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:54:45.987 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.988+0000 7f04e1400640 1 --2- 172.21.15.151:0/1573805738 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f04c80a3f30 0x7f04c80a63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:45.987 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.988+0000 7f04e1400640 1 -- 172.21.15.151:0/2488873868 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56386fdceb90 msgr2=0x56386fdd0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:45.987 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.988+0000 7f04e1400640 1 --2- 172.21.15.151:0/2488873868 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56386fdceb90 0x56386fdd0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:45.988 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.989+0000 7f04e1400640 1 -- 172.21.15.151:0/2488873868 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56386fdc42d0 msgr2=0x56386fdce650 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T01:54:45.988 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.989+0000 7f04e1400640 1 --2- 172.21.15.151:0/2488873868 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56386fdc42d0 0x56386fdce650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:45.988 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.989+0000 7f04e1400640 1 -- 172.21.15.151:0/2488873868 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56386fa3c210 con 0x56386fdc39b0 2024-04-26T01:54:45.989 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.989+0000 7f04e1400640 1 --2- 172.21.15.151:0/2488873868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56386fdc39b0 0x56386fdc3d90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f04d0031440 tx=0x7f04d0005140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbe63d7c22085003 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:45.989 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.990+0000 7f04e1400640 1 --2- 172.21.15.151:0/1573805738 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f04c80a3f30 0x7f04c80a63f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f04d0044000 tx=0x7f04d0046000 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:45.989 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.990+0000 7f04cf400640 1 -- 172.21.15.151:0/2488873868 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d004b020 con 0x56386fdc39b0 2024-04-26T01:54:45.989 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.990+0000 7f04cf400640 1 -- 172.21.15.151:0/2488873868 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04d0004c90 con 0x56386fdc39b0 2024-04-26T01:54:45.989 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.990+0000 7f04cf400640 1 -- 172.21.15.151:0/2488873868 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d0035040 con 0x56386fdc39b0 2024-04-26T01:54:45.990 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.991+0000 7f04ebd72740 1 -- 172.21.15.151:0/2488873868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56386fdc39b0 msgr2=0x56386fdc3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:45.990 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.991+0000 7f04ebd72740 1 --2- 172.21.15.151:0/2488873868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56386fdc39b0 0x56386fdc3d90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f04d0031440 tx=0x7f04d0005140 comp rx=0 tx=0).stop 2024-04-26T01:54:45.990 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.991+0000 7f04ebd72740 1 -- 172.21.15.151:0/2488873868 shutdown_connections 2024-04-26T01:54:45.991 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.991+0000 7f04ebd72740 1 --2- 172.21.15.151:0/2488873868 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56386fdceb90 0x56386fdd0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:45.991 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.991+0000 7f04ebd72740 1 --2- 172.21.15.151:0/2488873868 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56386fdc42d0 0x56386fdce650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:45.991 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.992+0000 7f04ebd72740 1 --2- 172.21.15.151:0/2488873868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56386fdc39b0 0x56386fdc3d90 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:45.991 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.992+0000 7f04ebd72740 1 -- 172.21.15.151:0/2488873868 >> 172.21.15.151:0/2488873868 conn(0x56386fdbfec0 msgr2=0x56386fdc02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:45.991 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.992+0000 7f04ebd72740 1 -- 172.21.15.151:0/2488873868 shutdown_connections 2024-04-26T01:54:45.991 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.992+0000 7f04ebd72740 1 -- 172.21.15.151:0/2488873868 wait complete. 2024-04-26T01:54:45.992 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.992+0000 7f04ebd72740 1 -- start start 2024-04-26T01:54:45.992 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.992+0000 7f04ebd72740 1 Processor -- start 2024-04-26T01:54:45.992 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.993+0000 7f04ebd72740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56386fdc39b0 0x56386fdcbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:45.992 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.993+0000 7f04ebd72740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56386fdc42d0 0x56386fdcc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:45.992 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.993+0000 7f04ebd72740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56386fdceb90 0x56386fdcc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:45.992 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.993+0000 7f04e1e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56386fdc39b0 0x56386fdcbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:45.993 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.993+0000 7f04ebd72740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56386fdd1760 con 0x56386fdc42d0 2024-04-26T01:54:45.993 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.993+0000 7f04e1400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56386fdc42d0 0x56386fdcc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:45.993 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.993+0000 7f04e1400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56386fdc42d0 0x56386fdcc0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49472/0 (socket says 172.21.15.151:49472) 2024-04-26T01:54:45.993 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.993+0000 7f04e1e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56386fdc39b0 0x56386fdcbbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:46852/0 (socket says 172.21.15.151:46852) 2024-04-26T01:54:45.993 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.993+0000 7f04e1400640 1 -- 172.21.15.151:0/3053137548 learned_addr learned my addr 172.21.15.151:0/3053137548 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:54:45.993 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.994+0000 7f04ebd72740 1 -- 172.21.15.151:0/3053137548 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56386fce3310 con 0x56386fdceb90 2024-04-26T01:54:45.993 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.994+0000 7f04ebd72740 1 -- 172.21.15.151:0/3053137548 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56386fdd18e0 con 0x56386fdc39b0 2024-04-26T01:54:45.993 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.994+0000 7f04ebd72740 10 client.0 _ll_register_callbacks cb 0x56386fcb1cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:54:45.993 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.994+0000 7f04e2800640 1 --2- 172.21.15.151:0/3053137548 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56386fdceb90 0x56386fdcc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:45.994 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.994+0000 7f04e1400640 1 -- 172.21.15.151:0/3053137548 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56386fdc39b0 msgr2=0x56386fdcbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:45.994 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.994+0000 7f04e1400640 1 --2- 172.21.15.151:0/3053137548 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56386fdc39b0 0x56386fdcbbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:45.994 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.994+0000 7f04e1400640 1 -- 172.21.15.151:0/3053137548 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56386fdceb90 msgr2=0x56386fdcc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:45.994 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.994+0000 7f04e1400640 1 --2- 172.21.15.151:0/3053137548 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56386fdceb90 0x56386fdcc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:45.994 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.994+0000 7f04e1400640 1 -- 172.21.15.151:0/3053137548 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56386fb08620 con 0x56386fdc42d0 2024-04-26T01:54:45.994 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.994+0000 7f04e2800640 1 --2- 172.21.15.151:0/3053137548 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56386fdceb90 0x56386fdcc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:54:45.994 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.995+0000 7f04e1400640 1 --2- 172.21.15.151:0/3053137548 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56386fdc42d0 0x56386fdcc0f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f04d0006010 tx=0x7f04d00044f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22e607e3cc88595b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:45.994 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.995+0000 7f04c3e00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-04-26T01:54:45.996 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.995+0000 7f04c3e00640 1 -- 172.21.15.151:0/3053137548 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d004b070 con 0x56386fdc42d0 2024-04-26T01:54:45.996 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.995+0000 7f04c3e00640 1 -- 172.21.15.151:0/3053137548 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f04d0004e40 con 0x56386fdc42d0 2024-04-26T01:54:45.996 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.995+0000 7f04ebd72740 1 -- 172.21.15.151:0/3053137548 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04d0004c90 con 0x56386fdc42d0 2024-04-26T01:54:45.996 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.996+0000 7f04c3e00640 1 -- 172.21.15.151:0/3053137548 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f04d005f050 con 0x56386fdc42d0 2024-04-26T01:54:45.996 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.996+0000 7f04ebd72740 10 client.15222 fetch_fsmap learned FSMap version 26 2024-04-26T01:54:45.996 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.996+0000 7f04ebd72740 1 -- 172.21.15.151:0/3053137548 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56386fddd840 con 0x56386fdc42d0 2024-04-26T01:54:45.996 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.996+0000 7f04c3e00640 1 -- 172.21.15.151:0/3053137548 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04d0004c90 con 0x56386fdc42d0 2024-04-26T01:54:45.996 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.997+0000 7f04c3e00640 1 -- 172.21.15.151:0/3053137548 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f04d0031950 con 0x56386fdc42d0 2024-04-26T01:54:45.996 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.997+0000 7f04ebd72740 10 client.15222 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T01:54:45.996 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.997+0000 7f04ebd72740 10 client.15222 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:54:45.997 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.997+0000 7f04ebd72740 20 client.15222 populate_metadata read hostname 'smithi151' 2024-04-26T01:54:45.997 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.997+0000 7f04ebd72740 1 --2- 172.21.15.151:0/3053137548 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x56386fddeb90 0x56386fde0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:45.997 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.997+0000 7f04ebd72740 4 client.15222 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:54:45.997 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.997+0000 7f04ebd72740 1 -- 172.21.15.151:0/3053137548 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56386fd3d8b0 con 0x56386fddeb90 2024-04-26T01:54:45.997 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.998+0000 7f04e1e00640 1 --2- 172.21.15.151:0/3053137548 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x56386fddeb90 0x56386fde0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:45.998 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.998+0000 7f04e1e00640 1 --2- 172.21.15.151:0/3053137548 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x56386fddeb90 0x56386fde0f80 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e349c76a74786a1 server_cookie=39ac44c0d2fb613f in_seq=0 out_seq=0 2024-04-26T01:54:45.998 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.998+0000 7f04c3e00640 10 client.15222 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:54:45.998 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.999+0000 7f04c3e00640 1 -- 172.21.15.151:0/3053137548 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56386fd3d8b0 con 0x56386fddeb90 2024-04-26T01:54:45.998 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.999+0000 7f04c3e00640 10 client.15222 handle_command_reply: tid=0 2024-04-26T01:54:45.998 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.999+0000 7f04ebd72740 1 client.15222 shutdown 2024-04-26T01:54:45.998 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.999+0000 7f04ebd72740 1 -- 172.21.15.151:0/3053137548 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56386fdc42d0 msgr2=0x56386fdcc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:45.998 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:45.999+0000 7f04ebd72740 1 --2- 172.21.15.151:0/3053137548 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56386fdc42d0 0x56386fdcc0f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f04d0006010 tx=0x7f04d00044f0 comp rx=0 tx=0).stop 2024-04-26T01:54:45.998 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.000+0000 7f04ebd72740 1 -- 172.21.15.151:0/3053137548 shutdown_connections 2024-04-26T01:54:45.998 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.000+0000 7f04ebd72740 1 --2- 172.21.15.151:0/3053137548 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x56386fddeb90 0x56386fde0f80 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:45.999 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.000+0000 7f04ebd72740 1 --2- 172.21.15.151:0/3053137548 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56386fdceb90 0x56386fdcc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:45.999 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.000+0000 7f04ebd72740 1 --2- 172.21.15.151:0/3053137548 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56386fdc42d0 0x56386fdcc0f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:45.999 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.000+0000 7f04ebd72740 1 --2- 172.21.15.151:0/3053137548 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56386fdc39b0 0x56386fdcbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:45.999 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.000+0000 7f04c3e00640 0 client.15222 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:54:45.999 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.000+0000 7f04ebd72740 1 -- 172.21.15.151:0/3053137548 >> 172.21.15.151:0/3053137548 conn(0x56386fdbfec0 msgr2=0x56386fdc5bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:46.000 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.002+0000 7f04ebd72740 1 -- 172.21.15.151:0/3053137548 shutdown_connections 2024-04-26T01:54:46.000 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.002+0000 7f04ebd72740 1 -- 172.21.15.151:0/3053137548 wait complete. 2024-04-26T01:54:46.000 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.002+0000 7f04ebd72740 20 client.15222 trim_cache size 0 max 16384 2024-04-26T01:54:46.031 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.032+0000 7f04ebd72740 1 Processor -- start 2024-04-26T01:54:46.031 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.032+0000 7f04ebd72740 1 -- start start 2024-04-26T01:54:46.031 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.033+0000 7f04ebd72740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56386fdc39b0 0x56386fdc6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:46.032 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.033+0000 7f04ebd72740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56386fdc42d0 0x56386fdcbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:46.032 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.033+0000 7f04ebd72740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56386fdceb90 0x56386fdcc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:46.032 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.033+0000 7f04ebd72740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f04e406e290 con 0x56386fdceb90 2024-04-26T01:54:46.032 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.033+0000 7f04ebd72740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f04e406bba0 con 0x56386fdc42d0 2024-04-26T01:54:46.032 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.033+0000 7f04ebd72740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f04e419c9d0 con 0x56386fdc39b0 2024-04-26T01:54:46.032 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.033+0000 7f04e1e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56386fdc39b0 0x56386fdc6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:46.032 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.033+0000 7f04e1e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56386fdc39b0 0x56386fdc6370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:46866/0 (socket says 172.21.15.151:46866) 2024-04-26T01:54:46.032 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.033+0000 7f04e1e00640 1 -- 172.21.15.151:0/1229667546 learned_addr learned my addr 172.21.15.151:0/1229667546 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:54:46.032 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.033+0000 7f04e2800640 1 --2- 172.21.15.151:0/1229667546 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56386fdceb90 0x56386fdcc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.033+0000 7f04e1400640 1 --2- 172.21.15.151:0/1229667546 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56386fdc42d0 0x56386fdcbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.034+0000 7f04e1e00640 1 -- 172.21.15.151:0/1229667546 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56386fdc42d0 msgr2=0x56386fdcbb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.034+0000 7f04e1e00640 1 --2- 172.21.15.151:0/1229667546 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56386fdc42d0 0x56386fdcbb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.034+0000 7f04e1e00640 1 -- 172.21.15.151:0/1229667546 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56386fdceb90 msgr2=0x56386fdcc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.034+0000 7f04e1e00640 1 --2- 172.21.15.151:0/1229667546 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56386fdceb90 0x56386fdcc0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.034+0000 7f04e1e00640 1 -- 172.21.15.151:0/1229667546 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56386fdd0f80 con 0x56386fdc39b0 2024-04-26T01:54:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.034+0000 7f04e1400640 1 --2- 172.21.15.151:0/1229667546 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56386fdc42d0 0x56386fdcbb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:54:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.034+0000 7f04e2800640 1 --2- 172.21.15.151:0/1229667546 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56386fdceb90 0x56386fdcc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:54:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.034+0000 7f04e1e00640 1 --2- 172.21.15.151:0/1229667546 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56386fdc39b0 0x56386fdc6370 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x56386fdc4f50 tx=0x7f04d400bd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fe46ba73b1f4e4a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.034+0000 7f04cea00640 1 -- 172.21.15.151:0/1229667546 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d4003d20 con 0x56386fdc39b0 2024-04-26T01:54:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.034+0000 7f04cea00640 1 -- 172.21.15.151:0/1229667546 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04d4003ec0 con 0x56386fdc39b0 2024-04-26T01:54:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.034+0000 7f04cea00640 1 -- 172.21.15.151:0/1229667546 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d40117c0 con 0x56386fdc39b0 2024-04-26T01:54:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.035+0000 7f04ebd72740 1 -- 172.21.15.151:0/1229667546 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56386fdc39b0 msgr2=0x56386fdc6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.035+0000 7f04ebd72740 1 --2- 172.21.15.151:0/1229667546 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56386fdc39b0 0x56386fdc6370 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x56386fdc4f50 tx=0x7f04d400bd30 comp rx=0 tx=0).stop 2024-04-26T01:54:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.035+0000 7f04ebd72740 1 -- 172.21.15.151:0/1229667546 shutdown_connections 2024-04-26T01:54:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.035+0000 7f04ebd72740 1 --2- 172.21.15.151:0/1229667546 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56386fdceb90 0x56386fdcc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.035+0000 7f04ebd72740 1 --2- 172.21.15.151:0/1229667546 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56386fdc42d0 0x56386fdcbb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.035+0000 7f04ebd72740 1 --2- 172.21.15.151:0/1229667546 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56386fdc39b0 0x56386fdc6370 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.035+0000 7f04ebd72740 1 -- 172.21.15.151:0/1229667546 >> 172.21.15.151:0/1229667546 conn(0x56386fdbfec0 msgr2=0x56386fdda3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.035+0000 7f04ebd72740 1 -- 172.21.15.151:0/1229667546 shutdown_connections 2024-04-26T01:54:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.036+0000 7f04ebd72740 1 -- 172.21.15.151:0/1229667546 wait complete. 2024-04-26T01:54:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.036+0000 7f04ebd72740 1 -- start start 2024-04-26T01:54:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.036+0000 7f04ebd72740 1 Processor -- start 2024-04-26T01:54:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.037+0000 7f04ebd72740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56386fdc39b0 0x56386fe3c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.037+0000 7f04ebd72740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56386fdc42d0 0x56386fe3c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.037+0000 7f04ebd72740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56386fdceb90 0x56386fde3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.038+0000 7f04ebd72740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f04e419d020 con 0x56386fdc42d0 2024-04-26T01:54:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.038+0000 7f04ebd72740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f04e419e470 con 0x56386fdceb90 2024-04-26T01:54:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.038+0000 7f04ebd72740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f04e419d6d0 con 0x56386fdc39b0 2024-04-26T01:54:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.038+0000 7f04ebd72740 10 client.0 _ll_register_callbacks cb 0x7f04d0003fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:54:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.038+0000 7f04e1e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56386fdc39b0 0x56386fe3c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.038+0000 7f04e1400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56386fdc42d0 0x56386fe3c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.038+0000 7f04e1e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56386fdc39b0 0x56386fe3c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:46874/0 (socket says 172.21.15.151:46874) 2024-04-26T01:54:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.038+0000 7f04e1e00640 1 -- 172.21.15.151:0/1620220396 learned_addr learned my addr 172.21.15.151:0/1620220396 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:54:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.038+0000 7f04e2800640 1 --2- 172.21.15.151:0/1620220396 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56386fdceb90 0x56386fde3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.038+0000 7f04e1e00640 1 -- 172.21.15.151:0/1620220396 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56386fdceb90 msgr2=0x56386fde3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.038+0000 7f04e1e00640 1 --2- 172.21.15.151:0/1620220396 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56386fdceb90 0x56386fde3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.038+0000 7f04e1e00640 1 -- 172.21.15.151:0/1620220396 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56386fdc42d0 msgr2=0x56386fe3c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.038+0000 7f04e1e00640 1 --2- 172.21.15.151:0/1620220396 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56386fdc42d0 0x56386fe3c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.038+0000 7f04e1e00640 1 -- 172.21.15.151:0/1620220396 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56386fdc4d80 con 0x56386fdc39b0 2024-04-26T01:54:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.038+0000 7f04e1e00640 1 --2- 172.21.15.151:0/1620220396 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56386fdc39b0 0x56386fe3c0e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f04d40058c0 tx=0x7f04d4003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e50ec47944d27311 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.038+0000 7f04c3e00640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-26T01:54:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.039+0000 7f04c3e00640 1 -- 172.21.15.151:0/1620220396 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d4033060 con 0x56386fdc39b0 2024-04-26T01:54:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.039+0000 7f04c3e00640 1 -- 172.21.15.151:0/1620220396 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f04d401d250 con 0x56386fdc39b0 2024-04-26T01:54:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.039+0000 7f04ebd72740 1 -- 172.21.15.151:0/1620220396 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04d4003ec0 con 0x56386fdc39b0 2024-04-26T01:54:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.039+0000 7f04c3e00640 1 -- 172.21.15.151:0/1620220396 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f04d401d450 con 0x56386fdc39b0 2024-04-26T01:54:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.040+0000 7f04ebd72740 10 client.25348 fetch_fsmap learned FSMap version 26 2024-04-26T01:54:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.040+0000 7f04ebd72740 1 -- 172.21.15.151:0/1620220396 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56386fe446c0 con 0x56386fdc39b0 2024-04-26T01:54:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.040+0000 7f04c3e00640 1 -- 172.21.15.151:0/1620220396 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04d4003ec0 con 0x56386fdc39b0 2024-04-26T01:54:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.040+0000 7f04c3e00640 1 -- 172.21.15.151:0/1620220396 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f04d402d030 con 0x56386fdc39b0 2024-04-26T01:54:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.040+0000 7f04ebd72740 10 client.25348 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T01:54:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.040+0000 7f04ebd72740 10 client.25348 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:54:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.040+0000 7f04ebd72740 20 client.25348 populate_metadata read hostname 'smithi151' 2024-04-26T01:54:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.040+0000 7f04ebd72740 1 --2- 172.21.15.151:0/1620220396 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x56386fe44d30 0x56386fe47150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.040+0000 7f04ebd72740 4 client.25348 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:54:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.040+0000 7f04ebd72740 1 -- 172.21.15.151:0/1620220396 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56386fd37320 con 0x56386fe44d30 2024-04-26T01:54:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.041+0000 7f04e1400640 1 --2- 172.21.15.151:0/1620220396 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x56386fe44d30 0x56386fe47150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.041+0000 7f04e1400640 1 --2- 172.21.15.151:0/1620220396 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x56386fe44d30 0x56386fe47150 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db6defa5af6ca98e server_cookie=9dfe33fb401726ce in_seq=0 out_seq=0 2024-04-26T01:54:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.041+0000 7f04c3e00640 10 client.25348 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:54:46.051 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.052+0000 7f04c3e00640 1 -- 172.21.15.151:0/1620220396 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56386fd37320 con 0x56386fe44d30 2024-04-26T01:54:46.051 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.052+0000 7f04c3e00640 10 client.25348 handle_command_reply: tid=0 2024-04-26T01:54:46.051 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.053+0000 7f04ebd72740 1 client.25348 shutdown 2024-04-26T01:54:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.053+0000 7f04ebd72740 1 -- 172.21.15.151:0/1620220396 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56386fdc39b0 msgr2=0x56386fe3c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.053+0000 7f04ebd72740 1 --2- 172.21.15.151:0/1620220396 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56386fdc39b0 0x56386fe3c0e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f04d40058c0 tx=0x7f04d4003ba0 comp rx=0 tx=0).stop 2024-04-26T01:54:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.053+0000 7f04ebd72740 1 -- 172.21.15.151:0/1620220396 shutdown_connections 2024-04-26T01:54:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.053+0000 7f04ebd72740 1 --2- 172.21.15.151:0/1620220396 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x56386fe44d30 0x56386fe47150 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.053+0000 7f04ebd72740 1 --2- 172.21.15.151:0/1620220396 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56386fdceb90 0x56386fde3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.053+0000 7f04ebd72740 1 --2- 172.21.15.151:0/1620220396 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56386fdc42d0 0x56386fe3c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.053+0000 7f04ebd72740 1 --2- 172.21.15.151:0/1620220396 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56386fdc39b0 0x56386fe3c0e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.053+0000 7f04ebd72740 1 -- 172.21.15.151:0/1620220396 >> 172.21.15.151:0/1620220396 conn(0x56386fdbfec0 msgr2=0x56386fdcd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.053+0000 7f04c3e00640 0 client.25348 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:54:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.054+0000 7f04ebd72740 1 -- 172.21.15.151:0/1620220396 shutdown_connections 2024-04-26T01:54:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.054+0000 7f04ebd72740 1 -- 172.21.15.151:0/1620220396 wait complete. 2024-04-26T01:54:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.054+0000 7f04ebd72740 20 client.25348 trim_cache size 0 max 16384 2024-04-26T01:54:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:{ 2024-04-26T01:54:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "message": "", 2024-04-26T01:54:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "return_code": 0 2024-04-26T01:54:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:} 2024-04-26T01:54:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.056+0000 7f04e3200640 1 -- 172.21.15.151:0/1573805738 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f04c80a3f30 msgr2=0x7f04c80a63f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.056+0000 7f04e3200640 1 --2- 172.21.15.151:0/1573805738 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f04c80a3f30 0x7f04c80a63f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f04d0044000 tx=0x7f04d0046000 comp rx=0 tx=0).stop 2024-04-26T01:54:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.056+0000 7f04e3200640 1 -- 172.21.15.151:0/1573805738 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04e419ea20 msgr2=0x7f04e419eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.056+0000 7f04e3200640 1 --2- 172.21.15.151:0/1573805738 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04e419ea20 0x7f04e419eed0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f04dc00eaa0 tx=0x7f04dc00ef60 comp rx=0 tx=0).stop 2024-04-26T01:54:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.056+0000 7f04e3200640 1 -- 172.21.15.151:0/1573805738 shutdown_connections 2024-04-26T01:54:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.056+0000 7f04e3200640 1 --2- 172.21.15.151:0/1573805738 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f04c80a3f30 0x7f04c80a63f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.056+0000 7f04e3200640 1 --2- 172.21.15.151:0/1573805738 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04e419ea20 0x7f04e419eed0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.056+0000 7f04e3200640 1 --2- 172.21.15.151:0/1573805738 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04e4105f80 0x7f04e41a4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.056+0000 7f04e3200640 1 --2- 172.21.15.151:0/1573805738 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f04e4100eb0 0x7f04e41a4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.056+0000 7f04e3200640 1 -- 172.21.15.151:0/1573805738 >> 172.21.15.151:0/1573805738 conn(0x7f04e40fc770 msgr2=0x7f04e40fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.056+0000 7f04e3200640 1 -- 172.21.15.151:0/1573805738 shutdown_connections 2024-04-26T01:54:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:46.056+0000 7f04e3200640 1 -- 172.21.15.151:0/1573805738 wait complete. 2024-04-26T01:54:46.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:54:45 smithi172 ceph-mon[32785]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-04-26T01:54:46.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:54:45 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:54:46.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:54:45 smithi165 ceph-mon[32648]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-04-26T01:54:46.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:54:45 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:54:46.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:54:45 smithi151 ceph-mon[28057]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-04-26T01:54:46.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:54:45 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:54:47.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:54:46 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:54:47.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:54:46 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:54:47.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:54:46 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:54:47.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:54:46 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:54:47.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:54:46 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:54:47.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:54:46 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:54:47.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:54:46 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:54:47.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:54:46 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:54:47.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:54:46 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:54:48.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:54:48 smithi151 ceph-mon[28057]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-04-26T01:54:48.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:54:48 smithi172 ceph-mon[32785]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-04-26T01:54:48.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:54:48 smithi165 ceph-mon[32648]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-04-26T01:54:50.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:54:50 smithi151 ceph-mon[28057]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-04-26T01:54:50.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:54:50 smithi172 ceph-mon[32785]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-04-26T01:54:50.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:54:50 smithi165 ceph-mon[32648]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-04-26T01:54:52.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:54:52 smithi151 ceph-mon[28057]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-04-26T01:54:52.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:54:52 smithi172 ceph-mon[32785]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-04-26T01:54:52.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:54:52 smithi165 ceph-mon[32648]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-04-26T01:54:54.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:54:54 smithi151 ceph-mon[28057]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-04-26T01:54:54.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:54:54 smithi172 ceph-mon[32785]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-04-26T01:54:54.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:54:54 smithi165 ceph-mon[32648]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-04-26T01:54:56.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:54:56 smithi172 ceph-mon[32785]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-04-26T01:54:56.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:54:56 smithi165 ceph-mon[32648]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-04-26T01:54:56.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:54:56 smithi151 ceph-mon[28057]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-04-26T01:54:58.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:54:58 smithi172 ceph-mon[32785]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-04-26T01:54:58.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:54:58 smithi165 ceph-mon[32648]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-04-26T01:54:58.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:54:58 smithi151 ceph-mon[28057]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-04-26T01:54:59.197 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.198+0000 7f7f3be00640 1 -- 172.21.15.151:0/4238701264 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7f3c102970 msgr2=0x7f7f3c1069b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:59.197 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.198+0000 7f7f3be00640 1 --2- 172.21.15.151:0/4238701264 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7f3c102970 0x7f7f3c1069b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7f24009900 tx=0x7f7f2402a410 comp rx=0 tx=0).stop 2024-04-26T01:54:59.197 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.198+0000 7f7f3be00640 1 -- 172.21.15.151:0/4238701264 shutdown_connections 2024-04-26T01:54:59.197 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.198+0000 7f7f3be00640 1 --2- 172.21.15.151:0/4238701264 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f3c107090 0x7f7f3c10ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:59.198 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.198+0000 7f7f3be00640 1 --2- 172.21.15.151:0/4238701264 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7f3c102970 0x7f7f3c1069b0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:59.198 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.198+0000 7f7f3be00640 1 --2- 172.21.15.151:0/4238701264 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f3c101fc0 0x7f7f3c1023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:59.198 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.198+0000 7f7f3be00640 1 -- 172.21.15.151:0/4238701264 >> 172.21.15.151:0/4238701264 conn(0x7f7f3c0fd8a0 msgr2=0x7f7f3c0ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:59.198 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.198+0000 7f7f3be00640 1 -- 172.21.15.151:0/4238701264 shutdown_connections 2024-04-26T01:54:59.198 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.198+0000 7f7f3be00640 1 -- 172.21.15.151:0/4238701264 wait complete. 2024-04-26T01:54:59.200 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.199+0000 7f7f3be00640 1 Processor -- start 2024-04-26T01:54:59.200 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.199+0000 7f7f3be00640 1 -- start start 2024-04-26T01:54:59.200 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.200+0000 7f7f3be00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f3c101fc0 0x7f7f3c074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:59.200 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.200+0000 7f7f3be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7f3c107090 0x7f7f3c074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:59.200 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.200+0000 7f7f3be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f3c074cd0 0x7f7f3c1a67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:59.200 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.200+0000 7f7f3be00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3c112020 con 0x7f7f3c107090 2024-04-26T01:54:59.201 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.200+0000 7f7f3be00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3c111ea0 con 0x7f7f3c074cd0 2024-04-26T01:54:59.201 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.200+0000 7f7f3be00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3c1121a0 con 0x7f7f3c101fc0 2024-04-26T01:54:59.201 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.200+0000 7f7f33e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7f3c107090 0x7f7f3c074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:59.201 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.200+0000 7f7f33e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7f3c107090 0x7f7f3c074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:45764/0 (socket says 172.21.15.151:45764) 2024-04-26T01:54:59.201 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.200+0000 7f7f33e00640 1 -- 172.21.15.151:0/3233424939 learned_addr learned my addr 172.21.15.151:0/3233424939 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:54:59.201 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.200+0000 7f7f3b400640 1 --2- 172.21.15.151:0/3233424939 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f3c074cd0 0x7f7f3c1a67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:59.201 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.200+0000 7f7f3aa00640 1 --2- 172.21.15.151:0/3233424939 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f3c101fc0 0x7f7f3c074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:59.201 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.200+0000 7f7f33e00640 1 -- 172.21.15.151:0/3233424939 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f3c101fc0 msgr2=0x7f7f3c074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:59.201 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.200+0000 7f7f33e00640 1 --2- 172.21.15.151:0/3233424939 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f3c101fc0 0x7f7f3c074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:59.201 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.200+0000 7f7f33e00640 1 -- 172.21.15.151:0/3233424939 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f3c074cd0 msgr2=0x7f7f3c1a67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:59.201 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.200+0000 7f7f33e00640 1 --2- 172.21.15.151:0/3233424939 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f3c074cd0 0x7f7f3c1a67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:59.201 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.200+0000 7f7f33e00640 1 -- 172.21.15.151:0/3233424939 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f3c1a6e70 con 0x7f7f3c107090 2024-04-26T01:54:59.202 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.201+0000 7f7f33e00640 1 --2- 172.21.15.151:0/3233424939 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7f3c107090 0x7f7f3c074790 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7f24005ab0 tx=0x7f7f24003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:59.202 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.201+0000 7f7f33400640 1 -- 172.21.15.151:0/3233424939 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f2403f070 con 0x7f7f3c107090 2024-04-26T01:54:59.202 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.201+0000 7f7f33400640 1 -- 172.21.15.151:0/3233424939 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f24031b50 con 0x7f7f3c107090 2024-04-26T01:54:59.202 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.201+0000 7f7f3be00640 1 -- 172.21.15.151:0/3233424939 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f3c1a7100 con 0x7f7f3c107090 2024-04-26T01:54:59.202 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.201+0000 7f7f33400640 1 -- 172.21.15.151:0/3233424939 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f24035030 con 0x7f7f3c107090 2024-04-26T01:54:59.202 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.201+0000 7f7f3be00640 1 -- 172.21.15.151:0/3233424939 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f3c1a75f0 con 0x7f7f3c107090 2024-04-26T01:54:59.204 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.205+0000 7f7f445eb740 1 Processor -- start 2024-04-26T01:54:59.204 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.206+0000 7f7f445eb740 1 -- start start 2024-04-26T01:54:59.205 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.206+0000 7f7f33400640 1 -- 172.21.15.151:0/3233424939 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 138931+0+0 (secure 0 0 0) 0x7f7f24042070 con 0x7f7f3c107090 2024-04-26T01:54:59.205 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.206+0000 7f7f445eb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de402c29b0 0x55de402c2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:59.205 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.206+0000 7f7f445eb740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55de402c32d0 0x55de402cd650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:59.205 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.206+0000 7f7f445eb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de402cdb90 0x55de402cff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.206+0000 7f7f445eb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55de40228900 con 0x55de402cdb90 2024-04-26T01:54:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.206+0000 7f7f445eb740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55de401ceef0 con 0x55de402c32d0 2024-04-26T01:54:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.206+0000 7f7f445eb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55de402248c0 con 0x55de402c29b0 2024-04-26T01:54:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.206+0000 7f7f3b400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de402c29b0 0x55de402c2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.206+0000 7f7f3b400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de402c29b0 0x55de402c2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:35916/0 (socket says 172.21.15.151:35916) 2024-04-26T01:54:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.206+0000 7f7f3b400640 1 -- 172.21.15.151:0/4059007383 learned_addr learned my addr 172.21.15.151:0/4059007383 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:54:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.206+0000 7f7f33e00640 1 --2- 172.21.15.151:0/4059007383 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de402cdb90 0x55de402cff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.206+0000 7f7f3aa00640 1 --2- 172.21.15.151:0/4059007383 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55de402c32d0 0x55de402cd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.207+0000 7f7f3b400640 1 -- 172.21.15.151:0/4059007383 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55de402c32d0 msgr2=0x55de402cd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.207+0000 7f7f3b400640 1 --2- 172.21.15.151:0/4059007383 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55de402c32d0 0x55de402cd650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.207+0000 7f7f3b400640 1 -- 172.21.15.151:0/4059007383 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de402cdb90 msgr2=0x55de402cff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.207+0000 7f7f3b400640 1 --2- 172.21.15.151:0/4059007383 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de402cdb90 0x55de402cff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:59.207 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.207+0000 7f7f3b400640 1 -- 172.21.15.151:0/4059007383 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de3ff3b210 con 0x55de402c29b0 2024-04-26T01:54:59.207 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.207+0000 7f7f33e00640 1 --2- 172.21.15.151:0/4059007383 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de402cdb90 0x55de402cff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:54:59.207 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.207+0000 7f7f3b400640 1 --2- 172.21.15.151:0/4059007383 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de402c29b0 0x55de402c2d90 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7f34009f50 tx=0x7f7f340118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8108e7cd5b333828 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:59.211 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.207+0000 7f7f2b400640 1 -- 172.21.15.151:0/4059007383 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f34004d20 con 0x55de402c29b0 2024-04-26T01:54:59.211 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.207+0000 7f7f2b400640 1 -- 172.21.15.151:0/4059007383 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f34004050 con 0x55de402c29b0 2024-04-26T01:54:59.211 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.207+0000 7f7f2b400640 1 -- 172.21.15.151:0/4059007383 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f34005220 con 0x55de402c29b0 2024-04-26T01:54:59.211 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.208+0000 7f7f33400640 1 --2- 172.21.15.151:0/3233424939 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f7f200a3ec0 0x7f7f200a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:59.212 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.208+0000 7f7f33400640 1 -- 172.21.15.151:0/3233424939 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f7f240f1040 con 0x7f7f3c107090 2024-04-26T01:54:59.212 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.208+0000 7f7f3aa00640 1 --2- 172.21.15.151:0/3233424939 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f7f200a3ec0 0x7f7f200a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:59.212 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.208+0000 7f7f445eb740 1 -- 172.21.15.151:0/4059007383 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de402c29b0 msgr2=0x55de402c2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:59.212 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.208+0000 7f7f445eb740 1 --2- 172.21.15.151:0/4059007383 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de402c29b0 0x55de402c2d90 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7f34009f50 tx=0x7f7f340118d0 comp rx=0 tx=0).stop 2024-04-26T01:54:59.212 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.208+0000 7f7f445eb740 1 -- 172.21.15.151:0/4059007383 shutdown_connections 2024-04-26T01:54:59.212 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.208+0000 7f7f445eb740 1 --2- 172.21.15.151:0/4059007383 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de402cdb90 0x55de402cff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:59.212 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.208+0000 7f7f445eb740 1 --2- 172.21.15.151:0/4059007383 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55de402c32d0 0x55de402cd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:59.212 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.208+0000 7f7f445eb740 1 --2- 172.21.15.151:0/4059007383 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de402c29b0 0x55de402c2d90 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:59.212 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.208+0000 7f7f445eb740 1 -- 172.21.15.151:0/4059007383 >> 172.21.15.151:0/4059007383 conn(0x55de402beec0 msgr2=0x55de402bf2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:59.212 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.208+0000 7f7f445eb740 1 -- 172.21.15.151:0/4059007383 shutdown_connections 2024-04-26T01:54:59.212 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.209+0000 7f7f445eb740 1 -- 172.21.15.151:0/4059007383 wait complete. 2024-04-26T01:54:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.209+0000 7f7f3aa00640 1 --2- 172.21.15.151:0/3233424939 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f7f200a3ec0 0x7f7f200a6380 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x55de402c3f50 tx=0x7f7f2c005f90 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.209+0000 7f7f445eb740 1 -- start start 2024-04-26T01:54:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.209+0000 7f7f445eb740 1 Processor -- start 2024-04-26T01:54:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.210+0000 7f7f445eb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de402c29b0 0x55de402caab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.210+0000 7f7f445eb740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55de402c32d0 0x55de402caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.210+0000 7f7f445eb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de402cdb90 0x55de402cb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.210+0000 7f7f445eb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55de402d07c0 con 0x55de402c29b0 2024-04-26T01:54:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.210+0000 7f7f445eb740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55de401e2310 con 0x55de402c32d0 2024-04-26T01:54:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.210+0000 7f7f445eb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55de402d0940 con 0x55de402cdb90 2024-04-26T01:54:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.210+0000 7f7f445eb740 10 client.0 _ll_register_callbacks cb 0x55de401b0cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:54:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.210+0000 7f7f3aa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de402c29b0 0x55de402caab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.210+0000 7f7f3aa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de402c29b0 0x55de402caab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:45786/0 (socket says 172.21.15.151:45786) 2024-04-26T01:54:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.210+0000 7f7f3aa00640 1 -- 172.21.15.151:0/2484461785 learned_addr learned my addr 172.21.15.151:0/2484461785 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:54:59.214 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.210+0000 7f7f33e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55de402c32d0 0x55de402caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:59.214 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.210+0000 7f7f3b400640 1 --2- 172.21.15.151:0/2484461785 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de402cdb90 0x55de402cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:59.214 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.210+0000 7f7f33e00640 1 -- 172.21.15.151:0/2484461785 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de402cdb90 msgr2=0x55de402cb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:59.214 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.210+0000 7f7f33e00640 1 --2- 172.21.15.151:0/2484461785 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de402cdb90 0x55de402cb530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:59.214 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.210+0000 7f7f33e00640 1 -- 172.21.15.151:0/2484461785 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de402c29b0 msgr2=0x55de402caab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:59.214 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.210+0000 7f7f33e00640 1 --2- 172.21.15.151:0/2484461785 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de402c29b0 0x55de402caab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:59.214 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.210+0000 7f7f33e00640 1 -- 172.21.15.151:0/2484461785 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de40007620 con 0x55de402c32d0 2024-04-26T01:54:59.214 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.211+0000 7f7f3aa00640 1 --2- 172.21.15.151:0/2484461785 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de402c29b0 0x55de402caab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:54:59.214 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.211+0000 7f7f33e00640 1 --2- 172.21.15.151:0/2484461785 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55de402c32d0 0x55de402caff0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7f240ee5c0 tx=0x7f7f24040fc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2af3f98b4361fa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:59.214 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.211+0000 7f7f1fe00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T01:54:59.214 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.211+0000 7f7f1fe00640 1 -- 172.21.15.151:0/2484461785 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f24049070 con 0x55de402c32d0 2024-04-26T01:54:59.214 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.211+0000 7f7f1fe00640 1 -- 172.21.15.151:0/2484461785 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f7f240b9530 con 0x55de402c32d0 2024-04-26T01:54:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.211+0000 7f7f445eb740 1 -- 172.21.15.151:0/2484461785 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f34004050 con 0x55de402c32d0 2024-04-26T01:54:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.211+0000 7f7f1fe00640 1 -- 172.21.15.151:0/2484461785 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7f240bca00 con 0x55de402c32d0 2024-04-26T01:54:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.211+0000 7f7f445eb740 10 client.25340 fetch_fsmap learned FSMap version 26 2024-04-26T01:54:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.211+0000 7f7f445eb740 1 -- 172.21.15.151:0/2484461785 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55de402dc9c0 con 0x55de402c32d0 2024-04-26T01:54:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.212+0000 7f7f1fe00640 1 -- 172.21.15.151:0/2484461785 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f34004050 con 0x55de402c32d0 2024-04-26T01:54:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.212+0000 7f7f1fe00640 1 -- 172.21.15.151:0/2484461785 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f7f240bd120 con 0x55de402c32d0 2024-04-26T01:54:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.212+0000 7f7f445eb740 10 client.25340 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T01:54:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.212+0000 7f7f445eb740 10 client.25340 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:54:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.212+0000 7f7f445eb740 20 client.25340 populate_metadata read hostname 'smithi151' 2024-04-26T01:54:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.212+0000 7f7f445eb740 1 --2- 172.21.15.151:0/2484461785 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55de402ddd60 0x55de402e01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.212+0000 7f7f445eb740 4 client.25340 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:54:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.212+0000 7f7f445eb740 1 -- 172.21.15.151:0/2484461785 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de4023c8b0 con 0x55de402ddd60 2024-04-26T01:54:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.213+0000 7f7f3aa00640 1 --2- 172.21.15.151:0/2484461785 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55de402ddd60 0x55de402e01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:59.216 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.213+0000 7f7f3aa00640 1 --2- 172.21.15.151:0/2484461785 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55de402ddd60 0x55de402e01f0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cb47a4c86c2db7c server_cookie=7a13c4eb677d340a in_seq=0 out_seq=0 2024-04-26T01:54:59.216 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.213+0000 7f7f1fe00640 10 client.25340 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:54:59.217 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.214+0000 7f7f1fe00640 1 -- 172.21.15.151:0/2484461785 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55de4023c8b0 con 0x55de402ddd60 2024-04-26T01:54:59.217 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.214+0000 7f7f1fe00640 10 client.25340 handle_command_reply: tid=0 2024-04-26T01:54:59.217 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.214+0000 7f7f445eb740 1 client.25340 shutdown 2024-04-26T01:54:59.217 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.214+0000 7f7f445eb740 1 -- 172.21.15.151:0/2484461785 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55de402c32d0 msgr2=0x55de402caff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:59.217 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.214+0000 7f7f445eb740 1 --2- 172.21.15.151:0/2484461785 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55de402c32d0 0x55de402caff0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7f240ee5c0 tx=0x7f7f24040fc0 comp rx=0 tx=0).stop 2024-04-26T01:54:59.217 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.214+0000 7f7f445eb740 1 -- 172.21.15.151:0/2484461785 shutdown_connections 2024-04-26T01:54:59.217 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.214+0000 7f7f445eb740 1 --2- 172.21.15.151:0/2484461785 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55de402ddd60 0x55de402e01f0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:59.217 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.214+0000 7f7f445eb740 1 --2- 172.21.15.151:0/2484461785 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de402cdb90 0x55de402cb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:59.217 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.214+0000 7f7f445eb740 1 --2- 172.21.15.151:0/2484461785 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55de402c32d0 0x55de402caff0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:59.217 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.214+0000 7f7f445eb740 1 --2- 172.21.15.151:0/2484461785 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de402c29b0 0x55de402caab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:59.217 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.214+0000 7f7f445eb740 1 -- 172.21.15.151:0/2484461785 >> 172.21.15.151:0/2484461785 conn(0x55de402beec0 msgr2=0x55de402c4bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:59.217 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.214+0000 7f7f1fe00640 0 client.25340 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:54:59.218 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.214+0000 7f7f445eb740 1 -- 172.21.15.151:0/2484461785 shutdown_connections 2024-04-26T01:54:59.218 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.214+0000 7f7f445eb740 1 -- 172.21.15.151:0/2484461785 wait complete. 2024-04-26T01:54:59.218 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.214+0000 7f7f445eb740 20 client.25340 trim_cache size 0 max 16384 2024-04-26T01:54:59.230 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.230+0000 7f7f445eb740 1 Processor -- start 2024-04-26T01:54:59.230 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.230+0000 7f7f445eb740 1 -- start start 2024-04-26T01:54:59.230 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.231+0000 7f7f445eb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de402c29b0 0x55de402d9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:59.230 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.231+0000 7f7f445eb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de402c32d0 0x55de402c5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:59.230 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.231+0000 7f7f445eb740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55de402cdb90 0x55de402e5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:59.230 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.231+0000 7f7f445eb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3c1b2370 con 0x55de402c32d0 2024-04-26T01:54:59.230 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.231+0000 7f7f445eb740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3c1afc40 con 0x55de402cdb90 2024-04-26T01:54:59.230 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.231+0000 7f7f445eb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3c1adab0 con 0x55de402c29b0 2024-04-26T01:54:59.230 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.231+0000 7f7f33e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de402c32d0 0x55de402c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:59.230 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.231+0000 7f7f3aa00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de402c29b0 0x55de402d9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:59.230 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.231+0000 7f7f33e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de402c32d0 0x55de402c5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:45794/0 (socket says 172.21.15.151:45794) 2024-04-26T01:54:59.231 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.231+0000 7f7f33e00640 1 -- 172.21.15.151:0/482139771 learned_addr learned my addr 172.21.15.151:0/482139771 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:54:59.231 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.231+0000 7f7f3b400640 1 --2- 172.21.15.151:0/482139771 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55de402cdb90 0x55de402e5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:59.231 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.231+0000 7f7f33e00640 1 -- 172.21.15.151:0/482139771 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de402c29b0 msgr2=0x55de402d9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:59.231 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.231+0000 7f7f33e00640 1 --2- 172.21.15.151:0/482139771 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de402c29b0 0x55de402d9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:59.231 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.231+0000 7f7f33e00640 1 -- 172.21.15.151:0/482139771 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55de402cdb90 msgr2=0x55de402e5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:59.231 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.231+0000 7f7f33e00640 1 --2- 172.21.15.151:0/482139771 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55de402cdb90 0x55de402e5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:59.231 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.231+0000 7f7f33e00640 1 -- 172.21.15.151:0/482139771 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de402e0870 con 0x55de402c32d0 2024-04-26T01:54:59.233 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.232+0000 7f7f33e00640 1 --2- 172.21.15.151:0/482139771 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de402c32d0 0x55de402c5370 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7f240ee700 tx=0x7f7f240bb3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb8cf6b859038eff server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:59.234 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.232+0000 7f7f2aa00640 1 -- 172.21.15.151:0/482139771 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f24049030 con 0x55de402c32d0 2024-04-26T01:54:59.234 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.232+0000 7f7f2aa00640 1 -- 172.21.15.151:0/482139771 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f240bcec0 con 0x55de402c32d0 2024-04-26T01:54:59.234 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.232+0000 7f7f2aa00640 1 -- 172.21.15.151:0/482139771 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f240bc820 con 0x55de402c32d0 2024-04-26T01:54:59.234 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.233+0000 7f7f445eb740 1 -- 172.21.15.151:0/482139771 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de402c32d0 msgr2=0x55de402c5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:59.235 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.233+0000 7f7f445eb740 1 --2- 172.21.15.151:0/482139771 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de402c32d0 0x55de402c5370 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7f240ee700 tx=0x7f7f240bb3d0 comp rx=0 tx=0).stop 2024-04-26T01:54:59.235 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.233+0000 7f7f445eb740 1 -- 172.21.15.151:0/482139771 shutdown_connections 2024-04-26T01:54:59.235 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.233+0000 7f7f445eb740 1 --2- 172.21.15.151:0/482139771 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55de402cdb90 0x55de402e5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:59.235 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.233+0000 7f7f445eb740 1 --2- 172.21.15.151:0/482139771 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de402c32d0 0x55de402c5370 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:59.235 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.233+0000 7f7f445eb740 1 --2- 172.21.15.151:0/482139771 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de402c29b0 0x55de402d9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:59.236 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.233+0000 7f7f445eb740 1 -- 172.21.15.151:0/482139771 >> 172.21.15.151:0/482139771 conn(0x55de402beec0 msgr2=0x55de402be020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:59.236 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.233+0000 7f7f445eb740 1 -- 172.21.15.151:0/482139771 shutdown_connections 2024-04-26T01:54:59.236 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.233+0000 7f7f445eb740 1 -- 172.21.15.151:0/482139771 wait complete. 2024-04-26T01:54:59.236 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.233+0000 7f7f445eb740 1 -- start start 2024-04-26T01:54:59.236 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.234+0000 7f7f445eb740 1 Processor -- start 2024-04-26T01:54:59.236 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.234+0000 7f7f445eb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de402c29b0 0x55de402d5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:59.236 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.234+0000 7f7f445eb740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55de402c32d0 0x55de402d60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:59.236 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.234+0000 7f7f445eb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de402cdb90 0x55de402d3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:59.236 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.234+0000 7f7f445eb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3c1ae100 con 0x55de402cdb90 2024-04-26T01:54:59.236 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.234+0000 7f7f445eb740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3c1af570 con 0x55de402c32d0 2024-04-26T01:54:59.236 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.234+0000 7f7f445eb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3c1ae7d0 con 0x55de402c29b0 2024-04-26T01:54:59.236 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.234+0000 7f7f445eb740 10 client.0 _ll_register_callbacks cb 0x7f7f24005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:54:59.236 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.234+0000 7f7f3b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de402cdb90 0x55de402d3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:59.237 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.234+0000 7f7f3b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de402cdb90 0x55de402d3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:45808/0 (socket says 172.21.15.151:45808) 2024-04-26T01:54:59.237 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.234+0000 7f7f3b400640 1 -- 172.21.15.151:0/1070056515 learned_addr learned my addr 172.21.15.151:0/1070056515 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:54:59.237 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.234+0000 7f7f3aa00640 1 --2- 172.21.15.151:0/1070056515 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de402c29b0 0x55de402d5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:59.237 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.234+0000 7f7f33e00640 1 --2- 172.21.15.151:0/1070056515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55de402c32d0 0x55de402d60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:59.237 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.235+0000 7f7f3b400640 1 -- 172.21.15.151:0/1070056515 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de402c29b0 msgr2=0x55de402d5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:59.237 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.235+0000 7f7f3b400640 1 --2- 172.21.15.151:0/1070056515 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de402c29b0 0x55de402d5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:59.237 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.235+0000 7f7f3b400640 1 -- 172.21.15.151:0/1070056515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55de402c32d0 msgr2=0x55de402d60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:59.237 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.235+0000 7f7f3b400640 1 --2- 172.21.15.151:0/1070056515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55de402c32d0 0x55de402d60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:59.237 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.235+0000 7f7f3b400640 1 -- 172.21.15.151:0/1070056515 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de402d3900 con 0x55de402cdb90 2024-04-26T01:54:59.237 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.235+0000 7f7f3aa00640 1 --2- 172.21.15.151:0/1070056515 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de402c29b0 0x55de402d5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:54:59.237 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.235+0000 7f7f3b400640 1 --2- 172.21.15.151:0/1070056515 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de402cdb90 0x55de402d3100 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7f34009f50 tx=0x7f7f34005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb4dab2d630f9790 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:59.237 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.235+0000 7f7f1fe00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-04-26T01:54:59.239 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.236+0000 7f7f1fe00640 1 -- 172.21.15.151:0/1070056515 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f340055c0 con 0x55de402cdb90 2024-04-26T01:54:59.239 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.236+0000 7f7f445eb740 1 -- 172.21.15.151:0/1070056515 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55de40236320 con 0x55de402cdb90 2024-04-26T01:54:59.239 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.236+0000 7f7f1fe00640 1 -- 172.21.15.151:0/1070056515 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f7f34005760 con 0x55de402cdb90 2024-04-26T01:54:59.239 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.236+0000 7f7f1fe00640 1 -- 172.21.15.151:0/1070056515 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7f34005960 con 0x55de402cdb90 2024-04-26T01:54:59.239 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.236+0000 7f7f445eb740 10 client.15246 fetch_fsmap learned FSMap version 26 2024-04-26T01:54:59.239 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.236+0000 7f7f445eb740 1 -- 172.21.15.151:0/1070056515 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55de40343fd0 con 0x55de402cdb90 2024-04-26T01:54:59.239 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.236+0000 7f7f1fe00640 1 -- 172.21.15.151:0/1070056515 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55de40236320 con 0x55de402cdb90 2024-04-26T01:54:59.239 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.236+0000 7f7f1fe00640 1 -- 172.21.15.151:0/1070056515 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f7f340195f0 con 0x55de402cdb90 2024-04-26T01:54:59.240 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.237+0000 7f7f445eb740 10 client.15246 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T01:54:59.240 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.237+0000 7f7f445eb740 10 client.15246 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:54:59.240 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.237+0000 7f7f445eb740 20 client.15246 populate_metadata read hostname 'smithi151' 2024-04-26T01:54:59.240 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.237+0000 7f7f445eb740 1 --2- 172.21.15.151:0/1070056515 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55de40344610 0x55de40346a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:59.240 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.237+0000 7f7f445eb740 4 client.15246 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:54:59.240 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.237+0000 7f7f445eb740 1 -- 172.21.15.151:0/1070056515 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55de40230190 con 0x55de40344610 2024-04-26T01:54:59.240 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.237+0000 7f7f3aa00640 1 --2- 172.21.15.151:0/1070056515 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55de40344610 0x55de40346a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:59.240 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.237+0000 7f7f3aa00640 1 --2- 172.21.15.151:0/1070056515 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55de40344610 0x55de40346a30 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f3547ae291720bc server_cookie=4a680e0e41468cf7 in_seq=0 out_seq=0 2024-04-26T01:54:59.240 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.237+0000 7f7f1fe00640 10 client.15246 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:54:59.242 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.243+0000 7f7f1fe00640 1 -- 172.21.15.151:0/1070056515 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55de40230190 con 0x55de40344610 2024-04-26T01:54:59.242 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.243+0000 7f7f1fe00640 10 client.15246 handle_command_reply: tid=0 2024-04-26T01:54:59.243 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.243+0000 7f7f445eb740 1 client.15246 shutdown 2024-04-26T01:54:59.243 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.243+0000 7f7f445eb740 1 -- 172.21.15.151:0/1070056515 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de402cdb90 msgr2=0x55de402d3100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:59.243 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.243+0000 7f7f445eb740 1 --2- 172.21.15.151:0/1070056515 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de402cdb90 0x55de402d3100 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7f34009f50 tx=0x7f7f34005040 comp rx=0 tx=0).stop 2024-04-26T01:54:59.243 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.244+0000 7f7f445eb740 1 -- 172.21.15.151:0/1070056515 shutdown_connections 2024-04-26T01:54:59.243 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.244+0000 7f7f445eb740 1 --2- 172.21.15.151:0/1070056515 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55de40344610 0x55de40346a30 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:59.243 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.244+0000 7f7f445eb740 1 --2- 172.21.15.151:0/1070056515 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de402cdb90 0x55de402d3100 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:59.243 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.244+0000 7f7f445eb740 1 --2- 172.21.15.151:0/1070056515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55de402c32d0 0x55de402d60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:59.243 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.244+0000 7f7f445eb740 1 --2- 172.21.15.151:0/1070056515 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de402c29b0 0x55de402d5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:59.243 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.244+0000 7f7f445eb740 1 -- 172.21.15.151:0/1070056515 >> 172.21.15.151:0/1070056515 conn(0x55de402beec0 msgr2=0x55de402bd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:59.243 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.244+0000 7f7f1fe00640 0 client.15246 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:54:59.243 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.244+0000 7f7f445eb740 1 -- 172.21.15.151:0/1070056515 shutdown_connections 2024-04-26T01:54:59.243 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.244+0000 7f7f445eb740 1 -- 172.21.15.151:0/1070056515 wait complete. 2024-04-26T01:54:59.243 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.244+0000 7f7f445eb740 20 client.15246 trim_cache size 0 max 16384 2024-04-26T01:54:59.244 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:{ 2024-04-26T01:54:59.244 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "message": "", 2024-04-26T01:54:59.244 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "return_code": 0 2024-04-26T01:54:59.244 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:} 2024-04-26T01:54:59.245 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.245+0000 7f7f3be00640 1 -- 172.21.15.151:0/3233424939 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f7f200a3ec0 msgr2=0x7f7f200a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:59.245 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.245+0000 7f7f3be00640 1 --2- 172.21.15.151:0/3233424939 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f7f200a3ec0 0x7f7f200a6380 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x55de402c3f50 tx=0x7f7f2c005f90 comp rx=0 tx=0).stop 2024-04-26T01:54:59.245 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.245+0000 7f7f3be00640 1 -- 172.21.15.151:0/3233424939 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7f3c107090 msgr2=0x7f7f3c074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:59.245 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.245+0000 7f7f3be00640 1 --2- 172.21.15.151:0/3233424939 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7f3c107090 0x7f7f3c074790 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7f24005ab0 tx=0x7f7f24003b90 comp rx=0 tx=0).stop 2024-04-26T01:54:59.245 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.245+0000 7f7f3be00640 1 -- 172.21.15.151:0/3233424939 shutdown_connections 2024-04-26T01:54:59.245 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.245+0000 7f7f3be00640 1 --2- 172.21.15.151:0/3233424939 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f7f200a3ec0 0x7f7f200a6380 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:59.245 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.245+0000 7f7f3be00640 1 --2- 172.21.15.151:0/3233424939 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f3c074cd0 0x7f7f3c1a67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:59.245 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.245+0000 7f7f3be00640 1 --2- 172.21.15.151:0/3233424939 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7f3c107090 0x7f7f3c074790 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:59.245 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.245+0000 7f7f3be00640 1 --2- 172.21.15.151:0/3233424939 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f3c101fc0 0x7f7f3c074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:59.245 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.245+0000 7f7f3be00640 1 -- 172.21.15.151:0/3233424939 >> 172.21.15.151:0/3233424939 conn(0x7f7f3c0fd8a0 msgr2=0x7f7f3c0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:59.246 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.245+0000 7f7f3be00640 1 -- 172.21.15.151:0/3233424939 shutdown_connections 2024-04-26T01:54:59.246 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:54:59.245+0000 7f7f3be00640 1 -- 172.21.15.151:0/3233424939 wait complete. 2024-04-26T01:55:00.571 INFO:tasks.workunit.client.0.smithi151.stderr:Updating files: 66% (8259/12492) Updating files: 67% (8370/12492) Updating files: 68% (8495/12492) Updating files: 69% (8620/12492) Updating files: 70% (8745/12492) Updating files: 71% (8870/12492) Updating files: 72% (8995/12492) Updating files: 73% (9120/12492) Updating files: 74% (9245/12492) Updating files: 75% (9369/12492) Updating files: 76% (9494/12492) Updating files: 77% (9619/12492) Updating files: 78% (9744/12492) Updating files: 79% (9869/12492) Updating files: 80% (9994/12492) Updating files: 81% (10119/12492) Updating files: 82% (10244/12492) Updating files: 83% (10369/12492) Updating files: 84% (10494/12492) Updating files: 85% (10619/12492) Updating files: 86% (10744/12492) Updating files: 87% (10869/12492) Updating files: 88% (10993/12492) Updating files: 89% (11118/12492) Updating files: 90% (11243/12492) Updating files: 91% (11368/12492) Updating files: 92% (11493/12492) Updating files: 93% (11618/12492) Updating files: 94% (11743/12492) Updating files: 95% (11868/12492) Updating files: 96% (11993/12492) Updating files: 97% (12118/12492) Updating files: 98% (12243/12492) Updating files: 99% (12368/12492) Updating files: 100% (12492/12492) Updating files: 100% (12492/12492), done. 2024-04-26T01:55:00.674 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:55:00 smithi151 ceph-mon[28057]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-26T01:55:00.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:55:00 smithi172 ceph-mon[32785]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-26T01:55:00.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:55:00 smithi165 ceph-mon[32648]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-26T01:55:00.965 INFO:tasks.workunit.client.0.smithi151.stderr:Note: switching to 'afa1933b0fdbb6c99c947d1eda34d661d23cd327'. 2024-04-26T01:55:00.965 INFO:tasks.workunit.client.0.smithi151.stderr: 2024-04-26T01:55:00.965 INFO:tasks.workunit.client.0.smithi151.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-26T01:55:00.965 INFO:tasks.workunit.client.0.smithi151.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-26T01:55:00.965 INFO:tasks.workunit.client.0.smithi151.stderr:state without impacting any branches by switching back to a branch. 2024-04-26T01:55:00.965 INFO:tasks.workunit.client.0.smithi151.stderr: 2024-04-26T01:55:00.965 INFO:tasks.workunit.client.0.smithi151.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-26T01:55:00.965 INFO:tasks.workunit.client.0.smithi151.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-26T01:55:00.966 INFO:tasks.workunit.client.0.smithi151.stderr: 2024-04-26T01:55:00.966 INFO:tasks.workunit.client.0.smithi151.stderr: git switch -c 2024-04-26T01:55:00.966 INFO:tasks.workunit.client.0.smithi151.stderr: 2024-04-26T01:55:00.966 INFO:tasks.workunit.client.0.smithi151.stderr:Or undo this operation with: 2024-04-26T01:55:00.966 INFO:tasks.workunit.client.0.smithi151.stderr: 2024-04-26T01:55:00.966 INFO:tasks.workunit.client.0.smithi151.stderr: git switch - 2024-04-26T01:55:00.966 INFO:tasks.workunit.client.0.smithi151.stderr: 2024-04-26T01:55:00.967 INFO:tasks.workunit.client.0.smithi151.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-26T01:55:00.967 INFO:tasks.workunit.client.0.smithi151.stderr: 2024-04-26T01:55:00.967 INFO:tasks.workunit.client.0.smithi151.stderr:HEAD is now at afa1933b0fd qa/suites/fs: add quiescer to the fs suite 2024-04-26T01:55:00.969 DEBUG:teuthology.orchestra.run.smithi151:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-26T01:55:01.074 INFO:tasks.workunit.client.0.smithi151.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-26T01:55:01.082 INFO:tasks.workunit.client.0.smithi151.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-26T01:55:01.083 INFO:tasks.workunit.client.0.smithi151.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-26T01:55:01.908 INFO:tasks.workunit.client.0.smithi151.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-26T01:55:02.006 INFO:tasks.workunit.client.0.smithi151.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-26T01:55:02.088 INFO:tasks.workunit.client.0.smithi151.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-26T01:55:02.093 INFO:tasks.workunit.client.0.smithi151.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-26T01:55:02.094 INFO:tasks.workunit.client.0.smithi151.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-26T01:55:02.168 INFO:tasks.workunit.client.0.smithi151.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-26T01:55:02.178 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-26T01:55:02.178 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-26T01:55:02.223 INFO:tasks.workunit:Running workunits matching suites/fsx.sh on client.0... 2024-04-26T01:55:02.224 INFO:tasks.workunit:Running workunit suites/fsx.sh... 2024-04-26T01:55:02.224 DEBUG:teuthology.orchestra.run.smithi151:workunit test suites/fsx.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=afa1933b0fdbb6c99c947d1eda34d661d23cd327 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh 2024-04-26T01:55:02.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:55:02 smithi172 ceph-mon[32785]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T01:55:02.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:55:02 smithi165 ceph-mon[32648]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T01:55:02.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:55:02 smithi151 ceph-mon[28057]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T01:55:04.648 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:55:04 smithi165 ceph-mon[32648]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-26T01:55:04.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:55:04 smithi172 ceph-mon[32785]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-26T01:55:04.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:55:04 smithi151 ceph-mon[28057]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-26T01:55:06.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:55:06 smithi172 ceph-mon[32785]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-26T01:55:06.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:55:06 smithi165 ceph-mon[32648]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-26T01:55:06.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:55:06 smithi151 ceph-mon[28057]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-26T01:55:08.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:55:08 smithi172 ceph-mon[32785]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-26T01:55:08.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:55:08 smithi165 ceph-mon[32648]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-26T01:55:08.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:55:08 smithi151 ceph-mon[28057]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-26T01:55:10.281 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:55:10.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:55:10 smithi151 ceph-mon[28057]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-04-26T01:55:10.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:55:10 smithi151 ceph-mon[28057]: mgrmap e27: x(active, since 92s), standbys: y, z 2024-04-26T01:55:10.740 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:55:10.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:55:10 smithi172 ceph-mon[32785]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-04-26T01:55:10.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:55:10 smithi172 ceph-mon[32785]: mgrmap e27: x(active, since 92s), standbys: y, z 2024-04-26T01:55:10.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:55:10 smithi165 ceph-mon[32648]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-04-26T01:55:10.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:55:10 smithi165 ceph-mon[32648]: mgrmap e27: x(active, since 92s), standbys: y, z 2024-04-26T01:55:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.385+0000 7f1e82200640 1 -- 172.21.15.151:0/809816988 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e7c074b70 msgr2=0x7f1e7c10e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:12.386 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.385+0000 7f1e82200640 1 --2- 172.21.15.151:0/809816988 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e7c074b70 0x7f1e7c10e300 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1e68009900 tx=0x7f1e6802a480 comp rx=0 tx=0).stop 2024-04-26T01:55:12.386 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.385+0000 7f1e82200640 1 -- 172.21.15.151:0/809816988 shutdown_connections 2024-04-26T01:55:12.386 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.385+0000 7f1e82200640 1 --2- 172.21.15.151:0/809816988 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1e7c10e840 0x7f1e7c110c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:12.386 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.385+0000 7f1e82200640 1 --2- 172.21.15.151:0/809816988 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e7c074b70 0x7f1e7c10e300 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:12.386 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.385+0000 7f1e82200640 1 --2- 172.21.15.151:0/809816988 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e7c074250 0x7f1e7c074630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:12.386 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.385+0000 7f1e82200640 1 -- 172.21.15.151:0/809816988 >> 172.21.15.151:0/809816988 conn(0x7f1e7c0793e0 msgr2=0x7f1e7c0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:12.386 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.385+0000 7f1e82200640 1 -- 172.21.15.151:0/809816988 shutdown_connections 2024-04-26T01:55:12.386 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.385+0000 7f1e82200640 1 -- 172.21.15.151:0/809816988 wait complete. 2024-04-26T01:55:12.386 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.386+0000 7f1e82200640 1 Processor -- start 2024-04-26T01:55:12.386 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.386+0000 7f1e82200640 1 -- start start 2024-04-26T01:55:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.387+0000 7f1e82200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e7c074250 0x7f1e7c1a0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.387+0000 7f1e82200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1e7c074b70 0x7f1e7c1a0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.387+0000 7f1e82200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e7c10e840 0x7f1e7c19a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.387+0000 7f1e82200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1e7c114440 con 0x7f1e7c074250 2024-04-26T01:55:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.387+0000 7f1e82200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1e7c1142c0 con 0x7f1e7c074b70 2024-04-26T01:55:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.387+0000 7f1e82200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1e7c1145c0 con 0x7f1e7c10e840 2024-04-26T01:55:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.387+0000 7f1e7b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e7c074250 0x7f1e7c1a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.387+0000 7f1e7aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1e7c074b70 0x7f1e7c1a0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.387+0000 7f1e7be00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e7c10e840 0x7f1e7c19a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.387+0000 7f1e7aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1e7c074b70 0x7f1e7c1a0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:54150/0 (socket says 172.21.15.151:54150) 2024-04-26T01:55:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.387+0000 7f1e7aa00640 1 -- 172.21.15.151:0/2917839175 learned_addr learned my addr 172.21.15.151:0/2917839175 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:55:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.387+0000 7f1e7be00640 1 -- 172.21.15.151:0/2917839175 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1e7c074b70 msgr2=0x7f1e7c1a0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.387+0000 7f1e7be00640 1 --2- 172.21.15.151:0/2917839175 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1e7c074b70 0x7f1e7c1a0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.387+0000 7f1e7be00640 1 -- 172.21.15.151:0/2917839175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e7c074250 msgr2=0x7f1e7c1a0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.387+0000 7f1e7be00640 1 --2- 172.21.15.151:0/2917839175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e7c074250 0x7f1e7c1a0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.387+0000 7f1e7be00640 1 -- 172.21.15.151:0/2917839175 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e7c19af60 con 0x7f1e7c10e840 2024-04-26T01:55:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.388+0000 7f1e7be00640 1 --2- 172.21.15.151:0/2917839175 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e7c10e840 0x7f1e7c19a6a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1e74005fd0 tx=0x7f1e7400ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.388+0000 7f1e73e00640 1 -- 172.21.15.151:0/2917839175 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7400cd80 con 0x7f1e7c10e840 2024-04-26T01:55:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.388+0000 7f1e82200640 1 -- 172.21.15.151:0/2917839175 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e7c19b250 con 0x7f1e7c10e840 2024-04-26T01:55:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.388+0000 7f1e73e00640 1 -- 172.21.15.151:0/2917839175 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e74005020 con 0x7f1e7c10e840 2024-04-26T01:55:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.388+0000 7f1e73e00640 1 -- 172.21.15.151:0/2917839175 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e74010690 con 0x7f1e7c10e840 2024-04-26T01:55:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.388+0000 7f1e82200640 1 -- 172.21.15.151:0/2917839175 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e7c1a7510 con 0x7f1e7c10e840 2024-04-26T01:55:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.392+0000 7f1e84f84740 1 Processor -- start 2024-04-26T01:55:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.392+0000 7f1e73e00640 1 -- 172.21.15.151:0/2917839175 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f1e740054d0 con 0x7f1e7c10e840 2024-04-26T01:55:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.392+0000 7f1e84f84740 1 -- start start 2024-04-26T01:55:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.392+0000 7f1e84f84740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563189d8e820 0x563189d8ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.392+0000 7f1e84f84740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563189d8f140 0x563189d994c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.392+0000 7f1e84f84740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563189d99a00 0x563189d9bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.392+0000 7f1e84f84740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563189cf4900 con 0x563189d8f140 2024-04-26T01:55:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.392+0000 7f1e84f84740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563189c9aef0 con 0x563189d8e820 2024-04-26T01:55:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.392+0000 7f1e84f84740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563189cf08c0 con 0x563189d99a00 2024-04-26T01:55:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.393+0000 7f1e7aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563189d8e820 0x563189d8ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.393+0000 7f1e7aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563189d8e820 0x563189d8ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:54160/0 (socket says 172.21.15.151:54160) 2024-04-26T01:55:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.393+0000 7f1e7aa00640 1 -- 172.21.15.151:0/1544461555 learned_addr learned my addr 172.21.15.151:0/1544461555 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:55:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.393+0000 7f1e7b400640 1 --2- 172.21.15.151:0/1544461555 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563189d8f140 0x563189d994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.393+0000 7f1e7aa00640 1 --2- 172.21.15.151:0/1544461555 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563189d99a00 0x563189d9bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:12.394 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.393+0000 7f1e7aa00640 1 -- 172.21.15.151:0/1544461555 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563189d99a00 msgr2=0x563189d9bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:12.394 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.393+0000 7f1e7aa00640 1 --2- 172.21.15.151:0/1544461555 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563189d99a00 0x563189d9bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:12.394 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.393+0000 7f1e7aa00640 1 -- 172.21.15.151:0/1544461555 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563189d8f140 msgr2=0x563189d994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:12.394 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.393+0000 7f1e7aa00640 1 --2- 172.21.15.151:0/1544461555 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563189d8f140 0x563189d994c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:12.395 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.393+0000 7f1e7aa00640 1 -- 172.21.15.151:0/1544461555 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563189a07210 con 0x563189d8e820 2024-04-26T01:55:12.395 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.394+0000 7f1e73e00640 1 --2- 172.21.15.151:0/2917839175 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f1e600a3dc0 0x7f1e600a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:12.395 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.394+0000 7f1e7aa00640 1 --2- 172.21.15.151:0/1544461555 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563189d8e820 0x563189d8ec00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1e68005ab0 tx=0x7f1e6802a480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe6453535fa7a7b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:12.395 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.394+0000 7f1e73e00640 1 -- 172.21.15.151:0/2917839175 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f1e740d0d80 con 0x7f1e7c10e840 2024-04-26T01:55:12.395 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.394+0000 7f1e7b400640 1 --2- 172.21.15.151:0/2917839175 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f1e600a3dc0 0x7f1e600a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:12.395 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.394+0000 7f1e67e00640 1 -- 172.21.15.151:0/1544461555 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e68035030 con 0x563189d8e820 2024-04-26T01:55:12.395 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.394+0000 7f1e67e00640 1 -- 172.21.15.151:0/1544461555 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e68005020 con 0x563189d8e820 2024-04-26T01:55:12.395 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.394+0000 7f1e67e00640 1 -- 172.21.15.151:0/1544461555 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e68004a00 con 0x563189d8e820 2024-04-26T01:55:12.395 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.395+0000 7f1e84f84740 1 -- 172.21.15.151:0/1544461555 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563189d8e820 msgr2=0x563189d8ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:12.395 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.395+0000 7f1e7b400640 1 --2- 172.21.15.151:0/2917839175 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f1e600a3dc0 0x7f1e600a6280 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x563189d8fdc0 tx=0x7f1e6c005f90 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:12.395 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.395+0000 7f1e84f84740 1 --2- 172.21.15.151:0/1544461555 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563189d8e820 0x563189d8ec00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1e68005ab0 tx=0x7f1e6802a480 comp rx=0 tx=0).stop 2024-04-26T01:55:12.395 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.395+0000 7f1e84f84740 1 -- 172.21.15.151:0/1544461555 shutdown_connections 2024-04-26T01:55:12.395 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.395+0000 7f1e84f84740 1 --2- 172.21.15.151:0/1544461555 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563189d99a00 0x563189d9bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:12.395 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.395+0000 7f1e84f84740 1 --2- 172.21.15.151:0/1544461555 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563189d8f140 0x563189d994c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:12.395 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.395+0000 7f1e84f84740 1 --2- 172.21.15.151:0/1544461555 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563189d8e820 0x563189d8ec00 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:12.396 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.395+0000 7f1e84f84740 1 -- 172.21.15.151:0/1544461555 >> 172.21.15.151:0/1544461555 conn(0x563189d8ad30 msgr2=0x563189d8b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:12.396 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.395+0000 7f1e84f84740 1 -- 172.21.15.151:0/1544461555 shutdown_connections 2024-04-26T01:55:12.396 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.395+0000 7f1e84f84740 1 -- 172.21.15.151:0/1544461555 wait complete. 2024-04-26T01:55:12.396 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.396+0000 7f1e84f84740 1 -- start start 2024-04-26T01:55:12.396 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.396+0000 7f1e84f84740 1 Processor -- start 2024-04-26T01:55:12.398 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.396+0000 7f1e84f84740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563189d8e820 0x563189d969d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:12.398 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.396+0000 7f1e84f84740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563189d8f140 0x563189d96f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:12.398 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.396+0000 7f1e84f84740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563189d99a00 0x563189d97450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:12.399 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.396+0000 7f1e84f84740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563189d9c5d0 con 0x563189d99a00 2024-04-26T01:55:12.399 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.396+0000 7f1e84f84740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563189cae310 con 0x563189d8e820 2024-04-26T01:55:12.399 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.396+0000 7f1e84f84740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563189d9c750 con 0x563189d8f140 2024-04-26T01:55:12.399 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.397+0000 7f1e84f84740 10 client.0 _ll_register_callbacks cb 0x563189c7ccb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:55:12.399 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.397+0000 7f1e7b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563189d8e820 0x563189d969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:12.399 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.397+0000 7f1e7b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563189d8e820 0x563189d969d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:54170/0 (socket says 172.21.15.151:54170) 2024-04-26T01:55:12.399 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.397+0000 7f1e7b400640 1 -- 172.21.15.151:0/1682963915 learned_addr learned my addr 172.21.15.151:0/1682963915 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:55:12.399 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.397+0000 7f1e7aa00640 1 --2- 172.21.15.151:0/1682963915 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563189d8f140 0x563189d96f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:12.399 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.397+0000 7f1e7be00640 1 --2- 172.21.15.151:0/1682963915 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563189d99a00 0x563189d97450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:12.399 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.397+0000 7f1e7aa00640 1 -- 172.21.15.151:0/1682963915 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563189d8e820 msgr2=0x563189d969d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:12.399 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.397+0000 7f1e7aa00640 1 --2- 172.21.15.151:0/1682963915 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563189d8e820 0x563189d969d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:12.399 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.397+0000 7f1e7aa00640 1 -- 172.21.15.151:0/1682963915 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563189d99a00 msgr2=0x563189d97450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:12.399 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.397+0000 7f1e7aa00640 1 --2- 172.21.15.151:0/1682963915 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563189d99a00 0x563189d97450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:12.400 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.397+0000 7f1e7aa00640 1 -- 172.21.15.151:0/1682963915 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563189ad3620 con 0x563189d8f140 2024-04-26T01:55:12.400 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.398+0000 7f1e7aa00640 1 --2- 172.21.15.151:0/1682963915 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563189d8f140 0x563189d96f10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1e68045000 tx=0x7f1e68005480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15247947469afc82 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:12.400 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.398+0000 7f1e64c00640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-26T01:55:12.400 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.398+0000 7f1e64c00640 1 -- 172.21.15.151:0/1682963915 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e6804e070 con 0x563189d8f140 2024-04-26T01:55:12.400 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.398+0000 7f1e84f84740 1 -- 172.21.15.151:0/1682963915 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e68005020 con 0x563189d8f140 2024-04-26T01:55:12.400 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.398+0000 7f1e64c00640 1 -- 172.21.15.151:0/1682963915 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f1e68003cd0 con 0x563189d8f140 2024-04-26T01:55:12.400 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.398+0000 7f1e64c00640 1 -- 172.21.15.151:0/1682963915 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1e6805a050 con 0x563189d8f140 2024-04-26T01:55:12.400 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.398+0000 7f1e84f84740 10 client.25378 fetch_fsmap learned FSMap version 26 2024-04-26T01:55:12.400 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.398+0000 7f1e84f84740 1 -- 172.21.15.151:0/1682963915 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563189d9c5d0 con 0x563189d8f140 2024-04-26T01:55:12.400 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.399+0000 7f1e64c00640 1 -- 172.21.15.151:0/1682963915 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e68005020 con 0x563189d8f140 2024-04-26T01:55:12.400 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.399+0000 7f1e64c00640 1 -- 172.21.15.151:0/1682963915 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f1e680314a0 con 0x563189d8f140 2024-04-26T01:55:12.400 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.399+0000 7f1e84f84740 10 client.25378 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T01:55:12.400 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.399+0000 7f1e84f84740 10 client.25378 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:55:12.400 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.399+0000 7f1e84f84740 20 client.25378 populate_metadata read hostname 'smithi151' 2024-04-26T01:55:12.400 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.399+0000 7f1e84f84740 1 --2- 172.21.15.151:0/1682963915 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x563189da9a40 0x563189dabe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:12.401 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.399+0000 7f1e84f84740 4 client.25378 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:55:12.401 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.399+0000 7f1e84f84740 1 -- 172.21.15.151:0/1682963915 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563189d088b0 con 0x563189da9a40 2024-04-26T01:55:12.401 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.399+0000 7f1e7b400640 1 --2- 172.21.15.151:0/1682963915 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x563189da9a40 0x563189dabe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:12.401 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.400+0000 7f1e7b400640 1 --2- 172.21.15.151:0/1682963915 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x563189da9a40 0x563189dabe30 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0e44eccf655777d server_cookie=c14f7b540d18f90f in_seq=0 out_seq=0 2024-04-26T01:55:12.401 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.400+0000 7f1e64c00640 10 client.25378 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:55:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.401+0000 7f1e64c00640 1 -- 172.21.15.151:0/1682963915 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563189d088b0 con 0x563189da9a40 2024-04-26T01:55:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.401+0000 7f1e64c00640 10 client.25378 handle_command_reply: tid=0 2024-04-26T01:55:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.401+0000 7f1e84f84740 1 client.25378 shutdown 2024-04-26T01:55:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.401+0000 7f1e84f84740 1 -- 172.21.15.151:0/1682963915 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563189d8f140 msgr2=0x563189d96f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.401+0000 7f1e84f84740 1 --2- 172.21.15.151:0/1682963915 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563189d8f140 0x563189d96f10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1e68045000 tx=0x7f1e68005480 comp rx=0 tx=0).stop 2024-04-26T01:55:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.401+0000 7f1e84f84740 1 -- 172.21.15.151:0/1682963915 shutdown_connections 2024-04-26T01:55:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.401+0000 7f1e84f84740 1 --2- 172.21.15.151:0/1682963915 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x563189da9a40 0x563189dabe30 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.401+0000 7f1e84f84740 1 --2- 172.21.15.151:0/1682963915 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563189d99a00 0x563189d97450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.401+0000 7f1e84f84740 1 --2- 172.21.15.151:0/1682963915 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563189d8f140 0x563189d96f10 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.401+0000 7f1e64c00640 0 client.25378 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:55:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.401+0000 7f1e84f84740 1 --2- 172.21.15.151:0/1682963915 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563189d8e820 0x563189d969d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.401+0000 7f1e84f84740 1 -- 172.21.15.151:0/1682963915 >> 172.21.15.151:0/1682963915 conn(0x563189d8ad30 msgr2=0x563189d90ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.401+0000 7f1e84f84740 1 -- 172.21.15.151:0/1682963915 shutdown_connections 2024-04-26T01:55:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.401+0000 7f1e84f84740 1 -- 172.21.15.151:0/1682963915 wait complete. 2024-04-26T01:55:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.401+0000 7f1e84f84740 20 client.25378 trim_cache size 0 max 16384 2024-04-26T01:55:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.418+0000 7f1e84f84740 1 Processor -- start 2024-04-26T01:55:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.418+0000 7f1e84f84740 1 -- start start 2024-04-26T01:55:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.419+0000 7f1e84f84740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563189d8e820 0x563189d8a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.419+0000 7f1e84f84740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563189d8f140 0x563189d97270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.419+0000 7f1e84f84740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563189d99a00 0x563189db1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.419+0000 7f1e84f84740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1e7c1b2400 con 0x563189d8e820 2024-04-26T01:55:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.419+0000 7f1e84f84740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1e7c1afcd0 con 0x563189d99a00 2024-04-26T01:55:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.419+0000 7f1e84f84740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1e7c1adb40 con 0x563189d8f140 2024-04-26T01:55:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.419+0000 7f1e7b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563189d8e820 0x563189d8a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.419+0000 7f1e7b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563189d8e820 0x563189d8a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:48360/0 (socket says 172.21.15.151:48360) 2024-04-26T01:55:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.419+0000 7f1e7b400640 1 -- 172.21.15.151:0/2081713854 learned_addr learned my addr 172.21.15.151:0/2081713854 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:55:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.419+0000 7f1e7aa00640 1 --2- 172.21.15.151:0/2081713854 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563189d8f140 0x563189d97270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.419+0000 7f1e7be00640 1 --2- 172.21.15.151:0/2081713854 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563189d99a00 0x563189db1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.419+0000 7f1e7aa00640 1 -- 172.21.15.151:0/2081713854 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563189d99a00 msgr2=0x563189db1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.420+0000 7f1e7aa00640 1 --2- 172.21.15.151:0/2081713854 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563189d99a00 0x563189db1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.420+0000 7f1e7aa00640 1 -- 172.21.15.151:0/2081713854 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563189d8e820 msgr2=0x563189d8a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.420+0000 7f1e7aa00640 1 --2- 172.21.15.151:0/2081713854 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563189d8e820 0x563189d8a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.420+0000 7f1e7aa00640 1 -- 172.21.15.151:0/2081713854 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563189d917d0 con 0x563189d8f140 2024-04-26T01:55:12.420 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.420+0000 7f1e7b400640 1 --2- 172.21.15.151:0/2081713854 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563189d8e820 0x563189d8a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:55:12.420 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.420+0000 7f1e7aa00640 1 --2- 172.21.15.151:0/2081713854 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563189d8f140 0x563189d97270 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1e68006010 tx=0x7f1e68004190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0172c9a02891874 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.420+0000 7f1e67400640 1 -- 172.21.15.151:0/2081713854 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e6804e070 con 0x563189d8f140 2024-04-26T01:55:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.420+0000 7f1e67400640 1 -- 172.21.15.151:0/2081713854 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e68041070 con 0x563189d8f140 2024-04-26T01:55:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.420+0000 7f1e67400640 1 -- 172.21.15.151:0/2081713854 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e68035070 con 0x563189d8f140 2024-04-26T01:55:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.421+0000 7f1e84f84740 1 -- 172.21.15.151:0/2081713854 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563189d8f140 msgr2=0x563189d97270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.421+0000 7f1e84f84740 1 --2- 172.21.15.151:0/2081713854 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563189d8f140 0x563189d97270 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1e68006010 tx=0x7f1e68004190 comp rx=0 tx=0).stop 2024-04-26T01:55:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.421+0000 7f1e84f84740 1 -- 172.21.15.151:0/2081713854 shutdown_connections 2024-04-26T01:55:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.421+0000 7f1e84f84740 1 --2- 172.21.15.151:0/2081713854 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563189d99a00 0x563189db1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.421+0000 7f1e84f84740 1 --2- 172.21.15.151:0/2081713854 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563189d8f140 0x563189d97270 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.421+0000 7f1e84f84740 1 --2- 172.21.15.151:0/2081713854 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563189d8e820 0x563189d8a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.421+0000 7f1e84f84740 1 -- 172.21.15.151:0/2081713854 >> 172.21.15.151:0/2081713854 conn(0x563189d8ad30 msgr2=0x563189d90ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.421+0000 7f1e84f84740 1 -- 172.21.15.151:0/2081713854 shutdown_connections 2024-04-26T01:55:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.421+0000 7f1e84f84740 1 -- 172.21.15.151:0/2081713854 wait complete. 2024-04-26T01:55:12.424 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.421+0000 7f1e84f84740 1 -- start start 2024-04-26T01:55:12.424 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.422+0000 7f1e84f84740 1 Processor -- start 2024-04-26T01:55:12.424 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.422+0000 7f1e84f84740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563189d8e820 0x563189da19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:12.424 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.422+0000 7f1e84f84740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563189d8f140 0x563189da1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:12.424 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.422+0000 7f1e84f84740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563189d99a00 0x563189d9ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:12.424 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.422+0000 7f1e84f84740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1e7c1ae190 con 0x563189d8e820 2024-04-26T01:55:12.424 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.422+0000 7f1e84f84740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1e7c1af600 con 0x563189d99a00 2024-04-26T01:55:12.424 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.422+0000 7f1e84f84740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1e7c1ae860 con 0x563189d8f140 2024-04-26T01:55:12.424 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.422+0000 7f1e84f84740 10 client.0 _ll_register_callbacks cb 0x7f1e68003810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:55:12.424 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.422+0000 7f1e7aa00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563189d8f140 0x563189da1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:12.424 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.422+0000 7f1e7b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563189d8e820 0x563189da19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:12.424 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.422+0000 7f1e7aa00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563189d8f140 0x563189da1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:58670/0 (socket says 172.21.15.151:58670) 2024-04-26T01:55:12.424 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.422+0000 7f1e7aa00640 1 -- 172.21.15.151:0/3846271747 learned_addr learned my addr 172.21.15.151:0/3846271747 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:55:12.424 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.423+0000 7f1e7be00640 1 --2- 172.21.15.151:0/3846271747 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563189d99a00 0x563189d9ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:12.425 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.423+0000 7f1e7b400640 1 -- 172.21.15.151:0/3846271747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563189d8f140 msgr2=0x563189da1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:12.425 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.423+0000 7f1e7b400640 1 --2- 172.21.15.151:0/3846271747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563189d8f140 0x563189da1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:12.425 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.423+0000 7f1e7b400640 1 -- 172.21.15.151:0/3846271747 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563189d99a00 msgr2=0x563189d9ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:12.425 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.423+0000 7f1e7b400640 1 --2- 172.21.15.151:0/3846271747 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563189d99a00 0x563189d9ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:12.425 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.423+0000 7f1e7b400640 1 -- 172.21.15.151:0/3846271747 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563189d9f740 con 0x563189d8e820 2024-04-26T01:55:12.425 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.423+0000 7f1e7b400640 1 --2- 172.21.15.151:0/3846271747 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563189d8e820 0x563189da19d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c012cb0 tx=0x7f1e6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2abd5e8f42482c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:12.425 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.423+0000 7f1e64c00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-04-26T01:55:12.425 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.423+0000 7f1e64c00640 1 -- 172.21.15.151:0/3846271747 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c0042d0 con 0x563189d8e820 2024-04-26T01:55:12.425 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.424+0000 7f1e84f84740 1 -- 172.21.15.151:0/3846271747 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e68041070 con 0x563189d8e820 2024-04-26T01:55:12.425 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.424+0000 7f1e64c00640 1 -- 172.21.15.151:0/3846271747 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f1e6c004470 con 0x563189d8e820 2024-04-26T01:55:12.425 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.424+0000 7f1e64c00640 1 -- 172.21.15.151:0/3846271747 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1e6c02b380 con 0x563189d8e820 2024-04-26T01:55:12.425 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.424+0000 7f1e84f84740 10 client.15264 fetch_fsmap learned FSMap version 26 2024-04-26T01:55:12.425 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.424+0000 7f1e84f84740 1 -- 172.21.15.151:0/3846271747 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563189e0fa10 con 0x563189d8e820 2024-04-26T01:55:12.425 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.424+0000 7f1e64c00640 1 -- 172.21.15.151:0/3846271747 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e68041070 con 0x563189d8e820 2024-04-26T01:55:12.425 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.424+0000 7f1e64c00640 1 -- 172.21.15.151:0/3846271747 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f1e6c03b050 con 0x563189d8e820 2024-04-26T01:55:12.426 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.424+0000 7f1e84f84740 10 client.15264 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T01:55:12.426 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.424+0000 7f1e84f84740 10 client.15264 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:55:12.426 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.424+0000 7f1e84f84740 20 client.15264 populate_metadata read hostname 'smithi151' 2024-04-26T01:55:12.426 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.424+0000 7f1e84f84740 1 --2- 172.21.15.151:0/3846271747 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x563189e10170 0x563189e12590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:12.426 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.425+0000 7f1e84f84740 4 client.15264 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:55:12.426 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.425+0000 7f1e84f84740 1 -- 172.21.15.151:0/3846271747 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563189d02320 con 0x563189e10170 2024-04-26T01:55:12.426 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.425+0000 7f1e7aa00640 1 --2- 172.21.15.151:0/3846271747 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x563189e10170 0x563189e12590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:12.427 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.425+0000 7f1e7aa00640 1 --2- 172.21.15.151:0/3846271747 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x563189e10170 0x563189e12590 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=690ec86eeed87118 server_cookie=21ef5ba0cb144400 in_seq=0 out_seq=0 2024-04-26T01:55:12.427 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.425+0000 7f1e64c00640 10 client.15264 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:55:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.433+0000 7f1e64c00640 1 -- 172.21.15.151:0/3846271747 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563189d02320 con 0x563189e10170 2024-04-26T01:55:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.433+0000 7f1e64c00640 10 client.15264 handle_command_reply: tid=0 2024-04-26T01:55:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.433+0000 7f1e84f84740 1 client.15264 shutdown 2024-04-26T01:55:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.433+0000 7f1e84f84740 1 -- 172.21.15.151:0/3846271747 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563189d8e820 msgr2=0x563189da19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.433+0000 7f1e84f84740 1 --2- 172.21.15.151:0/3846271747 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563189d8e820 0x563189da19d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c012cb0 tx=0x7f1e6c0046c0 comp rx=0 tx=0).stop 2024-04-26T01:55:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.433+0000 7f1e84f84740 1 -- 172.21.15.151:0/3846271747 shutdown_connections 2024-04-26T01:55:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.433+0000 7f1e84f84740 1 --2- 172.21.15.151:0/3846271747 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x563189e10170 0x563189e12590 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.433+0000 7f1e84f84740 1 --2- 172.21.15.151:0/3846271747 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563189d99a00 0x563189d9ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.433+0000 7f1e84f84740 1 --2- 172.21.15.151:0/3846271747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563189d8f140 0x563189da1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.433+0000 7f1e84f84740 1 --2- 172.21.15.151:0/3846271747 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563189d8e820 0x563189da19d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.433+0000 7f1e84f84740 1 -- 172.21.15.151:0/3846271747 >> 172.21.15.151:0/3846271747 conn(0x563189d8ad30 msgr2=0x563189d98200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.433+0000 7f1e64c00640 0 client.15264 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:55:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.433+0000 7f1e84f84740 1 -- 172.21.15.151:0/3846271747 shutdown_connections 2024-04-26T01:55:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.433+0000 7f1e84f84740 1 -- 172.21.15.151:0/3846271747 wait complete. 2024-04-26T01:55:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.433+0000 7f1e84f84740 20 client.15264 trim_cache size 0 max 16384 2024-04-26T01:55:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:{ 2024-04-26T01:55:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "message": "", 2024-04-26T01:55:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "return_code": 0 2024-04-26T01:55:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:} 2024-04-26T01:55:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.434+0000 7f1e82200640 1 -- 172.21.15.151:0/2917839175 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f1e600a3dc0 msgr2=0x7f1e600a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.434+0000 7f1e82200640 1 --2- 172.21.15.151:0/2917839175 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f1e600a3dc0 0x7f1e600a6280 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x563189d8fdc0 tx=0x7f1e6c005f90 comp rx=0 tx=0).stop 2024-04-26T01:55:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.434+0000 7f1e82200640 1 -- 172.21.15.151:0/2917839175 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e7c10e840 msgr2=0x7f1e7c19a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.434+0000 7f1e82200640 1 --2- 172.21.15.151:0/2917839175 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e7c10e840 0x7f1e7c19a6a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1e74005fd0 tx=0x7f1e7400ee60 comp rx=0 tx=0).stop 2024-04-26T01:55:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.434+0000 7f1e82200640 1 -- 172.21.15.151:0/2917839175 shutdown_connections 2024-04-26T01:55:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.434+0000 7f1e82200640 1 --2- 172.21.15.151:0/2917839175 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f1e600a3dc0 0x7f1e600a6280 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.434+0000 7f1e82200640 1 --2- 172.21.15.151:0/2917839175 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e7c10e840 0x7f1e7c19a6a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:12.435 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.434+0000 7f1e82200640 1 --2- 172.21.15.151:0/2917839175 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1e7c074b70 0x7f1e7c1a0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:12.435 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.434+0000 7f1e82200640 1 --2- 172.21.15.151:0/2917839175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e7c074250 0x7f1e7c1a0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:12.435 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.434+0000 7f1e82200640 1 -- 172.21.15.151:0/2917839175 >> 172.21.15.151:0/2917839175 conn(0x7f1e7c0793e0 msgr2=0x7f1e7c0fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:12.435 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.434+0000 7f1e82200640 1 -- 172.21.15.151:0/2917839175 shutdown_connections 2024-04-26T01:55:12.435 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:12.434+0000 7f1e82200640 1 -- 172.21.15.151:0/2917839175 wait complete. 2024-04-26T01:55:12.680 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:55:12 smithi151 ceph-mon[28057]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-04-26T01:55:12.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:55:12 smithi172 ceph-mon[32785]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-04-26T01:55:12.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:55:12 smithi165 ceph-mon[32648]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-04-26T01:55:12.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:12.998+0000 7f0d64800700 1 -- 172.21.15.151:0/3186202001 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d5c109fa0 msgr2=0x7f0d5c111b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:12.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:12.998+0000 7f0d64800700 1 --2- 172.21.15.151:0/3186202001 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d5c109fa0 0x7f0d5c111b70 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0d5800b0d0 tx=0x7f0d5800b3e0 comp rx=0 tx=0).stop 2024-04-26T01:55:12.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:12.998+0000 7f0d64800700 1 -- 172.21.15.151:0/3186202001 shutdown_connections 2024-04-26T01:55:12.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:12.998+0000 7f0d64800700 1 --2- 172.21.15.151:0/3186202001 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d5c109fa0 0x7f0d5c111b70 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:12.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:12.998+0000 7f0d64800700 1 --2- 172.21.15.151:0/3186202001 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d5c104ec0 0x7f0d5c1052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:12.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:12.998+0000 7f0d64800700 1 --2- 172.21.15.151:0/3186202001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d5c105870 0x7f0d5c1098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:12.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:12.998+0000 7f0d64800700 1 -- 172.21.15.151:0/3186202001 >> 172.21.15.151:0/3186202001 conn(0x7f0d5c100750 msgr2=0x7f0d5c102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:12.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:12.998+0000 7f0d64800700 1 -- 172.21.15.151:0/3186202001 shutdown_connections 2024-04-26T01:55:12.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:12.998+0000 7f0d64800700 1 -- 172.21.15.151:0/3186202001 wait complete. 2024-04-26T01:55:12.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:13.000+0000 7f0d64800700 1 Processor -- start 2024-04-26T01:55:12.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:13.000+0000 7f0d64800700 1 -- start start 2024-04-26T01:55:13.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:13.000+0000 7f0d64800700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d5c104ec0 0x7f0d5c19b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:13.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:13.000+0000 7f0d64800700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d5c105870 0x7f0d5c19b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:13.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:13.000+0000 7f0d64800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d5c109fa0 0x7f0d5c19fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:13.001 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:13.000+0000 7f0d64800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5c114e60 con 0x7f0d5c109fa0 2024-04-26T01:55:13.001 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:13.000+0000 7f0d64800700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5c114ce0 con 0x7f0d5c104ec0 2024-04-26T01:55:13.001 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:13.000+0000 7f0d64800700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5c114fe0 con 0x7f0d5c105870 2024-04-26T01:55:13.001 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:13.001+0000 7f0d62800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d5c109fa0 0x7f0d5c19fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:13.001 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:13.001+0000 7f0d61400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d5c105870 0x7f0d5c19b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:13.001 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:13.001+0000 7f0d62800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d5c109fa0 0x7f0d5c19fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:48376/0 (socket says 172.21.15.151:48376) 2024-04-26T01:55:13.001 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:13.001+0000 7f0d61e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d5c104ec0 0x7f0d5c19b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:13.001 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:13.001+0000 7f0d62800700 1 -- 172.21.15.151:0/3171405394 learned_addr learned my addr 172.21.15.151:0/3171405394 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:55:13.001 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:13.001+0000 7f0d61400700 1 -- 172.21.15.151:0/3171405394 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d5c104ec0 msgr2=0x7f0d5c19b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:13.001 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:13.001+0000 7f0d61400700 1 --2- 172.21.15.151:0/3171405394 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d5c104ec0 0x7f0d5c19b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:13.001 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:13.001+0000 7f0d61400700 1 -- 172.21.15.151:0/3171405394 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d5c109fa0 msgr2=0x7f0d5c19fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:13.001 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:13.001+0000 7f0d61400700 1 --2- 172.21.15.151:0/3171405394 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d5c109fa0 0x7f0d5c19fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:13.001 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:13.001+0000 7f0d61400700 1 -- 172.21.15.151:0/3171405394 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d5c1a0340 con 0x7f0d5c105870 2024-04-26T01:55:13.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:13.001+0000 7f0d62800700 1 --2- 172.21.15.151:0/3171405394 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d5c109fa0 0x7f0d5c19fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:55:13.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:13.002+0000 7f0d61400700 1 --2- 172.21.15.151:0/3171405394 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d5c105870 0x7f0d5c19b870 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c00e9c0 tx=0x7f0d4c00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:13.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:13.002+0000 7f0d56a00700 1 -- 172.21.15.151:0/3171405394 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c004220 con 0x7f0d5c105870 2024-04-26T01:55:13.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:13.002+0000 7f0d64800700 1 -- 172.21.15.151:0/3171405394 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d5c1a0630 con 0x7f0d5c105870 2024-04-26T01:55:13.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:13.002+0000 7f0d56a00700 1 -- 172.21.15.151:0/3171405394 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d4c0043c0 con 0x7f0d5c105870 2024-04-26T01:55:13.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:13.002+0000 7f0d56a00700 1 -- 172.21.15.151:0/3171405394 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c005020 con 0x7f0d5c105870 2024-04-26T01:55:13.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:13.002+0000 7f0d64800700 1 -- 172.21.15.151:0/3171405394 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d5c1a7eb0 con 0x7f0d5c105870 2024-04-26T01:55:13.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:13.003+0000 7f0d64800700 1 -- 172.21.15.151:0/3171405394 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d5c069730 con 0x7f0d5c105870 2024-04-26T01:55:13.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:13.005+0000 7f0d56a00700 1 -- 172.21.15.151:0/3171405394 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f0d4c004750 con 0x7f0d5c105870 2024-04-26T01:55:13.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:13.006+0000 7f0d56a00700 1 --2- 172.21.15.151:0/3171405394 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f0d440a3e30 0x7f0d440a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:13.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:13.007+0000 7f0d56a00700 1 -- 172.21.15.151:0/3171405394 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f0d4c0d0770 con 0x7f0d5c105870 2024-04-26T01:55:13.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:13.007+0000 7f0d61e00700 1 --2- 172.21.15.151:0/3171405394 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f0d440a3e30 0x7f0d440a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:13.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:13.008+0000 7f0d61e00700 1 --2- 172.21.15.151:0/3171405394 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f0d440a3e30 0x7f0d440a62f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0d50009680 tx=0x7f0d50009260 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:13.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:13.009+0000 7f0d56a00700 1 -- 172.21.15.151:0/3171405394 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0d4c09ce20 con 0x7f0d5c105870 2024-04-26T01:55:13.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:13.284+0000 7f0d64800700 1 -- 172.21.15.151:0/3171405394 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0d5c19c690 con 0x7f0d5c105870 2024-04-26T01:55:13.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:13.285+0000 7f0d56a00700 1 -- 172.21.15.151:0/3171405394 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11889 (secure 0 0 0) 0x7f0d4c09c720 con 0x7f0d5c105870 2024-04-26T01:55:13.285 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:55:13.285 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2937864097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2937864097},{"type":"v1","addr":"172.21.15.151:6835","nonce":2937864097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14373,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/2947090802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2947090802},{"type":"v1","addr":"172.21.15.151:6837","nonce":2947090802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/1356563280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1356563280},{"type":"v1","addr":"172.21.15.151:6839","nonce":1356563280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6841/1075240552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1075240552},{"type":"v1","addr":"172.21.15.151:6841","nonce":1075240552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/504675912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":504675912},{"type":"v1","addr":"172.21.15.165:6833","nonce":504675912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1207580015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1207580015},{"type":"v1","addr":"172.21.15.172:6833","nonce":1207580015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:49:40.746876+0000","modified":"2024-04-26T01:53:36.483847+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24505,"mds_2":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/3598138294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3598138294},{"type":"v1","addr":"172.21.15.165:6835","nonce":3598138294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/1847859054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":1847859054},{"type":"v1","addr":"172.21.15.165:6837","nonce":1847859054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/2309751171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2309751171},{"type":"v1","addr":"172.21.15.172:6835","nonce":2309751171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.172:6837/55972537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":55972537},{"type":"v1","addr":"172.21.15.172:6837","nonce":55972537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.172:6839/1242302810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1242302810},{"type":"v1","addr":"172.21.15.172:6839","nonce":1242302810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/1969716254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1969716254},{"type":"v1","addr":"172.21.15.165:6839","nonce":1969716254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24493,24505,24509]},"id":1}]} 2024-04-26T01:55:13.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:13.289+0000 7f0d64800700 1 -- 172.21.15.151:0/3171405394 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f0d440a3e30 msgr2=0x7f0d440a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:13.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:13.289+0000 7f0d64800700 1 --2- 172.21.15.151:0/3171405394 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f0d440a3e30 0x7f0d440a62f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0d50009680 tx=0x7f0d50009260 comp rx=0 tx=0).stop 2024-04-26T01:55:13.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:13.289+0000 7f0d64800700 1 -- 172.21.15.151:0/3171405394 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d5c105870 msgr2=0x7f0d5c19b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:13.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:13.289+0000 7f0d64800700 1 --2- 172.21.15.151:0/3171405394 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d5c105870 0x7f0d5c19b870 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c00e9c0 tx=0x7f0d4c00ecd0 comp rx=0 tx=0).stop 2024-04-26T01:55:13.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:13.289+0000 7f0d64800700 1 -- 172.21.15.151:0/3171405394 shutdown_connections 2024-04-26T01:55:13.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:13.289+0000 7f0d64800700 1 --2- 172.21.15.151:0/3171405394 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f0d440a3e30 0x7f0d440a62f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:13.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:13.289+0000 7f0d64800700 1 --2- 172.21.15.151:0/3171405394 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d5c109fa0 0x7f0d5c19fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:13.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:13.289+0000 7f0d64800700 1 --2- 172.21.15.151:0/3171405394 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d5c104ec0 0x7f0d5c19b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:13.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:13.289+0000 7f0d64800700 1 --2- 172.21.15.151:0/3171405394 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d5c105870 0x7f0d5c19b870 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:13.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:13.289+0000 7f0d64800700 1 -- 172.21.15.151:0/3171405394 >> 172.21.15.151:0/3171405394 conn(0x7f0d5c100750 msgr2=0x7f0d5c1012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:13.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:13.289+0000 7f0d64800700 1 -- 172.21.15.151:0/3171405394 shutdown_connections 2024-04-26T01:55:13.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:13.289+0000 7f0d64800700 1 -- 172.21.15.151:0/3171405394 wait complete. 2024-04-26T01:55:13.289 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 26 2024-04-26T01:55:13.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:55:13 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/3171405394' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:55:13.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:55:13 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/3171405394' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:55:13.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:55:13 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/3171405394' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:55:14.082 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24509 quiesce db --set-id 7b96ec7c --release --await' 2024-04-26T01:55:14.082 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24509 quiesce db --set-id 7b96ec7c --release --await 2024-04-26T01:55:14.510 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:55:14.694 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:55:14 smithi151 ceph-mon[28057]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-04-26T01:55:14.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:55:14 smithi172 ceph-mon[32785]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-04-26T01:55:14.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:55:14 smithi165 ceph-mon[32648]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-04-26T01:55:15.706 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:55:15 smithi151 ceph-mon[28057]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-04-26T01:55:15.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:55:15 smithi172 ceph-mon[32785]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-04-26T01:55:15.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:55:15 smithi165 ceph-mon[32648]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-04-26T01:55:16.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.675+0000 7fdfd8000700 1 -- 172.21.15.151:0/2511870510 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdfd01092d0 msgr2=0x7fdfd01096b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:16.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.675+0000 7fdfd8000700 1 --2- 172.21.15.151:0/2511870510 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdfd01092d0 0x7fdfd01096b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdfc40099c0 tx=0x7fdfc4009cd0 comp rx=0 tx=0).stop 2024-04-26T01:55:16.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.675+0000 7fdfd8000700 1 -- 172.21.15.151:0/2511870510 shutdown_connections 2024-04-26T01:55:16.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.675+0000 7fdfd8000700 1 --2- 172.21.15.151:0/2511870510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdfd010e3b0 0x7fdfd0115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:16.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.675+0000 7fdfd8000700 1 --2- 172.21.15.151:0/2511870510 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdfd01092d0 0x7fdfd01096b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:16.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.675+0000 7fdfd8000700 1 --2- 172.21.15.151:0/2511870510 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdfd0109c80 0x7fdfd010dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:16.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.675+0000 7fdfd8000700 1 -- 172.21.15.151:0/2511870510 >> 172.21.15.151:0/2511870510 conn(0x7fdfd0076210 msgr2=0x7fdfd0107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:16.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.675+0000 7fdfd8000700 1 -- 172.21.15.151:0/2511870510 shutdown_connections 2024-04-26T01:55:16.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.675+0000 7fdfd8000700 1 -- 172.21.15.151:0/2511870510 wait complete. 2024-04-26T01:55:16.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.676+0000 7fdfd8000700 1 Processor -- start 2024-04-26T01:55:16.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.676+0000 7fdfd8000700 1 -- start start 2024-04-26T01:55:16.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.677+0000 7fdfd8000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdfd01092d0 0x7fdfd019f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:16.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.677+0000 7fdfd8000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdfd0109c80 0x7fdfd019fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:16.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.677+0000 7fdfd8000700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdfd010e3b0 0x7fdfd01a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:16.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.677+0000 7fdfd8000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd0119270 con 0x7fdfd01092d0 2024-04-26T01:55:16.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.677+0000 7fdfd8000700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd01190f0 con 0x7fdfd0109c80 2024-04-26T01:55:16.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.677+0000 7fdfd8000700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd01193f0 con 0x7fdfd010e3b0 2024-04-26T01:55:16.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.677+0000 7fdfd5600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdfd01092d0 0x7fdfd019f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:16.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.677+0000 7fdfd5600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdfd01092d0 0x7fdfd019f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:48398/0 (socket says 172.21.15.151:48398) 2024-04-26T01:55:16.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.677+0000 7fdfd5600700 1 -- 172.21.15.151:0/1570916889 learned_addr learned my addr 172.21.15.151:0/1570916889 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:55:16.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.677+0000 7fdfd4c00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdfd0109c80 0x7fdfd019fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:16.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.677+0000 7fdfd6000700 1 --2- 172.21.15.151:0/1570916889 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdfd010e3b0 0x7fdfd01a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:16.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.678+0000 7fdfd5600700 1 -- 172.21.15.151:0/1570916889 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdfd010e3b0 msgr2=0x7fdfd01a4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:16.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.678+0000 7fdfd5600700 1 --2- 172.21.15.151:0/1570916889 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdfd010e3b0 0x7fdfd01a4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:16.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.678+0000 7fdfd5600700 1 -- 172.21.15.151:0/1570916889 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdfd0109c80 msgr2=0x7fdfd019fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:16.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.678+0000 7fdfd5600700 1 --2- 172.21.15.151:0/1570916889 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdfd0109c80 0x7fdfd019fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:16.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.678+0000 7fdfd5600700 1 -- 172.21.15.151:0/1570916889 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfd01a4750 con 0x7fdfd01092d0 2024-04-26T01:55:16.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.678+0000 7fdfd4c00700 1 --2- 172.21.15.151:0/1570916889 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdfd0109c80 0x7fdfd019fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:55:16.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.678+0000 7fdfd5600700 1 --2- 172.21.15.151:0/1570916889 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdfd01092d0 0x7fdfd019f740 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4006010 tx=0x7fdfc4005280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:16.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.678+0000 7fdfce000700 1 -- 172.21.15.151:0/1570916889 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc4026070 con 0x7fdfd01092d0 2024-04-26T01:55:16.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.678+0000 7fdfd8000700 1 -- 172.21.15.151:0/1570916889 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfd01a49e0 con 0x7fdfd01092d0 2024-04-26T01:55:16.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.678+0000 7fdfce000700 1 -- 172.21.15.151:0/1570916889 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfc4005680 con 0x7fdfd01092d0 2024-04-26T01:55:16.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.678+0000 7fdfce000700 1 -- 172.21.15.151:0/1570916889 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc400fb20 con 0x7fdfd01092d0 2024-04-26T01:55:16.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.679+0000 7fdfd8000700 1 -- 172.21.15.151:0/1570916889 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfd01ac2c0 con 0x7fdfd01092d0 2024-04-26T01:55:16.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.682+0000 7fdfce000700 1 -- 172.21.15.151:0/1570916889 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7fdfc4029070 con 0x7fdfd01092d0 2024-04-26T01:55:16.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.683+0000 7fdfce000700 1 --2- 172.21.15.151:0/1570916889 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fdfb80a3e30 0x7fdfb80a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:16.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.683+0000 7fdfe298fb80 1 Processor -- start 2024-04-26T01:55:16.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.683+0000 7fdfce000700 1 -- 172.21.15.151:0/1570916889 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fdfc40d0eb0 con 0x7fdfd01092d0 2024-04-26T01:55:16.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.683+0000 7fdfd4c00700 1 --2- 172.21.15.151:0/1570916889 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fdfb80a3e30 0x7fdfb80a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:16.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.683+0000 7fdfe298fb80 1 -- start start 2024-04-26T01:55:16.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.684+0000 7fdfe298fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564399d72d40 0x564399d73120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:16.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.684+0000 7fdfe298fb80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564399d73660 0x564399d7df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:16.697 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.684+0000 7fdfe298fb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564399d7e4d0 0x564399d808c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:16.697 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.684+0000 7fdfe298fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564399cc5f90 con 0x564399d72d40 2024-04-26T01:55:16.697 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.684+0000 7fdfe298fb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564399c6c830 con 0x564399d7e4d0 2024-04-26T01:55:16.697 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.684+0000 7fdfe298fb80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564399cc1f50 con 0x564399d73660 2024-04-26T01:55:16.697 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.684+0000 7fdfd5600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564399d72d40 0x564399d73120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:16.697 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.684+0000 7fdfd5600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564399d72d40 0x564399d73120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:48414/0 (socket says 172.21.15.151:48414) 2024-04-26T01:55:16.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.684+0000 7fdfd5600700 1 -- 172.21.15.151:0/1292078239 learned_addr learned my addr 172.21.15.151:0/1292078239 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:55:16.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.684+0000 7fdfd4c00700 1 --2- 172.21.15.151:0/1292078239 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564399d73660 0x564399d7df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:16.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.685+0000 7fdfd4c00700 1 --2- 172.21.15.151:0/1570916889 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fdfb80a3e30 0x7fdfb80a62f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdfd0078240 tx=0x7fdfbc009260 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:16.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.685+0000 7fdfd6000700 1 --2- 172.21.15.151:0/1292078239 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564399d7e4d0 0x564399d808c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:16.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.685+0000 7fdfd5600700 1 -- 172.21.15.151:0/1292078239 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564399d73660 msgr2=0x564399d7df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:16.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.685+0000 7fdfd5600700 1 --2- 172.21.15.151:0/1292078239 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564399d73660 0x564399d7df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:16.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.685+0000 7fdfd5600700 1 -- 172.21.15.151:0/1292078239 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564399d7e4d0 msgr2=0x564399d808c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:16.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.685+0000 7fdfd5600700 1 --2- 172.21.15.151:0/1292078239 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564399d7e4d0 0x564399d808c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:16.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.685+0000 7fdfd5600700 1 -- 172.21.15.151:0/1292078239 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564399c62780 con 0x564399d72d40 2024-04-26T01:55:16.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.685+0000 7fdfd4c00700 1 --2- 172.21.15.151:0/1292078239 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564399d73660 0x564399d7df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:55:16.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.685+0000 7fdfd5600700 1 --2- 172.21.15.151:0/1292078239 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564399d72d40 0x564399d73120 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdfc40a55a0 tx=0x7fdfc409c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cae5731d103e7dfe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:16.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.685+0000 7fdfc2a00700 1 -- 172.21.15.151:0/1292078239 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc409cef0 con 0x564399d72d40 2024-04-26T01:55:16.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.685+0000 7fdfc2a00700 1 -- 172.21.15.151:0/1292078239 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfc40a12a0 con 0x564399d72d40 2024-04-26T01:55:16.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.685+0000 7fdfc2a00700 1 -- 172.21.15.151:0/1292078239 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc40a5640 con 0x564399d72d40 2024-04-26T01:55:16.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.686+0000 7fdfe298fb80 1 -- 172.21.15.151:0/1292078239 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564399d72d40 msgr2=0x564399d73120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:16.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.686+0000 7fdfe298fb80 1 --2- 172.21.15.151:0/1292078239 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564399d72d40 0x564399d73120 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdfc40a55a0 tx=0x7fdfc409c980 comp rx=0 tx=0).stop 2024-04-26T01:55:16.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.686+0000 7fdfe298fb80 1 -- 172.21.15.151:0/1292078239 shutdown_connections 2024-04-26T01:55:16.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.686+0000 7fdfe298fb80 1 --2- 172.21.15.151:0/1292078239 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564399d7e4d0 0x564399d808c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:16.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.686+0000 7fdfe298fb80 1 --2- 172.21.15.151:0/1292078239 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564399d72d40 0x564399d73120 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:16.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.686+0000 7fdfe298fb80 1 --2- 172.21.15.151:0/1292078239 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564399d73660 0x564399d7df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:16.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.686+0000 7fdfe298fb80 1 -- 172.21.15.151:0/1292078239 >> 172.21.15.151:0/1292078239 conn(0x564399ca2050 msgr2=0x564399d6f250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:16.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.686+0000 7fdfe298fb80 1 -- 172.21.15.151:0/1292078239 shutdown_connections 2024-04-26T01:55:16.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.687+0000 7fdfe298fb80 1 -- 172.21.15.151:0/1292078239 wait complete. 2024-04-26T01:55:16.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.687+0000 7fdfe298fb80 1 -- start start 2024-04-26T01:55:16.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.687+0000 7fdfe298fb80 1 Processor -- start 2024-04-26T01:55:16.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.688+0000 7fdfe298fb80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564399d72d40 0x564399d7dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:16.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.688+0000 7fdfe298fb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564399d73660 0x564399d7b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:16.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.688+0000 7fdfe298fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564399d7e4d0 0x564399d7b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:16.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.688+0000 7fdfe298fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564399cc5f90 con 0x564399d7e4d0 2024-04-26T01:55:16.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.688+0000 7fdfe298fb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564399c7f960 con 0x564399d73660 2024-04-26T01:55:16.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.688+0000 7fdfe298fb80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564399d80e00 con 0x564399d72d40 2024-04-26T01:55:16.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.688+0000 7fdfe298fb80 10 client.0 _ll_register_callbacks cb 0x564399c5e820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:55:16.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.688+0000 7fdfd6000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564399d7e4d0 0x564399d7b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:16.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.688+0000 7fdfd5600700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564399d72d40 0x564399d7dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:16.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.688+0000 7fdfd6000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564399d7e4d0 0x564399d7b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:48426/0 (socket says 172.21.15.151:48426) 2024-04-26T01:55:16.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.688+0000 7fdfd6000700 1 -- 172.21.15.151:0/1268253139 learned_addr learned my addr 172.21.15.151:0/1268253139 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:55:16.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.688+0000 7fdfd4c00700 1 --2- 172.21.15.151:0/1268253139 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564399d73660 0x564399d7b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:16.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.689+0000 7fdfd5600700 1 -- 172.21.15.151:0/1268253139 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564399d73660 msgr2=0x564399d7b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:16.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.689+0000 7fdfd5600700 1 --2- 172.21.15.151:0/1268253139 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564399d73660 0x564399d7b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:16.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.689+0000 7fdfd5600700 1 -- 172.21.15.151:0/1268253139 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564399d7e4d0 msgr2=0x564399d7b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:16.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.689+0000 7fdfd5600700 1 --2- 172.21.15.151:0/1268253139 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564399d7e4d0 0x564399d7b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:16.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.689+0000 7fdfd5600700 1 -- 172.21.15.151:0/1268253139 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564399c139d0 con 0x564399d72d40 2024-04-26T01:55:16.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.689+0000 7fdfd5600700 1 --2- 172.21.15.151:0/1268253139 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564399d72d40 0x564399d7dd70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x564399d74d50 tx=0x7fdfc409b400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13dba56e232571d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:16.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.689+0000 7fdfb7400700 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-26T01:55:16.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.689+0000 7fdfb7400700 1 -- 172.21.15.151:0/1268253139 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc409af90 con 0x564399d72d40 2024-04-26T01:55:16.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.689+0000 7fdfb7400700 1 -- 172.21.15.151:0/1268253139 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fdfc400f460 con 0x564399d72d40 2024-04-26T01:55:16.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.689+0000 7fdfe298fb80 1 -- 172.21.15.151:0/1268253139 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfc40a12a0 con 0x564399d72d40 2024-04-26T01:55:16.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.690+0000 7fdfb7400700 1 -- 172.21.15.151:0/1268253139 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdfc409cef0 con 0x564399d72d40 2024-04-26T01:55:16.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.690+0000 7fdfe298fb80 10 client.25402 fetch_fsmap learned FSMap version 26 2024-04-26T01:55:16.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.690+0000 7fdfe298fb80 1 -- 172.21.15.151:0/1268253139 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564399c5cbf0 con 0x564399d72d40 2024-04-26T01:55:16.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.690+0000 7fdfb7400700 1 -- 172.21.15.151:0/1268253139 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfc40a12a0 con 0x564399d72d40 2024-04-26T01:55:16.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.690+0000 7fdfb7400700 1 -- 172.21.15.151:0/1268253139 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fdfc409bdb0 con 0x564399d72d40 2024-04-26T01:55:16.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.690+0000 7fdfe298fb80 10 client.25402 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T01:55:16.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.690+0000 7fdfe298fb80 10 client.25402 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-26T01:55:16.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.690+0000 7fdfe298fb80 20 client.25402 populate_metadata read hostname 'smithi151' 2024-04-26T01:55:16.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.690+0000 7fdfe298fb80 1 --2- 172.21.15.151:0/1268253139 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x564399d8d8e0 0x564399d8fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:16.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.690+0000 7fdfe298fb80 4 client.25402 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:55:16.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.690+0000 7fdfe298fb80 1 -- 172.21.15.151:0/1268253139 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564399cd3780 con 0x564399d8d8e0 2024-04-26T01:55:16.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.691+0000 7fdfd4c00700 1 --2- 172.21.15.151:0/1268253139 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x564399d8d8e0 0x564399d8fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:16.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.692+0000 7fdfb7400700 10 client.25402 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:55:16.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.692+0000 7fdfd4c00700 1 --2- 172.21.15.151:0/1268253139 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x564399d8d8e0 0x564399d8fcd0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67df3b7f41fae280 server_cookie=bf8be4a174718cda in_seq=0 out_seq=0 2024-04-26T01:55:16.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.692+0000 7fdfb7400700 1 -- 172.21.15.151:0/1268253139 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564399cd3780 con 0x564399d8d8e0 2024-04-26T01:55:16.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.692+0000 7fdfb7400700 10 client.25402 handle_command_reply: tid=0 2024-04-26T01:55:16.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.693+0000 7fdfe298fb80 1 client.25402 shutdown 2024-04-26T01:55:16.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.693+0000 7fdfe298fb80 1 -- 172.21.15.151:0/1268253139 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564399d72d40 msgr2=0x564399d7dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:16.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.693+0000 7fdfe298fb80 1 --2- 172.21.15.151:0/1268253139 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564399d72d40 0x564399d7dd70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x564399d74d50 tx=0x7fdfc409b400 comp rx=0 tx=0).stop 2024-04-26T01:55:16.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.693+0000 7fdfe298fb80 1 -- 172.21.15.151:0/1268253139 shutdown_connections 2024-04-26T01:55:16.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.693+0000 7fdfe298fb80 1 --2- 172.21.15.151:0/1268253139 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564399d7e4d0 0x564399d7b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:16.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.693+0000 7fdfe298fb80 1 --2- 172.21.15.151:0/1268253139 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564399d72d40 0x564399d7dd70 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:16.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.693+0000 7fdfe298fb80 1 --2- 172.21.15.151:0/1268253139 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x564399d8d8e0 0x564399d8fcd0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:16.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.693+0000 7fdfe298fb80 1 --2- 172.21.15.151:0/1268253139 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564399d73660 0x564399d7b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:16.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.693+0000 7fdfe298fb80 1 -- 172.21.15.151:0/1268253139 >> 172.21.15.151:0/1268253139 conn(0x564399ca2050 msgr2=0x564399d6ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:16.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.693+0000 7fdfb7400700 0 client.25402 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:55:16.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.693+0000 7fdfe298fb80 1 -- 172.21.15.151:0/1268253139 shutdown_connections 2024-04-26T01:55:16.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.693+0000 7fdfe298fb80 1 -- 172.21.15.151:0/1268253139 wait complete. 2024-04-26T01:55:16.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.693+0000 7fdfe298fb80 20 client.25402 trim_cache size 0 max 16384 2024-04-26T01:55:16.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.714+0000 7fdfe298fb80 1 Processor -- start 2024-04-26T01:55:16.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.714+0000 7fdfe298fb80 1 -- start start 2024-04-26T01:55:16.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.715+0000 7fdfe298fb80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564399d72d40 0x564399d79ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:16.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.715+0000 7fdfe298fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564399d73660 0x564399d7d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:16.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.715+0000 7fdfe298fb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564399d7e4d0 0x564399d89b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:16.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.715+0000 7fdfe298fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd01b5f10 con 0x564399d73660 2024-04-26T01:55:16.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.715+0000 7fdfe298fb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd01b37a0 con 0x564399d7e4d0 2024-04-26T01:55:16.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.715+0000 7fdfe298fb80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd01b1610 con 0x564399d72d40 2024-04-26T01:55:16.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.715+0000 7fdfd4c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564399d73660 0x564399d7d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:16.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.715+0000 7fdfd4c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564399d73660 0x564399d7d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:48438/0 (socket says 172.21.15.151:48438) 2024-04-26T01:55:16.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.715+0000 7fdfd4c00700 1 -- 172.21.15.151:0/2974001643 learned_addr learned my addr 172.21.15.151:0/2974001643 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:55:16.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.715+0000 7fdfd6000700 1 --2- 172.21.15.151:0/2974001643 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564399d7e4d0 0x564399d89b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:16.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.715+0000 7fdfd5600700 1 --2- 172.21.15.151:0/2974001643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564399d72d40 0x564399d79ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:16.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.716+0000 7fdfd5600700 1 -- 172.21.15.151:0/2974001643 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564399d7e4d0 msgr2=0x564399d89b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:16.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.716+0000 7fdfd5600700 1 --2- 172.21.15.151:0/2974001643 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564399d7e4d0 0x564399d89b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:16.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.716+0000 7fdfd5600700 1 -- 172.21.15.151:0/2974001643 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564399d73660 msgr2=0x564399d7d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:16.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.716+0000 7fdfd5600700 1 --2- 172.21.15.151:0/2974001643 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564399d73660 0x564399d7d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:16.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.716+0000 7fdfd5600700 1 -- 172.21.15.151:0/2974001643 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564399d79600 con 0x564399d72d40 2024-04-26T01:55:16.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.716+0000 7fdfd5600700 1 --2- 172.21.15.151:0/2974001643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564399d72d40 0x564399d79ec0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdfc409c170 tx=0x7fdfc409c980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c96973baaeb7427f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:16.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.716+0000 7fdfc2000700 1 -- 172.21.15.151:0/2974001643 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc40a12a0 con 0x564399d72d40 2024-04-26T01:55:16.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.716+0000 7fdfc2000700 1 -- 172.21.15.151:0/2974001643 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfc409d360 con 0x564399d72d40 2024-04-26T01:55:16.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.716+0000 7fdfc2000700 1 -- 172.21.15.151:0/2974001643 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc40a5640 con 0x564399d72d40 2024-04-26T01:55:16.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.717+0000 7fdfe298fb80 1 -- 172.21.15.151:0/2974001643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564399d72d40 msgr2=0x564399d79ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:16.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.717+0000 7fdfe298fb80 1 --2- 172.21.15.151:0/2974001643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564399d72d40 0x564399d79ec0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdfc409c170 tx=0x7fdfc409c980 comp rx=0 tx=0).stop 2024-04-26T01:55:16.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.717+0000 7fdfe298fb80 1 -- 172.21.15.151:0/2974001643 shutdown_connections 2024-04-26T01:55:16.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.717+0000 7fdfe298fb80 1 --2- 172.21.15.151:0/2974001643 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564399d7e4d0 0x564399d89b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:16.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.717+0000 7fdfe298fb80 1 --2- 172.21.15.151:0/2974001643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564399d72d40 0x564399d79ec0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:16.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.717+0000 7fdfe298fb80 1 --2- 172.21.15.151:0/2974001643 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564399d73660 0x564399d7d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:16.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.717+0000 7fdfe298fb80 1 -- 172.21.15.151:0/2974001643 >> 172.21.15.151:0/2974001643 conn(0x564399ca2050 msgr2=0x564399d6ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:16.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.717+0000 7fdfe298fb80 1 -- 172.21.15.151:0/2974001643 shutdown_connections 2024-04-26T01:55:16.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.717+0000 7fdfe298fb80 1 -- 172.21.15.151:0/2974001643 wait complete. 2024-04-26T01:55:16.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.718+0000 7fdfe298fb80 1 -- start start 2024-04-26T01:55:16.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.718+0000 7fdfe298fb80 1 Processor -- start 2024-04-26T01:55:16.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.718+0000 7fdfe298fb80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564399d72d40 0x564399d86070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:16.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.718+0000 7fdfe298fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564399d73660 0x564399d865b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:16.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.718+0000 7fdfe298fb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564399d7e4d0 0x564399d84620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:16.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.718+0000 7fdfe298fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd01b1c60 con 0x564399d73660 2024-04-26T01:55:16.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.718+0000 7fdfe298fb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd01b30d0 con 0x564399d7e4d0 2024-04-26T01:55:16.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.718+0000 7fdfe298fb80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd01b2330 con 0x564399d72d40 2024-04-26T01:55:16.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.718+0000 7fdfe298fb80 10 client.0 _ll_register_callbacks cb 0x7fdfbc009d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:55:16.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.719+0000 7fdfd4c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564399d73660 0x564399d865b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:16.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.719+0000 7fdfd4c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564399d73660 0x564399d865b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:48442/0 (socket says 172.21.15.151:48442) 2024-04-26T01:55:16.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.719+0000 7fdfd4c00700 1 -- 172.21.15.151:0/2338821601 learned_addr learned my addr 172.21.15.151:0/2338821601 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:55:16.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.719+0000 7fdfd5600700 1 --2- 172.21.15.151:0/2338821601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564399d72d40 0x564399d86070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:16.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.719+0000 7fdfd6000700 1 --2- 172.21.15.151:0/2338821601 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564399d7e4d0 0x564399d84620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:16.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.719+0000 7fdfd4c00700 1 -- 172.21.15.151:0/2338821601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564399d72d40 msgr2=0x564399d86070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:16.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.719+0000 7fdfd4c00700 1 --2- 172.21.15.151:0/2338821601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564399d72d40 0x564399d86070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:16.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.719+0000 7fdfd4c00700 1 -- 172.21.15.151:0/2338821601 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564399d7e4d0 msgr2=0x564399d84620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:16.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.719+0000 7fdfd4c00700 1 --2- 172.21.15.151:0/2338821601 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564399d7e4d0 0x564399d84620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:16.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.719+0000 7fdfd4c00700 1 -- 172.21.15.151:0/2338821601 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564399d84e30 con 0x564399d73660 2024-04-26T01:55:16.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.719+0000 7fdfd4c00700 1 --2- 172.21.15.151:0/2338821601 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564399d73660 0x564399d865b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc004570 tx=0x7fdfbc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed52d7e1fea678a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:16.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.719+0000 7fdfb7400700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-04-26T01:55:16.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.720+0000 7fdfb7400700 1 -- 172.21.15.151:0/2338821601 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc02c2b0 con 0x564399d73660 2024-04-26T01:55:16.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.720+0000 7fdfb7400700 1 -- 172.21.15.151:0/2338821601 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fdfbc02c450 con 0x564399d73660 2024-04-26T01:55:16.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.720+0000 7fdfe298fb80 1 -- 172.21.15.151:0/2338821601 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfc409d360 con 0x564399d73660 2024-04-26T01:55:16.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.720+0000 7fdfb7400700 1 -- 172.21.15.151:0/2338821601 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdfbc0045a0 con 0x564399d73660 2024-04-26T01:55:16.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.720+0000 7fdfe298fb80 10 client.15288 fetch_fsmap learned FSMap version 26 2024-04-26T01:55:16.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.720+0000 7fdfe298fb80 1 -- 172.21.15.151:0/2338821601 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564399df48e0 con 0x564399d73660 2024-04-26T01:55:16.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.720+0000 7fdfb7400700 1 -- 172.21.15.151:0/2338821601 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfc409d360 con 0x564399d73660 2024-04-26T01:55:16.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.721+0000 7fdfb7400700 1 -- 172.21.15.151:0/2338821601 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fdfbc03d050 con 0x564399d73660 2024-04-26T01:55:16.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.721+0000 7fdfe298fb80 10 client.15288 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T01:55:16.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.721+0000 7fdfe298fb80 10 client.15288 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-26T01:55:16.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.721+0000 7fdfe298fb80 20 client.15288 populate_metadata read hostname 'smithi151' 2024-04-26T01:55:16.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.721+0000 7fdfe298fb80 1 --2- 172.21.15.151:0/2338821601 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x564399df5000 0x564399df7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:16.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.721+0000 7fdfe298fb80 4 client.15288 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "7b96ec7c", "release": true, "await": true}] 2024-04-26T01:55:16.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.721+0000 7fdfe298fb80 1 -- 172.21.15.151:0/2338821601 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "quiesce db", "set_id": "7b96ec7c", "release": true, "await": true}) -- 0x564399ccd820 con 0x564399df5000 2024-04-26T01:55:16.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.721+0000 7fdfd5600700 1 --2- 172.21.15.151:0/2338821601 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x564399df5000 0x564399df7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:16.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.722+0000 7fdfd5600700 1 --2- 172.21.15.151:0/2338821601 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x564399df5000 0x564399df7420 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb8a5b1c82a0b94c server_cookie=6e3e4a6d3e1903d5 in_seq=0 out_seq=0 2024-04-26T01:55:16.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.722+0000 7fdfb7400700 10 client.15288 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:55:16.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.725+0000 7fdfb7400700 1 -- 172.21.15.151:0/2338821601 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x564399ccd820 con 0x564399df5000 2024-04-26T01:55:16.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.725+0000 7fdfb7400700 10 client.15288 handle_command_reply: tid=0 2024-04-26T01:55:16.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.725+0000 7fdfe298fb80 1 client.15288 shutdown 2024-04-26T01:55:16.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.725+0000 7fdfe298fb80 1 -- 172.21.15.151:0/2338821601 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564399d73660 msgr2=0x564399d865b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:16.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.725+0000 7fdfe298fb80 1 --2- 172.21.15.151:0/2338821601 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564399d73660 0x564399d865b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc004570 tx=0x7fdfbc0047c0 comp rx=0 tx=0).stop 2024-04-26T01:55:16.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.725+0000 7fdfe298fb80 1 -- 172.21.15.151:0/2338821601 shutdown_connections 2024-04-26T01:55:16.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.725+0000 7fdfe298fb80 1 --2- 172.21.15.151:0/2338821601 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x564399df5000 0x564399df7420 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:16.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.725+0000 7fdfe298fb80 1 --2- 172.21.15.151:0/2338821601 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564399d7e4d0 0x564399d84620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:16.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.725+0000 7fdfe298fb80 1 --2- 172.21.15.151:0/2338821601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564399d72d40 0x564399d86070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:16.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.725+0000 7fdfe298fb80 1 --2- 172.21.15.151:0/2338821601 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564399d73660 0x564399d865b0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:16.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.725+0000 7fdfb7400700 0 client.15288 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:55:16.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.725+0000 7fdfe298fb80 1 -- 172.21.15.151:0/2338821601 >> 172.21.15.151:0/2338821601 conn(0x564399ca2050 msgr2=0x564399d88b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:16.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.726+0000 7fdfe298fb80 1 -- 172.21.15.151:0/2338821601 shutdown_connections 2024-04-26T01:55:16.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.726+0000 7fdfe298fb80 1 -- 172.21.15.151:0/2338821601 wait complete. 2024-04-26T01:55:16.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.726+0000 7fdfe298fb80 20 client.15288 trim_cache size 0 max 16384 2024-04-26T01:55:16.725 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-26T01:55:16.726 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 26, 2024-04-26T01:55:16.726 INFO:teuthology.orchestra.run.smithi151.stdout: "leader": 24509, 2024-04-26T01:55:16.726 INFO:teuthology.orchestra.run.smithi151.stdout: "set_version": 4, 2024-04-26T01:55:16.726 INFO:teuthology.orchestra.run.smithi151.stdout: "sets": { 2024-04-26T01:55:16.726 INFO:teuthology.orchestra.run.smithi151.stdout: "7b96ec7c": { 2024-04-26T01:55:16.726 INFO:teuthology.orchestra.run.smithi151.stdout: "version": 4, 2024-04-26T01:55:16.726 INFO:teuthology.orchestra.run.smithi151.stdout: "age_ref": 0.0, 2024-04-26T01:55:16.726 INFO:teuthology.orchestra.run.smithi151.stdout: "state": { 2024-04-26T01:55:16.726 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "RELEASED", 2024-04-26T01:55:16.726 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 0.0 2024-04-26T01:55:16.727 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T01:55:16.727 INFO:teuthology.orchestra.run.smithi151.stdout: "timeout": 60.0, 2024-04-26T01:55:16.727 INFO:teuthology.orchestra.run.smithi151.stdout: "expiration": 85.6, 2024-04-26T01:55:16.727 INFO:teuthology.orchestra.run.smithi151.stdout: "members": { 2024-04-26T01:55:16.727 INFO:teuthology.orchestra.run.smithi151.stdout: "file:/": { 2024-04-26T01:55:16.727 INFO:teuthology.orchestra.run.smithi151.stdout: "excluded": false, 2024-04-26T01:55:16.727 INFO:teuthology.orchestra.run.smithi151.stdout: "state": { 2024-04-26T01:55:16.727 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "RELEASED", 2024-04-26T01:55:16.727 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 0.0 2024-04-26T01:55:16.727 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T01:55:16.727 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T01:55:16.727 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T01:55:16.727 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T01:55:16.727 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T01:55:16.728 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-26T01:55:16.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.727+0000 7fdfd8000700 1 -- 172.21.15.151:0/1570916889 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fdfb80a3e30 msgr2=0x7fdfb80a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:16.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.727+0000 7fdfd8000700 1 --2- 172.21.15.151:0/1570916889 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fdfb80a3e30 0x7fdfb80a62f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdfd0078240 tx=0x7fdfbc009260 comp rx=0 tx=0).stop 2024-04-26T01:55:16.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.727+0000 7fdfd8000700 1 -- 172.21.15.151:0/1570916889 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdfd01092d0 msgr2=0x7fdfd019f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:16.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.727+0000 7fdfd8000700 1 --2- 172.21.15.151:0/1570916889 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdfd01092d0 0x7fdfd019f740 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4006010 tx=0x7fdfc4005280 comp rx=0 tx=0).stop 2024-04-26T01:55:16.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.727+0000 7fdfd8000700 1 -- 172.21.15.151:0/1570916889 shutdown_connections 2024-04-26T01:55:16.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.727+0000 7fdfd8000700 1 --2- 172.21.15.151:0/1570916889 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fdfb80a3e30 0x7fdfb80a62f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:16.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.727+0000 7fdfd8000700 1 --2- 172.21.15.151:0/1570916889 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdfd010e3b0 0x7fdfd01a4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:16.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.727+0000 7fdfd8000700 1 --2- 172.21.15.151:0/1570916889 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdfd01092d0 0x7fdfd019f740 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:16.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.727+0000 7fdfd8000700 1 --2- 172.21.15.151:0/1570916889 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdfd0109c80 0x7fdfd019fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:16.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.727+0000 7fdfd8000700 1 -- 172.21.15.151:0/1570916889 >> 172.21.15.151:0/1570916889 conn(0x7fdfd0076210 msgr2=0x7fdfd010cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:16.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.727+0000 7fdfd8000700 1 -- 172.21.15.151:0/1570916889 shutdown_connections 2024-04-26T01:55:16.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:55:16.727+0000 7fdfd8000700 1 -- 172.21.15.151:0/1570916889 wait complete. 2024-04-26T01:55:16.736 INFO:tasks.workunit.client.0.smithi151.stderr:+ set -e 2024-04-26T01:55:16.736 INFO:tasks.workunit.client.0.smithi151.stderr:+ git clone https://git.ceph.com/xfstests-dev.git 2024-04-26T01:55:16.742 INFO:tasks.workunit.client.0.smithi151.stderr:Cloning into 'xfstests-dev'... 2024-04-26T01:55:17.624 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '7b96ec7c', seconds elapsed: 40.3 2024-04-26T01:55:17.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:55:17 smithi151 ceph-mon[28057]: pgmap v53: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-04-26T01:55:18.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:55:17 smithi172 ceph-mon[32785]: pgmap v53: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-04-26T01:55:18.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:55:17 smithi165 ceph-mon[32648]: pgmap v53: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-04-26T01:55:20.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:55:19 smithi172 ceph-mon[32785]: pgmap v54: 97 pgs: 97 active+clean; 1.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 10 op/s 2024-04-26T01:55:20.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:55:19 smithi165 ceph-mon[32648]: pgmap v54: 97 pgs: 97 active+clean; 1.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 10 op/s 2024-04-26T01:55:20.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:55:19 smithi151 ceph-mon[28057]: pgmap v54: 97 pgs: 97 active+clean; 1.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 10 op/s 2024-04-26T01:55:22.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:55:21 smithi165 ceph-mon[32648]: pgmap v55: 97 pgs: 97 active+clean; 1.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 7 op/s 2024-04-26T01:55:22.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:55:21 smithi151 ceph-mon[28057]: pgmap v55: 97 pgs: 97 active+clean; 1.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 7 op/s 2024-04-26T01:55:22.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:55:21 smithi172 ceph-mon[32785]: pgmap v55: 97 pgs: 97 active+clean; 1.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 7 op/s 2024-04-26T01:55:24.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:55:23 smithi165 ceph-mon[32648]: pgmap v56: 97 pgs: 97 active+clean; 15 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 1.1 MiB/s wr, 12 op/s 2024-04-26T01:55:24.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:55:23 smithi151 ceph-mon[28057]: pgmap v56: 97 pgs: 97 active+clean; 15 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 1.1 MiB/s wr, 12 op/s 2024-04-26T01:55:24.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:55:23 smithi172 ceph-mon[32785]: pgmap v56: 97 pgs: 97 active+clean; 15 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 1.1 MiB/s wr, 12 op/s 2024-04-26T01:55:25.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:55:24 smithi165 ceph-mon[32648]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T01:55:25.096 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:55:24 smithi151 ceph-mon[28057]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T01:55:25.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:55:24 smithi172 ceph-mon[32785]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T01:55:25.576 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.577+0000 7f84f3c00640 1 -- 172.21.15.151:0/1313477085 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84ec10e620 msgr2=0x7f84ec10ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:25.576 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.577+0000 7f84f3c00640 1 --2- 172.21.15.151:0/1313477085 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84ec10e620 0x7f84ec10ea00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f84d4009930 tx=0x7f84d402a420 comp rx=0 tx=0).stop 2024-04-26T01:55:25.576 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.578+0000 7f84f3c00640 1 -- 172.21.15.151:0/1313477085 shutdown_connections 2024-04-26T01:55:25.577 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.578+0000 7f84f3c00640 1 --2- 172.21.15.151:0/1313477085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84ec108f20 0x7f84ec1153e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:25.577 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.578+0000 7f84f3c00640 1 --2- 172.21.15.151:0/1313477085 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f84ec108560 0x7f84ec1089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:25.577 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.578+0000 7f84f3c00640 1 --2- 172.21.15.151:0/1313477085 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84ec10e620 0x7f84ec10ea00 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:25.577 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.578+0000 7f84f3c00640 1 -- 172.21.15.151:0/1313477085 >> 172.21.15.151:0/1313477085 conn(0x7f84ec0764e0 msgr2=0x7f84ec078900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:25.577 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.578+0000 7f84f3c00640 1 -- 172.21.15.151:0/1313477085 shutdown_connections 2024-04-26T01:55:25.577 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.578+0000 7f84f3c00640 1 -- 172.21.15.151:0/1313477085 wait complete. 2024-04-26T01:55:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.579+0000 7f84f3c00640 1 Processor -- start 2024-04-26T01:55:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.579+0000 7f84f3c00640 1 -- start start 2024-04-26T01:55:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.579+0000 7f84f3c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84ec108560 0x7f84ec1a48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.579+0000 7f84f3c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f84ec108f20 0x7f84ec1a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.579+0000 7f84f3c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84ec10e620 0x7f84ec19ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.579+0000 7f84f3c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f84ec1187b0 con 0x7f84ec108f20 2024-04-26T01:55:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.579+0000 7f84f3c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f84ec118630 con 0x7f84ec108560 2024-04-26T01:55:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.579+0000 7f84f3c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f84ec118930 con 0x7f84ec10e620 2024-04-26T01:55:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.579+0000 7f84f1200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84ec108560 0x7f84ec1a48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.579+0000 7f84e3e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f84ec108f20 0x7f84ec1a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.580+0000 7f84f1200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84ec108560 0x7f84ec1a48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:42548/0 (socket says 172.21.15.151:42548) 2024-04-26T01:55:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.580+0000 7f84f1200640 1 -- 172.21.15.151:0/2639668515 learned_addr learned my addr 172.21.15.151:0/2639668515 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:55:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.580+0000 7f84f1c00640 1 --2- 172.21.15.151:0/2639668515 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84ec10e620 0x7f84ec19ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.580+0000 7f84e3e00640 1 -- 172.21.15.151:0/2639668515 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84ec10e620 msgr2=0x7f84ec19ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.580+0000 7f84e3e00640 1 --2- 172.21.15.151:0/2639668515 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84ec10e620 0x7f84ec19ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.580+0000 7f84e3e00640 1 -- 172.21.15.151:0/2639668515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84ec108560 msgr2=0x7f84ec1a48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:25.580 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.580+0000 7f84e3e00640 1 --2- 172.21.15.151:0/2639668515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84ec108560 0x7f84ec1a48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:25.580 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.580+0000 7f84e3e00640 1 -- 172.21.15.151:0/2639668515 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84ec19f210 con 0x7f84ec108f20 2024-04-26T01:55:25.580 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.580+0000 7f84e3e00640 1 --2- 172.21.15.151:0/2639668515 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f84ec108f20 0x7f84ec1a4e20 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f84dc00e9f0 tx=0x7f84dc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:25.580 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.581+0000 7f84ea000640 1 -- 172.21.15.151:0/2639668515 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84dc00cd80 con 0x7f84ec108f20 2024-04-26T01:55:25.580 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.581+0000 7f84f3c00640 1 -- 172.21.15.151:0/2639668515 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84ec19f500 con 0x7f84ec108f20 2024-04-26T01:55:25.580 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.581+0000 7f84ea000640 1 -- 172.21.15.151:0/2639668515 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84dc0035f0 con 0x7f84ec108f20 2024-04-26T01:55:25.581 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.581+0000 7f84ea000640 1 -- 172.21.15.151:0/2639668515 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84dc005220 con 0x7f84ec108f20 2024-04-26T01:55:25.581 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.581+0000 7f84f3c00640 1 -- 172.21.15.151:0/2639668515 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84ec1ab7a0 con 0x7f84ec108f20 2024-04-26T01:55:25.582 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.584+0000 7f84ea000640 1 -- 172.21.15.151:0/2639668515 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f84dc0037a0 con 0x7f84ec108f20 2024-04-26T01:55:25.582 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.584+0000 7f84f69e4740 1 Processor -- start 2024-04-26T01:55:25.583 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.584+0000 7f84f69e4740 1 -- start start 2024-04-26T01:55:25.584 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.585+0000 7f84f69e4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b99983820 0x560b99983c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:25.584 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.585+0000 7f84f69e4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b99984140 0x560b9998e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:25.584 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.585+0000 7f84f69e4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560b9998ea00 0x560b99990df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:25.584 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.585+0000 7f84f69e4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560b998e9900 con 0x560b99984140 2024-04-26T01:55:25.584 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.585+0000 7f84f69e4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560b9988fef0 con 0x560b9998ea00 2024-04-26T01:55:25.584 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.585+0000 7f84f69e4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560b998e58c0 con 0x560b99983820 2024-04-26T01:55:25.584 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.585+0000 7f84f1c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b99983820 0x560b99983c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:25.584 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.585+0000 7f84f1c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b99983820 0x560b99983c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:47870/0 (socket says 172.21.15.151:47870) 2024-04-26T01:55:25.584 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.585+0000 7f84f1c00640 1 -- 172.21.15.151:0/2904565639 learned_addr learned my addr 172.21.15.151:0/2904565639 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:55:25.584 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.585+0000 7f84f1200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b99984140 0x560b9998e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:25.585 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.585+0000 7f84e3e00640 1 --2- 172.21.15.151:0/2904565639 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560b9998ea00 0x560b99990df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:25.585 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.585+0000 7f84f1c00640 1 -- 172.21.15.151:0/2904565639 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560b9998ea00 msgr2=0x560b99990df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:25.585 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.585+0000 7f84f1c00640 1 --2- 172.21.15.151:0/2904565639 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560b9998ea00 0x560b99990df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:25.585 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.585+0000 7f84f1c00640 1 -- 172.21.15.151:0/2904565639 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b99984140 msgr2=0x560b9998e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:25.585 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.585+0000 7f84f1c00640 1 --2- 172.21.15.151:0/2904565639 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b99984140 0x560b9998e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:25.585 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.585+0000 7f84f1c00640 1 -- 172.21.15.151:0/2904565639 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b995fc210 con 0x560b99983820 2024-04-26T01:55:25.585 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.586+0000 7f84f1c00640 1 --2- 172.21.15.151:0/2904565639 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b99983820 0x560b99983c00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f84e4009fd0 tx=0x7f84e40118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d13997ff73711575 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.586+0000 7f84ea000640 1 --2- 172.21.15.151:0/2639668515 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f84cc0a3dc0 0x7f84cc0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.586+0000 7f84e2000640 1 -- 172.21.15.151:0/2904565639 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e4004d20 con 0x560b99983820 2024-04-26T01:55:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.586+0000 7f84e2000640 1 -- 172.21.15.151:0/2904565639 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84e4004050 con 0x560b99983820 2024-04-26T01:55:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.586+0000 7f84e2000640 1 -- 172.21.15.151:0/2904565639 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e4005280 con 0x560b99983820 2024-04-26T01:55:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.586+0000 7f84ea000640 1 -- 172.21.15.151:0/2639668515 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f84dc0d0a10 con 0x7f84ec108f20 2024-04-26T01:55:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.586+0000 7f84f1200640 1 --2- 172.21.15.151:0/2639668515 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f84cc0a3dc0 0x7f84cc0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.587+0000 7f84f69e4740 1 -- 172.21.15.151:0/2904565639 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b99983820 msgr2=0x560b99983c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.587+0000 7f84f69e4740 1 --2- 172.21.15.151:0/2904565639 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b99983820 0x560b99983c00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f84e4009fd0 tx=0x7f84e40118d0 comp rx=0 tx=0).stop 2024-04-26T01:55:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.587+0000 7f84f69e4740 1 -- 172.21.15.151:0/2904565639 shutdown_connections 2024-04-26T01:55:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.587+0000 7f84f69e4740 1 --2- 172.21.15.151:0/2904565639 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560b9998ea00 0x560b99990df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.587+0000 7f84f69e4740 1 --2- 172.21.15.151:0/2904565639 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b99984140 0x560b9998e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.587+0000 7f84f69e4740 1 --2- 172.21.15.151:0/2904565639 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b99983820 0x560b99983c00 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.587+0000 7f84f69e4740 1 -- 172.21.15.151:0/2904565639 >> 172.21.15.151:0/2904565639 conn(0x560b9997fd30 msgr2=0x560b99980110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.587+0000 7f84f69e4740 1 -- 172.21.15.151:0/2904565639 shutdown_connections 2024-04-26T01:55:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.587+0000 7f84f1200640 1 --2- 172.21.15.151:0/2639668515 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f84cc0a3dc0 0x7f84cc0a6280 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f84d4009900 tx=0x7f84d40058d0 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.587+0000 7f84f69e4740 1 -- 172.21.15.151:0/2904565639 wait complete. 2024-04-26T01:55:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.588+0000 7f84f69e4740 1 -- start start 2024-04-26T01:55:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.588+0000 7f84f69e4740 1 Processor -- start 2024-04-26T01:55:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.588+0000 7f84f69e4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b99983820 0x560b9998b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.588+0000 7f84f69e4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560b99984140 0x560b9998bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.588+0000 7f84f69e4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b9998ea00 0x560b9998c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.588+0000 7f84f69e4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560b99991630 con 0x560b99983820 2024-04-26T01:55:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.588+0000 7f84f69e4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560b998a3310 con 0x560b99984140 2024-04-26T01:55:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.588+0000 7f84f69e4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560b999917b0 con 0x560b9998ea00 2024-04-26T01:55:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.588+0000 7f84f69e4740 10 client.0 _ll_register_callbacks cb 0x560b99871cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:55:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.588+0000 7f84f1200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b99983820 0x560b9998b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.589+0000 7f84f1200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b99983820 0x560b9998b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41290/0 (socket says 172.21.15.151:41290) 2024-04-26T01:55:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.589+0000 7f84f1200640 1 -- 172.21.15.151:0/2012545111 learned_addr learned my addr 172.21.15.151:0/2012545111 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:55:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.589+0000 7f84f1c00640 1 --2- 172.21.15.151:0/2012545111 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b9998ea00 0x560b9998c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.589+0000 7f84e3e00640 1 --2- 172.21.15.151:0/2012545111 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560b99984140 0x560b9998bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.589+0000 7f84f1200640 1 -- 172.21.15.151:0/2012545111 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b9998ea00 msgr2=0x560b9998c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.589+0000 7f84f1200640 1 --2- 172.21.15.151:0/2012545111 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b9998ea00 0x560b9998c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.589+0000 7f84f1200640 1 -- 172.21.15.151:0/2012545111 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560b99984140 msgr2=0x560b9998bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.589+0000 7f84f1200640 1 --2- 172.21.15.151:0/2012545111 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560b99984140 0x560b9998bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.589+0000 7f84f1200640 1 -- 172.21.15.151:0/2012545111 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b996c8620 con 0x560b99983820 2024-04-26T01:55:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.589+0000 7f84e3e00640 1 --2- 172.21.15.151:0/2012545111 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560b99984140 0x560b9998bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:55:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.589+0000 7f84f1200640 1 --2- 172.21.15.151:0/2012545111 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b99983820 0x560b9998b980 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f84d4009a50 tx=0x7f84d4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c2a24da150c9055 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.589+0000 7f84daa00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-04-26T01:55:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.590+0000 7f84daa00640 1 -- 172.21.15.151:0/2012545111 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d404d030 con 0x560b99983820 2024-04-26T01:55:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.590+0000 7f84daa00640 1 -- 172.21.15.151:0/2012545111 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f84d4003800 con 0x560b99983820 2024-04-26T01:55:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.590+0000 7f84f69e4740 1 -- 172.21.15.151:0/2012545111 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84e4004050 con 0x560b99983820 2024-04-26T01:55:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.590+0000 7f84daa00640 1 -- 172.21.15.151:0/2012545111 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f84d4035040 con 0x560b99983820 2024-04-26T01:55:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.590+0000 7f84f69e4740 10 client.15300 fetch_fsmap learned FSMap version 27 2024-04-26T01:55:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.590+0000 7f84f69e4740 1 -- 172.21.15.151:0/2012545111 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560b9999d870 con 0x560b99983820 2024-04-26T01:55:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.590+0000 7f84daa00640 1 -- 172.21.15.151:0/2012545111 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84e4004050 con 0x560b99983820 2024-04-26T01:55:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.591+0000 7f84daa00640 1 -- 172.21.15.151:0/2012545111 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 27) ==== 8342+0+0 (secure 0 0 0) 0x7f84d405a050 con 0x560b99983820 2024-04-26T01:55:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.591+0000 7f84f69e4740 10 client.15300 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T01:55:25.593 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.591+0000 7f84f69e4740 10 client.15300 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:55:25.593 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.591+0000 7f84f69e4740 20 client.15300 populate_metadata read hostname 'smithi151' 2024-04-26T01:55:25.593 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.591+0000 7f84f69e4740 1 --2- 172.21.15.151:0/2012545111 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x560b9999ed30 0x560b999a1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:25.593 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.591+0000 7f84f69e4740 4 client.15300 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:55:25.593 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.591+0000 7f84f69e4740 1 -- 172.21.15.151:0/2012545111 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b998fd8b0 con 0x560b9999ed30 2024-04-26T01:55:25.594 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.591+0000 7f84e3e00640 1 --2- 172.21.15.151:0/2012545111 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x560b9999ed30 0x560b999a1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:25.594 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.591+0000 7f84e3e00640 1 --2- 172.21.15.151:0/2012545111 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x560b9999ed30 0x560b999a1160 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f46e931d3d46030 server_cookie=a5d16c3f98f08022 in_seq=0 out_seq=0 2024-04-26T01:55:25.594 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.591+0000 7f84daa00640 10 client.15300 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:55:25.594 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.592+0000 7f84daa00640 1 -- 172.21.15.151:0/2012545111 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560b998fd8b0 con 0x560b9999ed30 2024-04-26T01:55:25.594 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.592+0000 7f84daa00640 10 client.15300 handle_command_reply: tid=0 2024-04-26T01:55:25.595 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.592+0000 7f84f69e4740 1 client.15300 shutdown 2024-04-26T01:55:25.595 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.592+0000 7f84f69e4740 1 -- 172.21.15.151:0/2012545111 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b99983820 msgr2=0x560b9998b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:25.595 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.592+0000 7f84f69e4740 1 --2- 172.21.15.151:0/2012545111 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b99983820 0x560b9998b980 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f84d4009a50 tx=0x7f84d4009260 comp rx=0 tx=0).stop 2024-04-26T01:55:25.595 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.592+0000 7f84f69e4740 1 -- 172.21.15.151:0/2012545111 shutdown_connections 2024-04-26T01:55:25.595 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.592+0000 7f84f69e4740 1 --2- 172.21.15.151:0/2012545111 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x560b9999ed30 0x560b999a1160 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:25.595 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.592+0000 7f84f69e4740 1 --2- 172.21.15.151:0/2012545111 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b9998ea00 0x560b9998c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:25.595 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.592+0000 7f84f69e4740 1 --2- 172.21.15.151:0/2012545111 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560b99984140 0x560b9998bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:25.595 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.592+0000 7f84f69e4740 1 --2- 172.21.15.151:0/2012545111 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b99983820 0x560b9998b980 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:25.595 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.592+0000 7f84daa00640 0 client.15300 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:55:25.595 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.592+0000 7f84f69e4740 1 -- 172.21.15.151:0/2012545111 >> 172.21.15.151:0/2012545111 conn(0x560b9997fd30 msgr2=0x560b99985ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:25.595 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.593+0000 7f84f69e4740 1 -- 172.21.15.151:0/2012545111 shutdown_connections 2024-04-26T01:55:25.595 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.593+0000 7f84f69e4740 1 -- 172.21.15.151:0/2012545111 wait complete. 2024-04-26T01:55:25.595 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.593+0000 7f84f69e4740 20 client.15300 trim_cache size 0 max 16384 2024-04-26T01:55:25.609 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.610+0000 7f84f69e4740 1 Processor -- start 2024-04-26T01:55:25.609 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.610+0000 7f84f69e4740 1 -- start start 2024-04-26T01:55:25.609 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.610+0000 7f84f69e4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b99983820 0x560b9999a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:25.609 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.610+0000 7f84f69e4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b99984140 0x560b999861e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:25.610 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.610+0000 7f84f69e4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560b9998ea00 0x560b999a6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:25.610 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.610+0000 7f84f69e4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f84ec1b6830 con 0x560b99983820 2024-04-26T01:55:25.610 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.610+0000 7f84f69e4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f84ec1b4100 con 0x560b9998ea00 2024-04-26T01:55:25.610 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.610+0000 7f84f69e4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f84ec1b1f70 con 0x560b99984140 2024-04-26T01:55:25.610 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.611+0000 7f84f1c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560b9998ea00 0x560b999a6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:25.610 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.611+0000 7f84e3e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b99984140 0x560b999861e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:25.610 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.611+0000 7f84e3e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b99984140 0x560b999861e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:47896/0 (socket says 172.21.15.151:47896) 2024-04-26T01:55:25.610 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.611+0000 7f84e3e00640 1 -- 172.21.15.151:0/1637194168 learned_addr learned my addr 172.21.15.151:0/1637194168 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:55:25.610 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.611+0000 7f84f1c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560b9998ea00 0x560b999a6a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:42562/0 (socket says 172.21.15.151:42562) 2024-04-26T01:55:25.610 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.611+0000 7f84f1200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b99983820 0x560b9999a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:25.610 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.611+0000 7f84e3e00640 1 -- 172.21.15.151:0/1637194168 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560b9998ea00 msgr2=0x560b999a6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:25.610 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.611+0000 7f84e3e00640 1 --2- 172.21.15.151:0/1637194168 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560b9998ea00 0x560b999a6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:25.610 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.611+0000 7f84e3e00640 1 -- 172.21.15.151:0/1637194168 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b99983820 msgr2=0x560b9999a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:25.610 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.611+0000 7f84e3e00640 1 --2- 172.21.15.151:0/1637194168 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b99983820 0x560b9999a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:25.611 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.611+0000 7f84e3e00640 1 -- 172.21.15.151:0/1637194168 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b99986720 con 0x560b99984140 2024-04-26T01:55:25.612 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.611+0000 7f84e3e00640 1 --2- 172.21.15.151:0/1637194168 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b99984140 0x560b999861e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f84dc09c900 tx=0x7f84dc0bd040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6036039173a677d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:25.612 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.611+0000 7f84e1600640 1 -- 172.21.15.151:0/1637194168 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84dc0a5630 con 0x560b99984140 2024-04-26T01:55:25.612 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.611+0000 7f84e1600640 1 -- 172.21.15.151:0/1637194168 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84dc09aa60 con 0x560b99984140 2024-04-26T01:55:25.612 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.612+0000 7f84e1600640 1 -- 172.21.15.151:0/1637194168 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84dc099030 con 0x560b99984140 2024-04-26T01:55:25.612 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.612+0000 7f84f69e4740 1 -- 172.21.15.151:0/1637194168 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b99984140 msgr2=0x560b999861e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:25.612 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.612+0000 7f84f69e4740 1 --2- 172.21.15.151:0/1637194168 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b99984140 0x560b999861e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f84dc09c900 tx=0x7f84dc0bd040 comp rx=0 tx=0).stop 2024-04-26T01:55:25.612 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.612+0000 7f84f69e4740 1 -- 172.21.15.151:0/1637194168 shutdown_connections 2024-04-26T01:55:25.612 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.612+0000 7f84f69e4740 1 --2- 172.21.15.151:0/1637194168 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560b9998ea00 0x560b999a6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:25.612 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.612+0000 7f84f69e4740 1 --2- 172.21.15.151:0/1637194168 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b99984140 0x560b999861e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:25.612 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.612+0000 7f84f69e4740 1 --2- 172.21.15.151:0/1637194168 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b99983820 0x560b9999a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:25.612 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.612+0000 7f84f69e4740 1 -- 172.21.15.151:0/1637194168 >> 172.21.15.151:0/1637194168 conn(0x560b9997fd30 msgr2=0x560b9997ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:25.612 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.612+0000 7f84f69e4740 1 -- 172.21.15.151:0/1637194168 shutdown_connections 2024-04-26T01:55:25.612 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.613+0000 7f84f69e4740 1 -- 172.21.15.151:0/1637194168 wait complete. 2024-04-26T01:55:25.612 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.613+0000 7f84f69e4740 1 -- start start 2024-04-26T01:55:25.613 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.613+0000 7f84f69e4740 1 Processor -- start 2024-04-26T01:55:25.615 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.614+0000 7f84f69e4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b99983820 0x560b99994730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:25.615 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.614+0000 7f84f69e4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b99984140 0x560b99994c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:25.615 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.614+0000 7f84f69e4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560b9998ea00 0x560b999951b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:25.615 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.614+0000 7f84f69e4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f84ec1b25c0 con 0x560b99984140 2024-04-26T01:55:25.615 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.614+0000 7f84f69e4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f84ec1b3a30 con 0x560b9998ea00 2024-04-26T01:55:25.616 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.614+0000 7f84f69e4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f84ec1b2c90 con 0x560b99983820 2024-04-26T01:55:25.616 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.614+0000 7f84f69e4740 10 client.0 _ll_register_callbacks cb 0x7f84d4031530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:55:25.616 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.614+0000 7f84f1200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b99983820 0x560b99994730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:25.616 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.614+0000 7f84f1c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560b9998ea00 0x560b999951b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:25.616 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.614+0000 7f84f1200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b99983820 0x560b99994730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:47898/0 (socket says 172.21.15.151:47898) 2024-04-26T01:55:25.616 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.614+0000 7f84f1200640 1 -- 172.21.15.151:0/636210753 learned_addr learned my addr 172.21.15.151:0/636210753 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:55:25.616 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.614+0000 7f84e3e00640 1 --2- 172.21.15.151:0/636210753 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b99984140 0x560b99994c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:25.616 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.615+0000 7f84f1c00640 1 -- 172.21.15.151:0/636210753 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b99983820 msgr2=0x560b99994730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:25.616 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.615+0000 7f84f1c00640 1 --2- 172.21.15.151:0/636210753 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b99983820 0x560b99994730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:25.616 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.615+0000 7f84f1c00640 1 -- 172.21.15.151:0/636210753 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b99984140 msgr2=0x560b99994c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:25.616 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.615+0000 7f84f1c00640 1 --2- 172.21.15.151:0/636210753 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b99984140 0x560b99994c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:25.616 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.615+0000 7f84f1c00640 1 -- 172.21.15.151:0/636210753 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b99a04590 con 0x560b9998ea00 2024-04-26T01:55:25.617 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.615+0000 7f84f1c00640 1 --2- 172.21.15.151:0/636210753 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560b9998ea00 0x560b999951b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f84e4005c40 tx=0x7f84e4009d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e9036e761ca1f7c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:25.617 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.615+0000 7f84daa00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T01:55:25.617 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.615+0000 7f84daa00640 1 -- 172.21.15.151:0/636210753 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e4004d10 con 0x560b9998ea00 2024-04-26T01:55:25.617 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.615+0000 7f84f69e4740 1 -- 172.21.15.151:0/636210753 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84dc09aa60 con 0x560b9998ea00 2024-04-26T01:55:25.617 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.615+0000 7f84daa00640 1 -- 172.21.15.151:0/636210753 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f84e4004000 con 0x560b9998ea00 2024-04-26T01:55:25.617 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.616+0000 7f84daa00640 1 -- 172.21.15.151:0/636210753 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f84e4004200 con 0x560b9998ea00 2024-04-26T01:55:25.617 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.616+0000 7f84f69e4740 10 client.25406 fetch_fsmap learned FSMap version 27 2024-04-26T01:55:25.617 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.616+0000 7f84f69e4740 1 -- 172.21.15.151:0/636210753 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560b99a058a0 con 0x560b9998ea00 2024-04-26T01:55:25.617 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.616+0000 7f84daa00640 1 -- 172.21.15.151:0/636210753 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84dc09aa60 con 0x560b9998ea00 2024-04-26T01:55:25.617 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.616+0000 7f84daa00640 1 -- 172.21.15.151:0/636210753 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 27) ==== 8342+0+0 (secure 0 0 0) 0x7f84e4034020 con 0x560b9998ea00 2024-04-26T01:55:25.617 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.616+0000 7f84f69e4740 10 client.25406 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T01:55:25.617 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.616+0000 7f84f69e4740 10 client.25406 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:55:25.617 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.616+0000 7f84f69e4740 20 client.25406 populate_metadata read hostname 'smithi151' 2024-04-26T01:55:25.617 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.616+0000 7f84f69e4740 1 --2- 172.21.15.151:0/636210753 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x560b99a05d90 0x560b99a081b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:25.618 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.616+0000 7f84f69e4740 4 client.25406 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:55:25.618 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.616+0000 7f84f69e4740 1 -- 172.21.15.151:0/636210753 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560b998f7320 con 0x560b99a05d90 2024-04-26T01:55:25.618 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.617+0000 7f84f1200640 1 --2- 172.21.15.151:0/636210753 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x560b99a05d90 0x560b99a081b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:25.618 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.617+0000 7f84f1200640 1 --2- 172.21.15.151:0/636210753 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x560b99a05d90 0x560b99a081b0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed940d90efc0e4bf server_cookie=b4cda1b8fd353120 in_seq=0 out_seq=0 2024-04-26T01:55:25.618 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:25.617+0000 7f84daa00640 10 client.25406 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:55:26.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:55:25 smithi165 ceph-mon[32648]: pgmap v57: 97 pgs: 97 active+clean; 15 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 1.1 MiB/s wr, 9 op/s 2024-04-26T01:55:26.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:55:25 smithi151 ceph-mon[28057]: pgmap v57: 97 pgs: 97 active+clean; 15 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 1.1 MiB/s wr, 9 op/s 2024-04-26T01:55:26.238 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:26.239+0000 7f84daa00640 1 -- 172.21.15.151:0/636210753 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560b998f7320 con 0x560b99a05d90 2024-04-26T01:55:26.238 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:26.239+0000 7f84daa00640 10 client.25406 handle_command_reply: tid=0 2024-04-26T01:55:26.238 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:26.239+0000 7f84f69e4740 1 client.25406 shutdown 2024-04-26T01:55:26.238 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:26.239+0000 7f84f69e4740 1 -- 172.21.15.151:0/636210753 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560b9998ea00 msgr2=0x560b999951b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:26.239 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:26.239+0000 7f84f69e4740 1 --2- 172.21.15.151:0/636210753 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560b9998ea00 0x560b999951b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f84e4005c40 tx=0x7f84e4009d40 comp rx=0 tx=0).stop 2024-04-26T01:55:26.239 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:26.239+0000 7f84f69e4740 1 -- 172.21.15.151:0/636210753 shutdown_connections 2024-04-26T01:55:26.239 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:26.239+0000 7f84f69e4740 1 --2- 172.21.15.151:0/636210753 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x560b99a05d90 0x560b99a081b0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:26.239 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:26.239+0000 7f84f69e4740 1 --2- 172.21.15.151:0/636210753 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560b9998ea00 0x560b999951b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:26.239 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:26.239+0000 7f84daa00640 0 client.25406 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:55:26.239 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:26.239+0000 7f84f69e4740 1 --2- 172.21.15.151:0/636210753 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b99984140 0x560b99994c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:26.239 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:26.239+0000 7f84f69e4740 1 --2- 172.21.15.151:0/636210753 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b99983820 0x560b99994730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:26.239 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:26.239+0000 7f84f69e4740 1 -- 172.21.15.151:0/636210753 >> 172.21.15.151:0/636210753 conn(0x560b9997fd30 msgr2=0x560b9997e600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:26.239 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:26.239+0000 7f84f69e4740 1 -- 172.21.15.151:0/636210753 shutdown_connections 2024-04-26T01:55:26.239 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:26.239+0000 7f84f69e4740 1 -- 172.21.15.151:0/636210753 wait complete. 2024-04-26T01:55:26.239 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:26.240+0000 7f84f69e4740 20 client.25406 trim_cache size 0 max 16384 2024-04-26T01:55:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:{ 2024-04-26T01:55:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "message": "", 2024-04-26T01:55:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "return_code": 0 2024-04-26T01:55:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:} 2024-04-26T01:55:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:26.240+0000 7f84f3c00640 1 -- 172.21.15.151:0/2639668515 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f84cc0a3dc0 msgr2=0x7f84cc0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:26.240+0000 7f84f3c00640 1 --2- 172.21.15.151:0/2639668515 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f84cc0a3dc0 0x7f84cc0a6280 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f84d4009900 tx=0x7f84d40058d0 comp rx=0 tx=0).stop 2024-04-26T01:55:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:26.240+0000 7f84f3c00640 1 -- 172.21.15.151:0/2639668515 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f84ec108f20 msgr2=0x7f84ec1a4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:26.240+0000 7f84f3c00640 1 --2- 172.21.15.151:0/2639668515 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f84ec108f20 0x7f84ec1a4e20 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f84dc00e9f0 tx=0x7f84dc00eeb0 comp rx=0 tx=0).stop 2024-04-26T01:55:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:26.241+0000 7f84f3c00640 1 -- 172.21.15.151:0/2639668515 shutdown_connections 2024-04-26T01:55:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:26.241+0000 7f84f3c00640 1 --2- 172.21.15.151:0/2639668515 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f84cc0a3dc0 0x7f84cc0a6280 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:26.241+0000 7f84f3c00640 1 --2- 172.21.15.151:0/2639668515 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84ec10e620 0x7f84ec19ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:26.241+0000 7f84f3c00640 1 --2- 172.21.15.151:0/2639668515 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f84ec108f20 0x7f84ec1a4e20 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:26.241+0000 7f84f3c00640 1 --2- 172.21.15.151:0/2639668515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84ec108560 0x7f84ec1a48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:26.241+0000 7f84f3c00640 1 -- 172.21.15.151:0/2639668515 >> 172.21.15.151:0/2639668515 conn(0x7f84ec0764e0 msgr2=0x7f84ec077ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:26.241+0000 7f84f3c00640 1 -- 172.21.15.151:0/2639668515 shutdown_connections 2024-04-26T01:55:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:26.241+0000 7f84f3c00640 1 -- 172.21.15.151:0/2639668515 wait complete. 2024-04-26T01:55:26.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:55:25 smithi172 ceph-mon[32785]: pgmap v57: 97 pgs: 97 active+clean; 15 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 1.1 MiB/s wr, 9 op/s 2024-04-26T01:55:28.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:55:27 smithi165 ceph-mon[32648]: pgmap v58: 97 pgs: 97 active+clean; 15 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 1.1 MiB/s wr, 9 op/s 2024-04-26T01:55:28.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:55:27 smithi151 ceph-mon[28057]: pgmap v58: 97 pgs: 97 active+clean; 15 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 1.1 MiB/s wr, 9 op/s 2024-04-26T01:55:28.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:55:27 smithi172 ceph-mon[32785]: pgmap v58: 97 pgs: 97 active+clean; 15 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 1.1 MiB/s wr, 9 op/s 2024-04-26T01:55:30.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:55:29 smithi165 ceph-mon[32648]: pgmap v59: 97 pgs: 97 active+clean; 18 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 260 op/s 2024-04-26T01:55:30.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:55:29 smithi165 ceph-mon[32648]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T01:55:30.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:55:29 smithi151 ceph-mon[28057]: pgmap v59: 97 pgs: 97 active+clean; 18 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 260 op/s 2024-04-26T01:55:30.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:55:29 smithi151 ceph-mon[28057]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T01:55:30.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:55:29 smithi172 ceph-mon[32785]: pgmap v59: 97 pgs: 97 active+clean; 18 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 260 op/s 2024-04-26T01:55:30.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:55:29 smithi172 ceph-mon[32785]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T01:55:32.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:55:31 smithi165 ceph-mon[32648]: pgmap v60: 97 pgs: 97 active+clean; 18 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 256 op/s 2024-04-26T01:55:32.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:55:31 smithi151 ceph-mon[28057]: pgmap v60: 97 pgs: 97 active+clean; 18 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 256 op/s 2024-04-26T01:55:32.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:55:31 smithi172 ceph-mon[32785]: pgmap v60: 97 pgs: 97 active+clean; 18 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 256 op/s 2024-04-26T01:55:33.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:55:33 smithi151 ceph-mon[28057]: pgmap v61: 97 pgs: 97 active+clean; 27 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 396 op/s 2024-04-26T01:55:34.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:55:33 smithi165 ceph-mon[32648]: pgmap v61: 97 pgs: 97 active+clean; 27 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 396 op/s 2024-04-26T01:55:34.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:55:33 smithi172 ceph-mon[32785]: pgmap v61: 97 pgs: 97 active+clean; 27 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 396 op/s 2024-04-26T01:55:36.068 INFO:tasks.workunit.client.0.smithi151.stderr:Updating files: 7% (321/4290) Updating files: 8% (344/4290) Updating files: 9% (387/4290) Updating files: 10% (429/4290) Updating files: 10% (449/4290) Updating files: 11% (472/4290) Updating files: 12% (515/4290) Updating files: 13% (558/4290) Updating files: 14% (601/4290) Updating files: 15% (644/4290) Updating files: 16% (687/4290) Updating files: 17% (730/4290) Updating files: 18% (773/4290) Updating files: 19% (816/4290) Updating files: 20% (858/4290) Updating files: 20% (860/4290) Updating files: 21% (901/4290) Updating files: 22% (944/4290) Updating files: 23% (987/4290) Updating files: 24% (1030/4290) Updating files: 25% (1073/4290) Updating files: 26% (1116/4290) Updating files: 27% (1159/4290) Updating files: 28% (1202/4290) Updating files: 29% (1245/4290) Updating files: 30% (1287/4290) Updating files: 30% (1288/4290) Updating files: 31% (1330/4290) Updating files: 32% (1373/4290) Updating files: 33% (1416/4290) Updating files: 34% (1459/4290) Updating files: 35% (1502/4290) Updating files: 35% (1505/4290) Updating files: 36% (1545/4290) Updating files: 37% (1588/4290) Updating files: 38% (1631/4290) Updating files: 39% (1674/4290) Updating files: 40% (1716/4290) Updating files: 41% (1759/4290) Updating files: 42% (1802/4290) Updating files: 43% (1845/4290) Updating files: 44% (1888/4290) Updating files: 45% (1931/4290) Updating files: 46% (1974/4290) Updating files: 47% (2017/4290) Updating files: 47% (2051/4290) Updating files: 48% (2060/4290) Updating files: 49% (2103/4290) Updating files: 50% (2145/4290) Updating files: 51% (2188/4290) Updating files: 52% (2231/4290) Updating files: 53% (2274/4290) Updating files: 54% (2317/4290) Updating files: 55% (2360/4290) Updating files: 55% (2395/4290) Updating files: 56% (2403/4290) Updating files: 57% (2446/4290) Updating files: 58% (2489/4290) Updating files: 59% (2532/4290) Updating files: 60% (2574/4290) Updating files: 61% (2617/4290) Updating files: 62% (2660/4290) Updating files: 63% (2703/4290) Updating files: 64% (2746/4290) Updating files: 64% (2784/4290) Updating files: 65% (2789/4290) Updating files: 66% (2832/4290) Updating files: 67% (2875/4290) Updating files: 68% (2918/4290) Updating files: 69% (2961/4290) Updating files: 70% (3003/4290) Updating files: 70% (3022/4290) Updating files: 71% (3046/4290) Updating files: 72% (3089/4290) Updating files: 73% (3132/4290) Updating files: 73% (3161/4290) Updating files: 74% (3175/4290) Updating files: 75% (3218/4290) Updating files: 76% (3261/4290) Updating files: 77% (3304/4290) Updating files: 78% (3347/4290) Updating files: 79% (3390/4290) Updating files: 80% (3432/4290) Updating files: 81% (3475/4290) Updating files: 82% (3518/4290) Updating files: 83% (3561/4290) Updating files: 84% (3604/4290) Updating files: 85% (3647/4290) Updating files: 86% (3690/4290) Updating files: 86% (3720/4290) Updating files: 87% (3733/4290) Updating files: 88% (3776/4290) Updating files: 89% (3819/4290) Updating files: 90% (3861/4290) Updating files: 91% (3904/4290) Updating files: 92% (3947/4290) Updating files: 93% (3990/4290) Updating files: 94% (4033/4290) Updating files: 94% (4059/4290) Updating files: 95% (4076/4290) Updating files: 96% (4119/4290) Updating files: 97% (4162/4290) Updating files: 98% (4205/4290) Updating files: 99% (4248/4290) Updating files: 100% (4290/4290) Updating files: 100% (4290/4290), done. 2024-04-26T01:55:36.088 INFO:tasks.workunit.client.0.smithi151.stderr:+ cd xfstests-dev 2024-04-26T01:55:36.088 INFO:tasks.workunit.client.0.smithi151.stderr:+ make -j4 2024-04-26T01:55:36.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:55:35 smithi165 ceph-mon[32648]: pgmap v62: 97 pgs: 97 active+clean; 27 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 1.3 MiB/s wr, 390 op/s 2024-04-26T01:55:36.108 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:55:35 smithi172 ceph-mon[32785]: pgmap v62: 97 pgs: 97 active+clean; 27 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 1.3 MiB/s wr, 390 op/s 2024-04-26T01:55:36.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:55:35 smithi151 ceph-mon[28057]: pgmap v62: 97 pgs: 97 active+clean; 27 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 1.3 MiB/s wr, 390 op/s 2024-04-26T01:55:36.144 INFO:tasks.workunit.client.0.smithi151.stdout:libtoolize -cfi 2024-04-26T01:55:36.960 INFO:tasks.workunit.client.0.smithi151.stdout:libtoolize: putting auxiliary files in '.'. 2024-04-26T01:55:36.960 INFO:tasks.workunit.client.0.smithi151.stdout:libtoolize: copying file './config.guess' 2024-04-26T01:55:36.974 INFO:tasks.workunit.client.0.smithi151.stdout:libtoolize: copying file './config.sub' 2024-04-26T01:55:36.987 INFO:tasks.workunit.client.0.smithi151.stdout:libtoolize: copying file './install-sh' 2024-04-26T01:55:37.002 INFO:tasks.workunit.client.0.smithi151.stdout:libtoolize: copying file './ltmain.sh' 2024-04-26T01:55:37.084 INFO:tasks.workunit.client.0.smithi151.stdout:libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. 2024-04-26T01:55:37.084 INFO:tasks.workunit.client.0.smithi151.stdout:libtoolize: copying file 'm4/libtool.m4' 2024-04-26T01:55:37.102 INFO:tasks.workunit.client.0.smithi151.stdout:libtoolize: copying file 'm4/ltoptions.m4' 2024-04-26T01:55:37.120 INFO:tasks.workunit.client.0.smithi151.stdout:libtoolize: copying file 'm4/ltsugar.m4' 2024-04-26T01:55:37.138 INFO:tasks.workunit.client.0.smithi151.stdout:libtoolize: copying file 'm4/ltversion.m4' 2024-04-26T01:55:37.157 INFO:tasks.workunit.client.0.smithi151.stdout:libtoolize: copying file 'm4/lt~obsolete.m4' 2024-04-26T01:55:37.181 INFO:tasks.workunit.client.0.smithi151.stdout:libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am. 2024-04-26T01:55:37.185 INFO:tasks.workunit.client.0.smithi151.stdout:cp include/install-sh . 2024-04-26T01:55:37.228 INFO:tasks.workunit.client.0.smithi151.stdout:aclocal -I m4 2024-04-26T01:55:38.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:55:37 smithi151 ceph-mon[28057]: pgmap v63: 97 pgs: 97 active+clean; 27 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 1.3 MiB/s wr, 390 op/s 2024-04-26T01:55:38.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:55:37 smithi172 ceph-mon[32785]: pgmap v63: 97 pgs: 97 active+clean; 27 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 1.3 MiB/s wr, 390 op/s 2024-04-26T01:55:38.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:55:37 smithi165 ceph-mon[32648]: pgmap v63: 97 pgs: 97 active+clean; 27 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 1.3 MiB/s wr, 390 op/s 2024-04-26T01:55:39.102 INFO:tasks.workunit.client.0.smithi151.stdout:autoheader 2024-04-26T01:55:39.376 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.377+0000 7f0a92a00640 1 -- 172.21.15.151:0/988581583 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a8c10c340 msgr2=0x7f0a8c10e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:39.376 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.377+0000 7f0a92a00640 1 --2- 172.21.15.151:0/988581583 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a8c10c340 0x7f0a8c10e7c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0a84009ca0 tx=0x7f0a8402a950 comp rx=0 tx=0).stop 2024-04-26T01:55:39.376 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.377+0000 7f0a92a00640 1 -- 172.21.15.151:0/988581583 shutdown_connections 2024-04-26T01:55:39.376 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.377+0000 7f0a92a00640 1 --2- 172.21.15.151:0/988581583 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a8c10c340 0x7f0a8c10e7c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:39.376 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.377+0000 7f0a92a00640 1 --2- 172.21.15.151:0/988581583 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a8c103890 0x7f0a8c10bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:39.377 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.377+0000 7f0a92a00640 1 --2- 172.21.15.151:0/988581583 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a8c102f70 0x7f0a8c103350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:39.377 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.377+0000 7f0a92a00640 1 -- 172.21.15.151:0/988581583 >> 172.21.15.151:0/988581583 conn(0x7f0a8c0fc770 msgr2=0x7f0a8c0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:39.377 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.377+0000 7f0a92a00640 1 -- 172.21.15.151:0/988581583 shutdown_connections 2024-04-26T01:55:39.377 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.377+0000 7f0a92a00640 1 -- 172.21.15.151:0/988581583 wait complete. 2024-04-26T01:55:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.378+0000 7f0a92a00640 1 Processor -- start 2024-04-26T01:55:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.378+0000 7f0a92a00640 1 -- start start 2024-04-26T01:55:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.378+0000 7f0a92a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a8c102f70 0x7f0a8c198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.378+0000 7f0a92a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a8c103890 0x7f0a8c198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.379+0000 7f0a92a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a8c10c340 0x7f0a8c19ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.379+0000 7f0a92a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0a8c111fb0 con 0x7f0a8c10c340 2024-04-26T01:55:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.379+0000 7f0a92a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0a8c111e30 con 0x7f0a8c102f70 2024-04-26T01:55:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.379+0000 7f0a92a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0a8c112130 con 0x7f0a8c103890 2024-04-26T01:55:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.379+0000 7f0a91600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a8c102f70 0x7f0a8c198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.379+0000 7f0a91600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a8c102f70 0x7f0a8c198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:46708/0 (socket says 172.21.15.151:46708) 2024-04-26T01:55:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.379+0000 7f0a91600640 1 -- 172.21.15.151:0/2158562628 learned_addr learned my addr 172.21.15.151:0/2158562628 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:55:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.379+0000 7f0a92000640 1 --2- 172.21.15.151:0/2158562628 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a8c10c340 0x7f0a8c19ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.379+0000 7f0a90c00640 1 --2- 172.21.15.151:0/2158562628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a8c103890 0x7f0a8c198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.379+0000 7f0a91600640 1 -- 172.21.15.151:0/2158562628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a8c103890 msgr2=0x7f0a8c198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.379+0000 7f0a91600640 1 --2- 172.21.15.151:0/2158562628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a8c103890 0x7f0a8c198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.379+0000 7f0a91600640 1 -- 172.21.15.151:0/2158562628 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a8c10c340 msgr2=0x7f0a8c19ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.379+0000 7f0a91600640 1 --2- 172.21.15.151:0/2158562628 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a8c10c340 0x7f0a8c19ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.379+0000 7f0a91600640 1 -- 172.21.15.151:0/2158562628 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a8c19d540 con 0x7f0a8c102f70 2024-04-26T01:55:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.380+0000 7f0a91600640 1 --2- 172.21.15.151:0/2158562628 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a8c102f70 0x7f0a8c198550 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0a8000e9f0 tx=0x7f0a8000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.380+0000 7f0a8a000640 1 -- 172.21.15.151:0/2158562628 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8000cd80 con 0x7f0a8c102f70 2024-04-26T01:55:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.380+0000 7f0a8a000640 1 -- 172.21.15.151:0/2158562628 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a800054d0 con 0x7f0a8c102f70 2024-04-26T01:55:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.380+0000 7f0a92a00640 1 -- 172.21.15.151:0/2158562628 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a8c19d830 con 0x7f0a8c102f70 2024-04-26T01:55:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.380+0000 7f0a8a000640 1 -- 172.21.15.151:0/2158562628 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a80010690 con 0x7f0a8c102f70 2024-04-26T01:55:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.380+0000 7f0a92a00640 1 -- 172.21.15.151:0/2158562628 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a8c1a50c0 con 0x7f0a8c102f70 2024-04-26T01:55:39.383 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.384+0000 7f0a96fbd740 1 Processor -- start 2024-04-26T01:55:39.383 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.385+0000 7f0a96fbd740 1 -- start start 2024-04-26T01:55:39.390 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.385+0000 7f0a8a000640 1 -- 172.21.15.151:0/2158562628 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f0a80004e90 con 0x7f0a8c102f70 2024-04-26T01:55:39.390 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.385+0000 7f0a96fbd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0f46ac9b0 0x55d0f46acd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:39.390 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.385+0000 7f0a96fbd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0f46ad2d0 0x55d0f46b7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:39.390 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.385+0000 7f0a96fbd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0f46b7b90 0x55d0f46b9f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:39.391 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.385+0000 7f0a96fbd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d0f4612900 con 0x55d0f46ac9b0 2024-04-26T01:55:39.391 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.385+0000 7f0a96fbd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d0f45b8ef0 con 0x55d0f46ad2d0 2024-04-26T01:55:39.391 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.385+0000 7f0a96fbd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d0f460e8c0 con 0x55d0f46b7b90 2024-04-26T01:55:39.391 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.385+0000 7f0a92000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0f46ac9b0 0x55d0f46acd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:39.391 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.385+0000 7f0a91600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0f46ad2d0 0x55d0f46b7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:39.391 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.386+0000 7f0a90c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0f46b7b90 0x55d0f46b9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:39.391 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.386+0000 7f0a92000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0f46ac9b0 0x55d0f46acd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:48124/0 (socket says 172.21.15.151:48124) 2024-04-26T01:55:39.391 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.386+0000 7f0a92000640 1 -- 172.21.15.151:0/4141980399 learned_addr learned my addr 172.21.15.151:0/4141980399 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:55:39.391 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.386+0000 7f0a91600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0f46ad2d0 0x55d0f46b7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:46722/0 (socket says 172.21.15.151:46722) 2024-04-26T01:55:39.391 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.386+0000 7f0a90c00640 1 -- 172.21.15.151:0/4141980399 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0f46ad2d0 msgr2=0x55d0f46b7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:39.392 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.386+0000 7f0a90c00640 1 --2- 172.21.15.151:0/4141980399 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0f46ad2d0 0x55d0f46b7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:39.392 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.386+0000 7f0a90c00640 1 -- 172.21.15.151:0/4141980399 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0f46ac9b0 msgr2=0x55d0f46acd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:39.392 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.386+0000 7f0a90c00640 1 --2- 172.21.15.151:0/4141980399 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0f46ac9b0 0x55d0f46acd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:39.392 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.386+0000 7f0a90c00640 1 -- 172.21.15.151:0/4141980399 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0f4325210 con 0x55d0f46b7b90 2024-04-26T01:55:39.392 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.386+0000 7f0a90c00640 1 --2- 172.21.15.151:0/4141980399 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0f46b7b90 0x55d0f46b9f80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0a78009bf0 tx=0x7f0a78011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f909769f5473ba31 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:39.392 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.386+0000 7f0a7e600640 1 -- 172.21.15.151:0/4141980399 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a78004050 con 0x55d0f46b7b90 2024-04-26T01:55:39.392 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.387+0000 7f0a7e600640 1 -- 172.21.15.151:0/4141980399 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a780054d0 con 0x55d0f46b7b90 2024-04-26T01:55:39.392 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.387+0000 7f0a7e600640 1 -- 172.21.15.151:0/4141980399 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a78004c60 con 0x55d0f46b7b90 2024-04-26T01:55:39.392 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.387+0000 7f0a8a000640 1 --2- 172.21.15.151:0/2158562628 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f0a740a3d90 0x7f0a740a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:39.392 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.387+0000 7f0a8a000640 1 -- 172.21.15.151:0/2158562628 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f0a800d0b00 con 0x7f0a8c102f70 2024-04-26T01:55:39.392 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.387+0000 7f0a92000640 1 --2- 172.21.15.151:0/2158562628 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f0a740a3d90 0x7f0a740a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:39.392 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.387+0000 7f0a96fbd740 1 -- 172.21.15.151:0/4141980399 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0f46b7b90 msgr2=0x55d0f46b9f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:39.393 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.387+0000 7f0a96fbd740 1 --2- 172.21.15.151:0/4141980399 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0f46b7b90 0x55d0f46b9f80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0a78009bf0 tx=0x7f0a78011810 comp rx=0 tx=0).stop 2024-04-26T01:55:39.393 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.388+0000 7f0a96fbd740 1 -- 172.21.15.151:0/4141980399 shutdown_connections 2024-04-26T01:55:39.393 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.388+0000 7f0a96fbd740 1 --2- 172.21.15.151:0/4141980399 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0f46b7b90 0x55d0f46b9f80 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:39.393 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.388+0000 7f0a96fbd740 1 --2- 172.21.15.151:0/4141980399 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0f46ad2d0 0x55d0f46b7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:39.393 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.388+0000 7f0a96fbd740 1 --2- 172.21.15.151:0/4141980399 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0f46ac9b0 0x55d0f46acd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:39.393 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.388+0000 7f0a96fbd740 1 -- 172.21.15.151:0/4141980399 >> 172.21.15.151:0/4141980399 conn(0x55d0f46a8ec0 msgr2=0x55d0f46a92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:39.393 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.388+0000 7f0a96fbd740 1 -- 172.21.15.151:0/4141980399 shutdown_connections 2024-04-26T01:55:39.393 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.388+0000 7f0a92000640 1 --2- 172.21.15.151:0/2158562628 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f0a740a3d90 0x7f0a740a6250 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0a84009ca0 tx=0x7f0a84005c50 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:39.393 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.388+0000 7f0a96fbd740 1 -- 172.21.15.151:0/4141980399 wait complete. 2024-04-26T01:55:39.393 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.388+0000 7f0a96fbd740 1 -- start start 2024-04-26T01:55:39.393 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.388+0000 7f0a96fbd740 1 Processor -- start 2024-04-26T01:55:39.393 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.389+0000 7f0a96fbd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0f46ac9b0 0x55d0f46b4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:39.394 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.389+0000 7f0a96fbd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0f46ad2d0 0x55d0f46b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:39.394 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.389+0000 7f0a96fbd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0f46b7b90 0x55d0f46b5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:39.394 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.389+0000 7f0a96fbd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d0f46ba7c0 con 0x55d0f46b7b90 2024-04-26T01:55:39.394 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.389+0000 7f0a96fbd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d0f45cc310 con 0x55d0f46ad2d0 2024-04-26T01:55:39.394 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.389+0000 7f0a96fbd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d0f46ba940 con 0x55d0f46ac9b0 2024-04-26T01:55:39.394 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.389+0000 7f0a96fbd740 10 client.0 _ll_register_callbacks cb 0x55d0f459acb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:55:39.394 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.389+0000 7f0a90c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0f46ad2d0 0x55d0f46b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:39.394 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.389+0000 7f0a92000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0f46b7b90 0x55d0f46b5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:39.394 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.389+0000 7f0a91600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0f46ac9b0 0x55d0f46b4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.389+0000 7f0a90c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0f46ad2d0 0x55d0f46b4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:46738/0 (socket says 172.21.15.151:46738) 2024-04-26T01:55:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.389+0000 7f0a92000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0f46b7b90 0x55d0f46b5500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:48128/0 (socket says 172.21.15.151:48128) 2024-04-26T01:55:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.389+0000 7f0a90c00640 1 -- 172.21.15.151:0/1813857926 learned_addr learned my addr 172.21.15.151:0/1813857926 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:55:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.390+0000 7f0a90c00640 1 -- 172.21.15.151:0/1813857926 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0f46ac9b0 msgr2=0x55d0f46b4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.390+0000 7f0a90c00640 1 --2- 172.21.15.151:0/1813857926 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0f46ac9b0 0x55d0f46b4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.390+0000 7f0a90c00640 1 -- 172.21.15.151:0/1813857926 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0f46b7b90 msgr2=0x55d0f46b5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.390+0000 7f0a90c00640 1 --2- 172.21.15.151:0/1813857926 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0f46b7b90 0x55d0f46b5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.390+0000 7f0a90c00640 1 -- 172.21.15.151:0/1813857926 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0f43f1620 con 0x55d0f46ad2d0 2024-04-26T01:55:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.390+0000 7f0a91600640 1 --2- 172.21.15.151:0/1813857926 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0f46ac9b0 0x55d0f46b4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:55:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.390+0000 7f0a92000640 1 --2- 172.21.15.151:0/1813857926 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0f46b7b90 0x55d0f46b5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:55:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.390+0000 7f0a90c00640 1 --2- 172.21.15.151:0/1813857926 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0f46ad2d0 0x55d0f46b4fc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0a78004650 tx=0x7f0a78009fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5387ec82b37b47a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:39.396 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.390+0000 7f0a72a00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T01:55:39.396 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.390+0000 7f0a72a00640 1 -- 172.21.15.151:0/1813857926 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a78029070 con 0x55d0f46ad2d0 2024-04-26T01:55:39.396 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.391+0000 7f0a72a00640 1 -- 172.21.15.151:0/1813857926 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f0a78003a00 con 0x55d0f46ad2d0 2024-04-26T01:55:39.396 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.391+0000 7f0a96fbd740 1 -- 172.21.15.151:0/1813857926 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a780054d0 con 0x55d0f46ad2d0 2024-04-26T01:55:39.396 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.391+0000 7f0a72a00640 1 -- 172.21.15.151:0/1813857926 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0a78035050 con 0x55d0f46ad2d0 2024-04-26T01:55:39.396 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.391+0000 7f0a96fbd740 10 client.25418 fetch_fsmap learned FSMap version 28 2024-04-26T01:55:39.396 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.391+0000 7f0a96fbd740 1 -- 172.21.15.151:0/1813857926 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d0f46c6980 con 0x55d0f46ad2d0 2024-04-26T01:55:39.396 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.391+0000 7f0a72a00640 1 -- 172.21.15.151:0/1813857926 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a780054d0 con 0x55d0f46ad2d0 2024-04-26T01:55:39.396 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.391+0000 7f0a72a00640 1 -- 172.21.15.151:0/1813857926 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f0a78004f20 con 0x55d0f46ad2d0 2024-04-26T01:55:39.396 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.391+0000 7f0a96fbd740 10 client.25418 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T01:55:39.396 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.391+0000 7f0a96fbd740 10 client.25418 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:55:39.396 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.392+0000 7f0a96fbd740 20 client.25418 populate_metadata read hostname 'smithi151' 2024-04-26T01:55:39.396 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.392+0000 7f0a96fbd740 1 --2- 172.21.15.151:0/1813857926 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55d0f46c7d30 0x55d0f46ca120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:39.397 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.392+0000 7f0a96fbd740 4 client.25418 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:55:39.397 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.392+0000 7f0a96fbd740 1 -- 172.21.15.151:0/1813857926 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0f46268b0 con 0x55d0f46c7d30 2024-04-26T01:55:39.397 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.392+0000 7f0a91600640 1 --2- 172.21.15.151:0/1813857926 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55d0f46c7d30 0x55d0f46ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:39.398 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.392+0000 7f0a91600640 1 --2- 172.21.15.151:0/1813857926 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55d0f46c7d30 0x55d0f46ca120 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a563105380058c2 server_cookie=f1b1c0a052f799f3 in_seq=0 out_seq=0 2024-04-26T01:55:39.398 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.392+0000 7f0a72a00640 10 client.25418 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:55:39.398 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.393+0000 7f0a72a00640 1 -- 172.21.15.151:0/1813857926 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d0f46268b0 con 0x55d0f46c7d30 2024-04-26T01:55:39.398 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.393+0000 7f0a72a00640 10 client.25418 handle_command_reply: tid=0 2024-04-26T01:55:39.398 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.393+0000 7f0a96fbd740 1 client.25418 shutdown 2024-04-26T01:55:39.398 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.393+0000 7f0a96fbd740 1 -- 172.21.15.151:0/1813857926 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0f46ad2d0 msgr2=0x55d0f46b4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:39.398 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.393+0000 7f0a96fbd740 1 --2- 172.21.15.151:0/1813857926 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0f46ad2d0 0x55d0f46b4fc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0a78004650 tx=0x7f0a78009fd0 comp rx=0 tx=0).stop 2024-04-26T01:55:39.399 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.393+0000 7f0a96fbd740 1 -- 172.21.15.151:0/1813857926 shutdown_connections 2024-04-26T01:55:39.399 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.393+0000 7f0a96fbd740 1 --2- 172.21.15.151:0/1813857926 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55d0f46c7d30 0x55d0f46ca120 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:39.399 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.393+0000 7f0a96fbd740 1 --2- 172.21.15.151:0/1813857926 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0f46b7b90 0x55d0f46b5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:39.399 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.393+0000 7f0a96fbd740 1 --2- 172.21.15.151:0/1813857926 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0f46ad2d0 0x55d0f46b4fc0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:39.399 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.393+0000 7f0a96fbd740 1 --2- 172.21.15.151:0/1813857926 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0f46ac9b0 0x55d0f46b4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:39.399 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.393+0000 7f0a72a00640 0 client.25418 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:55:39.399 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.393+0000 7f0a96fbd740 1 -- 172.21.15.151:0/1813857926 >> 172.21.15.151:0/1813857926 conn(0x55d0f46a8ec0 msgr2=0x55d0f46aec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:39.399 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.394+0000 7f0a96fbd740 1 -- 172.21.15.151:0/1813857926 shutdown_connections 2024-04-26T01:55:39.399 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.394+0000 7f0a96fbd740 1 -- 172.21.15.151:0/1813857926 wait complete. 2024-04-26T01:55:39.399 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.394+0000 7f0a96fbd740 20 client.25418 trim_cache size 0 max 16384 2024-04-26T01:55:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.410+0000 7f0a96fbd740 1 Processor -- start 2024-04-26T01:55:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.410+0000 7f0a96fbd740 1 -- start start 2024-04-26T01:55:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.410+0000 7f0a96fbd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0f46ac9b0 0x55d0f46c3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.410+0000 7f0a96fbd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0f46ad2d0 0x55d0f46af370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.410+0000 7f0a96fbd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0f46b7b90 0x55d0f46a7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.410+0000 7f0a96fbd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0a8c1b0170 con 0x55d0f46b7b90 2024-04-26T01:55:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.410+0000 7f0a96fbd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0a8c1ada40 con 0x55d0f46ad2d0 2024-04-26T01:55:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.410+0000 7f0a96fbd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0a8c1ab8b0 con 0x55d0f46ac9b0 2024-04-26T01:55:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.410+0000 7f0a92000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0f46b7b90 0x55d0f46a7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.410+0000 7f0a90c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0f46ad2d0 0x55d0f46af370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.410+0000 7f0a91600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0f46ac9b0 0x55d0f46c3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.410+0000 7f0a90c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0f46ad2d0 0x55d0f46af370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:46746/0 (socket says 172.21.15.151:46746) 2024-04-26T01:55:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.410+0000 7f0a92000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0f46b7b90 0x55d0f46a7ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:48134/0 (socket says 172.21.15.151:48134) 2024-04-26T01:55:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.410+0000 7f0a90c00640 1 -- 172.21.15.151:0/2844189141 learned_addr learned my addr 172.21.15.151:0/2844189141 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:55:39.415 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.411+0000 7f0a90c00640 1 -- 172.21.15.151:0/2844189141 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0f46ac9b0 msgr2=0x55d0f46c3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:39.415 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.411+0000 7f0a90c00640 1 --2- 172.21.15.151:0/2844189141 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0f46ac9b0 0x55d0f46c3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:39.415 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.411+0000 7f0a90c00640 1 -- 172.21.15.151:0/2844189141 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0f46b7b90 msgr2=0x55d0f46a7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:39.415 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.411+0000 7f0a90c00640 1 --2- 172.21.15.151:0/2844189141 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0f46b7b90 0x55d0f46a7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:39.415 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.411+0000 7f0a90c00640 1 -- 172.21.15.151:0/2844189141 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0f46b2fd0 con 0x55d0f46ad2d0 2024-04-26T01:55:39.416 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.411+0000 7f0a90c00640 1 --2- 172.21.15.151:0/2844189141 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0f46ad2d0 0x55d0f46af370 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0a78009b40 tx=0x7f0a78023480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d11fc261b3bfcb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:39.416 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.411+0000 7f0a7dc00640 1 -- 172.21.15.151:0/2844189141 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a78029070 con 0x55d0f46ad2d0 2024-04-26T01:55:39.416 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.411+0000 7f0a7dc00640 1 -- 172.21.15.151:0/2844189141 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a78023dd0 con 0x55d0f46ad2d0 2024-04-26T01:55:39.416 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.411+0000 7f0a7dc00640 1 -- 172.21.15.151:0/2844189141 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a78003a40 con 0x55d0f46ad2d0 2024-04-26T01:55:39.416 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.412+0000 7f0a96fbd740 1 -- 172.21.15.151:0/2844189141 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0f46ad2d0 msgr2=0x55d0f46af370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:39.416 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.412+0000 7f0a96fbd740 1 --2- 172.21.15.151:0/2844189141 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0f46ad2d0 0x55d0f46af370 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0a78009b40 tx=0x7f0a78023480 comp rx=0 tx=0).stop 2024-04-26T01:55:39.416 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.412+0000 7f0a96fbd740 1 -- 172.21.15.151:0/2844189141 shutdown_connections 2024-04-26T01:55:39.416 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.412+0000 7f0a96fbd740 1 --2- 172.21.15.151:0/2844189141 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0f46b7b90 0x55d0f46a7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:39.416 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.412+0000 7f0a96fbd740 1 --2- 172.21.15.151:0/2844189141 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0f46ad2d0 0x55d0f46af370 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:39.416 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.412+0000 7f0a96fbd740 1 --2- 172.21.15.151:0/2844189141 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0f46ac9b0 0x55d0f46c3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:39.416 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.412+0000 7f0a96fbd740 1 -- 172.21.15.151:0/2844189141 >> 172.21.15.151:0/2844189141 conn(0x55d0f46a8ec0 msgr2=0x55d0f46a7790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:39.417 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.412+0000 7f0a96fbd740 1 -- 172.21.15.151:0/2844189141 shutdown_connections 2024-04-26T01:55:39.417 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.412+0000 7f0a96fbd740 1 -- 172.21.15.151:0/2844189141 wait complete. 2024-04-26T01:55:39.417 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.412+0000 7f0a96fbd740 1 -- start start 2024-04-26T01:55:39.417 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.413+0000 7f0a96fbd740 1 Processor -- start 2024-04-26T01:55:39.417 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.413+0000 7f0a96fbd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0f46ac9b0 0x55d0f46bfdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:39.417 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.413+0000 7f0a96fbd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0f46ad2d0 0x55d0f4727860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:39.417 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.413+0000 7f0a96fbd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0f46b7b90 0x55d0f4727da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:39.417 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.413+0000 7f0a96fbd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0a8c1abf00 con 0x55d0f46ad2d0 2024-04-26T01:55:39.417 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.413+0000 7f0a96fbd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0a8c1ad370 con 0x55d0f46ac9b0 2024-04-26T01:55:39.417 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.413+0000 7f0a96fbd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0a8c1ac5d0 con 0x55d0f46b7b90 2024-04-26T01:55:39.418 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.413+0000 7f0a96fbd740 10 client.0 _ll_register_callbacks cb 0x7f0a78009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:55:39.418 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.413+0000 7f0a91600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0f46ac9b0 0x55d0f46bfdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:39.418 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.413+0000 7f0a90c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0f46ad2d0 0x55d0f4727860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:39.418 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.413+0000 7f0a92000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0f46b7b90 0x55d0f4727da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:39.418 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.414+0000 7f0a91600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0f46ac9b0 0x55d0f46bfdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:46748/0 (socket says 172.21.15.151:46748) 2024-04-26T01:55:39.418 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.414+0000 7f0a91600640 1 -- 172.21.15.151:0/4155037172 learned_addr learned my addr 172.21.15.151:0/4155037172 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:55:39.418 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.414+0000 7f0a90c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0f46ad2d0 0x55d0f4727860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:48148/0 (socket says 172.21.15.151:48148) 2024-04-26T01:55:39.418 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.414+0000 7f0a92000640 1 -- 172.21.15.151:0/4155037172 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0f46ac9b0 msgr2=0x55d0f46bfdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:39.418 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.414+0000 7f0a92000640 1 --2- 172.21.15.151:0/4155037172 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0f46ac9b0 0x55d0f46bfdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:39.418 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.414+0000 7f0a92000640 1 -- 172.21.15.151:0/4155037172 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0f46ad2d0 msgr2=0x55d0f4727860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:39.419 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.414+0000 7f0a92000640 1 --2- 172.21.15.151:0/4155037172 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0f46ad2d0 0x55d0f4727860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:39.419 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.414+0000 7f0a92000640 1 -- 172.21.15.151:0/4155037172 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0f46beb60 con 0x55d0f46b7b90 2024-04-26T01:55:39.419 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.414+0000 7f0a90c00640 1 --2- 172.21.15.151:0/4155037172 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0f46ad2d0 0x55d0f4727860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:55:39.419 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.414+0000 7f0a92000640 1 --2- 172.21.15.151:0/4155037172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0f46b7b90 0x55d0f4727da0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0a84005e30 tx=0x7f0a840095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9592240304ae56d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:39.419 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.414+0000 7f0a72a00640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-26T01:55:39.419 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.415+0000 7f0a72a00640 1 -- 172.21.15.151:0/4155037172 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a84034040 con 0x55d0f46b7b90 2024-04-26T01:55:39.419 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.415+0000 7f0a96fbd740 1 -- 172.21.15.151:0/4155037172 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a78023dd0 con 0x55d0f46b7b90 2024-04-26T01:55:39.419 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.415+0000 7f0a72a00640 1 -- 172.21.15.151:0/4155037172 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f0a84003e30 con 0x55d0f46b7b90 2024-04-26T01:55:39.419 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.415+0000 7f0a72a00640 1 -- 172.21.15.151:0/4155037172 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0a840046a0 con 0x55d0f46b7b90 2024-04-26T01:55:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.415+0000 7f0a96fbd740 10 client.25444 fetch_fsmap learned FSMap version 28 2024-04-26T01:55:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.415+0000 7f0a96fbd740 1 -- 172.21.15.151:0/4155037172 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d0f4726340 con 0x55d0f46b7b90 2024-04-26T01:55:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.415+0000 7f0a72a00640 1 -- 172.21.15.151:0/4155037172 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a78023dd0 con 0x55d0f46b7b90 2024-04-26T01:55:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.416+0000 7f0a72a00640 1 -- 172.21.15.151:0/4155037172 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f0a8403a430 con 0x55d0f46b7b90 2024-04-26T01:55:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.416+0000 7f0a96fbd740 10 client.25444 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T01:55:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.416+0000 7f0a96fbd740 10 client.25444 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:55:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.416+0000 7f0a96fbd740 20 client.25444 populate_metadata read hostname 'smithi151' 2024-04-26T01:55:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.416+0000 7f0a96fbd740 1 --2- 172.21.15.151:0/4155037172 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55d0f4726ac0 0x55d0f4738d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.416+0000 7f0a96fbd740 4 client.25444 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:55:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.416+0000 7f0a96fbd740 1 -- 172.21.15.151:0/4155037172 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d0f46ae090 con 0x55d0f4726ac0 2024-04-26T01:55:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.416+0000 7f0a91600640 1 --2- 172.21.15.151:0/4155037172 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55d0f4726ac0 0x55d0f4738d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:39.421 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.417+0000 7f0a72a00640 10 client.25444 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:55:39.421 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.417+0000 7f0a91600640 1 --2- 172.21.15.151:0/4155037172 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55d0f4726ac0 0x55d0f4738d70 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5624e0d9c5b9ebfa server_cookie=7a65f7b28f3bb6e4 in_seq=0 out_seq=0 2024-04-26T01:55:39.909 INFO:tasks.workunit.client.0.smithi151.stdout:autoconf 2024-04-26T01:55:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.951+0000 7f0a72a00640 1 -- 172.21.15.151:0/4155037172 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d0f46ae090 con 0x55d0f4726ac0 2024-04-26T01:55:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.951+0000 7f0a72a00640 10 client.25444 handle_command_reply: tid=0 2024-04-26T01:55:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.952+0000 7f0a96fbd740 1 client.25444 shutdown 2024-04-26T01:55:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.952+0000 7f0a96fbd740 1 -- 172.21.15.151:0/4155037172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0f46b7b90 msgr2=0x55d0f4727da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.952+0000 7f0a96fbd740 1 --2- 172.21.15.151:0/4155037172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0f46b7b90 0x55d0f4727da0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0a84005e30 tx=0x7f0a840095c0 comp rx=0 tx=0).stop 2024-04-26T01:55:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.952+0000 7f0a96fbd740 1 -- 172.21.15.151:0/4155037172 shutdown_connections 2024-04-26T01:55:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.952+0000 7f0a96fbd740 1 --2- 172.21.15.151:0/4155037172 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55d0f4726ac0 0x55d0f4738d70 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.952+0000 7f0a96fbd740 1 --2- 172.21.15.151:0/4155037172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0f46b7b90 0x55d0f4727da0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.952+0000 7f0a96fbd740 1 --2- 172.21.15.151:0/4155037172 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0f46ad2d0 0x55d0f4727860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.952+0000 7f0a72a00640 0 client.25444 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:55:39.952 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.952+0000 7f0a96fbd740 1 --2- 172.21.15.151:0/4155037172 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0f46ac9b0 0x55d0f46bfdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:39.952 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.952+0000 7f0a96fbd740 1 -- 172.21.15.151:0/4155037172 >> 172.21.15.151:0/4155037172 conn(0x55d0f46a8ec0 msgr2=0x55d0f46a8020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:39.952 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.952+0000 7f0a96fbd740 1 -- 172.21.15.151:0/4155037172 shutdown_connections 2024-04-26T01:55:39.952 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.952+0000 7f0a96fbd740 1 -- 172.21.15.151:0/4155037172 wait complete. 2024-04-26T01:55:39.952 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.952+0000 7f0a96fbd740 20 client.25444 trim_cache size 0 max 16384 2024-04-26T01:55:39.952 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:{ 2024-04-26T01:55:39.952 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "message": "", 2024-04-26T01:55:39.952 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "return_code": 0 2024-04-26T01:55:39.952 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:} 2024-04-26T01:55:39.953 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.953+0000 7f0a92a00640 1 -- 172.21.15.151:0/2158562628 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f0a740a3d90 msgr2=0x7f0a740a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:39.953 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.953+0000 7f0a92a00640 1 --2- 172.21.15.151:0/2158562628 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f0a740a3d90 0x7f0a740a6250 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0a84009ca0 tx=0x7f0a84005c50 comp rx=0 tx=0).stop 2024-04-26T01:55:39.953 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.953+0000 7f0a92a00640 1 -- 172.21.15.151:0/2158562628 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a8c102f70 msgr2=0x7f0a8c198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:39.954 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.953+0000 7f0a92a00640 1 --2- 172.21.15.151:0/2158562628 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a8c102f70 0x7f0a8c198550 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0a8000e9f0 tx=0x7f0a8000edf0 comp rx=0 tx=0).stop 2024-04-26T01:55:39.954 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.953+0000 7f0a92a00640 1 -- 172.21.15.151:0/2158562628 shutdown_connections 2024-04-26T01:55:39.954 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.953+0000 7f0a92a00640 1 --2- 172.21.15.151:0/2158562628 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f0a740a3d90 0x7f0a740a6250 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:39.954 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.953+0000 7f0a92a00640 1 --2- 172.21.15.151:0/2158562628 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a8c10c340 0x7f0a8c19ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:39.954 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.953+0000 7f0a92a00640 1 --2- 172.21.15.151:0/2158562628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a8c103890 0x7f0a8c198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:39.954 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.953+0000 7f0a92a00640 1 --2- 172.21.15.151:0/2158562628 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a8c102f70 0x7f0a8c198550 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:39.954 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.953+0000 7f0a92a00640 1 -- 172.21.15.151:0/2158562628 >> 172.21.15.151:0/2158562628 conn(0x7f0a8c0fc770 msgr2=0x7f0a8c0fe1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:39.954 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.953+0000 7f0a92a00640 1 -- 172.21.15.151:0/2158562628 shutdown_connections 2024-04-26T01:55:39.954 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:39.953+0000 7f0a92a00640 1 -- 172.21.15.151:0/2158562628 wait complete. 2024-04-26T01:55:40.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:55:39 smithi151 ceph-mon[28057]: pgmap v64: 97 pgs: 97 active+clean; 34 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 1019 KiB/s rd, 2.0 MiB/s wr, 550 op/s 2024-04-26T01:55:40.157 INFO:tasks.workunit.client.0.smithi151.stdout:./configure \ 2024-04-26T01:55:40.157 INFO:tasks.workunit.client.0.smithi151.stdout: --libexecdir=/usr/lib \ 2024-04-26T01:55:40.157 INFO:tasks.workunit.client.0.smithi151.stdout: --exec_prefix=/var/lib 2024-04-26T01:55:40.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:55:39 smithi172 ceph-mon[32785]: pgmap v64: 97 pgs: 97 active+clean; 34 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 1019 KiB/s rd, 2.0 MiB/s wr, 550 op/s 2024-04-26T01:55:40.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:55:39 smithi165 ceph-mon[32648]: pgmap v64: 97 pgs: 97 active+clean; 34 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 1019 KiB/s rd, 2.0 MiB/s wr, 550 op/s 2024-04-26T01:55:40.536 INFO:tasks.workunit.client.0.smithi151.stdout:checking build system type... x86_64-pc-linux-gnu 2024-04-26T01:55:40.536 INFO:tasks.workunit.client.0.smithi151.stdout:checking host system type... x86_64-pc-linux-gnu 2024-04-26T01:55:40.540 INFO:tasks.workunit.client.0.smithi151.stdout:checking how to print strings... printf 2024-04-26T01:55:40.540 INFO:tasks.workunit.client.0.smithi151.stdout:checking for gcc... gcc 2024-04-26T01:55:40.707 INFO:tasks.workunit.client.0.smithi151.stdout:checking whether the C compiler works... yes 2024-04-26T01:55:40.707 INFO:tasks.workunit.client.0.smithi151.stdout:checking for C compiler default output file name... a.out 2024-04-26T01:55:40.790 INFO:tasks.workunit.client.0.smithi151.stdout:checking for suffix of executables... 2024-04-26T01:55:40.873 INFO:tasks.workunit.client.0.smithi151.stdout:checking whether we are cross compiling... no 2024-04-26T01:55:40.946 INFO:tasks.workunit.client.0.smithi151.stdout:checking for suffix of object files... o 2024-04-26T01:55:41.004 INFO:tasks.workunit.client.0.smithi151.stdout:checking whether we are using the GNU C compiler... yes 2024-04-26T01:55:41.058 INFO:tasks.workunit.client.0.smithi151.stdout:checking whether gcc accepts -g... yes 2024-04-26T01:55:41.196 INFO:tasks.workunit.client.0.smithi151.stdout:checking for gcc option to accept ISO C89... none needed 2024-04-26T01:55:41.209 INFO:tasks.workunit.client.0.smithi151.stdout:checking for a sed that does not truncate output... /usr/bin/sed 2024-04-26T01:55:41.220 INFO:tasks.workunit.client.0.smithi151.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-04-26T01:55:41.224 INFO:tasks.workunit.client.0.smithi151.stdout:checking for egrep... /usr/bin/grep -E 2024-04-26T01:55:41.229 INFO:tasks.workunit.client.0.smithi151.stdout:checking for fgrep... /usr/bin/grep -F 2024-04-26T01:55:41.242 INFO:tasks.workunit.client.0.smithi151.stdout:checking for ld used by gcc... /usr/bin/ld 2024-04-26T01:55:41.248 INFO:tasks.workunit.client.0.smithi151.stdout:checking if the linker (/usr/bin/ld) is GNU ld... yes 2024-04-26T01:55:41.258 INFO:tasks.workunit.client.0.smithi151.stdout:checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B 2024-04-26T01:55:42.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:55:41 smithi151 ceph-mon[28057]: pgmap v65: 97 pgs: 97 active+clean; 34 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 1.6 MiB/s wr, 299 op/s 2024-04-26T01:55:42.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:55:41 smithi172 ceph-mon[32785]: pgmap v65: 97 pgs: 97 active+clean; 34 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 1.6 MiB/s wr, 299 op/s 2024-04-26T01:55:42.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:55:41 smithi165 ceph-mon[32648]: pgmap v65: 97 pgs: 97 active+clean; 34 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 1.6 MiB/s wr, 299 op/s 2024-04-26T01:55:42.388 INFO:tasks.workunit.client.0.smithi151.stdout:checking the name lister (/usr/bin/nm -B) interface... BSD nm 2024-04-26T01:55:42.388 INFO:tasks.workunit.client.0.smithi151.stdout:checking whether ln -s works... yes 2024-04-26T01:55:42.427 INFO:tasks.workunit.client.0.smithi151.stdout:checking the maximum length of command line arguments... 1572864 2024-04-26T01:55:42.437 INFO:tasks.workunit.client.0.smithi151.stdout:checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop 2024-04-26T01:55:42.438 INFO:tasks.workunit.client.0.smithi151.stdout:checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop 2024-04-26T01:55:42.438 INFO:tasks.workunit.client.0.smithi151.stdout:checking for /usr/bin/ld option to reload object files... -r 2024-04-26T01:55:42.439 INFO:tasks.workunit.client.0.smithi151.stdout:checking for objdump... objdump 2024-04-26T01:55:42.440 INFO:tasks.workunit.client.0.smithi151.stdout:checking how to recognize dependent libraries... pass_all 2024-04-26T01:55:42.442 INFO:tasks.workunit.client.0.smithi151.stdout:checking for dlltool... no 2024-04-26T01:55:42.442 INFO:tasks.workunit.client.0.smithi151.stdout:checking how to associate runtime and link libraries... printf %s\n 2024-04-26T01:55:42.443 INFO:tasks.workunit.client.0.smithi151.stdout:checking for ar... ar 2024-04-26T01:55:42.609 INFO:tasks.workunit.client.0.smithi151.stdout:checking for archiver @FILE support... @ 2024-04-26T01:55:42.610 INFO:tasks.workunit.client.0.smithi151.stdout:checking for strip... strip 2024-04-26T01:55:42.610 INFO:tasks.workunit.client.0.smithi151.stdout:checking for ranlib... ranlib 2024-04-26T01:55:42.611 INFO:tasks.workunit.client.0.smithi151.stdout:checking for gawk... gawk 2024-04-26T01:55:42.832 INFO:tasks.workunit.client.0.smithi151.stdout:checking command to parse /usr/bin/nm -B output from gcc object... ok 2024-04-26T01:55:42.836 INFO:tasks.workunit.client.0.smithi151.stdout:checking for sysroot... no 2024-04-26T01:55:42.857 INFO:tasks.workunit.client.0.smithi151.stdout:checking for a working dd... /usr/bin/dd 2024-04-26T01:55:42.878 INFO:tasks.workunit.client.0.smithi151.stdout:checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1 2024-04-26T01:55:42.943 INFO:tasks.workunit.client.0.smithi151.stdout:checking for mt... no 2024-04-26T01:55:42.969 INFO:tasks.workunit.client.0.smithi151.stdout:checking if : is a manifest tool... no 2024-04-26T01:55:43.120 INFO:tasks.workunit.client.0.smithi151.stdout:checking how to run the C preprocessor... gcc -E 2024-04-26T01:55:43.512 INFO:tasks.workunit.client.0.smithi151.stdout:checking for ANSI C header files... yes 2024-04-26T01:55:43.586 INFO:tasks.workunit.client.0.smithi151.stdout:checking for sys/types.h... yes 2024-04-26T01:55:43.678 INFO:tasks.workunit.client.0.smithi151.stdout:checking for sys/stat.h... yes 2024-04-26T01:55:43.765 INFO:tasks.workunit.client.0.smithi151.stdout:checking for stdlib.h... yes 2024-04-26T01:55:43.859 INFO:tasks.workunit.client.0.smithi151.stdout:checking for string.h... yes 2024-04-26T01:55:43.950 INFO:tasks.workunit.client.0.smithi151.stdout:checking for memory.h... yes 2024-04-26T01:55:44.045 INFO:tasks.workunit.client.0.smithi151.stdout:checking for strings.h... yes 2024-04-26T01:55:44.142 INFO:tasks.workunit.client.0.smithi151.stdout:checking for inttypes.h... yes 2024-04-26T01:55:44.238 INFO:tasks.workunit.client.0.smithi151.stdout:checking for stdint.h... yes 2024-04-26T01:55:44.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:55:43 smithi172 ceph-mon[32785]: pgmap v66: 97 pgs: 97 active+clean; 25 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 1013 KiB/s rd, 2.1 MiB/s wr, 863 op/s 2024-04-26T01:55:44.336 INFO:tasks.workunit.client.0.smithi151.stdout:checking for unistd.h... yes 2024-04-26T01:55:44.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:55:43 smithi165 ceph-mon[32648]: pgmap v66: 97 pgs: 97 active+clean; 25 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 1013 KiB/s rd, 2.1 MiB/s wr, 863 op/s 2024-04-26T01:55:44.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:55:43 smithi151 ceph-mon[28057]: pgmap v66: 97 pgs: 97 active+clean; 25 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 1013 KiB/s rd, 2.1 MiB/s wr, 863 op/s 2024-04-26T01:55:44.432 INFO:tasks.workunit.client.0.smithi151.stdout:checking for dlfcn.h... yes 2024-04-26T01:55:44.449 INFO:tasks.workunit.client.0.smithi151.stdout:checking for objdir... .libs 2024-04-26T01:55:44.694 INFO:tasks.workunit.client.0.smithi151.stdout:checking if gcc supports -fno-rtti -fno-exceptions... no 2024-04-26T01:55:44.694 INFO:tasks.workunit.client.0.smithi151.stdout:checking for gcc option to produce PIC... -fPIC -DPIC 2024-04-26T01:55:44.770 INFO:tasks.workunit.client.0.smithi151.stdout:checking if gcc PIC flag -fPIC -DPIC works... yes 2024-04-26T01:55:44.856 INFO:tasks.workunit.client.0.smithi151.stdout:checking if gcc static flag -static works... no 2024-04-26T01:55:44.987 INFO:tasks.workunit.client.0.smithi151.stdout:checking if gcc supports -c -o file.o... yes 2024-04-26T01:55:44.987 INFO:tasks.workunit.client.0.smithi151.stdout:checking if gcc supports -c -o file.o... (cached) yes 2024-04-26T01:55:45.013 INFO:tasks.workunit.client.0.smithi151.stdout:checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes 2024-04-26T01:55:45.116 INFO:tasks.workunit.client.0.smithi151.stdout:checking whether -lc should be explicitly linked in... no 2024-04-26T01:55:45.278 INFO:tasks.workunit.client.0.smithi151.stdout:checking dynamic linker characteristics... GNU/Linux ld.so 2024-04-26T01:55:45.278 INFO:tasks.workunit.client.0.smithi151.stdout:checking how to hardcode library paths into programs... immediate 2024-04-26T01:55:45.282 INFO:tasks.workunit.client.0.smithi151.stdout:checking whether stripping libraries is possible... yes 2024-04-26T01:55:45.282 INFO:tasks.workunit.client.0.smithi151.stdout:checking if libtool supports shared libraries... yes 2024-04-26T01:55:45.282 INFO:tasks.workunit.client.0.smithi151.stdout:checking whether to build shared libraries... yes 2024-04-26T01:55:45.282 INFO:tasks.workunit.client.0.smithi151.stdout:checking whether to build static libraries... yes 2024-04-26T01:55:45.298 INFO:tasks.workunit.client.0.smithi151.stdout:checking for gcc... (cached) gcc 2024-04-26T01:55:45.370 INFO:tasks.workunit.client.0.smithi151.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-04-26T01:55:45.370 INFO:tasks.workunit.client.0.smithi151.stdout:checking whether gcc accepts -g... (cached) yes 2024-04-26T01:55:45.371 INFO:tasks.workunit.client.0.smithi151.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-04-26T01:55:45.371 INFO:tasks.workunit.client.0.smithi151.stdout:checking for gmake... /usr/bin/gmake 2024-04-26T01:55:45.372 INFO:tasks.workunit.client.0.smithi151.stderr:./configure: line 12082: test: $(SHELL): binary operator expected 2024-04-26T01:55:45.372 INFO:tasks.workunit.client.0.smithi151.stdout:checking for tar... /bin/tar 2024-04-26T01:55:45.372 INFO:tasks.workunit.client.0.smithi151.stdout:checking for gzip... /bin/gzip 2024-04-26T01:55:45.417 INFO:tasks.workunit.client.0.smithi151.stdout:checking whether gcc -MM is supported... yes 2024-04-26T01:55:45.418 INFO:tasks.workunit.client.0.smithi151.stdout:checking for sort... /bin/sort 2024-04-26T01:55:45.418 INFO:tasks.workunit.client.0.smithi151.stdout:checking whether ln -s works... yes 2024-04-26T01:55:45.420 INFO:tasks.workunit.client.0.smithi151.stdout:checking for rpm... /bin/rpm 2024-04-26T01:55:45.432 INFO:tasks.workunit.client.0.smithi151.stdout:checking for rpmbuild... no 2024-04-26T01:55:45.543 INFO:tasks.workunit.client.0.smithi151.stdout:checking assert.h usability... yes 2024-04-26T01:55:45.580 INFO:tasks.workunit.client.0.smithi151.stdout:checking assert.h presence... yes 2024-04-26T01:55:45.580 INFO:tasks.workunit.client.0.smithi151.stdout:checking for assert.h... yes 2024-04-26T01:55:45.698 INFO:tasks.workunit.client.0.smithi151.stdout:checking bstring.h usability... no 2024-04-26T01:55:45.760 INFO:tasks.workunit.client.0.smithi151.stdout:checking bstring.h presence... no 2024-04-26T01:55:45.761 INFO:tasks.workunit.client.0.smithi151.stdout:checking for bstring.h... no 2024-04-26T01:55:45.853 INFO:tasks.workunit.client.0.smithi151.stdout:checking libgen.h usability... yes 2024-04-26T01:55:45.892 INFO:tasks.workunit.client.0.smithi151.stdout:checking libgen.h presence... yes 2024-04-26T01:55:45.892 INFO:tasks.workunit.client.0.smithi151.stdout:checking for libgen.h... yes 2024-04-26T01:55:45.998 INFO:tasks.workunit.client.0.smithi151.stdout:checking dirent.h usability... yes 2024-04-26T01:55:46.037 INFO:tasks.workunit.client.0.smithi151.stdout:checking dirent.h presence... yes 2024-04-26T01:55:46.037 INFO:tasks.workunit.client.0.smithi151.stdout:checking for dirent.h... yes 2024-04-26T01:55:46.139 INFO:tasks.workunit.client.0.smithi151.stdout:checking errno.h usability... yes 2024-04-26T01:55:46.179 INFO:tasks.workunit.client.0.smithi151.stdout:checking errno.h presence... yes 2024-04-26T01:55:46.180 INFO:tasks.workunit.client.0.smithi151.stdout:checking for errno.h... yes 2024-04-26T01:55:46.287 INFO:tasks.workunit.client.0.smithi151.stdout:checking malloc.h usability... yes 2024-04-26T01:55:46.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:55:45 smithi172 ceph-mon[32785]: pgmap v67: 97 pgs: 97 active+clean; 25 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 1.2 MiB/s wr, 723 op/s 2024-04-26T01:55:46.327 INFO:tasks.workunit.client.0.smithi151.stdout:checking malloc.h presence... yes 2024-04-26T01:55:46.327 INFO:tasks.workunit.client.0.smithi151.stdout:checking for malloc.h... yes 2024-04-26T01:55:46.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:55:45 smithi165 ceph-mon[32648]: pgmap v67: 97 pgs: 97 active+clean; 25 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 1.2 MiB/s wr, 723 op/s 2024-04-26T01:55:46.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:55:45 smithi151 ceph-mon[28057]: pgmap v67: 97 pgs: 97 active+clean; 25 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 1.2 MiB/s wr, 723 op/s 2024-04-26T01:55:46.460 INFO:tasks.workunit.client.0.smithi151.stdout:checking uuid.h usability... no 2024-04-26T01:55:46.523 INFO:tasks.workunit.client.0.smithi151.stdout:checking uuid.h presence... no 2024-04-26T01:55:46.523 INFO:tasks.workunit.client.0.smithi151.stdout:checking for uuid.h... no 2024-04-26T01:55:46.638 INFO:tasks.workunit.client.0.smithi151.stdout:checking uuid/uuid.h usability... yes 2024-04-26T01:55:46.674 INFO:tasks.workunit.client.0.smithi151.stdout:checking uuid/uuid.h presence... yes 2024-04-26T01:55:46.675 INFO:tasks.workunit.client.0.smithi151.stdout:checking for uuid/uuid.h... yes 2024-04-26T01:55:46.769 INFO:tasks.workunit.client.0.smithi151.stdout:checking sys/uuid.h usability... no 2024-04-26T01:55:46.823 INFO:tasks.workunit.client.0.smithi151.stdout:checking sys/uuid.h presence... no 2024-04-26T01:55:46.824 INFO:tasks.workunit.client.0.smithi151.stdout:checking for sys/uuid.h... no 2024-04-26T01:55:46.924 INFO:tasks.workunit.client.0.smithi151.stdout:checking sys/file.h usability... yes 2024-04-26T01:55:46.959 INFO:tasks.workunit.client.0.smithi151.stdout:checking sys/file.h presence... yes 2024-04-26T01:55:46.960 INFO:tasks.workunit.client.0.smithi151.stdout:checking for sys/file.h... yes 2024-04-26T01:55:47.036 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:55:46 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:55:47.081 INFO:tasks.workunit.client.0.smithi151.stdout:checking sys/fcntl.h usability... yes 2024-04-26T01:55:47.117 INFO:tasks.workunit.client.0.smithi151.stdout:checking sys/fcntl.h presence... yes 2024-04-26T01:55:47.117 INFO:tasks.workunit.client.0.smithi151.stdout:checking for sys/fcntl.h... yes 2024-04-26T01:55:47.217 INFO:tasks.workunit.client.0.smithi151.stdout:checking sys/syssgi.h usability... no 2024-04-26T01:55:47.275 INFO:tasks.workunit.client.0.smithi151.stdout:checking sys/syssgi.h presence... no 2024-04-26T01:55:47.275 INFO:tasks.workunit.client.0.smithi151.stdout:checking for sys/syssgi.h... no 2024-04-26T01:55:47.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:55:46 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:55:47.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:55:46 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:55:47.373 INFO:tasks.workunit.client.0.smithi151.stdout:checking sys/param.h usability... yes 2024-04-26T01:55:47.410 INFO:tasks.workunit.client.0.smithi151.stdout:checking sys/param.h presence... yes 2024-04-26T01:55:47.410 INFO:tasks.workunit.client.0.smithi151.stdout:checking for sys/param.h... yes 2024-04-26T01:55:47.437 INFO:tasks.workunit.client.0.smithi151.stdout:checking for sys/stat.h... (cached) yes 2024-04-26T01:55:47.517 INFO:tasks.workunit.client.0.smithi151.stdout:checking sys/statvfs.h usability... yes 2024-04-26T01:55:47.552 INFO:tasks.workunit.client.0.smithi151.stdout:checking sys/statvfs.h presence... yes 2024-04-26T01:55:47.552 INFO:tasks.workunit.client.0.smithi151.stdout:checking for sys/statvfs.h... yes 2024-04-26T01:55:47.640 INFO:tasks.workunit.client.0.smithi151.stdout:checking sys/time.h usability... yes 2024-04-26T01:55:47.674 INFO:tasks.workunit.client.0.smithi151.stdout:checking sys/time.h presence... yes 2024-04-26T01:55:47.675 INFO:tasks.workunit.client.0.smithi151.stdout:checking for sys/time.h... yes 2024-04-26T01:55:47.775 INFO:tasks.workunit.client.0.smithi151.stdout:checking sys/ioctl.h usability... yes 2024-04-26T01:55:47.813 INFO:tasks.workunit.client.0.smithi151.stdout:checking sys/ioctl.h presence... yes 2024-04-26T01:55:47.813 INFO:tasks.workunit.client.0.smithi151.stdout:checking for sys/ioctl.h... yes 2024-04-26T01:55:47.924 INFO:tasks.workunit.client.0.smithi151.stdout:checking sys/wait.h usability... yes 2024-04-26T01:55:47.961 INFO:tasks.workunit.client.0.smithi151.stdout:checking sys/wait.h presence... yes 2024-04-26T01:55:47.961 INFO:tasks.workunit.client.0.smithi151.stdout:checking for sys/wait.h... yes 2024-04-26T01:55:47.991 INFO:tasks.workunit.client.0.smithi151.stdout:checking for sys/types.h... (cached) yes 2024-04-26T01:55:48.009 INFO:tasks.workunit.client.0.smithi151.stdout:checking for strings.h... (cached) yes 2024-04-26T01:55:48.092 INFO:tasks.workunit.client.0.smithi151.stdout:checking err.h usability... yes 2024-04-26T01:55:48.142 INFO:tasks.workunit.client.0.smithi151.stdout:checking err.h presence... yes 2024-04-26T01:55:48.142 INFO:tasks.workunit.client.0.smithi151.stdout:checking for err.h... yes 2024-04-26T01:55:48.240 INFO:tasks.workunit.client.0.smithi151.stdout:checking linux/falloc.h usability... yes 2024-04-26T01:55:48.276 INFO:tasks.workunit.client.0.smithi151.stdout:checking linux/falloc.h presence... yes 2024-04-26T01:55:48.276 INFO:tasks.workunit.client.0.smithi151.stdout:checking for linux/falloc.h... yes 2024-04-26T01:55:48.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:55:47 smithi172 ceph-mon[32785]: pgmap v68: 97 pgs: 97 active+clean; 25 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 1.2 MiB/s wr, 724 op/s 2024-04-26T01:55:48.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:55:47 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:55:48.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:55:47 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:55:48.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:55:47 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:55:48.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:55:47 smithi165 ceph-mon[32648]: pgmap v68: 97 pgs: 97 active+clean; 25 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 1.2 MiB/s wr, 724 op/s 2024-04-26T01:55:48.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:55:47 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:55:48.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:55:47 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:55:48.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:55:47 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:55:48.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:55:47 smithi151 ceph-mon[28057]: pgmap v68: 97 pgs: 97 active+clean; 25 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 1.2 MiB/s wr, 724 op/s 2024-04-26T01:55:48.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:55:47 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:55:48.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:55:47 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:55:48.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:55:47 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:55:48.389 INFO:tasks.workunit.client.0.smithi151.stdout:checking sys/fs/xfs_fsops.h usability... no 2024-04-26T01:55:48.458 INFO:tasks.workunit.client.0.smithi151.stdout:checking sys/fs/xfs_fsops.h presence... no 2024-04-26T01:55:48.458 INFO:tasks.workunit.client.0.smithi151.stdout:checking for sys/fs/xfs_fsops.h... no 2024-04-26T01:55:48.558 INFO:tasks.workunit.client.0.smithi151.stdout:checking sys/fs/xfs_itable.h usability... no 2024-04-26T01:55:48.625 INFO:tasks.workunit.client.0.smithi151.stdout:checking sys/fs/xfs_itable.h presence... no 2024-04-26T01:55:48.625 INFO:tasks.workunit.client.0.smithi151.stdout:checking for sys/fs/xfs_itable.h... no 2024-04-26T01:55:48.733 INFO:tasks.workunit.client.0.smithi151.stdout:checking xfs/platform_defs.h usability... no 2024-04-26T01:55:48.801 INFO:tasks.workunit.client.0.smithi151.stdout:checking xfs/platform_defs.h presence... no 2024-04-26T01:55:48.801 INFO:tasks.workunit.client.0.smithi151.stdout:checking for xfs/platform_defs.h... no 2024-04-26T01:55:48.914 INFO:tasks.workunit.client.0.smithi151.stdout:checking btrfs/ioctl.h usability... no 2024-04-26T01:55:48.979 INFO:tasks.workunit.client.0.smithi151.stdout:checking btrfs/ioctl.h presence... no 2024-04-26T01:55:48.979 INFO:tasks.workunit.client.0.smithi151.stdout:checking for btrfs/ioctl.h... no 2024-04-26T01:55:49.110 INFO:tasks.workunit.client.0.smithi151.stdout:checking cifs/ioctl.h usability... no 2024-04-26T01:55:49.176 INFO:tasks.workunit.client.0.smithi151.stdout:checking cifs/ioctl.h presence... no 2024-04-26T01:55:49.176 INFO:tasks.workunit.client.0.smithi151.stdout:checking for cifs/ioctl.h... no 2024-04-26T01:55:49.273 INFO:tasks.workunit.client.0.smithi151.stdout:checking sys/mman.h usability... yes 2024-04-26T01:55:49.312 INFO:tasks.workunit.client.0.smithi151.stdout:checking sys/mman.h presence... yes 2024-04-26T01:55:49.313 INFO:tasks.workunit.client.0.smithi151.stdout:checking for sys/mman.h... yes 2024-04-26T01:55:49.425 INFO:tasks.workunit.client.0.smithi151.stdout:checking linux/ext4.h usability... no 2024-04-26T01:55:49.489 INFO:tasks.workunit.client.0.smithi151.stdout:checking linux/ext4.h presence... no 2024-04-26T01:55:49.489 INFO:tasks.workunit.client.0.smithi151.stdout:checking for linux/ext4.h... no 2024-04-26T01:55:49.559 INFO:tasks.workunit.client.0.smithi151.stdout:checking for xfs/xfs_log_format.h... no 2024-04-26T01:55:49.681 INFO:tasks.workunit.client.0.smithi151.stdout:checking for uuid_compare... no 2024-04-26T01:55:49.773 INFO:tasks.workunit.client.0.smithi151.stdout:checking for uuid_compare in -luuid... yes 2024-04-26T01:55:49.914 INFO:tasks.workunit.client.0.smithi151.stdout:checking for xfs/xfs.h... yes 2024-04-26T01:55:49.989 INFO:tasks.workunit.client.0.smithi151.stdout:checking for xfs/libxfs.h... no 2024-04-26T01:55:50.069 INFO:tasks.workunit.client.0.smithi151.stdout:checking whether xlog_assign_lsn is declared... no 2024-04-26T01:55:50.158 INFO:tasks.workunit.client.0.smithi151.stdout:checking for xfs/xqm.h... yes 2024-04-26T01:55:50.277 INFO:tasks.workunit.client.0.smithi151.stdout:checking xfsctl from xfs/xfs.h... ok 2024-04-26T01:55:50.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:55:49 smithi172 ceph-mon[32785]: pgmap v69: 97 pgs: 97 active+clean; 27 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 796 KiB/s rd, 1.3 MiB/s wr, 734 op/s 2024-04-26T01:55:50.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:55:49 smithi165 ceph-mon[32648]: pgmap v69: 97 pgs: 97 active+clean; 27 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 796 KiB/s rd, 1.3 MiB/s wr, 734 op/s 2024-04-26T01:55:50.356 INFO:tasks.workunit.client.0.smithi151.stdout:checking xfs/handle.h usability... yes 2024-04-26T01:55:50.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:55:49 smithi151 ceph-mon[28057]: pgmap v69: 97 pgs: 97 active+clean; 27 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 796 KiB/s rd, 1.3 MiB/s wr, 734 op/s 2024-04-26T01:55:50.392 INFO:tasks.workunit.client.0.smithi151.stdout:checking xfs/handle.h presence... yes 2024-04-26T01:55:50.392 INFO:tasks.workunit.client.0.smithi151.stdout:checking for xfs/handle.h... yes 2024-04-26T01:55:50.511 INFO:tasks.workunit.client.0.smithi151.stdout:checking for attr_list_by_handle in -lhandle... yes 2024-04-26T01:55:50.727 INFO:tasks.workunit.client.0.smithi151.stdout:checking sys/acl.h usability... yes 2024-04-26T01:55:50.769 INFO:tasks.workunit.client.0.smithi151.stdout:checking sys/acl.h presence... yes 2024-04-26T01:55:50.769 INFO:tasks.workunit.client.0.smithi151.stdout:checking for sys/acl.h... yes 2024-04-26T01:55:50.867 INFO:tasks.workunit.client.0.smithi151.stdout:checking acl/libacl.h usability... yes 2024-04-26T01:55:50.909 INFO:tasks.workunit.client.0.smithi151.stdout:checking acl/libacl.h presence... yes 2024-04-26T01:55:50.909 INFO:tasks.workunit.client.0.smithi151.stdout:checking for acl/libacl.h... yes 2024-04-26T01:55:51.011 INFO:tasks.workunit.client.0.smithi151.stdout:checking for acl_init in -lacl... yes 2024-04-26T01:55:51.104 INFO:tasks.workunit.client.0.smithi151.stdout:checking gdbm-ndbm.h usability... no 2024-04-26T01:55:51.168 INFO:tasks.workunit.client.0.smithi151.stdout:checking gdbm-ndbm.h presence... no 2024-04-26T01:55:51.168 INFO:tasks.workunit.client.0.smithi151.stdout:checking for gdbm-ndbm.h... no 2024-04-26T01:55:51.254 INFO:tasks.workunit.client.0.smithi151.stdout:checking gdbm/ndbm.h usability... no 2024-04-26T01:55:51.320 INFO:tasks.workunit.client.0.smithi151.stdout:checking gdbm/ndbm.h presence... no 2024-04-26T01:55:51.320 INFO:tasks.workunit.client.0.smithi151.stdout:checking for gdbm/ndbm.h... no 2024-04-26T01:55:51.406 INFO:tasks.workunit.client.0.smithi151.stdout:checking ndbm.h usability... no 2024-04-26T01:55:51.472 INFO:tasks.workunit.client.0.smithi151.stdout:checking ndbm.h presence... no 2024-04-26T01:55:51.472 INFO:tasks.workunit.client.0.smithi151.stdout:checking for ndbm.h... no 2024-04-26T01:55:51.547 INFO:tasks.workunit.client.0.smithi151.stdout:checking libaio.h usability... yes 2024-04-26T01:55:51.589 INFO:tasks.workunit.client.0.smithi151.stdout:checking libaio.h presence... yes 2024-04-26T01:55:51.589 INFO:tasks.workunit.client.0.smithi151.stdout:checking for libaio.h... yes 2024-04-26T01:55:51.680 INFO:tasks.workunit.client.0.smithi151.stdout:checking liburing.h usability... no 2024-04-26T01:55:51.744 INFO:tasks.workunit.client.0.smithi151.stdout:checking liburing.h presence... no 2024-04-26T01:55:51.744 INFO:tasks.workunit.client.0.smithi151.stdout:checking for liburing.h... no 2024-04-26T01:55:51.830 INFO:tasks.workunit.client.0.smithi151.stdout:checking sys/capability.h usability... no 2024-04-26T01:55:51.891 INFO:tasks.workunit.client.0.smithi151.stdout:checking sys/capability.h presence... no 2024-04-26T01:55:51.891 INFO:tasks.workunit.client.0.smithi151.stdout:checking for sys/capability.h... no 2024-04-26T01:55:51.979 INFO:tasks.workunit.client.0.smithi151.stdout:checking linux/fiemap.h usability... yes 2024-04-26T01:55:52.017 INFO:tasks.workunit.client.0.smithi151.stdout:checking linux/fiemap.h presence... yes 2024-04-26T01:55:52.018 INFO:tasks.workunit.client.0.smithi151.stdout:checking for linux/fiemap.h... yes 2024-04-26T01:55:52.109 INFO:tasks.workunit.client.0.smithi151.stdout:checking for fallocate... yes 2024-04-26T01:55:52.217 INFO:tasks.workunit.client.0.smithi151.stdout:checking for open_by_handle_at... yes 2024-04-26T01:55:52.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:55:51 smithi172 ceph-mon[32785]: pgmap v70: 97 pgs: 97 active+clean; 27 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 577 KiB/s wr, 574 op/s 2024-04-26T01:55:52.296 INFO:tasks.workunit.client.0.smithi151.stdout:checking linux/fs.h usability... yes 2024-04-26T01:55:52.335 INFO:tasks.workunit.client.0.smithi151.stdout:checking linux/fs.h presence... yes 2024-04-26T01:55:52.335 INFO:tasks.workunit.client.0.smithi151.stdout:checking for linux/fs.h... yes 2024-04-26T01:55:52.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:55:51 smithi165 ceph-mon[32648]: pgmap v70: 97 pgs: 97 active+clean; 27 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 577 KiB/s wr, 574 op/s 2024-04-26T01:55:52.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:55:51 smithi151 ceph-mon[28057]: pgmap v70: 97 pgs: 97 active+clean; 27 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 577 KiB/s wr, 574 op/s 2024-04-26T01:55:52.437 INFO:tasks.workunit.client.0.smithi151.stdout:checking btrfsutil.h usability... no 2024-04-26T01:55:52.499 INFO:tasks.workunit.client.0.smithi151.stdout:checking btrfsutil.h presence... no 2024-04-26T01:55:52.499 INFO:tasks.workunit.client.0.smithi151.stdout:checking for btrfsutil.h... no 2024-04-26T01:55:52.601 INFO:tasks.workunit.client.0.smithi151.stdout:checking for copy_file_range... yes 2024-04-26T01:55:52.703 INFO:tasks.workunit.client.0.smithi151.stdout:checking for SEEK_DATA... yes 2024-04-26T01:55:52.823 INFO:tasks.workunit.client.0.smithi151.stdout:checking for BMV_OF_SHARED... yes 2024-04-26T01:55:52.928 INFO:tasks.workunit.client.0.smithi151.stdout:checking for nftw... yes 2024-04-26T01:55:53.033 INFO:tasks.workunit.client.0.smithi151.stdout:checking for RLIMIT_NOFILE... yes 2024-04-26T01:55:53.090 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.091+0000 7f910b200640 1 -- 172.21.15.151:0/1843507897 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91041031e0 msgr2=0x7f91041035c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:53.090 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.091+0000 7f910b200640 1 --2- 172.21.15.151:0/1843507897 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91041031e0 0x7f91041035c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f90f4009860 tx=0x7f90f402a460 comp rx=0 tx=0).stop 2024-04-26T01:55:53.090 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.091+0000 7f910b200640 1 -- 172.21.15.151:0/1843507897 shutdown_connections 2024-04-26T01:55:53.090 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.091+0000 7f910b200640 1 --2- 172.21.15.151:0/1843507897 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f910410c520 0x7f910410e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:53.090 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.091+0000 7f910b200640 1 --2- 172.21.15.151:0/1843507897 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9104103b00 0x7f910410bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:53.090 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.091+0000 7f910b200640 1 --2- 172.21.15.151:0/1843507897 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91041031e0 0x7f91041035c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:53.091 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.091+0000 7f910b200640 1 -- 172.21.15.151:0/1843507897 >> 172.21.15.151:0/1843507897 conn(0x7f9104077fd0 msgr2=0x7f91040783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:53.091 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.091+0000 7f910b200640 1 -- 172.21.15.151:0/1843507897 shutdown_connections 2024-04-26T01:55:53.091 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.091+0000 7f910b200640 1 -- 172.21.15.151:0/1843507897 wait complete. 2024-04-26T01:55:53.092 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.092+0000 7f910b200640 1 Processor -- start 2024-04-26T01:55:53.092 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.092+0000 7f910b200640 1 -- start start 2024-04-26T01:55:53.092 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.093+0000 7f910b200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91041031e0 0x7f91041a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:53.092 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.093+0000 7f910b200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9104103b00 0x7f91041a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:53.092 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.093+0000 7f910b200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f910410c520 0x7f910419a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:53.092 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.093+0000 7f910b200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9104112200 con 0x7f9104103b00 2024-04-26T01:55:53.092 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.093+0000 7f910b200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9104112080 con 0x7f910410c520 2024-04-26T01:55:53.092 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.093+0000 7f910b200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9104112380 con 0x7f91041031e0 2024-04-26T01:55:53.092 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.093+0000 7f9103400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9104103b00 0x7f91041a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:53.092 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.093+0000 7f9103e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91041031e0 0x7f91041a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:53.093 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.093+0000 7f9109200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f910410c520 0x7f910419a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:53.093 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.093+0000 7f9103400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9104103b00 0x7f91041a0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:59988/0 (socket says 172.21.15.151:59988) 2024-04-26T01:55:53.093 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.093+0000 7f9103400640 1 -- 172.21.15.151:0/2036117715 learned_addr learned my addr 172.21.15.151:0/2036117715 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:55:53.093 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.093+0000 7f9103e00640 1 --2- 172.21.15.151:0/2036117715 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91041031e0 0x7f91041a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:34034/0 (socket says 172.21.15.151:34034) 2024-04-26T01:55:53.094 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.094+0000 7f9103e00640 1 -- 172.21.15.151:0/2036117715 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f910410c520 msgr2=0x7f910419a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:53.094 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.094+0000 7f9103e00640 1 --2- 172.21.15.151:0/2036117715 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f910410c520 0x7f910419a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:53.094 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.094+0000 7f9103e00640 1 -- 172.21.15.151:0/2036117715 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9104103b00 msgr2=0x7f91041a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:53.094 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.094+0000 7f9103e00640 1 --2- 172.21.15.151:0/2036117715 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9104103b00 0x7f91041a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:53.094 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.094+0000 7f9103e00640 1 -- 172.21.15.151:0/2036117715 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f910419afe0 con 0x7f91041031e0 2024-04-26T01:55:53.094 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.094+0000 7f9103400640 1 --2- 172.21.15.151:0/2036117715 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9104103b00 0x7f91041a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:55:53.094 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.094+0000 7f9103e00640 1 --2- 172.21.15.151:0/2036117715 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91041031e0 0x7f91041a0660 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f90f402a980 tx=0x7f90f4030fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:53.095 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.095+0000 7f9100c00640 1 -- 172.21.15.151:0/2036117715 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f4040070 con 0x7f91041031e0 2024-04-26T01:55:53.095 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.095+0000 7f910b200640 1 -- 172.21.15.151:0/2036117715 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f910419b270 con 0x7f91041031e0 2024-04-26T01:55:53.095 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.095+0000 7f9100c00640 1 -- 172.21.15.151:0/2036117715 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90f4005570 con 0x7f91041031e0 2024-04-26T01:55:53.095 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.095+0000 7f9100c00640 1 -- 172.21.15.151:0/2036117715 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f40046c0 con 0x7f91041031e0 2024-04-26T01:55:53.095 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.095+0000 7f910b200640 1 -- 172.21.15.151:0/2036117715 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91041a7400 con 0x7f91041031e0 2024-04-26T01:55:53.097 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.098+0000 7f910e1af740 1 Processor -- start 2024-04-26T01:55:53.097 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.098+0000 7f910e1af740 1 -- start start 2024-04-26T01:55:53.097 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.098+0000 7f910e1af740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d3e8ef820 0x557d3e8efc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:53.097 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.098+0000 7f910e1af740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557d3e8f0140 0x557d3e8fa4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:53.098 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.098+0000 7f910e1af740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557d3e8faa00 0x557d3e8fcdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:53.098 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.098+0000 7f910e1af740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557d3e855900 con 0x557d3e8faa00 2024-04-26T01:55:53.098 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.098+0000 7f910e1af740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x557d3e7fbef0 con 0x557d3e8f0140 2024-04-26T01:55:53.098 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.098+0000 7f910e1af740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557d3e8518c0 con 0x557d3e8ef820 2024-04-26T01:55:53.098 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.099+0000 7f9109200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d3e8ef820 0x557d3e8efc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:53.098 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.099+0000 7f9109200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d3e8ef820 0x557d3e8efc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:34050/0 (socket says 172.21.15.151:34050) 2024-04-26T01:55:53.098 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.099+0000 7f9109200640 1 -- 172.21.15.151:0/843544847 learned_addr learned my addr 172.21.15.151:0/843544847 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:55:53.098 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.099+0000 7f9103400640 1 --2- 172.21.15.151:0/843544847 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557d3e8faa00 0x557d3e8fcdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:53.100 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.099+0000 7f9109200640 1 -- 172.21.15.151:0/843544847 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557d3e8f0140 msgr2=0x557d3e8fa4c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T01:55:53.100 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.099+0000 7f9109200640 1 --2- 172.21.15.151:0/843544847 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557d3e8f0140 0x557d3e8fa4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:53.100 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.099+0000 7f9109200640 1 -- 172.21.15.151:0/843544847 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557d3e8faa00 msgr2=0x557d3e8fcdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:53.100 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.099+0000 7f9109200640 1 --2- 172.21.15.151:0/843544847 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557d3e8faa00 0x557d3e8fcdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:53.101 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.099+0000 7f9109200640 1 -- 172.21.15.151:0/843544847 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d3e568210 con 0x557d3e8ef820 2024-04-26T01:55:53.101 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.099+0000 7f9100c00640 1 -- 172.21.15.151:0/2036117715 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f90f4043070 con 0x7f91041031e0 2024-04-26T01:55:53.101 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.099+0000 7f9109200640 1 --2- 172.21.15.151:0/843544847 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d3e8ef820 0x557d3e8efc00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f90fc0099f0 tx=0x7f90fc0095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70ab62b9451131f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:53.101 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.100+0000 7f90f9600640 1 -- 172.21.15.151:0/843544847 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90fc01d070 con 0x557d3e8ef820 2024-04-26T01:55:53.101 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.100+0000 7f90f9600640 1 -- 172.21.15.151:0/843544847 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90fc003e40 con 0x557d3e8ef820 2024-04-26T01:55:53.101 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.100+0000 7f90f9600640 1 -- 172.21.15.151:0/843544847 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90fc00d9d0 con 0x557d3e8ef820 2024-04-26T01:55:53.101 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.101+0000 7f910e1af740 1 -- 172.21.15.151:0/843544847 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d3e8ef820 msgr2=0x557d3e8efc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:53.101 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.101+0000 7f910e1af740 1 --2- 172.21.15.151:0/843544847 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d3e8ef820 0x557d3e8efc00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f90fc0099f0 tx=0x7f90fc0095c0 comp rx=0 tx=0).stop 2024-04-26T01:55:53.101 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.101+0000 7f9100c00640 1 --2- 172.21.15.151:0/2036117715 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f90e80a3dc0 0x7f90e80a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:53.101 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.101+0000 7f910e1af740 1 -- 172.21.15.151:0/843544847 shutdown_connections 2024-04-26T01:55:53.101 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.101+0000 7f910e1af740 1 --2- 172.21.15.151:0/843544847 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557d3e8faa00 0x557d3e8fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:53.101 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.101+0000 7f910e1af740 1 --2- 172.21.15.151:0/843544847 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557d3e8f0140 0x557d3e8fa4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:53.102 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.101+0000 7f910e1af740 1 --2- 172.21.15.151:0/843544847 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d3e8ef820 0x557d3e8efc00 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:53.102 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.101+0000 7f910e1af740 1 -- 172.21.15.151:0/843544847 >> 172.21.15.151:0/843544847 conn(0x557d3e8ebd30 msgr2=0x557d3e8ec110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:53.102 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.101+0000 7f9100c00640 1 -- 172.21.15.151:0/2036117715 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f90f40f1240 con 0x7f91041031e0 2024-04-26T01:55:53.102 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.101+0000 7f910e1af740 1 -- 172.21.15.151:0/843544847 shutdown_connections 2024-04-26T01:55:53.102 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.101+0000 7f9109200640 1 --2- 172.21.15.151:0/2036117715 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f90e80a3dc0 0x7f90e80a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:53.102 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.101+0000 7f910e1af740 1 -- 172.21.15.151:0/843544847 wait complete. 2024-04-26T01:55:53.102 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.102+0000 7f910e1af740 1 -- start start 2024-04-26T01:55:53.102 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.102+0000 7f910e1af740 1 Processor -- start 2024-04-26T01:55:53.102 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.102+0000 7f9109200640 1 --2- 172.21.15.151:0/2036117715 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f90e80a3dc0 0x7f90e80a6280 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f90fc005c10 tx=0x7f90fc009a20 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:53.106 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.102+0000 7f910e1af740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557d3e8ef820 0x557d3e8f7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:53.106 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.102+0000 7f910e1af740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557d3e8f0140 0x557d3e8f7f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:53.106 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.102+0000 7f910e1af740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d3e8faa00 0x557d3e8f8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:53.106 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.103+0000 7f910e1af740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557d3e8fd630 con 0x557d3e8ef820 2024-04-26T01:55:53.106 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.103+0000 7f910e1af740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x557d3e80f310 con 0x557d3e8f0140 2024-04-26T01:55:53.106 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.103+0000 7f910e1af740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557d3e8fd7b0 con 0x557d3e8faa00 2024-04-26T01:55:53.106 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.103+0000 7f910e1af740 10 client.0 _ll_register_callbacks cb 0x557d3e7ddcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:55:53.106 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.103+0000 7f9103400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557d3e8f0140 0x557d3e8f7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:53.106 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.103+0000 7f9103400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557d3e8f0140 0x557d3e8f7f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:58846/0 (socket says 172.21.15.151:58846) 2024-04-26T01:55:53.106 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.103+0000 7f9103400640 1 -- 172.21.15.151:0/2845245112 learned_addr learned my addr 172.21.15.151:0/2845245112 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:55:53.107 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.103+0000 7f9103e00640 1 --2- 172.21.15.151:0/2845245112 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557d3e8ef820 0x557d3e8f7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:53.107 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.103+0000 7f9109200640 1 --2- 172.21.15.151:0/2845245112 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d3e8faa00 0x557d3e8f8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:53.107 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.103+0000 7f9103400640 1 -- 172.21.15.151:0/2845245112 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d3e8faa00 msgr2=0x557d3e8f8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:53.107 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.103+0000 7f9103400640 1 --2- 172.21.15.151:0/2845245112 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d3e8faa00 0x557d3e8f8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:53.107 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.103+0000 7f9103400640 1 -- 172.21.15.151:0/2845245112 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557d3e8ef820 msgr2=0x557d3e8f7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:53.107 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.103+0000 7f9103400640 1 --2- 172.21.15.151:0/2845245112 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557d3e8ef820 0x557d3e8f7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:53.107 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.103+0000 7f9103400640 1 -- 172.21.15.151:0/2845245112 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d3e634620 con 0x557d3e8f0140 2024-04-26T01:55:53.107 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.104+0000 7f9103400640 1 --2- 172.21.15.151:0/2845245112 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557d3e8f0140 0x557d3e8f7f50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f90f0012d20 tx=0x7f90f000f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f4db4b0cfef3d1d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:53.107 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.104+0000 7f90ee000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T01:55:53.107 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.104+0000 7f90ee000640 1 -- 172.21.15.151:0/2845245112 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f00120d0 con 0x557d3e8f0140 2024-04-26T01:55:53.107 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.104+0000 7f90ee000640 1 -- 172.21.15.151:0/2845245112 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f90f0005020 con 0x557d3e8f0140 2024-04-26T01:55:53.107 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.104+0000 7f910e1af740 1 -- 172.21.15.151:0/2845245112 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557d3e8698b0 con 0x557d3e8f0140 2024-04-26T01:55:53.107 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.104+0000 7f90ee000640 1 -- 172.21.15.151:0/2845245112 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f90f0012270 con 0x557d3e8f0140 2024-04-26T01:55:53.108 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.104+0000 7f910e1af740 10 client.25433 fetch_fsmap learned FSMap version 28 2024-04-26T01:55:53.108 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.104+0000 7f910e1af740 1 -- 172.21.15.151:0/2845245112 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557d3e909980 con 0x557d3e8f0140 2024-04-26T01:55:53.108 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.104+0000 7f90ee000640 1 -- 172.21.15.151:0/2845245112 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x557d3e8698b0 con 0x557d3e8f0140 2024-04-26T01:55:53.108 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.104+0000 7f90ee000640 1 -- 172.21.15.151:0/2845245112 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f90f0024050 con 0x557d3e8f0140 2024-04-26T01:55:53.108 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.105+0000 7f910e1af740 10 client.25433 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T01:55:53.108 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.105+0000 7f910e1af740 10 client.25433 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:55:53.108 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.105+0000 7f910e1af740 20 client.25433 populate_metadata read hostname 'smithi151' 2024-04-26T01:55:53.108 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.105+0000 7f910e1af740 1 --2- 172.21.15.151:0/2845245112 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x557d3e90ae40 0x557d3e90d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:53.108 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.105+0000 7f910e1af740 4 client.25433 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:55:53.108 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.105+0000 7f910e1af740 1 -- 172.21.15.151:0/2845245112 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d3e863320 con 0x557d3e90ae40 2024-04-26T01:55:53.108 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.105+0000 7f9103e00640 1 --2- 172.21.15.151:0/2845245112 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x557d3e90ae40 0x557d3e90d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:53.108 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.105+0000 7f9103e00640 1 --2- 172.21.15.151:0/2845245112 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x557d3e90ae40 0x557d3e90d270 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10290a702c34a69a server_cookie=11bbb145abf8226e in_seq=0 out_seq=0 2024-04-26T01:55:53.108 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.105+0000 7f90ee000640 10 client.25433 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:55:53.109 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.106+0000 7f90ee000640 1 -- 172.21.15.151:0/2845245112 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557d3e863320 con 0x557d3e90ae40 2024-04-26T01:55:53.109 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.106+0000 7f90ee000640 10 client.25433 handle_command_reply: tid=0 2024-04-26T01:55:53.109 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.106+0000 7f910e1af740 1 client.25433 shutdown 2024-04-26T01:55:53.109 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.107+0000 7f910e1af740 1 -- 172.21.15.151:0/2845245112 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557d3e8f0140 msgr2=0x557d3e8f7f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:53.109 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.107+0000 7f910e1af740 1 --2- 172.21.15.151:0/2845245112 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557d3e8f0140 0x557d3e8f7f50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f90f0012d20 tx=0x7f90f000f790 comp rx=0 tx=0).stop 2024-04-26T01:55:53.109 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.107+0000 7f910e1af740 1 -- 172.21.15.151:0/2845245112 shutdown_connections 2024-04-26T01:55:53.109 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.107+0000 7f910e1af740 1 --2- 172.21.15.151:0/2845245112 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x557d3e90ae40 0x557d3e90d270 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:53.109 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.107+0000 7f910e1af740 1 --2- 172.21.15.151:0/2845245112 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d3e8faa00 0x557d3e8f8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:53.109 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.107+0000 7f910e1af740 1 --2- 172.21.15.151:0/2845245112 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557d3e8f0140 0x557d3e8f7f50 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:53.109 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.107+0000 7f90ee000640 0 client.25433 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:55:53.109 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.107+0000 7f910e1af740 1 --2- 172.21.15.151:0/2845245112 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557d3e8ef820 0x557d3e8f7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:53.109 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.107+0000 7f910e1af740 1 -- 172.21.15.151:0/2845245112 >> 172.21.15.151:0/2845245112 conn(0x557d3e8ebd30 msgr2=0x557d3e8f1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:53.110 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.107+0000 7f910e1af740 1 -- 172.21.15.151:0/2845245112 shutdown_connections 2024-04-26T01:55:53.110 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.107+0000 7f910e1af740 1 -- 172.21.15.151:0/2845245112 wait complete. 2024-04-26T01:55:53.110 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.107+0000 7f910e1af740 20 client.25433 trim_cache size 0 max 16384 2024-04-26T01:55:53.118 INFO:tasks.workunit.client.0.smithi151.stdout:checking for XFS_IOC_EXCHANGE_RANGE... no 2024-04-26T01:55:53.123 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.124+0000 7f910e1af740 1 Processor -- start 2024-04-26T01:55:53.123 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.124+0000 7f910e1af740 1 -- start start 2024-04-26T01:55:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.125+0000 7f910e1af740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d3e8ef820 0x557d3e9064f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.125+0000 7f910e1af740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557d3e8f0140 0x557d3e8f21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.125+0000 7f910e1af740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557d3e8faa00 0x557d3e912b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:53.125 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.125+0000 7f910e1af740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f91041b2460 con 0x557d3e8faa00 2024-04-26T01:55:53.125 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.125+0000 7f910e1af740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f91041afd30 con 0x557d3e8f0140 2024-04-26T01:55:53.125 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.125+0000 7f910e1af740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f91041adb80 con 0x557d3e8ef820 2024-04-26T01:55:53.125 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.125+0000 7f9103400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557d3e8f0140 0x557d3e8f21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:53.125 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.125+0000 7f9103e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d3e8ef820 0x557d3e9064f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:53.125 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.125+0000 7f9103e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d3e8ef820 0x557d3e9064f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:34060/0 (socket says 172.21.15.151:34060) 2024-04-26T01:55:53.125 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.125+0000 7f9103e00640 1 -- 172.21.15.151:0/568964751 learned_addr learned my addr 172.21.15.151:0/568964751 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:55:53.125 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.125+0000 7f9103400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557d3e8f0140 0x557d3e8f21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:58854/0 (socket says 172.21.15.151:58854) 2024-04-26T01:55:53.125 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.125+0000 7f9109200640 1 --2- 172.21.15.151:0/568964751 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557d3e8faa00 0x557d3e912b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:53.125 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.126+0000 7f9103e00640 1 -- 172.21.15.151:0/568964751 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557d3e8f0140 msgr2=0x557d3e8f21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:53.125 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.126+0000 7f9103e00640 1 --2- 172.21.15.151:0/568964751 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557d3e8f0140 0x557d3e8f21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:53.125 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.126+0000 7f9103e00640 1 -- 172.21.15.151:0/568964751 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557d3e8faa00 msgr2=0x557d3e912b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:53.126 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.126+0000 7f9103e00640 1 --2- 172.21.15.151:0/568964751 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557d3e8faa00 0x557d3e912b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:53.126 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.126+0000 7f9103e00640 1 -- 172.21.15.151:0/568964751 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d3e8f0730 con 0x557d3e8ef820 2024-04-26T01:55:53.126 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.126+0000 7f9103400640 1 --2- 172.21.15.151:0/568964751 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557d3e8f0140 0x557d3e8f21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:55:53.126 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.126+0000 7f9109200640 1 --2- 172.21.15.151:0/568964751 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557d3e8faa00 0x557d3e912b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:55:53.126 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.126+0000 7f9103e00640 1 --2- 172.21.15.151:0/568964751 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d3e8ef820 0x557d3e9064f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f90f40eeab0 tx=0x7f90f4032fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2ec2ab2922138d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:53.129 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.126+0000 7f90f8c00640 1 -- 172.21.15.151:0/568964751 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f404a040 con 0x557d3e8ef820 2024-04-26T01:55:53.129 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.126+0000 7f90f8c00640 1 -- 172.21.15.151:0/568964751 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90f40bd300 con 0x557d3e8ef820 2024-04-26T01:55:53.129 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.126+0000 7f90f8c00640 1 -- 172.21.15.151:0/568964751 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f4031b10 con 0x557d3e8ef820 2024-04-26T01:55:53.129 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.127+0000 7f910e1af740 1 -- 172.21.15.151:0/568964751 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d3e8ef820 msgr2=0x557d3e9064f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:53.129 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.127+0000 7f910e1af740 1 --2- 172.21.15.151:0/568964751 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d3e8ef820 0x557d3e9064f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f90f40eeab0 tx=0x7f90f4032fb0 comp rx=0 tx=0).stop 2024-04-26T01:55:53.130 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.127+0000 7f910e1af740 1 -- 172.21.15.151:0/568964751 shutdown_connections 2024-04-26T01:55:53.130 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.127+0000 7f910e1af740 1 --2- 172.21.15.151:0/568964751 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557d3e8faa00 0x557d3e912b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:53.130 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.127+0000 7f910e1af740 1 --2- 172.21.15.151:0/568964751 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557d3e8f0140 0x557d3e8f21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:53.130 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.127+0000 7f910e1af740 1 --2- 172.21.15.151:0/568964751 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d3e8ef820 0x557d3e9064f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:53.130 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.127+0000 7f910e1af740 1 -- 172.21.15.151:0/568964751 >> 172.21.15.151:0/568964751 conn(0x557d3e8ebd30 msgr2=0x557d3e8eae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:53.130 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.127+0000 7f910e1af740 1 -- 172.21.15.151:0/568964751 shutdown_connections 2024-04-26T01:55:53.130 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.127+0000 7f910e1af740 1 -- 172.21.15.151:0/568964751 wait complete. 2024-04-26T01:55:53.130 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.128+0000 7f910e1af740 1 -- start start 2024-04-26T01:55:53.130 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.128+0000 7f910e1af740 1 Processor -- start 2024-04-26T01:55:53.130 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.128+0000 7f910e1af740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d3e8f0140 0x557d3e9006f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:53.130 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.128+0000 7f910e1af740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557d3e8faa00 0x557d3e900c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.129+0000 7f910e1af740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557d3e9011c0 0x557d3e9703c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.129+0000 7f910e1af740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f91041ae1f0 con 0x557d3e9011c0 2024-04-26T01:55:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.129+0000 7f910e1af740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f91041af660 con 0x557d3e8faa00 2024-04-26T01:55:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.129+0000 7f910e1af740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f91041ae8c0 con 0x557d3e8f0140 2024-04-26T01:55:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.129+0000 7f910e1af740 10 client.0 _ll_register_callbacks cb 0x7f90f00127f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:55:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.129+0000 7f9103400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557d3e8faa00 0x557d3e900c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.129+0000 7f9103e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d3e8f0140 0x557d3e9006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.129+0000 7f9103400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557d3e8faa00 0x557d3e900c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:58858/0 (socket says 172.21.15.151:58858) 2024-04-26T01:55:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.129+0000 7f9103400640 1 -- 172.21.15.151:0/1957080358 learned_addr learned my addr 172.21.15.151:0/1957080358 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:55:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.129+0000 7f9109200640 1 --2- 172.21.15.151:0/1957080358 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557d3e9011c0 0x557d3e9703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.129+0000 7f9103400640 1 -- 172.21.15.151:0/1957080358 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d3e8f0140 msgr2=0x557d3e9006f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.129+0000 7f9103400640 1 --2- 172.21.15.151:0/1957080358 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d3e8f0140 0x557d3e9006f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.129+0000 7f9103400640 1 -- 172.21.15.151:0/1957080358 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557d3e9011c0 msgr2=0x557d3e9703c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:53.132 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.129+0000 7f9103400640 1 --2- 172.21.15.151:0/1957080358 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557d3e9011c0 0x557d3e9703c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:53.132 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.129+0000 7f9103400640 1 -- 172.21.15.151:0/1957080358 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d3e970b00 con 0x557d3e8faa00 2024-04-26T01:55:53.132 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.129+0000 7f9103e00640 1 --2- 172.21.15.151:0/1957080358 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d3e8f0140 0x557d3e9006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:55:53.132 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.129+0000 7f9103400640 1 --2- 172.21.15.151:0/1957080358 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557d3e8faa00 0x557d3e900c30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x557d3e8f7270 tx=0x7f90f001ccb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9c2a0c84a8ce5c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:53.132 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.130+0000 7f90ee000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T01:55:53.132 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.130+0000 7f90ee000640 1 -- 172.21.15.151:0/1957080358 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f0004890 con 0x557d3e8faa00 2024-04-26T01:55:53.132 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.130+0000 7f910e1af740 1 -- 172.21.15.151:0/1957080358 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90f40bd300 con 0x557d3e8faa00 2024-04-26T01:55:53.132 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.130+0000 7f90ee000640 1 -- 172.21.15.151:0/1957080358 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f90f0004a30 con 0x557d3e8faa00 2024-04-26T01:55:53.132 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.130+0000 7f90ee000640 1 -- 172.21.15.151:0/1957080358 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f90f0004c30 con 0x557d3e8faa00 2024-04-26T01:55:53.132 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.130+0000 7f910e1af740 10 client.25439 fetch_fsmap learned FSMap version 28 2024-04-26T01:55:53.132 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.130+0000 7f910e1af740 1 -- 172.21.15.151:0/1957080358 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557d3e971e10 con 0x557d3e8faa00 2024-04-26T01:55:53.132 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.130+0000 7f90ee000640 1 -- 172.21.15.151:0/1957080358 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90f40bd300 con 0x557d3e8faa00 2024-04-26T01:55:53.133 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.130+0000 7f90ee000640 1 -- 172.21.15.151:0/1957080358 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f90f0037050 con 0x557d3e8faa00 2024-04-26T01:55:53.133 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.131+0000 7f910e1af740 10 client.25439 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T01:55:53.133 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.131+0000 7f910e1af740 10 client.25439 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:55:53.133 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.131+0000 7f910e1af740 20 client.25439 populate_metadata read hostname 'smithi151' 2024-04-26T01:55:53.133 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.131+0000 7f910e1af740 1 --2- 172.21.15.151:0/1957080358 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x557d3e972650 0x557d3e974a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:53.133 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.131+0000 7f910e1af740 4 client.25439 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:55:53.133 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.131+0000 7f910e1af740 1 -- 172.21.15.151:0/1957080358 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557d3e85d190 con 0x557d3e972650 2024-04-26T01:55:53.133 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.131+0000 7f9103e00640 1 --2- 172.21.15.151:0/1957080358 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x557d3e972650 0x557d3e974a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:53.133 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.131+0000 7f9103e00640 1 --2- 172.21.15.151:0/1957080358 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x557d3e972650 0x557d3e974a70 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2c5f4d8d9874ad4 server_cookie=29a4839338e2ef81 in_seq=0 out_seq=0 2024-04-26T01:55:53.134 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.131+0000 7f90ee000640 10 client.25439 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:55:53.225 INFO:tasks.workunit.client.0.smithi151.stdout:checking for FICLONE... yes 2024-04-26T01:55:53.320 INFO:tasks.workunit.client.0.smithi151.stdout:checking for renameat2... yes 2024-04-26T01:55:53.419 INFO:tasks.workunit.client.0.smithi151.stdout:checking for reallocarray... yes 2024-04-26T01:55:53.592 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.593+0000 7f90ee000640 1 -- 172.21.15.151:0/1957080358 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557d3e85d190 con 0x557d3e972650 2024-04-26T01:55:53.593 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.593+0000 7f90ee000640 10 client.25439 handle_command_reply: tid=0 2024-04-26T01:55:53.593 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.593+0000 7f910e1af740 1 client.25439 shutdown 2024-04-26T01:55:53.593 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.593+0000 7f910e1af740 1 -- 172.21.15.151:0/1957080358 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557d3e8faa00 msgr2=0x557d3e900c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:53.593 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.593+0000 7f910e1af740 1 --2- 172.21.15.151:0/1957080358 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557d3e8faa00 0x557d3e900c30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x557d3e8f7270 tx=0x7f90f001ccb0 comp rx=0 tx=0).stop 2024-04-26T01:55:53.593 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.593+0000 7f910e1af740 1 -- 172.21.15.151:0/1957080358 shutdown_connections 2024-04-26T01:55:53.593 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.593+0000 7f910e1af740 1 --2- 172.21.15.151:0/1957080358 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x557d3e972650 0x557d3e974a70 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:53.594 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.593+0000 7f910e1af740 1 --2- 172.21.15.151:0/1957080358 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557d3e9011c0 0x557d3e9703c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:53.594 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.593+0000 7f910e1af740 1 --2- 172.21.15.151:0/1957080358 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557d3e8faa00 0x557d3e900c30 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:53.594 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.593+0000 7f910e1af740 1 --2- 172.21.15.151:0/1957080358 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d3e8f0140 0x557d3e9006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:53.594 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.593+0000 7f90ee000640 0 client.25439 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:55:53.594 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.593+0000 7f910e1af740 1 -- 172.21.15.151:0/1957080358 >> 172.21.15.151:0/1957080358 conn(0x557d3e8ebd30 msgr2=0x557d3e8ea600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:53.594 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.593+0000 7f910e1af740 1 -- 172.21.15.151:0/1957080358 shutdown_connections 2024-04-26T01:55:53.594 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.593+0000 7f910e1af740 1 -- 172.21.15.151:0/1957080358 wait complete. 2024-04-26T01:55:53.594 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.594+0000 7f910e1af740 20 client.25439 trim_cache size 0 max 16384 2024-04-26T01:55:53.595 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:{ 2024-04-26T01:55:53.595 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "message": "", 2024-04-26T01:55:53.595 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "return_code": 0 2024-04-26T01:55:53.595 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:} 2024-04-26T01:55:53.596 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.595+0000 7f910b200640 1 -- 172.21.15.151:0/2036117715 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f90e80a3dc0 msgr2=0x7f90e80a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:53.596 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.595+0000 7f910b200640 1 --2- 172.21.15.151:0/2036117715 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f90e80a3dc0 0x7f90e80a6280 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f90fc005c10 tx=0x7f90fc009a20 comp rx=0 tx=0).stop 2024-04-26T01:55:53.596 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.595+0000 7f910b200640 1 -- 172.21.15.151:0/2036117715 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91041031e0 msgr2=0x7f91041a0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:53.596 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.595+0000 7f910b200640 1 --2- 172.21.15.151:0/2036117715 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91041031e0 0x7f91041a0660 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f90f402a980 tx=0x7f90f4030fa0 comp rx=0 tx=0).stop 2024-04-26T01:55:53.596 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.595+0000 7f910b200640 1 -- 172.21.15.151:0/2036117715 shutdown_connections 2024-04-26T01:55:53.596 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.595+0000 7f910b200640 1 --2- 172.21.15.151:0/2036117715 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f90e80a3dc0 0x7f90e80a6280 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:53.596 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.595+0000 7f910b200640 1 --2- 172.21.15.151:0/2036117715 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f910410c520 0x7f910419a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:53.596 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.595+0000 7f910b200640 1 --2- 172.21.15.151:0/2036117715 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9104103b00 0x7f91041a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:53.596 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.595+0000 7f910b200640 1 --2- 172.21.15.151:0/2036117715 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91041031e0 0x7f91041a0660 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.595+0000 7f910b200640 1 -- 172.21.15.151:0/2036117715 >> 172.21.15.151:0/2036117715 conn(0x7f9104077fd0 msgr2=0x7f91041083f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.595+0000 7f910b200640 1 -- 172.21.15.151:0/2036117715 shutdown_connections 2024-04-26T01:55:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:55:53.595+0000 7f910b200640 1 -- 172.21.15.151:0/2036117715 wait complete. 2024-04-26T01:55:53.614 INFO:tasks.workunit.client.0.smithi151.stdout:checking for struct mount_attr... yes 2024-04-26T01:55:53.741 INFO:tasks.workunit.client.0.smithi151.stdout:checking for struct btrfs_qgroup_limit... yes 2024-04-26T01:55:53.873 INFO:tasks.workunit.client.0.smithi151.stdout:checking for struct btrfs_qgroup_inherit... yes 2024-04-26T01:55:54.008 INFO:tasks.workunit.client.0.smithi151.stdout:checking for struct btrfs_ioctl_vol_args... yes 2024-04-26T01:55:54.144 INFO:tasks.workunit.client.0.smithi151.stdout:checking for struct btrfs_ioctl_vol_args_v2... yes 2024-04-26T01:55:54.284 INFO:tasks.workunit.client.0.smithi151.stdout:checking for struct btrfs_ioctl_ino_lookup_args... yes 2024-04-26T01:55:54.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:55:53 smithi172 ceph-mon[32785]: pgmap v71: 97 pgs: 97 active+clean; 28 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 705 KiB/s wr, 583 op/s 2024-04-26T01:55:54.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:55:53 smithi165 ceph-mon[32648]: pgmap v71: 97 pgs: 97 active+clean; 28 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 705 KiB/s wr, 583 op/s 2024-04-26T01:55:54.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:55:53 smithi151 ceph-mon[28057]: pgmap v71: 97 pgs: 97 active+clean; 28 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 705 KiB/s wr, 583 op/s 2024-04-26T01:55:54.425 INFO:tasks.workunit.client.0.smithi151.stdout:checking for struct btrfs_ioctl_ino_lookup_user_args... yes 2024-04-26T01:55:54.569 INFO:tasks.workunit.client.0.smithi151.stdout:checking for struct btrfs_ioctl_get_subvol_rootref_args... yes 2024-04-26T01:55:54.665 INFO:tasks.workunit.client.0.smithi151.stdout:checking linux/btrfs.h usability... yes 2024-04-26T01:55:54.702 INFO:tasks.workunit.client.0.smithi151.stdout:checking linux/btrfs.h presence... yes 2024-04-26T01:55:54.702 INFO:tasks.workunit.client.0.smithi151.stdout:checking for linux/btrfs.h... yes 2024-04-26T01:55:54.811 INFO:tasks.workunit.client.0.smithi151.stdout:checking linux/btrfs_tree.h usability... yes 2024-04-26T01:55:54.848 INFO:tasks.workunit.client.0.smithi151.stdout:checking linux/btrfs_tree.h presence... yes 2024-04-26T01:55:54.848 INFO:tasks.workunit.client.0.smithi151.stdout:checking for linux/btrfs_tree.h... yes 2024-04-26T01:55:54.938 INFO:tasks.workunit.client.0.smithi151.stdout:checking for struct btrfs_ioctl_vol_args_v2.subvolid... yes 2024-04-26T01:55:55.013 INFO:tasks.workunit.client.0.smithi151.stdout:checking whether BTRFS_IOC_SNAP_DESTROY_V2 is declared... yes 2024-04-26T01:55:55.056 INFO:tasks.workunit.client.0.smithi151.stdout:configure: creating ./config.status 2024-04-26T01:55:56.275 INFO:tasks.workunit.client.0.smithi151.stdout:config.status: creating include/builddefs 2024-04-26T01:55:56.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:55:55 smithi172 ceph-mon[32785]: pgmap v72: 97 pgs: 97 active+clean; 28 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 257 KiB/s wr, 18 op/s 2024-04-26T01:55:56.324 INFO:tasks.workunit.client.0.smithi151.stdout:config.status: creating include/config.h 2024-04-26T01:55:56.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:55:55 smithi165 ceph-mon[32648]: pgmap v72: 97 pgs: 97 active+clean; 28 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 257 KiB/s wr, 18 op/s 2024-04-26T01:55:56.360 INFO:tasks.workunit.client.0.smithi151.stdout:config.status: executing libtool commands 2024-04-26T01:55:56.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:55:55 smithi151 ceph-mon[28057]: pgmap v72: 97 pgs: 97 active+clean; 28 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 257 KiB/s wr, 18 op/s 2024-04-26T01:55:56.531 INFO:tasks.workunit.client.0.smithi151.stdout:Building include 2024-04-26T01:55:56.531 INFO:tasks.workunit.client.0.smithi151.stdout:Building lib 2024-04-26T01:55:56.532 INFO:tasks.workunit.client.0.smithi151.stdout:Building tests 2024-04-26T01:55:56.543 INFO:tasks.workunit.client.0.smithi151.stdout: [LTDEP] 2024-04-26T01:55:56.553 INFO:tasks.workunit.client.0.smithi151.stdout:Building btrfs 2024-04-26T01:55:56.553 INFO:tasks.workunit.client.0.smithi151.stdout:Building ceph 2024-04-26T01:55:56.555 INFO:tasks.workunit.client.0.smithi151.stdout:Building cifs 2024-04-26T01:55:56.572 INFO:tasks.workunit.client.0.smithi151.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/ceph/group.list 2024-04-26T01:55:56.572 INFO:tasks.workunit.client.0.smithi151.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/cifs/group.list 2024-04-26T01:55:56.597 INFO:tasks.workunit.client.0.smithi151.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/btrfs/group.list 2024-04-26T01:55:56.621 INFO:tasks.workunit.client.0.smithi151.stdout:Building ext4 2024-04-26T01:55:56.637 INFO:tasks.workunit.client.0.smithi151.stdout:Building f2fs 2024-04-26T01:55:56.650 INFO:tasks.workunit.client.0.smithi151.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/f2fs/group.list 2024-04-26T01:55:56.653 INFO:tasks.workunit.client.0.smithi151.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/ext4/group.list 2024-04-26T01:55:56.709 INFO:tasks.workunit.client.0.smithi151.stdout:Building generic 2024-04-26T01:55:56.784 INFO:tasks.workunit.client.0.smithi151.stdout:Building nfs 2024-04-26T01:55:56.798 INFO:tasks.workunit.client.0.smithi151.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/nfs/group.list 2024-04-26T01:55:56.831 INFO:tasks.workunit.client.0.smithi151.stdout:Building ocfs2 2024-04-26T01:55:56.846 INFO:tasks.workunit.client.0.smithi151.stdout:Building overlay 2024-04-26T01:55:56.846 INFO:tasks.workunit.client.0.smithi151.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/ocfs2/group.list 2024-04-26T01:55:56.881 INFO:tasks.workunit.client.0.smithi151.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/overlay/group.list 2024-04-26T01:55:56.907 INFO:tasks.workunit.client.0.smithi151.stdout:Building perf 2024-04-26T01:55:56.922 INFO:tasks.workunit.client.0.smithi151.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/perf/group.list 2024-04-26T01:55:56.923 INFO:tasks.workunit.client.0.smithi151.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/generic/group.list 2024-04-26T01:55:56.939 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] dataascii.lo 2024-04-26T01:55:56.960 INFO:tasks.workunit.client.0.smithi151.stdout:Building selftest 2024-04-26T01:55:56.973 INFO:tasks.workunit.client.0.smithi151.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/selftest/group.list 2024-04-26T01:55:57.017 INFO:tasks.workunit.client.0.smithi151.stdout:Building shared 2024-04-26T01:55:57.030 INFO:tasks.workunit.client.0.smithi151.stdout:Building tmpfs 2024-04-26T01:55:57.031 INFO:tasks.workunit.client.0.smithi151.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/shared/group.list 2024-04-26T01:55:57.046 INFO:tasks.workunit.client.0.smithi151.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/tmpfs/group.list 2024-04-26T01:55:57.090 INFO:tasks.workunit.client.0.smithi151.stdout:Building udf 2024-04-26T01:55:57.092 INFO:tasks.workunit.client.0.smithi151.stdout:Building xfs 2024-04-26T01:55:57.103 INFO:tasks.workunit.client.0.smithi151.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/udf/group.list 2024-04-26T01:55:57.143 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] databin.lo 2024-04-26T01:55:57.282 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] datapid.lo 2024-04-26T01:55:57.288 INFO:tasks.workunit.client.0.smithi151.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/xfs/group.list 2024-04-26T01:55:57.302 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] file_lock.lo 2024-04-26T01:55:57.442 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] forker.lo 2024-04-26T01:55:57.504 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] pattern.lo 2024-04-26T01:55:57.601 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] open_flags.lo 2024-04-26T01:55:57.676 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] random_range.lo 2024-04-26T01:55:58.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:55:57 smithi172 ceph-mon[32785]: pgmap v73: 97 pgs: 97 active+clean; 28 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 257 KiB/s wr, 18 op/s 2024-04-26T01:55:58.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:55:57 smithi165 ceph-mon[32648]: pgmap v73: 97 pgs: 97 active+clean; 28 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 257 KiB/s wr, 18 op/s 2024-04-26T01:55:58.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:55:57 smithi151 ceph-mon[28057]: pgmap v73: 97 pgs: 97 active+clean; 28 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 257 KiB/s wr, 18 op/s 2024-04-26T01:55:58.971 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] string_to_tokens.lo 2024-04-26T01:55:58.974 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] str_to_bytes.lo 2024-04-26T01:55:58.989 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] tlibio.lo 2024-04-26T01:55:59.184 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] write_log.lo 2024-04-26T01:55:59.192 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] random.lo 2024-04-26T01:55:59.550 INFO:tasks.workunit.client.0.smithi151.stdout: [LD] 2024-04-26T01:56:00.086 INFO:tasks.workunit.client.0.smithi151.stdout:Building ltp 2024-04-26T01:56:00.087 INFO:tasks.workunit.client.0.smithi151.stdout:Building src 2024-04-26T01:56:00.087 INFO:tasks.workunit.client.0.smithi151.stdout:Building m4 2024-04-26T01:56:00.087 INFO:tasks.workunit.client.0.smithi151.stdout:Building common 2024-04-26T01:56:00.098 INFO:tasks.workunit.client.0.smithi151.stdout: [DEP] 2024-04-26T01:56:00.115 INFO:tasks.workunit.client.0.smithi151.stdout: [DEP] 2024-04-26T01:56:00.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:55:59 smithi172 ceph-mon[32785]: pgmap v74: 97 pgs: 97 active+clean; 26 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 510 KiB/s wr, 156 op/s 2024-04-26T01:56:00.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:55:59 smithi165 ceph-mon[32648]: pgmap v74: 97 pgs: 97 active+clean; 26 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 510 KiB/s wr, 156 op/s 2024-04-26T01:56:00.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:55:59 smithi151 ceph-mon[28057]: pgmap v74: 97 pgs: 97 active+clean; 26 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 510 KiB/s wr, 156 op/s 2024-04-26T01:56:00.518 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] doio 2024-04-26T01:56:00.520 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] fsstress 2024-04-26T01:56:00.520 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] fsx 2024-04-26T01:56:01.421 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] growfiles 2024-04-26T01:56:01.483 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] iogen 2024-04-26T01:56:02.044 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] aio-stress 2024-04-26T01:56:02.174 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:56:01 smithi151 ceph-mon[28057]: pgmap v75: 97 pgs: 97 active+clean; 26 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 381 KiB/s wr, 146 op/s 2024-04-26T01:56:02.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:56:01 smithi172 ceph-mon[32785]: pgmap v75: 97 pgs: 97 active+clean; 26 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 381 KiB/s wr, 146 op/s 2024-04-26T01:56:02.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:56:01 smithi165 ceph-mon[32648]: pgmap v75: 97 pgs: 97 active+clean; 26 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 381 KiB/s wr, 146 op/s 2024-04-26T01:56:04.073 INFO:tasks.workunit.client.0.smithi151.stdout: [DEP] 2024-04-26T01:56:04.077 INFO:tasks.workunit.client.0.smithi151.stdout: [DEP] 2024-04-26T01:56:04.077 INFO:tasks.workunit.client.0.smithi151.stdout: [DEP] 2024-04-26T01:56:04.081 INFO:tasks.workunit.client.0.smithi151.stdout: [DEP] 2024-04-26T01:56:04.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:56:03 smithi172 ceph-mon[32785]: pgmap v76: 97 pgs: 97 active+clean; 26 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 410 KiB/s wr, 164 op/s 2024-04-26T01:56:04.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:56:03 smithi165 ceph-mon[32648]: pgmap v76: 97 pgs: 97 active+clean; 26 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 410 KiB/s wr, 164 op/s 2024-04-26T01:56:04.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:56:03 smithi151 ceph-mon[28057]: pgmap v76: 97 pgs: 97 active+clean; 26 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 410 KiB/s wr, 164 op/s 2024-04-26T01:56:04.590 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] dirstress 2024-04-26T01:56:04.593 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] fill 2024-04-26T01:56:04.595 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] fill2 2024-04-26T01:56:04.596 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] getpagesize 2024-04-26T01:56:04.880 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] holes 2024-04-26T01:56:04.941 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] lstat64 2024-04-26T01:56:05.016 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] nametest 2024-04-26T01:56:05.040 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] permname 2024-04-26T01:56:05.168 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-04-26T01:56:05.169 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-04-26T01:56:05.169 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:56:05.252 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] randholes 2024-04-26T01:56:05.273 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] runas 2024-04-26T01:56:05.404 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] truncfile 2024-04-26T01:56:05.424 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] usemem 2024-04-26T01:56:05.642 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] mmapcat 2024-04-26T01:56:05.676 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:56:05.745 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] append_reader 2024-04-26T01:56:05.747 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] append_writer 2024-04-26T01:56:05.766 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] dirperf 2024-04-26T01:56:06.061 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] metaperf 2024-04-26T01:56:06.061 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] devzero 2024-04-26T01:56:06.063 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] feature 2024-04-26T01:56:06.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:56:05 smithi151 ceph-mon[28057]: pgmap v77: 97 pgs: 97 active+clean; 34 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 1.1 MiB/s wr, 281 op/s 2024-04-26T01:56:06.177 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] alloc 2024-04-26T01:56:06.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:56:05 smithi172 ceph-mon[32785]: pgmap v77: 97 pgs: 97 active+clean; 34 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 1.1 MiB/s wr, 281 op/s 2024-04-26T01:56:06.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:56:05 smithi165 ceph-mon[32648]: pgmap v77: 97 pgs: 97 active+clean; 34 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 1.1 MiB/s wr, 281 op/s 2024-04-26T01:56:06.429 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] fault 2024-04-26T01:56:06.468 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] fstest 2024-04-26T01:56:06.493 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] t_access_root 2024-04-26T01:56:06.530 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] godown 2024-04-26T01:56:06.774 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] resvtest 2024-04-26T01:56:06.779 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.779+0000 7fa22a000640 1 -- 172.21.15.151:0/1118351797 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa2241072b0 msgr2=0x7fa22410ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:06.779 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.779+0000 7fa22a000640 1 --2- 172.21.15.151:0/1118351797 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa2241072b0 0x7fa22410ee80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa21c009be0 tx=0x7fa21c02a6f0 comp rx=0 tx=0).stop 2024-04-26T01:56:06.779 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.779+0000 7fa22a000640 1 -- 172.21.15.151:0/1118351797 shutdown_connections 2024-04-26T01:56:06.780 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.779+0000 7fa22a000640 1 --2- 172.21.15.151:0/1118351797 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa2241072b0 0x7fa22410ee80 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:06.780 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.779+0000 7fa22a000640 1 --2- 172.21.15.151:0/1118351797 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa224102b90 0x7fa224106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:06.780 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.779+0000 7fa22a000640 1 --2- 172.21.15.151:0/1118351797 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2241021e0 0x7fa2241025c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:06.780 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.779+0000 7fa22a000640 1 -- 172.21.15.151:0/1118351797 >> 172.21.15.151:0/1118351797 conn(0x7fa2240fda00 msgr2=0x7fa2240ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:06.780 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.779+0000 7fa22a000640 1 -- 172.21.15.151:0/1118351797 shutdown_connections 2024-04-26T01:56:06.780 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.779+0000 7fa22a000640 1 -- 172.21.15.151:0/1118351797 wait complete. 2024-04-26T01:56:06.780 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.781+0000 7fa22a000640 1 Processor -- start 2024-04-26T01:56:06.780 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.781+0000 7fa22a000640 1 -- start start 2024-04-26T01:56:06.782 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.781+0000 7fa22a000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2241021e0 0x7fa2241a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:06.782 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.781+0000 7fa22a000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa2241072b0 0x7fa2241a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:06.782 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.781+0000 7fa22a000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa22419a6c0 0x7fa22419ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:06.783 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.781+0000 7fa22a000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa224112200 con 0x7fa22419a6c0 2024-04-26T01:56:06.783 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.781+0000 7fa22a000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa224112080 con 0x7fa2241021e0 2024-04-26T01:56:06.783 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.781+0000 7fa22a000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa224112380 con 0x7fa2241072b0 2024-04-26T01:56:06.783 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.782+0000 7fa223400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2241021e0 0x7fa2241a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:06.783 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.782+0000 7fa223400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2241021e0 0x7fa2241a0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:48228/0 (socket says 172.21.15.151:48228) 2024-04-26T01:56:06.783 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.782+0000 7fa223400640 1 -- 172.21.15.151:0/2885200500 learned_addr learned my addr 172.21.15.151:0/2885200500 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:56:06.783 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.782+0000 7fa222a00640 1 --2- 172.21.15.151:0/2885200500 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa2241072b0 0x7fa2241a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:06.783 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.782+0000 7fa223e00640 1 --2- 172.21.15.151:0/2885200500 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa22419a6c0 0x7fa22419ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:06.783 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.782+0000 7fa223400640 1 -- 172.21.15.151:0/2885200500 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa2241072b0 msgr2=0x7fa2241a0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:06.783 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.782+0000 7fa223400640 1 --2- 172.21.15.151:0/2885200500 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa2241072b0 0x7fa2241a0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:06.783 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.783+0000 7fa223400640 1 -- 172.21.15.151:0/2885200500 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa22419a6c0 msgr2=0x7fa22419ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:06.783 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.783+0000 7fa223400640 1 --2- 172.21.15.151:0/2885200500 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa22419a6c0 0x7fa22419ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.783+0000 7fa223400640 1 -- 172.21.15.151:0/2885200500 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa22419b2c0 con 0x7fa2241021e0 2024-04-26T01:56:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.783+0000 7fa223e00640 1 --2- 172.21.15.151:0/2885200500 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa22419a6c0 0x7fa22419ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:56:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.783+0000 7fa223400640 1 --2- 172.21.15.151:0/2885200500 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2241021e0 0x7fa2241a0600 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa21400e9c0 tx=0x7fa21400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.784+0000 7fa21be00640 1 -- 172.21.15.151:0/2885200500 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa21400ccc0 con 0x7fa2241021e0 2024-04-26T01:56:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.784+0000 7fa21be00640 1 -- 172.21.15.151:0/2885200500 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2140054d0 con 0x7fa2241021e0 2024-04-26T01:56:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.784+0000 7fa22a000640 1 -- 172.21.15.151:0/2885200500 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2241a73a0 con 0x7fa2241021e0 2024-04-26T01:56:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.784+0000 7fa21be00640 1 -- 172.21.15.151:0/2885200500 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa214010430 con 0x7fa2241021e0 2024-04-26T01:56:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.784+0000 7fa22a000640 1 -- 172.21.15.151:0/2885200500 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2241a7800 con 0x7fa2241021e0 2024-04-26T01:56:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.788+0000 7fa21be00640 1 -- 172.21.15.151:0/2885200500 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7fa214004f80 con 0x7fa2241021e0 2024-04-26T01:56:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.788+0000 7fa22cf87740 1 Processor -- start 2024-04-26T01:56:06.788 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.789+0000 7fa22cf87740 1 -- start start 2024-04-26T01:56:06.788 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.789+0000 7fa22cf87740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c1d43ed030 0x55c1d43ed410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:06.788 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.789+0000 7fa22cf87740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c1d43ed950 0x55c1d43f7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:06.788 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.789+0000 7fa22cf87740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c1d43f8210 0x55c1d43fa600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:06.788 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.789+0000 7fa22cf87740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c1d434a900 con 0x55c1d43f8210 2024-04-26T01:56:06.788 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.789+0000 7fa22cf87740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55c1d42f0ef0 con 0x55c1d43ed950 2024-04-26T01:56:06.789 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.789+0000 7fa22cf87740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c1d43468c0 con 0x55c1d43ed030 2024-04-26T01:56:06.789 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.790+0000 7fa223e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c1d43ed030 0x55c1d43ed410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.790+0000 7fa223e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c1d43ed030 0x55c1d43ed410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:47816/0 (socket says 172.21.15.151:47816) 2024-04-26T01:56:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.790+0000 7fa223e00640 1 -- 172.21.15.151:0/46131042 learned_addr learned my addr 172.21.15.151:0/46131042 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:56:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.790+0000 7fa222a00640 1 --2- 172.21.15.151:0/46131042 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c1d43f8210 0x55c1d43fa600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.790+0000 7fa223e00640 1 -- 172.21.15.151:0/46131042 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c1d43ed950 msgr2=0x55c1d43f7cd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T01:56:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.790+0000 7fa223e00640 1 --2- 172.21.15.151:0/46131042 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c1d43ed950 0x55c1d43f7cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.790+0000 7fa223e00640 1 -- 172.21.15.151:0/46131042 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c1d43f8210 msgr2=0x55c1d43fa600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.790+0000 7fa223e00640 1 --2- 172.21.15.151:0/46131042 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c1d43f8210 0x55c1d43fa600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.790+0000 7fa223e00640 1 -- 172.21.15.151:0/46131042 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1d405d210 con 0x55c1d43ed030 2024-04-26T01:56:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.791+0000 7fa223e00640 1 --2- 172.21.15.151:0/46131042 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c1d43ed030 0x55c1d43ed410 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa21c004af0 tx=0x7fa21c0304b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=175d3777f454ca4e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.791+0000 7fa21be00640 1 --2- 172.21.15.151:0/2885200500 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fa2080a3e40 0x7fa2080a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.791+0000 7fa20fe00640 1 -- 172.21.15.151:0/46131042 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa21c040030 con 0x55c1d43ed030 2024-04-26T01:56:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.791+0000 7fa20fe00640 1 -- 172.21.15.151:0/46131042 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa21c003e20 con 0x55c1d43ed030 2024-04-26T01:56:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.791+0000 7fa20fe00640 1 -- 172.21.15.151:0/46131042 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa21c036040 con 0x55c1d43ed030 2024-04-26T01:56:06.791 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] writemod 2024-04-26T01:56:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.791+0000 7fa21be00640 1 -- 172.21.15.151:0/2885200500 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fa2140d0bd0 con 0x7fa2241021e0 2024-04-26T01:56:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.792+0000 7fa223400640 1 --2- 172.21.15.151:0/2885200500 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fa2080a3e40 0x7fa2080a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:06.792 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.792+0000 7fa22cf87740 1 -- 172.21.15.151:0/46131042 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c1d43ed030 msgr2=0x55c1d43ed410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:06.792 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.792+0000 7fa22cf87740 1 --2- 172.21.15.151:0/46131042 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c1d43ed030 0x55c1d43ed410 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa21c004af0 tx=0x7fa21c0304b0 comp rx=0 tx=0).stop 2024-04-26T01:56:06.792 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.792+0000 7fa22cf87740 1 -- 172.21.15.151:0/46131042 shutdown_connections 2024-04-26T01:56:06.793 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.792+0000 7fa22cf87740 1 --2- 172.21.15.151:0/46131042 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c1d43f8210 0x55c1d43fa600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:06.793 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.792+0000 7fa22cf87740 1 --2- 172.21.15.151:0/46131042 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c1d43ed950 0x55c1d43f7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:06.793 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.792+0000 7fa22cf87740 1 --2- 172.21.15.151:0/46131042 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c1d43ed030 0x55c1d43ed410 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:06.793 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.792+0000 7fa22cf87740 1 -- 172.21.15.151:0/46131042 >> 172.21.15.151:0/46131042 conn(0x55c1d43e9540 msgr2=0x55c1d43e9920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:06.793 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.793+0000 7fa223400640 1 --2- 172.21.15.151:0/2885200500 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fa2080a3e40 0x7fa2080a6300 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x55c1d43ee5d0 tx=0x7fa2140bd040 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:06.793 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.793+0000 7fa22cf87740 1 -- 172.21.15.151:0/46131042 shutdown_connections 2024-04-26T01:56:06.793 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.793+0000 7fa22cf87740 1 -- 172.21.15.151:0/46131042 wait complete. 2024-04-26T01:56:06.793 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.793+0000 7fa22cf87740 1 -- start start 2024-04-26T01:56:06.793 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.793+0000 7fa22cf87740 1 Processor -- start 2024-04-26T01:56:06.793 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.794+0000 7fa22cf87740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c1d43ed030 0x55c1d43f5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:06.793 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.794+0000 7fa22cf87740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c1d43ed950 0x55c1d43f56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:06.793 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.794+0000 7fa22cf87740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c1d43f8210 0x55c1d43f5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:06.794 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.794+0000 7fa22cf87740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c1d43fade0 con 0x55c1d43ed950 2024-04-26T01:56:06.794 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.794+0000 7fa22cf87740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55c1d4304310 con 0x55c1d43f8210 2024-04-26T01:56:06.794 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.794+0000 7fa22cf87740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c1d43faf60 con 0x55c1d43ed030 2024-04-26T01:56:06.794 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.794+0000 7fa22cf87740 10 client.0 _ll_register_callbacks cb 0x55c1d42d2cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:56:06.795 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.795+0000 7fa222a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c1d43ed950 0x55c1d43f56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:06.795 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.795+0000 7fa222a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c1d43ed950 0x55c1d43f56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37688/0 (socket says 172.21.15.151:37688) 2024-04-26T01:56:06.795 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.795+0000 7fa223e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c1d43f8210 0x55c1d43f5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:06.796 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.795+0000 7fa222a00640 1 -- 172.21.15.151:0/735281161 learned_addr learned my addr 172.21.15.151:0/735281161 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:56:06.796 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.795+0000 7fa223e00640 1 --2- 172.21.15.151:0/735281161 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c1d43ed030 0x55c1d43f5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:06.796 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.795+0000 7fa223e00640 1 -- 172.21.15.151:0/735281161 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c1d43ed030 msgr2=0x55c1d43f5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:06.796 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.795+0000 7fa223e00640 1 --2- 172.21.15.151:0/735281161 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c1d43ed030 0x55c1d43f5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:06.796 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.795+0000 7fa223e00640 1 -- 172.21.15.151:0/735281161 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c1d43ed950 msgr2=0x55c1d43f56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:06.796 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.795+0000 7fa223e00640 1 --2- 172.21.15.151:0/735281161 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c1d43ed950 0x55c1d43f56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:06.796 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.795+0000 7fa223e00640 1 -- 172.21.15.151:0/735281161 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1d4129620 con 0x55c1d43f8210 2024-04-26T01:56:06.797 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.796+0000 7fa223e00640 1 --2- 172.21.15.151:0/735281161 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c1d43f8210 0x55c1d43f5c10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa21c02ac10 tx=0x7fa21c0304b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acd528e1098bb283 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:06.797 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.796+0000 7fa20cc00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T01:56:06.797 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.796+0000 7fa20cc00640 1 -- 172.21.15.151:0/735281161 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa21c040040 con 0x55c1d43f8210 2024-04-26T01:56:06.797 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.796+0000 7fa20cc00640 1 -- 172.21.15.151:0/735281161 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fa21c030be0 con 0x55c1d43f8210 2024-04-26T01:56:06.797 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.796+0000 7fa22cf87740 1 -- 172.21.15.151:0/735281161 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa21c003e20 con 0x55c1d43f8210 2024-04-26T01:56:06.798 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.797+0000 7fa20cc00640 1 -- 172.21.15.151:0/735281161 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa21c05a050 con 0x55c1d43f8210 2024-04-26T01:56:06.798 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.797+0000 7fa22cf87740 10 client.25448 fetch_fsmap learned FSMap version 28 2024-04-26T01:56:06.798 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.797+0000 7fa22cf87740 1 -- 172.21.15.151:0/735281161 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c1d4407090 con 0x55c1d43f8210 2024-04-26T01:56:06.798 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.797+0000 7fa20cc00640 1 -- 172.21.15.151:0/735281161 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa21c003e20 con 0x55c1d43f8210 2024-04-26T01:56:06.798 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.798+0000 7fa20cc00640 1 -- 172.21.15.151:0/735281161 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7fa21c0544c0 con 0x55c1d43f8210 2024-04-26T01:56:06.799 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.798+0000 7fa22cf87740 10 client.25448 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T01:56:06.799 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.798+0000 7fa22cf87740 10 client.25448 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:56:06.799 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.798+0000 7fa22cf87740 20 client.25448 populate_metadata read hostname 'smithi151' 2024-04-26T01:56:06.799 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.798+0000 7fa22cf87740 1 --2- 172.21.15.151:0/735281161 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55c1d44083e0 0x55c1d440a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:06.799 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.798+0000 7fa22cf87740 4 client.25448 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:56:06.799 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.798+0000 7fa22cf87740 1 -- 172.21.15.151:0/735281161 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c1d435e8b0 con 0x55c1d44083e0 2024-04-26T01:56:06.799 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.798+0000 7fa222a00640 1 --2- 172.21.15.151:0/735281161 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55c1d44083e0 0x55c1d440a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:06.799 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.799+0000 7fa222a00640 1 --2- 172.21.15.151:0/735281161 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55c1d44083e0 0x55c1d440a7d0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=734efb41bf69259a server_cookie=d51d2fb609dd57b3 in_seq=0 out_seq=0 2024-04-26T01:56:06.799 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.799+0000 7fa20cc00640 10 client.25448 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:56:06.800 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.800+0000 7fa20cc00640 1 -- 172.21.15.151:0/735281161 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c1d435e8b0 con 0x55c1d44083e0 2024-04-26T01:56:06.800 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.800+0000 7fa20cc00640 10 client.25448 handle_command_reply: tid=0 2024-04-26T01:56:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.800+0000 7fa22cf87740 1 client.25448 shutdown 2024-04-26T01:56:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.800+0000 7fa22cf87740 1 -- 172.21.15.151:0/735281161 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c1d43f8210 msgr2=0x55c1d43f5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.801+0000 7fa22cf87740 1 --2- 172.21.15.151:0/735281161 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c1d43f8210 0x55c1d43f5c10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa21c02ac10 tx=0x7fa21c0304b0 comp rx=0 tx=0).stop 2024-04-26T01:56:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.801+0000 7fa22cf87740 1 -- 172.21.15.151:0/735281161 shutdown_connections 2024-04-26T01:56:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.801+0000 7fa22cf87740 1 --2- 172.21.15.151:0/735281161 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55c1d44083e0 0x55c1d440a7d0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.801+0000 7fa22cf87740 1 --2- 172.21.15.151:0/735281161 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c1d43f8210 0x55c1d43f5c10 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.801+0000 7fa22cf87740 1 --2- 172.21.15.151:0/735281161 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c1d43ed950 0x55c1d43f56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.801+0000 7fa22cf87740 1 --2- 172.21.15.151:0/735281161 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c1d43ed030 0x55c1d43f5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.801+0000 7fa22cf87740 1 -- 172.21.15.151:0/735281161 >> 172.21.15.151:0/735281161 conn(0x55c1d43e9540 msgr2=0x55c1d43dfe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.801+0000 7fa20cc00640 0 client.25448 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:56:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.801+0000 7fa22cf87740 1 -- 172.21.15.151:0/735281161 shutdown_connections 2024-04-26T01:56:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.801+0000 7fa22cf87740 1 -- 172.21.15.151:0/735281161 wait complete. 2024-04-26T01:56:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.801+0000 7fa22cf87740 20 client.25448 trim_cache size 0 max 16384 2024-04-26T01:56:06.831 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.832+0000 7fa22cf87740 1 Processor -- start 2024-04-26T01:56:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.833+0000 7fa22cf87740 1 -- start start 2024-04-26T01:56:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.833+0000 7fa22cf87740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c1d43ed030 0x55c1d4403cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.833+0000 7fa22cf87740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c1d43ed950 0x55c1d43f3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.833+0000 7fa22cf87740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c1d43f8210 0x55c1d43f5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.833+0000 7fa22cf87740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa2241ae0e0 con 0x55c1d43f8210 2024-04-26T01:56:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.833+0000 7fa22cf87740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa2241ab9b0 con 0x55c1d43ed950 2024-04-26T01:56:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.833+0000 7fa22cf87740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa2241a9840 con 0x55c1d43ed030 2024-04-26T01:56:06.833 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.834+0000 7fa223e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c1d43ed030 0x55c1d4403cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:06.833 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.834+0000 7fa222a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c1d43ed950 0x55c1d43f3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:06.833 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.834+0000 7fa223e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c1d43ed030 0x55c1d4403cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:47838/0 (socket says 172.21.15.151:47838) 2024-04-26T01:56:06.833 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.834+0000 7fa222a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c1d43ed950 0x55c1d43f3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:48266/0 (socket says 172.21.15.151:48266) 2024-04-26T01:56:06.833 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.834+0000 7fa223e00640 1 -- 172.21.15.151:0/4258177872 learned_addr learned my addr 172.21.15.151:0/4258177872 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:56:06.833 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.834+0000 7fa223e00640 1 --2- 172.21.15.151:0/4258177872 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c1d43f8210 0x55c1d43f5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:06.833 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.834+0000 7fa222a00640 1 -- 172.21.15.151:0/4258177872 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c1d43ed030 msgr2=0x55c1d4403cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:06.833 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.834+0000 7fa222a00640 1 --2- 172.21.15.151:0/4258177872 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c1d43ed030 0x55c1d4403cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:06.834 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.834+0000 7fa222a00640 1 -- 172.21.15.151:0/4258177872 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c1d43f8210 msgr2=0x55c1d43f5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:06.834 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.834+0000 7fa222a00640 1 --2- 172.21.15.151:0/4258177872 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c1d43f8210 0x55c1d43f5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:06.834 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.834+0000 7fa222a00640 1 -- 172.21.15.151:0/4258177872 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1d43f4140 con 0x55c1d43ed950 2024-04-26T01:56:06.834 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.834+0000 7fa222a00640 1 --2- 172.21.15.151:0/4258177872 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c1d43ed950 0x55c1d43f3a60 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x55c1d440ad40 tx=0x7fa210005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12871644fb72139a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:06.834 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.835+0000 7fa20f400640 1 -- 172.21.15.151:0/4258177872 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2100120d0 con 0x55c1d43ed950 2024-04-26T01:56:06.834 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.835+0000 7fa20f400640 1 -- 172.21.15.151:0/4258177872 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa210012270 con 0x55c1d43ed950 2024-04-26T01:56:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.835+0000 7fa20f400640 1 -- 172.21.15.151:0/4258177872 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2100108d0 con 0x55c1d43ed950 2024-04-26T01:56:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.836+0000 7fa22cf87740 1 -- 172.21.15.151:0/4258177872 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c1d43ed950 msgr2=0x55c1d43f3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.836+0000 7fa22cf87740 1 --2- 172.21.15.151:0/4258177872 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c1d43ed950 0x55c1d43f3a60 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x55c1d440ad40 tx=0x7fa210005f90 comp rx=0 tx=0).stop 2024-04-26T01:56:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.836+0000 7fa22cf87740 1 -- 172.21.15.151:0/4258177872 shutdown_connections 2024-04-26T01:56:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.836+0000 7fa22cf87740 1 --2- 172.21.15.151:0/4258177872 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c1d43f8210 0x55c1d43f5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.836+0000 7fa22cf87740 1 --2- 172.21.15.151:0/4258177872 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c1d43ed950 0x55c1d43f3a60 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.836+0000 7fa22cf87740 1 --2- 172.21.15.151:0/4258177872 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c1d43ed030 0x55c1d4403cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.836+0000 7fa22cf87740 1 -- 172.21.15.151:0/4258177872 >> 172.21.15.151:0/4258177872 conn(0x55c1d43e9540 msgr2=0x55c1d43dfe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.837+0000 7fa22cf87740 1 -- 172.21.15.151:0/4258177872 shutdown_connections 2024-04-26T01:56:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.837+0000 7fa22cf87740 1 -- 172.21.15.151:0/4258177872 wait complete. 2024-04-26T01:56:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.837+0000 7fa22cf87740 1 -- start start 2024-04-26T01:56:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.837+0000 7fa22cf87740 1 Processor -- start 2024-04-26T01:56:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.838+0000 7fa22cf87740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c1d43ed030 0x55c1d44001a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.838+0000 7fa22cf87740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c1d43ed950 0x55c1d44006e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.838+0000 7fa22cf87740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c1d43f8210 0x55c1d43fd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.838+0000 7fa22cf87740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa2241a9e90 con 0x55c1d43ed950 2024-04-26T01:56:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.838+0000 7fa22cf87740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa2241ab2e0 con 0x55c1d43f8210 2024-04-26T01:56:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.838+0000 7fa22cf87740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa2241aa540 con 0x55c1d43ed030 2024-04-26T01:56:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.838+0000 7fa22cf87740 10 client.0 _ll_register_callbacks cb 0x7fa21c030a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:56:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.838+0000 7fa223e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c1d43ed030 0x55c1d44001a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.838+0000 7fa223e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c1d43ed030 0x55c1d44001a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:47846/0 (socket says 172.21.15.151:47846) 2024-04-26T01:56:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.838+0000 7fa223e00640 1 -- 172.21.15.151:0/1437751020 learned_addr learned my addr 172.21.15.151:0/1437751020 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:56:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.838+0000 7fa223e00640 1 --2- 172.21.15.151:0/1437751020 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c1d43f8210 0x55c1d43fd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.838+0000 7fa222a00640 1 --2- 172.21.15.151:0/1437751020 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c1d43ed950 0x55c1d44006e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.839+0000 7fa223e00640 1 -- 172.21.15.151:0/1437751020 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c1d43f8210 msgr2=0x55c1d43fd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.839+0000 7fa223e00640 1 --2- 172.21.15.151:0/1437751020 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c1d43f8210 0x55c1d43fd760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.839+0000 7fa223e00640 1 -- 172.21.15.151:0/1437751020 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c1d43ed950 msgr2=0x55c1d44006e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.839+0000 7fa223e00640 1 --2- 172.21.15.151:0/1437751020 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c1d43ed950 0x55c1d44006e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.839+0000 7fa223e00640 1 -- 172.21.15.151:0/1437751020 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1d43fdf60 con 0x55c1d43ed030 2024-04-26T01:56:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.839+0000 7fa222a00640 1 --2- 172.21.15.151:0/1437751020 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c1d43ed950 0x55c1d44006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:56:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.839+0000 7fa223e00640 1 --2- 172.21.15.151:0/1437751020 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c1d43ed030 0x55c1d44001a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa21c009b30 tx=0x7fa21c009860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e79a7460fc3cecb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.839+0000 7fa20cc00640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-26T01:56:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.839+0000 7fa20cc00640 1 -- 172.21.15.151:0/1437751020 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa21c036070 con 0x55c1d43ed030 2024-04-26T01:56:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.839+0000 7fa22cf87740 1 -- 172.21.15.151:0/1437751020 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa210012270 con 0x55c1d43ed030 2024-04-26T01:56:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.839+0000 7fa20cc00640 1 -- 172.21.15.151:0/1437751020 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fa21c004b40 con 0x55c1d43ed030 2024-04-26T01:56:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.840+0000 7fa20cc00640 1 -- 172.21.15.151:0/1437751020 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa21c040040 con 0x55c1d43ed030 2024-04-26T01:56:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.840+0000 7fa22cf87740 10 client.25480 fetch_fsmap learned FSMap version 28 2024-04-26T01:56:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.840+0000 7fa22cf87740 1 -- 172.21.15.151:0/1437751020 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c1d446e2a0 con 0x55c1d43ed030 2024-04-26T01:56:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.840+0000 7fa20cc00640 1 -- 172.21.15.151:0/1437751020 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa210012270 con 0x55c1d43ed030 2024-04-26T01:56:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.840+0000 7fa20cc00640 1 -- 172.21.15.151:0/1437751020 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7fa21c0540b0 con 0x55c1d43ed030 2024-04-26T01:56:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.840+0000 7fa22cf87740 10 client.25480 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T01:56:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.840+0000 7fa22cf87740 10 client.25480 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:56:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.840+0000 7fa22cf87740 20 client.25480 populate_metadata read hostname 'smithi151' 2024-04-26T01:56:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.840+0000 7fa22cf87740 1 --2- 172.21.15.151:0/1437751020 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55c1d446e910 0x55c1d4470d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.840+0000 7fa22cf87740 4 client.25480 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:56:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.840+0000 7fa22cf87740 1 -- 172.21.15.151:0/1437751020 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c1d4358320 con 0x55c1d446e910 2024-04-26T01:56:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.841+0000 7fa222a00640 1 --2- 172.21.15.151:0/1437751020 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55c1d446e910 0x55c1d4470d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.842+0000 7fa222a00640 1 --2- 172.21.15.151:0/1437751020 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55c1d446e910 0x55c1d4470d00 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=126f9c44713c17d server_cookie=e183cccc21e9506d in_seq=0 out_seq=0 2024-04-26T01:56:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.842+0000 7fa20cc00640 10 client.25480 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:56:06.870 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] writev_on_pagefault 2024-04-26T01:56:06.966 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.967+0000 7fa20cc00640 1 -- 172.21.15.151:0/1437751020 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c1d4358320 con 0x55c1d446e910 2024-04-26T01:56:06.966 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.967+0000 7fa20cc00640 10 client.25480 handle_command_reply: tid=0 2024-04-26T01:56:06.966 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.967+0000 7fa22cf87740 1 client.25480 shutdown 2024-04-26T01:56:06.966 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.967+0000 7fa22cf87740 1 -- 172.21.15.151:0/1437751020 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c1d43ed030 msgr2=0x55c1d44001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:06.966 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.967+0000 7fa22cf87740 1 --2- 172.21.15.151:0/1437751020 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c1d43ed030 0x55c1d44001a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa21c009b30 tx=0x7fa21c009860 comp rx=0 tx=0).stop 2024-04-26T01:56:06.967 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.968+0000 7fa22cf87740 1 -- 172.21.15.151:0/1437751020 shutdown_connections 2024-04-26T01:56:06.967 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.968+0000 7fa22cf87740 1 --2- 172.21.15.151:0/1437751020 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55c1d446e910 0x55c1d4470d00 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:06.967 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.968+0000 7fa22cf87740 1 --2- 172.21.15.151:0/1437751020 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c1d43f8210 0x55c1d43fd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:06.967 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.968+0000 7fa22cf87740 1 --2- 172.21.15.151:0/1437751020 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c1d43ed950 0x55c1d44006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:06.967 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.968+0000 7fa22cf87740 1 --2- 172.21.15.151:0/1437751020 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c1d43ed030 0x55c1d44001a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:06.967 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.968+0000 7fa22cf87740 1 -- 172.21.15.151:0/1437751020 >> 172.21.15.151:0/1437751020 conn(0x55c1d43e9540 msgr2=0x55c1d43ef270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:06.967 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.968+0000 7fa20cc00640 0 client.25480 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:56:06.967 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.968+0000 7fa22cf87740 1 -- 172.21.15.151:0/1437751020 shutdown_connections 2024-04-26T01:56:06.968 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.968+0000 7fa22cf87740 1 -- 172.21.15.151:0/1437751020 wait complete. 2024-04-26T01:56:06.968 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.969+0000 7fa22cf87740 20 client.25480 trim_cache size 0 max 16384 2024-04-26T01:56:06.968 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:{ 2024-04-26T01:56:06.968 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "message": "", 2024-04-26T01:56:06.968 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "return_code": 0 2024-04-26T01:56:06.968 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:} 2024-04-26T01:56:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.969+0000 7fa22a000640 1 -- 172.21.15.151:0/2885200500 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fa2080a3e40 msgr2=0x7fa2080a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.969+0000 7fa22a000640 1 --2- 172.21.15.151:0/2885200500 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fa2080a3e40 0x7fa2080a6300 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x55c1d43ee5d0 tx=0x7fa2140bd040 comp rx=0 tx=0).stop 2024-04-26T01:56:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.970+0000 7fa22a000640 1 -- 172.21.15.151:0/2885200500 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2241021e0 msgr2=0x7fa2241a0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.970+0000 7fa22a000640 1 --2- 172.21.15.151:0/2885200500 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2241021e0 0x7fa2241a0600 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa21400e9c0 tx=0x7fa21400ee80 comp rx=0 tx=0).stop 2024-04-26T01:56:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.970+0000 7fa22a000640 1 -- 172.21.15.151:0/2885200500 shutdown_connections 2024-04-26T01:56:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.970+0000 7fa22a000640 1 --2- 172.21.15.151:0/2885200500 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fa2080a3e40 0x7fa2080a6300 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.970+0000 7fa22a000640 1 --2- 172.21.15.151:0/2885200500 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa22419a6c0 0x7fa22419ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.970+0000 7fa22a000640 1 --2- 172.21.15.151:0/2885200500 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa2241072b0 0x7fa2241a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.970+0000 7fa22a000640 1 --2- 172.21.15.151:0/2885200500 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2241021e0 0x7fa2241a0600 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:06.970 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.970+0000 7fa22a000640 1 -- 172.21.15.151:0/2885200500 >> 172.21.15.151:0/2885200500 conn(0x7fa2240fda00 msgr2=0x7fa2240fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:06.970 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.970+0000 7fa22a000640 1 -- 172.21.15.151:0/2885200500 shutdown_connections 2024-04-26T01:56:06.970 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:06.970+0000 7fa22a000640 1 -- 172.21.15.151:0/2885200500 wait complete. 2024-04-26T01:56:06.988 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] makeextents 2024-04-26T01:56:07.091 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] itrash 2024-04-26T01:56:07.143 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] rename 2024-04-26T01:56:07.264 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] multi_open_unlink 2024-04-26T01:56:07.311 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] unwritten_sync 2024-04-26T01:56:07.371 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] genhashnames 2024-04-26T01:56:07.412 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] t_holes 2024-04-26T01:56:07.590 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] t_mmap_writev 2024-04-26T01:56:07.640 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] t_truncate_cmtime 2024-04-26T01:56:07.712 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] dirhash_collide 2024-04-26T01:56:07.737 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] t_rename_overwrite 2024-04-26T01:56:07.892 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] holetest 2024-04-26T01:56:07.951 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] t_truncate_self 2024-04-26T01:56:08.022 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] af_unix 2024-04-26T01:56:08.078 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:56:07 smithi151 ceph-mon[28057]: pgmap v78: 97 pgs: 97 active+clean; 34 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 1.1 MiB/s wr, 281 op/s 2024-04-26T01:56:08.082 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] t_mmap_stale_pmd 2024-04-26T01:56:08.228 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] t_mmap_cow_race 2024-04-26T01:56:08.264 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] t_mmap_fallocate 2024-04-26T01:56:08.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:56:07 smithi172 ceph-mon[32785]: pgmap v78: 97 pgs: 97 active+clean; 34 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 1.1 MiB/s wr, 281 op/s 2024-04-26T01:56:08.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:56:07 smithi165 ceph-mon[32648]: pgmap v78: 97 pgs: 97 active+clean; 34 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 1.1 MiB/s wr, 281 op/s 2024-04-26T01:56:08.366 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] fsync-err 2024-04-26T01:56:08.391 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] t_mmap_write_ro 2024-04-26T01:56:08.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:08.494+0000 7f5031a00700 1 -- 172.21.15.151:0/1200195010 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f502c074c70 msgr2=0x7f502c075050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:08.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:08.494+0000 7f5031a00700 1 --2- 172.21.15.151:0/1200195010 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f502c074c70 0x7f502c075050 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f501c0098f0 tx=0x7f501c009c00 comp rx=0 tx=0).stop 2024-04-26T01:56:08.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:08.494+0000 7f5031a00700 1 -- 172.21.15.151:0/1200195010 shutdown_connections 2024-04-26T01:56:08.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:08.494+0000 7f5031a00700 1 --2- 172.21.15.151:0/1200195010 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f502c10f440 0x7f502c111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:08.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:08.494+0000 7f5031a00700 1 --2- 172.21.15.151:0/1200195010 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f502c074c70 0x7f502c075050 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:08.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:08.494+0000 7f5031a00700 1 --2- 172.21.15.151:0/1200195010 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f502c075590 0x7f502c10ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:08.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:08.494+0000 7f5031a00700 1 -- 172.21.15.151:0/1200195010 >> 172.21.15.151:0/1200195010 conn(0x7f502c079ba0 msgr2=0x7f502c079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:08.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:08.495+0000 7f5031a00700 1 -- 172.21.15.151:0/1200195010 shutdown_connections 2024-04-26T01:56:08.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:08.496+0000 7f5031a00700 1 -- 172.21.15.151:0/1200195010 wait complete. 2024-04-26T01:56:08.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:08.497+0000 7f5031a00700 1 Processor -- start 2024-04-26T01:56:08.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:08.497+0000 7f5031a00700 1 -- start start 2024-04-26T01:56:08.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:08.498+0000 7f5031a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f502c074c70 0x7f502c19b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:08.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:08.498+0000 7f5031a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f502c075590 0x7f502c19b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:08.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:08.498+0000 7f5031a00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f502c10f440 0x7f502c19fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:08.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:08.498+0000 7f5031a00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f502c114e20 con 0x7f502c075590 2024-04-26T01:56:08.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:08.498+0000 7f5031a00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f502c114ca0 con 0x7f502c074c70 2024-04-26T01:56:08.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:08.498+0000 7f5031a00700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f502c114fa0 con 0x7f502c10f440 2024-04-26T01:56:08.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:08.498+0000 7f502aa00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f502c074c70 0x7f502c19b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:08.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:08.498+0000 7f502aa00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f502c074c70 0x7f502c19b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:48280/0 (socket says 172.21.15.151:48280) 2024-04-26T01:56:08.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:08.498+0000 7f502aa00700 1 -- 172.21.15.151:0/3542159643 learned_addr learned my addr 172.21.15.151:0/3542159643 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:56:08.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:08.499+0000 7f502b400700 1 --2- 172.21.15.151:0/3542159643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f502c10f440 0x7f502c19fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:08.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:08.499+0000 7f502a000700 1 --2- 172.21.15.151:0/3542159643 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f502c075590 0x7f502c19b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:08.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:08.499+0000 7f502aa00700 1 -- 172.21.15.151:0/3542159643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f502c10f440 msgr2=0x7f502c19fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:08.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:08.499+0000 7f502aa00700 1 --2- 172.21.15.151:0/3542159643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f502c10f440 0x7f502c19fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:08.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:08.499+0000 7f502aa00700 1 -- 172.21.15.151:0/3542159643 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f502c075590 msgr2=0x7f502c19b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:08.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:08.499+0000 7f502aa00700 1 --2- 172.21.15.151:0/3542159643 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f502c075590 0x7f502c19b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:08.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:08.499+0000 7f502aa00700 1 -- 172.21.15.151:0/3542159643 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f502c1a0340 con 0x7f502c074c70 2024-04-26T01:56:08.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:08.499+0000 7f502a000700 1 --2- 172.21.15.151:0/3542159643 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f502c075590 0x7f502c19b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:56:08.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:08.500+0000 7f502aa00700 1 --2- 172.21.15.151:0/3542159643 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f502c074c70 0x7f502c19b330 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f501c00f990 tx=0x7f501c00fa70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:08.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:08.500+0000 7f5023400700 1 -- 172.21.15.151:0/3542159643 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f501c005400 con 0x7f502c074c70 2024-04-26T01:56:08.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:08.500+0000 7f5023400700 1 -- 172.21.15.151:0/3542159643 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f501c0055a0 con 0x7f502c074c70 2024-04-26T01:56:08.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:08.500+0000 7f5023400700 1 -- 172.21.15.151:0/3542159643 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f501c004df0 con 0x7f502c074c70 2024-04-26T01:56:08.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:08.501+0000 7f5031a00700 1 -- 172.21.15.151:0/3542159643 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f502c1a05d0 con 0x7f502c074c70 2024-04-26T01:56:08.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:08.501+0000 7f5031a00700 1 -- 172.21.15.151:0/3542159643 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f502c1a0970 con 0x7f502c074c70 2024-04-26T01:56:08.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:08.503+0000 7f5031a00700 1 -- 172.21.15.151:0/3542159643 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f502c069730 con 0x7f502c074c70 2024-04-26T01:56:08.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:08.504+0000 7f5023400700 1 -- 172.21.15.151:0/3542159643 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f501c00b560 con 0x7f502c074c70 2024-04-26T01:56:08.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:08.506+0000 7f5023400700 1 --2- 172.21.15.151:0/3542159643 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f50100a3e30 0x7f50100a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:08.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:08.506+0000 7f5023400700 1 -- 172.21.15.151:0/3542159643 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f501c0d17d0 con 0x7f502c074c70 2024-04-26T01:56:08.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:08.506+0000 7f502a000700 1 --2- 172.21.15.151:0/3542159643 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f50100a3e30 0x7f50100a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:08.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:08.507+0000 7f502a000700 1 --2- 172.21.15.151:0/3542159643 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f50100a3e30 0x7f50100a62f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f502c111070 tx=0x7f5018009260 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:08.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:08.510+0000 7f5023400700 1 -- 172.21.15.151:0/3542159643 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f501c013030 con 0x7f502c074c70 2024-04-26T01:56:08.547 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] t_ext4_dax_journal_corruption 2024-04-26T01:56:08.581 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] t_ext4_dax_inline_corruption 2024-04-26T01:56:08.711 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] t_ofd_locks 2024-04-26T01:56:08.723 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] t_mmap_collision 2024-04-26T01:56:08.872 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] mmap-write-concurrent 2024-04-26T01:56:08.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:08.912+0000 7f5031a00700 1 -- 172.21.15.151:0/3542159643 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f502c02df80 con 0x7f502c074c70 2024-04-26T01:56:08.912 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:08.913+0000 7f5023400700 1 -- 172.21.15.151:0/3542159643 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11892 (secure 0 0 0) 0x7f501c09df10 con 0x7f502c074c70 2024-04-26T01:56:08.912 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:56:08.912 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2937864097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2937864097},{"type":"v1","addr":"172.21.15.151:6835","nonce":2937864097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14373,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/2947090802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2947090802},{"type":"v1","addr":"172.21.15.151:6837","nonce":2947090802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/1356563280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1356563280},{"type":"v1","addr":"172.21.15.151:6839","nonce":1356563280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6841/1075240552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1075240552},{"type":"v1","addr":"172.21.15.151:6841","nonce":1075240552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/504675912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":504675912},{"type":"v1","addr":"172.21.15.165:6833","nonce":504675912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1207580015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1207580015},{"type":"v1","addr":"172.21.15.172:6833","nonce":1207580015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:49:40.746876+0000","modified":"2024-04-26T01:55:28.126804+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24505,"mds_2":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/3598138294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3598138294},{"type":"v1","addr":"172.21.15.165:6835","nonce":3598138294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/1847859054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":1847859054},{"type":"v1","addr":"172.21.15.165:6837","nonce":1847859054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/2309751171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2309751171},{"type":"v1","addr":"172.21.15.172:6835","nonce":2309751171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.172:6837/55972537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":55972537},{"type":"v1","addr":"172.21.15.172:6837","nonce":55972537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.172:6839/1242302810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1242302810},{"type":"v1","addr":"172.21.15.172:6839","nonce":1242302810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/1969716254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1969716254},{"type":"v1","addr":"172.21.15.165:6839","nonce":1969716254}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24505,24509,24493]},"id":1}]} 2024-04-26T01:56:08.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:08.916+0000 7f5031a00700 1 -- 172.21.15.151:0/3542159643 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f50100a3e30 msgr2=0x7f50100a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:08.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:08.916+0000 7f5031a00700 1 --2- 172.21.15.151:0/3542159643 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f50100a3e30 0x7f50100a62f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f502c111070 tx=0x7f5018009260 comp rx=0 tx=0).stop 2024-04-26T01:56:08.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:08.916+0000 7f5031a00700 1 -- 172.21.15.151:0/3542159643 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f502c074c70 msgr2=0x7f502c19b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:08.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:08.916+0000 7f5031a00700 1 --2- 172.21.15.151:0/3542159643 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f502c074c70 0x7f502c19b330 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f501c00f990 tx=0x7f501c00fa70 comp rx=0 tx=0).stop 2024-04-26T01:56:08.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:08.917+0000 7f5031a00700 1 -- 172.21.15.151:0/3542159643 shutdown_connections 2024-04-26T01:56:08.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:08.917+0000 7f5031a00700 1 --2- 172.21.15.151:0/3542159643 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f50100a3e30 0x7f50100a62f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:08.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:08.917+0000 7f5031a00700 1 --2- 172.21.15.151:0/3542159643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f502c10f440 0x7f502c19fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:08.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:08.917+0000 7f5031a00700 1 --2- 172.21.15.151:0/3542159643 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f502c074c70 0x7f502c19b330 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:08.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:08.917+0000 7f5031a00700 1 --2- 172.21.15.151:0/3542159643 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f502c075590 0x7f502c19b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:08.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:08.917+0000 7f5031a00700 1 -- 172.21.15.151:0/3542159643 >> 172.21.15.151:0/3542159643 conn(0x7f502c079ba0 msgr2=0x7f502c0ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:08.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:08.917+0000 7f5031a00700 1 -- 172.21.15.151:0/3542159643 shutdown_connections 2024-04-26T01:56:08.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:08.917+0000 7f5031a00700 1 -- 172.21.15.151:0/3542159643 wait complete. 2024-04-26T01:56:08.918 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 28 2024-04-26T01:56:08.926 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] t_get_file_time 2024-04-26T01:56:09.118 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] t_create_short_dirs 2024-04-26T01:56:09.133 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] t_create_long_dirs 2024-04-26T01:56:09.197 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] t_enospc 2024-04-26T01:56:09.213 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] t_mmap_writev_overlap 2024-04-26T01:56:09.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:56:08 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/3542159643' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:56:09.318 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:56:08 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/3542159643' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:56:09.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:56:08 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/3542159643' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:56:09.430 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] checkpoint_journal 2024-04-26T01:56:09.451 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] mmap-rw-fault 2024-04-26T01:56:09.551 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] allocstale 2024-04-26T01:56:09.638 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] t_mmap_cow_memory_failure 2024-04-26T01:56:09.736 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] fake-dump-rootino 2024-04-26T01:56:09.773 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] dio-buf-fault 2024-04-26T01:56:09.872 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] rewinddir-test 2024-04-26T01:56:09.946 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] readdir-while-renames 2024-04-26T01:56:10.030 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] xfsctl 2024-04-26T01:56:10.052 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] bstat 2024-04-26T01:56:10.112 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24509 quiesce db / --timeout 60 --expiration 90.0 --await-for 42.0' 2024-04-26T01:56:10.112 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24509 quiesce db / --timeout 60 --expiration 90.0 --await-for 42.0 2024-04-26T01:56:10.207 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] t_mtab 2024-04-26T01:56:10.277 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] getdevicesize 2024-04-26T01:56:10.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:56:10 smithi165 ceph-mon[32648]: pgmap v79: 97 pgs: 97 active+clean; 30 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 1.2 MiB/s wr, 335 op/s 2024-04-26T01:56:10.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:56:10 smithi151 ceph-mon[28057]: pgmap v79: 97 pgs: 97 active+clean; 30 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 1.2 MiB/s wr, 335 op/s 2024-04-26T01:56:10.408 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] preallo_rw_pattern_reader 2024-04-26T01:56:10.457 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] preallo_rw_pattern_writer 2024-04-26T01:56:10.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:56:10 smithi172 ceph-mon[32785]: pgmap v79: 97 pgs: 97 active+clean; 30 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 1.2 MiB/s wr, 335 op/s 2024-04-26T01:56:10.594 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] ftrunc 2024-04-26T01:56:10.598 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] trunc 2024-04-26T01:56:10.613 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:56:10.745 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] fs_perms 2024-04-26T01:56:10.786 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] testx 2024-04-26T01:56:10.880 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] looptest 2024-04-26T01:56:10.921 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] locktest 2024-04-26T01:56:11.063 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] unwritten_mmap 2024-04-26T01:56:11.063 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] bulkstat_unlink_test 2024-04-26T01:56:11.243 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] deduperace 2024-04-26T01:56:11.370 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] bulkstat_unlink_test_modified 2024-04-26T01:56:11.419 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] t_dir_offset 2024-04-26T01:56:11.424 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] t_futimens 2024-04-26T01:56:11.583 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] t_immutable 2024-04-26T01:56:11.690 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] stale_handle 2024-04-26T01:56:11.705 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] pwrite_mmap_blocked 2024-04-26T01:56:11.706 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] t_dir_offset2 2024-04-26T01:56:11.987 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] seek_sanity_test 2024-04-26T01:56:11.997 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] seek_copy_test 2024-04-26T01:56:12.034 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] t_readdir_1 2024-04-26T01:56:12.069 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:56:12 smithi151 ceph-mon[28057]: pgmap v80: 97 pgs: 97 active+clean; 30 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 959 KiB/s wr, 198 op/s 2024-04-26T01:56:12.323 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] t_readdir_2 2024-04-26T01:56:12.335 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] fsync-tester 2024-04-26T01:56:12.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:56:12 smithi165 ceph-mon[32648]: pgmap v80: 97 pgs: 97 active+clean; 30 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 959 KiB/s wr, 198 op/s 2024-04-26T01:56:12.525 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] nsexec 2024-04-26T01:56:12.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:56:12 smithi172 ceph-mon[32785]: pgmap v80: 97 pgs: 97 active+clean; 30 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 959 KiB/s wr, 198 op/s 2024-04-26T01:56:12.632 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] cloner 2024-04-26T01:56:12.663 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] renameat2 2024-04-26T01:56:12.774 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] t_getcwd 2024-04-26T01:56:12.854 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] e4compact 2024-04-26T01:56:12.965 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] test-nextquota 2024-04-26T01:56:12.984 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] punch-alternating 2024-04-26T01:56:13.092 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] attr-list-by-handle-cursor-test 2024-04-26T01:56:13.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.180+0000 7f82a1a00700 1 -- 172.21.15.151:0/1838202527 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f829c102af0 msgr2=0x7f829c102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:13.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.180+0000 7f82a1a00700 1 --2- 172.21.15.151:0/1838202527 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f829c102af0 0x7f829c102f70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8288009990 tx=0x7f8288009ca0 comp rx=0 tx=0).stop 2024-04-26T01:56:13.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.180+0000 7f82a1a00700 1 -- 172.21.15.151:0/1838202527 shutdown_connections 2024-04-26T01:56:13.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.180+0000 7f82a1a00700 1 --2- 172.21.15.151:0/1838202527 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f829c1034b0 0x7f829c10f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:13.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.180+0000 7f82a1a00700 1 --2- 172.21.15.151:0/1838202527 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f829c107940 0x7f829c107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:13.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.180+0000 7f82a1a00700 1 --2- 172.21.15.151:0/1838202527 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f829c102af0 0x7f829c102f70 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:13.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.180+0000 7f82a1a00700 1 -- 172.21.15.151:0/1838202527 >> 172.21.15.151:0/1838202527 conn(0x7f829c079ba0 msgr2=0x7f829c079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:13.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.182+0000 7f82a1a00700 1 -- 172.21.15.151:0/1838202527 shutdown_connections 2024-04-26T01:56:13.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.182+0000 7f82a1a00700 1 -- 172.21.15.151:0/1838202527 wait complete. 2024-04-26T01:56:13.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.183+0000 7f82a1a00700 1 Processor -- start 2024-04-26T01:56:13.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.184+0000 7f82a1a00700 1 -- start start 2024-04-26T01:56:13.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.184+0000 7f82a1a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f829c102af0 0x7f829c1a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:13.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.184+0000 7f82a1a00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f829c1034b0 0x7f829c1a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:13.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.184+0000 7f82a1a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f829c107940 0x7f829c10e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:13.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.184+0000 7f82a1a00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f829c112d00 con 0x7f829c107940 2024-04-26T01:56:13.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.184+0000 7f82a1a00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f829c112b80 con 0x7f829c102af0 2024-04-26T01:56:13.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.184+0000 7f82a1a00700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f829c112e80 con 0x7f829c1034b0 2024-04-26T01:56:13.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.185+0000 7f829a000700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f829c1034b0 0x7f829c1a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:13.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.185+0000 7f829a000700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f829c1034b0 0x7f829c1a4890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:37000/0 (socket says 172.21.15.151:37000) 2024-04-26T01:56:13.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.185+0000 7f829a000700 1 -- 172.21.15.151:0/2876510180 learned_addr learned my addr 172.21.15.151:0/2876510180 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:56:13.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.185+0000 7f829aa00700 1 --2- 172.21.15.151:0/2876510180 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f829c102af0 0x7f829c1a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:13.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.185+0000 7f829a000700 1 -- 172.21.15.151:0/2876510180 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f829c102af0 msgr2=0x7f829c1a4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:13.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.185+0000 7f829a000700 1 --2- 172.21.15.151:0/2876510180 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f829c102af0 0x7f829c1a4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:13.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.185+0000 7f829a000700 1 -- 172.21.15.151:0/2876510180 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f829c107940 msgr2=0x7f829c10e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:13.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.185+0000 7f829a000700 1 --2- 172.21.15.151:0/2876510180 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f829c107940 0x7f829c10e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:13.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.185+0000 7f829a000700 1 -- 172.21.15.151:0/2876510180 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f829c10f170 con 0x7f829c1034b0 2024-04-26T01:56:13.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.186+0000 7f829a000700 1 --2- 172.21.15.151:0/2876510180 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f829c1034b0 0x7f829c1a4890 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8288000c00 tx=0x7f82880057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:13.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.186+0000 7f8293400700 1 -- 172.21.15.151:0/2876510180 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8288027070 con 0x7f829c1034b0 2024-04-26T01:56:13.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.186+0000 7f8293400700 1 -- 172.21.15.151:0/2876510180 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8288022470 con 0x7f829c1034b0 2024-04-26T01:56:13.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.186+0000 7f8293400700 1 -- 172.21.15.151:0/2876510180 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8288017650 con 0x7f829c1034b0 2024-04-26T01:56:13.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.186+0000 7f82a1a00700 1 -- 172.21.15.151:0/2876510180 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f829c10f400 con 0x7f829c1034b0 2024-04-26T01:56:13.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.186+0000 7f82a1a00700 1 -- 172.21.15.151:0/2876510180 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f829c1ac280 con 0x7f829c1034b0 2024-04-26T01:56:13.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.191+0000 7f8293400700 1 -- 172.21.15.151:0/2876510180 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f8288009d10 con 0x7f829c1034b0 2024-04-26T01:56:13.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.193+0000 7f82ac3c5b80 1 Processor -- start 2024-04-26T01:56:13.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.193+0000 7f82ac3c5b80 1 -- start start 2024-04-26T01:56:13.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.194+0000 7f82ac3c5b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bacd7eadd0 0x55bacd7eb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:13.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.194+0000 7f82ac3c5b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bacd7eb6f0 0x55bacd7f6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:13.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.194+0000 7f82ac3c5b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bacd7f6560 0x55bacd7f8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:13.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.194+0000 7f82ac3c5b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55bacd73e020 con 0x55bacd7f6560 2024-04-26T01:56:13.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.194+0000 7f82ac3c5b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55bacd6e48c0 con 0x55bacd7eadd0 2024-04-26T01:56:13.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.194+0000 7f82ac3c5b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55bacd739fe0 con 0x55bacd7eb6f0 2024-04-26T01:56:13.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.194+0000 7f829aa00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bacd7eb6f0 0x55bacd7f6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:13.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.194+0000 7f829aa00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bacd7eb6f0 0x55bacd7f6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:37016/0 (socket says 172.21.15.151:37016) 2024-04-26T01:56:13.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.194+0000 7f829aa00700 1 -- 172.21.15.151:0/927353412 learned_addr learned my addr 172.21.15.151:0/927353412 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:56:13.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.194+0000 7f829b400700 1 --2- 172.21.15.151:0/927353412 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bacd7eadd0 0x55bacd7eb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:13.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.194+0000 7f8293400700 1 --2- 172.21.15.151:0/2876510180 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f82800a3ed0 0x7f82800a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:13.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.195+0000 7f8293400700 1 -- 172.21.15.151:0/2876510180 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f82880d1610 con 0x7f829c1034b0 2024-04-26T01:56:13.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.195+0000 7f829aa00700 1 -- 172.21.15.151:0/927353412 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bacd7eadd0 msgr2=0x55bacd7eb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:13.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.195+0000 7f829aa00700 1 --2- 172.21.15.151:0/927353412 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bacd7eadd0 0x55bacd7eb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:13.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.195+0000 7f829aa00700 1 -- 172.21.15.151:0/927353412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bacd7f6560 msgr2=0x55bacd7f8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:13.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.195+0000 7f829aa00700 1 --2- 172.21.15.151:0/927353412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bacd7f6560 0x55bacd7f8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:13.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.195+0000 7f829aa00700 1 -- 172.21.15.151:0/927353412 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bacd6da810 con 0x55bacd7eb6f0 2024-04-26T01:56:13.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.196+0000 7f829aa00700 1 --2- 172.21.15.151:0/927353412 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bacd7eb6f0 0x55bacd7f6020 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f828c009d80 tx=0x7f828c011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ec30dd4b4765fc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:13.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.196+0000 7f829b400700 1 --2- 172.21.15.151:0/2876510180 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f82800a3ed0 0x7f82800a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:13.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.196+0000 7f8287e00700 1 -- 172.21.15.151:0/927353412 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f828c004230 con 0x55bacd7eb6f0 2024-04-26T01:56:13.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.197+0000 7f8287e00700 1 -- 172.21.15.151:0/927353412 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f828c0043d0 con 0x55bacd7eb6f0 2024-04-26T01:56:13.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.197+0000 7f8287e00700 1 -- 172.21.15.151:0/927353412 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f828c0197f0 con 0x55bacd7eb6f0 2024-04-26T01:56:13.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.197+0000 7f82ac3c5b80 1 -- 172.21.15.151:0/927353412 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bacd7eb6f0 msgr2=0x55bacd7f6020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:13.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.197+0000 7f82ac3c5b80 1 --2- 172.21.15.151:0/927353412 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bacd7eb6f0 0x55bacd7f6020 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f828c009d80 tx=0x7f828c011720 comp rx=0 tx=0).stop 2024-04-26T01:56:13.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.197+0000 7f829b400700 1 --2- 172.21.15.151:0/2876510180 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f82800a3ed0 0x7f82800a6390 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f829400bae0 tx=0x7f8294009bd0 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:13.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.198+0000 7f82ac3c5b80 1 -- 172.21.15.151:0/927353412 shutdown_connections 2024-04-26T01:56:13.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.198+0000 7f82ac3c5b80 1 --2- 172.21.15.151:0/927353412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bacd7f6560 0x55bacd7f8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:13.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.198+0000 7f82ac3c5b80 1 --2- 172.21.15.151:0/927353412 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bacd7eadd0 0x55bacd7eb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:13.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.198+0000 7f82ac3c5b80 1 --2- 172.21.15.151:0/927353412 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bacd7eb6f0 0x55bacd7f6020 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:13.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.198+0000 7f82ac3c5b80 1 -- 172.21.15.151:0/927353412 >> 172.21.15.151:0/927353412 conn(0x55bacd71a0e0 msgr2=0x55bacd7e72e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:13.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.198+0000 7f82ac3c5b80 1 -- 172.21.15.151:0/927353412 shutdown_connections 2024-04-26T01:56:13.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.198+0000 7f82ac3c5b80 1 -- 172.21.15.151:0/927353412 wait complete. 2024-04-26T01:56:13.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.199+0000 7f82ac3c5b80 1 -- start start 2024-04-26T01:56:13.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.200+0000 7f82ac3c5b80 1 Processor -- start 2024-04-26T01:56:13.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.200+0000 7f82ac3c5b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bacd7eadd0 0x55bacd7f5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:13.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.200+0000 7f82ac3c5b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bacd7eb6f0 0x55bacd7f30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:13.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.200+0000 7f82ac3c5b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bacd7f6560 0x55bacd7f3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:13.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.201+0000 7f829aa00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bacd7eadd0 0x55bacd7f5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:13.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.201+0000 7f829aa00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bacd7eadd0 0x55bacd7f5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:43608/0 (socket says 172.21.15.151:43608) 2024-04-26T01:56:13.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.201+0000 7f829aa00700 1 -- 172.21.15.151:0/1852152032 learned_addr learned my addr 172.21.15.151:0/1852152032 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:56:13.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.201+0000 7f82ac3c5b80 1 -- 172.21.15.151:0/1852152032 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55bacd7f9050 con 0x55bacd7f6560 2024-04-26T01:56:13.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.201+0000 7f82ac3c5b80 1 -- 172.21.15.151:0/1852152032 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55bacd6f79f0 con 0x55bacd7eadd0 2024-04-26T01:56:13.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.201+0000 7f82ac3c5b80 1 -- 172.21.15.151:0/1852152032 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55bacd7f91d0 con 0x55bacd7eb6f0 2024-04-26T01:56:13.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.201+0000 7f82ac3c5b80 10 client.0 _ll_register_callbacks cb 0x55bacd6d68b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:56:13.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.201+0000 7f829a000700 1 --2- 172.21.15.151:0/1852152032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bacd7eb6f0 0x55bacd7f30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:13.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.201+0000 7f829b400700 1 --2- 172.21.15.151:0/1852152032 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bacd7f6560 0x55bacd7f3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:13.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.201+0000 7f829aa00700 1 -- 172.21.15.151:0/1852152032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bacd7eb6f0 msgr2=0x55bacd7f30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:13.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.201+0000 7f829aa00700 1 --2- 172.21.15.151:0/1852152032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bacd7eb6f0 0x55bacd7f30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:13.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.201+0000 7f829aa00700 1 -- 172.21.15.151:0/1852152032 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bacd7f6560 msgr2=0x55bacd7f3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:13.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.201+0000 7f829aa00700 1 --2- 172.21.15.151:0/1852152032 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bacd7f6560 0x55bacd7f3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:13.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.202+0000 7f829aa00700 1 -- 172.21.15.151:0/1852152032 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bacd68ba60 con 0x55bacd7eadd0 2024-04-26T01:56:13.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.202+0000 7f829aa00700 1 --2- 172.21.15.151:0/1852152032 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bacd7eadd0 0x55bacd7f5dc0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f828c000c00 tx=0x7f828c005320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cee1d3f75cc250be server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:13.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.202+0000 7f8284c00700 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T01:56:13.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.202+0000 7f8284c00700 1 -- 172.21.15.151:0/1852152032 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f828c004950 con 0x55bacd7eadd0 2024-04-26T01:56:13.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.202+0000 7f8284c00700 1 -- 172.21.15.151:0/1852152032 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f828c004af0 con 0x55bacd7eadd0 2024-04-26T01:56:13.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.203+0000 7f82ac3c5b80 1 -- 172.21.15.151:0/1852152032 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f828c0043d0 con 0x55bacd7eadd0 2024-04-26T01:56:13.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.203+0000 7f8284c00700 1 -- 172.21.15.151:0/1852152032 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f828c005ce0 con 0x55bacd7eadd0 2024-04-26T01:56:13.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.203+0000 7f82ac3c5b80 10 client.25469 fetch_fsmap learned FSMap version 28 2024-04-26T01:56:13.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.203+0000 7f82ac3c5b80 1 -- 172.21.15.151:0/1852152032 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bacd6d4c80 con 0x55bacd7eadd0 2024-04-26T01:56:13.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.203+0000 7f8284c00700 1 -- 172.21.15.151:0/1852152032 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f828c0043d0 con 0x55bacd7eadd0 2024-04-26T01:56:13.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.204+0000 7f8284c00700 1 -- 172.21.15.151:0/1852152032 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f828c032020 con 0x55bacd7eadd0 2024-04-26T01:56:13.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.204+0000 7f82ac3c5b80 10 client.25469 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T01:56:13.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.204+0000 7f82ac3c5b80 10 client.25469 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-26T01:56:13.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.204+0000 7f82ac3c5b80 20 client.25469 populate_metadata read hostname 'smithi151' 2024-04-26T01:56:13.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.204+0000 7f82ac3c5b80 1 --2- 172.21.15.151:0/1852152032 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55bacd805d80 0x55bacd808170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:13.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.204+0000 7f82ac3c5b80 4 client.25469 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:56:13.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.204+0000 7f82ac3c5b80 1 -- 172.21.15.151:0/1852152032 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bacd74b810 con 0x55bacd805d80 2024-04-26T01:56:13.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.205+0000 7f829a000700 1 --2- 172.21.15.151:0/1852152032 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55bacd805d80 0x55bacd808170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:13.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.206+0000 7f829a000700 1 --2- 172.21.15.151:0/1852152032 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55bacd805d80 0x55bacd808170 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=383f39979cbfe949 server_cookie=5a7e54274d95ed57 in_seq=0 out_seq=0 2024-04-26T01:56:13.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.206+0000 7f8284c00700 10 client.25469 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:56:13.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.206+0000 7f8284c00700 1 -- 172.21.15.151:0/1852152032 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bacd74b810 con 0x55bacd805d80 2024-04-26T01:56:13.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.206+0000 7f8284c00700 10 client.25469 handle_command_reply: tid=0 2024-04-26T01:56:13.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.207+0000 7f82ac3c5b80 1 client.25469 shutdown 2024-04-26T01:56:13.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.208+0000 7f82ac3c5b80 1 -- 172.21.15.151:0/1852152032 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bacd7eadd0 msgr2=0x55bacd7f5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:13.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.208+0000 7f82ac3c5b80 1 --2- 172.21.15.151:0/1852152032 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bacd7eadd0 0x55bacd7f5dc0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f828c000c00 tx=0x7f828c005320 comp rx=0 tx=0).stop 2024-04-26T01:56:13.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.209+0000 7f82ac3c5b80 1 -- 172.21.15.151:0/1852152032 shutdown_connections 2024-04-26T01:56:13.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.209+0000 7f82ac3c5b80 1 --2- 172.21.15.151:0/1852152032 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bacd7f6560 0x55bacd7f3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:13.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.209+0000 7f82ac3c5b80 1 --2- 172.21.15.151:0/1852152032 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55bacd805d80 0x55bacd808170 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:13.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.209+0000 7f82ac3c5b80 1 --2- 172.21.15.151:0/1852152032 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bacd7eadd0 0x55bacd7f5dc0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:13.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.209+0000 7f82ac3c5b80 1 --2- 172.21.15.151:0/1852152032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bacd7eb6f0 0x55bacd7f30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:13.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.209+0000 7f8284c00700 0 client.25469 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:56:13.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.209+0000 7f82ac3c5b80 1 -- 172.21.15.151:0/1852152032 >> 172.21.15.151:0/1852152032 conn(0x55bacd71a0e0 msgr2=0x55bacd7e6ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:13.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.209+0000 7f82ac3c5b80 1 -- 172.21.15.151:0/1852152032 shutdown_connections 2024-04-26T01:56:13.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.209+0000 7f82ac3c5b80 1 -- 172.21.15.151:0/1852152032 wait complete. 2024-04-26T01:56:13.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.210+0000 7f82ac3c5b80 20 client.25469 trim_cache size 0 max 16384 2024-04-26T01:56:13.238 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] listxattr 2024-04-26T01:56:13.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.251+0000 7f82ac3c5b80 1 Processor -- start 2024-04-26T01:56:13.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.251+0000 7f82ac3c5b80 1 -- start start 2024-04-26T01:56:13.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.251+0000 7f82ac3c5b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bacd7eadd0 0x55bacd801f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:13.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.251+0000 7f82ac3c5b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bacd7eb6f0 0x55bacd7fe480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:13.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.251+0000 7f82ac3c5b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bacd7f6560 0x55bacd7fe9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:13.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.251+0000 7f82ac3c5b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f829c1b5f30 con 0x55bacd7eb6f0 2024-04-26T01:56:13.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.251+0000 7f82ac3c5b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f829c1b37c0 con 0x55bacd7f6560 2024-04-26T01:56:13.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.251+0000 7f82ac3c5b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f829c1b1650 con 0x55bacd7eadd0 2024-04-26T01:56:13.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.252+0000 7f829a000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bacd7eb6f0 0x55bacd7fe480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:13.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.252+0000 7f829a000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bacd7eb6f0 0x55bacd7fe480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33000/0 (socket says 172.21.15.151:33000) 2024-04-26T01:56:13.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.252+0000 7f829a000700 1 -- 172.21.15.151:0/3348927015 learned_addr learned my addr 172.21.15.151:0/3348927015 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:56:13.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.252+0000 7f829aa00700 1 --2- 172.21.15.151:0/3348927015 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bacd7eadd0 0x55bacd801f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:13.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.252+0000 7f829b400700 1 --2- 172.21.15.151:0/3348927015 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bacd7f6560 0x55bacd7fe9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:13.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.252+0000 7f829a000700 1 -- 172.21.15.151:0/3348927015 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bacd7eadd0 msgr2=0x55bacd801f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:13.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.252+0000 7f829a000700 1 --2- 172.21.15.151:0/3348927015 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bacd7eadd0 0x55bacd801f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:13.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.253+0000 7f829a000700 1 -- 172.21.15.151:0/3348927015 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bacd7f6560 msgr2=0x55bacd7fe9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:13.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.253+0000 7f829a000700 1 --2- 172.21.15.151:0/3348927015 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bacd7f6560 0x55bacd7fe9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:13.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.253+0000 7f829a000700 1 -- 172.21.15.151:0/3348927015 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bacd7ecc40 con 0x55bacd7eb6f0 2024-04-26T01:56:13.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.253+0000 7f829b400700 1 --2- 172.21.15.151:0/3348927015 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bacd7f6560 0x55bacd7fe9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T01:56:13.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.253+0000 7f829a000700 1 --2- 172.21.15.151:0/3348927015 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bacd7eb6f0 0x55bacd7fe480 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f828809d3b0 tx=0x7f82880a4040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5af8f4b3b1583d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:13.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.254+0000 7f8287400700 1 -- 172.21.15.151:0/3348927015 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82880880b0 con 0x55bacd7eb6f0 2024-04-26T01:56:13.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.254+0000 7f8287400700 1 -- 172.21.15.151:0/3348927015 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f828809ddb0 con 0x55bacd7eb6f0 2024-04-26T01:56:13.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.254+0000 7f8287400700 1 -- 172.21.15.151:0/3348927015 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82880cf050 con 0x55bacd7eb6f0 2024-04-26T01:56:13.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.255+0000 7f82ac3c5b80 1 -- 172.21.15.151:0/3348927015 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bacd7eb6f0 msgr2=0x55bacd7fe480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:13.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.255+0000 7f82ac3c5b80 1 --2- 172.21.15.151:0/3348927015 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bacd7eb6f0 0x55bacd7fe480 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f828809d3b0 tx=0x7f82880a4040 comp rx=0 tx=0).stop 2024-04-26T01:56:13.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.256+0000 7f82ac3c5b80 1 -- 172.21.15.151:0/3348927015 shutdown_connections 2024-04-26T01:56:13.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.256+0000 7f82ac3c5b80 1 --2- 172.21.15.151:0/3348927015 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bacd7f6560 0x55bacd7fe9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:13.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.256+0000 7f82ac3c5b80 1 --2- 172.21.15.151:0/3348927015 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bacd7eadd0 0x55bacd801f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:13.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.256+0000 7f82ac3c5b80 1 --2- 172.21.15.151:0/3348927015 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bacd7eb6f0 0x55bacd7fe480 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:13.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.256+0000 7f82ac3c5b80 1 -- 172.21.15.151:0/3348927015 >> 172.21.15.151:0/3348927015 conn(0x55bacd71a0e0 msgr2=0x55bacd7f5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:13.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.256+0000 7f82ac3c5b80 1 -- 172.21.15.151:0/3348927015 shutdown_connections 2024-04-26T01:56:13.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.256+0000 7f82ac3c5b80 1 -- 172.21.15.151:0/3348927015 wait complete. 2024-04-26T01:56:13.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.257+0000 7f82ac3c5b80 1 -- start start 2024-04-26T01:56:13.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.257+0000 7f82ac3c5b80 1 Processor -- start 2024-04-26T01:56:13.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.259+0000 7f82ac3c5b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bacd7eadd0 0x55bacd7fd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:13.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.259+0000 7f82ac3c5b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bacd7eb6f0 0x55bacd7fda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:13.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.259+0000 7f82ac3c5b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bacd7f6560 0x55bacd86c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:13.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.259+0000 7f82ac3c5b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f829c1b1ca0 con 0x55bacd7eadd0 2024-04-26T01:56:13.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.259+0000 7f82ac3c5b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f829c1b30f0 con 0x55bacd7f6560 2024-04-26T01:56:13.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.259+0000 7f82ac3c5b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f829c1b2350 con 0x55bacd7eb6f0 2024-04-26T01:56:13.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.259+0000 7f82ac3c5b80 10 client.0 _ll_register_callbacks cb 0x55bacd6d68b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:56:13.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.259+0000 7f829a000700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bacd7eb6f0 0x55bacd7fda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:13.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.259+0000 7f829a000700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bacd7eb6f0 0x55bacd7fda20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:37026/0 (socket says 172.21.15.151:37026) 2024-04-26T01:56:13.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.259+0000 7f829a000700 1 -- 172.21.15.151:0/3204850906 learned_addr learned my addr 172.21.15.151:0/3204850906 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:56:13.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.260+0000 7f829a000700 1 -- 172.21.15.151:0/3204850906 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bacd7f6560 msgr2=0x55bacd86c170 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T01:56:13.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.260+0000 7f829a000700 1 --2- 172.21.15.151:0/3204850906 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bacd7f6560 0x55bacd86c170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:13.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.260+0000 7f829a000700 1 -- 172.21.15.151:0/3204850906 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bacd7eadd0 msgr2=0x55bacd7fd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:13.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.260+0000 7f829a000700 1 --2- 172.21.15.151:0/3204850906 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bacd7eadd0 0x55bacd7fd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:13.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.260+0000 7f829a000700 1 -- 172.21.15.151:0/3204850906 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bacd7ebd70 con 0x55bacd7eb6f0 2024-04-26T01:56:13.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.262+0000 7f829a000700 1 --2- 172.21.15.151:0/3204850906 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bacd7eb6f0 0x55bacd7fda20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f828809d3b0 tx=0x7f8288009520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9552827962a04855 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:13.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.262+0000 7f8284c00700 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-26T01:56:13.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.262+0000 7f8284c00700 1 -- 172.21.15.151:0/3204850906 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82880b5070 con 0x55bacd7eb6f0 2024-04-26T01:56:13.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.262+0000 7f8284c00700 1 -- 172.21.15.151:0/3204850906 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f82880cf050 con 0x55bacd7eb6f0 2024-04-26T01:56:13.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.262+0000 7f82ac3c5b80 1 -- 172.21.15.151:0/3204850906 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f828809ddb0 con 0x55bacd7eb6f0 2024-04-26T01:56:13.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.264+0000 7f8284c00700 1 -- 172.21.15.151:0/3204850906 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f828809d6b0 con 0x55bacd7eb6f0 2024-04-26T01:56:13.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.264+0000 7f82ac3c5b80 10 client.25504 fetch_fsmap learned FSMap version 28 2024-04-26T01:56:13.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.264+0000 7f82ac3c5b80 1 -- 172.21.15.151:0/3204850906 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bacd86d700 con 0x55bacd7eb6f0 2024-04-26T01:56:13.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.264+0000 7f8284c00700 1 -- 172.21.15.151:0/3204850906 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f828809ddb0 con 0x55bacd7eb6f0 2024-04-26T01:56:13.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.264+0000 7f8284c00700 1 -- 172.21.15.151:0/3204850906 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f828809d910 con 0x55bacd7eb6f0 2024-04-26T01:56:13.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.264+0000 7f82ac3c5b80 10 client.25504 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T01:56:13.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.265+0000 7f82ac3c5b80 10 client.25504 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-26T01:56:13.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.265+0000 7f82ac3c5b80 20 client.25504 populate_metadata read hostname 'smithi151' 2024-04-26T01:56:13.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.265+0000 7f82ac3c5b80 1 --2- 172.21.15.151:0/3204850906 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55bacd86de60 0x55bacd870280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:13.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.265+0000 7f82ac3c5b80 4 client.25504 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 90.0, "await_for": 42.0}] 2024-04-26T01:56:13.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.265+0000 7f82ac3c5b80 1 -- 172.21.15.151:0/3204850906 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 90.0, "await_for": 42.0}) -- 0x55bacd7e6030 con 0x55bacd86de60 2024-04-26T01:56:13.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.265+0000 7f829aa00700 1 --2- 172.21.15.151:0/3204850906 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55bacd86de60 0x55bacd870280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:13.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.266+0000 7f829aa00700 1 --2- 172.21.15.151:0/3204850906 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55bacd86de60 0x55bacd870280 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2dac1441421bcfef server_cookie=53ac5319de57aff1 in_seq=0 out_seq=0 2024-04-26T01:56:13.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:13.266+0000 7f8284c00700 10 client.25504 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:56:14.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:56:14 smithi165 ceph-mon[32648]: pgmap v81: 97 pgs: 97 active+clean; 30 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 960 KiB/s wr, 198 op/s 2024-04-26T01:56:14.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:56:14 smithi151 ceph-mon[28057]: pgmap v81: 97 pgs: 97 active+clean; 30 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 960 KiB/s wr, 198 op/s 2024-04-26T01:56:14.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:56:14 smithi172 ceph-mon[32785]: pgmap v81: 97 pgs: 97 active+clean; 30 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 960 KiB/s wr, 198 op/s 2024-04-26T01:56:16.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:56:16 smithi165 ceph-mon[32648]: pgmap v82: 97 pgs: 97 active+clean; 30 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 945 KiB/s wr, 183 op/s 2024-04-26T01:56:16.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:56:16 smithi151 ceph-mon[28057]: pgmap v82: 97 pgs: 97 active+clean; 30 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 945 KiB/s wr, 183 op/s 2024-04-26T01:56:16.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:56:16 smithi172 ceph-mon[32785]: pgmap v82: 97 pgs: 97 active+clean; 30 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 945 KiB/s wr, 183 op/s 2024-04-26T01:56:16.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:16.640+0000 7f8284c00700 1 -- 172.21.15.151:0/3204850906 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x55bacd7e6030 con 0x55bacd86de60 2024-04-26T01:56:16.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:16.640+0000 7f8284c00700 10 client.25504 handle_command_reply: tid=0 2024-04-26T01:56:16.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:16.640+0000 7f82ac3c5b80 1 client.25504 shutdown 2024-04-26T01:56:16.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:16.641+0000 7f82ac3c5b80 1 -- 172.21.15.151:0/3204850906 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bacd7eb6f0 msgr2=0x55bacd7fda20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:16.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:16.641+0000 7f82ac3c5b80 1 --2- 172.21.15.151:0/3204850906 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bacd7eb6f0 0x55bacd7fda20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f828809d3b0 tx=0x7f8288009520 comp rx=0 tx=0).stop 2024-04-26T01:56:16.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:16.641+0000 7f82ac3c5b80 1 -- 172.21.15.151:0/3204850906 shutdown_connections 2024-04-26T01:56:16.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:16.641+0000 7f82ac3c5b80 1 --2- 172.21.15.151:0/3204850906 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55bacd86de60 0x55bacd870280 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:16.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:16.641+0000 7f82ac3c5b80 1 --2- 172.21.15.151:0/3204850906 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bacd7f6560 0x55bacd86c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:16.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:16.641+0000 7f82ac3c5b80 1 --2- 172.21.15.151:0/3204850906 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bacd7eadd0 0x55bacd7fd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:16.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:16.641+0000 7f82ac3c5b80 1 --2- 172.21.15.151:0/3204850906 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bacd7eb6f0 0x55bacd7fda20 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:16.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:16.641+0000 7f8284c00700 0 client.25504 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:56:16.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:16.641+0000 7f82ac3c5b80 1 -- 172.21.15.151:0/3204850906 >> 172.21.15.151:0/3204850906 conn(0x55bacd71a0e0 msgr2=0x55bacd800f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:16.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:16.642+0000 7f82ac3c5b80 1 -- 172.21.15.151:0/3204850906 shutdown_connections 2024-04-26T01:56:16.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:16.642+0000 7f82ac3c5b80 1 -- 172.21.15.151:0/3204850906 wait complete. 2024-04-26T01:56:16.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:16.642+0000 7f82ac3c5b80 20 client.25504 trim_cache size 0 max 16384 2024-04-26T01:56:16.642 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-26T01:56:16.642 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 28, 2024-04-26T01:56:16.642 INFO:teuthology.orchestra.run.smithi151.stdout: "leader": 24509, 2024-04-26T01:56:16.642 INFO:teuthology.orchestra.run.smithi151.stdout: "set_version": 6, 2024-04-26T01:56:16.642 INFO:teuthology.orchestra.run.smithi151.stdout: "sets": { 2024-04-26T01:56:16.642 INFO:teuthology.orchestra.run.smithi151.stdout: "bc374c20": { 2024-04-26T01:56:16.643 INFO:teuthology.orchestra.run.smithi151.stdout: "version": 6, 2024-04-26T01:56:16.643 INFO:teuthology.orchestra.run.smithi151.stdout: "age_ref": 0.0, 2024-04-26T01:56:16.643 INFO:teuthology.orchestra.run.smithi151.stdout: "state": { 2024-04-26T01:56:16.643 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "QUIESCED", 2024-04-26T01:56:16.643 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 0.0 2024-04-26T01:56:16.643 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T01:56:16.643 INFO:teuthology.orchestra.run.smithi151.stdout: "timeout": 60.0, 2024-04-26T01:56:16.643 INFO:teuthology.orchestra.run.smithi151.stdout: "expiration": 90.0, 2024-04-26T01:56:16.643 INFO:teuthology.orchestra.run.smithi151.stdout: "members": { 2024-04-26T01:56:16.643 INFO:teuthology.orchestra.run.smithi151.stdout: "file:/": { 2024-04-26T01:56:16.643 INFO:teuthology.orchestra.run.smithi151.stdout: "excluded": false, 2024-04-26T01:56:16.643 INFO:teuthology.orchestra.run.smithi151.stdout: "state": { 2024-04-26T01:56:16.643 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "QUIESCED", 2024-04-26T01:56:16.643 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 0.0 2024-04-26T01:56:16.643 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T01:56:16.644 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T01:56:16.644 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T01:56:16.644 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T01:56:16.644 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T01:56:16.644 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-26T01:56:16.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:16.643+0000 7f82a1a00700 1 -- 172.21.15.151:0/2876510180 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f82800a3ed0 msgr2=0x7f82800a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:16.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:16.643+0000 7f82a1a00700 1 --2- 172.21.15.151:0/2876510180 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f82800a3ed0 0x7f82800a6390 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f829400bae0 tx=0x7f8294009bd0 comp rx=0 tx=0).stop 2024-04-26T01:56:16.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:16.643+0000 7f82a1a00700 1 -- 172.21.15.151:0/2876510180 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f829c1034b0 msgr2=0x7f829c1a4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:16.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:16.643+0000 7f82a1a00700 1 --2- 172.21.15.151:0/2876510180 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f829c1034b0 0x7f829c1a4890 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8288000c00 tx=0x7f82880057c0 comp rx=0 tx=0).stop 2024-04-26T01:56:16.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:16.643+0000 7f82a1a00700 1 -- 172.21.15.151:0/2876510180 shutdown_connections 2024-04-26T01:56:16.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:16.643+0000 7f82a1a00700 1 --2- 172.21.15.151:0/2876510180 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f82800a3ed0 0x7f82800a6390 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:16.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:16.643+0000 7f82a1a00700 1 --2- 172.21.15.151:0/2876510180 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f829c107940 0x7f829c10e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:16.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:16.643+0000 7f82a1a00700 1 --2- 172.21.15.151:0/2876510180 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f829c102af0 0x7f829c1a4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:16.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:16.643+0000 7f82a1a00700 1 --2- 172.21.15.151:0/2876510180 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f829c1034b0 0x7f829c1a4890 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:16.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:16.643+0000 7f82a1a00700 1 -- 172.21.15.151:0/2876510180 >> 172.21.15.151:0/2876510180 conn(0x7f829c079ba0 msgr2=0x7f829c100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:16.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:16.644+0000 7f82a1a00700 1 -- 172.21.15.151:0/2876510180 shutdown_connections 2024-04-26T01:56:16.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:16.644+0000 7f82a1a00700 1 -- 172.21.15.151:0/2876510180 wait complete. 2024-04-26T01:56:17.530 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'bc374c20', quiesce took 12.4 seconds. Will release after: 30.0 2024-04-26T01:56:18.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:56:18 smithi165 ceph-mon[32648]: pgmap v83: 97 pgs: 97 active+clean; 30 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 138 KiB/s wr, 58 op/s 2024-04-26T01:56:18.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:56:18 smithi151 ceph-mon[28057]: pgmap v83: 97 pgs: 97 active+clean; 30 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 138 KiB/s wr, 58 op/s 2024-04-26T01:56:18.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:56:18 smithi172 ceph-mon[32785]: pgmap v83: 97 pgs: 97 active+clean; 30 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 138 KiB/s wr, 58 op/s 2024-04-26T01:56:20.112 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.113+0000 7fc6c3200640 1 -- 172.21.15.151:0/1070962827 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6c410e5a0 msgr2=0x7fc6c41109f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:20.112 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.113+0000 7fc6c3200640 1 --2- 172.21.15.151:0/1070962827 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6c410e5a0 0x7fc6c41109f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc009ca0 tx=0x7fc6bc02a950 comp rx=0 tx=0).stop 2024-04-26T01:56:20.112 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.113+0000 7fc6c3200640 1 -- 172.21.15.151:0/1070962827 shutdown_connections 2024-04-26T01:56:20.112 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.113+0000 7fc6c3200640 1 --2- 172.21.15.151:0/1070962827 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6c410e5a0 0x7fc6c41109f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.113+0000 7fc6c3200640 1 --2- 172.21.15.151:0/1070962827 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6c4074ca0 0x7fc6c410e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.113+0000 7fc6c3200640 1 --2- 172.21.15.151:0/1070962827 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc6c4074380 0x7fc6c4074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.113+0000 7fc6c3200640 1 -- 172.21.15.151:0/1070962827 >> 172.21.15.151:0/1070962827 conn(0x7fc6c40fd890 msgr2=0x7fc6c40ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.113+0000 7fc6c3200640 1 -- 172.21.15.151:0/1070962827 shutdown_connections 2024-04-26T01:56:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.113+0000 7fc6c3200640 1 -- 172.21.15.151:0/1070962827 wait complete. 2024-04-26T01:56:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.114+0000 7fc6c3200640 1 Processor -- start 2024-04-26T01:56:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.114+0000 7fc6c3200640 1 -- start start 2024-04-26T01:56:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.115+0000 7fc6c3200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6c4074380 0x7fc6c419a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.115+0000 7fc6c3200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc6c4074ca0 0x7fc6c419acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.115+0000 7fc6c3200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6c410e5a0 0x7fc6c419eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.115+0000 7fc6c3200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c4114200 con 0x7fc6c410e5a0 2024-04-26T01:56:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.115+0000 7fc6c3200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c4114080 con 0x7fc6c4074ca0 2024-04-26T01:56:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.115+0000 7fc6c3200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c4114380 con 0x7fc6c4074380 2024-04-26T01:56:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.115+0000 7fc6c2800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6c410e5a0 0x7fc6c419eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.115+0000 7fc6c1e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6c4074380 0x7fc6c419a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.115+0000 7fc6c1400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc6c4074ca0 0x7fc6c419acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.115+0000 7fc6c2800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6c410e5a0 0x7fc6c419eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:35676/0 (socket says 172.21.15.151:35676) 2024-04-26T01:56:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.115+0000 7fc6c2800640 1 -- 172.21.15.151:0/855096660 learned_addr learned my addr 172.21.15.151:0/855096660 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:56:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.115+0000 7fc6c1e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6c4074380 0x7fc6c419a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:38938/0 (socket says 172.21.15.151:38938) 2024-04-26T01:56:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.115+0000 7fc6c1400640 1 -- 172.21.15.151:0/855096660 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6c4074380 msgr2=0x7fc6c419a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.115+0000 7fc6c1400640 1 --2- 172.21.15.151:0/855096660 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6c4074380 0x7fc6c419a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.115+0000 7fc6c1400640 1 -- 172.21.15.151:0/855096660 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6c410e5a0 msgr2=0x7fc6c419eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.115+0000 7fc6c1400640 1 --2- 172.21.15.151:0/855096660 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6c410e5a0 0x7fc6c419eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.115+0000 7fc6c1400640 1 -- 172.21.15.151:0/855096660 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6c419f720 con 0x7fc6c4074ca0 2024-04-26T01:56:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.116+0000 7fc6c1400640 1 --2- 172.21.15.151:0/855096660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc6c4074ca0 0x7fc6c419acc0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc6b000e9f0 tx=0x7fc6b000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.116+0000 7fc6baa00640 1 -- 172.21.15.151:0/855096660 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b000ccc0 con 0x7fc6c4074ca0 2024-04-26T01:56:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.116+0000 7fc6baa00640 1 -- 172.21.15.151:0/855096660 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6b00108e0 con 0x7fc6c4074ca0 2024-04-26T01:56:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.116+0000 7fc6baa00640 1 -- 172.21.15.151:0/855096660 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b0018880 con 0x7fc6c4074ca0 2024-04-26T01:56:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.116+0000 7fc6c3200640 1 -- 172.21.15.151:0/855096660 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6c419fa10 con 0x7fc6c4074ca0 2024-04-26T01:56:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.116+0000 7fc6c3200640 1 -- 172.21.15.151:0/855096660 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6c41a72a0 con 0x7fc6c4074ca0 2024-04-26T01:56:20.119 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.120+0000 7fc6cbc64740 1 Processor -- start 2024-04-26T01:56:20.119 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.120+0000 7fc6baa00640 1 -- 172.21.15.151:0/855096660 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7fc6b0010430 con 0x7fc6c4074ca0 2024-04-26T01:56:20.119 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.121+0000 7fc6cbc64740 1 -- start start 2024-04-26T01:56:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.121+0000 7fc6cbc64740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5638a2e8f9b0 0x5638a2e8fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.121+0000 7fc6cbc64740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638a2e902d0 0x5638a2e9a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.121+0000 7fc6cbc64740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5638a2e9ab90 0x5638a2e9cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.121+0000 7fc6cbc64740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5638a2df5900 con 0x5638a2e9ab90 2024-04-26T01:56:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.121+0000 7fc6cbc64740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5638a2d9bef0 con 0x5638a2e902d0 2024-04-26T01:56:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.121+0000 7fc6cbc64740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5638a2df18c0 con 0x5638a2e8f9b0 2024-04-26T01:56:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.121+0000 7fc6c2800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5638a2e8f9b0 0x5638a2e8fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.121+0000 7fc6c1400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5638a2e9ab90 0x5638a2e9cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.121+0000 7fc6c2800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5638a2e8f9b0 0x5638a2e8fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:38948/0 (socket says 172.21.15.151:38948) 2024-04-26T01:56:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.121+0000 7fc6c2800640 1 -- 172.21.15.151:0/1393820879 learned_addr learned my addr 172.21.15.151:0/1393820879 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:56:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.121+0000 7fc6c1e00640 1 --2- 172.21.15.151:0/1393820879 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638a2e902d0 0x5638a2e9a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.122+0000 7fc6c2800640 1 -- 172.21.15.151:0/1393820879 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638a2e902d0 msgr2=0x5638a2e9a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.122+0000 7fc6c2800640 1 --2- 172.21.15.151:0/1393820879 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638a2e902d0 0x5638a2e9a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.122+0000 7fc6c2800640 1 -- 172.21.15.151:0/1393820879 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5638a2e9ab90 msgr2=0x5638a2e9cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.122+0000 7fc6c2800640 1 --2- 172.21.15.151:0/1393820879 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5638a2e9ab90 0x5638a2e9cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.122+0000 7fc6c2800640 1 -- 172.21.15.151:0/1393820879 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638a2b08210 con 0x5638a2e8f9b0 2024-04-26T01:56:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.122+0000 7fc6c1e00640 1 --2- 172.21.15.151:0/1393820879 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638a2e902d0 0x5638a2e9a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:56:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.122+0000 7fc6c2800640 1 --2- 172.21.15.151:0/1393820879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5638a2e8f9b0 0x5638a2e8fd90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc02f410 tx=0x7fc6bc02f440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4523c8854c2712d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.122+0000 7fc6baa00640 1 --2- 172.21.15.151:0/855096660 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fc6a80a3d10 0x7fc6a80a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.122+0000 7fc6af400640 1 -- 172.21.15.151:0/1393820879 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6bc048070 con 0x5638a2e8f9b0 2024-04-26T01:56:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.122+0000 7fc6baa00640 1 -- 172.21.15.151:0/855096660 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fc6b00d0a70 con 0x7fc6c4074ca0 2024-04-26T01:56:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.122+0000 7fc6af400640 1 -- 172.21.15.151:0/1393820879 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6bc003e20 con 0x5638a2e8f9b0 2024-04-26T01:56:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.122+0000 7fc6af400640 1 -- 172.21.15.151:0/1393820879 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6bc034040 con 0x5638a2e8f9b0 2024-04-26T01:56:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.123+0000 7fc6c1e00640 1 --2- 172.21.15.151:0/855096660 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fc6a80a3d10 0x7fc6a80a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:20.124 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.123+0000 7fc6cbc64740 1 -- 172.21.15.151:0/1393820879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5638a2e8f9b0 msgr2=0x5638a2e8fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:20.124 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.123+0000 7fc6cbc64740 1 --2- 172.21.15.151:0/1393820879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5638a2e8f9b0 0x5638a2e8fd90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc02f410 tx=0x7fc6bc02f440 comp rx=0 tx=0).stop 2024-04-26T01:56:20.124 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.123+0000 7fc6cbc64740 1 -- 172.21.15.151:0/1393820879 shutdown_connections 2024-04-26T01:56:20.124 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.123+0000 7fc6cbc64740 1 --2- 172.21.15.151:0/1393820879 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5638a2e9ab90 0x5638a2e9cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:20.124 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.123+0000 7fc6c1e00640 1 --2- 172.21.15.151:0/855096660 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fc6a80a3d10 0x7fc6a80a61d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x5638a2e90f50 tx=0x7fc6b4005f90 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:20.124 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.123+0000 7fc6cbc64740 1 --2- 172.21.15.151:0/1393820879 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638a2e902d0 0x5638a2e9a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:20.124 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.123+0000 7fc6cbc64740 1 --2- 172.21.15.151:0/1393820879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5638a2e8f9b0 0x5638a2e8fd90 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:20.124 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.123+0000 7fc6cbc64740 1 -- 172.21.15.151:0/1393820879 >> 172.21.15.151:0/1393820879 conn(0x5638a2e8bec0 msgr2=0x5638a2e8c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:20.124 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.123+0000 7fc6cbc64740 1 -- 172.21.15.151:0/1393820879 shutdown_connections 2024-04-26T01:56:20.124 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.123+0000 7fc6cbc64740 1 -- 172.21.15.151:0/1393820879 wait complete. 2024-04-26T01:56:20.124 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.124+0000 7fc6cbc64740 1 -- start start 2024-04-26T01:56:20.124 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.124+0000 7fc6cbc64740 1 Processor -- start 2024-04-26T01:56:20.128 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.124+0000 7fc6cbc64740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5638a2e8f9b0 0x5638a2e97bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:20.128 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.124+0000 7fc6cbc64740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5638a2e902d0 0x5638a2e98130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:20.129 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.124+0000 7fc6cbc64740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638a2e9ab90 0x5638a2e98670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:20.129 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.124+0000 7fc6cbc64740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5638a2e9d760 con 0x5638a2e902d0 2024-04-26T01:56:20.129 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.124+0000 7fc6cbc64740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5638a2daf310 con 0x5638a2e9ab90 2024-04-26T01:56:20.129 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.125+0000 7fc6cbc64740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5638a2e9d8e0 con 0x5638a2e8f9b0 2024-04-26T01:56:20.129 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.125+0000 7fc6cbc64740 10 client.0 _ll_register_callbacks cb 0x5638a2d7dcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:56:20.129 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.125+0000 7fc6c1400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5638a2e902d0 0x5638a2e98130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:20.129 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.125+0000 7fc6c1400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5638a2e902d0 0x5638a2e98130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:35682/0 (socket says 172.21.15.151:35682) 2024-04-26T01:56:20.129 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.125+0000 7fc6c1400640 1 -- 172.21.15.151:0/2297210874 learned_addr learned my addr 172.21.15.151:0/2297210874 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:56:20.129 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.125+0000 7fc6c1e00640 1 --2- 172.21.15.151:0/2297210874 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5638a2e8f9b0 0x5638a2e97bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:20.129 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.125+0000 7fc6c2800640 1 --2- 172.21.15.151:0/2297210874 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638a2e9ab90 0x5638a2e98670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:20.129 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.125+0000 7fc6c1400640 1 -- 172.21.15.151:0/2297210874 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5638a2e8f9b0 msgr2=0x5638a2e97bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:20.129 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.125+0000 7fc6c1400640 1 --2- 172.21.15.151:0/2297210874 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5638a2e8f9b0 0x5638a2e97bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:20.129 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.125+0000 7fc6c1400640 1 -- 172.21.15.151:0/2297210874 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638a2e9ab90 msgr2=0x5638a2e98670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:20.130 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.125+0000 7fc6c1400640 1 --2- 172.21.15.151:0/2297210874 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638a2e9ab90 0x5638a2e98670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:20.130 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.125+0000 7fc6c1400640 1 -- 172.21.15.151:0/2297210874 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638a2bd4620 con 0x5638a2e902d0 2024-04-26T01:56:20.130 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.125+0000 7fc6c1400640 1 --2- 172.21.15.151:0/2297210874 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5638a2e902d0 0x5638a2e98130 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc6b009c7d0 tx=0x7fc6b000cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d29435c6b682a94 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:20.130 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.125+0000 7fc6a3e00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-04-26T01:56:20.130 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.126+0000 7fc6a3e00640 1 -- 172.21.15.151:0/2297210874 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b00d0e60 con 0x5638a2e902d0 2024-04-26T01:56:20.130 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.126+0000 7fc6a3e00640 1 -- 172.21.15.151:0/2297210874 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fc6b00ce450 con 0x5638a2e902d0 2024-04-26T01:56:20.130 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.126+0000 7fc6cbc64740 1 -- 172.21.15.151:0/2297210874 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6bc003e20 con 0x5638a2e902d0 2024-04-26T01:56:20.130 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.126+0000 7fc6a3e00640 1 -- 172.21.15.151:0/2297210874 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc6b009c430 con 0x5638a2e902d0 2024-04-26T01:56:20.130 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.126+0000 7fc6cbc64740 10 client.15387 fetch_fsmap learned FSMap version 28 2024-04-26T01:56:20.130 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.126+0000 7fc6cbc64740 1 -- 172.21.15.151:0/2297210874 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5638a2ea9b50 con 0x5638a2e902d0 2024-04-26T01:56:20.130 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.126+0000 7fc6a3e00640 1 -- 172.21.15.151:0/2297210874 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6bc003e20 con 0x5638a2e902d0 2024-04-26T01:56:20.130 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.126+0000 7fc6a3e00640 1 -- 172.21.15.151:0/2297210874 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7fc6b00bbbe0 con 0x5638a2e902d0 2024-04-26T01:56:20.130 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.126+0000 7fc6cbc64740 10 client.15387 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T01:56:20.131 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.127+0000 7fc6cbc64740 10 client.15387 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:56:20.131 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.127+0000 7fc6cbc64740 20 client.15387 populate_metadata read hostname 'smithi151' 2024-04-26T01:56:20.131 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.127+0000 7fc6cbc64740 1 --2- 172.21.15.151:0/2297210874 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5638a2eab010 0x5638a2ead420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:20.131 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.127+0000 7fc6cbc64740 4 client.15387 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:56:20.131 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.127+0000 7fc6cbc64740 1 -- 172.21.15.151:0/2297210874 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638a2e098b0 con 0x5638a2eab010 2024-04-26T01:56:20.131 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.127+0000 7fc6c1e00640 1 --2- 172.21.15.151:0/2297210874 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5638a2eab010 0x5638a2ead420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:20.131 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.127+0000 7fc6c1e00640 1 --2- 172.21.15.151:0/2297210874 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5638a2eab010 0x5638a2ead420 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4ed35604ee4e83c server_cookie=5d26926aaa147a14 in_seq=0 out_seq=0 2024-04-26T01:56:20.131 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.127+0000 7fc6a3e00640 10 client.15387 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:56:20.131 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.128+0000 7fc6a3e00640 1 -- 172.21.15.151:0/2297210874 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5638a2e098b0 con 0x5638a2eab010 2024-04-26T01:56:20.131 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.128+0000 7fc6a3e00640 10 client.15387 handle_command_reply: tid=0 2024-04-26T01:56:20.131 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.128+0000 7fc6cbc64740 1 client.15387 shutdown 2024-04-26T01:56:20.131 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.129+0000 7fc6cbc64740 1 -- 172.21.15.151:0/2297210874 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5638a2e902d0 msgr2=0x5638a2e98130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:20.131 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.129+0000 7fc6cbc64740 1 --2- 172.21.15.151:0/2297210874 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5638a2e902d0 0x5638a2e98130 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc6b009c7d0 tx=0x7fc6b000cef0 comp rx=0 tx=0).stop 2024-04-26T01:56:20.132 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.129+0000 7fc6cbc64740 1 -- 172.21.15.151:0/2297210874 shutdown_connections 2024-04-26T01:56:20.132 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.129+0000 7fc6cbc64740 1 --2- 172.21.15.151:0/2297210874 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5638a2eab010 0x5638a2ead420 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:20.132 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.129+0000 7fc6cbc64740 1 --2- 172.21.15.151:0/2297210874 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638a2e9ab90 0x5638a2e98670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:20.132 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.129+0000 7fc6cbc64740 1 --2- 172.21.15.151:0/2297210874 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5638a2e902d0 0x5638a2e98130 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:20.132 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.129+0000 7fc6cbc64740 1 --2- 172.21.15.151:0/2297210874 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5638a2e8f9b0 0x5638a2e97bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:20.132 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.129+0000 7fc6a3e00640 0 client.15387 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:56:20.132 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.129+0000 7fc6cbc64740 1 -- 172.21.15.151:0/2297210874 >> 172.21.15.151:0/2297210874 conn(0x5638a2e8bec0 msgr2=0x5638a2e91bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:20.132 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.129+0000 7fc6cbc64740 1 -- 172.21.15.151:0/2297210874 shutdown_connections 2024-04-26T01:56:20.132 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.129+0000 7fc6cbc64740 1 -- 172.21.15.151:0/2297210874 wait complete. 2024-04-26T01:56:20.132 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.129+0000 7fc6cbc64740 20 client.15387 trim_cache size 0 max 16384 2024-04-26T01:56:20.145 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.145+0000 7fc6cbc64740 1 Processor -- start 2024-04-26T01:56:20.145 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.145+0000 7fc6cbc64740 1 -- start start 2024-04-26T01:56:20.146 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.146+0000 7fc6cbc64740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638a2e8f9b0 0x5638a2ea6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:20.146 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.146+0000 7fc6cbc64740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5638a2e902d0 0x5638a2e92370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:20.146 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.146+0000 7fc6cbc64740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5638a2e9ab90 0x5638a2e99b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:20.146 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.146+0000 7fc6cbc64740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c41b2270 con 0x5638a2e9ab90 2024-04-26T01:56:20.146 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.146+0000 7fc6cbc64740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c41afb40 con 0x5638a2e8f9b0 2024-04-26T01:56:20.146 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.146+0000 7fc6cbc64740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c41ad9b0 con 0x5638a2e902d0 2024-04-26T01:56:20.146 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.146+0000 7fc6c2800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5638a2e9ab90 0x5638a2e99b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:20.146 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.146+0000 7fc6c1e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638a2e8f9b0 0x5638a2ea6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:20.146 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.146+0000 7fc6c1e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638a2e8f9b0 0x5638a2ea6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:49624/0 (socket says 172.21.15.151:49624) 2024-04-26T01:56:20.146 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.146+0000 7fc6c1e00640 1 -- 172.21.15.151:0/2190925893 learned_addr learned my addr 172.21.15.151:0/2190925893 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:56:20.146 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.146+0000 7fc6c2800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5638a2e9ab90 0x5638a2e99b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:35694/0 (socket says 172.21.15.151:35694) 2024-04-26T01:56:20.146 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.146+0000 7fc6c1400640 1 --2- 172.21.15.151:0/2190925893 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5638a2e902d0 0x5638a2e92370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.146+0000 7fc6c1e00640 1 -- 172.21.15.151:0/2190925893 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5638a2e902d0 msgr2=0x5638a2e92370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.146+0000 7fc6c1e00640 1 --2- 172.21.15.151:0/2190925893 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5638a2e902d0 0x5638a2e92370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.146+0000 7fc6c1e00640 1 -- 172.21.15.151:0/2190925893 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5638a2e9ab90 msgr2=0x5638a2e99b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.146+0000 7fc6c1e00640 1 --2- 172.21.15.151:0/2190925893 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5638a2e9ab90 0x5638a2e99b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.147+0000 7fc6c1e00640 1 -- 172.21.15.151:0/2190925893 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638a2eadaa0 con 0x5638a2e8f9b0 2024-04-26T01:56:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.147+0000 7fc6c2800640 1 --2- 172.21.15.151:0/2190925893 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5638a2e9ab90 0x5638a2e99b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:56:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.147+0000 7fc6c1e00640 1 --2- 172.21.15.151:0/2190925893 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638a2e8f9b0 0x5638a2ea6620 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc6b4009440 tx=0x7fc6b4019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1046284c1ae6dc50 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.147+0000 7fc6aea00640 1 -- 172.21.15.151:0/2190925893 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b401ddc0 con 0x5638a2e8f9b0 2024-04-26T01:56:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.147+0000 7fc6aea00640 1 -- 172.21.15.151:0/2190925893 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6b40054d0 con 0x5638a2e8f9b0 2024-04-26T01:56:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.147+0000 7fc6aea00640 1 -- 172.21.15.151:0/2190925893 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b402b290 con 0x5638a2e8f9b0 2024-04-26T01:56:20.150 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.148+0000 7fc6cbc64740 1 -- 172.21.15.151:0/2190925893 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638a2e8f9b0 msgr2=0x5638a2ea6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:20.150 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.148+0000 7fc6cbc64740 1 --2- 172.21.15.151:0/2190925893 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638a2e8f9b0 0x5638a2ea6620 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc6b4009440 tx=0x7fc6b4019000 comp rx=0 tx=0).stop 2024-04-26T01:56:20.150 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.148+0000 7fc6cbc64740 1 -- 172.21.15.151:0/2190925893 shutdown_connections 2024-04-26T01:56:20.150 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.148+0000 7fc6cbc64740 1 --2- 172.21.15.151:0/2190925893 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5638a2e9ab90 0x5638a2e99b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:20.150 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.148+0000 7fc6cbc64740 1 --2- 172.21.15.151:0/2190925893 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5638a2e902d0 0x5638a2e92370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:20.151 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.148+0000 7fc6cbc64740 1 --2- 172.21.15.151:0/2190925893 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638a2e8f9b0 0x5638a2ea6620 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:20.151 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.148+0000 7fc6cbc64740 1 -- 172.21.15.151:0/2190925893 >> 172.21.15.151:0/2190925893 conn(0x5638a2e8bec0 msgr2=0x5638a2e99420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:20.151 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.148+0000 7fc6cbc64740 1 -- 172.21.15.151:0/2190925893 shutdown_connections 2024-04-26T01:56:20.151 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.148+0000 7fc6cbc64740 1 -- 172.21.15.151:0/2190925893 wait complete. 2024-04-26T01:56:20.151 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.148+0000 7fc6cbc64740 1 -- start start 2024-04-26T01:56:20.151 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.148+0000 7fc6cbc64740 1 Processor -- start 2024-04-26T01:56:20.151 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.149+0000 7fc6cbc64740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638a2e8f9b0 0x5638a2ea2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:20.151 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.149+0000 7fc6cbc64740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5638a2e902d0 0x5638a2ea0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:20.151 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.149+0000 7fc6cbc64740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5638a2e9ab90 0x5638a2ea0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:20.151 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.149+0000 7fc6cbc64740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c41ae000 con 0x5638a2e902d0 2024-04-26T01:56:20.151 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.149+0000 7fc6cbc64740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c41af470 con 0x5638a2e8f9b0 2024-04-26T01:56:20.151 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.149+0000 7fc6cbc64740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c41ae6d0 con 0x5638a2e9ab90 2024-04-26T01:56:20.152 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.149+0000 7fc6cbc64740 10 client.0 _ll_register_callbacks cb 0x7fc6b0098d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:56:20.152 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.149+0000 7fc6c1400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5638a2e902d0 0x5638a2ea0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:20.152 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.149+0000 7fc6c1400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5638a2e902d0 0x5638a2ea0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:35698/0 (socket says 172.21.15.151:35698) 2024-04-26T01:56:20.152 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.149+0000 7fc6c1400640 1 -- 172.21.15.151:0/1566917193 learned_addr learned my addr 172.21.15.151:0/1566917193 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:56:20.152 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.149+0000 7fc6c2800640 1 --2- 172.21.15.151:0/1566917193 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5638a2e9ab90 0x5638a2ea0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:20.152 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.149+0000 7fc6c1e00640 1 --2- 172.21.15.151:0/1566917193 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638a2e8f9b0 0x5638a2ea2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:20.152 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.150+0000 7fc6c1400640 1 -- 172.21.15.151:0/1566917193 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5638a2e9ab90 msgr2=0x5638a2ea0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:20.152 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.150+0000 7fc6c1400640 1 --2- 172.21.15.151:0/1566917193 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5638a2e9ab90 0x5638a2ea0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:20.152 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.150+0000 7fc6c1400640 1 -- 172.21.15.151:0/1566917193 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638a2e8f9b0 msgr2=0x5638a2ea2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:20.152 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.150+0000 7fc6c1400640 1 --2- 172.21.15.151:0/1566917193 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638a2e8f9b0 0x5638a2ea2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:20.152 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.150+0000 7fc6c1400640 1 -- 172.21.15.151:0/1566917193 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638a2ea0d70 con 0x5638a2e902d0 2024-04-26T01:56:20.152 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.150+0000 7fc6c1400640 1 --2- 172.21.15.151:0/1566917193 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5638a2e902d0 0x5638a2ea0140 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc6b009ca60 tx=0x7fc6b00ce2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d92f2bd36b1354e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:20.152 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.150+0000 7fc6a3e00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-04-26T01:56:20.153 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.150+0000 7fc6a3e00640 1 -- 172.21.15.151:0/1566917193 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b0010720 con 0x5638a2e902d0 2024-04-26T01:56:20.153 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.150+0000 7fc6a3e00640 1 -- 172.21.15.151:0/1566917193 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fc6b00ce450 con 0x5638a2e902d0 2024-04-26T01:56:20.153 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.150+0000 7fc6cbc64740 1 -- 172.21.15.151:0/1566917193 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6b40054d0 con 0x5638a2e902d0 2024-04-26T01:56:20.153 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.150+0000 7fc6a3e00640 1 -- 172.21.15.151:0/1566917193 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc6b00ce650 con 0x5638a2e902d0 2024-04-26T01:56:20.153 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.150+0000 7fc6cbc64740 10 client.15393 fetch_fsmap learned FSMap version 28 2024-04-26T01:56:20.153 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.150+0000 7fc6cbc64740 1 -- 172.21.15.151:0/1566917193 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5638a2f114d0 con 0x5638a2e902d0 2024-04-26T01:56:20.153 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.151+0000 7fc6a3e00640 1 -- 172.21.15.151:0/1566917193 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6b40054d0 con 0x5638a2e902d0 2024-04-26T01:56:20.153 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.151+0000 7fc6a3e00640 1 -- 172.21.15.151:0/1566917193 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7fc6b009ba20 con 0x5638a2e902d0 2024-04-26T01:56:20.153 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.151+0000 7fc6cbc64740 10 client.15393 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T01:56:20.153 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.151+0000 7fc6cbc64740 10 client.15393 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:56:20.153 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.151+0000 7fc6cbc64740 20 client.15393 populate_metadata read hostname 'smithi151' 2024-04-26T01:56:20.153 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.151+0000 7fc6cbc64740 1 --2- 172.21.15.151:0/1566917193 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5638a2f11ce0 0x5638a2f14100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:20.153 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.151+0000 7fc6cbc64740 4 client.15393 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:56:20.153 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.151+0000 7fc6cbc64740 1 -- 172.21.15.151:0/1566917193 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5638a2e03320 con 0x5638a2f11ce0 2024-04-26T01:56:20.154 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.151+0000 7fc6c1e00640 1 --2- 172.21.15.151:0/1566917193 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5638a2f11ce0 0x5638a2f14100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:20.154 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.152+0000 7fc6c1e00640 1 --2- 172.21.15.151:0/1566917193 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5638a2f11ce0 0x5638a2f14100 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad7364e42ffe9503 server_cookie=f260f31f2c72eeb7 in_seq=0 out_seq=0 2024-04-26T01:56:20.154 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.152+0000 7fc6a3e00640 10 client.15393 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:56:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.270+0000 7fc6a3e00640 1 -- 172.21.15.151:0/1566917193 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5638a2e03320 con 0x5638a2f11ce0 2024-04-26T01:56:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.270+0000 7fc6a3e00640 10 client.15393 handle_command_reply: tid=0 2024-04-26T01:56:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.270+0000 7fc6cbc64740 1 client.15393 shutdown 2024-04-26T01:56:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.270+0000 7fc6cbc64740 1 -- 172.21.15.151:0/1566917193 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5638a2e902d0 msgr2=0x5638a2ea0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.270+0000 7fc6cbc64740 1 --2- 172.21.15.151:0/1566917193 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5638a2e902d0 0x5638a2ea0140 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc6b009ca60 tx=0x7fc6b00ce2b0 comp rx=0 tx=0).stop 2024-04-26T01:56:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.270+0000 7fc6cbc64740 1 -- 172.21.15.151:0/1566917193 shutdown_connections 2024-04-26T01:56:20.270 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.270+0000 7fc6cbc64740 1 --2- 172.21.15.151:0/1566917193 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5638a2f11ce0 0x5638a2f14100 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:20.270 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.270+0000 7fc6cbc64740 1 --2- 172.21.15.151:0/1566917193 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5638a2e9ab90 0x5638a2ea0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:20.270 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.270+0000 7fc6cbc64740 1 --2- 172.21.15.151:0/1566917193 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5638a2e902d0 0x5638a2ea0140 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:20.270 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.270+0000 7fc6cbc64740 1 --2- 172.21.15.151:0/1566917193 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638a2e8f9b0 0x5638a2ea2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:20.270 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.270+0000 7fc6a3e00640 0 client.15393 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:56:20.270 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.270+0000 7fc6cbc64740 1 -- 172.21.15.151:0/1566917193 >> 172.21.15.151:0/1566917193 conn(0x5638a2e8bec0 msgr2=0x5638a2e99cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:20.270 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.270+0000 7fc6cbc64740 1 -- 172.21.15.151:0/1566917193 shutdown_connections 2024-04-26T01:56:20.270 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.270+0000 7fc6cbc64740 1 -- 172.21.15.151:0/1566917193 wait complete. 2024-04-26T01:56:20.270 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.271+0000 7fc6cbc64740 20 client.15393 trim_cache size 0 max 16384 2024-04-26T01:56:20.271 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:{ 2024-04-26T01:56:20.271 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "message": "", 2024-04-26T01:56:20.271 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "return_code": 0 2024-04-26T01:56:20.271 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:} 2024-04-26T01:56:20.271 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.272+0000 7fc6c3200640 1 -- 172.21.15.151:0/855096660 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fc6a80a3d10 msgr2=0x7fc6a80a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:20.272 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.272+0000 7fc6c3200640 1 --2- 172.21.15.151:0/855096660 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fc6a80a3d10 0x7fc6a80a61d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x5638a2e90f50 tx=0x7fc6b4005f90 comp rx=0 tx=0).stop 2024-04-26T01:56:20.272 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.272+0000 7fc6c3200640 1 -- 172.21.15.151:0/855096660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc6c4074ca0 msgr2=0x7fc6c419acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:20.272 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.272+0000 7fc6c3200640 1 --2- 172.21.15.151:0/855096660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc6c4074ca0 0x7fc6c419acc0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc6b000e9f0 tx=0x7fc6b000eeb0 comp rx=0 tx=0).stop 2024-04-26T01:56:20.272 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.272+0000 7fc6c3200640 1 -- 172.21.15.151:0/855096660 shutdown_connections 2024-04-26T01:56:20.272 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.272+0000 7fc6c3200640 1 --2- 172.21.15.151:0/855096660 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fc6a80a3d10 0x7fc6a80a61d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:20.272 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.272+0000 7fc6c3200640 1 --2- 172.21.15.151:0/855096660 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6c410e5a0 0x7fc6c419eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:20.272 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.272+0000 7fc6c3200640 1 --2- 172.21.15.151:0/855096660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc6c4074ca0 0x7fc6c419acc0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:20.272 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.272+0000 7fc6c3200640 1 --2- 172.21.15.151:0/855096660 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6c4074380 0x7fc6c419a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:20.272 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.272+0000 7fc6c3200640 1 -- 172.21.15.151:0/855096660 >> 172.21.15.151:0/855096660 conn(0x7fc6c40fd890 msgr2=0x7fc6c40ff0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:20.272 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.272+0000 7fc6c3200640 1 -- 172.21.15.151:0/855096660 shutdown_connections 2024-04-26T01:56:20.272 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:20.272+0000 7fc6c3200640 1 -- 172.21.15.151:0/855096660 wait complete. 2024-04-26T01:56:20.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:56:20 smithi165 ceph-mon[32648]: pgmap v84: 97 pgs: 97 active+clean; 37 MiB data, 591 MiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 738 KiB/s wr, 187 op/s 2024-04-26T01:56:20.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:56:20 smithi151 ceph-mon[28057]: pgmap v84: 97 pgs: 97 active+clean; 37 MiB data, 591 MiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 738 KiB/s wr, 187 op/s 2024-04-26T01:56:20.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:56:20 smithi172 ceph-mon[32785]: pgmap v84: 97 pgs: 97 active+clean; 37 MiB data, 591 MiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 738 KiB/s wr, 187 op/s 2024-04-26T01:56:22.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:56:22 smithi165 ceph-mon[32648]: pgmap v85: 97 pgs: 97 active+clean; 37 MiB data, 591 MiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 614 KiB/s wr, 133 op/s 2024-04-26T01:56:22.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:56:22 smithi151 ceph-mon[28057]: pgmap v85: 97 pgs: 97 active+clean; 37 MiB data, 591 MiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 614 KiB/s wr, 133 op/s 2024-04-26T01:56:22.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:56:22 smithi172 ceph-mon[32785]: pgmap v85: 97 pgs: 97 active+clean; 37 MiB data, 591 MiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 614 KiB/s wr, 133 op/s 2024-04-26T01:56:24.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:56:24 smithi151 ceph-mon[28057]: pgmap v86: 97 pgs: 97 active+clean; 33 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 640 KiB/s wr, 142 op/s 2024-04-26T01:56:24.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:56:24 smithi172 ceph-mon[32785]: pgmap v86: 97 pgs: 97 active+clean; 33 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 640 KiB/s wr, 142 op/s 2024-04-26T01:56:24.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:56:24 smithi165 ceph-mon[32648]: pgmap v86: 97 pgs: 97 active+clean; 33 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 640 KiB/s wr, 142 op/s 2024-04-26T01:56:26.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:56:26 smithi151 ceph-mon[28057]: pgmap v87: 97 pgs: 97 active+clean; 29 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 698 KiB/s wr, 173 op/s 2024-04-26T01:56:26.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:56:26 smithi172 ceph-mon[32785]: pgmap v87: 97 pgs: 97 active+clean; 29 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 698 KiB/s wr, 173 op/s 2024-04-26T01:56:26.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:56:26 smithi165 ceph-mon[32648]: pgmap v87: 97 pgs: 97 active+clean; 29 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 698 KiB/s wr, 173 op/s 2024-04-26T01:56:28.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:56:28 smithi151 ceph-mon[28057]: pgmap v88: 97 pgs: 97 active+clean; 29 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 684 KiB/s wr, 169 op/s 2024-04-26T01:56:28.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:56:28 smithi172 ceph-mon[32785]: pgmap v88: 97 pgs: 97 active+clean; 29 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 684 KiB/s wr, 169 op/s 2024-04-26T01:56:28.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:56:28 smithi165 ceph-mon[32648]: pgmap v88: 97 pgs: 97 active+clean; 29 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 684 KiB/s wr, 169 op/s 2024-04-26T01:56:30.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:56:30 smithi151 ceph-mon[28057]: pgmap v89: 97 pgs: 97 active+clean; 29 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 684 KiB/s wr, 172 op/s 2024-04-26T01:56:30.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:56:30 smithi172 ceph-mon[32785]: pgmap v89: 97 pgs: 97 active+clean; 29 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 684 KiB/s wr, 172 op/s 2024-04-26T01:56:30.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:56:30 smithi165 ceph-mon[32648]: pgmap v89: 97 pgs: 97 active+clean; 29 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 684 KiB/s wr, 172 op/s 2024-04-26T01:56:32.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:56:32 smithi151 ceph-mon[28057]: pgmap v90: 97 pgs: 97 active+clean; 29 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 84 KiB/s wr, 43 op/s 2024-04-26T01:56:32.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:56:32 smithi172 ceph-mon[32785]: pgmap v90: 97 pgs: 97 active+clean; 29 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 84 KiB/s wr, 43 op/s 2024-04-26T01:56:32.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:56:32 smithi165 ceph-mon[32648]: pgmap v90: 97 pgs: 97 active+clean; 29 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 84 KiB/s wr, 43 op/s 2024-04-26T01:56:33.410 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.412+0000 7efc55200640 1 -- 172.21.15.151:0/2314641864 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc50103830 msgr2=0x7efc5010bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:33.410 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.412+0000 7efc55200640 1 --2- 172.21.15.151:0/2314641864 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc50103830 0x7efc5010bd10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7efc38009860 tx=0x7efc3802a3d0 comp rx=0 tx=0).stop 2024-04-26T01:56:33.411 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.412+0000 7efc55200640 1 -- 172.21.15.151:0/2314641864 shutdown_connections 2024-04-26T01:56:33.411 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.412+0000 7efc55200640 1 --2- 172.21.15.151:0/2314641864 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc5010c330 0x7efc5010e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:33.411 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.412+0000 7efc55200640 1 --2- 172.21.15.151:0/2314641864 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc50103830 0x7efc5010bd10 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:33.411 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.412+0000 7efc55200640 1 --2- 172.21.15.151:0/2314641864 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efc50102f10 0x7efc501032f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:33.411 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.412+0000 7efc55200640 1 -- 172.21.15.151:0/2314641864 >> 172.21.15.151:0/2314641864 conn(0x7efc500fc770 msgr2=0x7efc500feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:33.411 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.412+0000 7efc55200640 1 -- 172.21.15.151:0/2314641864 shutdown_connections 2024-04-26T01:56:33.411 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.412+0000 7efc55200640 1 -- 172.21.15.151:0/2314641864 wait complete. 2024-04-26T01:56:33.413 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.414+0000 7efc55200640 1 Processor -- start 2024-04-26T01:56:33.413 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.414+0000 7efc55200640 1 -- start start 2024-04-26T01:56:33.413 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.414+0000 7efc55200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efc50102f10 0x7efc501a0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:33.413 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.414+0000 7efc55200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc50103830 0x7efc501a0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:33.413 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.414+0000 7efc55200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc5010c330 0x7efc5019a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:33.413 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.414+0000 7efc55200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efc50111ff0 con 0x7efc50103830 2024-04-26T01:56:33.413 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.414+0000 7efc55200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7efc50111e70 con 0x7efc50102f10 2024-04-26T01:56:33.413 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.414+0000 7efc55200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efc50112170 con 0x7efc5010c330 2024-04-26T01:56:33.413 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.414+0000 7efc47e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc50103830 0x7efc501a0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:33.413 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.415+0000 7efc47e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc50103830 0x7efc501a0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37792/0 (socket says 172.21.15.151:37792) 2024-04-26T01:56:33.413 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.415+0000 7efc47e00640 1 -- 172.21.15.151:0/1842066507 learned_addr learned my addr 172.21.15.151:0/1842066507 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:56:33.413 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.415+0000 7efc4f400640 1 --2- 172.21.15.151:0/1842066507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efc50102f10 0x7efc501a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:33.414 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.415+0000 7efc4fe00640 1 --2- 172.21.15.151:0/1842066507 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc5010c330 0x7efc5019a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:33.414 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.415+0000 7efc47e00640 1 -- 172.21.15.151:0/1842066507 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc5010c330 msgr2=0x7efc5019a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:33.414 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.415+0000 7efc47e00640 1 --2- 172.21.15.151:0/1842066507 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc5010c330 0x7efc5019a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:33.414 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.415+0000 7efc47e00640 1 -- 172.21.15.151:0/1842066507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efc50102f10 msgr2=0x7efc501a0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:33.414 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.415+0000 7efc47e00640 1 --2- 172.21.15.151:0/1842066507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efc50102f10 0x7efc501a0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:33.414 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.415+0000 7efc47e00640 1 -- 172.21.15.151:0/1842066507 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc5019af10 con 0x7efc50103830 2024-04-26T01:56:33.414 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.415+0000 7efc4f400640 1 --2- 172.21.15.151:0/1842066507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efc50102f10 0x7efc501a0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:56:33.414 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.415+0000 7efc47e00640 1 --2- 172.21.15.151:0/1842066507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc50103830 0x7efc501a0ad0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7efc38005940 tx=0x7efc38030ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.416+0000 7efc4cc00640 1 -- 172.21.15.151:0/1842066507 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc38040070 con 0x7efc50103830 2024-04-26T01:56:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.416+0000 7efc55200640 1 -- 172.21.15.151:0/1842066507 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc5019b1a0 con 0x7efc50103830 2024-04-26T01:56:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.416+0000 7efc4cc00640 1 -- 172.21.15.151:0/1842066507 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc38005570 con 0x7efc50103830 2024-04-26T01:56:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.416+0000 7efc4cc00640 1 -- 172.21.15.151:0/1842066507 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc380046c0 con 0x7efc50103830 2024-04-26T01:56:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.416+0000 7efc55200640 1 -- 172.21.15.151:0/1842066507 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc501a7510 con 0x7efc50103830 2024-04-26T01:56:33.416 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.419+0000 7efc595cd740 1 Processor -- start 2024-04-26T01:56:33.417 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.419+0000 7efc595cd740 1 -- start start 2024-04-26T01:56:33.418 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.420+0000 7efc595cd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1b39e09b0 0x55e1b39e0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:33.418 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.420+0000 7efc595cd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e1b39e12d0 0x55e1b39eb650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:33.418 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.420+0000 7efc595cd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1b39ebb90 0x55e1b39edf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:33.418 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.420+0000 7efc595cd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e1b3946900 con 0x55e1b39ebb90 2024-04-26T01:56:33.418 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.420+0000 7efc595cd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e1b38ecef0 con 0x55e1b39e12d0 2024-04-26T01:56:33.418 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.420+0000 7efc595cd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e1b39428c0 con 0x55e1b39e09b0 2024-04-26T01:56:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.420+0000 7efc4cc00640 1 -- 172.21.15.151:0/1842066507 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7efc38043070 con 0x7efc50103830 2024-04-26T01:56:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.420+0000 7efc4fe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1b39e09b0 0x55e1b39e0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.420+0000 7efc4fe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1b39e09b0 0x55e1b39e0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:54194/0 (socket says 172.21.15.151:54194) 2024-04-26T01:56:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.420+0000 7efc4f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e1b39e12d0 0x55e1b39eb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.420+0000 7efc4fe00640 1 -- 172.21.15.151:0/141063566 learned_addr learned my addr 172.21.15.151:0/141063566 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:56:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.420+0000 7efc47e00640 1 --2- 172.21.15.151:0/141063566 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1b39ebb90 0x55e1b39edf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.420+0000 7efc4fe00640 1 -- 172.21.15.151:0/141063566 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e1b39e12d0 msgr2=0x55e1b39eb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.420+0000 7efc4fe00640 1 --2- 172.21.15.151:0/141063566 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e1b39e12d0 0x55e1b39eb650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.420+0000 7efc4fe00640 1 -- 172.21.15.151:0/141063566 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1b39ebb90 msgr2=0x55e1b39edf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.420+0000 7efc4fe00640 1 --2- 172.21.15.151:0/141063566 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1b39ebb90 0x55e1b39edf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.420+0000 7efc4fe00640 1 -- 172.21.15.151:0/141063566 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1b3659210 con 0x55e1b39e09b0 2024-04-26T01:56:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.421+0000 7efc4fe00640 1 --2- 172.21.15.151:0/141063566 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1b39e09b0 0x55e1b39e0d90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7efc480099f0 tx=0x7efc480095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79dd8bce98971c16 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:33.423 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.421+0000 7efc44c00640 1 -- 172.21.15.151:0/141063566 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc4801d070 con 0x55e1b39e09b0 2024-04-26T01:56:33.423 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.421+0000 7efc44c00640 1 -- 172.21.15.151:0/141063566 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc48003e40 con 0x55e1b39e09b0 2024-04-26T01:56:33.423 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.421+0000 7efc44c00640 1 -- 172.21.15.151:0/141063566 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc4800d9d0 con 0x55e1b39e09b0 2024-04-26T01:56:33.423 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.422+0000 7efc4cc00640 1 --2- 172.21.15.151:0/1842066507 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7efc340a3d10 0x7efc340a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:33.424 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.422+0000 7efc4cc00640 1 -- 172.21.15.151:0/1842066507 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7efc380f1580 con 0x7efc50103830 2024-04-26T01:56:33.424 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.422+0000 7efc595cd740 1 -- 172.21.15.151:0/141063566 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1b39e09b0 msgr2=0x55e1b39e0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:33.424 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.422+0000 7efc595cd740 1 --2- 172.21.15.151:0/141063566 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1b39e09b0 0x55e1b39e0d90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7efc480099f0 tx=0x7efc480095c0 comp rx=0 tx=0).stop 2024-04-26T01:56:33.424 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.422+0000 7efc4f400640 1 --2- 172.21.15.151:0/1842066507 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7efc340a3d10 0x7efc340a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:33.424 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.422+0000 7efc595cd740 1 -- 172.21.15.151:0/141063566 shutdown_connections 2024-04-26T01:56:33.424 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.422+0000 7efc595cd740 1 --2- 172.21.15.151:0/141063566 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1b39ebb90 0x55e1b39edf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:33.424 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.422+0000 7efc595cd740 1 --2- 172.21.15.151:0/141063566 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e1b39e12d0 0x55e1b39eb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:33.424 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.422+0000 7efc595cd740 1 --2- 172.21.15.151:0/141063566 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1b39e09b0 0x55e1b39e0d90 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:33.424 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.422+0000 7efc595cd740 1 -- 172.21.15.151:0/141063566 >> 172.21.15.151:0/141063566 conn(0x55e1b39dcec0 msgr2=0x55e1b39dd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:33.424 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.422+0000 7efc595cd740 1 -- 172.21.15.151:0/141063566 shutdown_connections 2024-04-26T01:56:33.424 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.422+0000 7efc595cd740 1 -- 172.21.15.151:0/141063566 wait complete. 2024-04-26T01:56:33.424 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.423+0000 7efc4f400640 1 --2- 172.21.15.151:0/1842066507 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7efc340a3d10 0x7efc340a61d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x55e1b39e1f50 tx=0x7efc40005f90 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:33.425 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.423+0000 7efc595cd740 1 -- start start 2024-04-26T01:56:33.425 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.423+0000 7efc595cd740 1 Processor -- start 2024-04-26T01:56:33.425 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.423+0000 7efc595cd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e1b39e09b0 0x55e1b39e8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:33.425 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.423+0000 7efc595cd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1b39e12d0 0x55e1b39e9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:33.425 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.423+0000 7efc595cd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1b39ebb90 0x55e1b39e95a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:33.425 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.423+0000 7efc595cd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e1b39ee7c0 con 0x55e1b39e12d0 2024-04-26T01:56:33.425 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.424+0000 7efc595cd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e1b3900310 con 0x55e1b39e09b0 2024-04-26T01:56:33.425 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.424+0000 7efc595cd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e1b39ee940 con 0x55e1b39ebb90 2024-04-26T01:56:33.425 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.424+0000 7efc595cd740 10 client.0 _ll_register_callbacks cb 0x55e1b38cecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:56:33.425 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.424+0000 7efc47e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1b39e12d0 0x55e1b39e9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:33.425 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.424+0000 7efc4f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e1b39e09b0 0x55e1b39e8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:33.425 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.424+0000 7efc47e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1b39e12d0 0x55e1b39e9060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37810/0 (socket says 172.21.15.151:37810) 2024-04-26T01:56:33.425 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.424+0000 7efc47e00640 1 -- 172.21.15.151:0/3134002687 learned_addr learned my addr 172.21.15.151:0/3134002687 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:56:33.425 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.424+0000 7efc4fe00640 1 --2- 172.21.15.151:0/3134002687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1b39ebb90 0x55e1b39e95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.424+0000 7efc47e00640 1 -- 172.21.15.151:0/3134002687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1b39ebb90 msgr2=0x55e1b39e95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.424+0000 7efc47e00640 1 --2- 172.21.15.151:0/3134002687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1b39ebb90 0x55e1b39e95a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.424+0000 7efc47e00640 1 -- 172.21.15.151:0/3134002687 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e1b39e09b0 msgr2=0x55e1b39e8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.424+0000 7efc47e00640 1 --2- 172.21.15.151:0/3134002687 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e1b39e09b0 0x55e1b39e8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.424+0000 7efc47e00640 1 -- 172.21.15.151:0/3134002687 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1b3725620 con 0x55e1b39e12d0 2024-04-26T01:56:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.424+0000 7efc47e00640 1 --2- 172.21.15.151:0/3134002687 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1b39e12d0 0x55e1b39e9060 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7efc380eed40 tx=0x7efc380f4f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ae5ae6b4ea7d81 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.424+0000 7efc3d600640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-04-26T01:56:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.425+0000 7efc3d600640 1 -- 172.21.15.151:0/3134002687 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3804a030 con 0x55e1b39e12d0 2024-04-26T01:56:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.425+0000 7efc3d600640 1 -- 172.21.15.151:0/3134002687 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7efc38031400 con 0x55e1b39e12d0 2024-04-26T01:56:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.425+0000 7efc595cd740 1 -- 172.21.15.151:0/3134002687 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e1b395a8b0 con 0x55e1b39e12d0 2024-04-26T01:56:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.425+0000 7efc3d600640 1 -- 172.21.15.151:0/3134002687 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efc380bd440 con 0x55e1b39e12d0 2024-04-26T01:56:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.425+0000 7efc595cd740 10 client.15405 fetch_fsmap learned FSMap version 28 2024-04-26T01:56:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.425+0000 7efc595cd740 1 -- 172.21.15.151:0/3134002687 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e1b39faa20 con 0x55e1b39e12d0 2024-04-26T01:56:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.425+0000 7efc3d600640 1 -- 172.21.15.151:0/3134002687 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55e1b395a8b0 con 0x55e1b39e12d0 2024-04-26T01:56:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.426+0000 7efc3d600640 1 -- 172.21.15.151:0/3134002687 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7efc380eef60 con 0x55e1b39e12d0 2024-04-26T01:56:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.426+0000 7efc595cd740 10 client.15405 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T01:56:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.426+0000 7efc595cd740 10 client.15405 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:56:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.426+0000 7efc595cd740 20 client.15405 populate_metadata read hostname 'smithi151' 2024-04-26T01:56:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.426+0000 7efc595cd740 1 --2- 172.21.15.151:0/3134002687 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55e1b39fbe60 0x55e1b39fe300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.426+0000 7efc595cd740 4 client.15405 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:56:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.426+0000 7efc595cd740 1 -- 172.21.15.151:0/3134002687 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e1b3954320 con 0x55e1b39fbe60 2024-04-26T01:56:33.428 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.426+0000 7efc4f400640 1 --2- 172.21.15.151:0/3134002687 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55e1b39fbe60 0x55e1b39fe300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:33.428 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.427+0000 7efc3d600640 10 client.15405 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:56:33.428 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.427+0000 7efc4f400640 1 --2- 172.21.15.151:0/3134002687 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55e1b39fbe60 0x55e1b39fe300 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dff3e75a0be7a5df server_cookie=dc6a435a17327bf in_seq=0 out_seq=0 2024-04-26T01:56:33.428 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.428+0000 7efc3d600640 1 -- 172.21.15.151:0/3134002687 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e1b3954320 con 0x55e1b39fbe60 2024-04-26T01:56:33.429 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.428+0000 7efc3d600640 10 client.15405 handle_command_reply: tid=0 2024-04-26T01:56:33.429 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.428+0000 7efc595cd740 1 client.15405 shutdown 2024-04-26T01:56:33.429 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.428+0000 7efc595cd740 1 -- 172.21.15.151:0/3134002687 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1b39e12d0 msgr2=0x55e1b39e9060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:33.429 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.428+0000 7efc595cd740 1 --2- 172.21.15.151:0/3134002687 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1b39e12d0 0x55e1b39e9060 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7efc380eed40 tx=0x7efc380f4f40 comp rx=0 tx=0).stop 2024-04-26T01:56:33.429 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.428+0000 7efc595cd740 1 -- 172.21.15.151:0/3134002687 shutdown_connections 2024-04-26T01:56:33.429 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.428+0000 7efc595cd740 1 --2- 172.21.15.151:0/3134002687 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55e1b39fbe60 0x55e1b39fe300 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:33.429 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.428+0000 7efc595cd740 1 --2- 172.21.15.151:0/3134002687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1b39ebb90 0x55e1b39e95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:33.429 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.428+0000 7efc595cd740 1 --2- 172.21.15.151:0/3134002687 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1b39e12d0 0x55e1b39e9060 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:33.429 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.428+0000 7efc3d600640 0 client.15405 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:56:33.429 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.428+0000 7efc595cd740 1 --2- 172.21.15.151:0/3134002687 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e1b39e09b0 0x55e1b39e8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:33.429 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.428+0000 7efc595cd740 1 -- 172.21.15.151:0/3134002687 >> 172.21.15.151:0/3134002687 conn(0x55e1b39dcec0 msgr2=0x55e1b39e2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:33.429 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.428+0000 7efc595cd740 1 -- 172.21.15.151:0/3134002687 shutdown_connections 2024-04-26T01:56:33.429 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.428+0000 7efc595cd740 1 -- 172.21.15.151:0/3134002687 wait complete. 2024-04-26T01:56:33.429 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.429+0000 7efc595cd740 20 client.15405 trim_cache size 0 max 16384 2024-04-26T01:56:33.443 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.444+0000 7efc595cd740 1 Processor -- start 2024-04-26T01:56:33.443 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.444+0000 7efc595cd740 1 -- start start 2024-04-26T01:56:33.443 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.445+0000 7efc595cd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e1b39e09b0 0x55e1b39e7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:33.443 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.445+0000 7efc595cd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1b39e12d0 0x55e1b39f7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:33.443 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.445+0000 7efc595cd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1b39ebb90 0x55e1b39e3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:33.443 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.445+0000 7efc595cd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efc501b23d0 con 0x55e1b39e12d0 2024-04-26T01:56:33.443 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.445+0000 7efc595cd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7efc501afca0 con 0x55e1b39e09b0 2024-04-26T01:56:33.443 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.445+0000 7efc595cd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efc501adaf0 con 0x55e1b39ebb90 2024-04-26T01:56:33.443 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.445+0000 7efc47e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1b39e12d0 0x55e1b39f7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:33.443 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.445+0000 7efc47e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1b39e12d0 0x55e1b39f7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37816/0 (socket says 172.21.15.151:37816) 2024-04-26T01:56:33.443 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.445+0000 7efc47e00640 1 -- 172.21.15.151:0/3300903441 learned_addr learned my addr 172.21.15.151:0/3300903441 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:56:33.443 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.445+0000 7efc4f400640 1 --2- 172.21.15.151:0/3300903441 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e1b39e09b0 0x55e1b39e7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:33.444 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.445+0000 7efc4fe00640 1 --2- 172.21.15.151:0/3300903441 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1b39ebb90 0x55e1b39e3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:33.445 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.446+0000 7efc4f400640 1 -- 172.21.15.151:0/3300903441 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1b39ebb90 msgr2=0x55e1b39e3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:33.445 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.446+0000 7efc4f400640 1 --2- 172.21.15.151:0/3300903441 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1b39ebb90 0x55e1b39e3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:33.445 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.446+0000 7efc4f400640 1 -- 172.21.15.151:0/3300903441 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1b39e12d0 msgr2=0x55e1b39f7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:33.445 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.446+0000 7efc4f400640 1 --2- 172.21.15.151:0/3300903441 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1b39e12d0 0x55e1b39f7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:33.445 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.446+0000 7efc4f400640 1 -- 172.21.15.151:0/3300903441 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1b39fe980 con 0x55e1b39e09b0 2024-04-26T01:56:33.445 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.446+0000 7efc47e00640 1 --2- 172.21.15.151:0/3300903441 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1b39e12d0 0x55e1b39f7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:56:33.445 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.446+0000 7efc4f400640 1 --2- 172.21.15.151:0/3300903441 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e1b39e09b0 0x55e1b39e7ac0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x55e1b39fe900 tx=0x7efc4001b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8da6e69b5a7fa0a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:33.445 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.446+0000 7efc3fe00640 1 -- 172.21.15.151:0/3300903441 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc4002b530 con 0x55e1b39e09b0 2024-04-26T01:56:33.445 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.446+0000 7efc3fe00640 1 -- 172.21.15.151:0/3300903441 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc400054d0 con 0x55e1b39e09b0 2024-04-26T01:56:33.445 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.446+0000 7efc3fe00640 1 -- 172.21.15.151:0/3300903441 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc40004a40 con 0x55e1b39e09b0 2024-04-26T01:56:33.446 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.447+0000 7efc595cd740 1 -- 172.21.15.151:0/3300903441 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e1b39e09b0 msgr2=0x55e1b39e7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:33.446 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.447+0000 7efc595cd740 1 --2- 172.21.15.151:0/3300903441 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e1b39e09b0 0x55e1b39e7ac0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x55e1b39fe900 tx=0x7efc4001b000 comp rx=0 tx=0).stop 2024-04-26T01:56:33.446 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.447+0000 7efc595cd740 1 -- 172.21.15.151:0/3300903441 shutdown_connections 2024-04-26T01:56:33.446 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.447+0000 7efc595cd740 1 --2- 172.21.15.151:0/3300903441 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1b39ebb90 0x55e1b39e3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:33.446 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.447+0000 7efc595cd740 1 --2- 172.21.15.151:0/3300903441 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1b39e12d0 0x55e1b39f7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:33.446 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.447+0000 7efc595cd740 1 --2- 172.21.15.151:0/3300903441 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e1b39e09b0 0x55e1b39e7ac0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:33.446 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.447+0000 7efc595cd740 1 -- 172.21.15.151:0/3300903441 >> 172.21.15.151:0/3300903441 conn(0x55e1b39dcec0 msgr2=0x55e1b39e9370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:33.446 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.447+0000 7efc595cd740 1 -- 172.21.15.151:0/3300903441 shutdown_connections 2024-04-26T01:56:33.446 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.447+0000 7efc595cd740 1 -- 172.21.15.151:0/3300903441 wait complete. 2024-04-26T01:56:33.449 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.448+0000 7efc595cd740 1 -- start start 2024-04-26T01:56:33.449 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.448+0000 7efc595cd740 1 Processor -- start 2024-04-26T01:56:33.449 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.448+0000 7efc595cd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1b39e09b0 0x55e1b39f3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:33.449 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.448+0000 7efc595cd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1b39e12d0 0x55e1b39f11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:33.449 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.448+0000 7efc595cd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e1b39ebb90 0x55e1b39f1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:33.449 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.448+0000 7efc595cd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efc501ae160 con 0x55e1b39e09b0 2024-04-26T01:56:33.449 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.448+0000 7efc595cd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7efc501af5d0 con 0x55e1b39ebb90 2024-04-26T01:56:33.449 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.448+0000 7efc595cd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efc501ae830 con 0x55e1b39e12d0 2024-04-26T01:56:33.449 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.448+0000 7efc595cd740 10 client.0 _ll_register_callbacks cb 0x7efc380f4b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:56:33.450 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.449+0000 7efc47e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1b39e12d0 0x55e1b39f11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:33.450 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.449+0000 7efc4f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1b39e09b0 0x55e1b39f3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:33.450 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.449+0000 7efc47e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1b39e12d0 0x55e1b39f11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:54202/0 (socket says 172.21.15.151:54202) 2024-04-26T01:56:33.450 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.449+0000 7efc47e00640 1 -- 172.21.15.151:0/4255912511 learned_addr learned my addr 172.21.15.151:0/4255912511 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:56:33.450 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.449+0000 7efc4fe00640 1 --2- 172.21.15.151:0/4255912511 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e1b39ebb90 0x55e1b39f1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:33.450 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.449+0000 7efc4f400640 1 -- 172.21.15.151:0/4255912511 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1b39e12d0 msgr2=0x55e1b39f11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:33.450 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.449+0000 7efc4f400640 1 --2- 172.21.15.151:0/4255912511 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1b39e12d0 0x55e1b39f11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:33.450 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.449+0000 7efc4f400640 1 -- 172.21.15.151:0/4255912511 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e1b39ebb90 msgr2=0x55e1b39f1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:33.450 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.449+0000 7efc4f400640 1 --2- 172.21.15.151:0/4255912511 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e1b39ebb90 0x55e1b39f1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:33.450 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.449+0000 7efc4f400640 1 -- 172.21.15.151:0/4255912511 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1b39f1dc0 con 0x55e1b39e09b0 2024-04-26T01:56:33.450 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.449+0000 7efc47e00640 1 --2- 172.21.15.151:0/4255912511 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1b39e12d0 0x55e1b39f11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:56:33.450 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.450+0000 7efc4f400640 1 --2- 172.21.15.151:0/4255912511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1b39e09b0 0x55e1b39f3e30 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x55e1b39fe900 tx=0x7efc400046e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bcb55d9b5b5cd97 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:33.451 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.450+0000 7efc3d600640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-04-26T01:56:33.451 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.450+0000 7efc3d600640 1 -- 172.21.15.151:0/4255912511 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc4003a020 con 0x55e1b39e09b0 2024-04-26T01:56:33.451 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.450+0000 7efc3d600640 1 -- 172.21.15.151:0/4255912511 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7efc4002b530 con 0x55e1b39e09b0 2024-04-26T01:56:33.451 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.450+0000 7efc595cd740 1 -- 172.21.15.151:0/4255912511 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc400054d0 con 0x55e1b39e09b0 2024-04-26T01:56:33.451 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.450+0000 7efc3d600640 1 -- 172.21.15.151:0/4255912511 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efc40045050 con 0x55e1b39e09b0 2024-04-26T01:56:33.451 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.450+0000 7efc595cd740 10 client.15411 fetch_fsmap learned FSMap version 28 2024-04-26T01:56:33.451 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.450+0000 7efc595cd740 1 -- 172.21.15.151:0/4255912511 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e1b3a62370 con 0x55e1b39e09b0 2024-04-26T01:56:33.451 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.450+0000 7efc3d600640 1 -- 172.21.15.151:0/4255912511 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc400054d0 con 0x55e1b39e09b0 2024-04-26T01:56:33.451 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.451+0000 7efc3d600640 1 -- 172.21.15.151:0/4255912511 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7efc40036480 con 0x55e1b39e09b0 2024-04-26T01:56:33.451 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.451+0000 7efc595cd740 10 client.15411 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T01:56:33.451 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.451+0000 7efc595cd740 10 client.15411 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:56:33.451 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.451+0000 7efc595cd740 20 client.15411 populate_metadata read hostname 'smithi151' 2024-04-26T01:56:33.451 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.451+0000 7efc595cd740 1 --2- 172.21.15.151:0/4255912511 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55e1b3a62b10 0x55e1b3a64f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:33.451 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.451+0000 7efc595cd740 4 client.15411 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:56:33.452 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.451+0000 7efc595cd740 1 -- 172.21.15.151:0/4255912511 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e1b394e190 con 0x55e1b3a62b10 2024-04-26T01:56:33.452 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.451+0000 7efc47e00640 1 --2- 172.21.15.151:0/4255912511 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55e1b3a62b10 0x55e1b3a64f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:33.452 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.452+0000 7efc47e00640 1 --2- 172.21.15.151:0/4255912511 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55e1b3a62b10 0x55e1b3a64f30 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cfe2bea51f306dc server_cookie=90b839831ab108db in_seq=0 out_seq=0 2024-04-26T01:56:33.452 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.452+0000 7efc3d600640 10 client.15411 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:56:33.473 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.474+0000 7efc3d600640 1 -- 172.21.15.151:0/4255912511 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e1b394e190 con 0x55e1b3a62b10 2024-04-26T01:56:33.473 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.475+0000 7efc3d600640 10 client.15411 handle_command_reply: tid=0 2024-04-26T01:56:33.473 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.475+0000 7efc595cd740 1 client.15411 shutdown 2024-04-26T01:56:33.473 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.475+0000 7efc595cd740 1 -- 172.21.15.151:0/4255912511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1b39e09b0 msgr2=0x55e1b39f3e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:33.473 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.475+0000 7efc595cd740 1 --2- 172.21.15.151:0/4255912511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1b39e09b0 0x55e1b39f3e30 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x55e1b39fe900 tx=0x7efc400046e0 comp rx=0 tx=0).stop 2024-04-26T01:56:33.473 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.475+0000 7efc595cd740 1 -- 172.21.15.151:0/4255912511 shutdown_connections 2024-04-26T01:56:33.473 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.475+0000 7efc595cd740 1 --2- 172.21.15.151:0/4255912511 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55e1b3a62b10 0x55e1b3a64f30 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:33.473 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.475+0000 7efc595cd740 1 --2- 172.21.15.151:0/4255912511 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e1b39ebb90 0x55e1b39f1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:33.473 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.475+0000 7efc595cd740 1 --2- 172.21.15.151:0/4255912511 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1b39e12d0 0x55e1b39f11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:33.473 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.475+0000 7efc3d600640 0 client.15411 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:56:33.474 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.475+0000 7efc595cd740 1 --2- 172.21.15.151:0/4255912511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1b39e09b0 0x55e1b39f3e30 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:33.474 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.475+0000 7efc595cd740 1 -- 172.21.15.151:0/4255912511 >> 172.21.15.151:0/4255912511 conn(0x55e1b39dcec0 msgr2=0x55e1b39db790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:33.474 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.475+0000 7efc595cd740 1 -- 172.21.15.151:0/4255912511 shutdown_connections 2024-04-26T01:56:33.474 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.475+0000 7efc595cd740 1 -- 172.21.15.151:0/4255912511 wait complete. 2024-04-26T01:56:33.474 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.475+0000 7efc595cd740 20 client.15411 trim_cache size 0 max 16384 2024-04-26T01:56:33.474 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:{ 2024-04-26T01:56:33.474 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "message": "", 2024-04-26T01:56:33.474 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "return_code": 0 2024-04-26T01:56:33.475 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:} 2024-04-26T01:56:33.475 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.476+0000 7efc55200640 1 -- 172.21.15.151:0/1842066507 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7efc340a3d10 msgr2=0x7efc340a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:33.475 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.476+0000 7efc55200640 1 --2- 172.21.15.151:0/1842066507 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7efc340a3d10 0x7efc340a61d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x55e1b39e1f50 tx=0x7efc40005f90 comp rx=0 tx=0).stop 2024-04-26T01:56:33.475 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.476+0000 7efc55200640 1 -- 172.21.15.151:0/1842066507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc50103830 msgr2=0x7efc501a0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:33.476 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.476+0000 7efc55200640 1 --2- 172.21.15.151:0/1842066507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc50103830 0x7efc501a0ad0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7efc38005940 tx=0x7efc38030ef0 comp rx=0 tx=0).stop 2024-04-26T01:56:33.476 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.476+0000 7efc55200640 1 -- 172.21.15.151:0/1842066507 shutdown_connections 2024-04-26T01:56:33.476 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.476+0000 7efc55200640 1 --2- 172.21.15.151:0/1842066507 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7efc340a3d10 0x7efc340a61d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:33.476 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.476+0000 7efc55200640 1 --2- 172.21.15.151:0/1842066507 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc5010c330 0x7efc5019a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:33.476 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.476+0000 7efc55200640 1 --2- 172.21.15.151:0/1842066507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc50103830 0x7efc501a0ad0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:33.476 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.476+0000 7efc55200640 1 --2- 172.21.15.151:0/1842066507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efc50102f10 0x7efc501a0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:33.476 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.476+0000 7efc55200640 1 -- 172.21.15.151:0/1842066507 >> 172.21.15.151:0/1842066507 conn(0x7efc500fc770 msgr2=0x7efc50101c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:33.476 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.477+0000 7efc55200640 1 -- 172.21.15.151:0/1842066507 shutdown_connections 2024-04-26T01:56:33.476 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:33.477+0000 7efc55200640 1 -- 172.21.15.151:0/1842066507 wait complete. 2024-04-26T01:56:34.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:56:34 smithi151 ceph-mon[28057]: pgmap v91: 97 pgs: 97 active+clean; 29 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 87 KiB/s wr, 44 op/s 2024-04-26T01:56:34.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:56:34 smithi172 ceph-mon[32785]: pgmap v91: 97 pgs: 97 active+clean; 29 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 87 KiB/s wr, 44 op/s 2024-04-26T01:56:34.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:56:34 smithi165 ceph-mon[32648]: pgmap v91: 97 pgs: 97 active+clean; 29 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 87 KiB/s wr, 44 op/s 2024-04-26T01:56:36.398 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:56:36 smithi165 ceph-mon[32648]: pgmap v92: 97 pgs: 97 active+clean; 29 MiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 74 KiB/s wr, 37 op/s 2024-04-26T01:56:36.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:56:36 smithi172 ceph-mon[32785]: pgmap v92: 97 pgs: 97 active+clean; 29 MiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 74 KiB/s wr, 37 op/s 2024-04-26T01:56:36.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:56:36 smithi151 ceph-mon[28057]: pgmap v92: 97 pgs: 97 active+clean; 29 MiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 74 KiB/s wr, 37 op/s 2024-04-26T01:56:38.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:56:38 smithi172 ceph-mon[32785]: pgmap v93: 97 pgs: 97 active+clean; 29 MiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 16 KiB/s wr, 6 op/s 2024-04-26T01:56:38.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:56:38 smithi165 ceph-mon[32648]: pgmap v93: 97 pgs: 97 active+clean; 29 MiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 16 KiB/s wr, 6 op/s 2024-04-26T01:56:38.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:56:38 smithi151 ceph-mon[28057]: pgmap v93: 97 pgs: 97 active+clean; 29 MiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 16 KiB/s wr, 6 op/s 2024-04-26T01:56:40.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:56:40 smithi172 ceph-mon[32785]: pgmap v94: 97 pgs: 97 active+clean; 31 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 178 KiB/s wr, 19 op/s 2024-04-26T01:56:40.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:56:40 smithi165 ceph-mon[32648]: pgmap v94: 97 pgs: 97 active+clean; 31 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 178 KiB/s wr, 19 op/s 2024-04-26T01:56:40.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:56:40 smithi151 ceph-mon[28057]: pgmap v94: 97 pgs: 97 active+clean; 31 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 178 KiB/s wr, 19 op/s 2024-04-26T01:56:42.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:56:42 smithi172 ceph-mon[32785]: pgmap v95: 97 pgs: 97 active+clean; 31 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 177 KiB/s wr, 16 op/s 2024-04-26T01:56:42.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:56:42 smithi165 ceph-mon[32648]: pgmap v95: 97 pgs: 97 active+clean; 31 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 177 KiB/s wr, 16 op/s 2024-04-26T01:56:42.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:56:42 smithi151 ceph-mon[28057]: pgmap v95: 97 pgs: 97 active+clean; 31 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 177 KiB/s wr, 16 op/s 2024-04-26T01:56:44.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:56:44 smithi172 ceph-mon[32785]: pgmap v96: 97 pgs: 97 active+clean; 31 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 190 KiB/s wr, 17 op/s 2024-04-26T01:56:44.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:56:44 smithi165 ceph-mon[32648]: pgmap v96: 97 pgs: 97 active+clean; 31 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 190 KiB/s wr, 17 op/s 2024-04-26T01:56:44.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:56:44 smithi151 ceph-mon[28057]: pgmap v96: 97 pgs: 97 active+clean; 31 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 190 KiB/s wr, 17 op/s 2024-04-26T01:56:46.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:56:46 smithi172 ceph-mon[32785]: pgmap v97: 97 pgs: 97 active+clean; 31 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 187 KiB/s wr, 18 op/s 2024-04-26T01:56:46.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:56:46 smithi165 ceph-mon[32648]: pgmap v97: 97 pgs: 97 active+clean; 31 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 187 KiB/s wr, 18 op/s 2024-04-26T01:56:46.617 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.618+0000 7f7182400640 1 -- 172.21.15.151:0/1745626910 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f717c108490 msgr2=0x7f717c1088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:46.618 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.618+0000 7f7182400640 1 --2- 172.21.15.151:0/1745626910 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f717c108490 0x7f717c1088d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7164009930 tx=0x7f716402a440 comp rx=0 tx=0).stop 2024-04-26T01:56:46.618 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.619+0000 7f7182400640 1 -- 172.21.15.151:0/1745626910 shutdown_connections 2024-04-26T01:56:46.618 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.619+0000 7f7182400640 1 --2- 172.21.15.151:0/1745626910 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f717c108e10 0x7f717c115250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:46.618 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.619+0000 7f7182400640 1 --2- 172.21.15.151:0/1745626910 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f717c108490 0x7f717c1088d0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:46.618 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.619+0000 7f7182400640 1 --2- 172.21.15.151:0/1745626910 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f717c10e420 0x7f717c10e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:46.618 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.619+0000 7f7182400640 1 -- 172.21.15.151:0/1745626910 >> 172.21.15.151:0/1745626910 conn(0x7f717c077d30 msgr2=0x7f717c106d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:46.618 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.619+0000 7f7182400640 1 -- 172.21.15.151:0/1745626910 shutdown_connections 2024-04-26T01:56:46.618 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.619+0000 7f7182400640 1 -- 172.21.15.151:0/1745626910 wait complete. 2024-04-26T01:56:46.620 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.620+0000 7f7182400640 1 Processor -- start 2024-04-26T01:56:46.621 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.620+0000 7f7182400640 1 -- start start 2024-04-26T01:56:46.621 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.620+0000 7f7182400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f717c108490 0x7f717c1a47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:46.621 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.620+0000 7f7182400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f717c108e10 0x7f717c1a4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:46.621 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.620+0000 7f7182400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f717c10e420 0x7f717c19e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:46.621 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.620+0000 7f7182400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f717c1185f0 con 0x7f717c108490 2024-04-26T01:56:46.621 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.620+0000 7f7182400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f717c118470 con 0x7f717c10e420 2024-04-26T01:56:46.621 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.620+0000 7f7182400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f717c118770 con 0x7f717c108e10 2024-04-26T01:56:46.621 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.621+0000 7f7181a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f717c10e420 0x7f717c19e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:46.621 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.621+0000 7f7181000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f717c108490 0x7f717c1a47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:46.621 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.621+0000 7f7181a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f717c10e420 0x7f717c19e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:48914/0 (socket says 172.21.15.151:48914) 2024-04-26T01:56:46.621 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.621+0000 7f7181a00640 1 -- 172.21.15.151:0/444136238 learned_addr learned my addr 172.21.15.151:0/444136238 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:56:46.621 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.621+0000 7f7173e00640 1 --2- 172.21.15.151:0/444136238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f717c108e10 0x7f717c1a4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:46.621 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.621+0000 7f7181a00640 1 -- 172.21.15.151:0/444136238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f717c108e10 msgr2=0x7f717c1a4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:46.622 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.621+0000 7f7181a00640 1 --2- 172.21.15.151:0/444136238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f717c108e10 0x7f717c1a4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:46.622 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.621+0000 7f7181a00640 1 -- 172.21.15.151:0/444136238 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f717c108490 msgr2=0x7f717c1a47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:46.622 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.621+0000 7f7181a00640 1 --2- 172.21.15.151:0/444136238 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f717c108490 0x7f717c1a47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:46.622 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.621+0000 7f7181a00640 1 -- 172.21.15.151:0/444136238 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f717c19f0f0 con 0x7f717c10e420 2024-04-26T01:56:46.622 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.622+0000 7f7181a00640 1 --2- 172.21.15.151:0/444136238 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f717c10e420 0x7f717c19e970 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f717400eb10 tx=0x7f717400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:46.622 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.622+0000 7f717a000640 1 -- 172.21.15.151:0/444136238 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7174004b20 con 0x7f717c10e420 2024-04-26T01:56:46.622 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.622+0000 7f717a000640 1 -- 172.21.15.151:0/444136238 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7174004cc0 con 0x7f717c10e420 2024-04-26T01:56:46.622 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.622+0000 7f7182400640 1 -- 172.21.15.151:0/444136238 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f717c19f3e0 con 0x7f717c10e420 2024-04-26T01:56:46.622 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.622+0000 7f717a000640 1 -- 172.21.15.151:0/444136238 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7174010690 con 0x7f717c10e420 2024-04-26T01:56:46.622 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.622+0000 7f7182400640 1 -- 172.21.15.151:0/444136238 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f717c1ab680 con 0x7f717c10e420 2024-04-26T01:56:46.623 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:56:46 smithi151 ceph-mon[28057]: pgmap v97: 97 pgs: 97 active+clean; 31 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 187 KiB/s wr, 18 op/s 2024-04-26T01:56:46.624 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.625+0000 7f71867b8740 1 Processor -- start 2024-04-26T01:56:46.624 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.625+0000 7f717a000640 1 -- 172.21.15.151:0/444136238 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f7174005020 con 0x7f717c10e420 2024-04-26T01:56:46.624 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.625+0000 7f71867b8740 1 -- start start 2024-04-26T01:56:46.627 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.626+0000 7f71867b8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5638ec9c69b0 0x5638ec9c6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:46.627 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.626+0000 7f71867b8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638ec9c72d0 0x5638ec9d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:46.627 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.626+0000 7f71867b8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5638ec9d1b90 0x5638ec9d3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:46.627 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.626+0000 7f71867b8740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5638ec92c900 con 0x5638ec9c69b0 2024-04-26T01:56:46.627 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.626+0000 7f71867b8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5638ec8d2ef0 con 0x5638ec9c72d0 2024-04-26T01:56:46.627 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.626+0000 7f71867b8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5638ec9288c0 con 0x5638ec9d1b90 2024-04-26T01:56:46.627 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.626+0000 7f7173e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5638ec9c69b0 0x5638ec9c6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:46.628 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.626+0000 7f7181000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638ec9c72d0 0x5638ec9d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:46.628 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.626+0000 7f7173e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5638ec9c69b0 0x5638ec9c6d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:46048/0 (socket says 172.21.15.151:46048) 2024-04-26T01:56:46.628 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.626+0000 7f7173e00640 1 -- 172.21.15.151:0/2363685197 learned_addr learned my addr 172.21.15.151:0/2363685197 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:56:46.628 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.626+0000 7f7173e00640 1 --2- 172.21.15.151:0/2363685197 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5638ec9d1b90 0x5638ec9d3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:46.628 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.627+0000 7f7181000640 1 -- 172.21.15.151:0/2363685197 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5638ec9d1b90 msgr2=0x5638ec9d3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:46.628 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.627+0000 7f7181000640 1 --2- 172.21.15.151:0/2363685197 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5638ec9d1b90 0x5638ec9d3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:46.628 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.627+0000 7f7181000640 1 -- 172.21.15.151:0/2363685197 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5638ec9c69b0 msgr2=0x5638ec9c6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:46.628 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.627+0000 7f7181000640 1 --2- 172.21.15.151:0/2363685197 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5638ec9c69b0 0x5638ec9c6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:46.628 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.627+0000 7f7181000640 1 -- 172.21.15.151:0/2363685197 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638ec63f210 con 0x5638ec9c72d0 2024-04-26T01:56:46.628 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.627+0000 7f717a000640 1 --2- 172.21.15.151:0/444136238 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f71600a3d10 0x7f71600a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:46.628 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.627+0000 7f7181000640 1 --2- 172.21.15.151:0/2363685197 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638ec9c72d0 0x5638ec9d1650 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f716c009bf0 tx=0x7f716c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c01866f483c035b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:46.628 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.627+0000 7f717a000640 1 -- 172.21.15.151:0/444136238 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f71740d0d40 con 0x7f717c10e420 2024-04-26T01:56:46.628 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.627+0000 7f7173e00640 1 --2- 172.21.15.151:0/444136238 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f71600a3d10 0x7f71600a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.627+0000 7f7172000640 1 -- 172.21.15.151:0/2363685197 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716c004050 con 0x5638ec9c72d0 2024-04-26T01:56:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.627+0000 7f7172000640 1 -- 172.21.15.151:0/2363685197 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f716c0054d0 con 0x5638ec9c72d0 2024-04-26T01:56:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.627+0000 7f7172000640 1 -- 172.21.15.151:0/2363685197 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716c004c80 con 0x5638ec9c72d0 2024-04-26T01:56:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.628+0000 7f71867b8740 1 -- 172.21.15.151:0/2363685197 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638ec9c72d0 msgr2=0x5638ec9d1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.628+0000 7f7173e00640 1 --2- 172.21.15.151:0/444136238 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f71600a3d10 0x7f71600a61d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x5638ec9c42e0 tx=0x7f7164005fd0 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.628+0000 7f71867b8740 1 --2- 172.21.15.151:0/2363685197 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638ec9c72d0 0x5638ec9d1650 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f716c009bf0 tx=0x7f716c011810 comp rx=0 tx=0).stop 2024-04-26T01:56:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.628+0000 7f71867b8740 1 -- 172.21.15.151:0/2363685197 shutdown_connections 2024-04-26T01:56:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.628+0000 7f71867b8740 1 --2- 172.21.15.151:0/2363685197 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5638ec9d1b90 0x5638ec9d3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.628+0000 7f71867b8740 1 --2- 172.21.15.151:0/2363685197 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638ec9c72d0 0x5638ec9d1650 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.628+0000 7f71867b8740 1 --2- 172.21.15.151:0/2363685197 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5638ec9c69b0 0x5638ec9c6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.628+0000 7f71867b8740 1 -- 172.21.15.151:0/2363685197 >> 172.21.15.151:0/2363685197 conn(0x5638ec9c2ec0 msgr2=0x5638ec9c32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.628+0000 7f71867b8740 1 -- 172.21.15.151:0/2363685197 shutdown_connections 2024-04-26T01:56:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.628+0000 7f71867b8740 1 -- 172.21.15.151:0/2363685197 wait complete. 2024-04-26T01:56:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.629+0000 7f71867b8740 1 -- start start 2024-04-26T01:56:46.634 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.629+0000 7f71867b8740 1 Processor -- start 2024-04-26T01:56:46.634 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.629+0000 7f71867b8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5638ec9c69b0 0x5638ec9ceb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:46.634 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.629+0000 7f71867b8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5638ec9c72d0 0x5638ec9cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:46.634 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.630+0000 7f71867b8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638ec9d1b90 0x5638ec9cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:46.634 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.630+0000 7f71867b8740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5638ec9d47c0 con 0x5638ec9c69b0 2024-04-26T01:56:46.634 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.630+0000 7f71867b8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5638ec8e6310 con 0x5638ec9d1b90 2024-04-26T01:56:46.634 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.630+0000 7f71867b8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5638ec9d4940 con 0x5638ec9c72d0 2024-04-26T01:56:46.634 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.630+0000 7f71867b8740 10 client.0 _ll_register_callbacks cb 0x5638ec8b4cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:56:46.634 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.630+0000 7f7181a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638ec9d1b90 0x5638ec9cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:46.634 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.630+0000 7f7181000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5638ec9c69b0 0x5638ec9ceb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:46.635 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.630+0000 7f7181a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638ec9d1b90 0x5638ec9cf5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:48934/0 (socket says 172.21.15.151:48934) 2024-04-26T01:56:46.635 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.630+0000 7f7181a00640 1 -- 172.21.15.151:0/1018510428 learned_addr learned my addr 172.21.15.151:0/1018510428 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:56:46.635 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.630+0000 7f7173e00640 1 --2- 172.21.15.151:0/1018510428 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5638ec9c72d0 0x5638ec9cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:46.635 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.630+0000 7f7181a00640 1 -- 172.21.15.151:0/1018510428 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5638ec9c72d0 msgr2=0x5638ec9cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:46.635 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.630+0000 7f7181a00640 1 --2- 172.21.15.151:0/1018510428 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5638ec9c72d0 0x5638ec9cf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:46.635 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.630+0000 7f7181a00640 1 -- 172.21.15.151:0/1018510428 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5638ec9c69b0 msgr2=0x5638ec9ceb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:46.635 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.630+0000 7f7181a00640 1 --2- 172.21.15.151:0/1018510428 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5638ec9c69b0 0x5638ec9ceb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:46.635 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.630+0000 7f7181a00640 1 -- 172.21.15.151:0/1018510428 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638ec70b620 con 0x5638ec9d1b90 2024-04-26T01:56:46.635 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.630+0000 7f7173e00640 1 --2- 172.21.15.151:0/1018510428 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5638ec9c72d0 0x5638ec9cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:56:46.635 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.630+0000 7f7181a00640 1 --2- 172.21.15.151:0/1018510428 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638ec9d1b90 0x5638ec9cf5b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f71740996c0 tx=0x7f717409c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a249b740a220a78 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:46.635 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.630+0000 7f716aa00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T01:56:46.635 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.631+0000 7f716aa00640 1 -- 172.21.15.151:0/1018510428 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f717409d150 con 0x5638ec9d1b90 2024-04-26T01:56:46.635 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.631+0000 7f716aa00640 1 -- 172.21.15.151:0/1018510428 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f71740a54d0 con 0x5638ec9d1b90 2024-04-26T01:56:46.636 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.631+0000 7f71867b8740 1 -- 172.21.15.151:0/1018510428 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f716c0054d0 con 0x5638ec9d1b90 2024-04-26T01:56:46.636 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.631+0000 7f716aa00640 1 -- 172.21.15.151:0/1018510428 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f71740a10a0 con 0x5638ec9d1b90 2024-04-26T01:56:46.636 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.631+0000 7f71867b8740 10 client.25517 fetch_fsmap learned FSMap version 28 2024-04-26T01:56:46.636 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.631+0000 7f71867b8740 1 -- 172.21.15.151:0/1018510428 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5638ec9e0a80 con 0x5638ec9d1b90 2024-04-26T01:56:46.636 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.631+0000 7f716aa00640 1 -- 172.21.15.151:0/1018510428 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f716c0054d0 con 0x5638ec9d1b90 2024-04-26T01:56:46.636 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.631+0000 7f716aa00640 1 -- 172.21.15.151:0/1018510428 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f71740bd070 con 0x5638ec9d1b90 2024-04-26T01:56:46.636 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.631+0000 7f71867b8740 10 client.25517 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T01:56:46.636 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.631+0000 7f71867b8740 10 client.25517 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:56:46.636 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.632+0000 7f71867b8740 20 client.25517 populate_metadata read hostname 'smithi151' 2024-04-26T01:56:46.636 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.632+0000 7f71867b8740 1 --2- 172.21.15.151:0/1018510428 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5638ec9e1f40 0x5638ec9e4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:46.636 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.632+0000 7f71867b8740 4 client.25517 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:56:46.636 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.632+0000 7f71867b8740 1 -- 172.21.15.151:0/1018510428 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638ec9408b0 con 0x5638ec9e1f40 2024-04-26T01:56:46.636 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.632+0000 7f7181000640 1 --2- 172.21.15.151:0/1018510428 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5638ec9e1f40 0x5638ec9e4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:46.636 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.632+0000 7f7181000640 1 --2- 172.21.15.151:0/1018510428 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5638ec9e1f40 0x5638ec9e4350 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8a989e7a2965fac server_cookie=2f4e76f5d38ef8ef in_seq=0 out_seq=0 2024-04-26T01:56:46.637 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.632+0000 7f716aa00640 10 client.25517 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:56:46.637 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.633+0000 7f716aa00640 1 -- 172.21.15.151:0/1018510428 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5638ec9408b0 con 0x5638ec9e1f40 2024-04-26T01:56:46.637 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.633+0000 7f716aa00640 10 client.25517 handle_command_reply: tid=0 2024-04-26T01:56:46.637 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.633+0000 7f71867b8740 1 client.25517 shutdown 2024-04-26T01:56:46.637 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.633+0000 7f71867b8740 1 -- 172.21.15.151:0/1018510428 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638ec9d1b90 msgr2=0x5638ec9cf5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:46.637 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.634+0000 7f71867b8740 1 --2- 172.21.15.151:0/1018510428 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638ec9d1b90 0x5638ec9cf5b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f71740996c0 tx=0x7f717409c650 comp rx=0 tx=0).stop 2024-04-26T01:56:46.637 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.634+0000 7f71867b8740 1 -- 172.21.15.151:0/1018510428 shutdown_connections 2024-04-26T01:56:46.637 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.634+0000 7f71867b8740 1 --2- 172.21.15.151:0/1018510428 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5638ec9e1f40 0x5638ec9e4350 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:46.637 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.634+0000 7f71867b8740 1 --2- 172.21.15.151:0/1018510428 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638ec9d1b90 0x5638ec9cf5b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:46.637 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.634+0000 7f71867b8740 1 --2- 172.21.15.151:0/1018510428 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5638ec9c72d0 0x5638ec9cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:46.637 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.634+0000 7f716aa00640 0 client.25517 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:56:46.637 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.634+0000 7f71867b8740 1 --2- 172.21.15.151:0/1018510428 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5638ec9c69b0 0x5638ec9ceb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:46.637 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.634+0000 7f71867b8740 1 -- 172.21.15.151:0/1018510428 >> 172.21.15.151:0/1018510428 conn(0x5638ec9c2ec0 msgr2=0x5638ec9c8c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:46.638 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.634+0000 7f71867b8740 1 -- 172.21.15.151:0/1018510428 shutdown_connections 2024-04-26T01:56:46.638 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.634+0000 7f71867b8740 1 -- 172.21.15.151:0/1018510428 wait complete. 2024-04-26T01:56:46.638 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.634+0000 7f71867b8740 20 client.25517 trim_cache size 0 max 16384 2024-04-26T01:56:46.650 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.650+0000 7f71867b8740 1 Processor -- start 2024-04-26T01:56:46.650 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.650+0000 7f71867b8740 1 -- start start 2024-04-26T01:56:46.650 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.650+0000 7f71867b8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5638ec9c69b0 0x5638ec9dd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:46.650 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.650+0000 7f71867b8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638ec9c72d0 0x5638ec9c9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:46.650 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.650+0000 7f71867b8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5638ec9d1b90 0x5638ec9c1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:46.650 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.650+0000 7f71867b8740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f717c1b6680 con 0x5638ec9c69b0 2024-04-26T01:56:46.650 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.650+0000 7f71867b8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f717c1b3f50 con 0x5638ec9c72d0 2024-04-26T01:56:46.650 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.650+0000 7f71867b8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f717c1b1da0 con 0x5638ec9d1b90 2024-04-26T01:56:46.650 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.650+0000 7f7181000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5638ec9c69b0 0x5638ec9dd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:46.650 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.650+0000 7f7181000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5638ec9c69b0 0x5638ec9dd680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:46058/0 (socket says 172.21.15.151:46058) 2024-04-26T01:56:46.650 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.651+0000 7f7181000640 1 -- 172.21.15.151:0/1164590610 learned_addr learned my addr 172.21.15.151:0/1164590610 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:56:46.651 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.651+0000 7f7173e00640 1 --2- 172.21.15.151:0/1164590610 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638ec9c72d0 0x5638ec9c9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:46.651 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.651+0000 7f7181a00640 1 --2- 172.21.15.151:0/1164590610 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5638ec9d1b90 0x5638ec9c1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:46.651 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.651+0000 7f7181000640 1 -- 172.21.15.151:0/1164590610 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5638ec9d1b90 msgr2=0x5638ec9c1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:46.651 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.651+0000 7f7181000640 1 --2- 172.21.15.151:0/1164590610 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5638ec9d1b90 0x5638ec9c1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:46.651 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.651+0000 7f7181000640 1 -- 172.21.15.151:0/1164590610 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638ec9c72d0 msgr2=0x5638ec9c9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:46.651 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.651+0000 7f7181000640 1 --2- 172.21.15.151:0/1164590610 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638ec9c72d0 0x5638ec9c9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:46.651 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.651+0000 7f7181000640 1 -- 172.21.15.151:0/1164590610 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638ec9c7c60 con 0x5638ec9c69b0 2024-04-26T01:56:46.651 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.651+0000 7f7173e00640 1 --2- 172.21.15.151:0/1164590610 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638ec9c72d0 0x5638ec9c9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:56:46.651 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.651+0000 7f7181a00640 1 --2- 172.21.15.151:0/1164590610 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5638ec9d1b90 0x5638ec9c1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:56:46.651 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.651+0000 7f7181000640 1 --2- 172.21.15.151:0/1164590610 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5638ec9c69b0 0x5638ec9dd680 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f716c009b40 tx=0x7f716c009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7ddb218d83bb424 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:46.655 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.652+0000 7f7171600640 1 -- 172.21.15.151:0/1164590610 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716c02edd0 con 0x5638ec9c69b0 2024-04-26T01:56:46.655 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.652+0000 7f7171600640 1 -- 172.21.15.151:0/1164590610 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f716c004500 con 0x5638ec9c69b0 2024-04-26T01:56:46.655 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.652+0000 7f7171600640 1 -- 172.21.15.151:0/1164590610 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716c0035f0 con 0x5638ec9c69b0 2024-04-26T01:56:46.656 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.652+0000 7f71867b8740 1 -- 172.21.15.151:0/1164590610 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5638ec9c69b0 msgr2=0x5638ec9dd680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:46.656 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.652+0000 7f71867b8740 1 --2- 172.21.15.151:0/1164590610 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5638ec9c69b0 0x5638ec9dd680 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f716c009b40 tx=0x7f716c009850 comp rx=0 tx=0).stop 2024-04-26T01:56:46.656 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.652+0000 7f71867b8740 1 -- 172.21.15.151:0/1164590610 shutdown_connections 2024-04-26T01:56:46.656 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.652+0000 7f71867b8740 1 --2- 172.21.15.151:0/1164590610 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5638ec9d1b90 0x5638ec9c1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:46.656 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.652+0000 7f71867b8740 1 --2- 172.21.15.151:0/1164590610 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638ec9c72d0 0x5638ec9c9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:46.656 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.652+0000 7f71867b8740 1 --2- 172.21.15.151:0/1164590610 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5638ec9c69b0 0x5638ec9dd680 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:46.656 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.652+0000 7f71867b8740 1 -- 172.21.15.151:0/1164590610 >> 172.21.15.151:0/1164590610 conn(0x5638ec9c2ec0 msgr2=0x5638ec9c1790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:46.656 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.652+0000 7f71867b8740 1 -- 172.21.15.151:0/1164590610 shutdown_connections 2024-04-26T01:56:46.656 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.652+0000 7f71867b8740 1 -- 172.21.15.151:0/1164590610 wait complete. 2024-04-26T01:56:46.656 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.653+0000 7f71867b8740 1 -- start start 2024-04-26T01:56:46.656 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.653+0000 7f71867b8740 1 Processor -- start 2024-04-26T01:56:46.656 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.653+0000 7f71867b8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5638ec9c69b0 0x5638ec9d6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:46.656 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.653+0000 7f71867b8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5638ec9c72d0 0x5638ec9d70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:46.657 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.654+0000 7f71867b8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638ec9d1b90 0x5638eca464d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:46.657 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.654+0000 7f71867b8740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f717c1b2410 con 0x5638ec9c72d0 2024-04-26T01:56:46.657 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.654+0000 7f71867b8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f717c1b3880 con 0x5638ec9d1b90 2024-04-26T01:56:46.657 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.654+0000 7f71867b8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f717c1b2ae0 con 0x5638ec9c69b0 2024-04-26T01:56:46.657 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.654+0000 7f71867b8740 10 client.0 _ll_register_callbacks cb 0x7f7174099020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:56:46.657 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.654+0000 7f7173e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5638ec9c72d0 0x5638ec9d70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:46.657 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.654+0000 7f7173e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5638ec9c72d0 0x5638ec9d70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:46068/0 (socket says 172.21.15.151:46068) 2024-04-26T01:56:46.657 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.654+0000 7f7173e00640 1 -- 172.21.15.151:0/1777137894 learned_addr learned my addr 172.21.15.151:0/1777137894 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:56:46.657 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.654+0000 7f7181000640 1 --2- 172.21.15.151:0/1777137894 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5638ec9c69b0 0x5638ec9d6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:46.657 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.654+0000 7f7181a00640 1 --2- 172.21.15.151:0/1777137894 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638ec9d1b90 0x5638eca464d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:46.657 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.654+0000 7f7173e00640 1 -- 172.21.15.151:0/1777137894 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5638ec9c69b0 msgr2=0x5638ec9d6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:46.658 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.654+0000 7f7173e00640 1 --2- 172.21.15.151:0/1777137894 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5638ec9c69b0 0x5638ec9d6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:46.658 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.654+0000 7f7173e00640 1 -- 172.21.15.151:0/1777137894 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638ec9d1b90 msgr2=0x5638eca464d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:46.658 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.654+0000 7f7173e00640 1 --2- 172.21.15.151:0/1777137894 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638ec9d1b90 0x5638eca464d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:46.658 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.654+0000 7f7173e00640 1 -- 172.21.15.151:0/1777137894 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638eca46a10 con 0x5638ec9c72d0 2024-04-26T01:56:46.658 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.654+0000 7f7181a00640 1 --2- 172.21.15.151:0/1777137894 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638ec9d1b90 0x5638eca464d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:56:46.658 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.655+0000 7f7173e00640 1 --2- 172.21.15.151:0/1777137894 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5638ec9c72d0 0x5638ec9d70a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7164009930 tx=0x7f7164000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65f49df4258b355b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:46.658 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.655+0000 7f716aa00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-04-26T01:56:46.658 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.655+0000 7f716aa00640 1 -- 172.21.15.151:0/1777137894 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7164035040 con 0x5638ec9c72d0 2024-04-26T01:56:46.658 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.655+0000 7f71867b8740 1 -- 172.21.15.151:0/1777137894 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5638ec93a320 con 0x5638ec9c72d0 2024-04-26T01:56:46.658 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.655+0000 7f716aa00640 1 -- 172.21.15.151:0/1777137894 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f71640037f0 con 0x5638ec9c72d0 2024-04-26T01:56:46.658 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.655+0000 7f716aa00640 1 -- 172.21.15.151:0/1777137894 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f71640039f0 con 0x5638ec9c72d0 2024-04-26T01:56:46.659 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.655+0000 7f71867b8740 10 client.15429 fetch_fsmap learned FSMap version 28 2024-04-26T01:56:46.659 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.655+0000 7f71867b8740 1 -- 172.21.15.151:0/1777137894 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5638eca47bf0 con 0x5638ec9c72d0 2024-04-26T01:56:46.659 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.656+0000 7f716aa00640 1 -- 172.21.15.151:0/1777137894 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5638ec93a320 con 0x5638ec9c72d0 2024-04-26T01:56:46.659 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.656+0000 7f716aa00640 1 -- 172.21.15.151:0/1777137894 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f716405c020 con 0x5638ec9c72d0 2024-04-26T01:56:46.659 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.656+0000 7f71867b8740 10 client.15429 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T01:56:46.659 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.656+0000 7f71867b8740 10 client.15429 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:56:46.659 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.656+0000 7f71867b8740 20 client.15429 populate_metadata read hostname 'smithi151' 2024-04-26T01:56:46.659 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.656+0000 7f71867b8740 1 --2- 172.21.15.151:0/1777137894 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5638eca48400 0x5638eca4a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:46.659 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.656+0000 7f71867b8740 4 client.15429 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:56:46.659 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.656+0000 7f71867b8740 1 -- 172.21.15.151:0/1777137894 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5638ec934190 con 0x5638eca48400 2024-04-26T01:56:46.659 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.656+0000 7f7181000640 1 --2- 172.21.15.151:0/1777137894 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5638eca48400 0x5638eca4a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:46.659 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.657+0000 7f7181000640 1 --2- 172.21.15.151:0/1777137894 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5638eca48400 0x5638eca4a820 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92d8c1bd5dce5b73 server_cookie=575e097a6009cd20 in_seq=0 out_seq=0 2024-04-26T01:56:46.659 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.657+0000 7f716aa00640 10 client.15429 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:56:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.677+0000 7f716aa00640 1 -- 172.21.15.151:0/1777137894 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5638ec934190 con 0x5638eca48400 2024-04-26T01:56:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.677+0000 7f716aa00640 10 client.15429 handle_command_reply: tid=0 2024-04-26T01:56:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.677+0000 7f71867b8740 1 client.15429 shutdown 2024-04-26T01:56:46.677 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.678+0000 7f71867b8740 1 -- 172.21.15.151:0/1777137894 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5638ec9c72d0 msgr2=0x5638ec9d70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:46.677 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.678+0000 7f71867b8740 1 --2- 172.21.15.151:0/1777137894 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5638ec9c72d0 0x5638ec9d70a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7164009930 tx=0x7f7164000bc0 comp rx=0 tx=0).stop 2024-04-26T01:56:46.677 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.678+0000 7f71867b8740 1 -- 172.21.15.151:0/1777137894 shutdown_connections 2024-04-26T01:56:46.677 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.678+0000 7f71867b8740 1 --2- 172.21.15.151:0/1777137894 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5638eca48400 0x5638eca4a820 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:46.677 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.678+0000 7f71867b8740 1 --2- 172.21.15.151:0/1777137894 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638ec9d1b90 0x5638eca464d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:46.677 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.678+0000 7f71867b8740 1 --2- 172.21.15.151:0/1777137894 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5638ec9c72d0 0x5638ec9d70a0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:46.677 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.678+0000 7f71867b8740 1 --2- 172.21.15.151:0/1777137894 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5638ec9c69b0 0x5638ec9d6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:46.677 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.678+0000 7f716aa00640 0 client.15429 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:56:46.677 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.678+0000 7f71867b8740 1 -- 172.21.15.151:0/1777137894 >> 172.21.15.151:0/1777137894 conn(0x5638ec9c2ec0 msgr2=0x5638ec9c2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:46.678 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.678+0000 7f71867b8740 1 -- 172.21.15.151:0/1777137894 shutdown_connections 2024-04-26T01:56:46.678 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.678+0000 7f71867b8740 1 -- 172.21.15.151:0/1777137894 wait complete. 2024-04-26T01:56:46.678 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.678+0000 7f71867b8740 20 client.15429 trim_cache size 0 max 16384 2024-04-26T01:56:46.679 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:{ 2024-04-26T01:56:46.679 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "message": "", 2024-04-26T01:56:46.679 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "return_code": 0 2024-04-26T01:56:46.679 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:} 2024-04-26T01:56:46.680 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.679+0000 7f7182400640 1 -- 172.21.15.151:0/444136238 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f71600a3d10 msgr2=0x7f71600a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:46.680 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.679+0000 7f7182400640 1 --2- 172.21.15.151:0/444136238 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f71600a3d10 0x7f71600a61d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x5638ec9c42e0 tx=0x7f7164005fd0 comp rx=0 tx=0).stop 2024-04-26T01:56:46.680 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.679+0000 7f7182400640 1 -- 172.21.15.151:0/444136238 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f717c10e420 msgr2=0x7f717c19e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:46.680 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.679+0000 7f7182400640 1 --2- 172.21.15.151:0/444136238 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f717c10e420 0x7f717c19e970 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f717400eb10 tx=0x7f717400c2d0 comp rx=0 tx=0).stop 2024-04-26T01:56:46.680 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.679+0000 7f7182400640 1 -- 172.21.15.151:0/444136238 shutdown_connections 2024-04-26T01:56:46.680 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.679+0000 7f7182400640 1 --2- 172.21.15.151:0/444136238 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f71600a3d10 0x7f71600a61d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:46.680 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.679+0000 7f7182400640 1 --2- 172.21.15.151:0/444136238 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f717c10e420 0x7f717c19e970 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:46.680 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.679+0000 7f7182400640 1 --2- 172.21.15.151:0/444136238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f717c108e10 0x7f717c1a4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:46.680 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.679+0000 7f7182400640 1 --2- 172.21.15.151:0/444136238 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f717c108490 0x7f717c1a47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:46.680 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.679+0000 7f7182400640 1 -- 172.21.15.151:0/444136238 >> 172.21.15.151:0/444136238 conn(0x7f717c077d30 msgr2=0x7f717c078f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:46.680 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.680+0000 7f7182400640 1 -- 172.21.15.151:0/444136238 shutdown_connections 2024-04-26T01:56:46.681 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:46.680+0000 7f7182400640 1 -- 172.21.15.151:0/444136238 wait complete. 2024-04-26T01:56:47.531 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:56:47.990 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:56:48.451 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:56:48 smithi151 ceph-mon[28057]: pgmap v98: 97 pgs: 97 active+clean; 31 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 174 KiB/s wr, 16 op/s 2024-04-26T01:56:48.451 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:56:48 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:56:48.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:56:48 smithi172 ceph-mon[32785]: pgmap v98: 97 pgs: 97 active+clean; 31 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 174 KiB/s wr, 16 op/s 2024-04-26T01:56:48.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:56:48 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:56:48.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:56:48 smithi165 ceph-mon[32648]: pgmap v98: 97 pgs: 97 active+clean; 31 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 174 KiB/s wr, 16 op/s 2024-04-26T01:56:48.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:56:48 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:56:50.349 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:56:50 smithi151 ceph-mon[28057]: pgmap v99: 97 pgs: 97 active+clean; 31 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 185 KiB/s wr, 25 op/s 2024-04-26T01:56:50.349 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:56:50 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:56:50.349 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:56:50 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:56:50.349 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:56:50 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:56:50.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:50.425+0000 7fdbc3e00700 1 -- 172.21.15.151:0/767356309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbbc103450 msgr2=0x7fdbbc10f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:50.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:50.425+0000 7fdbc3e00700 1 --2- 172.21.15.151:0/767356309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbbc103450 0x7fdbbc10f8b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fdbb4009990 tx=0x7fdbb4009ca0 comp rx=0 tx=0).stop 2024-04-26T01:56:50.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:50.425+0000 7fdbc3e00700 1 -- 172.21.15.151:0/767356309 shutdown_connections 2024-04-26T01:56:50.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:50.425+0000 7fdbc3e00700 1 --2- 172.21.15.151:0/767356309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbbc103450 0x7fdbbc10f8b0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:50.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:50.425+0000 7fdbc3e00700 1 --2- 172.21.15.151:0/767356309 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdbbc108a50 0x7fdbbc108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:50.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:50.425+0000 7fdbc3e00700 1 --2- 172.21.15.151:0/767356309 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdbbc102a90 0x7fdbbc102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:50.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:50.425+0000 7fdbc3e00700 1 -- 172.21.15.151:0/767356309 >> 172.21.15.151:0/767356309 conn(0x7fdbbc079ba0 msgr2=0x7fdbbc079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:50.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:50.425+0000 7fdbc3e00700 1 -- 172.21.15.151:0/767356309 shutdown_connections 2024-04-26T01:56:50.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:50.426+0000 7fdbc3e00700 1 -- 172.21.15.151:0/767356309 wait complete. 2024-04-26T01:56:50.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:50.427+0000 7fdbc3e00700 1 Processor -- start 2024-04-26T01:56:50.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:50.428+0000 7fdbc3e00700 1 -- start start 2024-04-26T01:56:50.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:50.428+0000 7fdbc3e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbbc102a90 0x7fdbbc19b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:50.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:50.428+0000 7fdbc3e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdbbc103450 0x7fdbbc19b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:50.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:50.428+0000 7fdbc3e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdbbc108a50 0x7fdbbc19fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:50.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:50.428+0000 7fdbc3e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdbbc112c50 con 0x7fdbbc108a50 2024-04-26T01:56:50.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:50.428+0000 7fdbc3e00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fdbbc112ad0 con 0x7fdbbc103450 2024-04-26T01:56:50.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:50.428+0000 7fdbc3e00700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdbbc112dd0 con 0x7fdbbc102a90 2024-04-26T01:56:50.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:50.429+0000 7fdbc0a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdbbc103450 0x7fdbbc19b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:50.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:50.429+0000 7fdbc1400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbbc102a90 0x7fdbbc19b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:50.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:50.429+0000 7fdbc0a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdbbc103450 0x7fdbbc19b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:46810/0 (socket says 172.21.15.151:46810) 2024-04-26T01:56:50.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:50.429+0000 7fdbc0a00700 1 -- 172.21.15.151:0/3937455312 learned_addr learned my addr 172.21.15.151:0/3937455312 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:56:50.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:50.429+0000 7fdbc1e00700 1 --2- 172.21.15.151:0/3937455312 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdbbc108a50 0x7fdbbc19fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:50.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:50.429+0000 7fdbc0a00700 1 -- 172.21.15.151:0/3937455312 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbbc102a90 msgr2=0x7fdbbc19b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:50.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:50.429+0000 7fdbc0a00700 1 --2- 172.21.15.151:0/3937455312 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbbc102a90 0x7fdbbc19b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:50.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:50.429+0000 7fdbc0a00700 1 -- 172.21.15.151:0/3937455312 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdbbc108a50 msgr2=0x7fdbbc19fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:50.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:50.429+0000 7fdbc0a00700 1 --2- 172.21.15.151:0/3937455312 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdbbc108a50 0x7fdbbc19fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:50.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:50.429+0000 7fdbc0a00700 1 -- 172.21.15.151:0/3937455312 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbbc1a0330 con 0x7fdbbc103450 2024-04-26T01:56:50.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:50.430+0000 7fdbc0a00700 1 --2- 172.21.15.151:0/3937455312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdbbc103450 0x7fdbbc19b860 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdba800e9c0 tx=0x7fdba800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:50.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:50.430+0000 7fdbba000700 1 -- 172.21.15.151:0/3937455312 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba800cbd0 con 0x7fdbbc103450 2024-04-26T01:56:50.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:50.430+0000 7fdbba000700 1 -- 172.21.15.151:0/3937455312 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdba800ee60 con 0x7fdbbc103450 2024-04-26T01:56:50.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:50.430+0000 7fdbba000700 1 -- 172.21.15.151:0/3937455312 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba80055c0 con 0x7fdbbc103450 2024-04-26T01:56:50.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:50.430+0000 7fdbc3e00700 1 -- 172.21.15.151:0/3937455312 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbbc1a0620 con 0x7fdbbc103450 2024-04-26T01:56:50.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:50.430+0000 7fdbc3e00700 1 -- 172.21.15.151:0/3937455312 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbbc1a7ea0 con 0x7fdbbc103450 2024-04-26T01:56:50.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:50.432+0000 7fdbc3e00700 1 -- 172.21.15.151:0/3937455312 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbbc069730 con 0x7fdbbc103450 2024-04-26T01:56:50.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:50.433+0000 7fdbba000700 1 -- 172.21.15.151:0/3937455312 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7fdba8004750 con 0x7fdbbc103450 2024-04-26T01:56:50.434 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:50.435+0000 7fdbba000700 1 --2- 172.21.15.151:0/3937455312 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fdba40a3e30 0x7fdba40a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:50.434 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:50.435+0000 7fdbba000700 1 -- 172.21.15.151:0/3937455312 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fdba80d09e0 con 0x7fdbbc103450 2024-04-26T01:56:50.434 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:50.435+0000 7fdbc1400700 1 --2- 172.21.15.151:0/3937455312 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fdba40a3e30 0x7fdba40a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:50.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:50.436+0000 7fdbc1400700 1 --2- 172.21.15.151:0/3937455312 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fdba40a3e30 0x7fdba40a62f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdbb000f540 tx=0x7fdbb000b040 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:50.437 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:50.439+0000 7fdbba000700 1 -- 172.21.15.151:0/3937455312 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdba809d090 con 0x7fdbbc103450 2024-04-26T01:56:50.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:56:50 smithi172 ceph-mon[32785]: pgmap v99: 97 pgs: 97 active+clean; 31 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 185 KiB/s wr, 25 op/s 2024-04-26T01:56:50.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:56:50 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:56:50.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:56:50 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:56:50.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:56:50 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:56:50.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:56:50 smithi165 ceph-mon[32648]: pgmap v99: 97 pgs: 97 active+clean; 31 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 185 KiB/s wr, 25 op/s 2024-04-26T01:56:50.598 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:56:50 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:56:50.598 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:56:50 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:56:50.598 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:56:50 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:56:50.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:50.719+0000 7fdbc3e00700 1 -- 172.21.15.151:0/3937455312 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdbbc1a8070 con 0x7fdbbc103450 2024-04-26T01:56:50.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:50.720+0000 7fdbba000700 1 -- 172.21.15.151:0/3937455312 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11892 (secure 0 0 0) 0x7fdba809c990 con 0x7fdbbc103450 2024-04-26T01:56:50.720 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:56:50.720 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2937864097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2937864097},{"type":"v1","addr":"172.21.15.151:6835","nonce":2937864097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14373,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/2947090802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2947090802},{"type":"v1","addr":"172.21.15.151:6837","nonce":2947090802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/1356563280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1356563280},{"type":"v1","addr":"172.21.15.151:6839","nonce":1356563280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6841/1075240552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1075240552},{"type":"v1","addr":"172.21.15.151:6841","nonce":1075240552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/504675912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":504675912},{"type":"v1","addr":"172.21.15.165:6833","nonce":504675912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1207580015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1207580015},{"type":"v1","addr":"172.21.15.172:6833","nonce":1207580015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:49:40.746876+0000","modified":"2024-04-26T01:55:28.126804+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24505,"mds_2":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/3598138294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3598138294},{"type":"v1","addr":"172.21.15.165:6835","nonce":3598138294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/1847859054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":1847859054},{"type":"v1","addr":"172.21.15.165:6837","nonce":1847859054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/2309751171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2309751171},{"type":"v1","addr":"172.21.15.172:6835","nonce":2309751171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.172:6837/55972537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":55972537},{"type":"v1","addr":"172.21.15.172:6837","nonce":55972537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.172:6839/1242302810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1242302810},{"type":"v1","addr":"172.21.15.172:6839","nonce":1242302810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/1969716254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1969716254},{"type":"v1","addr":"172.21.15.165:6839","nonce":1969716254}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24505,24509,24493]},"id":1}]} 2024-04-26T01:56:50.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:50.723+0000 7fdbc3e00700 1 -- 172.21.15.151:0/3937455312 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fdba40a3e30 msgr2=0x7fdba40a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:50.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:50.723+0000 7fdbc3e00700 1 --2- 172.21.15.151:0/3937455312 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fdba40a3e30 0x7fdba40a62f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdbb000f540 tx=0x7fdbb000b040 comp rx=0 tx=0).stop 2024-04-26T01:56:50.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:50.723+0000 7fdbc3e00700 1 -- 172.21.15.151:0/3937455312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdbbc103450 msgr2=0x7fdbbc19b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:50.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:50.723+0000 7fdbc3e00700 1 --2- 172.21.15.151:0/3937455312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdbbc103450 0x7fdbbc19b860 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdba800e9c0 tx=0x7fdba800ecd0 comp rx=0 tx=0).stop 2024-04-26T01:56:50.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:50.724+0000 7fdbc3e00700 1 -- 172.21.15.151:0/3937455312 shutdown_connections 2024-04-26T01:56:50.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:50.724+0000 7fdbc3e00700 1 --2- 172.21.15.151:0/3937455312 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fdba40a3e30 0x7fdba40a62f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:50.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:50.724+0000 7fdbc3e00700 1 --2- 172.21.15.151:0/3937455312 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdbbc108a50 0x7fdbbc19fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:50.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:50.724+0000 7fdbc3e00700 1 --2- 172.21.15.151:0/3937455312 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbbc102a90 0x7fdbbc19b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:50.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:50.724+0000 7fdbc3e00700 1 --2- 172.21.15.151:0/3937455312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdbbc103450 0x7fdbbc19b860 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:50.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:50.724+0000 7fdbc3e00700 1 -- 172.21.15.151:0/3937455312 >> 172.21.15.151:0/3937455312 conn(0x7fdbbc079ba0 msgr2=0x7fdbbc10df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:50.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:50.724+0000 7fdbc3e00700 1 -- 172.21.15.151:0/3937455312 shutdown_connections 2024-04-26T01:56:50.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:50.724+0000 7fdbc3e00700 1 -- 172.21.15.151:0/3937455312 wait complete. 2024-04-26T01:56:50.724 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 28 2024-04-26T01:56:51.504 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:56:51 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/3937455312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:56:51.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:56:51 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/3937455312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:56:51.571 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24509 quiesce db --set-id bc374c20 --release --await' 2024-04-26T01:56:51.571 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24509 quiesce db --set-id bc374c20 --release --await 2024-04-26T01:56:51.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:56:51 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/3937455312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:56:52.017 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:56:52.484 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:56:52 smithi151 ceph-mon[28057]: pgmap v100: 97 pgs: 97 active+clean; 31 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 24 KiB/s wr, 12 op/s 2024-04-26T01:56:52.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:56:52 smithi172 ceph-mon[32785]: pgmap v100: 97 pgs: 97 active+clean; 31 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 24 KiB/s wr, 12 op/s 2024-04-26T01:56:52.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:56:52 smithi165 ceph-mon[32648]: pgmap v100: 97 pgs: 97 active+clean; 31 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 24 KiB/s wr, 12 op/s 2024-04-26T01:56:54.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.335+0000 7f8a1ba00700 1 -- 172.21.15.151:0/348473586 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a141092d0 msgr2=0x7f8a141096b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:54.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.335+0000 7f8a1ba00700 1 --2- 172.21.15.151:0/348473586 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a141092d0 0x7f8a141096b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8a080099c0 tx=0x7f8a08009cd0 comp rx=0 tx=0).stop 2024-04-26T01:56:54.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.335+0000 7f8a1ba00700 1 -- 172.21.15.151:0/348473586 shutdown_connections 2024-04-26T01:56:54.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.335+0000 7f8a1ba00700 1 --2- 172.21.15.151:0/348473586 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a1410e3b0 0x7f8a14115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:54.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.335+0000 7f8a1ba00700 1 --2- 172.21.15.151:0/348473586 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a141092d0 0x7f8a141096b0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:54.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.335+0000 7f8a1ba00700 1 --2- 172.21.15.151:0/348473586 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a14109c80 0x7f8a1410dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:54.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.335+0000 7f8a1ba00700 1 -- 172.21.15.151:0/348473586 >> 172.21.15.151:0/348473586 conn(0x7f8a14076210 msgr2=0x7f8a14107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:54.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.335+0000 7f8a1ba00700 1 -- 172.21.15.151:0/348473586 shutdown_connections 2024-04-26T01:56:54.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.335+0000 7f8a1ba00700 1 -- 172.21.15.151:0/348473586 wait complete. 2024-04-26T01:56:54.336 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:56:54 smithi151 ceph-mon[28057]: pgmap v101: 97 pgs: 97 active+clean; 31 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 24 KiB/s wr, 13 op/s 2024-04-26T01:56:54.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.337+0000 7f8a1ba00700 1 Processor -- start 2024-04-26T01:56:54.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.337+0000 7f8a1ba00700 1 -- start start 2024-04-26T01:56:54.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.337+0000 7f8a1ba00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a141092d0 0x7f8a141a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:54.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.337+0000 7f8a1ba00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a14109c80 0x7f8a141a5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:54.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.337+0000 7f8a1ba00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a1410e3b0 0x7f8a1419f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:54.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.337+0000 7f8a1ba00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8a14119290 con 0x7f8a14109c80 2024-04-26T01:56:54.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.338+0000 7f8a1ba00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8a14119110 con 0x7f8a141092d0 2024-04-26T01:56:54.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.338+0000 7f8a1ba00700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8a14119410 con 0x7f8a1410e3b0 2024-04-26T01:56:54.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.338+0000 7f8a19000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a141092d0 0x7f8a141a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:54.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.338+0000 7f8a13e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a14109c80 0x7f8a141a5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:54.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.338+0000 7f8a19000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a141092d0 0x7f8a141a53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:46826/0 (socket says 172.21.15.151:46826) 2024-04-26T01:56:54.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.338+0000 7f8a19000700 1 -- 172.21.15.151:0/3061325396 learned_addr learned my addr 172.21.15.151:0/3061325396 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:56:54.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.338+0000 7f8a19a00700 1 --2- 172.21.15.151:0/3061325396 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a1410e3b0 0x7f8a1419f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:54.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.338+0000 7f8a19000700 1 -- 172.21.15.151:0/3061325396 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a1410e3b0 msgr2=0x7f8a1419f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:54.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.338+0000 7f8a19000700 1 --2- 172.21.15.151:0/3061325396 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a1410e3b0 0x7f8a1419f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:54.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.338+0000 7f8a19000700 1 -- 172.21.15.151:0/3061325396 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a14109c80 msgr2=0x7f8a141a5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:54.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.338+0000 7f8a19000700 1 --2- 172.21.15.151:0/3061325396 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a14109c80 0x7f8a141a5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:54.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.338+0000 7f8a19000700 1 -- 172.21.15.151:0/3061325396 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a1419fd10 con 0x7f8a141092d0 2024-04-26T01:56:54.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.338+0000 7f8a19a00700 1 --2- 172.21.15.151:0/3061325396 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a1410e3b0 0x7f8a1419f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:56:54.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.339+0000 7f8a19000700 1 --2- 172.21.15.151:0/3061325396 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a141092d0 0x7f8a141a53f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8a08005910 tx=0x7f8a08004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:54.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.339+0000 7f8a11600700 1 -- 172.21.15.151:0/3061325396 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a08026070 con 0x7f8a141092d0 2024-04-26T01:56:54.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.339+0000 7f8a11600700 1 -- 172.21.15.151:0/3061325396 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a08003c70 con 0x7f8a141092d0 2024-04-26T01:56:54.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.339+0000 7f8a1ba00700 1 -- 172.21.15.151:0/3061325396 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a1419ffa0 con 0x7f8a141092d0 2024-04-26T01:56:54.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.339+0000 7f8a11600700 1 -- 172.21.15.151:0/3061325396 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0800fb50 con 0x7f8a141092d0 2024-04-26T01:56:54.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.339+0000 7f8a1ba00700 1 -- 172.21.15.151:0/3061325396 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a141ac2c0 con 0x7f8a141092d0 2024-04-26T01:56:54.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.344+0000 7f8a11600700 1 -- 172.21.15.151:0/3061325396 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f8a08029070 con 0x7f8a141092d0 2024-04-26T01:56:54.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.345+0000 7f8a263edb80 1 Processor -- start 2024-04-26T01:56:54.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.345+0000 7f8a263edb80 1 -- start start 2024-04-26T01:56:54.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.345+0000 7f8a11600700 1 --2- 172.21.15.151:0/3061325396 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f89fc0a3db0 0x7f89fc0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:54.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.345+0000 7f8a11600700 1 -- 172.21.15.151:0/3061325396 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f8a080d0ff0 con 0x7f8a141092d0 2024-04-26T01:56:54.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.345+0000 7f8a263edb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6ae55bd40 0x55c6ae55c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:54.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.345+0000 7f8a263edb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6ae55c660 0x55c6ae566f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:54.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.346+0000 7f8a263edb80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6ae5674d0 0x55c6ae5698c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:54.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.346+0000 7f8a263edb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c6ae4aef90 con 0x55c6ae55c660 2024-04-26T01:56:54.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.346+0000 7f8a263edb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55c6ae455830 con 0x55c6ae55bd40 2024-04-26T01:56:54.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.346+0000 7f8a263edb80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c6ae4aaf50 con 0x55c6ae5674d0 2024-04-26T01:56:54.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.346+0000 7f8a19a00700 1 --2- 172.21.15.151:0/3061325396 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f89fc0a3db0 0x7f89fc0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:54.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.346+0000 7f8a13e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6ae55c660 0x55c6ae566f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:54.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.346+0000 7f8a13e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6ae55c660 0x55c6ae566f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37946/0 (socket says 172.21.15.151:37946) 2024-04-26T01:56:54.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.346+0000 7f8a13e00700 1 -- 172.21.15.151:0/3331257050 learned_addr learned my addr 172.21.15.151:0/3331257050 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:56:54.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.346+0000 7f8a19000700 1 --2- 172.21.15.151:0/3331257050 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6ae55bd40 0x55c6ae55c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:54.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.346+0000 7f8a19000700 1 -- 172.21.15.151:0/3331257050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6ae5674d0 msgr2=0x55c6ae5698c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:54.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.346+0000 7f8a19000700 1 --2- 172.21.15.151:0/3331257050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6ae5674d0 0x55c6ae5698c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:54.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.346+0000 7f8a19000700 1 -- 172.21.15.151:0/3331257050 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6ae55c660 msgr2=0x55c6ae566f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:54.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.346+0000 7f8a19000700 1 --2- 172.21.15.151:0/3331257050 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6ae55c660 0x55c6ae566f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:54.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.346+0000 7f8a19000700 1 -- 172.21.15.151:0/3331257050 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6ae44b780 con 0x55c6ae55bd40 2024-04-26T01:56:54.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.347+0000 7f8a19a00700 1 --2- 172.21.15.151:0/3061325396 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f89fc0a3db0 0x7f89fc0a6270 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c009fd0 tx=0x7f8a0c009920 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:54.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.347+0000 7f8a19000700 1 --2- 172.21.15.151:0/3331257050 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6ae55bd40 0x55c6ae55c120 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8a080ce770 tx=0x7f8a080036f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce759430d8d54d6e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:54.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.347+0000 7f8a06000700 1 -- 172.21.15.151:0/3331257050 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a08009e60 con 0x55c6ae55bd40 2024-04-26T01:56:54.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.347+0000 7f8a06000700 1 -- 172.21.15.151:0/3331257050 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a080bd040 con 0x55c6ae55bd40 2024-04-26T01:56:54.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.347+0000 7f8a06000700 1 -- 172.21.15.151:0/3331257050 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0809d2b0 con 0x55c6ae55bd40 2024-04-26T01:56:54.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.348+0000 7f8a263edb80 1 -- 172.21.15.151:0/3331257050 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6ae55bd40 msgr2=0x55c6ae55c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:54.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.348+0000 7f8a263edb80 1 --2- 172.21.15.151:0/3331257050 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6ae55bd40 0x55c6ae55c120 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8a080ce770 tx=0x7f8a080036f0 comp rx=0 tx=0).stop 2024-04-26T01:56:54.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.348+0000 7f8a263edb80 1 -- 172.21.15.151:0/3331257050 shutdown_connections 2024-04-26T01:56:54.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.348+0000 7f8a263edb80 1 --2- 172.21.15.151:0/3331257050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6ae5674d0 0x55c6ae5698c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:54.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.348+0000 7f8a263edb80 1 --2- 172.21.15.151:0/3331257050 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6ae55bd40 0x55c6ae55c120 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:54.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.348+0000 7f8a263edb80 1 --2- 172.21.15.151:0/3331257050 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6ae55c660 0x55c6ae566f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:54.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.348+0000 7f8a263edb80 1 -- 172.21.15.151:0/3331257050 >> 172.21.15.151:0/3331257050 conn(0x55c6ae48b050 msgr2=0x55c6ae558250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:54.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.348+0000 7f8a263edb80 1 -- 172.21.15.151:0/3331257050 shutdown_connections 2024-04-26T01:56:54.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.348+0000 7f8a263edb80 1 -- 172.21.15.151:0/3331257050 wait complete. 2024-04-26T01:56:54.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.349+0000 7f8a263edb80 1 -- start start 2024-04-26T01:56:54.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.349+0000 7f8a263edb80 1 Processor -- start 2024-04-26T01:56:54.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.349+0000 7f8a263edb80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6ae55bd40 0x55c6ae566c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:54.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.349+0000 7f8a263edb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6ae55c660 0x55c6ae563f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:54.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.349+0000 7f8a263edb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6ae5674d0 0x55c6ae564470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:54.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.349+0000 7f8a263edb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c6ae455830 con 0x55c6ae5674d0 2024-04-26T01:56:54.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.349+0000 7f8a263edb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55c6ae468960 con 0x55c6ae55c660 2024-04-26T01:56:54.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.349+0000 7f8a263edb80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c6ae569e00 con 0x55c6ae55bd40 2024-04-26T01:56:54.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.349+0000 7f8a263edb80 10 client.0 _ll_register_callbacks cb 0x55c6ae447820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:56:54.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.350+0000 7f8a19a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6ae5674d0 0x55c6ae564470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:54.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.350+0000 7f8a19000700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6ae55bd40 0x55c6ae566c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:54.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.350+0000 7f8a19a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6ae5674d0 0x55c6ae564470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37958/0 (socket says 172.21.15.151:37958) 2024-04-26T01:56:54.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.350+0000 7f8a19a00700 1 -- 172.21.15.151:0/527520951 learned_addr learned my addr 172.21.15.151:0/527520951 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:56:54.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.350+0000 7f8a13e00700 1 --2- 172.21.15.151:0/527520951 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6ae55c660 0x55c6ae563f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:54.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.350+0000 7f8a19000700 1 -- 172.21.15.151:0/527520951 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6ae55c660 msgr2=0x55c6ae563f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:54.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.350+0000 7f8a19000700 1 --2- 172.21.15.151:0/527520951 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6ae55c660 0x55c6ae563f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:54.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.350+0000 7f8a19000700 1 -- 172.21.15.151:0/527520951 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6ae5674d0 msgr2=0x55c6ae564470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:54.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.350+0000 7f8a19000700 1 --2- 172.21.15.151:0/527520951 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6ae5674d0 0x55c6ae564470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:54.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.350+0000 7f8a19000700 1 -- 172.21.15.151:0/527520951 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6ae3fc9d0 con 0x55c6ae55bd40 2024-04-26T01:56:54.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.350+0000 7f8a13e00700 1 --2- 172.21.15.151:0/527520951 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6ae55c660 0x55c6ae563f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:56:54.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.350+0000 7f8a19000700 1 --2- 172.21.15.151:0/527520951 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6ae55bd40 0x55c6ae566c30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8a0809d210 tx=0x7f8a080a5c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a8a8d005234c2fe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:54.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.350+0000 7f89faa00700 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-26T01:56:54.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.351+0000 7f89faa00700 1 -- 172.21.15.151:0/527520951 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a080ba060 con 0x55c6ae55bd40 2024-04-26T01:56:54.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.351+0000 7f89faa00700 1 -- 172.21.15.151:0/527520951 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f8a0809d4b0 con 0x55c6ae55bd40 2024-04-26T01:56:54.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.351+0000 7f8a263edb80 1 -- 172.21.15.151:0/527520951 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a080bd040 con 0x55c6ae55bd40 2024-04-26T01:56:54.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.351+0000 7f89faa00700 1 -- 172.21.15.151:0/527520951 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8a08009e60 con 0x55c6ae55bd40 2024-04-26T01:56:54.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.351+0000 7f8a263edb80 10 client.25573 fetch_fsmap learned FSMap version 28 2024-04-26T01:56:54.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.351+0000 7f8a263edb80 1 -- 172.21.15.151:0/527520951 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c6ae445bf0 con 0x55c6ae55bd40 2024-04-26T01:56:54.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.351+0000 7f89faa00700 1 -- 172.21.15.151:0/527520951 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a080bd040 con 0x55c6ae55bd40 2024-04-26T01:56:54.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.351+0000 7f89faa00700 1 -- 172.21.15.151:0/527520951 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f8a0809c530 con 0x55c6ae55bd40 2024-04-26T01:56:54.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.351+0000 7f8a263edb80 10 client.25573 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T01:56:54.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.351+0000 7f8a263edb80 10 client.25573 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-26T01:56:54.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.352+0000 7f8a263edb80 20 client.25573 populate_metadata read hostname 'smithi151' 2024-04-26T01:56:54.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.352+0000 7f8a263edb80 1 --2- 172.21.15.151:0/527520951 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55c6ae576810 0x55c6ae578c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:54.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.352+0000 7f8a263edb80 4 client.25573 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:56:54.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.352+0000 7f8a263edb80 1 -- 172.21.15.151:0/527520951 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6ae4bc780 con 0x55c6ae576810 2024-04-26T01:56:54.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.352+0000 7f8a13e00700 1 --2- 172.21.15.151:0/527520951 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55c6ae576810 0x55c6ae578c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:54.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.353+0000 7f8a13e00700 1 --2- 172.21.15.151:0/527520951 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55c6ae576810 0x55c6ae578c00 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8625f77e34d6f22 server_cookie=7719160ec38bcfbf in_seq=0 out_seq=0 2024-04-26T01:56:54.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.353+0000 7f89faa00700 10 client.25573 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:56:54.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.353+0000 7f89faa00700 1 -- 172.21.15.151:0/527520951 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c6ae4bc780 con 0x55c6ae576810 2024-04-26T01:56:54.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.353+0000 7f89faa00700 10 client.25573 handle_command_reply: tid=0 2024-04-26T01:56:54.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.353+0000 7f8a263edb80 1 client.25573 shutdown 2024-04-26T01:56:54.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.354+0000 7f8a263edb80 1 -- 172.21.15.151:0/527520951 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6ae55bd40 msgr2=0x55c6ae566c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:54.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.354+0000 7f8a263edb80 1 --2- 172.21.15.151:0/527520951 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6ae55bd40 0x55c6ae566c30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8a0809d210 tx=0x7f8a080a5c30 comp rx=0 tx=0).stop 2024-04-26T01:56:54.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.354+0000 7f8a263edb80 1 -- 172.21.15.151:0/527520951 shutdown_connections 2024-04-26T01:56:54.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.354+0000 7f8a263edb80 1 --2- 172.21.15.151:0/527520951 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6ae5674d0 0x55c6ae564470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:54.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.354+0000 7f8a263edb80 1 --2- 172.21.15.151:0/527520951 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6ae55bd40 0x55c6ae566c30 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:54.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.354+0000 7f8a263edb80 1 --2- 172.21.15.151:0/527520951 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55c6ae576810 0x55c6ae578c00 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:54.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.354+0000 7f8a263edb80 1 --2- 172.21.15.151:0/527520951 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6ae55c660 0x55c6ae563f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:54.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.354+0000 7f8a263edb80 1 -- 172.21.15.151:0/527520951 >> 172.21.15.151:0/527520951 conn(0x55c6ae48b050 msgr2=0x55c6ae557e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:54.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.354+0000 7f89faa00700 0 client.25573 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:56:54.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.354+0000 7f8a263edb80 1 -- 172.21.15.151:0/527520951 shutdown_connections 2024-04-26T01:56:54.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.354+0000 7f8a263edb80 1 -- 172.21.15.151:0/527520951 wait complete. 2024-04-26T01:56:54.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.354+0000 7f8a263edb80 20 client.25573 trim_cache size 0 max 16384 2024-04-26T01:56:54.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.375+0000 7f8a263edb80 1 Processor -- start 2024-04-26T01:56:54.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.375+0000 7f8a263edb80 1 -- start start 2024-04-26T01:56:54.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.376+0000 7f8a263edb80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6ae55bd40 0x55c6ae572b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:54.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.376+0000 7f8a263edb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6ae55c660 0x55c6ae565e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:54.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.376+0000 7f8a263edb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6ae5674d0 0x55c6ae56f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:54.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.376+0000 7f8a263edb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8a141b5f50 con 0x55c6ae5674d0 2024-04-26T01:56:54.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.376+0000 7f8a263edb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8a141b37e0 con 0x55c6ae55c660 2024-04-26T01:56:54.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.376+0000 7f8a263edb80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8a141b1650 con 0x55c6ae55bd40 2024-04-26T01:56:54.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.376+0000 7f8a19000700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6ae55bd40 0x55c6ae572b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:54.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.376+0000 7f8a19000700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6ae55bd40 0x55c6ae572b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:41038/0 (socket says 172.21.15.151:41038) 2024-04-26T01:56:54.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.376+0000 7f8a19000700 1 -- 172.21.15.151:0/1891655383 learned_addr learned my addr 172.21.15.151:0/1891655383 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:56:54.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.376+0000 7f8a13e00700 1 --2- 172.21.15.151:0/1891655383 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6ae55c660 0x55c6ae565e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:54.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.376+0000 7f8a19a00700 1 --2- 172.21.15.151:0/1891655383 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6ae5674d0 0x55c6ae56f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:54.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.376+0000 7f8a19000700 1 -- 172.21.15.151:0/1891655383 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6ae55c660 msgr2=0x55c6ae565e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:54.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.376+0000 7f8a19000700 1 --2- 172.21.15.151:0/1891655383 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6ae55c660 0x55c6ae565e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:54.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.376+0000 7f8a19000700 1 -- 172.21.15.151:0/1891655383 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6ae5674d0 msgr2=0x55c6ae56f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:54.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.376+0000 7f8a19000700 1 --2- 172.21.15.151:0/1891655383 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6ae5674d0 0x55c6ae56f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:54.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.376+0000 7f8a19000700 1 -- 172.21.15.151:0/1891655383 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6ae5594c0 con 0x55c6ae55bd40 2024-04-26T01:56:54.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.376+0000 7f8a19a00700 1 --2- 172.21.15.151:0/1891655383 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6ae5674d0 0x55c6ae56f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:56:54.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.377+0000 7f8a19000700 1 --2- 172.21.15.151:0/1891655383 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6ae55bd40 0x55c6ae572b80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8a0809cd50 tx=0x7f8a080a14d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3998e1b0e1af8e8b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:54.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.377+0000 7f8a05600700 1 -- 172.21.15.151:0/1891655383 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a080ba070 con 0x55c6ae55bd40 2024-04-26T01:56:54.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.377+0000 7f8a05600700 1 -- 172.21.15.151:0/1891655383 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a080bd070 con 0x55c6ae55bd40 2024-04-26T01:56:54.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.377+0000 7f8a05600700 1 -- 172.21.15.151:0/1891655383 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a080a5a00 con 0x55c6ae55bd40 2024-04-26T01:56:54.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.378+0000 7f8a263edb80 1 -- 172.21.15.151:0/1891655383 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6ae55bd40 msgr2=0x55c6ae572b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:54.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.378+0000 7f8a263edb80 1 --2- 172.21.15.151:0/1891655383 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6ae55bd40 0x55c6ae572b80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8a0809cd50 tx=0x7f8a080a14d0 comp rx=0 tx=0).stop 2024-04-26T01:56:54.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.378+0000 7f8a263edb80 1 -- 172.21.15.151:0/1891655383 shutdown_connections 2024-04-26T01:56:54.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.378+0000 7f8a263edb80 1 --2- 172.21.15.151:0/1891655383 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6ae5674d0 0x55c6ae56f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:54.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.378+0000 7f8a263edb80 1 --2- 172.21.15.151:0/1891655383 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6ae55bd40 0x55c6ae572b80 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:54.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.378+0000 7f8a263edb80 1 --2- 172.21.15.151:0/1891655383 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6ae55c660 0x55c6ae565e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:54.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.378+0000 7f8a263edb80 1 -- 172.21.15.151:0/1891655383 >> 172.21.15.151:0/1891655383 conn(0x55c6ae48b050 msgr2=0x55c6ae565760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:54.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.378+0000 7f8a263edb80 1 -- 172.21.15.151:0/1891655383 shutdown_connections 2024-04-26T01:56:54.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.378+0000 7f8a263edb80 1 -- 172.21.15.151:0/1891655383 wait complete. 2024-04-26T01:56:54.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.378+0000 7f8a263edb80 1 -- start start 2024-04-26T01:56:54.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.378+0000 7f8a263edb80 1 Processor -- start 2024-04-26T01:56:54.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.379+0000 7f8a263edb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6ae55bd40 0x55c6ae56db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:54.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.379+0000 7f8a263edb80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6ae55c660 0x55c6ae56e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:54.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.379+0000 7f8a263edb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6ae5674d0 0x55c6ae56e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:54.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.379+0000 7f8a263edb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8a141b1ca0 con 0x55c6ae55bd40 2024-04-26T01:56:54.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.379+0000 7f8a263edb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8a141b3110 con 0x55c6ae5674d0 2024-04-26T01:56:54.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.379+0000 7f8a263edb80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8a141b2370 con 0x55c6ae55c660 2024-04-26T01:56:54.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.379+0000 7f8a263edb80 10 client.0 _ll_register_callbacks cb 0x55c6ae447820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:56:54.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.379+0000 7f8a19000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6ae55bd40 0x55c6ae56db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:54.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.379+0000 7f8a19000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6ae55bd40 0x55c6ae56db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37974/0 (socket says 172.21.15.151:37974) 2024-04-26T01:56:54.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.379+0000 7f8a19000700 1 -- 172.21.15.151:0/1235252935 learned_addr learned my addr 172.21.15.151:0/1235252935 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:56:54.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.379+0000 7f8a19a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6ae5674d0 0x55c6ae56e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:54.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.379+0000 7f8a13e00700 1 --2- 172.21.15.151:0/1235252935 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6ae55c660 0x55c6ae56e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:54.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.380+0000 7f8a13e00700 1 -- 172.21.15.151:0/1235252935 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6ae5674d0 msgr2=0x55c6ae56e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:54.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.380+0000 7f8a13e00700 1 --2- 172.21.15.151:0/1235252935 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6ae5674d0 0x55c6ae56e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:54.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.380+0000 7f8a13e00700 1 -- 172.21.15.151:0/1235252935 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6ae55bd40 msgr2=0x55c6ae56db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:54.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.380+0000 7f8a13e00700 1 --2- 172.21.15.151:0/1235252935 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6ae55bd40 0x55c6ae56db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:54.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.380+0000 7f8a13e00700 1 -- 172.21.15.151:0/1235252935 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6ae5dcc70 con 0x55c6ae55c660 2024-04-26T01:56:54.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.380+0000 7f8a19a00700 1 --2- 172.21.15.151:0/1235252935 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6ae5674d0 0x55c6ae56e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:56:54.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.380+0000 7f8a13e00700 1 --2- 172.21.15.151:0/1235252935 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6ae55c660 0x55c6ae56e070 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x55c6ae569f50 tx=0x7f8a00012b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66bad041ca93690d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:54.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.380+0000 7f89faa00700 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-26T01:56:54.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.380+0000 7f89faa00700 1 -- 172.21.15.151:0/1235252935 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a00004a30 con 0x55c6ae55c660 2024-04-26T01:56:54.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.380+0000 7f89faa00700 1 -- 172.21.15.151:0/1235252935 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f8a00004bd0 con 0x55c6ae55c660 2024-04-26T01:56:54.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.380+0000 7f8a263edb80 1 -- 172.21.15.151:0/1235252935 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a080bd070 con 0x55c6ae55c660 2024-04-26T01:56:54.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.381+0000 7f89faa00700 1 -- 172.21.15.151:0/1235252935 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8a00031050 con 0x55c6ae55c660 2024-04-26T01:56:54.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.381+0000 7f8a263edb80 10 client.25579 fetch_fsmap learned FSMap version 28 2024-04-26T01:56:54.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.381+0000 7f8a263edb80 1 -- 172.21.15.151:0/1235252935 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c6ae5de010 con 0x55c6ae55c660 2024-04-26T01:56:54.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.381+0000 7f89faa00700 1 -- 172.21.15.151:0/1235252935 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a080bd070 con 0x55c6ae55c660 2024-04-26T01:56:54.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.381+0000 7f89faa00700 1 -- 172.21.15.151:0/1235252935 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f8a000223d0 con 0x55c6ae55c660 2024-04-26T01:56:54.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.381+0000 7f8a263edb80 10 client.25579 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T01:56:54.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.381+0000 7f8a263edb80 10 client.25579 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-26T01:56:54.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.381+0000 7f8a263edb80 20 client.25579 populate_metadata read hostname 'smithi151' 2024-04-26T01:56:54.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.381+0000 7f8a263edb80 1 --2- 172.21.15.151:0/1235252935 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55c6ae5de740 0x55c6ae5e0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:54.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.381+0000 7f8a263edb80 4 client.25579 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "bc374c20", "release": true, "await": true}] 2024-04-26T01:56:54.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.381+0000 7f8a263edb80 1 -- 172.21.15.151:0/1235252935 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "quiesce db", "set_id": "bc374c20", "release": true, "await": true}) -- 0x55c6ae4b6820 con 0x55c6ae5de740 2024-04-26T01:56:54.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.382+0000 7f8a19000700 1 --2- 172.21.15.151:0/1235252935 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55c6ae5de740 0x55c6ae5e0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:54.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.382+0000 7f8a19000700 1 --2- 172.21.15.151:0/1235252935 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55c6ae5de740 0x55c6ae5e0b60 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be6a6e068113f1e3 server_cookie=ff97ae0cffeccf11 in_seq=0 out_seq=0 2024-04-26T01:56:54.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.382+0000 7f89faa00700 10 client.25579 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:56:54.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:56:54 smithi172 ceph-mon[32785]: pgmap v101: 97 pgs: 97 active+clean; 31 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 24 KiB/s wr, 13 op/s 2024-04-26T01:56:54.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.579+0000 7f89faa00700 1 -- 172.21.15.151:0/1235252935 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x55c6ae4b6820 con 0x55c6ae5de740 2024-04-26T01:56:54.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.579+0000 7f89faa00700 10 client.25579 handle_command_reply: tid=0 2024-04-26T01:56:54.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.580+0000 7f8a263edb80 1 client.25579 shutdown 2024-04-26T01:56:54.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.580+0000 7f8a263edb80 1 -- 172.21.15.151:0/1235252935 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6ae55c660 msgr2=0x55c6ae56e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:54.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.580+0000 7f8a263edb80 1 --2- 172.21.15.151:0/1235252935 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6ae55c660 0x55c6ae56e070 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x55c6ae569f50 tx=0x7f8a00012b20 comp rx=0 tx=0).stop 2024-04-26T01:56:54.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.580+0000 7f8a263edb80 1 -- 172.21.15.151:0/1235252935 shutdown_connections 2024-04-26T01:56:54.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.580+0000 7f8a263edb80 1 --2- 172.21.15.151:0/1235252935 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55c6ae5de740 0x55c6ae5e0b60 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:54.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.580+0000 7f8a263edb80 1 --2- 172.21.15.151:0/1235252935 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6ae5674d0 0x55c6ae56e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:54.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.580+0000 7f8a263edb80 1 --2- 172.21.15.151:0/1235252935 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6ae55bd40 0x55c6ae56db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:54.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.580+0000 7f89faa00700 0 client.25579 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:56:54.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.580+0000 7f8a263edb80 1 --2- 172.21.15.151:0/1235252935 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6ae55c660 0x55c6ae56e070 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:54.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.580+0000 7f8a263edb80 1 -- 172.21.15.151:0/1235252935 >> 172.21.15.151:0/1235252935 conn(0x55c6ae48b050 msgr2=0x55c6ae565ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:54.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.581+0000 7f8a263edb80 1 -- 172.21.15.151:0/1235252935 shutdown_connections 2024-04-26T01:56:54.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.581+0000 7f8a263edb80 1 -- 172.21.15.151:0/1235252935 wait complete. 2024-04-26T01:56:54.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.581+0000 7f8a263edb80 20 client.25579 trim_cache size 0 max 16384 2024-04-26T01:56:54.581 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-26T01:56:54.582 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 28, 2024-04-26T01:56:54.582 INFO:teuthology.orchestra.run.smithi151.stdout: "leader": 24509, 2024-04-26T01:56:54.582 INFO:teuthology.orchestra.run.smithi151.stdout: "set_version": 8, 2024-04-26T01:56:54.582 INFO:teuthology.orchestra.run.smithi151.stdout: "sets": { 2024-04-26T01:56:54.582 INFO:teuthology.orchestra.run.smithi151.stdout: "bc374c20": { 2024-04-26T01:56:54.582 INFO:teuthology.orchestra.run.smithi151.stdout: "version": 8, 2024-04-26T01:56:54.582 INFO:teuthology.orchestra.run.smithi151.stdout: "age_ref": 0.0, 2024-04-26T01:56:54.582 INFO:teuthology.orchestra.run.smithi151.stdout: "state": { 2024-04-26T01:56:54.582 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "RELEASED", 2024-04-26T01:56:54.582 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 0.0 2024-04-26T01:56:54.582 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T01:56:54.582 INFO:teuthology.orchestra.run.smithi151.stdout: "timeout": 60.0, 2024-04-26T01:56:54.582 INFO:teuthology.orchestra.run.smithi151.stdout: "expiration": 90.0, 2024-04-26T01:56:54.583 INFO:teuthology.orchestra.run.smithi151.stdout: "members": { 2024-04-26T01:56:54.583 INFO:teuthology.orchestra.run.smithi151.stdout: "file:/": { 2024-04-26T01:56:54.583 INFO:teuthology.orchestra.run.smithi151.stdout: "excluded": false, 2024-04-26T01:56:54.583 INFO:teuthology.orchestra.run.smithi151.stdout: "state": { 2024-04-26T01:56:54.583 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "RELEASED", 2024-04-26T01:56:54.583 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 0.0 2024-04-26T01:56:54.583 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T01:56:54.583 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T01:56:54.583 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T01:56:54.583 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T01:56:54.583 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T01:56:54.583 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-26T01:56:54.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.583+0000 7f8a1ba00700 1 -- 172.21.15.151:0/3061325396 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f89fc0a3db0 msgr2=0x7f89fc0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:54.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.583+0000 7f8a1ba00700 1 --2- 172.21.15.151:0/3061325396 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f89fc0a3db0 0x7f89fc0a6270 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c009fd0 tx=0x7f8a0c009920 comp rx=0 tx=0).stop 2024-04-26T01:56:54.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.583+0000 7f8a1ba00700 1 -- 172.21.15.151:0/3061325396 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a141092d0 msgr2=0x7f8a141a53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:54.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.583+0000 7f8a1ba00700 1 --2- 172.21.15.151:0/3061325396 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a141092d0 0x7f8a141a53f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8a08005910 tx=0x7f8a08004270 comp rx=0 tx=0).stop 2024-04-26T01:56:54.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.583+0000 7f8a1ba00700 1 -- 172.21.15.151:0/3061325396 shutdown_connections 2024-04-26T01:56:54.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.583+0000 7f8a1ba00700 1 --2- 172.21.15.151:0/3061325396 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f89fc0a3db0 0x7f89fc0a6270 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:54.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.583+0000 7f8a1ba00700 1 --2- 172.21.15.151:0/3061325396 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a1410e3b0 0x7f8a1419f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:54.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.583+0000 7f8a1ba00700 1 --2- 172.21.15.151:0/3061325396 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a141092d0 0x7f8a141a53f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:54.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.583+0000 7f8a1ba00700 1 --2- 172.21.15.151:0/3061325396 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a14109c80 0x7f8a141a5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:54.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.583+0000 7f8a1ba00700 1 -- 172.21.15.151:0/3061325396 >> 172.21.15.151:0/3061325396 conn(0x7f8a14076210 msgr2=0x7f8a1410cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:54.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.584+0000 7f8a1ba00700 1 -- 172.21.15.151:0/3061325396 shutdown_connections 2024-04-26T01:56:54.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:54.584+0000 7f8a1ba00700 1 -- 172.21.15.151:0/3061325396 wait complete. 2024-04-26T01:56:54.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:56:54 smithi165 ceph-mon[32648]: pgmap v101: 97 pgs: 97 active+clean; 31 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 24 KiB/s wr, 13 op/s 2024-04-26T01:56:54.643 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] dio-interleaved 2024-04-26T01:56:54.659 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] t_dir_type 2024-04-26T01:56:54.732 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] dio-invalidate-cache 2024-04-26T01:56:54.782 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] stat_test 2024-04-26T01:56:54.994 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] t_encrypted_d_revalidate 2024-04-26T01:56:54.999 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] attr_replace_test 2024-04-26T01:56:55.115 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] swapon 2024-04-26T01:56:55.304 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] mkswap 2024-04-26T01:56:55.332 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] t_attr_corruption 2024-04-26T01:56:55.334 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] t_open_tmpfiles 2024-04-26T01:56:55.441 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] fscrypt-crypt-util 2024-04-26T01:56:55.590 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'bc374c20', seconds elapsed: 50.4 2024-04-26T01:56:55.590 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-04-26T01:56:55.590 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:56:55.608 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] bulkstat_null_ocount 2024-04-26T01:56:55.641 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] splice-test 2024-04-26T01:56:55.696 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] chprojid_fail 2024-04-26T01:56:55.951 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] detached_mounts_propagation 2024-04-26T01:56:56.033 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] ext4_resize 2024-04-26T01:56:56.064 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] t_readdir_3 2024-04-26T01:56:56.082 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:56:56.308 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] splice2pipe 2024-04-26T01:56:56.323 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] uuid_ioctl 2024-04-26T01:56:56.438 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] t_snapshot_deleted_subvolume 2024-04-26T01:56:56.450 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] fiemap-fault 2024-04-26T01:56:56.532 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:56:56 smithi151 ceph-mon[28057]: pgmap v102: 97 pgs: 97 active+clean; 31 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-04-26T01:56:56.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:56:56 smithi172 ceph-mon[32785]: pgmap v102: 97 pgs: 97 active+clean; 31 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-04-26T01:56:56.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:56:56 smithi165 ceph-mon[32648]: pgmap v102: 97 pgs: 97 active+clean; 31 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-04-26T01:56:56.647 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] fiemap-tester 2024-04-26T01:56:56.650 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] t_stripealign 2024-04-26T01:56:56.763 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] open_by_handle 2024-04-26T01:56:56.767 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] t_mmap_dio 2024-04-26T01:56:56.960 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] xfsfind 2024-04-26T01:56:57.053 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] t_reflink_read_race 2024-04-26T01:56:57.087 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] fssum 2024-04-26T01:56:57.197 INFO:tasks.workunit.client.0.smithi151.stdout:Building log-writes 2024-04-26T01:56:57.227 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] replay-log 2024-04-26T01:56:57.324 INFO:tasks.workunit.client.0.smithi151.stdout:Building perf 2024-04-26T01:56:57.358 INFO:tasks.workunit.client.0.smithi151.stdout:Building aio-dio-regress 2024-04-26T01:56:57.380 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] aiocp 2024-04-26T01:56:57.387 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] aio-dio-append-write-fallocate-race 2024-04-26T01:56:57.692 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] aio-dio-append-write-read-race 2024-04-26T01:56:57.734 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] aio-dio-cow-race 2024-04-26T01:56:57.763 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] aio-dio-cycle-write 2024-04-26T01:56:57.776 INFO:tasks.workunit.client.0.smithi151.stdout:Building vfs 2024-04-26T01:56:57.793 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] vfstest 2024-04-26T01:56:58.027 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] mount-idmapped 2024-04-26T01:56:58.028 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] aio-dio-eof-race 2024-04-26T01:56:58.138 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] aio-dio-extend-stat 2024-04-26T01:56:58.196 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:56:58 smithi151 ceph-mon[28057]: pgmap v103: 97 pgs: 97 active+clean; 31 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-26T01:56:58.348 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] aio-dio-fcntl-race 2024-04-26T01:56:58.433 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] aio-dio-hole-filling-race 2024-04-26T01:56:58.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:56:58 smithi172 ceph-mon[32785]: pgmap v103: 97 pgs: 97 active+clean; 31 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-26T01:56:58.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:56:58 smithi165 ceph-mon[32648]: pgmap v103: 97 pgs: 97 active+clean; 31 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-26T01:56:58.661 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] aio-dio-invalidate-failure 2024-04-26T01:56:58.702 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] aio-dio-invalidate-readahead 2024-04-26T01:56:58.725 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] aiodio_sparse2 2024-04-26T01:56:58.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:58.952+0000 7fc0b6800700 1 -- 172.21.15.151:0/3541294963 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0b006a750 msgr2=0x7fc0b006abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:58.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:58.952+0000 7fc0b6800700 1 --2- 172.21.15.151:0/3541294963 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0b006a750 0x7fc0b006abb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc09c009990 tx=0x7fc09c009ca0 comp rx=0 tx=0).stop 2024-04-26T01:56:58.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:58.953+0000 7fc0b6800700 1 -- 172.21.15.151:0/3541294963 shutdown_connections 2024-04-26T01:56:58.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:58.953+0000 7fc0b6800700 1 --2- 172.21.15.151:0/3541294963 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc0b006b0f0 0x7fc0b010f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:58.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:58.953+0000 7fc0b6800700 1 --2- 172.21.15.151:0/3541294963 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0b0108a30 0x7fc0b0108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:58.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:58.953+0000 7fc0b6800700 1 --2- 172.21.15.151:0/3541294963 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0b006a750 0x7fc0b006abb0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:58.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:58.953+0000 7fc0b6800700 1 -- 172.21.15.151:0/3541294963 >> 172.21.15.151:0/3541294963 conn(0x7fc0b00788f0 msgr2=0x7fc0b0078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:58.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:58.953+0000 7fc0b6800700 1 -- 172.21.15.151:0/3541294963 shutdown_connections 2024-04-26T01:56:58.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:58.953+0000 7fc0b6800700 1 -- 172.21.15.151:0/3541294963 wait complete. 2024-04-26T01:56:58.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:58.955+0000 7fc0b6800700 1 Processor -- start 2024-04-26T01:56:58.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:58.955+0000 7fc0b6800700 1 -- start start 2024-04-26T01:56:58.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:58.955+0000 7fc0b6800700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0b006a750 0x7fc0b0199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:58.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:58.955+0000 7fc0b6800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0b006b0f0 0x7fc0b0199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:58.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:58.956+0000 7fc0b6800700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc0b0108a30 0x7fc0b019d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:58.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:58.956+0000 7fc0b6800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b0112c20 con 0x7fc0b006b0f0 2024-04-26T01:56:58.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:58.956+0000 7fc0b6800700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b0112aa0 con 0x7fc0b0108a30 2024-04-26T01:56:58.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:58.956+0000 7fc0b6800700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b0112da0 con 0x7fc0b006a750 2024-04-26T01:56:58.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:58.956+0000 7fc0af400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0b006a750 0x7fc0b0199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:58.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:58.956+0000 7fc0afe00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc0b0108a30 0x7fc0b019d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:58.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:58.956+0000 7fc0afe00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc0b0108a30 0x7fc0b019d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:46878/0 (socket says 172.21.15.151:46878) 2024-04-26T01:56:58.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:58.956+0000 7fc0afe00700 1 -- 172.21.15.151:0/2367679310 learned_addr learned my addr 172.21.15.151:0/2367679310 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:56:58.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:58.956+0000 7fc0af400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0b006a750 0x7fc0b0199150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:41060/0 (socket says 172.21.15.151:41060) 2024-04-26T01:56:58.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:58.957+0000 7fc0afe00700 1 -- 172.21.15.151:0/2367679310 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0b006a750 msgr2=0x7fc0b0199150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:58.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:58.957+0000 7fc0afe00700 1 --2- 172.21.15.151:0/2367679310 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0b006a750 0x7fc0b0199150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:58.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:58.957+0000 7fc0afe00700 1 -- 172.21.15.151:0/2367679310 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0b006b0f0 msgr2=0x7fc0b0199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:58.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:58.957+0000 7fc0afe00700 1 --2- 172.21.15.151:0/2367679310 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0b006b0f0 0x7fc0b0199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:58.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:58.957+0000 7fc0afe00700 1 -- 172.21.15.151:0/2367679310 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0b019e110 con 0x7fc0b0108a30 2024-04-26T01:56:58.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:58.957+0000 7fc0afe00700 1 --2- 172.21.15.151:0/2367679310 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc0b0108a30 0x7fc0b019d9d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc0a800ef30 tx=0x7fc0a800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:58.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:58.958+0000 7fc0a7e00700 1 -- 172.21.15.151:0/2367679310 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a80092e0 con 0x7fc0b0108a30 2024-04-26T01:56:58.957 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:58.958+0000 7fc0a7e00700 1 -- 172.21.15.151:0/2367679310 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0a8004bb0 con 0x7fc0b0108a30 2024-04-26T01:56:58.957 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:58.958+0000 7fc0b6800700 1 -- 172.21.15.151:0/2367679310 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0b019e3a0 con 0x7fc0b0108a30 2024-04-26T01:56:58.957 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:58.958+0000 7fc0a7e00700 1 -- 172.21.15.151:0/2367679310 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a8004280 con 0x7fc0b0108a30 2024-04-26T01:56:58.957 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:58.958+0000 7fc0b6800700 1 -- 172.21.15.151:0/2367679310 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0b01a5c80 con 0x7fc0b0108a30 2024-04-26T01:56:58.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:58.959+0000 7fc0b6800700 1 -- 172.21.15.151:0/2367679310 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0b0069730 con 0x7fc0b0108a30 2024-04-26T01:56:58.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:58.961+0000 7fc0a7e00700 1 -- 172.21.15.151:0/2367679310 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7fc0a801b030 con 0x7fc0b0108a30 2024-04-26T01:56:58.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:58.962+0000 7fc0a7e00700 1 --2- 172.21.15.151:0/2367679310 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fc0940a3fb0 0x7fc0940a6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:58.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:58.963+0000 7fc0a7e00700 1 -- 172.21.15.151:0/2367679310 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fc0a8004020 con 0x7fc0b0108a30 2024-04-26T01:56:58.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:58.963+0000 7fc0af400700 1 --2- 172.21.15.151:0/2367679310 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fc0940a3fb0 0x7fc0940a6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:58.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:58.964+0000 7fc0af400700 1 --2- 172.21.15.151:0/2367679310 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fc0940a3fb0 0x7fc0940a6470 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc0a0009be0 tx=0x7fc0a0009260 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:58.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:58.966+0000 7fc0a7e00700 1 -- 172.21.15.151:0/2367679310 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc0a8022020 con 0x7fc0b0108a30 2024-04-26T01:56:59.001 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] aio-dio-subblock-eof-read 2024-04-26T01:56:59.053 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] aio-dio-write-verify 2024-04-26T01:56:59.138 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] aio-free-ring-with-bogus-nr-pages 2024-04-26T01:56:59.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:59.289+0000 7fc0b6800700 1 -- 172.21.15.151:0/2367679310 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc0b002df80 con 0x7fc0b0108a30 2024-04-26T01:56:59.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:59.290+0000 7fc0a7e00700 1 -- 172.21.15.151:0/2367679310 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11892 (secure 0 0 0) 0x7fc0a8010040 con 0x7fc0b0108a30 2024-04-26T01:56:59.289 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:56:59.289 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2937864097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2937864097},{"type":"v1","addr":"172.21.15.151:6835","nonce":2937864097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14373,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/2947090802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2947090802},{"type":"v1","addr":"172.21.15.151:6837","nonce":2947090802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/1356563280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1356563280},{"type":"v1","addr":"172.21.15.151:6839","nonce":1356563280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6841/1075240552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1075240552},{"type":"v1","addr":"172.21.15.151:6841","nonce":1075240552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/504675912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":504675912},{"type":"v1","addr":"172.21.15.165:6833","nonce":504675912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1207580015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1207580015},{"type":"v1","addr":"172.21.15.172:6833","nonce":1207580015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:49:40.746876+0000","modified":"2024-04-26T01:55:28.126804+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24505,"mds_2":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/3598138294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3598138294},{"type":"v1","addr":"172.21.15.165:6835","nonce":3598138294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/1847859054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":1847859054},{"type":"v1","addr":"172.21.15.165:6837","nonce":1847859054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/2309751171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2309751171},{"type":"v1","addr":"172.21.15.172:6835","nonce":2309751171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.172:6837/55972537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":55972537},{"type":"v1","addr":"172.21.15.172:6837","nonce":55972537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.172:6839/1242302810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1242302810},{"type":"v1","addr":"172.21.15.172:6839","nonce":1242302810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/1969716254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1969716254},{"type":"v1","addr":"172.21.15.165:6839","nonce":1969716254}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24505,24509,24493]},"id":1}]} 2024-04-26T01:56:59.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:59.293+0000 7fc0b6800700 1 -- 172.21.15.151:0/2367679310 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fc0940a3fb0 msgr2=0x7fc0940a6470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:59.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:59.293+0000 7fc0b6800700 1 --2- 172.21.15.151:0/2367679310 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fc0940a3fb0 0x7fc0940a6470 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc0a0009be0 tx=0x7fc0a0009260 comp rx=0 tx=0).stop 2024-04-26T01:56:59.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:59.293+0000 7fc0b6800700 1 -- 172.21.15.151:0/2367679310 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc0b0108a30 msgr2=0x7fc0b019d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:59.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:59.293+0000 7fc0b6800700 1 --2- 172.21.15.151:0/2367679310 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc0b0108a30 0x7fc0b019d9d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc0a800ef30 tx=0x7fc0a800c3a0 comp rx=0 tx=0).stop 2024-04-26T01:56:59.293 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:59.293+0000 7fc0b6800700 1 -- 172.21.15.151:0/2367679310 shutdown_connections 2024-04-26T01:56:59.293 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:59.293+0000 7fc0b6800700 1 --2- 172.21.15.151:0/2367679310 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fc0940a3fb0 0x7fc0940a6470 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:59.293 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:59.293+0000 7fc0b6800700 1 --2- 172.21.15.151:0/2367679310 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc0b0108a30 0x7fc0b019d9d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:59.293 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:59.293+0000 7fc0b6800700 1 --2- 172.21.15.151:0/2367679310 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0b006a750 0x7fc0b0199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:59.293 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:59.293+0000 7fc0b6800700 1 --2- 172.21.15.151:0/2367679310 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0b006b0f0 0x7fc0b0199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:59.293 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:59.293+0000 7fc0b6800700 1 -- 172.21.15.151:0/2367679310 >> 172.21.15.151:0/2367679310 conn(0x7fc0b00788f0 msgr2=0x7fc0b00ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:59.293 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:59.294+0000 7fc0b6800700 1 -- 172.21.15.151:0/2367679310 shutdown_connections 2024-04-26T01:56:59.293 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:56:59.294+0000 7fc0b6800700 1 -- 172.21.15.151:0/2367679310 wait complete. 2024-04-26T01:56:59.296 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 28 2024-04-26T01:56:59.343 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] aio-io-setup-with-nonwritable-context-pointer 2024-04-26T01:56:59.433 INFO:tasks.workunit.client.0.smithi151.stdout: [CC] aio-last-ref-held-by-io 2024-04-26T01:56:59.825 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.826+0000 7f4a94e00640 1 -- 172.21.15.151:0/3677143414 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a9006a780 msgr2=0x7f4a9006abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:59.825 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.826+0000 7f4a94e00640 1 --2- 172.21.15.151:0/3677143414 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a9006a780 0x7f4a9006abe0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c009900 tx=0x7f4a7c02a480 comp rx=0 tx=0).stop 2024-04-26T01:56:59.825 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.826+0000 7f4a94e00640 1 -- 172.21.15.151:0/3677143414 shutdown_connections 2024-04-26T01:56:59.825 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.826+0000 7f4a94e00640 1 --2- 172.21.15.151:0/3677143414 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a9006b120 0x7f4a9010edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:59.825 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.826+0000 7f4a94e00640 1 --2- 172.21.15.151:0/3677143414 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a9006a780 0x7f4a9006abe0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:59.825 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.826+0000 7f4a94e00640 1 --2- 172.21.15.151:0/3677143414 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4a901080e0 0x7f4a901084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:59.826 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.826+0000 7f4a94e00640 1 -- 172.21.15.151:0/3677143414 >> 172.21.15.151:0/3677143414 conn(0x7f4a90076c50 msgr2=0x7f4a90077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:59.826 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.826+0000 7f4a94e00640 1 -- 172.21.15.151:0/3677143414 shutdown_connections 2024-04-26T01:56:59.826 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.826+0000 7f4a94e00640 1 -- 172.21.15.151:0/3677143414 wait complete. 2024-04-26T01:56:59.828 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.827+0000 7f4a94e00640 1 Processor -- start 2024-04-26T01:56:59.828 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.827+0000 7f4a94e00640 1 -- start start 2024-04-26T01:56:59.828 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.827+0000 7f4a94e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a9006a780 0x7f4a901a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:59.828 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.827+0000 7f4a94e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a9006b120 0x7f4a901a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:59.828 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.827+0000 7f4a94e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4a901080e0 0x7f4a9019a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:59.828 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.827+0000 7f4a94e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4a901121d0 con 0x7f4a9006b120 2024-04-26T01:56:59.828 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.827+0000 7f4a94e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4a90112050 con 0x7f4a901080e0 2024-04-26T01:56:59.828 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.827+0000 7f4a94e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4a90112350 con 0x7f4a9006a780 2024-04-26T01:56:59.828 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.828+0000 7f4a8d400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a9006b120 0x7f4a901a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:59.828 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.828+0000 7f4a8d400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a9006b120 0x7f4a901a0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52764/0 (socket says 172.21.15.151:52764) 2024-04-26T01:56:59.828 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.828+0000 7f4a8d400640 1 -- 172.21.15.151:0/6720121 learned_addr learned my addr 172.21.15.151:0/6720121 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:56:59.829 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.828+0000 7f4a8de00640 1 --2- 172.21.15.151:0/6720121 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a9006a780 0x7f4a901a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:59.829 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.828+0000 7f4a8e800640 1 --2- 172.21.15.151:0/6720121 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4a901080e0 0x7f4a9019a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:59.829 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.828+0000 7f4a8d400640 1 -- 172.21.15.151:0/6720121 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a9006a780 msgr2=0x7f4a901a0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:59.829 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.828+0000 7f4a8d400640 1 --2- 172.21.15.151:0/6720121 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a9006a780 0x7f4a901a0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:59.829 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.828+0000 7f4a8d400640 1 -- 172.21.15.151:0/6720121 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4a901080e0 msgr2=0x7f4a9019a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:59.829 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.828+0000 7f4a8d400640 1 --2- 172.21.15.151:0/6720121 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4a901080e0 0x7f4a9019a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:59.829 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.828+0000 7f4a8d400640 1 -- 172.21.15.151:0/6720121 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a9019aef0 con 0x7f4a9006b120 2024-04-26T01:56:59.829 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.828+0000 7f4a8d400640 1 --2- 172.21.15.151:0/6720121 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a9006b120 0x7f4a901a0ab0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c005ab0 tx=0x7f4a7c003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:59.829 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.829+0000 7f4a86a00640 1 -- 172.21.15.151:0/6720121 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c03f070 con 0x7f4a9006b120 2024-04-26T01:56:59.829 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.829+0000 7f4a86a00640 1 -- 172.21.15.151:0/6720121 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a7c031d60 con 0x7f4a9006b120 2024-04-26T01:56:59.829 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.829+0000 7f4a94e00640 1 -- 172.21.15.151:0/6720121 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a9019b180 con 0x7f4a9006b120 2024-04-26T01:56:59.829 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.829+0000 7f4a86a00640 1 -- 172.21.15.151:0/6720121 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c035030 con 0x7f4a9006b120 2024-04-26T01:56:59.829 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.829+0000 7f4a94e00640 1 -- 172.21.15.151:0/6720121 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a901a7310 con 0x7f4a9006b120 2024-04-26T01:56:59.831 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.832+0000 7f4a97c34740 1 Processor -- start 2024-04-26T01:56:59.831 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.832+0000 7f4a86a00640 1 -- 172.21.15.151:0/6720121 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f4a7c042070 con 0x7f4a9006b120 2024-04-26T01:56:59.831 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.832+0000 7f4a97c34740 1 -- start start 2024-04-26T01:56:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.832+0000 7f4a97c34740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a82cf0820 0x560a82cf0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.832+0000 7f4a97c34740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560a82cf1140 0x560a82cfb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.833+0000 7f4a97c34740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a82cfba00 0x560a82cfddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.833+0000 7f4a97c34740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560a82c56900 con 0x560a82cf1140 2024-04-26T01:56:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.833+0000 7f4a97c34740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560a82bfcef0 con 0x560a82cfba00 2024-04-26T01:56:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.833+0000 7f4a97c34740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560a82c528c0 con 0x560a82cf0820 2024-04-26T01:56:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.833+0000 7f4a8e800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a82cf0820 0x560a82cf0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.833+0000 7f4a8de00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560a82cf1140 0x560a82cfb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.833+0000 7f4a8e800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a82cf0820 0x560a82cf0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:36860/0 (socket says 172.21.15.151:36860) 2024-04-26T01:56:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.833+0000 7f4a8e800640 1 -- 172.21.15.151:0/1979331655 learned_addr learned my addr 172.21.15.151:0/1979331655 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:56:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.833+0000 7f4a8d400640 1 --2- 172.21.15.151:0/1979331655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a82cfba00 0x560a82cfddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.833+0000 7f4a8e800640 1 -- 172.21.15.151:0/1979331655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a82cfba00 msgr2=0x560a82cfddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.833+0000 7f4a8e800640 1 --2- 172.21.15.151:0/1979331655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a82cfba00 0x560a82cfddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.833+0000 7f4a8e800640 1 -- 172.21.15.151:0/1979331655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560a82cf1140 msgr2=0x560a82cfb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.833+0000 7f4a8e800640 1 --2- 172.21.15.151:0/1979331655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560a82cf1140 0x560a82cfb4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.833+0000 7f4a8e800640 1 -- 172.21.15.151:0/1979331655 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a82969210 con 0x560a82cf0820 2024-04-26T01:56:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.834+0000 7f4a8e800640 1 --2- 172.21.15.151:0/1979331655 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a82cf0820 0x560a82cf0c00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4a88003a40 tx=0x7f4a88003f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11f50d8496c109df server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.834+0000 7f4a86a00640 1 --2- 172.21.15.151:0/6720121 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f4a740a3dc0 0x7f4a740a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.834+0000 7f4a86a00640 1 -- 172.21.15.151:0/6720121 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f4a7c0f0f80 con 0x7f4a9006b120 2024-04-26T01:56:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.834+0000 7f4a7b400640 1 -- 172.21.15.151:0/1979331655 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8801a470 con 0x560a82cf0820 2024-04-26T01:56:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.834+0000 7f4a8de00640 1 --2- 172.21.15.151:0/6720121 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f4a740a3dc0 0x7f4a740a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.834+0000 7f4a7b400640 1 -- 172.21.15.151:0/1979331655 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a88015070 con 0x560a82cf0820 2024-04-26T01:56:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.834+0000 7f4a7b400640 1 -- 172.21.15.151:0/1979331655 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a88010820 con 0x560a82cf0820 2024-04-26T01:56:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.835+0000 7f4a97c34740 1 -- 172.21.15.151:0/1979331655 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a82cf0820 msgr2=0x560a82cf0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.835+0000 7f4a8de00640 1 --2- 172.21.15.151:0/6720121 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f4a740a3dc0 0x7f4a740a6280 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x560a82cf1dc0 tx=0x7f4a80005f90 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.835+0000 7f4a97c34740 1 --2- 172.21.15.151:0/1979331655 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a82cf0820 0x560a82cf0c00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4a88003a40 tx=0x7f4a88003f00 comp rx=0 tx=0).stop 2024-04-26T01:56:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.835+0000 7f4a97c34740 1 -- 172.21.15.151:0/1979331655 shutdown_connections 2024-04-26T01:56:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.835+0000 7f4a97c34740 1 --2- 172.21.15.151:0/1979331655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a82cfba00 0x560a82cfddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.835+0000 7f4a97c34740 1 --2- 172.21.15.151:0/1979331655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560a82cf1140 0x560a82cfb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.835+0000 7f4a97c34740 1 --2- 172.21.15.151:0/1979331655 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a82cf0820 0x560a82cf0c00 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.835+0000 7f4a97c34740 1 -- 172.21.15.151:0/1979331655 >> 172.21.15.151:0/1979331655 conn(0x560a82cecd30 msgr2=0x560a82ced110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.835+0000 7f4a97c34740 1 -- 172.21.15.151:0/1979331655 shutdown_connections 2024-04-26T01:56:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.835+0000 7f4a97c34740 1 -- 172.21.15.151:0/1979331655 wait complete. 2024-04-26T01:56:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.836+0000 7f4a97c34740 1 -- start start 2024-04-26T01:56:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.836+0000 7f4a97c34740 1 Processor -- start 2024-04-26T01:56:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.836+0000 7f4a97c34740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a82cf0820 0x560a82cf8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.836+0000 7f4a97c34740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560a82cf1140 0x560a82cf8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.836+0000 7f4a97c34740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a82cfba00 0x560a82cf94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.836+0000 7f4a97c34740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560a82cfe630 con 0x560a82cf1140 2024-04-26T01:56:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.836+0000 7f4a97c34740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560a82c10310 con 0x560a82cf0820 2024-04-26T01:56:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.836+0000 7f4a97c34740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560a82cfe7b0 con 0x560a82cfba00 2024-04-26T01:56:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.836+0000 7f4a97c34740 10 client.0 _ll_register_callbacks cb 0x560a82bdecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:56:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.836+0000 7f4a8d400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560a82cf1140 0x560a82cf8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.837+0000 7f4a8de00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a82cf0820 0x560a82cf8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.837+0000 7f4a8d400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560a82cf1140 0x560a82cf8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52786/0 (socket says 172.21.15.151:52786) 2024-04-26T01:56:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.837+0000 7f4a8d400640 1 -- 172.21.15.151:0/705725060 learned_addr learned my addr 172.21.15.151:0/705725060 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:56:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.837+0000 7f4a8e800640 1 --2- 172.21.15.151:0/705725060 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a82cfba00 0x560a82cf94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:59.840 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.837+0000 7f4a8de00640 1 -- 172.21.15.151:0/705725060 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a82cfba00 msgr2=0x560a82cf94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:59.840 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.837+0000 7f4a8de00640 1 --2- 172.21.15.151:0/705725060 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a82cfba00 0x560a82cf94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:59.840 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.837+0000 7f4a8de00640 1 -- 172.21.15.151:0/705725060 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560a82cf1140 msgr2=0x560a82cf8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:59.840 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.837+0000 7f4a8de00640 1 --2- 172.21.15.151:0/705725060 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560a82cf1140 0x560a82cf8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:59.840 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.837+0000 7f4a8de00640 1 -- 172.21.15.151:0/705725060 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a82a35620 con 0x560a82cf0820 2024-04-26T01:56:59.840 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.837+0000 7f4a8d400640 1 --2- 172.21.15.151:0/705725060 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560a82cf1140 0x560a82cf8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:56:59.840 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.837+0000 7f4a8de00640 1 --2- 172.21.15.151:0/705725060 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a82cf0820 0x560a82cf8a30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4a80012cb0 tx=0x7f4a80004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3b955278a293d95 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:59.840 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.837+0000 7f4a6fe00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T01:56:59.840 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.837+0000 7f4a6fe00640 1 -- 172.21.15.151:0/705725060 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8001f640 con 0x560a82cf0820 2024-04-26T01:56:59.840 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.838+0000 7f4a97c34740 1 -- 172.21.15.151:0/705725060 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a88015070 con 0x560a82cf0820 2024-04-26T01:56:59.840 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.838+0000 7f4a6fe00640 1 -- 172.21.15.151:0/705725060 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f4a8001f7e0 con 0x560a82cf0820 2024-04-26T01:56:59.840 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.838+0000 7f4a6fe00640 1 -- 172.21.15.151:0/705725060 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4a8001f9e0 con 0x560a82cf0820 2024-04-26T01:56:59.840 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.838+0000 7f4a97c34740 10 client.25565 fetch_fsmap learned FSMap version 28 2024-04-26T01:56:59.841 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.838+0000 7f4a97c34740 1 -- 172.21.15.151:0/705725060 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560a82d0a9d0 con 0x560a82cf0820 2024-04-26T01:56:59.841 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.838+0000 7f4a6fe00640 1 -- 172.21.15.151:0/705725060 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a88015070 con 0x560a82cf0820 2024-04-26T01:56:59.841 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.838+0000 7f4a6fe00640 1 -- 172.21.15.151:0/705725060 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f4a80027070 con 0x560a82cf0820 2024-04-26T01:56:59.841 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.838+0000 7f4a97c34740 10 client.25565 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T01:56:59.841 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.838+0000 7f4a97c34740 10 client.25565 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:56:59.841 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.838+0000 7f4a97c34740 20 client.25565 populate_metadata read hostname 'smithi151' 2024-04-26T01:56:59.841 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.838+0000 7f4a97c34740 1 --2- 172.21.15.151:0/705725060 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x560a82d0bee0 0x560a82d0e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:59.841 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.838+0000 7f4a97c34740 4 client.25565 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:56:59.841 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.838+0000 7f4a97c34740 1 -- 172.21.15.151:0/705725060 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a82c6a8b0 con 0x560a82d0bee0 2024-04-26T01:56:59.841 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.839+0000 7f4a8d400640 1 --2- 172.21.15.151:0/705725060 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x560a82d0bee0 0x560a82d0e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:59.841 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.839+0000 7f4a8d400640 1 --2- 172.21.15.151:0/705725060 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x560a82d0bee0 0x560a82d0e330 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d7de3beb1ef5496 server_cookie=30e1023c4530506d in_seq=0 out_seq=0 2024-04-26T01:56:59.841 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.839+0000 7f4a6fe00640 10 client.25565 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:56:59.843 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.840+0000 7f4a6fe00640 1 -- 172.21.15.151:0/705725060 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560a82c6a8b0 con 0x560a82d0bee0 2024-04-26T01:56:59.843 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.840+0000 7f4a6fe00640 10 client.25565 handle_command_reply: tid=0 2024-04-26T01:56:59.843 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.840+0000 7f4a97c34740 1 client.25565 shutdown 2024-04-26T01:56:59.843 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.840+0000 7f4a97c34740 1 -- 172.21.15.151:0/705725060 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a82cf0820 msgr2=0x560a82cf8a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:59.843 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.840+0000 7f4a97c34740 1 --2- 172.21.15.151:0/705725060 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a82cf0820 0x560a82cf8a30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4a80012cb0 tx=0x7f4a80004730 comp rx=0 tx=0).stop 2024-04-26T01:56:59.843 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.840+0000 7f4a97c34740 1 -- 172.21.15.151:0/705725060 shutdown_connections 2024-04-26T01:56:59.843 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.840+0000 7f4a97c34740 1 --2- 172.21.15.151:0/705725060 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x560a82d0bee0 0x560a82d0e330 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:59.843 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.840+0000 7f4a97c34740 1 --2- 172.21.15.151:0/705725060 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a82cfba00 0x560a82cf94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:59.843 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.840+0000 7f4a97c34740 1 --2- 172.21.15.151:0/705725060 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560a82cf1140 0x560a82cf8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:59.843 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.840+0000 7f4a97c34740 1 --2- 172.21.15.151:0/705725060 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a82cf0820 0x560a82cf8a30 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:59.843 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.840+0000 7f4a97c34740 1 -- 172.21.15.151:0/705725060 >> 172.21.15.151:0/705725060 conn(0x560a82cecd30 msgr2=0x560a82cf2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:59.844 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.840+0000 7f4a6fe00640 0 client.25565 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:56:59.844 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.840+0000 7f4a97c34740 1 -- 172.21.15.151:0/705725060 shutdown_connections 2024-04-26T01:56:59.844 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.840+0000 7f4a97c34740 1 -- 172.21.15.151:0/705725060 wait complete. 2024-04-26T01:56:59.844 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.840+0000 7f4a97c34740 20 client.25565 trim_cache size 0 max 16384 2024-04-26T01:56:59.855 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.857+0000 7f4a97c34740 1 Processor -- start 2024-04-26T01:56:59.855 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.857+0000 7f4a97c34740 1 -- start start 2024-04-26T01:56:59.856 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.857+0000 7f4a97c34740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a82cf0820 0x560a82cf3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:59.856 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.857+0000 7f4a97c34740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a82cf1140 0x560a82d13c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:59.856 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.857+0000 7f4a97c34740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560a82cfba00 0x560a82d14180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:59.856 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.857+0000 7f4a97c34740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4a901b23c0 con 0x560a82cfba00 2024-04-26T01:56:59.856 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.857+0000 7f4a97c34740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4a901afc90 con 0x560a82cf1140 2024-04-26T01:56:59.856 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.857+0000 7f4a97c34740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4a901adb00 con 0x560a82cf0820 2024-04-26T01:56:59.857 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.857+0000 7f4a8e800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560a82cfba00 0x560a82d14180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:59.857 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.857+0000 7f4a8e800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560a82cfba00 0x560a82d14180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52800/0 (socket says 172.21.15.151:52800) 2024-04-26T01:56:59.857 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.857+0000 7f4a8e800640 1 -- 172.21.15.151:0/3407635295 learned_addr learned my addr 172.21.15.151:0/3407635295 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:56:59.857 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.857+0000 7f4a8de00640 1 --2- 172.21.15.151:0/3407635295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a82cf0820 0x560a82cf3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:59.857 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.858+0000 7f4a8d400640 1 --2- 172.21.15.151:0/3407635295 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a82cf1140 0x560a82d13c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:59.857 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.858+0000 7f4a8d400640 1 -- 172.21.15.151:0/3407635295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a82cf0820 msgr2=0x560a82cf3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:59.858 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.858+0000 7f4a8d400640 1 --2- 172.21.15.151:0/3407635295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a82cf0820 0x560a82cf3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:59.858 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.858+0000 7f4a8d400640 1 -- 172.21.15.151:0/3407635295 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560a82cfba00 msgr2=0x560a82d14180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:59.858 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.858+0000 7f4a8d400640 1 --2- 172.21.15.151:0/3407635295 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560a82cfba00 0x560a82d14180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:59.858 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.858+0000 7f4a8d400640 1 -- 172.21.15.151:0/3407635295 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a82d0a7e0 con 0x560a82cf1140 2024-04-26T01:56:59.858 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.858+0000 7f4a8e800640 1 --2- 172.21.15.151:0/3407635295 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560a82cfba00 0x560a82d14180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:56:59.858 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.858+0000 7f4a8d400640 1 --2- 172.21.15.151:0/3407635295 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a82cf1140 0x560a82d13c40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c000c00 tx=0x7f4a7c0091c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e24242cc4f596d30 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:59.858 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.858+0000 7f4a7aa00640 1 -- 172.21.15.151:0/3407635295 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c049070 con 0x560a82cf1140 2024-04-26T01:56:59.858 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.858+0000 7f4a7aa00640 1 -- 172.21.15.151:0/3407635295 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a7c0bcfe0 con 0x560a82cf1140 2024-04-26T01:56:59.858 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.858+0000 7f4a7aa00640 1 -- 172.21.15.151:0/3407635295 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c0bb2e0 con 0x560a82cf1140 2024-04-26T01:56:59.861 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.859+0000 7f4a97c34740 1 -- 172.21.15.151:0/3407635295 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a82cf1140 msgr2=0x560a82d13c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:59.861 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.859+0000 7f4a97c34740 1 --2- 172.21.15.151:0/3407635295 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a82cf1140 0x560a82d13c40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c000c00 tx=0x7f4a7c0091c0 comp rx=0 tx=0).stop 2024-04-26T01:56:59.861 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.859+0000 7f4a97c34740 1 -- 172.21.15.151:0/3407635295 shutdown_connections 2024-04-26T01:56:59.861 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.859+0000 7f4a97c34740 1 --2- 172.21.15.151:0/3407635295 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560a82cfba00 0x560a82d14180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:59.861 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.859+0000 7f4a97c34740 1 --2- 172.21.15.151:0/3407635295 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a82cf1140 0x560a82d13c40 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:59.861 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.859+0000 7f4a97c34740 1 --2- 172.21.15.151:0/3407635295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a82cf0820 0x560a82cf3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:59.861 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.859+0000 7f4a97c34740 1 -- 172.21.15.151:0/3407635295 >> 172.21.15.151:0/3407635295 conn(0x560a82cecd30 msgr2=0x560a82cf89f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:59.862 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.859+0000 7f4a97c34740 1 -- 172.21.15.151:0/3407635295 shutdown_connections 2024-04-26T01:56:59.862 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.859+0000 7f4a97c34740 1 -- 172.21.15.151:0/3407635295 wait complete. 2024-04-26T01:56:59.862 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.860+0000 7f4a97c34740 1 -- start start 2024-04-26T01:56:59.862 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.860+0000 7f4a97c34740 1 Processor -- start 2024-04-26T01:56:59.862 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.860+0000 7f4a97c34740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a82cf0820 0x560a82d01640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:59.862 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.860+0000 7f4a97c34740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a82cf1140 0x560a82d01b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:59.862 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.860+0000 7f4a97c34740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560a82cfba00 0x560a82d020c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:59.862 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.860+0000 7f4a97c34740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4a901ae150 con 0x560a82cfba00 2024-04-26T01:56:59.862 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.860+0000 7f4a97c34740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4a901af5c0 con 0x560a82cf1140 2024-04-26T01:56:59.862 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.860+0000 7f4a97c34740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4a901ae820 con 0x560a82cf0820 2024-04-26T01:56:59.862 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.860+0000 7f4a97c34740 10 client.0 _ll_register_callbacks cb 0x7f4a800044b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:56:59.862 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.861+0000 7f4a8e800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560a82cfba00 0x560a82d020c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:59.862 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.861+0000 7f4a8d400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a82cf1140 0x560a82d01b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.861+0000 7f4a8e800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560a82cfba00 0x560a82d020c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52808/0 (socket says 172.21.15.151:52808) 2024-04-26T01:56:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.861+0000 7f4a8e800640 1 -- 172.21.15.151:0/1481728402 learned_addr learned my addr 172.21.15.151:0/1481728402 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:56:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.861+0000 7f4a8de00640 1 --2- 172.21.15.151:0/1481728402 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a82cf0820 0x560a82d01640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.861+0000 7f4a8e800640 1 -- 172.21.15.151:0/1481728402 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a82cf0820 msgr2=0x560a82d01640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.861+0000 7f4a8e800640 1 --2- 172.21.15.151:0/1481728402 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a82cf0820 0x560a82d01640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.861+0000 7f4a8e800640 1 -- 172.21.15.151:0/1481728402 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a82cf1140 msgr2=0x560a82d01b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.861+0000 7f4a8e800640 1 --2- 172.21.15.151:0/1481728402 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a82cf1140 0x560a82d01b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.861+0000 7f4a8e800640 1 -- 172.21.15.151:0/1481728402 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a82d71710 con 0x560a82cfba00 2024-04-26T01:56:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.861+0000 7f4a8d400640 1 --2- 172.21.15.151:0/1481728402 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a82cf1140 0x560a82d01b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:56:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.861+0000 7f4a8e800640 1 --2- 172.21.15.151:0/1481728402 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560a82cfba00 0x560a82d020c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4a88003fd0 tx=0x7f4a88009a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8c40e62bc0b9c55 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.861+0000 7f4a6fe00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-04-26T01:56:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.862+0000 7f4a6fe00640 1 -- 172.21.15.151:0/1481728402 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8802c060 con 0x560a82cfba00 2024-04-26T01:56:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.862+0000 7f4a97c34740 1 -- 172.21.15.151:0/1481728402 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a7c0bcfe0 con 0x560a82cfba00 2024-04-26T01:56:59.864 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.862+0000 7f4a6fe00640 1 -- 172.21.15.151:0/1481728402 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f4a8801a670 con 0x560a82cfba00 2024-04-26T01:56:59.864 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.862+0000 7f4a6fe00640 1 -- 172.21.15.151:0/1481728402 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4a8801a870 con 0x560a82cfba00 2024-04-26T01:56:59.864 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.862+0000 7f4a97c34740 10 client.15474 fetch_fsmap learned FSMap version 28 2024-04-26T01:56:59.864 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.862+0000 7f4a97c34740 1 -- 172.21.15.151:0/1481728402 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560a82d729c0 con 0x560a82cfba00 2024-04-26T01:56:59.864 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.862+0000 7f4a6fe00640 1 -- 172.21.15.151:0/1481728402 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a7c0bcfe0 con 0x560a82cfba00 2024-04-26T01:56:59.864 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.863+0000 7f4a6fe00640 1 -- 172.21.15.151:0/1481728402 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f4a8802f020 con 0x560a82cfba00 2024-04-26T01:56:59.864 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.863+0000 7f4a97c34740 10 client.15474 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T01:56:59.864 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.863+0000 7f4a97c34740 10 client.15474 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:56:59.864 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.863+0000 7f4a97c34740 20 client.15474 populate_metadata read hostname 'smithi151' 2024-04-26T01:56:59.864 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.863+0000 7f4a97c34740 1 --2- 172.21.15.151:0/1481728402 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x560a82d72f60 0x560a82d75380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:59.864 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.863+0000 7f4a97c34740 4 client.15474 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:56:59.864 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.863+0000 7f4a97c34740 1 -- 172.21.15.151:0/1481728402 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560a82c64320 con 0x560a82d72f60 2024-04-26T01:56:59.865 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.863+0000 7f4a8de00640 1 --2- 172.21.15.151:0/1481728402 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x560a82d72f60 0x560a82d75380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:59.865 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.864+0000 7f4a8de00640 1 --2- 172.21.15.151:0/1481728402 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x560a82d72f60 0x560a82d75380 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c34bb648322e6b9 server_cookie=c8ad1f1b98f96ec8 in_seq=0 out_seq=0 2024-04-26T01:56:59.865 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.864+0000 7f4a6fe00640 10 client.15474 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:56:59.904 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.904+0000 7f4a6fe00640 1 -- 172.21.15.151:0/1481728402 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560a82c64320 con 0x560a82d72f60 2024-04-26T01:56:59.904 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.904+0000 7f4a6fe00640 10 client.15474 handle_command_reply: tid=0 2024-04-26T01:56:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.904+0000 7f4a97c34740 1 client.15474 shutdown 2024-04-26T01:56:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.905+0000 7f4a97c34740 1 -- 172.21.15.151:0/1481728402 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560a82cfba00 msgr2=0x560a82d020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.905+0000 7f4a97c34740 1 --2- 172.21.15.151:0/1481728402 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560a82cfba00 0x560a82d020c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4a88003fd0 tx=0x7f4a88009a90 comp rx=0 tx=0).stop 2024-04-26T01:56:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.905+0000 7f4a97c34740 1 -- 172.21.15.151:0/1481728402 shutdown_connections 2024-04-26T01:56:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.905+0000 7f4a97c34740 1 --2- 172.21.15.151:0/1481728402 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x560a82d72f60 0x560a82d75380 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.905+0000 7f4a97c34740 1 --2- 172.21.15.151:0/1481728402 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560a82cfba00 0x560a82d020c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.905+0000 7f4a97c34740 1 --2- 172.21.15.151:0/1481728402 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a82cf1140 0x560a82d01b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.905+0000 7f4a97c34740 1 --2- 172.21.15.151:0/1481728402 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a82cf0820 0x560a82d01640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.905+0000 7f4a97c34740 1 -- 172.21.15.151:0/1481728402 >> 172.21.15.151:0/1481728402 conn(0x560a82cecd30 msgr2=0x560a82cf9260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.905+0000 7f4a6fe00640 0 client.15474 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:56:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.905+0000 7f4a97c34740 1 -- 172.21.15.151:0/1481728402 shutdown_connections 2024-04-26T01:56:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.905+0000 7f4a97c34740 1 -- 172.21.15.151:0/1481728402 wait complete. 2024-04-26T01:56:59.906 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.905+0000 7f4a97c34740 20 client.15474 trim_cache size 0 max 16384 2024-04-26T01:56:59.906 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:{ 2024-04-26T01:56:59.906 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "message": "", 2024-04-26T01:56:59.906 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "return_code": 0 2024-04-26T01:56:59.906 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:} 2024-04-26T01:56:59.907 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.906+0000 7f4a94e00640 1 -- 172.21.15.151:0/6720121 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f4a740a3dc0 msgr2=0x7f4a740a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:59.907 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.906+0000 7f4a94e00640 1 --2- 172.21.15.151:0/6720121 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f4a740a3dc0 0x7f4a740a6280 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x560a82cf1dc0 tx=0x7f4a80005f90 comp rx=0 tx=0).stop 2024-04-26T01:56:59.907 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.906+0000 7f4a94e00640 1 -- 172.21.15.151:0/6720121 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a9006b120 msgr2=0x7f4a901a0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:59.907 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.906+0000 7f4a94e00640 1 --2- 172.21.15.151:0/6720121 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a9006b120 0x7f4a901a0ab0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c005ab0 tx=0x7f4a7c003c80 comp rx=0 tx=0).stop 2024-04-26T01:56:59.907 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.906+0000 7f4a94e00640 1 -- 172.21.15.151:0/6720121 shutdown_connections 2024-04-26T01:56:59.907 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.906+0000 7f4a94e00640 1 --2- 172.21.15.151:0/6720121 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f4a740a3dc0 0x7f4a740a6280 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:59.907 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.906+0000 7f4a94e00640 1 --2- 172.21.15.151:0/6720121 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4a901080e0 0x7f4a9019a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:59.907 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.906+0000 7f4a94e00640 1 --2- 172.21.15.151:0/6720121 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a9006b120 0x7f4a901a0ab0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:59.907 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.906+0000 7f4a94e00640 1 --2- 172.21.15.151:0/6720121 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a9006a780 0x7f4a901a0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:59.907 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.906+0000 7f4a94e00640 1 -- 172.21.15.151:0/6720121 >> 172.21.15.151:0/6720121 conn(0x7f4a90076c50 msgr2=0x7f4a900ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.906+0000 7f4a94e00640 1 -- 172.21.15.151:0/6720121 shutdown_connections 2024-04-26T01:56:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:56:59.906+0000 7f4a94e00640 1 -- 172.21.15.151:0/6720121 wait complete. 2024-04-26T01:57:00.252 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24509 quiesce db / --timeout 60 --expiration 90.0 --await-for 42.0' 2024-04-26T01:57:00.252 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24509 quiesce db / --timeout 60 --expiration 90.0 --await-for 42.0 2024-04-26T01:57:00.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:57:00 smithi172 ceph-mon[32785]: pgmap v104: 97 pgs: 97 active+clean; 31 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-26T01:57:00.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:57:00 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/2367679310' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:57:00.537 INFO:tasks.workunit.client.0.smithi151.stderr:+ cd .. 2024-04-26T01:57:00.537 INFO:tasks.workunit.client.0.smithi151.stderr:+ cp xfstests-dev/ltp/fsx . 2024-04-26T01:57:00.546 INFO:tasks.workunit.client.0.smithi151.stderr:+ OPTIONS=-z 2024-04-26T01:57:00.546 INFO:tasks.workunit.client.0.smithi151.stderr:+ ./fsx -z 1MB -N 50000 -p 10000 -l 1048576 2024-04-26T01:57:00.548 INFO:tasks.workunit.client.0.smithi151.stdout:Seed set to 1 2024-04-26T01:57:00.578 INFO:tasks.workunit.client.0.smithi151.stderr:main: filesystem does not support fallocate mode 0, disabling! 2024-04-26T01:57:00.578 INFO:tasks.workunit.client.0.smithi151.stderr:main: filesystem does not support fallocate mode FALLOC_FL_KEEP_SIZE, disabling! 2024-04-26T01:57:00.581 INFO:tasks.workunit.client.0.smithi151.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2024-04-26T01:57:00.581 INFO:tasks.workunit.client.0.smithi151.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2024-04-26T01:57:00.581 INFO:tasks.workunit.client.0.smithi151.stderr:main: filesystem does not support clone range, disabling! 2024-04-26T01:57:00.585 INFO:tasks.workunit.client.0.smithi151.stderr:main: filesystem does not support dedupe range, disabling! 2024-04-26T01:57:00.587 INFO:tasks.workunit.client.0.smithi151.stderr:main: filesystem does not support exchange range, disabling! 2024-04-26T01:57:00.587 INFO:tasks.workunit.client.0.smithi151.stdout:skipping zero size read 2024-04-26T01:57:00.587 INFO:tasks.workunit.client.0.smithi151.stdout:skipping zero size read 2024-04-26T01:57:00.588 INFO:tasks.workunit.client.0.smithi151.stdout:truncating to largest ever: 0x81d54 2024-04-26T01:57:00.594 INFO:tasks.workunit.client.0.smithi151.stdout:copying to largest ever: 0xb86d7 2024-04-26T01:57:00.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:57:00 smithi165 ceph-mon[32648]: pgmap v104: 97 pgs: 97 active+clean; 31 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-26T01:57:00.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:57:00 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/2367679310' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:57:00.602 INFO:tasks.workunit.client.0.smithi151.stdout:copying to largest ever: 0xe627e 2024-04-26T01:57:00.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:57:00 smithi151 ceph-mon[28057]: pgmap v104: 97 pgs: 97 active+clean; 31 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-26T01:57:00.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:57:00 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/2367679310' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:57:00.651 INFO:tasks.workunit.client.0.smithi151.stdout:truncating to largest ever: 0xf3194 2024-04-26T01:57:00.676 INFO:tasks.workunit.client.0.smithi151.stdout:copying to largest ever: 0xf3ceb 2024-04-26T01:57:00.683 INFO:tasks.workunit.client.0.smithi151.stdout:truncating to largest ever: 0xf47e4 2024-04-26T01:57:00.744 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:57:00.802 INFO:tasks.workunit.client.0.smithi151.stdout:copying to largest ever: 0xfaccc 2024-04-26T01:57:01.348 INFO:tasks.workunit.client.0.smithi151.stdout:copying to largest ever: 0xfe423 2024-04-26T01:57:02.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:57:02 smithi172 ceph-mon[32785]: pgmap v105: 97 pgs: 97 active+clean; 31 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 4.1 KiB/s wr, 10 op/s 2024-04-26T01:57:02.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:57:02 smithi165 ceph-mon[32648]: pgmap v105: 97 pgs: 97 active+clean; 31 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 4.1 KiB/s wr, 10 op/s 2024-04-26T01:57:02.607 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:57:02 smithi151 ceph-mon[28057]: pgmap v105: 97 pgs: 97 active+clean; 31 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 4.1 KiB/s wr, 10 op/s 2024-04-26T01:57:03.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.179+0000 7f7b08a00700 1 -- 172.21.15.151:0/1417422297 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b0406b100 msgr2=0x7f7b0410f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:03.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.179+0000 7f7b08a00700 1 --2- 172.21.15.151:0/1417422297 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b0406b100 0x7f7b0410f8c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7afc009f90 tx=0x7f7afc00b330 comp rx=0 tx=0).stop 2024-04-26T01:57:03.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.179+0000 7f7b08a00700 1 -- 172.21.15.151:0/1417422297 shutdown_connections 2024-04-26T01:57:03.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.179+0000 7f7b08a00700 1 --2- 172.21.15.151:0/1417422297 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b0406b100 0x7f7b0410f8c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:03.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.179+0000 7f7b08a00700 1 --2- 172.21.15.151:0/1417422297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7b04108a60 0x7f7b04108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:03.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.179+0000 7f7b08a00700 1 --2- 172.21.15.151:0/1417422297 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b0406a740 0x7f7b0406abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:03.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.179+0000 7f7b08a00700 1 -- 172.21.15.151:0/1417422297 >> 172.21.15.151:0/1417422297 conn(0x7f7b040788e0 msgr2=0x7f7b04078cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:03.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.179+0000 7f7b08a00700 1 -- 172.21.15.151:0/1417422297 shutdown_connections 2024-04-26T01:57:03.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.180+0000 7f7b08a00700 1 -- 172.21.15.151:0/1417422297 wait complete. 2024-04-26T01:57:03.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.181+0000 7f7b08a00700 1 Processor -- start 2024-04-26T01:57:03.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.181+0000 7f7b08a00700 1 -- start start 2024-04-26T01:57:03.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.181+0000 7f7b08a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7b0406a740 0x7f7b04101a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:03.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.181+0000 7f7b08a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b0406b100 0x7f7b04101f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:03.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.181+0000 7f7b08a00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b04108a60 0x7f7b04102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:03.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.181+0000 7f7b08a00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7b04112c40 con 0x7f7b0406b100 2024-04-26T01:57:03.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.181+0000 7f7b08a00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7b04112ac0 con 0x7f7b0406a740 2024-04-26T01:57:03.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.181+0000 7f7b08a00700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7b04112dc0 con 0x7f7b04108a60 2024-04-26T01:57:03.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.182+0000 7f7b01e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7b0406a740 0x7f7b04101a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:03.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.182+0000 7f7b01e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7b0406a740 0x7f7b04101a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:35162/0 (socket says 172.21.15.151:35162) 2024-04-26T01:57:03.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.182+0000 7f7b01e00700 1 -- 172.21.15.151:0/1407683604 learned_addr learned my addr 172.21.15.151:0/1407683604 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:57:03.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.182+0000 7f7b01400700 1 --2- 172.21.15.151:0/1407683604 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b0406b100 0x7f7b04101f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:03.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.182+0000 7f7b02800700 1 --2- 172.21.15.151:0/1407683604 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b04108a60 0x7f7b04102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:03.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.182+0000 7f7b01e00700 1 -- 172.21.15.151:0/1407683604 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b04108a60 msgr2=0x7f7b04102490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:03.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.182+0000 7f7b01e00700 1 --2- 172.21.15.151:0/1407683604 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b04108a60 0x7f7b04102490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:03.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.182+0000 7f7b01e00700 1 -- 172.21.15.151:0/1407683604 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b0406b100 msgr2=0x7f7b04101f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:03.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.182+0000 7f7b01e00700 1 --2- 172.21.15.151:0/1407683604 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b0406b100 0x7f7b04101f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:03.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.182+0000 7f7b01e00700 1 -- 172.21.15.151:0/1407683604 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b041a7560 con 0x7f7b0406a740 2024-04-26T01:57:03.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.183+0000 7f7b01e00700 1 --2- 172.21.15.151:0/1407683604 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7b0406a740 0x7f7b04101a10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7af400e9c0 tx=0x7f7af400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:03.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.183+0000 7f7afaa00700 1 -- 172.21.15.151:0/1407683604 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af4004a30 con 0x7f7b0406a740 2024-04-26T01:57:03.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.183+0000 7f7afaa00700 1 -- 172.21.15.151:0/1407683604 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7af400cc70 con 0x7f7b0406a740 2024-04-26T01:57:03.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.183+0000 7f7b08a00700 1 -- 172.21.15.151:0/1407683604 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b041a7850 con 0x7f7b0406a740 2024-04-26T01:57:03.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.183+0000 7f7b08a00700 1 -- 172.21.15.151:0/1407683604 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b041a7de0 con 0x7f7b0406a740 2024-04-26T01:57:03.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.183+0000 7f7afaa00700 1 -- 172.21.15.151:0/1407683604 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af4005490 con 0x7f7b0406a740 2024-04-26T01:57:03.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.186+0000 7f7afaa00700 1 -- 172.21.15.151:0/1407683604 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f7af400c7c0 con 0x7f7b0406a740 2024-04-26T01:57:03.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.188+0000 7f7afaa00700 1 --2- 172.21.15.151:0/1407683604 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f7ae80a3ed0 0x7f7ae80a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:03.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.188+0000 7f7afaa00700 1 -- 172.21.15.151:0/1407683604 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f7af40d0670 con 0x7f7b0406a740 2024-04-26T01:57:03.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.188+0000 7f7b13449b80 1 Processor -- start 2024-04-26T01:57:03.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.188+0000 7f7b01400700 1 --2- 172.21.15.151:0/1407683604 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f7ae80a3ed0 0x7f7ae80a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:03.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.189+0000 7f7b13449b80 1 -- start start 2024-04-26T01:57:03.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.189+0000 7f7b13449b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55740798f9b0 0x55740798fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:03.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.189+0000 7f7b13449b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5574079902d0 0x55740799ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:03.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.189+0000 7f7b13449b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55740799b180 0x55740799d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:03.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.189+0000 7f7b13449b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5574078e3020 con 0x55740798f9b0 2024-04-26T01:57:03.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.189+0000 7f7b13449b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5574078898c0 con 0x55740799b180 2024-04-26T01:57:03.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.189+0000 7f7b13449b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5574078defe0 con 0x5574079902d0 2024-04-26T01:57:03.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.189+0000 7f7b01e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55740798f9b0 0x55740798fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:03.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.190+0000 7f7b01400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5574079902d0 0x55740799ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:03.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.190+0000 7f7b01e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55740798f9b0 0x55740798fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52848/0 (socket says 172.21.15.151:52848) 2024-04-26T01:57:03.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.190+0000 7f7b01e00700 1 -- 172.21.15.151:0/2407411885 learned_addr learned my addr 172.21.15.151:0/2407411885 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:57:03.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.190+0000 7f7b02800700 1 --2- 172.21.15.151:0/2407411885 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55740799b180 0x55740799d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:03.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.190+0000 7f7b02800700 1 -- 172.21.15.151:0/2407411885 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5574079902d0 msgr2=0x55740799ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:03.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.190+0000 7f7b02800700 1 --2- 172.21.15.151:0/2407411885 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5574079902d0 0x55740799ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:03.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.190+0000 7f7b02800700 1 -- 172.21.15.151:0/2407411885 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55740798f9b0 msgr2=0x55740798fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:03.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.190+0000 7f7b02800700 1 --2- 172.21.15.151:0/2407411885 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55740798f9b0 0x55740798fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:03.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.190+0000 7f7b02800700 1 -- 172.21.15.151:0/2407411885 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55740787f810 con 0x55740799b180 2024-04-26T01:57:03.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.190+0000 7f7b01e00700 1 --2- 172.21.15.151:0/2407411885 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55740798f9b0 0x55740798fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:57:03.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.190+0000 7f7b01400700 1 --2- 172.21.15.151:0/1407683604 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f7ae80a3ed0 0x7f7ae80a6390 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7af0009680 tx=0x7f7af0011910 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:03.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.191+0000 7f7b02800700 1 --2- 172.21.15.151:0/2407411885 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55740799b180 0x55740799d570 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7afc00b6c0 tx=0x7f7afc0092e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f703425711e781a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:03.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.191+0000 7f7aef400700 1 -- 172.21.15.151:0/2407411885 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7afc020070 con 0x55740799b180 2024-04-26T01:57:03.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.191+0000 7f7aef400700 1 -- 172.21.15.151:0/2407411885 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7afc003c00 con 0x55740799b180 2024-04-26T01:57:03.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.191+0000 7f7aef400700 1 -- 172.21.15.151:0/2407411885 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7afc007ad0 con 0x55740799b180 2024-04-26T01:57:03.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.192+0000 7f7b13449b80 1 -- 172.21.15.151:0/2407411885 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55740799b180 msgr2=0x55740799d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:03.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.192+0000 7f7b13449b80 1 --2- 172.21.15.151:0/2407411885 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55740799b180 0x55740799d570 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7afc00b6c0 tx=0x7f7afc0092e0 comp rx=0 tx=0).stop 2024-04-26T01:57:03.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.192+0000 7f7b13449b80 1 -- 172.21.15.151:0/2407411885 shutdown_connections 2024-04-26T01:57:03.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.192+0000 7f7b13449b80 1 --2- 172.21.15.151:0/2407411885 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55740799b180 0x55740799d570 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:03.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.192+0000 7f7b13449b80 1 --2- 172.21.15.151:0/2407411885 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55740798f9b0 0x55740798fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:03.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.192+0000 7f7b13449b80 1 --2- 172.21.15.151:0/2407411885 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5574079902d0 0x55740799ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:03.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.192+0000 7f7b13449b80 1 -- 172.21.15.151:0/2407411885 >> 172.21.15.151:0/2407411885 conn(0x5574078be730 msgr2=0x55740798be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:03.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.192+0000 7f7b13449b80 1 -- 172.21.15.151:0/2407411885 shutdown_connections 2024-04-26T01:57:03.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.192+0000 7f7b13449b80 1 -- 172.21.15.151:0/2407411885 wait complete. 2024-04-26T01:57:03.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.193+0000 7f7b13449b80 1 -- start start 2024-04-26T01:57:03.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.193+0000 7f7b13449b80 1 Processor -- start 2024-04-26T01:57:03.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.194+0000 7f7b13449b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55740798f9b0 0x55740799a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:03.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.194+0000 7f7b13449b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5574079902d0 0x557407997b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:03.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.194+0000 7f7b13449b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55740799b180 0x557407998090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:03.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.194+0000 7f7b13449b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55740799dc40 con 0x55740798f9b0 2024-04-26T01:57:03.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.194+0000 7f7b13449b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55740789c9f0 con 0x5574079902d0 2024-04-26T01:57:03.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.194+0000 7f7b13449b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55740799ddc0 con 0x55740799b180 2024-04-26T01:57:03.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.194+0000 7f7b13449b80 10 client.0 _ll_register_callbacks cb 0x55740787b8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:57:03.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.194+0000 7f7b01e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55740798f9b0 0x55740799a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:03.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.194+0000 7f7b01e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55740798f9b0 0x55740799a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52856/0 (socket says 172.21.15.151:52856) 2024-04-26T01:57:03.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.194+0000 7f7b01e00700 1 -- 172.21.15.151:0/1047854656 learned_addr learned my addr 172.21.15.151:0/1047854656 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:57:03.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.194+0000 7f7b01400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5574079902d0 0x557407997b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:03.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.195+0000 7f7b01400700 1 -- 172.21.15.151:0/1047854656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55740799b180 msgr2=0x557407998090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T01:57:03.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.195+0000 7f7b01400700 1 --2- 172.21.15.151:0/1047854656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55740799b180 0x557407998090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:03.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.195+0000 7f7b01400700 1 -- 172.21.15.151:0/1047854656 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55740798f9b0 msgr2=0x55740799a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:03.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.195+0000 7f7b01400700 1 --2- 172.21.15.151:0/1047854656 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55740798f9b0 0x55740799a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:03.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.195+0000 7f7b01400700 1 -- 172.21.15.151:0/1047854656 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557407830a60 con 0x5574079902d0 2024-04-26T01:57:03.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.195+0000 7f7b01400700 1 --2- 172.21.15.151:0/1047854656 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5574079902d0 0x557407997b50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7b040fff40 tx=0x7f7af00047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f50a9763408b7ac0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:03.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.195+0000 7f7ae3e00700 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T01:57:03.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.195+0000 7f7ae3e00700 1 -- 172.21.15.151:0/1047854656 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af001a040 con 0x5574079902d0 2024-04-26T01:57:03.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.195+0000 7f7b13449b80 1 -- 172.21.15.151:0/1047854656 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7afc003c00 con 0x5574079902d0 2024-04-26T01:57:03.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.195+0000 7f7ae3e00700 1 -- 172.21.15.151:0/1047854656 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f7af000f7a0 con 0x5574079902d0 2024-04-26T01:57:03.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.196+0000 7f7ae3e00700 1 -- 172.21.15.151:0/1047854656 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7af000f9a0 con 0x5574079902d0 2024-04-26T01:57:03.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.196+0000 7f7b13449b80 10 client.25583 fetch_fsmap learned FSMap version 28 2024-04-26T01:57:03.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.196+0000 7f7b13449b80 1 -- 172.21.15.151:0/1047854656 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557407879c80 con 0x5574079902d0 2024-04-26T01:57:03.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.196+0000 7f7ae3e00700 1 -- 172.21.15.151:0/1047854656 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7afc003c00 con 0x5574079902d0 2024-04-26T01:57:03.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.196+0000 7f7ae3e00700 1 -- 172.21.15.151:0/1047854656 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f7af001d030 con 0x5574079902d0 2024-04-26T01:57:03.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.196+0000 7f7b13449b80 10 client.25583 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T01:57:03.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.196+0000 7f7b13449b80 10 client.25583 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-26T01:57:03.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.196+0000 7f7b13449b80 20 client.25583 populate_metadata read hostname 'smithi151' 2024-04-26T01:57:03.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.196+0000 7f7b13449b80 1 --2- 172.21.15.151:0/1047854656 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5574079aaa10 0x5574079ace20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:03.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.196+0000 7f7b13449b80 4 client.25583 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:57:03.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.196+0000 7f7b13449b80 1 -- 172.21.15.151:0/1047854656 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574078f0810 con 0x5574079aaa10 2024-04-26T01:57:03.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.197+0000 7f7b01e00700 1 --2- 172.21.15.151:0/1047854656 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5574079aaa10 0x5574079ace20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:03.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.197+0000 7f7b01e00700 1 --2- 172.21.15.151:0/1047854656 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5574079aaa10 0x5574079ace20 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18bc53ef0563eb1c server_cookie=33868663bba8bd0f in_seq=0 out_seq=0 2024-04-26T01:57:03.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.197+0000 7f7ae3e00700 10 client.25583 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:57:03.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.198+0000 7f7ae3e00700 1 -- 172.21.15.151:0/1047854656 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5574078f0810 con 0x5574079aaa10 2024-04-26T01:57:03.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.198+0000 7f7ae3e00700 10 client.25583 handle_command_reply: tid=0 2024-04-26T01:57:03.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.198+0000 7f7b13449b80 1 client.25583 shutdown 2024-04-26T01:57:03.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.198+0000 7f7b13449b80 1 -- 172.21.15.151:0/1047854656 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5574079902d0 msgr2=0x557407997b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:03.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.198+0000 7f7b13449b80 1 --2- 172.21.15.151:0/1047854656 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5574079902d0 0x557407997b50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7b040fff40 tx=0x7f7af00047c0 comp rx=0 tx=0).stop 2024-04-26T01:57:03.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.198+0000 7f7b13449b80 1 -- 172.21.15.151:0/1047854656 shutdown_connections 2024-04-26T01:57:03.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.198+0000 7f7b13449b80 1 --2- 172.21.15.151:0/1047854656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55740799b180 0x557407998090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:03.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.198+0000 7f7b13449b80 1 --2- 172.21.15.151:0/1047854656 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55740798f9b0 0x55740799a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:03.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.198+0000 7f7b13449b80 1 --2- 172.21.15.151:0/1047854656 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5574079aaa10 0x5574079ace20 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:03.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.198+0000 7f7b13449b80 1 --2- 172.21.15.151:0/1047854656 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5574079902d0 0x557407997b50 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:03.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.198+0000 7f7b13449b80 1 -- 172.21.15.151:0/1047854656 >> 172.21.15.151:0/1047854656 conn(0x5574078be730 msgr2=0x55740798ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:03.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.198+0000 7f7ae3e00700 0 client.25583 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:57:03.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.199+0000 7f7b13449b80 1 -- 172.21.15.151:0/1047854656 shutdown_connections 2024-04-26T01:57:03.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.199+0000 7f7b13449b80 1 -- 172.21.15.151:0/1047854656 wait complete. 2024-04-26T01:57:03.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.199+0000 7f7b13449b80 20 client.25583 trim_cache size 0 max 16384 2024-04-26T01:57:03.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.221+0000 7f7b13449b80 1 Processor -- start 2024-04-26T01:57:03.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.221+0000 7f7b13449b80 1 -- start start 2024-04-26T01:57:03.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.221+0000 7f7b13449b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55740798f9b0 0x5574079b2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:03.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.221+0000 7f7b13449b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5574079902d0 0x5574079b2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:03.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.221+0000 7f7b13449b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55740799b180 0x557407994dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:03.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.221+0000 7f7b13449b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae400fcc0 con 0x55740798f9b0 2024-04-26T01:57:03.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.221+0000 7f7b13449b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae400d550 con 0x55740799b180 2024-04-26T01:57:03.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.221+0000 7f7b13449b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae400b3a0 con 0x5574079902d0 2024-04-26T01:57:03.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.222+0000 7f7b02800700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55740799b180 0x557407994dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:03.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.222+0000 7f7b01400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5574079902d0 0x5574079b2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:03.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.222+0000 7f7b02800700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55740799b180 0x557407994dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:35188/0 (socket says 172.21.15.151:35188) 2024-04-26T01:57:03.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.222+0000 7f7b02800700 1 -- 172.21.15.151:0/391742039 learned_addr learned my addr 172.21.15.151:0/391742039 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:57:03.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.222+0000 7f7b01400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5574079902d0 0x5574079b2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:36910/0 (socket says 172.21.15.151:36910) 2024-04-26T01:57:03.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.222+0000 7f7b01e00700 1 --2- 172.21.15.151:0/391742039 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55740798f9b0 0x5574079b2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:03.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.222+0000 7f7b02800700 1 -- 172.21.15.151:0/391742039 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5574079902d0 msgr2=0x5574079b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:03.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.222+0000 7f7b02800700 1 --2- 172.21.15.151:0/391742039 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5574079902d0 0x5574079b2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:03.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.222+0000 7f7b02800700 1 -- 172.21.15.151:0/391742039 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55740798f9b0 msgr2=0x5574079b2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:03.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.222+0000 7f7b02800700 1 --2- 172.21.15.151:0/391742039 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55740798f9b0 0x5574079b2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:03.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.222+0000 7f7b02800700 1 -- 172.21.15.151:0/391742039 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574079ad4a0 con 0x55740799b180 2024-04-26T01:57:03.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.222+0000 7f7b01e00700 1 --2- 172.21.15.151:0/391742039 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55740798f9b0 0x5574079b2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:57:03.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.223+0000 7f7b02800700 1 --2- 172.21.15.151:0/391742039 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55740799b180 0x557407994dd0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7afc009960 tx=0x7f7afc00bb80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=634f8e206abb235c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:03.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.223+0000 7f7aeea00700 1 -- 172.21.15.151:0/391742039 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7afc020030 con 0x55740799b180 2024-04-26T01:57:03.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.223+0000 7f7aeea00700 1 -- 172.21.15.151:0/391742039 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7afc0050c0 con 0x55740799b180 2024-04-26T01:57:03.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.223+0000 7f7aeea00700 1 -- 172.21.15.151:0/391742039 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7afc007640 con 0x55740799b180 2024-04-26T01:57:03.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.224+0000 7f7b13449b80 1 -- 172.21.15.151:0/391742039 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55740799b180 msgr2=0x557407994dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:03.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.224+0000 7f7b13449b80 1 --2- 172.21.15.151:0/391742039 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55740799b180 0x557407994dd0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7afc009960 tx=0x7f7afc00bb80 comp rx=0 tx=0).stop 2024-04-26T01:57:03.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.224+0000 7f7b13449b80 1 -- 172.21.15.151:0/391742039 shutdown_connections 2024-04-26T01:57:03.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.224+0000 7f7b13449b80 1 --2- 172.21.15.151:0/391742039 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55740799b180 0x557407994dd0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:03.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.224+0000 7f7b13449b80 1 --2- 172.21.15.151:0/391742039 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55740798f9b0 0x5574079b2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:03.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.224+0000 7f7b13449b80 1 --2- 172.21.15.151:0/391742039 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5574079902d0 0x5574079b2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:03.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.224+0000 7f7b13449b80 1 -- 172.21.15.151:0/391742039 >> 172.21.15.151:0/391742039 conn(0x5574078be730 msgr2=0x55740798b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:03.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.224+0000 7f7b13449b80 1 -- 172.21.15.151:0/391742039 shutdown_connections 2024-04-26T01:57:03.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.224+0000 7f7b13449b80 1 -- 172.21.15.151:0/391742039 wait complete. 2024-04-26T01:57:03.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.224+0000 7f7b13449b80 1 -- start start 2024-04-26T01:57:03.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.225+0000 7f7b13449b80 1 Processor -- start 2024-04-26T01:57:03.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.225+0000 7f7b13449b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55740798f9b0 0x5574079a3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:03.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.225+0000 7f7b13449b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574079902d0 0x5574079a1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:03.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.225+0000 7f7b13449b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55740799b180 0x5574079a1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:03.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.225+0000 7f7b13449b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae400ba10 con 0x5574079902d0 2024-04-26T01:57:03.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.225+0000 7f7b13449b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae400ce80 con 0x55740799b180 2024-04-26T01:57:03.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.225+0000 7f7b13449b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae400c0e0 con 0x55740798f9b0 2024-04-26T01:57:03.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.225+0000 7f7b13449b80 10 client.0 _ll_register_callbacks cb 0x55740787b8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:57:03.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.225+0000 7f7b01e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55740798f9b0 0x5574079a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:03.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.225+0000 7f7b01e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55740798f9b0 0x5574079a3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:36922/0 (socket says 172.21.15.151:36922) 2024-04-26T01:57:03.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.225+0000 7f7b02800700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55740799b180 0x5574079a1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:03.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.225+0000 7f7b01e00700 1 -- 172.21.15.151:0/4231114390 learned_addr learned my addr 172.21.15.151:0/4231114390 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:57:03.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.225+0000 7f7b01400700 1 --2- 172.21.15.151:0/4231114390 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574079902d0 0x5574079a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:03.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.226+0000 7f7b01e00700 1 -- 172.21.15.151:0/4231114390 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55740799b180 msgr2=0x5574079a1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:03.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.226+0000 7f7b01e00700 1 --2- 172.21.15.151:0/4231114390 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55740799b180 0x5574079a1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:03.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.226+0000 7f7b01e00700 1 -- 172.21.15.151:0/4231114390 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574079902d0 msgr2=0x5574079a1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:03.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.226+0000 7f7b01e00700 1 --2- 172.21.15.151:0/4231114390 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574079902d0 0x5574079a1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:03.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.226+0000 7f7b01e00700 1 -- 172.21.15.151:0/4231114390 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574079a2700 con 0x55740798f9b0 2024-04-26T01:57:03.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.226+0000 7f7b02800700 1 --2- 172.21.15.151:0/4231114390 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55740799b180 0x5574079a1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:57:03.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.226+0000 7f7b01e00700 1 --2- 172.21.15.151:0/4231114390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55740798f9b0 0x5574079a3550 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7af4098ff0 tx=0x7f7af4009550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1a30b8ca80df5b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:03.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.226+0000 7f7ae3e00700 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-26T01:57:03.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.226+0000 7f7ae3e00700 1 -- 172.21.15.151:0/4231114390 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af409cd20 con 0x55740798f9b0 2024-04-26T01:57:03.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.226+0000 7f7ae3e00700 1 -- 172.21.15.151:0/4231114390 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f7af4016330 con 0x55740798f9b0 2024-04-26T01:57:03.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.226+0000 7f7b13449b80 1 -- 172.21.15.151:0/4231114390 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7afc0050c0 con 0x55740798f9b0 2024-04-26T01:57:03.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.227+0000 7f7ae3e00700 1 -- 172.21.15.151:0/4231114390 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7af4016530 con 0x55740798f9b0 2024-04-26T01:57:03.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.227+0000 7f7b13449b80 10 client.25618 fetch_fsmap learned FSMap version 28 2024-04-26T01:57:03.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.227+0000 7f7b13449b80 1 -- 172.21.15.151:0/4231114390 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557407a12100 con 0x55740798f9b0 2024-04-26T01:57:03.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.227+0000 7f7ae3e00700 1 -- 172.21.15.151:0/4231114390 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7afc0050c0 con 0x55740798f9b0 2024-04-26T01:57:03.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.227+0000 7f7ae3e00700 1 -- 172.21.15.151:0/4231114390 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f7af409c910 con 0x55740798f9b0 2024-04-26T01:57:03.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.227+0000 7f7b13449b80 10 client.25618 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T01:57:03.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.227+0000 7f7b13449b80 10 client.25618 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-26T01:57:03.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.227+0000 7f7b13449b80 20 client.25618 populate_metadata read hostname 'smithi151' 2024-04-26T01:57:03.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.227+0000 7f7b13449b80 1 --2- 172.21.15.151:0/4231114390 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x557407a12860 0x557407a14c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:03.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.227+0000 7f7b13449b80 4 client.25618 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 90.0, "await_for": 42.0}] 2024-04-26T01:57:03.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.227+0000 7f7b13449b80 1 -- 172.21.15.151:0/4231114390 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 90.0, "await_for": 42.0}) -- 0x5574078ea8b0 con 0x557407a12860 2024-04-26T01:57:03.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.227+0000 7f7b01400700 1 --2- 172.21.15.151:0/4231114390 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x557407a12860 0x557407a14c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:03.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.228+0000 7f7ae3e00700 10 client.25618 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:57:03.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:03.228+0000 7f7b01400700 1 --2- 172.21.15.151:0/4231114390 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x557407a12860 0x557407a14c80 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8b191c6c0efc32a server_cookie=e8759c49e0fccbf9 in_seq=0 out_seq=0 2024-04-26T01:57:04.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:04.308+0000 7f7ae3e00700 1 -- 172.21.15.151:0/4231114390 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5574078ea8b0 con 0x557407a12860 2024-04-26T01:57:04.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:04.308+0000 7f7ae3e00700 10 client.25618 handle_command_reply: tid=0 2024-04-26T01:57:04.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:04.308+0000 7f7b13449b80 1 client.25618 shutdown 2024-04-26T01:57:04.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:04.309+0000 7f7b13449b80 1 -- 172.21.15.151:0/4231114390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55740798f9b0 msgr2=0x5574079a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:04.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:04.309+0000 7f7b13449b80 1 --2- 172.21.15.151:0/4231114390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55740798f9b0 0x5574079a3550 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7af4098ff0 tx=0x7f7af4009550 comp rx=0 tx=0).stop 2024-04-26T01:57:04.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:04.309+0000 7f7b13449b80 1 -- 172.21.15.151:0/4231114390 shutdown_connections 2024-04-26T01:57:04.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:04.309+0000 7f7b13449b80 1 --2- 172.21.15.151:0/4231114390 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x557407a12860 0x557407a14c80 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:04.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:04.309+0000 7f7b13449b80 1 --2- 172.21.15.151:0/4231114390 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55740799b180 0x5574079a1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:04.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:04.309+0000 7f7b13449b80 1 --2- 172.21.15.151:0/4231114390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55740798f9b0 0x5574079a3550 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:04.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:04.309+0000 7f7b13449b80 1 --2- 172.21.15.151:0/4231114390 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574079902d0 0x5574079a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:04.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:04.309+0000 7f7b13449b80 1 -- 172.21.15.151:0/4231114390 >> 172.21.15.151:0/4231114390 conn(0x5574078be730 msgr2=0x557407999380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:04.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:04.309+0000 7f7ae3e00700 0 client.25618 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:57:04.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:04.309+0000 7f7b13449b80 1 -- 172.21.15.151:0/4231114390 shutdown_connections 2024-04-26T01:57:04.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:04.309+0000 7f7b13449b80 1 -- 172.21.15.151:0/4231114390 wait complete. 2024-04-26T01:57:04.310 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-26T01:57:04.311 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 28, 2024-04-26T01:57:04.311 INFO:teuthology.orchestra.run.smithi151.stdout: "leader": 24509, 2024-04-26T01:57:04.311 INFO:teuthology.orchestra.run.smithi151.stdout: "set_version": 10, 2024-04-26T01:57:04.311 INFO:teuthology.orchestra.run.smithi151.stdout: "sets": { 2024-04-26T01:57:04.311 INFO:teuthology.orchestra.run.smithi151.stdout: "4b386810": { 2024-04-26T01:57:04.311 INFO:teuthology.orchestra.run.smithi151.stdout: "version": 10, 2024-04-26T01:57:04.311 INFO:teuthology.orchestra.run.smithi151.stdout: "age_ref": 0.0, 2024-04-26T01:57:04.311 INFO:teuthology.orchestra.run.smithi151.stdout: "state": { 2024-04-26T01:57:04.311 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "QUIESCED", 2024-04-26T01:57:04.311 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 0.0 2024-04-26T01:57:04.311 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T01:57:04.311 INFO:teuthology.orchestra.run.smithi151.stdout: "timeout": 60.0, 2024-04-26T01:57:04.311 INFO:teuthology.orchestra.run.smithi151.stdout: "expiration": 90.0, 2024-04-26T01:57:04.312 INFO:teuthology.orchestra.run.smithi151.stdout: "members": { 2024-04-26T01:57:04.312 INFO:teuthology.orchestra.run.smithi151.stdout: "file:/": { 2024-04-26T01:57:04.312 INFO:teuthology.orchestra.run.smithi151.stdout: "excluded": false, 2024-04-26T01:57:04.312 INFO:teuthology.orchestra.run.smithi151.stdout: "state": { 2024-04-26T01:57:04.312 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "QUIESCED", 2024-04-26T01:57:04.312 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 0.0 2024-04-26T01:57:04.312 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T01:57:04.312 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T01:57:04.312 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T01:57:04.312 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T01:57:04.312 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T01:57:04.312 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-26T01:57:04.313 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:04.310+0000 7f7b13449b80 20 client.25618 trim_cache size 0 max 16384 2024-04-26T01:57:04.313 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:04.310+0000 7f7b08a00700 1 -- 172.21.15.151:0/1407683604 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f7ae80a3ed0 msgr2=0x7f7ae80a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:04.313 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:04.310+0000 7f7b08a00700 1 --2- 172.21.15.151:0/1407683604 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f7ae80a3ed0 0x7f7ae80a6390 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7af0009680 tx=0x7f7af0011910 comp rx=0 tx=0).stop 2024-04-26T01:57:04.313 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:04.310+0000 7f7b08a00700 1 -- 172.21.15.151:0/1407683604 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7b0406a740 msgr2=0x7f7b04101a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:04.313 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:04.311+0000 7f7b08a00700 1 --2- 172.21.15.151:0/1407683604 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7b0406a740 0x7f7b04101a10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7af400e9c0 tx=0x7f7af400ecd0 comp rx=0 tx=0).stop 2024-04-26T01:57:04.313 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:04.311+0000 7f7b08a00700 1 -- 172.21.15.151:0/1407683604 shutdown_connections 2024-04-26T01:57:04.313 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:04.311+0000 7f7b08a00700 1 --2- 172.21.15.151:0/1407683604 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f7ae80a3ed0 0x7f7ae80a6390 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:04.313 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:04.311+0000 7f7b08a00700 1 --2- 172.21.15.151:0/1407683604 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b04108a60 0x7f7b04102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:04.313 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:04.311+0000 7f7b08a00700 1 --2- 172.21.15.151:0/1407683604 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7b0406a740 0x7f7b04101a10 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:04.313 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:04.311+0000 7f7b08a00700 1 --2- 172.21.15.151:0/1407683604 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b0406b100 0x7f7b04101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:04.313 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:04.311+0000 7f7b08a00700 1 -- 172.21.15.151:0/1407683604 >> 172.21.15.151:0/1407683604 conn(0x7f7b040788e0 msgr2=0x7f7b04100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:04.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:04.311+0000 7f7b08a00700 1 -- 172.21.15.151:0/1407683604 shutdown_connections 2024-04-26T01:57:04.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:04.311+0000 7f7b08a00700 1 -- 172.21.15.151:0/1407683604 wait complete. 2024-04-26T01:57:04.535 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:57:04 smithi151 ceph-mon[28057]: pgmap v106: 97 pgs: 97 active+clean; 32 MiB data, 602 MiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 186 KiB/s wr, 54 op/s 2024-04-26T01:57:04.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:57:04 smithi172 ceph-mon[32785]: pgmap v106: 97 pgs: 97 active+clean; 32 MiB data, 602 MiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 186 KiB/s wr, 54 op/s 2024-04-26T01:57:04.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:57:04 smithi165 ceph-mon[32648]: pgmap v106: 97 pgs: 97 active+clean; 32 MiB data, 602 MiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 186 KiB/s wr, 54 op/s 2024-04-26T01:57:05.363 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '4b386810', quiesce took 9.8 seconds. Will release after: 30.0 2024-04-26T01:57:06.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:57:06 smithi172 ceph-mon[32785]: pgmap v107: 97 pgs: 97 active+clean; 33 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 750 KiB/s wr, 147 op/s 2024-04-26T01:57:06.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:57:06 smithi165 ceph-mon[32648]: pgmap v107: 97 pgs: 97 active+clean; 33 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 750 KiB/s wr, 147 op/s 2024-04-26T01:57:06.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:57:06 smithi151 ceph-mon[28057]: pgmap v107: 97 pgs: 97 active+clean; 33 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 750 KiB/s wr, 147 op/s 2024-04-26T01:57:08.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:57:08 smithi172 ceph-mon[32785]: pgmap v108: 97 pgs: 97 active+clean; 33 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 750 KiB/s wr, 146 op/s 2024-04-26T01:57:08.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:57:08 smithi165 ceph-mon[32648]: pgmap v108: 97 pgs: 97 active+clean; 33 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 750 KiB/s wr, 146 op/s 2024-04-26T01:57:08.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:57:08 smithi151 ceph-mon[28057]: pgmap v108: 97 pgs: 97 active+clean; 33 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 750 KiB/s wr, 146 op/s 2024-04-26T01:57:10.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:57:10 smithi172 ceph-mon[32785]: pgmap v109: 97 pgs: 97 active+clean; 35 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 1.0 MiB/s wr, 176 op/s 2024-04-26T01:57:10.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:57:10 smithi165 ceph-mon[32648]: pgmap v109: 97 pgs: 97 active+clean; 35 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 1.0 MiB/s wr, 176 op/s 2024-04-26T01:57:10.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:57:10 smithi151 ceph-mon[28057]: pgmap v109: 97 pgs: 97 active+clean; 35 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 1.0 MiB/s wr, 176 op/s 2024-04-26T01:57:12.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:57:12 smithi172 ceph-mon[32785]: pgmap v110: 97 pgs: 97 active+clean; 35 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 1023 KiB/s wr, 169 op/s 2024-04-26T01:57:12.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:57:12 smithi165 ceph-mon[32648]: pgmap v110: 97 pgs: 97 active+clean; 35 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 1023 KiB/s wr, 169 op/s 2024-04-26T01:57:12.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:57:12 smithi151 ceph-mon[28057]: pgmap v110: 97 pgs: 97 active+clean; 35 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 1023 KiB/s wr, 169 op/s 2024-04-26T01:57:13.049 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.050+0000 7f4298a00640 1 -- 172.21.15.151:0/2600221859 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4294104bb0 msgr2=0x7f4294108bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:13.049 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.050+0000 7f4298a00640 1 --2- 172.21.15.151:0/2600221859 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4294104bb0 0x7f4294108bf0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f427c009900 tx=0x7f427c02a3f0 comp rx=0 tx=0).stop 2024-04-26T01:57:13.049 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.050+0000 7f4298a00640 1 -- 172.21.15.151:0/2600221859 shutdown_connections 2024-04-26T01:57:13.049 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.050+0000 7f4298a00640 1 --2- 172.21.15.151:0/2600221859 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f42941092d0 0x7f4294110ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:13.049 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.050+0000 7f4298a00640 1 --2- 172.21.15.151:0/2600221859 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4294104bb0 0x7f4294108bf0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:13.049 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.050+0000 7f4298a00640 1 --2- 172.21.15.151:0/2600221859 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4294104200 0x7f42941045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:13.050 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.050+0000 7f4298a00640 1 -- 172.21.15.151:0/2600221859 >> 172.21.15.151:0/2600221859 conn(0x7f42940ffa60 msgr2=0x7f4294101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:13.050 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.050+0000 7f4298a00640 1 -- 172.21.15.151:0/2600221859 shutdown_connections 2024-04-26T01:57:13.050 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.050+0000 7f4298a00640 1 -- 172.21.15.151:0/2600221859 wait complete. 2024-04-26T01:57:13.052 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.051+0000 7f4298a00640 1 Processor -- start 2024-04-26T01:57:13.052 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.051+0000 7f4298a00640 1 -- start start 2024-04-26T01:57:13.052 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.052+0000 7f4298a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4294104200 0x7f42941a0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:13.052 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.052+0000 7f4298a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42941092d0 0x7f42941a0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:13.052 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.052+0000 7f4298a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f429419a600 0x7f429419aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:13.052 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.052+0000 7f4298a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4294114260 con 0x7f4294104200 2024-04-26T01:57:13.052 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.052+0000 7f4298a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f42941140e0 con 0x7f429419a600 2024-04-26T01:57:13.052 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.052+0000 7f4298a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f42941143e0 con 0x7f42941092d0 2024-04-26T01:57:13.052 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.052+0000 7f4293400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4294104200 0x7f42941a0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:13.053 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.052+0000 7f4293400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4294104200 0x7f42941a0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52172/0 (socket says 172.21.15.151:52172) 2024-04-26T01:57:13.053 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.052+0000 7f4293400640 1 -- 172.21.15.151:0/2703737760 learned_addr learned my addr 172.21.15.151:0/2703737760 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:57:13.053 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.052+0000 7f428be00640 1 --2- 172.21.15.151:0/2703737760 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42941092d0 0x7f42941a0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:13.053 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.052+0000 7f4293e00640 1 --2- 172.21.15.151:0/2703737760 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f429419a600 0x7f429419aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:13.053 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.053+0000 7f4293e00640 1 -- 172.21.15.151:0/2703737760 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42941092d0 msgr2=0x7f42941a0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:13.053 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.053+0000 7f4293e00640 1 --2- 172.21.15.151:0/2703737760 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42941092d0 0x7f42941a0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:13.053 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.053+0000 7f4293e00640 1 -- 172.21.15.151:0/2703737760 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4294104200 msgr2=0x7f42941a0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:13.053 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.053+0000 7f4293e00640 1 --2- 172.21.15.151:0/2703737760 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4294104200 0x7f42941a0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:13.053 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.053+0000 7f4293e00640 1 -- 172.21.15.151:0/2703737760 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f429419b120 con 0x7f429419a600 2024-04-26T01:57:13.053 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.053+0000 7f4293400640 1 --2- 172.21.15.151:0/2703737760 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4294104200 0x7f42941a0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:57:13.053 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.053+0000 7f4293e00640 1 --2- 172.21.15.151:0/2703737760 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f429419a600 0x7f429419aab0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f428c00ec50 tx=0x7f428c00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:13.053 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.053+0000 7f4290c00640 1 -- 172.21.15.151:0/2703737760 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f428c003b70 con 0x7f429419a600 2024-04-26T01:57:13.054 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.053+0000 7f4298a00640 1 -- 172.21.15.151:0/2703737760 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f429419b410 con 0x7f429419a600 2024-04-26T01:57:13.054 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.053+0000 7f4290c00640 1 -- 172.21.15.151:0/2703737760 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f428c003d10 con 0x7f429419a600 2024-04-26T01:57:13.054 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.053+0000 7f4290c00640 1 -- 172.21.15.151:0/2703737760 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f428c010830 con 0x7f429419a600 2024-04-26T01:57:13.054 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.053+0000 7f4298a00640 1 -- 172.21.15.151:0/2703737760 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42941a7610 con 0x7f429419a600 2024-04-26T01:57:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.057+0000 7f429cda0740 1 Processor -- start 2024-04-26T01:57:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.057+0000 7f4290c00640 1 -- 172.21.15.151:0/2703737760 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f428c003ec0 con 0x7f429419a600 2024-04-26T01:57:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.057+0000 7f429cda0740 1 -- start start 2024-04-26T01:57:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.057+0000 7f429cda0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf3e3da9b0 0x55cf3e3dad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.057+0000 7f429cda0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf3e3db2d0 0x55cf3e3e5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.057+0000 7f429cda0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf3e3e5b90 0x55cf3e3e7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.057+0000 7f429cda0740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55cf3e340900 con 0x55cf3e3da9b0 2024-04-26T01:57:13.058 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.057+0000 7f429cda0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55cf3e2e6ef0 con 0x55cf3e3db2d0 2024-04-26T01:57:13.058 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.057+0000 7f429cda0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55cf3e33c8c0 con 0x55cf3e3e5b90 2024-04-26T01:57:13.058 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.058+0000 7f428be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf3e3da9b0 0x55cf3e3dad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:13.058 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.058+0000 7f428be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf3e3da9b0 0x55cf3e3dad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52176/0 (socket says 172.21.15.151:52176) 2024-04-26T01:57:13.058 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.058+0000 7f428be00640 1 -- 172.21.15.151:0/111339868 learned_addr learned my addr 172.21.15.151:0/111339868 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:57:13.058 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.058+0000 7f4293400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf3e3db2d0 0x55cf3e3e5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:13.058 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.058+0000 7f428be00640 1 --2- 172.21.15.151:0/111339868 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf3e3e5b90 0x55cf3e3e7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:13.058 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.058+0000 7f4293400640 1 -- 172.21.15.151:0/111339868 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf3e3e5b90 msgr2=0x55cf3e3e7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:13.058 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.058+0000 7f4293400640 1 --2- 172.21.15.151:0/111339868 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf3e3e5b90 0x55cf3e3e7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:13.058 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.058+0000 7f4293400640 1 -- 172.21.15.151:0/111339868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf3e3da9b0 msgr2=0x55cf3e3dad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:13.058 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.058+0000 7f4293400640 1 --2- 172.21.15.151:0/111339868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf3e3da9b0 0x55cf3e3dad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:13.059 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.058+0000 7f4293400640 1 -- 172.21.15.151:0/111339868 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf3e053210 con 0x55cf3e3db2d0 2024-04-26T01:57:13.059 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.058+0000 7f428be00640 1 --2- 172.21.15.151:0/111339868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf3e3da9b0 0x55cf3e3dad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:57:13.059 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.058+0000 7f4293400640 1 --2- 172.21.15.151:0/111339868 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf3e3db2d0 0x55cf3e3e5650 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4284011510 tx=0x7f42840119d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f468d3e64ad9a5d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:13.059 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.058+0000 7f4290c00640 1 --2- 172.21.15.151:0/2703737760 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f42780a3d10 0x7f42780a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:13.059 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.059+0000 7f4283e00640 1 -- 172.21.15.151:0/111339868 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f428401e070 con 0x55cf3e3db2d0 2024-04-26T01:57:13.059 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.059+0000 7f4283e00640 1 -- 172.21.15.151:0/111339868 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f428400fa70 con 0x55cf3e3db2d0 2024-04-26T01:57:13.059 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.059+0000 7f4283e00640 1 -- 172.21.15.151:0/111339868 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4284019b60 con 0x55cf3e3db2d0 2024-04-26T01:57:13.059 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.059+0000 7f4290c00640 1 -- 172.21.15.151:0/2703737760 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f428c0d0c90 con 0x7f429419a600 2024-04-26T01:57:13.059 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.059+0000 7f428be00640 1 --2- 172.21.15.151:0/2703737760 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f42780a3d10 0x7f42780a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:13.063 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.059+0000 7f429cda0740 1 -- 172.21.15.151:0/111339868 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf3e3db2d0 msgr2=0x55cf3e3e5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:13.063 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.059+0000 7f429cda0740 1 --2- 172.21.15.151:0/111339868 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf3e3db2d0 0x55cf3e3e5650 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4284011510 tx=0x7f42840119d0 comp rx=0 tx=0).stop 2024-04-26T01:57:13.063 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.059+0000 7f429cda0740 1 -- 172.21.15.151:0/111339868 shutdown_connections 2024-04-26T01:57:13.063 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.059+0000 7f429cda0740 1 --2- 172.21.15.151:0/111339868 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf3e3e5b90 0x55cf3e3e7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:13.064 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.059+0000 7f429cda0740 1 --2- 172.21.15.151:0/111339868 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf3e3db2d0 0x55cf3e3e5650 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:13.064 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.059+0000 7f429cda0740 1 --2- 172.21.15.151:0/111339868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf3e3da9b0 0x55cf3e3dad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:13.064 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.059+0000 7f429cda0740 1 -- 172.21.15.151:0/111339868 >> 172.21.15.151:0/111339868 conn(0x55cf3e3d6ec0 msgr2=0x55cf3e3d72a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:13.064 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.059+0000 7f428be00640 1 --2- 172.21.15.151:0/2703737760 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f42780a3d10 0x7f42780a61d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f427c000c00 tx=0x7f427c005e50 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:13.064 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.059+0000 7f429cda0740 1 -- 172.21.15.151:0/111339868 shutdown_connections 2024-04-26T01:57:13.064 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.060+0000 7f429cda0740 1 -- 172.21.15.151:0/111339868 wait complete. 2024-04-26T01:57:13.064 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.060+0000 7f429cda0740 1 -- start start 2024-04-26T01:57:13.064 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.060+0000 7f429cda0740 1 Processor -- start 2024-04-26T01:57:13.064 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.061+0000 7f429cda0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf3e3da9b0 0x55cf3e3e2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:13.064 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.061+0000 7f429cda0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf3e3db2d0 0x55cf3e3e3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:13.064 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.061+0000 7f429cda0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf3e3e5b90 0x55cf3e3e35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:13.064 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.061+0000 7f429cda0740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55cf3e3e87c0 con 0x55cf3e3da9b0 2024-04-26T01:57:13.064 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.061+0000 7f429cda0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55cf3e2fa310 con 0x55cf3e3e5b90 2024-04-26T01:57:13.065 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.061+0000 7f429cda0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55cf3e3e8940 con 0x55cf3e3db2d0 2024-04-26T01:57:13.065 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.061+0000 7f429cda0740 10 client.0 _ll_register_callbacks cb 0x55cf3e2c8cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:57:13.065 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.061+0000 7f428be00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf3e3db2d0 0x55cf3e3e3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:13.065 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.061+0000 7f4293400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf3e3da9b0 0x55cf3e3e2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:13.065 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.061+0000 7f4293400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf3e3da9b0 0x55cf3e3e2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52182/0 (socket says 172.21.15.151:52182) 2024-04-26T01:57:13.065 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.061+0000 7f4293400640 1 -- 172.21.15.151:0/166831890 learned_addr learned my addr 172.21.15.151:0/166831890 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:57:13.065 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.061+0000 7f4293e00640 1 --2- 172.21.15.151:0/166831890 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf3e3e5b90 0x55cf3e3e35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:13.065 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.061+0000 7f428be00640 1 -- 172.21.15.151:0/166831890 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf3e3e5b90 msgr2=0x55cf3e3e35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:13.065 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.061+0000 7f428be00640 1 --2- 172.21.15.151:0/166831890 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf3e3e5b90 0x55cf3e3e35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:13.065 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.061+0000 7f428be00640 1 -- 172.21.15.151:0/166831890 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf3e3da9b0 msgr2=0x55cf3e3e2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:13.065 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.061+0000 7f428be00640 1 --2- 172.21.15.151:0/166831890 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf3e3da9b0 0x55cf3e3e2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:13.065 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.061+0000 7f428be00640 1 -- 172.21.15.151:0/166831890 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf3e11f620 con 0x55cf3e3db2d0 2024-04-26T01:57:13.065 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.061+0000 7f4293400640 1 --2- 172.21.15.151:0/166831890 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf3e3da9b0 0x55cf3e3e2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:57:13.065 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.062+0000 7f428be00640 1 --2- 172.21.15.151:0/166831890 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf3e3db2d0 0x55cf3e3e3060 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f427c006010 tx=0x7f427c02a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e250a2312e51af71 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:13.065 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.062+0000 7f4280c00640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-26T01:57:13.066 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.062+0000 7f4280c00640 1 -- 172.21.15.151:0/166831890 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427c035040 con 0x55cf3e3db2d0 2024-04-26T01:57:13.066 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.062+0000 7f429cda0740 1 -- 172.21.15.151:0/166831890 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f428400fa70 con 0x55cf3e3db2d0 2024-04-26T01:57:13.066 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.062+0000 7f4280c00640 1 -- 172.21.15.151:0/166831890 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f427c0055f0 con 0x55cf3e3db2d0 2024-04-26T01:57:13.066 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.062+0000 7f4280c00640 1 -- 172.21.15.151:0/166831890 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f427c004e30 con 0x55cf3e3db2d0 2024-04-26T01:57:13.066 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.062+0000 7f429cda0740 10 client.25630 fetch_fsmap learned FSMap version 28 2024-04-26T01:57:13.066 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.062+0000 7f429cda0740 1 -- 172.21.15.151:0/166831890 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cf3e3f4a70 con 0x55cf3e3db2d0 2024-04-26T01:57:13.066 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.063+0000 7f4280c00640 1 -- 172.21.15.151:0/166831890 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f428400fa70 con 0x55cf3e3db2d0 2024-04-26T01:57:13.066 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.063+0000 7f4280c00640 1 -- 172.21.15.151:0/166831890 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f427c05c050 con 0x55cf3e3db2d0 2024-04-26T01:57:13.066 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.063+0000 7f429cda0740 10 client.25630 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T01:57:13.066 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.063+0000 7f429cda0740 10 client.25630 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:57:13.066 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.063+0000 7f429cda0740 20 client.25630 populate_metadata read hostname 'smithi151' 2024-04-26T01:57:13.066 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.063+0000 7f429cda0740 1 --2- 172.21.15.151:0/166831890 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55cf3e3f5ee0 0x55cf3e3f8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:13.066 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.063+0000 7f429cda0740 4 client.25630 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:57:13.066 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.063+0000 7f429cda0740 1 -- 172.21.15.151:0/166831890 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf3e3548b0 con 0x55cf3e3f5ee0 2024-04-26T01:57:13.067 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.064+0000 7f4293400640 1 --2- 172.21.15.151:0/166831890 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55cf3e3f5ee0 0x55cf3e3f8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:13.067 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.064+0000 7f4293400640 1 --2- 172.21.15.151:0/166831890 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55cf3e3f5ee0 0x55cf3e3f8330 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8acbf4837d16526d server_cookie=63867a311568137f in_seq=0 out_seq=0 2024-04-26T01:57:13.067 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.064+0000 7f4280c00640 10 client.25630 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:57:13.067 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.065+0000 7f4280c00640 1 -- 172.21.15.151:0/166831890 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cf3e3548b0 con 0x55cf3e3f5ee0 2024-04-26T01:57:13.067 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.065+0000 7f4280c00640 10 client.25630 handle_command_reply: tid=0 2024-04-26T01:57:13.067 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.065+0000 7f429cda0740 1 client.25630 shutdown 2024-04-26T01:57:13.068 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.065+0000 7f429cda0740 1 -- 172.21.15.151:0/166831890 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf3e3db2d0 msgr2=0x55cf3e3e3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:13.068 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.065+0000 7f429cda0740 1 --2- 172.21.15.151:0/166831890 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf3e3db2d0 0x55cf3e3e3060 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f427c006010 tx=0x7f427c02a3f0 comp rx=0 tx=0).stop 2024-04-26T01:57:13.068 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.065+0000 7f429cda0740 1 -- 172.21.15.151:0/166831890 shutdown_connections 2024-04-26T01:57:13.068 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.065+0000 7f429cda0740 1 --2- 172.21.15.151:0/166831890 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55cf3e3f5ee0 0x55cf3e3f8330 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:13.068 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.065+0000 7f429cda0740 1 --2- 172.21.15.151:0/166831890 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf3e3e5b90 0x55cf3e3e35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:13.068 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.065+0000 7f429cda0740 1 --2- 172.21.15.151:0/166831890 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf3e3db2d0 0x55cf3e3e3060 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:13.068 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.065+0000 7f4280c00640 0 client.25630 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:57:13.068 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.065+0000 7f429cda0740 1 --2- 172.21.15.151:0/166831890 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf3e3da9b0 0x55cf3e3e2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:13.068 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.065+0000 7f429cda0740 1 -- 172.21.15.151:0/166831890 >> 172.21.15.151:0/166831890 conn(0x55cf3e3d6ec0 msgr2=0x55cf3e3dcc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:13.068 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.065+0000 7f429cda0740 1 -- 172.21.15.151:0/166831890 shutdown_connections 2024-04-26T01:57:13.068 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.065+0000 7f429cda0740 1 -- 172.21.15.151:0/166831890 wait complete. 2024-04-26T01:57:13.069 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.066+0000 7f429cda0740 20 client.25630 trim_cache size 0 max 16384 2024-04-26T01:57:13.081 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.082+0000 7f429cda0740 1 Processor -- start 2024-04-26T01:57:13.081 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.083+0000 7f429cda0740 1 -- start start 2024-04-26T01:57:13.082 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.083+0000 7f429cda0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf3e3da9b0 0x55cf3e3f1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:13.082 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.083+0000 7f429cda0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf3e3db2d0 0x55cf3e3fdc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:13.082 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.083+0000 7f429cda0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf3e3e5b90 0x55cf3e3fe180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:13.082 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.083+0000 7f429cda0740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f42941b2340 con 0x55cf3e3e5b90 2024-04-26T01:57:13.082 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.083+0000 7f429cda0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f42941afc10 con 0x55cf3e3da9b0 2024-04-26T01:57:13.082 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.083+0000 7f429cda0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f42941ada80 con 0x55cf3e3db2d0 2024-04-26T01:57:13.084 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.083+0000 7f4293e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf3e3e5b90 0x55cf3e3fe180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:13.084 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.083+0000 7f4293e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf3e3e5b90 0x55cf3e3fe180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52192/0 (socket says 172.21.15.151:52192) 2024-04-26T01:57:13.084 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.083+0000 7f4293e00640 1 -- 172.21.15.151:0/3616787245 learned_addr learned my addr 172.21.15.151:0/3616787245 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:57:13.084 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.083+0000 7f428be00640 1 --2- 172.21.15.151:0/3616787245 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf3e3db2d0 0x55cf3e3fdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:13.084 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.084+0000 7f4293400640 1 --2- 172.21.15.151:0/3616787245 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf3e3da9b0 0x55cf3e3f1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:13.084 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.084+0000 7f4293e00640 1 -- 172.21.15.151:0/3616787245 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf3e3db2d0 msgr2=0x55cf3e3fdc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:13.084 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.084+0000 7f4293e00640 1 --2- 172.21.15.151:0/3616787245 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf3e3db2d0 0x55cf3e3fdc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:13.084 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.084+0000 7f4293e00640 1 -- 172.21.15.151:0/3616787245 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf3e3da9b0 msgr2=0x55cf3e3f1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:13.084 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.084+0000 7f4293e00640 1 --2- 172.21.15.151:0/3616787245 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf3e3da9b0 0x55cf3e3f1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:13.084 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.084+0000 7f4293e00640 1 -- 172.21.15.151:0/3616787245 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf3e3f2b30 con 0x55cf3e3e5b90 2024-04-26T01:57:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.084+0000 7f428be00640 1 --2- 172.21.15.151:0/3616787245 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf3e3db2d0 0x55cf3e3fdc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:57:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.084+0000 7f4293400640 1 --2- 172.21.15.151:0/3616787245 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf3e3da9b0 0x55cf3e3f1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T01:57:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.084+0000 7f4293e00640 1 --2- 172.21.15.151:0/3616787245 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf3e3e5b90 0x55cf3e3fe180 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f428c09cbd0 tx=0x7f428c0987c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=744fdfae7e8eb37e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.084+0000 7f4283400640 1 -- 172.21.15.151:0/3616787245 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f428c0b5020 con 0x55cf3e3e5b90 2024-04-26T01:57:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.084+0000 7f4283400640 1 -- 172.21.15.151:0/3616787245 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f428c087040 con 0x55cf3e3e5b90 2024-04-26T01:57:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.084+0000 7f4283400640 1 -- 172.21.15.151:0/3616787245 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f428c0a58b0 con 0x55cf3e3e5b90 2024-04-26T01:57:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.085+0000 7f429cda0740 1 -- 172.21.15.151:0/3616787245 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf3e3e5b90 msgr2=0x55cf3e3fe180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.085+0000 7f429cda0740 1 --2- 172.21.15.151:0/3616787245 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf3e3e5b90 0x55cf3e3fe180 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f428c09cbd0 tx=0x7f428c0987c0 comp rx=0 tx=0).stop 2024-04-26T01:57:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.085+0000 7f429cda0740 1 -- 172.21.15.151:0/3616787245 shutdown_connections 2024-04-26T01:57:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.085+0000 7f429cda0740 1 --2- 172.21.15.151:0/3616787245 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf3e3e5b90 0x55cf3e3fe180 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.085+0000 7f429cda0740 1 --2- 172.21.15.151:0/3616787245 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf3e3db2d0 0x55cf3e3fdc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.085+0000 7f429cda0740 1 --2- 172.21.15.151:0/3616787245 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf3e3da9b0 0x55cf3e3f1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.085+0000 7f429cda0740 1 -- 172.21.15.151:0/3616787245 >> 172.21.15.151:0/3616787245 conn(0x55cf3e3d6ec0 msgr2=0x55cf3e3dde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:13.086 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.085+0000 7f429cda0740 1 -- 172.21.15.151:0/3616787245 shutdown_connections 2024-04-26T01:57:13.086 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.085+0000 7f429cda0740 1 -- 172.21.15.151:0/3616787245 wait complete. 2024-04-26T01:57:13.086 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.085+0000 7f429cda0740 1 -- start start 2024-04-26T01:57:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.086+0000 7f429cda0740 1 Processor -- start 2024-04-26T01:57:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.086+0000 7f429cda0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf3e3da9b0 0x55cf3e3edc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.086+0000 7f429cda0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf3e3db2d0 0x55cf3e3eb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.086+0000 7f429cda0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf3e3e5b90 0x55cf3e3eb640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.086+0000 7f429cda0740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f42941ae0d0 con 0x55cf3e3e5b90 2024-04-26T01:57:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.086+0000 7f429cda0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f42941af540 con 0x55cf3e3db2d0 2024-04-26T01:57:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.086+0000 7f429cda0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f42941ae7a0 con 0x55cf3e3da9b0 2024-04-26T01:57:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.086+0000 7f429cda0740 10 client.0 _ll_register_callbacks cb 0x7f427c004450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:57:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.086+0000 7f4293e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf3e3e5b90 0x55cf3e3eb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.086+0000 7f4293e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf3e3e5b90 0x55cf3e3eb640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52200/0 (socket says 172.21.15.151:52200) 2024-04-26T01:57:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.086+0000 7f4293400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf3e3da9b0 0x55cf3e3edc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.086+0000 7f4293e00640 1 -- 172.21.15.151:0/2400370788 learned_addr learned my addr 172.21.15.151:0/2400370788 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:57:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.086+0000 7f428be00640 1 --2- 172.21.15.151:0/2400370788 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf3e3db2d0 0x55cf3e3eb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.087+0000 7f4293e00640 1 -- 172.21.15.151:0/2400370788 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf3e3da9b0 msgr2=0x55cf3e3edc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.087+0000 7f4293e00640 1 --2- 172.21.15.151:0/2400370788 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf3e3da9b0 0x55cf3e3edc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.087+0000 7f4293e00640 1 -- 172.21.15.151:0/2400370788 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf3e3db2d0 msgr2=0x55cf3e3eb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.087+0000 7f4293e00640 1 --2- 172.21.15.151:0/2400370788 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf3e3db2d0 0x55cf3e3eb100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.087+0000 7f4293e00640 1 -- 172.21.15.151:0/2400370788 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf3e3ebb80 con 0x55cf3e3e5b90 2024-04-26T01:57:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.087+0000 7f428be00640 1 --2- 172.21.15.151:0/2400370788 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf3e3db2d0 0x55cf3e3eb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:57:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.087+0000 7f4293e00640 1 --2- 172.21.15.151:0/2400370788 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf3e3e5b90 0x55cf3e3eb640 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f428c0a5380 tx=0x7f428c0a1060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f18d54f9255a89e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.087+0000 7f4280c00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-04-26T01:57:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.087+0000 7f4280c00640 1 -- 172.21.15.151:0/2400370788 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f428c0bb020 con 0x55cf3e3e5b90 2024-04-26T01:57:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.087+0000 7f4280c00640 1 -- 172.21.15.151:0/2400370788 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f428c0bd0b0 con 0x55cf3e3e5b90 2024-04-26T01:57:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.087+0000 7f429cda0740 1 -- 172.21.15.151:0/2400370788 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f428c087040 con 0x55cf3e3e5b90 2024-04-26T01:57:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.087+0000 7f4280c00640 1 -- 172.21.15.151:0/2400370788 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f428c0b5070 con 0x55cf3e3e5b90 2024-04-26T01:57:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.087+0000 7f429cda0740 10 client.15510 fetch_fsmap learned FSMap version 28 2024-04-26T01:57:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.088+0000 7f429cda0740 1 -- 172.21.15.151:0/2400370788 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cf3e45c150 con 0x55cf3e3e5b90 2024-04-26T01:57:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.088+0000 7f4280c00640 1 -- 172.21.15.151:0/2400370788 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f428c087040 con 0x55cf3e3e5b90 2024-04-26T01:57:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.088+0000 7f4280c00640 1 -- 172.21.15.151:0/2400370788 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f428c0041b0 con 0x55cf3e3e5b90 2024-04-26T01:57:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.088+0000 7f429cda0740 10 client.15510 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T01:57:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.088+0000 7f429cda0740 10 client.15510 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:57:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.088+0000 7f429cda0740 20 client.15510 populate_metadata read hostname 'smithi151' 2024-04-26T01:57:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.088+0000 7f429cda0740 1 --2- 172.21.15.151:0/2400370788 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55cf3e45c870 0x55cf3e45ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.088+0000 7f429cda0740 4 client.15510 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:57:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.088+0000 7f429cda0740 1 -- 172.21.15.151:0/2400370788 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cf3e34e320 con 0x55cf3e45c870 2024-04-26T01:57:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.089+0000 7f4293400640 1 --2- 172.21.15.151:0/2400370788 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55cf3e45c870 0x55cf3e45ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.089+0000 7f4293400640 1 --2- 172.21.15.151:0/2400370788 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55cf3e45c870 0x55cf3e45ec60 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98621fe7c2b358ea server_cookie=6d1c5bf38d41e70d in_seq=0 out_seq=0 2024-04-26T01:57:13.092 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.089+0000 7f4280c00640 10 client.15510 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:57:13.117 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.117+0000 7f4280c00640 1 -- 172.21.15.151:0/2400370788 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cf3e34e320 con 0x55cf3e45c870 2024-04-26T01:57:13.117 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.117+0000 7f4280c00640 10 client.15510 handle_command_reply: tid=0 2024-04-26T01:57:13.117 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.117+0000 7f429cda0740 1 client.15510 shutdown 2024-04-26T01:57:13.117 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.118+0000 7f429cda0740 1 -- 172.21.15.151:0/2400370788 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf3e3e5b90 msgr2=0x55cf3e3eb640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:13.117 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.118+0000 7f429cda0740 1 --2- 172.21.15.151:0/2400370788 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf3e3e5b90 0x55cf3e3eb640 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f428c0a5380 tx=0x7f428c0a1060 comp rx=0 tx=0).stop 2024-04-26T01:57:13.117 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.118+0000 7f429cda0740 1 -- 172.21.15.151:0/2400370788 shutdown_connections 2024-04-26T01:57:13.118 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.118+0000 7f429cda0740 1 --2- 172.21.15.151:0/2400370788 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55cf3e45c870 0x55cf3e45ec60 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:13.118 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.118+0000 7f429cda0740 1 --2- 172.21.15.151:0/2400370788 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf3e3e5b90 0x55cf3e3eb640 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:13.118 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.118+0000 7f4280c00640 0 client.15510 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:57:13.118 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.118+0000 7f429cda0740 1 --2- 172.21.15.151:0/2400370788 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cf3e3db2d0 0x55cf3e3eb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:13.118 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.118+0000 7f429cda0740 1 --2- 172.21.15.151:0/2400370788 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf3e3da9b0 0x55cf3e3edc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:13.118 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.118+0000 7f429cda0740 1 -- 172.21.15.151:0/2400370788 >> 172.21.15.151:0/2400370788 conn(0x55cf3e3d6ec0 msgr2=0x55cf3e3d5790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:13.118 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.118+0000 7f429cda0740 1 -- 172.21.15.151:0/2400370788 shutdown_connections 2024-04-26T01:57:13.118 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.118+0000 7f429cda0740 1 -- 172.21.15.151:0/2400370788 wait complete. 2024-04-26T01:57:13.118 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.119+0000 7f429cda0740 20 client.15510 trim_cache size 0 max 16384 2024-04-26T01:57:13.122 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:{ 2024-04-26T01:57:13.122 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "message": "", 2024-04-26T01:57:13.122 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "return_code": 0 2024-04-26T01:57:13.122 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:} 2024-04-26T01:57:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.119+0000 7f4298a00640 1 -- 172.21.15.151:0/2703737760 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f42780a3d10 msgr2=0x7f42780a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.119+0000 7f4298a00640 1 --2- 172.21.15.151:0/2703737760 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f42780a3d10 0x7f42780a61d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f427c000c00 tx=0x7f427c005e50 comp rx=0 tx=0).stop 2024-04-26T01:57:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.119+0000 7f4298a00640 1 -- 172.21.15.151:0/2703737760 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f429419a600 msgr2=0x7f429419aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.119+0000 7f4298a00640 1 --2- 172.21.15.151:0/2703737760 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f429419a600 0x7f429419aab0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f428c00ec50 tx=0x7f428c00c6a0 comp rx=0 tx=0).stop 2024-04-26T01:57:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.120+0000 7f4298a00640 1 -- 172.21.15.151:0/2703737760 shutdown_connections 2024-04-26T01:57:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.120+0000 7f4298a00640 1 --2- 172.21.15.151:0/2703737760 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f42780a3d10 0x7f42780a61d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.120+0000 7f4298a00640 1 --2- 172.21.15.151:0/2703737760 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f429419a600 0x7f429419aab0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.120+0000 7f4298a00640 1 --2- 172.21.15.151:0/2703737760 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42941092d0 0x7f42941a0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.120+0000 7f4298a00640 1 --2- 172.21.15.151:0/2703737760 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4294104200 0x7f42941a0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:13.124 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.120+0000 7f4298a00640 1 -- 172.21.15.151:0/2703737760 >> 172.21.15.151:0/2703737760 conn(0x7f42940ffa60 msgr2=0x7f4294100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:13.124 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.120+0000 7f4298a00640 1 -- 172.21.15.151:0/2703737760 shutdown_connections 2024-04-26T01:57:13.124 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:13.120+0000 7f4298a00640 1 -- 172.21.15.151:0/2703737760 wait complete. 2024-04-26T01:57:14.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:57:14 smithi172 ceph-mon[32785]: pgmap v111: 97 pgs: 97 active+clean; 35 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 756 KiB/s rd, 1.0 MiB/s wr, 170 op/s 2024-04-26T01:57:14.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:57:14 smithi165 ceph-mon[32648]: pgmap v111: 97 pgs: 97 active+clean; 35 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 756 KiB/s rd, 1.0 MiB/s wr, 170 op/s 2024-04-26T01:57:14.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:57:14 smithi151 ceph-mon[28057]: pgmap v111: 97 pgs: 97 active+clean; 35 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 756 KiB/s rd, 1.0 MiB/s wr, 170 op/s 2024-04-26T01:57:16.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:57:16 smithi172 ceph-mon[32785]: pgmap v112: 97 pgs: 97 active+clean; 35 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 851 KiB/s wr, 128 op/s 2024-04-26T01:57:16.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:57:16 smithi165 ceph-mon[32648]: pgmap v112: 97 pgs: 97 active+clean; 35 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 851 KiB/s wr, 128 op/s 2024-04-26T01:57:16.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:57:16 smithi151 ceph-mon[28057]: pgmap v112: 97 pgs: 97 active+clean; 35 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 851 KiB/s wr, 128 op/s 2024-04-26T01:57:18.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:57:18 smithi172 ceph-mon[32785]: pgmap v113: 97 pgs: 97 active+clean; 35 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 287 KiB/s wr, 34 op/s 2024-04-26T01:57:18.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:57:18 smithi165 ceph-mon[32648]: pgmap v113: 97 pgs: 97 active+clean; 35 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 287 KiB/s wr, 34 op/s 2024-04-26T01:57:18.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:57:18 smithi151 ceph-mon[28057]: pgmap v113: 97 pgs: 97 active+clean; 35 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 287 KiB/s wr, 34 op/s 2024-04-26T01:57:20.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:57:20 smithi172 ceph-mon[32785]: pgmap v114: 97 pgs: 97 active+clean; 35 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 303 KiB/s wr, 43 op/s 2024-04-26T01:57:20.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:57:20 smithi165 ceph-mon[32648]: pgmap v114: 97 pgs: 97 active+clean; 35 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 303 KiB/s wr, 43 op/s 2024-04-26T01:57:20.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:57:20 smithi151 ceph-mon[28057]: pgmap v114: 97 pgs: 97 active+clean; 35 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 303 KiB/s wr, 43 op/s 2024-04-26T01:57:22.538 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:57:22 smithi172 ceph-mon[32785]: pgmap v115: 97 pgs: 97 active+clean; 35 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 26 KiB/s wr, 13 op/s 2024-04-26T01:57:22.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:57:22 smithi165 ceph-mon[32648]: pgmap v115: 97 pgs: 97 active+clean; 35 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 26 KiB/s wr, 13 op/s 2024-04-26T01:57:22.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:57:22 smithi151 ceph-mon[28057]: pgmap v115: 97 pgs: 97 active+clean; 35 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 26 KiB/s wr, 13 op/s 2024-04-26T01:57:24.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:57:24 smithi172 ceph-mon[32785]: pgmap v116: 97 pgs: 97 active+clean; 35 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 31 KiB/s wr, 14 op/s 2024-04-26T01:57:24.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:57:24 smithi165 ceph-mon[32648]: pgmap v116: 97 pgs: 97 active+clean; 35 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 31 KiB/s wr, 14 op/s 2024-04-26T01:57:24.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:57:24 smithi151 ceph-mon[28057]: pgmap v116: 97 pgs: 97 active+clean; 35 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 31 KiB/s wr, 14 op/s 2024-04-26T01:57:26.269 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.269+0000 7ff812600640 1 -- 172.21.15.151:0/707370664 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff80c1043c0 msgr2=0x7ff80c1047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:26.269 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.269+0000 7ff812600640 1 --2- 172.21.15.151:0/707370664 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff80c1043c0 0x7ff80c1047a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc009900 tx=0x7ff7fc02a3f0 comp rx=0 tx=0).stop 2024-04-26T01:57:26.269 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.269+0000 7ff812600640 1 -- 172.21.15.151:0/707370664 shutdown_connections 2024-04-26T01:57:26.269 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.269+0000 7ff812600640 1 --2- 172.21.15.151:0/707370664 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff80c109490 0x7ff80c111060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:26.269 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.270+0000 7ff812600640 1 --2- 172.21.15.151:0/707370664 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff80c104d70 0x7ff80c108db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:26.269 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.270+0000 7ff812600640 1 --2- 172.21.15.151:0/707370664 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff80c1043c0 0x7ff80c1047a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:26.269 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.270+0000 7ff812600640 1 -- 172.21.15.151:0/707370664 >> 172.21.15.151:0/707370664 conn(0x7ff80c0ffc20 msgr2=0x7ff80c102040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:26.269 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.270+0000 7ff812600640 1 -- 172.21.15.151:0/707370664 shutdown_connections 2024-04-26T01:57:26.269 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.270+0000 7ff812600640 1 -- 172.21.15.151:0/707370664 wait complete. 2024-04-26T01:57:26.272 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.271+0000 7ff812600640 1 Processor -- start 2024-04-26T01:57:26.272 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.271+0000 7ff812600640 1 -- start start 2024-04-26T01:57:26.272 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.271+0000 7ff812600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff80c1043c0 0x7ff80c1a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:26.272 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.271+0000 7ff812600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff80c109490 0x7ff80c1a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:26.272 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.271+0000 7ff812600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff80c19a6f0 0x7ff80c19ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:26.272 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.271+0000 7ff812600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff80c114450 con 0x7ff80c19a6f0 2024-04-26T01:57:26.272 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.271+0000 7ff812600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff80c1142d0 con 0x7ff80c1043c0 2024-04-26T01:57:26.272 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.271+0000 7ff812600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff80c1145d0 con 0x7ff80c109490 2024-04-26T01:57:26.272 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.271+0000 7ff80be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff80c19a6f0 0x7ff80c19ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:26.272 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.271+0000 7ff80aa00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff80c109490 0x7ff80c1a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:26.272 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.272+0000 7ff80be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff80c19a6f0 0x7ff80c19ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:51694/0 (socket says 172.21.15.151:51694) 2024-04-26T01:57:26.272 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.272+0000 7ff80be00640 1 -- 172.21.15.151:0/4090011642 learned_addr learned my addr 172.21.15.151:0/4090011642 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:57:26.272 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.272+0000 7ff80b400640 1 --2- 172.21.15.151:0/4090011642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff80c1043c0 0x7ff80c1a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:26.273 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.272+0000 7ff80be00640 1 -- 172.21.15.151:0/4090011642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff80c109490 msgr2=0x7ff80c1a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:26.273 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.272+0000 7ff80be00640 1 --2- 172.21.15.151:0/4090011642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff80c109490 0x7ff80c1a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:26.273 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.272+0000 7ff80be00640 1 -- 172.21.15.151:0/4090011642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff80c1043c0 msgr2=0x7ff80c1a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:26.273 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.272+0000 7ff80be00640 1 --2- 172.21.15.151:0/4090011642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff80c1043c0 0x7ff80c1a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:26.273 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.272+0000 7ff80be00640 1 -- 172.21.15.151:0/4090011642 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff80c19b2f0 con 0x7ff80c19a6f0 2024-04-26T01:57:26.273 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.272+0000 7ff80b400640 1 --2- 172.21.15.151:0/4090011642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff80c1043c0 0x7ff80c1a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:57:26.273 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.272+0000 7ff80be00640 1 --2- 172.21.15.151:0/4090011642 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff80c19a6f0 0x7ff80c19ab70 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff80400eb10 tx=0x7ff80400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:26.273 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.273+0000 7ff803e00640 1 -- 172.21.15.151:0/4090011642 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff804004b20 con 0x7ff80c19a6f0 2024-04-26T01:57:26.273 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.273+0000 7ff803e00640 1 -- 172.21.15.151:0/4090011642 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff804004cc0 con 0x7ff80c19a6f0 2024-04-26T01:57:26.273 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.273+0000 7ff812600640 1 -- 172.21.15.151:0/4090011642 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff80c1a73d0 con 0x7ff80c19a6f0 2024-04-26T01:57:26.273 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.273+0000 7ff803e00640 1 -- 172.21.15.151:0/4090011642 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff804010670 con 0x7ff80c19a6f0 2024-04-26T01:57:26.273 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.273+0000 7ff812600640 1 -- 172.21.15.151:0/4090011642 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff80c1a7830 con 0x7ff80c19a6f0 2024-04-26T01:57:26.275 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.276+0000 7ff8153c6740 1 Processor -- start 2024-04-26T01:57:26.275 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.276+0000 7ff803e00640 1 -- 172.21.15.151:0/4090011642 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7ff804005020 con 0x7ff80c19a6f0 2024-04-26T01:57:26.275 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.276+0000 7ff8153c6740 1 -- start start 2024-04-26T01:57:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.277+0000 7ff8153c6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55abff259820 0x55abff259c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.277+0000 7ff8153c6740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abff25a140 0x55abff2644c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.277+0000 7ff8153c6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55abff264a00 0x55abff266df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.277+0000 7ff8153c6740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55abff1bf900 con 0x55abff25a140 2024-04-26T01:57:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.277+0000 7ff8153c6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55abff165ef0 con 0x55abff259820 2024-04-26T01:57:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.277+0000 7ff8153c6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55abff1bb8c0 con 0x55abff264a00 2024-04-26T01:57:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.277+0000 7ff80b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abff25a140 0x55abff2644c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.277+0000 7ff80aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55abff259820 0x55abff259c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.277+0000 7ff80aa00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55abff264a00 0x55abff266df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.277+0000 7ff80aa00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55abff264a00 0x55abff266df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:33552/0 (socket says 172.21.15.151:33552) 2024-04-26T01:57:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.277+0000 7ff80aa00640 1 -- 172.21.15.151:0/1784298818 learned_addr learned my addr 172.21.15.151:0/1784298818 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:57:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.277+0000 7ff80b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abff25a140 0x55abff2644c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:51702/0 (socket says 172.21.15.151:51702) 2024-04-26T01:57:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.277+0000 7ff80aa00640 1 -- 172.21.15.151:0/1784298818 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55abff259820 msgr2=0x55abff259c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.277+0000 7ff80aa00640 1 --2- 172.21.15.151:0/1784298818 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55abff259820 0x55abff259c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.277+0000 7ff80aa00640 1 -- 172.21.15.151:0/1784298818 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abff25a140 msgr2=0x55abff2644c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.277+0000 7ff80aa00640 1 --2- 172.21.15.151:0/1784298818 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abff25a140 0x55abff2644c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.277+0000 7ff80aa00640 1 -- 172.21.15.151:0/1784298818 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abfeed2210 con 0x55abff264a00 2024-04-26T01:57:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.278+0000 7ff80aa00640 1 --2- 172.21.15.151:0/1784298818 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55abff264a00 0x55abff266df0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8012750 tx=0x7ff7f800b390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5887869df1112514 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.278+0000 7ff803e00640 1 --2- 172.21.15.151:0/4090011642 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7ff7f00a3e40 0x7ff7f00a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.278+0000 7ff800c00640 1 -- 172.21.15.151:0/1784298818 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f8005020 con 0x55abff264a00 2024-04-26T01:57:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.278+0000 7ff800c00640 1 -- 172.21.15.151:0/1784298818 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7f80047a0 con 0x55abff264a00 2024-04-26T01:57:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.278+0000 7ff800c00640 1 -- 172.21.15.151:0/1784298818 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f800f9c0 con 0x55abff264a00 2024-04-26T01:57:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.278+0000 7ff803e00640 1 -- 172.21.15.151:0/4090011642 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7ff8040d0c20 con 0x7ff80c19a6f0 2024-04-26T01:57:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.278+0000 7ff80b400640 1 --2- 172.21.15.151:0/4090011642 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7ff7f00a3e40 0x7ff7f00a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:26.284 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.279+0000 7ff8153c6740 1 -- 172.21.15.151:0/1784298818 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55abff264a00 msgr2=0x55abff266df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:26.284 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.279+0000 7ff8153c6740 1 --2- 172.21.15.151:0/1784298818 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55abff264a00 0x55abff266df0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8012750 tx=0x7ff7f800b390 comp rx=0 tx=0).stop 2024-04-26T01:57:26.284 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.279+0000 7ff80b400640 1 --2- 172.21.15.151:0/4090011642 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7ff7f00a3e40 0x7ff7f00a6300 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc005ab0 tx=0x7ff7fc005a20 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:26.284 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.279+0000 7ff8153c6740 1 -- 172.21.15.151:0/1784298818 shutdown_connections 2024-04-26T01:57:26.284 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.279+0000 7ff8153c6740 1 --2- 172.21.15.151:0/1784298818 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55abff264a00 0x55abff266df0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:26.284 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.279+0000 7ff8153c6740 1 --2- 172.21.15.151:0/1784298818 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abff25a140 0x55abff2644c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:26.284 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.279+0000 7ff8153c6740 1 --2- 172.21.15.151:0/1784298818 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55abff259820 0x55abff259c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:26.284 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.279+0000 7ff8153c6740 1 -- 172.21.15.151:0/1784298818 >> 172.21.15.151:0/1784298818 conn(0x55abff255d30 msgr2=0x55abff256110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:26.284 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.279+0000 7ff8153c6740 1 -- 172.21.15.151:0/1784298818 shutdown_connections 2024-04-26T01:57:26.284 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.279+0000 7ff8153c6740 1 -- 172.21.15.151:0/1784298818 wait complete. 2024-04-26T01:57:26.284 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.280+0000 7ff8153c6740 1 -- start start 2024-04-26T01:57:26.285 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.280+0000 7ff8153c6740 1 Processor -- start 2024-04-26T01:57:26.285 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.280+0000 7ff8153c6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55abff259820 0x55abff261930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:26.285 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.280+0000 7ff8153c6740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abff25a140 0x55abff261e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:26.285 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.280+0000 7ff8153c6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55abff264a00 0x55abff2623b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:26.285 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.280+0000 7ff8153c6740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55abff267630 con 0x55abff25a140 2024-04-26T01:57:26.285 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.280+0000 7ff8153c6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55abff179310 con 0x55abff259820 2024-04-26T01:57:26.285 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.280+0000 7ff8153c6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55abff2677b0 con 0x55abff264a00 2024-04-26T01:57:26.285 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.280+0000 7ff8153c6740 10 client.0 _ll_register_callbacks cb 0x55abff147cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:57:26.285 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.281+0000 7ff80aa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abff25a140 0x55abff261e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:26.285 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.281+0000 7ff80aa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abff25a140 0x55abff261e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:51704/0 (socket says 172.21.15.151:51704) 2024-04-26T01:57:26.285 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.281+0000 7ff80aa00640 1 -- 172.21.15.151:0/4199293496 learned_addr learned my addr 172.21.15.151:0/4199293496 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:57:26.285 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.281+0000 7ff80b400640 1 --2- 172.21.15.151:0/4199293496 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55abff259820 0x55abff261930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:26.286 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.281+0000 7ff80be00640 1 --2- 172.21.15.151:0/4199293496 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55abff264a00 0x55abff2623b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:26.286 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.281+0000 7ff80aa00640 1 -- 172.21.15.151:0/4199293496 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55abff264a00 msgr2=0x55abff2623b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:26.286 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.281+0000 7ff80aa00640 1 --2- 172.21.15.151:0/4199293496 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55abff264a00 0x55abff2623b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:26.286 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.281+0000 7ff80aa00640 1 -- 172.21.15.151:0/4199293496 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55abff259820 msgr2=0x55abff261930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:26.286 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.281+0000 7ff80aa00640 1 --2- 172.21.15.151:0/4199293496 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55abff259820 0x55abff261930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:26.286 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.281+0000 7ff80aa00640 1 -- 172.21.15.151:0/4199293496 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abfef9e620 con 0x55abff25a140 2024-04-26T01:57:26.286 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.281+0000 7ff80be00640 1 --2- 172.21.15.151:0/4199293496 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55abff264a00 0x55abff2623b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:57:26.286 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.281+0000 7ff80aa00640 1 --2- 172.21.15.151:0/4199293496 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abff25a140 0x55abff261e70 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8004b80 tx=0x7ff7f8004bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da7d01dd201d255 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:26.286 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.281+0000 7ff7f5600640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-04-26T01:57:26.286 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.282+0000 7ff7f5600640 1 -- 172.21.15.151:0/4199293496 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f8005430 con 0x55abff25a140 2024-04-26T01:57:26.286 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.282+0000 7ff7f5600640 1 -- 172.21.15.151:0/4199293496 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7ff7f8004040 con 0x55abff25a140 2024-04-26T01:57:26.286 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.282+0000 7ff8153c6740 1 -- 172.21.15.151:0/4199293496 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7f80047a0 con 0x55abff25a140 2024-04-26T01:57:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.282+0000 7ff7f5600640 1 -- 172.21.15.151:0/4199293496 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff7f8004240 con 0x55abff25a140 2024-04-26T01:57:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.282+0000 7ff8153c6740 10 client.15522 fetch_fsmap learned FSMap version 28 2024-04-26T01:57:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.282+0000 7ff8153c6740 1 -- 172.21.15.151:0/4199293496 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55abff2738a0 con 0x55abff25a140 2024-04-26T01:57:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.282+0000 7ff7f5600640 1 -- 172.21.15.151:0/4199293496 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7f80047a0 con 0x55abff25a140 2024-04-26T01:57:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.282+0000 7ff7f5600640 1 -- 172.21.15.151:0/4199293496 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7ff7f8030050 con 0x55abff25a140 2024-04-26T01:57:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.283+0000 7ff8153c6740 10 client.15522 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T01:57:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.283+0000 7ff8153c6740 10 client.15522 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:57:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.283+0000 7ff8153c6740 20 client.15522 populate_metadata read hostname 'smithi151' 2024-04-26T01:57:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.283+0000 7ff8153c6740 1 --2- 172.21.15.151:0/4199293496 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55abff274c50 0x55abff277040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.283+0000 7ff8153c6740 4 client.15522 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:57:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.283+0000 7ff8153c6740 1 -- 172.21.15.151:0/4199293496 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55abff1d38b0 con 0x55abff274c50 2024-04-26T01:57:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.283+0000 7ff80b400640 1 --2- 172.21.15.151:0/4199293496 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55abff274c50 0x55abff277040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.284+0000 7ff80b400640 1 --2- 172.21.15.151:0/4199293496 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55abff274c50 0x55abff277040 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f8b125e889bb789 server_cookie=da1f125af3634f88 in_seq=0 out_seq=0 2024-04-26T01:57:26.288 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.284+0000 7ff7f5600640 10 client.15522 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:57:26.288 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.284+0000 7ff7f5600640 1 -- 172.21.15.151:0/4199293496 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55abff1d38b0 con 0x55abff274c50 2024-04-26T01:57:26.288 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.284+0000 7ff7f5600640 10 client.15522 handle_command_reply: tid=0 2024-04-26T01:57:26.288 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.284+0000 7ff8153c6740 1 client.15522 shutdown 2024-04-26T01:57:26.288 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.285+0000 7ff8153c6740 1 -- 172.21.15.151:0/4199293496 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abff25a140 msgr2=0x55abff261e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:26.288 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.285+0000 7ff8153c6740 1 --2- 172.21.15.151:0/4199293496 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abff25a140 0x55abff261e70 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8004b80 tx=0x7ff7f8004bb0 comp rx=0 tx=0).stop 2024-04-26T01:57:26.288 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.285+0000 7ff8153c6740 1 -- 172.21.15.151:0/4199293496 shutdown_connections 2024-04-26T01:57:26.288 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.285+0000 7ff8153c6740 1 --2- 172.21.15.151:0/4199293496 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55abff274c50 0x55abff277040 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:26.288 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.285+0000 7ff8153c6740 1 --2- 172.21.15.151:0/4199293496 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55abff264a00 0x55abff2623b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:26.288 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.285+0000 7ff8153c6740 1 --2- 172.21.15.151:0/4199293496 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abff25a140 0x55abff261e70 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:26.288 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.285+0000 7ff8153c6740 1 --2- 172.21.15.151:0/4199293496 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55abff259820 0x55abff261930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:26.288 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.285+0000 7ff7f5600640 0 client.15522 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:57:26.288 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.285+0000 7ff8153c6740 1 -- 172.21.15.151:0/4199293496 >> 172.21.15.151:0/4199293496 conn(0x55abff255d30 msgr2=0x55abff25bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:26.289 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.285+0000 7ff8153c6740 1 -- 172.21.15.151:0/4199293496 shutdown_connections 2024-04-26T01:57:26.289 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.285+0000 7ff8153c6740 1 -- 172.21.15.151:0/4199293496 wait complete. 2024-04-26T01:57:26.289 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.285+0000 7ff8153c6740 20 client.15522 trim_cache size 0 max 16384 2024-04-26T01:57:26.299 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.301+0000 7ff8153c6740 1 Processor -- start 2024-04-26T01:57:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.301+0000 7ff8153c6740 1 -- start start 2024-04-26T01:57:26.301 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.301+0000 7ff8153c6740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abff259820 0x55abff270520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:26.301 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.301+0000 7ff8153c6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55abff25a140 0x55abff2621d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:26.301 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.301+0000 7ff8153c6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55abff264a00 0x55abff27c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:26.301 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.301+0000 7ff8153c6740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff80c1b2410 con 0x55abff259820 2024-04-26T01:57:26.302 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.301+0000 7ff8153c6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff80c1afce0 con 0x55abff25a140 2024-04-26T01:57:26.302 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.301+0000 7ff8153c6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff80c1adb50 con 0x55abff264a00 2024-04-26T01:57:26.302 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.302+0000 7ff80b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abff259820 0x55abff270520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:26.302 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.302+0000 7ff80aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55abff25a140 0x55abff2621d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:26.302 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.302+0000 7ff80aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55abff25a140 0x55abff2621d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:60190/0 (socket says 172.21.15.151:60190) 2024-04-26T01:57:26.302 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.302+0000 7ff80aa00640 1 -- 172.21.15.151:0/2033457248 learned_addr learned my addr 172.21.15.151:0/2033457248 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:57:26.302 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.302+0000 7ff80be00640 1 --2- 172.21.15.151:0/2033457248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55abff264a00 0x55abff27c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:26.302 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.302+0000 7ff80aa00640 1 -- 172.21.15.151:0/2033457248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55abff264a00 msgr2=0x55abff27c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:26.302 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.302+0000 7ff80aa00640 1 --2- 172.21.15.151:0/2033457248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55abff264a00 0x55abff27c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:26.302 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.302+0000 7ff80aa00640 1 -- 172.21.15.151:0/2033457248 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abff259820 msgr2=0x55abff270520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:26.302 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.302+0000 7ff80aa00640 1 --2- 172.21.15.151:0/2033457248 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abff259820 0x55abff270520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:26.303 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.302+0000 7ff80aa00640 1 -- 172.21.15.151:0/2033457248 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abff256f60 con 0x55abff25a140 2024-04-26T01:57:26.303 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.302+0000 7ff80b400640 1 --2- 172.21.15.151:0/2033457248 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abff259820 0x55abff270520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:57:26.303 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.302+0000 7ff80aa00640 1 --2- 172.21.15.151:0/2033457248 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55abff25a140 0x55abff2621d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8004e90 tx=0x7ff7f8005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c09018b1666ee5d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:26.303 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.303+0000 7ff7f7e00640 1 -- 172.21.15.151:0/2033457248 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f8004c20 con 0x55abff25a140 2024-04-26T01:57:26.303 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.303+0000 7ff7f7e00640 1 -- 172.21.15.151:0/2033457248 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7f800be20 con 0x55abff25a140 2024-04-26T01:57:26.303 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.303+0000 7ff7f7e00640 1 -- 172.21.15.151:0/2033457248 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f800f730 con 0x55abff25a140 2024-04-26T01:57:26.306 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.303+0000 7ff8153c6740 1 -- 172.21.15.151:0/2033457248 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55abff25a140 msgr2=0x55abff2621d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.303+0000 7ff8153c6740 1 --2- 172.21.15.151:0/2033457248 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55abff25a140 0x55abff2621d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8004e90 tx=0x7ff7f8005850 comp rx=0 tx=0).stop 2024-04-26T01:57:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.303+0000 7ff8153c6740 1 -- 172.21.15.151:0/2033457248 shutdown_connections 2024-04-26T01:57:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.303+0000 7ff8153c6740 1 --2- 172.21.15.151:0/2033457248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55abff264a00 0x55abff27c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.303+0000 7ff8153c6740 1 --2- 172.21.15.151:0/2033457248 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55abff25a140 0x55abff2621d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.303+0000 7ff8153c6740 1 --2- 172.21.15.151:0/2033457248 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abff259820 0x55abff270520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.303+0000 7ff8153c6740 1 -- 172.21.15.151:0/2033457248 >> 172.21.15.151:0/2033457248 conn(0x55abff255d30 msgr2=0x55abff263160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.304+0000 7ff8153c6740 1 -- 172.21.15.151:0/2033457248 shutdown_connections 2024-04-26T01:57:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.304+0000 7ff8153c6740 1 -- 172.21.15.151:0/2033457248 wait complete. 2024-04-26T01:57:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.304+0000 7ff8153c6740 1 -- start start 2024-04-26T01:57:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.304+0000 7ff8153c6740 1 Processor -- start 2024-04-26T01:57:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.305+0000 7ff8153c6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55abff259820 0x55abff26c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.305+0000 7ff8153c6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55abff25a140 0x55abff26cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.305+0000 7ff8153c6740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abff264a00 0x55abff269fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.305+0000 7ff8153c6740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff80c1ae1a0 con 0x55abff264a00 2024-04-26T01:57:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.305+0000 7ff8153c6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff80c1af610 con 0x55abff259820 2024-04-26T01:57:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.305+0000 7ff8153c6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff80c1ae870 con 0x55abff25a140 2024-04-26T01:57:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.305+0000 7ff8153c6740 10 client.0 _ll_register_callbacks cb 0x7ff7f8012650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:57:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.305+0000 7ff80be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abff264a00 0x55abff269fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.305+0000 7ff80b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55abff259820 0x55abff26c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.305+0000 7ff80b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55abff259820 0x55abff26c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:60202/0 (socket says 172.21.15.151:60202) 2024-04-26T01:57:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.305+0000 7ff80b400640 1 -- 172.21.15.151:0/745558633 learned_addr learned my addr 172.21.15.151:0/745558633 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:57:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.305+0000 7ff80aa00640 1 --2- 172.21.15.151:0/745558633 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55abff25a140 0x55abff26cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.305+0000 7ff80be00640 1 -- 172.21.15.151:0/745558633 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55abff25a140 msgr2=0x55abff26cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.305+0000 7ff80be00640 1 --2- 172.21.15.151:0/745558633 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55abff25a140 0x55abff26cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.305+0000 7ff80be00640 1 -- 172.21.15.151:0/745558633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55abff259820 msgr2=0x55abff26c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.305+0000 7ff80be00640 1 --2- 172.21.15.151:0/745558633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55abff259820 0x55abff26c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.305+0000 7ff80be00640 1 -- 172.21.15.151:0/745558633 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abff26a7d0 con 0x55abff264a00 2024-04-26T01:57:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.305+0000 7ff80aa00640 1 --2- 172.21.15.151:0/745558633 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55abff25a140 0x55abff26cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:57:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.306+0000 7ff80be00640 1 --2- 172.21.15.151:0/745558633 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abff264a00 0x55abff269fd0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff8040ce380 tx=0x7ff80409c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=410f13cc057e664d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.306+0000 7ff7f5600640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-04-26T01:57:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.306+0000 7ff7f5600640 1 -- 172.21.15.151:0/745558633 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8040b5070 con 0x55abff264a00 2024-04-26T01:57:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.306+0000 7ff7f5600640 1 -- 172.21.15.151:0/745558633 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7ff80409ce30 con 0x55abff264a00 2024-04-26T01:57:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.306+0000 7ff8153c6740 1 -- 172.21.15.151:0/745558633 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7f800be20 con 0x55abff264a00 2024-04-26T01:57:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.306+0000 7ff7f5600640 1 -- 172.21.15.151:0/745558633 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff8040a0f80 con 0x55abff264a00 2024-04-26T01:57:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.306+0000 7ff8153c6740 10 client.15528 fetch_fsmap learned FSMap version 28 2024-04-26T01:57:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.306+0000 7ff8153c6740 1 -- 172.21.15.151:0/745558633 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55abff2dad40 con 0x55abff264a00 2024-04-26T01:57:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.306+0000 7ff7f5600640 1 -- 172.21.15.151:0/745558633 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7f800be20 con 0x55abff264a00 2024-04-26T01:57:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.307+0000 7ff7f5600640 1 -- 172.21.15.151:0/745558633 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7ff804098920 con 0x55abff264a00 2024-04-26T01:57:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.307+0000 7ff8153c6740 10 client.15528 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T01:57:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.307+0000 7ff8153c6740 10 client.15528 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:57:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.307+0000 7ff8153c6740 20 client.15528 populate_metadata read hostname 'smithi151' 2024-04-26T01:57:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.307+0000 7ff8153c6740 1 --2- 172.21.15.151:0/745558633 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55abff2db470 0x55abff2dd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.307+0000 7ff8153c6740 4 client.15528 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:57:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.307+0000 7ff8153c6740 1 -- 172.21.15.151:0/745558633 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55abff1cd320 con 0x55abff2db470 2024-04-26T01:57:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.307+0000 7ff80b400640 1 --2- 172.21.15.151:0/745558633 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55abff2db470 0x55abff2dd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.308+0000 7ff80b400640 1 --2- 172.21.15.151:0/745558633 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55abff2db470 0x55abff2dd890 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c478ede0accade5 server_cookie=8c1f601c8f42b903 in_seq=0 out_seq=0 2024-04-26T01:57:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.308+0000 7ff7f5600640 10 client.15528 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:57:26.330 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.331+0000 7ff7f5600640 1 -- 172.21.15.151:0/745558633 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55abff1cd320 con 0x55abff2db470 2024-04-26T01:57:26.330 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.331+0000 7ff7f5600640 10 client.15528 handle_command_reply: tid=0 2024-04-26T01:57:26.330 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.331+0000 7ff8153c6740 1 client.15528 shutdown 2024-04-26T01:57:26.331 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.331+0000 7ff8153c6740 1 -- 172.21.15.151:0/745558633 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abff264a00 msgr2=0x55abff269fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:26.331 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.331+0000 7ff8153c6740 1 --2- 172.21.15.151:0/745558633 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abff264a00 0x55abff269fd0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff8040ce380 tx=0x7ff80409c530 comp rx=0 tx=0).stop 2024-04-26T01:57:26.331 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.331+0000 7ff8153c6740 1 -- 172.21.15.151:0/745558633 shutdown_connections 2024-04-26T01:57:26.331 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.331+0000 7ff8153c6740 1 --2- 172.21.15.151:0/745558633 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55abff2db470 0x55abff2dd890 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:26.331 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.331+0000 7ff8153c6740 1 --2- 172.21.15.151:0/745558633 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abff264a00 0x55abff269fd0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:26.331 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.331+0000 7ff7f5600640 0 client.15528 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:57:26.331 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.331+0000 7ff8153c6740 1 --2- 172.21.15.151:0/745558633 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55abff25a140 0x55abff26cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:26.331 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.331+0000 7ff8153c6740 1 --2- 172.21.15.151:0/745558633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55abff259820 0x55abff26c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:26.331 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.331+0000 7ff8153c6740 1 -- 172.21.15.151:0/745558633 >> 172.21.15.151:0/745558633 conn(0x55abff255d30 msgr2=0x55abff2639f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:26.331 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.331+0000 7ff8153c6740 1 -- 172.21.15.151:0/745558633 shutdown_connections 2024-04-26T01:57:26.331 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.331+0000 7ff8153c6740 1 -- 172.21.15.151:0/745558633 wait complete. 2024-04-26T01:57:26.331 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.332+0000 7ff8153c6740 20 client.15528 trim_cache size 0 max 16384 2024-04-26T01:57:26.332 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:{ 2024-04-26T01:57:26.332 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "message": "", 2024-04-26T01:57:26.332 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "return_code": 0 2024-04-26T01:57:26.332 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:} 2024-04-26T01:57:26.333 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.332+0000 7ff812600640 1 -- 172.21.15.151:0/4090011642 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7ff7f00a3e40 msgr2=0x7ff7f00a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:26.333 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.332+0000 7ff812600640 1 --2- 172.21.15.151:0/4090011642 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7ff7f00a3e40 0x7ff7f00a6300 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc005ab0 tx=0x7ff7fc005a20 comp rx=0 tx=0).stop 2024-04-26T01:57:26.333 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.332+0000 7ff812600640 1 -- 172.21.15.151:0/4090011642 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff80c19a6f0 msgr2=0x7ff80c19ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:26.333 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.332+0000 7ff812600640 1 --2- 172.21.15.151:0/4090011642 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff80c19a6f0 0x7ff80c19ab70 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff80400eb10 tx=0x7ff80400c2d0 comp rx=0 tx=0).stop 2024-04-26T01:57:26.333 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.332+0000 7ff812600640 1 -- 172.21.15.151:0/4090011642 shutdown_connections 2024-04-26T01:57:26.333 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.333+0000 7ff812600640 1 --2- 172.21.15.151:0/4090011642 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7ff7f00a3e40 0x7ff7f00a6300 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:26.333 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.333+0000 7ff812600640 1 --2- 172.21.15.151:0/4090011642 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff80c19a6f0 0x7ff80c19ab70 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:26.333 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.333+0000 7ff812600640 1 --2- 172.21.15.151:0/4090011642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff80c109490 0x7ff80c1a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:26.333 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.333+0000 7ff812600640 1 --2- 172.21.15.151:0/4090011642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff80c1043c0 0x7ff80c1a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:26.333 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.333+0000 7ff812600640 1 -- 172.21.15.151:0/4090011642 >> 172.21.15.151:0/4090011642 conn(0x7ff80c0ffc20 msgr2=0x7ff80c1007a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:26.333 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.333+0000 7ff812600640 1 -- 172.21.15.151:0/4090011642 shutdown_connections 2024-04-26T01:57:26.333 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:26.333+0000 7ff812600640 1 -- 172.21.15.151:0/4090011642 wait complete. 2024-04-26T01:57:26.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:57:26 smithi172 ceph-mon[32785]: pgmap v117: 97 pgs: 97 active+clean; 35 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 49 KiB/s wr, 15 op/s 2024-04-26T01:57:26.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:57:26 smithi165 ceph-mon[32648]: pgmap v117: 97 pgs: 97 active+clean; 35 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 49 KiB/s wr, 15 op/s 2024-04-26T01:57:26.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:57:26 smithi151 ceph-mon[28057]: pgmap v117: 97 pgs: 97 active+clean; 35 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 49 KiB/s wr, 15 op/s 2024-04-26T01:57:28.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:57:28 smithi172 ceph-mon[32785]: pgmap v118: 97 pgs: 97 active+clean; 35 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 49 KiB/s wr, 13 op/s 2024-04-26T01:57:28.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:57:28 smithi165 ceph-mon[32648]: pgmap v118: 97 pgs: 97 active+clean; 35 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 49 KiB/s wr, 13 op/s 2024-04-26T01:57:28.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:57:28 smithi151 ceph-mon[28057]: pgmap v118: 97 pgs: 97 active+clean; 35 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 49 KiB/s wr, 13 op/s 2024-04-26T01:57:30.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:57:30 smithi172 ceph-mon[32785]: pgmap v119: 97 pgs: 97 active+clean; 36 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 125 KiB/s wr, 22 op/s 2024-04-26T01:57:30.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:57:30 smithi165 ceph-mon[32648]: pgmap v119: 97 pgs: 97 active+clean; 36 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 125 KiB/s wr, 22 op/s 2024-04-26T01:57:30.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:57:30 smithi151 ceph-mon[28057]: pgmap v119: 97 pgs: 97 active+clean; 36 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 125 KiB/s wr, 22 op/s 2024-04-26T01:57:32.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:57:32 smithi172 ceph-mon[32785]: pgmap v120: 97 pgs: 97 active+clean; 36 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 109 KiB/s wr, 12 op/s 2024-04-26T01:57:32.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:57:32 smithi165 ceph-mon[32648]: pgmap v120: 97 pgs: 97 active+clean; 36 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 109 KiB/s wr, 12 op/s 2024-04-26T01:57:32.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:57:32 smithi151 ceph-mon[28057]: pgmap v120: 97 pgs: 97 active+clean; 36 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 109 KiB/s wr, 12 op/s 2024-04-26T01:57:34.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:57:34 smithi165 ceph-mon[32648]: pgmap v121: 97 pgs: 97 active+clean; 36 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 109 KiB/s wr, 13 op/s 2024-04-26T01:57:34.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:57:34 smithi165 ceph-mon[32648]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T01:57:34.627 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:57:34 smithi151 ceph-mon[28057]: pgmap v121: 97 pgs: 97 active+clean; 36 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 109 KiB/s wr, 13 op/s 2024-04-26T01:57:34.627 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:57:34 smithi151 ceph-mon[28057]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T01:57:34.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:57:34 smithi172 ceph-mon[32785]: pgmap v121: 97 pgs: 97 active+clean; 36 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 109 KiB/s wr, 13 op/s 2024-04-26T01:57:34.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:57:34 smithi172 ceph-mon[32785]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T01:57:35.364 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:57:35.817 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:57:36.589 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:57:36 smithi172 ceph-mon[32785]: pgmap v122: 97 pgs: 97 active+clean; 36 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 104 KiB/s wr, 13 op/s 2024-04-26T01:57:36.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:57:36 smithi165 ceph-mon[32648]: pgmap v122: 97 pgs: 97 active+clean; 36 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 104 KiB/s wr, 13 op/s 2024-04-26T01:57:36.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:57:36 smithi151 ceph-mon[28057]: pgmap v122: 97 pgs: 97 active+clean; 36 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 104 KiB/s wr, 13 op/s 2024-04-26T01:57:38.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:38.235+0000 7fbc4c600700 1 -- 172.21.15.151:0/3219973185 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc44102af0 msgr2=0x7fbc44102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:38.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:38.235+0000 7fbc4c600700 1 --2- 172.21.15.151:0/3219973185 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc44102af0 0x7fbc44102f70 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fbc300099c0 tx=0x7fbc30009cd0 comp rx=0 tx=0).stop 2024-04-26T01:57:38.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:38.235+0000 7fbc4c600700 1 -- 172.21.15.151:0/3219973185 shutdown_connections 2024-04-26T01:57:38.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:38.235+0000 7fbc4c600700 1 --2- 172.21.15.151:0/3219973185 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc441034b0 0x7fbc4410f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:38.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:38.235+0000 7fbc4c600700 1 --2- 172.21.15.151:0/3219973185 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc44107940 0x7fbc44107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:38.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:38.235+0000 7fbc4c600700 1 --2- 172.21.15.151:0/3219973185 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc44102af0 0x7fbc44102f70 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:38.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:38.235+0000 7fbc4c600700 1 -- 172.21.15.151:0/3219973185 >> 172.21.15.151:0/3219973185 conn(0x7fbc44079ba0 msgr2=0x7fbc44079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:38.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:38.235+0000 7fbc4c600700 1 -- 172.21.15.151:0/3219973185 shutdown_connections 2024-04-26T01:57:38.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:38.235+0000 7fbc4c600700 1 -- 172.21.15.151:0/3219973185 wait complete. 2024-04-26T01:57:38.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:38.237+0000 7fbc4c600700 1 Processor -- start 2024-04-26T01:57:38.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:38.237+0000 7fbc4c600700 1 -- start start 2024-04-26T01:57:38.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:38.237+0000 7fbc4c600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc44102af0 0x7fbc44199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:38.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:38.237+0000 7fbc4c600700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc441034b0 0x7fbc441996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:38.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:38.237+0000 7fbc4c600700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc44107940 0x7fbc4419da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:38.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:38.237+0000 7fbc4c600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbc44112c90 con 0x7fbc44102af0 2024-04-26T01:57:38.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:38.237+0000 7fbc4c600700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbc44112b10 con 0x7fbc441034b0 2024-04-26T01:57:38.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:38.237+0000 7fbc4c600700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbc44112e10 con 0x7fbc44107940 2024-04-26T01:57:38.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:38.238+0000 7fbc49200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc441034b0 0x7fbc441996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:38.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:38.238+0000 7fbc49c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc44102af0 0x7fbc44199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:38.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:38.238+0000 7fbc49200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc441034b0 0x7fbc441996c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:57626/0 (socket says 172.21.15.151:57626) 2024-04-26T01:57:38.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:38.238+0000 7fbc49200700 1 -- 172.21.15.151:0/3387038063 learned_addr learned my addr 172.21.15.151:0/3387038063 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:57:38.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:38.238+0000 7fbc4a600700 1 --2- 172.21.15.151:0/3387038063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc44107940 0x7fbc4419da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:38.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:38.238+0000 7fbc49200700 1 -- 172.21.15.151:0/3387038063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc44107940 msgr2=0x7fbc4419da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:38.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:38.238+0000 7fbc49200700 1 --2- 172.21.15.151:0/3387038063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc44107940 0x7fbc4419da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:38.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:38.238+0000 7fbc49200700 1 -- 172.21.15.151:0/3387038063 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc44102af0 msgr2=0x7fbc44199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:38.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:38.238+0000 7fbc49200700 1 --2- 172.21.15.151:0/3387038063 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc44102af0 0x7fbc44199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:38.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:38.238+0000 7fbc49200700 1 -- 172.21.15.151:0/3387038063 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc4419e140 con 0x7fbc441034b0 2024-04-26T01:57:38.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:38.238+0000 7fbc49c00700 1 --2- 172.21.15.151:0/3387038063 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc44102af0 0x7fbc44199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:57:38.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:38.239+0000 7fbc49200700 1 --2- 172.21.15.151:0/3387038063 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc441034b0 0x7fbc441996c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbc30006010 tx=0x7fbc30005280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:38.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:38.239+0000 7fbc42000700 1 -- 172.21.15.151:0/3387038063 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc30026070 con 0x7fbc441034b0 2024-04-26T01:57:38.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:38.239+0000 7fbc4c600700 1 -- 172.21.15.151:0/3387038063 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc4419e3d0 con 0x7fbc441034b0 2024-04-26T01:57:38.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:38.239+0000 7fbc42000700 1 -- 172.21.15.151:0/3387038063 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc30005680 con 0x7fbc441034b0 2024-04-26T01:57:38.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:38.239+0000 7fbc42000700 1 -- 172.21.15.151:0/3387038063 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc3000fb20 con 0x7fbc441034b0 2024-04-26T01:57:38.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:38.239+0000 7fbc4c600700 1 -- 172.21.15.151:0/3387038063 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc441a5cb0 con 0x7fbc441034b0 2024-04-26T01:57:38.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:38.240+0000 7fbc4c600700 1 -- 172.21.15.151:0/3387038063 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc441041b0 con 0x7fbc441034b0 2024-04-26T01:57:38.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:38.242+0000 7fbc42000700 1 -- 172.21.15.151:0/3387038063 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7fbc30029070 con 0x7fbc441034b0 2024-04-26T01:57:38.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:38.244+0000 7fbc42000700 1 --2- 172.21.15.151:0/3387038063 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fbc2c0a3db0 0x7fbc2c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:38.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:38.244+0000 7fbc42000700 1 -- 172.21.15.151:0/3387038063 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fbc300d16c0 con 0x7fbc441034b0 2024-04-26T01:57:38.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:38.244+0000 7fbc49c00700 1 --2- 172.21.15.151:0/3387038063 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fbc2c0a3db0 0x7fbc2c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:38.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:38.245+0000 7fbc49c00700 1 --2- 172.21.15.151:0/3387038063 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fbc2c0a3db0 0x7fbc2c0a6270 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbc38009fd0 tx=0x7fbc38009260 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:38.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:38.246+0000 7fbc42000700 1 -- 172.21.15.151:0/3387038063 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbc30030020 con 0x7fbc441034b0 2024-04-26T01:57:38.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:38.521+0000 7fbc4c600700 1 -- 172.21.15.151:0/3387038063 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbc4402df80 con 0x7fbc441034b0 2024-04-26T01:57:38.520 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:57:38 smithi151 ceph-mon[28057]: pgmap v123: 97 pgs: 97 active+clean; 36 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 76 KiB/s wr, 10 op/s 2024-04-26T01:57:38.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:38.522+0000 7fbc42000700 1 -- 172.21.15.151:0/3387038063 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11890 (secure 0 0 0) 0x7fbc30013070 con 0x7fbc441034b0 2024-04-26T01:57:38.521 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:57:38.521 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2937864097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2937864097},{"type":"v1","addr":"172.21.15.151:6835","nonce":2937864097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14373,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/2947090802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2947090802},{"type":"v1","addr":"172.21.15.151:6837","nonce":2947090802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/1356563280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1356563280},{"type":"v1","addr":"172.21.15.151:6839","nonce":1356563280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6841/1075240552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1075240552},{"type":"v1","addr":"172.21.15.151:6841","nonce":1075240552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/504675912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":504675912},{"type":"v1","addr":"172.21.15.165:6833","nonce":504675912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1207580015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1207580015},{"type":"v1","addr":"172.21.15.172:6833","nonce":1207580015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:49:40.746876+0000","modified":"2024-04-26T01:57:33.131215+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24505,"mds_2":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/3598138294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3598138294},{"type":"v1","addr":"172.21.15.165:6835","nonce":3598138294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/1847859054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":1847859054},{"type":"v1","addr":"172.21.15.165:6837","nonce":1847859054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/2309751171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2309751171},{"type":"v1","addr":"172.21.15.172:6835","nonce":2309751171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.172:6837/55972537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":55972537},{"type":"v1","addr":"172.21.15.172:6837","nonce":55972537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.172:6839/1242302810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1242302810},{"type":"v1","addr":"172.21.15.172:6839","nonce":1242302810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/1969716254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1969716254},{"type":"v1","addr":"172.21.15.165:6839","nonce":1969716254}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24493,24505,24509]},"id":1}]} 2024-04-26T01:57:38.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:38.525+0000 7fbc4c600700 1 -- 172.21.15.151:0/3387038063 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fbc2c0a3db0 msgr2=0x7fbc2c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:38.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:38.525+0000 7fbc4c600700 1 --2- 172.21.15.151:0/3387038063 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fbc2c0a3db0 0x7fbc2c0a6270 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbc38009fd0 tx=0x7fbc38009260 comp rx=0 tx=0).stop 2024-04-26T01:57:38.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:38.525+0000 7fbc4c600700 1 -- 172.21.15.151:0/3387038063 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc441034b0 msgr2=0x7fbc441996c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:38.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:38.525+0000 7fbc4c600700 1 --2- 172.21.15.151:0/3387038063 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc441034b0 0x7fbc441996c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbc30006010 tx=0x7fbc30005280 comp rx=0 tx=0).stop 2024-04-26T01:57:38.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:38.526+0000 7fbc4c600700 1 -- 172.21.15.151:0/3387038063 shutdown_connections 2024-04-26T01:57:38.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:38.526+0000 7fbc4c600700 1 --2- 172.21.15.151:0/3387038063 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fbc2c0a3db0 0x7fbc2c0a6270 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:38.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:38.526+0000 7fbc4c600700 1 --2- 172.21.15.151:0/3387038063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc44107940 0x7fbc4419da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:38.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:38.526+0000 7fbc4c600700 1 --2- 172.21.15.151:0/3387038063 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc44102af0 0x7fbc44199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:38.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:38.526+0000 7fbc4c600700 1 --2- 172.21.15.151:0/3387038063 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc441034b0 0x7fbc441996c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:38.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:38.526+0000 7fbc4c600700 1 -- 172.21.15.151:0/3387038063 >> 172.21.15.151:0/3387038063 conn(0x7fbc44079ba0 msgr2=0x7fbc44100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:38.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:38.526+0000 7fbc4c600700 1 -- 172.21.15.151:0/3387038063 shutdown_connections 2024-04-26T01:57:38.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:38.526+0000 7fbc4c600700 1 -- 172.21.15.151:0/3387038063 wait complete. 2024-04-26T01:57:38.526 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 29 2024-04-26T01:57:38.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:57:38 smithi165 ceph-mon[32648]: pgmap v123: 97 pgs: 97 active+clean; 36 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 76 KiB/s wr, 10 op/s 2024-04-26T01:57:38.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:57:38 smithi172 ceph-mon[32785]: pgmap v123: 97 pgs: 97 active+clean; 36 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 76 KiB/s wr, 10 op/s 2024-04-26T01:57:39.477 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24509 quiesce db --set-id 4b386810 --release --await' 2024-04-26T01:57:39.477 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24509 quiesce db --set-id 4b386810 --release --await 2024-04-26T01:57:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.483+0000 7f3a1d400640 1 -- 172.21.15.151:0/1028341764 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3a181043c0 msgr2=0x7f3a181047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.483+0000 7f3a1d400640 1 --2- 172.21.15.151:0/1028341764 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3a181043c0 0x7f3a181047a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3a08009860 tx=0x7f3a0802a340 comp rx=0 tx=0).stop 2024-04-26T01:57:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.483+0000 7f3a1d400640 1 -- 172.21.15.151:0/1028341764 shutdown_connections 2024-04-26T01:57:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.483+0000 7f3a1d400640 1 --2- 172.21.15.151:0/1028341764 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a18109490 0x7f3a18111060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.483+0000 7f3a1d400640 1 --2- 172.21.15.151:0/1028341764 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3a18104d70 0x7f3a18108db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.483+0000 7f3a1d400640 1 --2- 172.21.15.151:0/1028341764 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3a181043c0 0x7f3a181047a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.483+0000 7f3a1d400640 1 -- 172.21.15.151:0/1028341764 >> 172.21.15.151:0/1028341764 conn(0x7f3a180ffc20 msgr2=0x7f3a18102040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.484+0000 7f3a1d400640 1 -- 172.21.15.151:0/1028341764 shutdown_connections 2024-04-26T01:57:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.484+0000 7f3a1d400640 1 -- 172.21.15.151:0/1028341764 wait complete. 2024-04-26T01:57:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.485+0000 7f3a1d400640 1 Processor -- start 2024-04-26T01:57:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.485+0000 7f3a1d400640 1 -- start start 2024-04-26T01:57:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.485+0000 7f3a1d400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3a181043c0 0x7f3a181a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.485+0000 7f3a1d400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a18109490 0x7f3a181a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.485+0000 7f3a1d400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3a1819a770 0x7f3a1819abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.485+0000 7f3a1d400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3a181143d0 con 0x7f3a181043c0 2024-04-26T01:57:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.485+0000 7f3a1d400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3a18114250 con 0x7f3a1819a770 2024-04-26T01:57:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.485+0000 7f3a1d400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3a18114550 con 0x7f3a18109490 2024-04-26T01:57:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.485+0000 7f3a16800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3a181043c0 0x7f3a181a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.486+0000 7f3a0fe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a18109490 0x7f3a181a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.486+0000 7f3a16800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3a181043c0 0x7f3a181a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:51568/0 (socket says 172.21.15.151:51568) 2024-04-26T01:57:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.486+0000 7f3a0fe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a18109490 0x7f3a181a0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:39192/0 (socket says 172.21.15.151:39192) 2024-04-26T01:57:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.486+0000 7f3a16800640 1 -- 172.21.15.151:0/3440634483 learned_addr learned my addr 172.21.15.151:0/3440634483 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:57:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.486+0000 7f3a17200640 1 --2- 172.21.15.151:0/3440634483 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3a1819a770 0x7f3a1819abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.486+0000 7f3a16800640 1 -- 172.21.15.151:0/3440634483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a18109490 msgr2=0x7f3a181a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.486+0000 7f3a16800640 1 --2- 172.21.15.151:0/3440634483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a18109490 0x7f3a181a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.486+0000 7f3a16800640 1 -- 172.21.15.151:0/3440634483 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3a1819a770 msgr2=0x7f3a1819abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.486+0000 7f3a16800640 1 --2- 172.21.15.151:0/3440634483 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3a1819a770 0x7f3a1819abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.486+0000 7f3a16800640 1 -- 172.21.15.151:0/3440634483 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a1819b370 con 0x7f3a181043c0 2024-04-26T01:57:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.486+0000 7f3a17200640 1 --2- 172.21.15.151:0/3440634483 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3a1819a770 0x7f3a1819abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:57:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.486+0000 7f3a0fe00640 1 --2- 172.21.15.151:0/3440634483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a18109490 0x7f3a181a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:57:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.486+0000 7f3a16800640 1 --2- 172.21.15.151:0/3440634483 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3a181043c0 0x7f3a181a06b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3a080303f0 tx=0x7f3a08032da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.487+0000 7f3a0f400640 1 -- 172.21.15.151:0/3440634483 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a08040020 con 0x7f3a181043c0 2024-04-26T01:57:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.487+0000 7f3a0f400640 1 -- 172.21.15.151:0/3440634483 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a08030dd0 con 0x7f3a181043c0 2024-04-26T01:57:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.487+0000 7f3a0f400640 1 -- 172.21.15.151:0/3440634483 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a08037040 con 0x7f3a181043c0 2024-04-26T01:57:39.488 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.487+0000 7f3a1d400640 1 -- 172.21.15.151:0/3440634483 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a1819b600 con 0x7f3a181043c0 2024-04-26T01:57:39.488 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.487+0000 7f3a1d400640 1 -- 172.21.15.151:0/3440634483 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a181a7740 con 0x7f3a181043c0 2024-04-26T01:57:39.489 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.490+0000 7f3a0f400640 1 -- 172.21.15.151:0/3440634483 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f3a08043020 con 0x7f3a181043c0 2024-04-26T01:57:39.489 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.491+0000 7f3a2038a740 1 Processor -- start 2024-04-26T01:57:39.489 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.491+0000 7f3a2038a740 1 -- start start 2024-04-26T01:57:39.490 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.491+0000 7f3a2038a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a40a581820 0x55a40a581c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:39.490 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.491+0000 7f3a2038a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a40a582140 0x55a40a58c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:39.490 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.491+0000 7f3a2038a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a40a58ca00 0x55a40a58edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.491+0000 7f3a2038a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a40a4e7900 con 0x55a40a58ca00 2024-04-26T01:57:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.491+0000 7f3a2038a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a40a48def0 con 0x55a40a582140 2024-04-26T01:57:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.491+0000 7f3a2038a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a40a4e38c0 con 0x55a40a581820 2024-04-26T01:57:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.491+0000 7f3a17200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a40a581820 0x55a40a581c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.491+0000 7f3a0fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a40a58ca00 0x55a40a58edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.492+0000 7f3a17200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a40a581820 0x55a40a581c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:39202/0 (socket says 172.21.15.151:39202) 2024-04-26T01:57:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.492+0000 7f3a17200640 1 -- 172.21.15.151:0/655689204 learned_addr learned my addr 172.21.15.151:0/655689204 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:57:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.492+0000 7f3a16800640 1 --2- 172.21.15.151:0/655689204 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a40a582140 0x55a40a58c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.492+0000 7f3a17200640 1 -- 172.21.15.151:0/655689204 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a40a582140 msgr2=0x55a40a58c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.492+0000 7f3a17200640 1 --2- 172.21.15.151:0/655689204 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a40a582140 0x55a40a58c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.492+0000 7f3a17200640 1 -- 172.21.15.151:0/655689204 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a40a58ca00 msgr2=0x55a40a58edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.492+0000 7f3a17200640 1 --2- 172.21.15.151:0/655689204 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a40a58ca00 0x55a40a58edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.492+0000 7f3a17200640 1 -- 172.21.15.151:0/655689204 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a40a1fa210 con 0x55a40a581820 2024-04-26T01:57:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.492+0000 7f3a0f400640 1 --2- 172.21.15.151:0/3440634483 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f39fc0a3dc0 0x7f39fc0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.492+0000 7f3a0f400640 1 -- 172.21.15.151:0/3440634483 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f3a080f0330 con 0x7f3a181043c0 2024-04-26T01:57:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.492+0000 7f3a17200640 1 --2- 172.21.15.151:0/655689204 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a40a581820 0x55a40a581c00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3a10011790 tx=0x7f3a10011c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5149c96341637b3e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.492+0000 7f3a16800640 1 --2- 172.21.15.151:0/3440634483 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f39fc0a3dc0 0x7f39fc0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.493+0000 7f3a07400640 1 -- 172.21.15.151:0/655689204 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a10003c30 con 0x55a40a581820 2024-04-26T01:57:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.493+0000 7f3a07400640 1 -- 172.21.15.151:0/655689204 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a1000fe50 con 0x55a40a581820 2024-04-26T01:57:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.493+0000 7f3a07400640 1 -- 172.21.15.151:0/655689204 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a100193e0 con 0x55a40a581820 2024-04-26T01:57:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.494+0000 7f3a2038a740 1 -- 172.21.15.151:0/655689204 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a40a581820 msgr2=0x55a40a581c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.494+0000 7f3a16800640 1 --2- 172.21.15.151:0/3440634483 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f39fc0a3dc0 0x7f39fc0a6280 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x55a40a582dc0 tx=0x7f3a080b8040 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.494+0000 7f3a2038a740 1 --2- 172.21.15.151:0/655689204 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a40a581820 0x55a40a581c00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3a10011790 tx=0x7f3a10011c50 comp rx=0 tx=0).stop 2024-04-26T01:57:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.494+0000 7f3a2038a740 1 -- 172.21.15.151:0/655689204 shutdown_connections 2024-04-26T01:57:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.494+0000 7f3a2038a740 1 --2- 172.21.15.151:0/655689204 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a40a58ca00 0x55a40a58edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.494+0000 7f3a2038a740 1 --2- 172.21.15.151:0/655689204 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a40a582140 0x55a40a58c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.494+0000 7f3a2038a740 1 --2- 172.21.15.151:0/655689204 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a40a581820 0x55a40a581c00 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:39.495 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.494+0000 7f3a2038a740 1 -- 172.21.15.151:0/655689204 >> 172.21.15.151:0/655689204 conn(0x55a40a57dd30 msgr2=0x55a40a57e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:39.495 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.494+0000 7f3a2038a740 1 -- 172.21.15.151:0/655689204 shutdown_connections 2024-04-26T01:57:39.495 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.494+0000 7f3a2038a740 1 -- 172.21.15.151:0/655689204 wait complete. 2024-04-26T01:57:39.495 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.494+0000 7f3a2038a740 1 -- start start 2024-04-26T01:57:39.495 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.495+0000 7f3a2038a740 1 Processor -- start 2024-04-26T01:57:39.498 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.495+0000 7f3a2038a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a40a581820 0x55a40a5899e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:39.498 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.495+0000 7f3a2038a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a40a582140 0x55a40a589f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:39.498 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.495+0000 7f3a2038a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a40a58ca00 0x55a40a58a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:39.498 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.495+0000 7f3a2038a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a40a58f630 con 0x55a40a58ca00 2024-04-26T01:57:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.495+0000 7f3a2038a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a40a4a1310 con 0x55a40a582140 2024-04-26T01:57:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.495+0000 7f3a2038a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a40a58f7b0 con 0x55a40a581820 2024-04-26T01:57:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.495+0000 7f3a2038a740 10 client.0 _ll_register_callbacks cb 0x55a40a46fcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:57:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.495+0000 7f3a17200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a40a58ca00 0x55a40a58a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.496+0000 7f3a0fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a40a582140 0x55a40a589f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.496+0000 7f3a17200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a40a581820 0x55a40a5899e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.496+0000 7f3a0fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a40a582140 0x55a40a589f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:57658/0 (socket says 172.21.15.151:57658) 2024-04-26T01:57:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.496+0000 7f3a0fe00640 1 -- 172.21.15.151:0/186173081 learned_addr learned my addr 172.21.15.151:0/186173081 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:57:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.496+0000 7f3a0fe00640 1 -- 172.21.15.151:0/186173081 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a40a581820 msgr2=0x55a40a5899e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.496+0000 7f3a0fe00640 1 --2- 172.21.15.151:0/186173081 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a40a581820 0x55a40a5899e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.496+0000 7f3a0fe00640 1 -- 172.21.15.151:0/186173081 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a40a58ca00 msgr2=0x55a40a58a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.496+0000 7f3a0fe00640 1 --2- 172.21.15.151:0/186173081 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a40a58ca00 0x55a40a58a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.496+0000 7f3a0fe00640 1 -- 172.21.15.151:0/186173081 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a40a2c6620 con 0x55a40a582140 2024-04-26T01:57:39.500 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.496+0000 7f3a0fe00640 1 --2- 172.21.15.151:0/186173081 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a40a582140 0x55a40a589f20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3a00011ce0 tx=0x7f3a0000f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f5672ea8bcd43cb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:39.500 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.496+0000 7f39fbe00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T01:57:39.500 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.496+0000 7f39fbe00640 1 -- 172.21.15.151:0/186173081 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0001e020 con 0x55a40a582140 2024-04-26T01:57:39.500 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.497+0000 7f3a2038a740 1 -- 172.21.15.151:0/186173081 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a1000fe50 con 0x55a40a582140 2024-04-26T01:57:39.500 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.497+0000 7f39fbe00640 1 -- 172.21.15.151:0/186173081 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f3a000047a0 con 0x55a40a582140 2024-04-26T01:57:39.500 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.497+0000 7f39fbe00640 1 -- 172.21.15.151:0/186173081 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3a000049a0 con 0x55a40a582140 2024-04-26T01:57:39.500 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.497+0000 7f3a2038a740 10 client.25640 fetch_fsmap learned FSMap version 29 2024-04-26T01:57:39.500 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.497+0000 7f3a2038a740 1 -- 172.21.15.151:0/186173081 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a40a59b940 con 0x55a40a582140 2024-04-26T01:57:39.500 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.497+0000 7f39fbe00640 1 -- 172.21.15.151:0/186173081 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a1000fe50 con 0x55a40a582140 2024-04-26T01:57:39.500 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.497+0000 7f39fbe00640 1 -- 172.21.15.151:0/186173081 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 29) ==== 8342+0+0 (secure 0 0 0) 0x7f3a00019070 con 0x55a40a582140 2024-04-26T01:57:39.500 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.497+0000 7f3a2038a740 10 client.25640 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T01:57:39.500 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.497+0000 7f3a2038a740 10 client.25640 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:57:39.500 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.497+0000 7f3a2038a740 20 client.25640 populate_metadata read hostname 'smithi151' 2024-04-26T01:57:39.501 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.497+0000 7f3a2038a740 1 --2- 172.21.15.151:0/186173081 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55a40a59cec0 0x55a40a59f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:39.501 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.497+0000 7f3a2038a740 4 client.25640 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:57:39.501 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.497+0000 7f3a2038a740 1 -- 172.21.15.151:0/186173081 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a40a4fb8b0 con 0x55a40a59cec0 2024-04-26T01:57:39.501 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.498+0000 7f3a17200640 1 --2- 172.21.15.151:0/186173081 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55a40a59cec0 0x55a40a59f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:39.501 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.498+0000 7f3a17200640 1 --2- 172.21.15.151:0/186173081 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55a40a59cec0 0x55a40a59f2d0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7744ebdcbea00778 server_cookie=fbee9544695d07db in_seq=0 out_seq=0 2024-04-26T01:57:39.501 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.498+0000 7f39fbe00640 10 client.25640 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:57:39.501 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.499+0000 7f39fbe00640 1 -- 172.21.15.151:0/186173081 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a40a4fb8b0 con 0x55a40a59cec0 2024-04-26T01:57:39.501 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.499+0000 7f39fbe00640 10 client.25640 handle_command_reply: tid=0 2024-04-26T01:57:39.501 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.499+0000 7f3a2038a740 1 client.25640 shutdown 2024-04-26T01:57:39.501 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.499+0000 7f3a2038a740 1 -- 172.21.15.151:0/186173081 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a40a582140 msgr2=0x55a40a589f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:39.501 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.499+0000 7f3a2038a740 1 --2- 172.21.15.151:0/186173081 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a40a582140 0x55a40a589f20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3a00011ce0 tx=0x7f3a0000f790 comp rx=0 tx=0).stop 2024-04-26T01:57:39.501 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.499+0000 7f3a2038a740 1 -- 172.21.15.151:0/186173081 shutdown_connections 2024-04-26T01:57:39.501 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.499+0000 7f3a2038a740 1 --2- 172.21.15.151:0/186173081 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55a40a59cec0 0x55a40a59f2d0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:39.502 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.499+0000 7f3a2038a740 1 --2- 172.21.15.151:0/186173081 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a40a58ca00 0x55a40a58a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:39.502 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.499+0000 7f3a2038a740 1 --2- 172.21.15.151:0/186173081 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a40a582140 0x55a40a589f20 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:39.502 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.500+0000 7f39fbe00640 0 client.25640 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:57:39.502 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.500+0000 7f3a2038a740 1 --2- 172.21.15.151:0/186173081 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a40a581820 0x55a40a5899e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:39.502 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.500+0000 7f3a2038a740 1 -- 172.21.15.151:0/186173081 >> 172.21.15.151:0/186173081 conn(0x55a40a57dd30 msgr2=0x55a40a583a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:39.502 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.500+0000 7f3a2038a740 1 -- 172.21.15.151:0/186173081 shutdown_connections 2024-04-26T01:57:39.502 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.500+0000 7f3a2038a740 1 -- 172.21.15.151:0/186173081 wait complete. 2024-04-26T01:57:39.502 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.500+0000 7f3a2038a740 20 client.25640 trim_cache size 0 max 16384 2024-04-26T01:57:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.516+0000 7f3a2038a740 1 Processor -- start 2024-04-26T01:57:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.516+0000 7f3a2038a740 1 -- start start 2024-04-26T01:57:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.517+0000 7f3a2038a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a40a581820 0x55a40a57d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.517+0000 7f3a2038a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a40a582140 0x55a40a5838e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.517+0000 7f3a2038a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a40a58ca00 0x55a40a5a4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.517+0000 7f3a2038a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3a181b24d0 con 0x55a40a58ca00 2024-04-26T01:57:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.517+0000 7f3a2038a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3a181afda0 con 0x55a40a582140 2024-04-26T01:57:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.517+0000 7f3a2038a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3a181adc10 con 0x55a40a581820 2024-04-26T01:57:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.517+0000 7f3a17200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a40a58ca00 0x55a40a5a4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.517+0000 7f3a17200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a40a581820 0x55a40a57d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.517+0000 7f3a17200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a40a581820 0x55a40a57d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:39224/0 (socket says 172.21.15.151:39224) 2024-04-26T01:57:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.517+0000 7f3a17200640 1 -- 172.21.15.151:0/2965773328 learned_addr learned my addr 172.21.15.151:0/2965773328 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:57:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.517+0000 7f3a0fe00640 1 --2- 172.21.15.151:0/2965773328 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a40a582140 0x55a40a5838e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.517+0000 7f3a17200640 1 -- 172.21.15.151:0/2965773328 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a40a582140 msgr2=0x55a40a5838e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.517+0000 7f3a17200640 1 --2- 172.21.15.151:0/2965773328 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a40a582140 0x55a40a5838e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.517+0000 7f3a17200640 1 -- 172.21.15.151:0/2965773328 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a40a58ca00 msgr2=0x55a40a5a4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.517+0000 7f3a17200640 1 --2- 172.21.15.151:0/2965773328 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a40a58ca00 0x55a40a5a4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.517+0000 7f3a17200640 1 -- 172.21.15.151:0/2965773328 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a40a59f950 con 0x55a40a581820 2024-04-26T01:57:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.518+0000 7f3a17200640 1 --2- 172.21.15.151:0/2965773328 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a40a581820 0x55a40a57d610 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3a1000f790 tx=0x7f3a10005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9832b539f1a024c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.518+0000 7f3a06a00640 1 -- 172.21.15.151:0/2965773328 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a10004050 con 0x55a40a581820 2024-04-26T01:57:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.518+0000 7f3a06a00640 1 -- 172.21.15.151:0/2965773328 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a100059f0 con 0x55a40a581820 2024-04-26T01:57:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.518+0000 7f3a06a00640 1 -- 172.21.15.151:0/2965773328 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a10004890 con 0x55a40a581820 2024-04-26T01:57:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.519+0000 7f3a2038a740 1 -- 172.21.15.151:0/2965773328 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a40a581820 msgr2=0x55a40a57d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.519+0000 7f3a2038a740 1 --2- 172.21.15.151:0/2965773328 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a40a581820 0x55a40a57d610 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3a1000f790 tx=0x7f3a10005bd0 comp rx=0 tx=0).stop 2024-04-26T01:57:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.519+0000 7f3a2038a740 1 -- 172.21.15.151:0/2965773328 shutdown_connections 2024-04-26T01:57:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.519+0000 7f3a2038a740 1 --2- 172.21.15.151:0/2965773328 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a40a58ca00 0x55a40a5a4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.519+0000 7f3a2038a740 1 --2- 172.21.15.151:0/2965773328 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a40a582140 0x55a40a5838e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.519+0000 7f3a2038a740 1 --2- 172.21.15.151:0/2965773328 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a40a581820 0x55a40a57d610 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.519+0000 7f3a2038a740 1 -- 172.21.15.151:0/2965773328 >> 172.21.15.151:0/2965773328 conn(0x55a40a57dd30 msgr2=0x55a40a5831d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.519+0000 7f3a2038a740 1 -- 172.21.15.151:0/2965773328 shutdown_connections 2024-04-26T01:57:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.519+0000 7f3a2038a740 1 -- 172.21.15.151:0/2965773328 wait complete. 2024-04-26T01:57:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.519+0000 7f3a2038a740 1 -- start start 2024-04-26T01:57:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.520+0000 7f3a2038a740 1 Processor -- start 2024-04-26T01:57:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.520+0000 7f3a2038a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a40a581820 0x55a40a5924d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.520+0000 7f3a2038a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a40a582140 0x55a40a592a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.520+0000 7f3a2038a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a40a58ca00 0x55a40a592f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.520+0000 7f3a2038a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3a181ae260 con 0x55a40a582140 2024-04-26T01:57:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.520+0000 7f3a2038a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3a181af6d0 con 0x55a40a58ca00 2024-04-26T01:57:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.520+0000 7f3a2038a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3a181ae930 con 0x55a40a581820 2024-04-26T01:57:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.520+0000 7f3a2038a740 10 client.0 _ll_register_callbacks cb 0x7f3a000117e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:57:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.520+0000 7f3a0fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a40a582140 0x55a40a592a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.520+0000 7f3a17200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a40a581820 0x55a40a5924d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.520+0000 7f3a0fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a40a582140 0x55a40a592a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:51584/0 (socket says 172.21.15.151:51584) 2024-04-26T01:57:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.520+0000 7f3a0fe00640 1 -- 172.21.15.151:0/1132247925 learned_addr learned my addr 172.21.15.151:0/1132247925 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:57:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.520+0000 7f3a17200640 1 --2- 172.21.15.151:0/1132247925 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a40a58ca00 0x55a40a592f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.521+0000 7f3a17200640 1 -- 172.21.15.151:0/1132247925 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a40a58ca00 msgr2=0x55a40a592f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.521+0000 7f3a17200640 1 --2- 172.21.15.151:0/1132247925 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a40a58ca00 0x55a40a592f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.521+0000 7f3a17200640 1 -- 172.21.15.151:0/1132247925 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a40a582140 msgr2=0x55a40a592a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.521+0000 7f3a17200640 1 --2- 172.21.15.151:0/1132247925 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a40a582140 0x55a40a592a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.521+0000 7f3a17200640 1 -- 172.21.15.151:0/1132247925 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a40a6024b0 con 0x55a40a581820 2024-04-26T01:57:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.521+0000 7f3a0fe00640 1 --2- 172.21.15.151:0/1132247925 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a40a582140 0x55a40a592a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:57:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.521+0000 7f3a17200640 1 --2- 172.21.15.151:0/1132247925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a40a581820 0x55a40a5924d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3a100115a0 tx=0x7f3a10000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23d2045c52f0fea2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.521+0000 7f39fbe00640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-26T01:57:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.521+0000 7f39fbe00640 1 -- 172.21.15.151:0/1132247925 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a10003d60 con 0x55a40a581820 2024-04-26T01:57:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.521+0000 7f3a2038a740 1 -- 172.21.15.151:0/1132247925 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a100059f0 con 0x55a40a581820 2024-04-26T01:57:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.521+0000 7f39fbe00640 1 -- 172.21.15.151:0/1132247925 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f3a1001f070 con 0x55a40a581820 2024-04-26T01:57:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.522+0000 7f39fbe00640 1 -- 172.21.15.151:0/1132247925 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3a10011020 con 0x55a40a581820 2024-04-26T01:57:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.522+0000 7f3a2038a740 10 client.25678 fetch_fsmap learned FSMap version 29 2024-04-26T01:57:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.522+0000 7f3a2038a740 1 -- 172.21.15.151:0/1132247925 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a40a6037a0 con 0x55a40a581820 2024-04-26T01:57:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.522+0000 7f39fbe00640 1 -- 172.21.15.151:0/1132247925 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a100059f0 con 0x55a40a581820 2024-04-26T01:57:39.524 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.522+0000 7f39fbe00640 1 -- 172.21.15.151:0/1132247925 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8342+0+0 (secure 0 0 0) 0x7f3a10018070 con 0x55a40a581820 2024-04-26T01:57:39.524 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.522+0000 7f3a2038a740 10 client.25678 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T01:57:39.524 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.522+0000 7f3a2038a740 10 client.25678 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:57:39.524 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.522+0000 7f3a2038a740 20 client.25678 populate_metadata read hostname 'smithi151' 2024-04-26T01:57:39.524 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.522+0000 7f3a2038a740 1 --2- 172.21.15.151:0/1132247925 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55a40a603c10 0x55a40a606000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:39.524 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.522+0000 7f3a2038a740 4 client.25678 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:57:39.524 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.522+0000 7f3a2038a740 1 -- 172.21.15.151:0/1132247925 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a40a4f5320 con 0x55a40a603c10 2024-04-26T01:57:39.524 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.523+0000 7f3a0fe00640 1 --2- 172.21.15.151:0/1132247925 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55a40a603c10 0x55a40a606000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.523+0000 7f3a0fe00640 1 --2- 172.21.15.151:0/1132247925 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55a40a603c10 0x55a40a606000 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69aa31002246dea1 server_cookie=fccb6ff52d1f637 in_seq=0 out_seq=0 2024-04-26T01:57:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.523+0000 7f39fbe00640 10 client.25678 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:57:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.541+0000 7f39fbe00640 1 -- 172.21.15.151:0/1132247925 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a40a4f5320 con 0x55a40a603c10 2024-04-26T01:57:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.541+0000 7f39fbe00640 10 client.25678 handle_command_reply: tid=0 2024-04-26T01:57:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.541+0000 7f3a2038a740 1 client.25678 shutdown 2024-04-26T01:57:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.541+0000 7f3a2038a740 1 -- 172.21.15.151:0/1132247925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a40a581820 msgr2=0x55a40a5924d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.541+0000 7f3a2038a740 1 --2- 172.21.15.151:0/1132247925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a40a581820 0x55a40a5924d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3a100115a0 tx=0x7f3a10000f40 comp rx=0 tx=0).stop 2024-04-26T01:57:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.541+0000 7f3a2038a740 1 -- 172.21.15.151:0/1132247925 shutdown_connections 2024-04-26T01:57:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.541+0000 7f3a2038a740 1 --2- 172.21.15.151:0/1132247925 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55a40a603c10 0x55a40a606000 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.541+0000 7f3a2038a740 1 --2- 172.21.15.151:0/1132247925 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a40a58ca00 0x55a40a592f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.541+0000 7f3a2038a740 1 --2- 172.21.15.151:0/1132247925 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a40a582140 0x55a40a592a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.541+0000 7f3a2038a740 1 --2- 172.21.15.151:0/1132247925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a40a581820 0x55a40a5924d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.541+0000 7f39fbe00640 0 client.25678 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:57:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.541+0000 7f3a2038a740 1 -- 172.21.15.151:0/1132247925 >> 172.21.15.151:0/1132247925 conn(0x55a40a57dd30 msgr2=0x55a40a583a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.541+0000 7f3a2038a740 1 -- 172.21.15.151:0/1132247925 shutdown_connections 2024-04-26T01:57:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.541+0000 7f3a2038a740 1 -- 172.21.15.151:0/1132247925 wait complete. 2024-04-26T01:57:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.542+0000 7f3a2038a740 20 client.25678 trim_cache size 0 max 16384 2024-04-26T01:57:39.542 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:{ 2024-04-26T01:57:39.542 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "message": "", 2024-04-26T01:57:39.542 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "return_code": 0 2024-04-26T01:57:39.542 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:} 2024-04-26T01:57:39.542 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.542+0000 7f3a1d400640 1 -- 172.21.15.151:0/3440634483 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f39fc0a3dc0 msgr2=0x7f39fc0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.542+0000 7f3a1d400640 1 --2- 172.21.15.151:0/3440634483 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f39fc0a3dc0 0x7f39fc0a6280 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x55a40a582dc0 tx=0x7f3a080b8040 comp rx=0 tx=0).stop 2024-04-26T01:57:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.542+0000 7f3a1d400640 1 -- 172.21.15.151:0/3440634483 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3a181043c0 msgr2=0x7f3a181a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.542+0000 7f3a1d400640 1 --2- 172.21.15.151:0/3440634483 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3a181043c0 0x7f3a181a06b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3a080303f0 tx=0x7f3a08032da0 comp rx=0 tx=0).stop 2024-04-26T01:57:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.543+0000 7f3a1d400640 1 -- 172.21.15.151:0/3440634483 shutdown_connections 2024-04-26T01:57:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.543+0000 7f3a1d400640 1 --2- 172.21.15.151:0/3440634483 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f39fc0a3dc0 0x7f39fc0a6280 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.543+0000 7f3a1d400640 1 --2- 172.21.15.151:0/3440634483 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3a1819a770 0x7f3a1819abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.543+0000 7f3a1d400640 1 --2- 172.21.15.151:0/3440634483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a18109490 0x7f3a181a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.543+0000 7f3a1d400640 1 --2- 172.21.15.151:0/3440634483 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3a181043c0 0x7f3a181a06b0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.543+0000 7f3a1d400640 1 -- 172.21.15.151:0/3440634483 >> 172.21.15.151:0/3440634483 conn(0x7f3a180ffc20 msgr2=0x7f3a18101f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.543+0000 7f3a1d400640 1 -- 172.21.15.151:0/3440634483 shutdown_connections 2024-04-26T01:57:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:39.543+0000 7f3a1d400640 1 -- 172.21.15.151:0/3440634483 wait complete. 2024-04-26T01:57:39.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:57:39 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/3387038063' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:57:39.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:57:39 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/3387038063' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:57:39.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:57:39 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/3387038063' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:57:39.904 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:57:40.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:57:40 smithi165 ceph-mon[32648]: pgmap v124: 97 pgs: 97 active+clean; 36 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 78 KiB/s wr, 13 op/s 2024-04-26T01:57:40.598 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:57:40 smithi151 ceph-mon[28057]: pgmap v124: 97 pgs: 97 active+clean; 36 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 78 KiB/s wr, 13 op/s 2024-04-26T01:57:40.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:57:40 smithi172 ceph-mon[32785]: pgmap v124: 97 pgs: 97 active+clean; 36 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 78 KiB/s wr, 13 op/s 2024-04-26T01:57:42.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.260+0000 7f024fe00700 1 -- 172.21.15.151:0/480457442 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0248109c80 msgr2=0x7f024810dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:42.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.260+0000 7f024fe00700 1 --2- 172.21.15.151:0/480457442 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0248109c80 0x7f024810dcd0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0234009990 tx=0x7f0234009ca0 comp rx=0 tx=0).stop 2024-04-26T01:57:42.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.260+0000 7f024fe00700 1 -- 172.21.15.151:0/480457442 shutdown_connections 2024-04-26T01:57:42.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.260+0000 7f024fe00700 1 --2- 172.21.15.151:0/480457442 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f024810e3b0 0x7f0248115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:42.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.260+0000 7f024fe00700 1 --2- 172.21.15.151:0/480457442 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02481092d0 0x7f02481096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:42.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.260+0000 7f024fe00700 1 --2- 172.21.15.151:0/480457442 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0248109c80 0x7f024810dcd0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:42.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.260+0000 7f024fe00700 1 -- 172.21.15.151:0/480457442 >> 172.21.15.151:0/480457442 conn(0x7f0248076210 msgr2=0x7f0248107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:42.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.261+0000 7f024fe00700 1 -- 172.21.15.151:0/480457442 shutdown_connections 2024-04-26T01:57:42.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.261+0000 7f024fe00700 1 -- 172.21.15.151:0/480457442 wait complete. 2024-04-26T01:57:42.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.262+0000 7f024fe00700 1 Processor -- start 2024-04-26T01:57:42.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.262+0000 7f024fe00700 1 -- start start 2024-04-26T01:57:42.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.262+0000 7f024fe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02481092d0 0x7f02481a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:42.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.262+0000 7f024fe00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0248109c80 0x7f02481a5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:42.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.262+0000 7f024fe00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f024810e3b0 0x7f024819f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:42.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.262+0000 7f024fe00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0248119290 con 0x7f02481092d0 2024-04-26T01:57:42.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.262+0000 7f024fe00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0248119110 con 0x7f0248109c80 2024-04-26T01:57:42.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.262+0000 7f024fe00700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0248119410 con 0x7f024810e3b0 2024-04-26T01:57:42.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.263+0000 7f024d400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02481092d0 0x7f02481a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:42.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.263+0000 7f024ca00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0248109c80 0x7f02481a5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:42.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.263+0000 7f024ca00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0248109c80 0x7f02481a5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:34652/0 (socket says 172.21.15.151:34652) 2024-04-26T01:57:42.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.263+0000 7f024ca00700 1 -- 172.21.15.151:0/4239506900 learned_addr learned my addr 172.21.15.151:0/4239506900 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:57:42.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.263+0000 7f024d400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02481092d0 0x7f02481a5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:45084/0 (socket says 172.21.15.151:45084) 2024-04-26T01:57:42.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.263+0000 7f024de00700 1 --2- 172.21.15.151:0/4239506900 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f024810e3b0 0x7f024819f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:42.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.263+0000 7f024ca00700 1 -- 172.21.15.151:0/4239506900 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f024810e3b0 msgr2=0x7f024819f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:42.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.263+0000 7f024ca00700 1 --2- 172.21.15.151:0/4239506900 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f024810e3b0 0x7f024819f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:42.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.263+0000 7f024ca00700 1 -- 172.21.15.151:0/4239506900 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02481092d0 msgr2=0x7f02481a5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:42.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.263+0000 7f024ca00700 1 --2- 172.21.15.151:0/4239506900 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02481092d0 0x7f02481a5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:42.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.263+0000 7f024ca00700 1 -- 172.21.15.151:0/4239506900 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f024819fe50 con 0x7f0248109c80 2024-04-26T01:57:42.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.264+0000 7f024ca00700 1 --2- 172.21.15.151:0/4239506900 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0248109c80 0x7f02481a5a90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0234000c00 tx=0x7f02340057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:42.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.264+0000 7f0246000700 1 -- 172.21.15.151:0/4239506900 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0234027070 con 0x7f0248109c80 2024-04-26T01:57:42.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.264+0000 7f024fe00700 1 -- 172.21.15.151:0/4239506900 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02481a00e0 con 0x7f0248109c80 2024-04-26T01:57:42.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.264+0000 7f0246000700 1 -- 172.21.15.151:0/4239506900 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f023400b610 con 0x7f0248109c80 2024-04-26T01:57:42.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.264+0000 7f024fe00700 1 -- 172.21.15.151:0/4239506900 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02481ac370 con 0x7f0248109c80 2024-04-26T01:57:42.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.264+0000 7f0246000700 1 -- 172.21.15.151:0/4239506900 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0234022e50 con 0x7f0248109c80 2024-04-26T01:57:42.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.267+0000 7f0246000700 1 -- 172.21.15.151:0/4239506900 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f023402a070 con 0x7f0248109c80 2024-04-26T01:57:42.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.269+0000 7f025a792b80 1 Processor -- start 2024-04-26T01:57:42.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.269+0000 7f0246000700 1 --2- 172.21.15.151:0/4239506900 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f02300a3fd0 0x7f02300a6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:42.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.269+0000 7f025a792b80 1 -- start start 2024-04-26T01:57:42.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.269+0000 7f0246000700 1 -- 172.21.15.151:0/4239506900 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f02340d1420 con 0x7f0248109c80 2024-04-26T01:57:42.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.269+0000 7f024de00700 1 --2- 172.21.15.151:0/4239506900 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f02300a3fd0 0x7f02300a6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:42.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.269+0000 7f025a792b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa5d58bd40 0x55fa5d58c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:42.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.269+0000 7f025a792b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fa5d58c660 0x55fa5d596f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:42.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.269+0000 7f025a792b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa5d5974d0 0x55fa5d5998c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:42.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.269+0000 7f025a792b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55fa5d4def90 con 0x55fa5d58bd40 2024-04-26T01:57:42.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.269+0000 7f025a792b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55fa5d485830 con 0x55fa5d5974d0 2024-04-26T01:57:42.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.269+0000 7f025a792b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55fa5d4daf50 con 0x55fa5d58c660 2024-04-26T01:57:42.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.270+0000 7f024d400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa5d58bd40 0x55fa5d58c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:42.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.270+0000 7f024d400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa5d58bd40 0x55fa5d58c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:45096/0 (socket says 172.21.15.151:45096) 2024-04-26T01:57:42.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.270+0000 7f024d400700 1 -- 172.21.15.151:0/3931262880 learned_addr learned my addr 172.21.15.151:0/3931262880 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:57:42.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.270+0000 7f024de00700 1 --2- 172.21.15.151:0/3931262880 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa5d5974d0 0x55fa5d5998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:42.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.270+0000 7f024ca00700 1 --2- 172.21.15.151:0/3931262880 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fa5d58c660 0x55fa5d596f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:42.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.270+0000 7f024d400700 1 -- 172.21.15.151:0/3931262880 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fa5d58c660 msgr2=0x55fa5d596f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:42.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.270+0000 7f024d400700 1 --2- 172.21.15.151:0/3931262880 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fa5d58c660 0x55fa5d596f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:42.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.270+0000 7f024d400700 1 -- 172.21.15.151:0/3931262880 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa5d5974d0 msgr2=0x55fa5d5998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:42.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.270+0000 7f024d400700 1 --2- 172.21.15.151:0/3931262880 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa5d5974d0 0x55fa5d5998c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:42.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.270+0000 7f024d400700 1 -- 172.21.15.151:0/3931262880 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa5d47b780 con 0x55fa5d58bd40 2024-04-26T01:57:42.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.270+0000 7f024de00700 1 --2- 172.21.15.151:0/4239506900 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f02300a3fd0 0x7f02300a6490 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f024000f870 tx=0x7f024000fc30 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:42.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.271+0000 7f024d400700 1 --2- 172.21.15.151:0/3931262880 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa5d58bd40 0x55fa5d58c120 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f023c009d80 tx=0x7f023c011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebef39aa1dac6d49 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:42.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.271+0000 7f023aa00700 1 -- 172.21.15.151:0/3931262880 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f023c004230 con 0x55fa5d58bd40 2024-04-26T01:57:42.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.271+0000 7f023aa00700 1 -- 172.21.15.151:0/3931262880 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f023c0043d0 con 0x55fa5d58bd40 2024-04-26T01:57:42.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.271+0000 7f023aa00700 1 -- 172.21.15.151:0/3931262880 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f023c005610 con 0x55fa5d58bd40 2024-04-26T01:57:42.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.272+0000 7f025a792b80 1 -- 172.21.15.151:0/3931262880 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa5d58bd40 msgr2=0x55fa5d58c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:42.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.272+0000 7f025a792b80 1 --2- 172.21.15.151:0/3931262880 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa5d58bd40 0x55fa5d58c120 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f023c009d80 tx=0x7f023c011720 comp rx=0 tx=0).stop 2024-04-26T01:57:42.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.272+0000 7f025a792b80 1 -- 172.21.15.151:0/3931262880 shutdown_connections 2024-04-26T01:57:42.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.272+0000 7f025a792b80 1 --2- 172.21.15.151:0/3931262880 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa5d5974d0 0x55fa5d5998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:42.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.272+0000 7f025a792b80 1 --2- 172.21.15.151:0/3931262880 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa5d58bd40 0x55fa5d58c120 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:42.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.272+0000 7f025a792b80 1 --2- 172.21.15.151:0/3931262880 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fa5d58c660 0x55fa5d596f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:42.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.272+0000 7f025a792b80 1 -- 172.21.15.151:0/3931262880 >> 172.21.15.151:0/3931262880 conn(0x55fa5d4bb050 msgr2=0x55fa5d588250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:42.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.272+0000 7f025a792b80 1 -- 172.21.15.151:0/3931262880 shutdown_connections 2024-04-26T01:57:42.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.272+0000 7f025a792b80 1 -- 172.21.15.151:0/3931262880 wait complete. 2024-04-26T01:57:42.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.273+0000 7f025a792b80 1 -- start start 2024-04-26T01:57:42.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.273+0000 7f025a792b80 1 Processor -- start 2024-04-26T01:57:42.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.274+0000 7f025a792b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa5d58bd40 0x55fa5d596be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:42.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.274+0000 7f025a792b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fa5d58c660 0x55fa5d593ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:42.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.274+0000 7f025a792b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa5d5974d0 0x55fa5d594420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:42.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.274+0000 7f025a792b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55fa5d599fc0 con 0x55fa5d58bd40 2024-04-26T01:57:42.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.274+0000 7f025a792b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55fa5d498960 con 0x55fa5d5974d0 2024-04-26T01:57:42.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.274+0000 7f025a792b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55fa5d59a140 con 0x55fa5d58c660 2024-04-26T01:57:42.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.274+0000 7f025a792b80 10 client.0 _ll_register_callbacks cb 0x55fa5d477820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:57:42.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.274+0000 7f024d400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa5d58bd40 0x55fa5d596be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:42.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.274+0000 7f024ca00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fa5d58c660 0x55fa5d593ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:42.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.274+0000 7f024de00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa5d5974d0 0x55fa5d594420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:42.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.274+0000 7f024ca00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fa5d58c660 0x55fa5d593ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:34124/0 (socket says 172.21.15.151:34124) 2024-04-26T01:57:42.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.274+0000 7f024ca00700 1 -- 172.21.15.151:0/18275911 learned_addr learned my addr 172.21.15.151:0/18275911 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:57:42.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.274+0000 7f024ca00700 1 -- 172.21.15.151:0/18275911 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa5d5974d0 msgr2=0x55fa5d594420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:42.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.274+0000 7f024ca00700 1 --2- 172.21.15.151:0/18275911 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa5d5974d0 0x55fa5d594420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:42.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.274+0000 7f024ca00700 1 -- 172.21.15.151:0/18275911 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa5d58bd40 msgr2=0x55fa5d596be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:42.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.274+0000 7f024ca00700 1 --2- 172.21.15.151:0/18275911 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa5d58bd40 0x55fa5d596be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:42.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.274+0000 7f024ca00700 1 -- 172.21.15.151:0/18275911 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa5d42c9d0 con 0x55fa5d58c660 2024-04-26T01:57:42.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.275+0000 7f024ca00700 1 --2- 172.21.15.151:0/18275911 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fa5d58c660 0x55fa5d593ee0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f023409d3f0 tx=0x7f023409d2b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=712ae4cb297bc001 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:42.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.275+0000 7f022f400700 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-26T01:57:42.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.275+0000 7f022f400700 1 -- 172.21.15.151:0/18275911 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f023409db60 con 0x55fa5d58c660 2024-04-26T01:57:42.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.275+0000 7f022f400700 1 -- 172.21.15.151:0/18275911 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f02340a5bb0 con 0x55fa5d58c660 2024-04-26T01:57:42.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.275+0000 7f025a792b80 1 -- 172.21.15.151:0/18275911 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f023c0043d0 con 0x55fa5d58c660 2024-04-26T01:57:42.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.275+0000 7f022f400700 1 -- 172.21.15.151:0/18275911 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f02340bb050 con 0x55fa5d58c660 2024-04-26T01:57:42.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.275+0000 7f025a792b80 10 client.25690 fetch_fsmap learned FSMap version 29 2024-04-26T01:57:42.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.275+0000 7f025a792b80 1 -- 172.21.15.151:0/18275911 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fa5d475bf0 con 0x55fa5d58c660 2024-04-26T01:57:42.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.275+0000 7f022f400700 1 -- 172.21.15.151:0/18275911 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f023c0043d0 con 0x55fa5d58c660 2024-04-26T01:57:42.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.276+0000 7f022f400700 1 -- 172.21.15.151:0/18275911 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8342+0+0 (secure 0 0 0) 0x7f023400fad0 con 0x55fa5d58c660 2024-04-26T01:57:42.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.276+0000 7f025a792b80 10 client.25690 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T01:57:42.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.276+0000 7f025a792b80 10 client.25690 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-26T01:57:42.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.276+0000 7f025a792b80 20 client.25690 populate_metadata read hostname 'smithi151' 2024-04-26T01:57:42.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.276+0000 7f025a792b80 1 --2- 172.21.15.151:0/18275911 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55fa5d5a6cf0 0x55fa5d5a9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:42.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.276+0000 7f025a792b80 4 client.25690 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:57:42.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.276+0000 7f025a792b80 1 -- 172.21.15.151:0/18275911 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa5d4ec780 con 0x55fa5d5a6cf0 2024-04-26T01:57:42.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.276+0000 7f024d400700 1 --2- 172.21.15.151:0/18275911 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55fa5d5a6cf0 0x55fa5d5a9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:42.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.277+0000 7f024d400700 1 --2- 172.21.15.151:0/18275911 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55fa5d5a6cf0 0x55fa5d5a9170 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f8acecac5f25bbf server_cookie=ac30d237c1b1e85b in_seq=0 out_seq=0 2024-04-26T01:57:42.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.277+0000 7f022f400700 10 client.25690 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:57:42.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.278+0000 7f022f400700 1 -- 172.21.15.151:0/18275911 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fa5d4ec780 con 0x55fa5d5a6cf0 2024-04-26T01:57:42.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.278+0000 7f022f400700 10 client.25690 handle_command_reply: tid=0 2024-04-26T01:57:42.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.278+0000 7f025a792b80 1 client.25690 shutdown 2024-04-26T01:57:42.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.278+0000 7f025a792b80 1 -- 172.21.15.151:0/18275911 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fa5d58c660 msgr2=0x55fa5d593ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:42.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.278+0000 7f025a792b80 1 --2- 172.21.15.151:0/18275911 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fa5d58c660 0x55fa5d593ee0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f023409d3f0 tx=0x7f023409d2b0 comp rx=0 tx=0).stop 2024-04-26T01:57:42.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.278+0000 7f025a792b80 1 -- 172.21.15.151:0/18275911 shutdown_connections 2024-04-26T01:57:42.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.278+0000 7f025a792b80 1 --2- 172.21.15.151:0/18275911 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55fa5d5a6cf0 0x55fa5d5a9170 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:42.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.278+0000 7f025a792b80 1 --2- 172.21.15.151:0/18275911 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa5d5974d0 0x55fa5d594420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:42.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.278+0000 7f025a792b80 1 --2- 172.21.15.151:0/18275911 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa5d58bd40 0x55fa5d596be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:42.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.278+0000 7f025a792b80 1 --2- 172.21.15.151:0/18275911 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fa5d58c660 0x55fa5d593ee0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:42.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.278+0000 7f022f400700 0 client.25690 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:57:42.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.278+0000 7f025a792b80 1 -- 172.21.15.151:0/18275911 >> 172.21.15.151:0/18275911 conn(0x55fa5d4bb050 msgr2=0x55fa5d587e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:42.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.279+0000 7f025a792b80 1 -- 172.21.15.151:0/18275911 shutdown_connections 2024-04-26T01:57:42.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.279+0000 7f025a792b80 1 -- 172.21.15.151:0/18275911 wait complete. 2024-04-26T01:57:42.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.279+0000 7f025a792b80 20 client.25690 trim_cache size 0 max 16384 2024-04-26T01:57:42.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.300+0000 7f025a792b80 1 Processor -- start 2024-04-26T01:57:42.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.300+0000 7f025a792b80 1 -- start start 2024-04-26T01:57:42.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.301+0000 7f025a792b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fa5d58bd40 0x55fa5d595d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:42.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.301+0000 7f025a792b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa5d58c660 0x55fa5d5a2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:42.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.301+0000 7f025a792b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa5d5974d0 0x55fa5d58edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:42.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.301+0000 7f025a792b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f02481b5f40 con 0x55fa5d5974d0 2024-04-26T01:57:42.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.301+0000 7f025a792b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f02481b37d0 con 0x55fa5d58c660 2024-04-26T01:57:42.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.301+0000 7f025a792b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f02481b1680 con 0x55fa5d58bd40 2024-04-26T01:57:42.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.301+0000 7f024de00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa5d5974d0 0x55fa5d58edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:42.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.301+0000 7f024de00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa5d5974d0 0x55fa5d58edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:45106/0 (socket says 172.21.15.151:45106) 2024-04-26T01:57:42.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.301+0000 7f024de00700 1 -- 172.21.15.151:0/1759035756 learned_addr learned my addr 172.21.15.151:0/1759035756 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:57:42.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.301+0000 7f024d400700 1 --2- 172.21.15.151:0/1759035756 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fa5d58bd40 0x55fa5d595d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:42.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.301+0000 7f024ca00700 1 --2- 172.21.15.151:0/1759035756 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa5d58c660 0x55fa5d5a2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:42.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.301+0000 7f024d400700 1 -- 172.21.15.151:0/1759035756 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa5d58c660 msgr2=0x55fa5d5a2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:42.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.301+0000 7f024d400700 1 --2- 172.21.15.151:0/1759035756 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa5d58c660 0x55fa5d5a2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:42.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.301+0000 7f024d400700 1 -- 172.21.15.151:0/1759035756 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa5d5974d0 msgr2=0x55fa5d58edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:42.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.301+0000 7f024d400700 1 --2- 172.21.15.151:0/1759035756 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa5d5974d0 0x55fa5d58edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:42.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.301+0000 7f024d400700 1 -- 172.21.15.151:0/1759035756 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa5d58dbb0 con 0x55fa5d58bd40 2024-04-26T01:57:42.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.302+0000 7f024d400700 1 --2- 172.21.15.151:0/1759035756 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fa5d58bd40 0x55fa5d595d80 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f023c0046c0 tx=0x7f023c005e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1c0af5dfb739549 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:42.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.302+0000 7f023a000700 1 -- 172.21.15.151:0/1759035756 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f023c01e040 con 0x55fa5d58bd40 2024-04-26T01:57:42.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.302+0000 7f023a000700 1 -- 172.21.15.151:0/1759035756 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f023c02a070 con 0x55fa5d58bd40 2024-04-26T01:57:42.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.302+0000 7f023a000700 1 -- 172.21.15.151:0/1759035756 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f023c0046f0 con 0x55fa5d58bd40 2024-04-26T01:57:42.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.303+0000 7f025a792b80 1 -- 172.21.15.151:0/1759035756 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fa5d58bd40 msgr2=0x55fa5d595d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:42.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.303+0000 7f025a792b80 1 --2- 172.21.15.151:0/1759035756 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fa5d58bd40 0x55fa5d595d80 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f023c0046c0 tx=0x7f023c005e40 comp rx=0 tx=0).stop 2024-04-26T01:57:42.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.303+0000 7f025a792b80 1 -- 172.21.15.151:0/1759035756 shutdown_connections 2024-04-26T01:57:42.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.303+0000 7f025a792b80 1 --2- 172.21.15.151:0/1759035756 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa5d5974d0 0x55fa5d58edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:42.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.303+0000 7f025a792b80 1 --2- 172.21.15.151:0/1759035756 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fa5d58bd40 0x55fa5d595d80 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:42.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.303+0000 7f025a792b80 1 --2- 172.21.15.151:0/1759035756 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa5d58c660 0x55fa5d5a2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:42.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.303+0000 7f025a792b80 1 -- 172.21.15.151:0/1759035756 >> 172.21.15.151:0/1759035756 conn(0x55fa5d4bb050 msgr2=0x55fa5d587c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:42.304 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.303+0000 7f025a792b80 1 -- 172.21.15.151:0/1759035756 shutdown_connections 2024-04-26T01:57:42.304 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.303+0000 7f025a792b80 1 -- 172.21.15.151:0/1759035756 wait complete. 2024-04-26T01:57:42.304 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.303+0000 7f025a792b80 1 -- start start 2024-04-26T01:57:42.304 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.304+0000 7f025a792b80 1 Processor -- start 2024-04-26T01:57:42.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.304+0000 7f025a792b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fa5d58bd40 0x55fa5d59c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:42.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.304+0000 7f025a792b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa5d58c660 0x55fa5d59cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:42.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.304+0000 7f025a792b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa5d5974d0 0x55fa5d607340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:42.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.304+0000 7f025a792b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f02481b1cd0 con 0x55fa5d58c660 2024-04-26T01:57:42.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.304+0000 7f025a792b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f02481b3100 con 0x55fa5d5974d0 2024-04-26T01:57:42.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.304+0000 7f025a792b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f02481b2360 con 0x55fa5d58bd40 2024-04-26T01:57:42.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.304+0000 7f025a792b80 10 client.0 _ll_register_callbacks cb 0x55fa5d477820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:57:42.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.304+0000 7f024ca00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa5d58c660 0x55fa5d59cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:42.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.305+0000 7f024d400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fa5d58bd40 0x55fa5d59c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:42.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.305+0000 7f024d400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fa5d58bd40 0x55fa5d59c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:34128/0 (socket says 172.21.15.151:34128) 2024-04-26T01:57:42.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.305+0000 7f024d400700 1 -- 172.21.15.151:0/78905302 learned_addr learned my addr 172.21.15.151:0/78905302 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:57:42.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.305+0000 7f024de00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa5d5974d0 0x55fa5d607340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:42.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.305+0000 7f024d400700 1 -- 172.21.15.151:0/78905302 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa5d5974d0 msgr2=0x55fa5d607340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:42.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.305+0000 7f024d400700 1 --2- 172.21.15.151:0/78905302 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa5d5974d0 0x55fa5d607340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:42.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.305+0000 7f024d400700 1 -- 172.21.15.151:0/78905302 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa5d58c660 msgr2=0x55fa5d59cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:42.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.305+0000 7f024d400700 1 --2- 172.21.15.151:0/78905302 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa5d58c660 0x55fa5d59cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:42.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.305+0000 7f024d400700 1 -- 172.21.15.151:0/78905302 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa5d58cce0 con 0x55fa5d58bd40 2024-04-26T01:57:42.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.305+0000 7f024d400700 1 --2- 172.21.15.151:0/78905302 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fa5d58bd40 0x55fa5d59c6b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f023c02e040 tx=0x7f023c0052d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2727d95735bd039c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:42.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.305+0000 7f022f400700 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-26T01:57:42.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.305+0000 7f022f400700 1 -- 172.21.15.151:0/78905302 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f023c005480 con 0x55fa5d58bd40 2024-04-26T01:57:42.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.306+0000 7f022f400700 1 -- 172.21.15.151:0/78905302 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f023c02b2c0 con 0x55fa5d58bd40 2024-04-26T01:57:42.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.306+0000 7f025a792b80 1 -- 172.21.15.151:0/78905302 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f023c02a070 con 0x55fa5d58bd40 2024-04-26T01:57:42.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.306+0000 7f022f400700 1 -- 172.21.15.151:0/78905302 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f023c005620 con 0x55fa5d58bd40 2024-04-26T01:57:42.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.306+0000 7f025a792b80 10 client.25696 fetch_fsmap learned FSMap version 29 2024-04-26T01:57:42.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.306+0000 7f025a792b80 1 -- 172.21.15.151:0/78905302 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fa5d60eb90 con 0x55fa5d58bd40 2024-04-26T01:57:42.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.306+0000 7f022f400700 1 -- 172.21.15.151:0/78905302 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f023c02a070 con 0x55fa5d58bd40 2024-04-26T01:57:42.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.306+0000 7f022f400700 1 -- 172.21.15.151:0/78905302 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8342+0+0 (secure 0 0 0) 0x7f023c03d020 con 0x55fa5d58bd40 2024-04-26T01:57:42.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.306+0000 7f025a792b80 10 client.25696 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T01:57:42.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.306+0000 7f025a792b80 10 client.25696 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-26T01:57:42.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.306+0000 7f025a792b80 20 client.25696 populate_metadata read hostname 'smithi151' 2024-04-26T01:57:42.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.306+0000 7f025a792b80 1 --2- 172.21.15.151:0/78905302 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55fa5d60efd0 0x55fa5d6113f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:42.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.306+0000 7f025a792b80 4 client.25696 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "4b386810", "release": true, "await": true}] 2024-04-26T01:57:42.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.306+0000 7f025a792b80 1 -- 172.21.15.151:0/78905302 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "quiesce db", "set_id": "4b386810", "release": true, "await": true}) -- 0x55fa5d4e6820 con 0x55fa5d60efd0 2024-04-26T01:57:42.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.307+0000 7f024ca00700 1 --2- 172.21.15.151:0/78905302 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55fa5d60efd0 0x55fa5d6113f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:42.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.307+0000 7f024ca00700 1 --2- 172.21.15.151:0/78905302 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55fa5d60efd0 0x55fa5d6113f0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb3ce265477735af server_cookie=e611a718899f94b6 in_seq=0 out_seq=0 2024-04-26T01:57:42.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.307+0000 7f022f400700 10 client.25696 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:57:42.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.566+0000 7f022f400700 1 -- 172.21.15.151:0/78905302 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55fa5d4e6820 con 0x55fa5d60efd0 2024-04-26T01:57:42.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.566+0000 7f022f400700 10 client.25696 handle_command_reply: tid=0 2024-04-26T01:57:42.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.566+0000 7f025a792b80 1 client.25696 shutdown 2024-04-26T01:57:42.566 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:57:42 smithi151 ceph-mon[28057]: pgmap v125: 97 pgs: 97 active+clean; 36 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.5 KiB/s wr, 5 op/s 2024-04-26T01:57:42.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.567+0000 7f025a792b80 1 -- 172.21.15.151:0/78905302 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fa5d58bd40 msgr2=0x55fa5d59c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:42.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.567+0000 7f025a792b80 1 --2- 172.21.15.151:0/78905302 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fa5d58bd40 0x55fa5d59c6b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f023c02e040 tx=0x7f023c0052d0 comp rx=0 tx=0).stop 2024-04-26T01:57:42.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.567+0000 7f025a792b80 1 -- 172.21.15.151:0/78905302 shutdown_connections 2024-04-26T01:57:42.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.567+0000 7f025a792b80 1 --2- 172.21.15.151:0/78905302 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55fa5d60efd0 0x55fa5d6113f0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:42.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.567+0000 7f025a792b80 1 --2- 172.21.15.151:0/78905302 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa5d5974d0 0x55fa5d607340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:42.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.567+0000 7f025a792b80 1 --2- 172.21.15.151:0/78905302 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fa5d58bd40 0x55fa5d59c6b0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:42.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.567+0000 7f025a792b80 1 --2- 172.21.15.151:0/78905302 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa5d58c660 0x55fa5d59cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:42.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.567+0000 7f025a792b80 1 -- 172.21.15.151:0/78905302 >> 172.21.15.151:0/78905302 conn(0x55fa5d4bb050 msgr2=0x55fa5d5a1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:42.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.567+0000 7f022f400700 0 client.25696 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:57:42.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.568+0000 7f025a792b80 1 -- 172.21.15.151:0/78905302 shutdown_connections 2024-04-26T01:57:42.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.568+0000 7f025a792b80 1 -- 172.21.15.151:0/78905302 wait complete. 2024-04-26T01:57:42.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.568+0000 7f025a792b80 20 client.25696 trim_cache size 0 max 16384 2024-04-26T01:57:42.568 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-26T01:57:42.568 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 29, 2024-04-26T01:57:42.568 INFO:teuthology.orchestra.run.smithi151.stdout: "leader": 24509, 2024-04-26T01:57:42.568 INFO:teuthology.orchestra.run.smithi151.stdout: "set_version": 12, 2024-04-26T01:57:42.568 INFO:teuthology.orchestra.run.smithi151.stdout: "sets": { 2024-04-26T01:57:42.568 INFO:teuthology.orchestra.run.smithi151.stdout: "4b386810": { 2024-04-26T01:57:42.569 INFO:teuthology.orchestra.run.smithi151.stdout: "version": 12, 2024-04-26T01:57:42.569 INFO:teuthology.orchestra.run.smithi151.stdout: "age_ref": 0.0, 2024-04-26T01:57:42.569 INFO:teuthology.orchestra.run.smithi151.stdout: "state": { 2024-04-26T01:57:42.569 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "RELEASED", 2024-04-26T01:57:42.569 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 0.0 2024-04-26T01:57:42.569 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T01:57:42.569 INFO:teuthology.orchestra.run.smithi151.stdout: "timeout": 60.0, 2024-04-26T01:57:42.569 INFO:teuthology.orchestra.run.smithi151.stdout: "expiration": 90.0, 2024-04-26T01:57:42.569 INFO:teuthology.orchestra.run.smithi151.stdout: "members": { 2024-04-26T01:57:42.569 INFO:teuthology.orchestra.run.smithi151.stdout: "file:/": { 2024-04-26T01:57:42.569 INFO:teuthology.orchestra.run.smithi151.stdout: "excluded": false, 2024-04-26T01:57:42.569 INFO:teuthology.orchestra.run.smithi151.stdout: "state": { 2024-04-26T01:57:42.570 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "RELEASED", 2024-04-26T01:57:42.570 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 0.0 2024-04-26T01:57:42.570 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T01:57:42.570 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T01:57:42.570 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T01:57:42.570 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T01:57:42.570 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T01:57:42.570 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-26T01:57:42.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.571+0000 7f024fe00700 1 -- 172.21.15.151:0/4239506900 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f02300a3fd0 msgr2=0x7f02300a6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:42.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.571+0000 7f024fe00700 1 --2- 172.21.15.151:0/4239506900 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f02300a3fd0 0x7f02300a6490 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f024000f870 tx=0x7f024000fc30 comp rx=0 tx=0).stop 2024-04-26T01:57:42.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.571+0000 7f024fe00700 1 -- 172.21.15.151:0/4239506900 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0248109c80 msgr2=0x7f02481a5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:42.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.571+0000 7f024fe00700 1 --2- 172.21.15.151:0/4239506900 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0248109c80 0x7f02481a5a90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0234000c00 tx=0x7f02340057c0 comp rx=0 tx=0).stop 2024-04-26T01:57:42.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.571+0000 7f024fe00700 1 -- 172.21.15.151:0/4239506900 shutdown_connections 2024-04-26T01:57:42.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.571+0000 7f024fe00700 1 --2- 172.21.15.151:0/4239506900 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f02300a3fd0 0x7f02300a6490 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:42.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.571+0000 7f024fe00700 1 --2- 172.21.15.151:0/4239506900 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f024810e3b0 0x7f024819f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:42.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.571+0000 7f024fe00700 1 --2- 172.21.15.151:0/4239506900 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02481092d0 0x7f02481a5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:42.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.571+0000 7f024fe00700 1 --2- 172.21.15.151:0/4239506900 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0248109c80 0x7f02481a5a90 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:42.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.571+0000 7f024fe00700 1 -- 172.21.15.151:0/4239506900 >> 172.21.15.151:0/4239506900 conn(0x7f0248076210 msgr2=0x7f0248076870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:42.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.572+0000 7f024fe00700 1 -- 172.21.15.151:0/4239506900 shutdown_connections 2024-04-26T01:57:42.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:57:42.572+0000 7f024fe00700 1 -- 172.21.15.151:0/4239506900 wait complete. 2024-04-26T01:57:42.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:57:42 smithi165 ceph-mon[32648]: pgmap v125: 97 pgs: 97 active+clean; 36 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.5 KiB/s wr, 5 op/s 2024-04-26T01:57:42.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:57:42 smithi172 ceph-mon[32785]: pgmap v125: 97 pgs: 97 active+clean; 36 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.5 KiB/s wr, 5 op/s 2024-04-26T01:57:43.569 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '4b386810', seconds elapsed: 48.0 2024-04-26T01:57:44.340 INFO:tasks.workunit.client.0.smithi151.stdout:copying to largest ever: 0xfe492 2024-04-26T01:57:44.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:57:44 smithi165 ceph-mon[32648]: pgmap v126: 97 pgs: 97 active+clean; 36 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-04-26T01:57:44.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:57:44 smithi151 ceph-mon[28057]: pgmap v126: 97 pgs: 97 active+clean; 36 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-04-26T01:57:44.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:57:44 smithi172 ceph-mon[32785]: pgmap v126: 97 pgs: 97 active+clean; 36 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-04-26T01:57:46.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:57:46 smithi165 ceph-mon[32648]: pgmap v127: 97 pgs: 97 active+clean; 37 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 83 KiB/s wr, 22 op/s 2024-04-26T01:57:46.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:57:46 smithi151 ceph-mon[28057]: pgmap v127: 97 pgs: 97 active+clean; 37 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 83 KiB/s wr, 22 op/s 2024-04-26T01:57:46.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:57:46 smithi172 ceph-mon[32785]: pgmap v127: 97 pgs: 97 active+clean; 37 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 83 KiB/s wr, 22 op/s 2024-04-26T01:57:47.615 INFO:tasks.workunit.client.0.smithi151.stdout:copying to largest ever: 0xff038 2024-04-26T01:57:48.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:57:48 smithi165 ceph-mon[32648]: pgmap v128: 97 pgs: 97 active+clean; 37 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 83 KiB/s wr, 20 op/s 2024-04-26T01:57:48.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:57:48 smithi151 ceph-mon[28057]: pgmap v128: 97 pgs: 97 active+clean; 37 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 83 KiB/s wr, 20 op/s 2024-04-26T01:57:48.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:57:48 smithi172 ceph-mon[32785]: pgmap v128: 97 pgs: 97 active+clean; 37 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 83 KiB/s wr, 20 op/s 2024-04-26T01:57:49.073 INFO:tasks.workunit.client.0.smithi151.stdout:truncating to largest ever: 0xffcac 2024-04-26T01:57:49.282 INFO:tasks.workunit.client.0.smithi151.stdout:10000 copy from 0x2d237 to 0x30552, (0x331b bytes) at 0x82aed 2024-04-26T01:57:50.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:57:50 smithi165 ceph-mon[32648]: pgmap v129: 97 pgs: 97 active+clean; 39 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 224 op/s 2024-04-26T01:57:50.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:57:50 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:57:50.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:57:50 smithi151 ceph-mon[28057]: pgmap v129: 97 pgs: 97 active+clean; 39 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 224 op/s 2024-04-26T01:57:50.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:57:50 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:57:50.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:57:50 smithi172 ceph-mon[32785]: pgmap v129: 97 pgs: 97 active+clean; 39 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 224 op/s 2024-04-26T01:57:50.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:57:50 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:57:51.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:57:51 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:57:51.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:57:51 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:57:51.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:57:51 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:57:51.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:57:51 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:57:51.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:57:51 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:57:51.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:57:51 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:57:51.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:57:51 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:57:51.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:57:51 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:57:51.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:57:51 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:57:52.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:57:52 smithi151 ceph-mon[28057]: pgmap v130: 97 pgs: 97 active+clean; 39 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 221 op/s 2024-04-26T01:57:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.694+0000 7fa87fe00640 1 -- 172.21.15.151:0/1965897822 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa880108e10 msgr2=0x7fa880115250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:52.693 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.694+0000 7fa87fe00640 1 --2- 172.21.15.151:0/1965897822 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa880108e10 0x7fa880115250 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa878009cd0 tx=0x7fa87802a8e0 comp rx=0 tx=0).stop 2024-04-26T01:57:52.693 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.695+0000 7fa87fe00640 1 -- 172.21.15.151:0/1965897822 shutdown_connections 2024-04-26T01:57:52.693 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.695+0000 7fa87fe00640 1 --2- 172.21.15.151:0/1965897822 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa880108e10 0x7fa880115250 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:52.693 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.695+0000 7fa87fe00640 1 --2- 172.21.15.151:0/1965897822 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa880108490 0x7fa8801088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:52.693 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.695+0000 7fa87fe00640 1 --2- 172.21.15.151:0/1965897822 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa88010e420 0x7fa88010e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:52.693 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.695+0000 7fa87fe00640 1 -- 172.21.15.151:0/1965897822 >> 172.21.15.151:0/1965897822 conn(0x7fa880077d30 msgr2=0x7fa880106d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:52.693 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.695+0000 7fa87fe00640 1 -- 172.21.15.151:0/1965897822 shutdown_connections 2024-04-26T01:57:52.693 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.695+0000 7fa87fe00640 1 -- 172.21.15.151:0/1965897822 wait complete. 2024-04-26T01:57:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.696+0000 7fa87fe00640 1 Processor -- start 2024-04-26T01:57:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.696+0000 7fa87fe00640 1 -- start start 2024-04-26T01:57:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.696+0000 7fa87fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa880108490 0x7fa8801a4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.696+0000 7fa87fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa880108e10 0x7fa8801a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.696+0000 7fa87fe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa88010e420 0x7fa88019e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.696+0000 7fa87fe00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa880118630 con 0x7fa880108e10 2024-04-26T01:57:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.696+0000 7fa87fe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa8801184b0 con 0x7fa880108490 2024-04-26T01:57:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.696+0000 7fa87fe00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa8801187b0 con 0x7fa88010e420 2024-04-26T01:57:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.697+0000 7fa87f400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa88010e420 0x7fa88019e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.697+0000 7fa87ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa880108490 0x7fa8801a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.697+0000 7fa87f400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa88010e420 0x7fa88019e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:36470/0 (socket says 172.21.15.151:36470) 2024-04-26T01:57:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.697+0000 7fa87f400640 1 -- 172.21.15.151:0/3567382975 learned_addr learned my addr 172.21.15.151:0/3567382975 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:57:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.697+0000 7fa87e000640 1 --2- 172.21.15.151:0/3567382975 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa880108e10 0x7fa8801a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.697+0000 7fa87ea00640 1 -- 172.21.15.151:0/3567382975 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa88010e420 msgr2=0x7fa88019e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.697+0000 7fa87ea00640 1 --2- 172.21.15.151:0/3567382975 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa88010e420 0x7fa88019e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.697+0000 7fa87ea00640 1 -- 172.21.15.151:0/3567382975 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa880108e10 msgr2=0x7fa8801a4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.697+0000 7fa87ea00640 1 --2- 172.21.15.151:0/3567382975 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa880108e10 0x7fa8801a4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.697+0000 7fa87ea00640 1 -- 172.21.15.151:0/3567382975 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa88019f0b0 con 0x7fa880108490 2024-04-26T01:57:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.697+0000 7fa87f400640 1 --2- 172.21.15.151:0/3567382975 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa88010e420 0x7fa88019e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:57:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.698+0000 7fa87ea00640 1 --2- 172.21.15.151:0/3567382975 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa880108490 0x7fa8801a4780 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa87000e9f0 tx=0x7fa87000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.698+0000 7fa884a00640 1 -- 172.21.15.151:0/3567382975 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87000ccc0 con 0x7fa880108490 2024-04-26T01:57:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.698+0000 7fa87fe00640 1 -- 172.21.15.151:0/3567382975 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa88019f3a0 con 0x7fa880108490 2024-04-26T01:57:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.698+0000 7fa884a00640 1 -- 172.21.15.151:0/3567382975 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8700108e0 con 0x7fa880108490 2024-04-26T01:57:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.698+0000 7fa884a00640 1 -- 172.21.15.151:0/3567382975 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa870018880 con 0x7fa880108490 2024-04-26T01:57:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.698+0000 7fa87fe00640 1 -- 172.21.15.151:0/3567382975 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8801ab640 con 0x7fa880108490 2024-04-26T01:57:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.702+0000 7fa888c29740 1 Processor -- start 2024-04-26T01:57:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.703+0000 7fa888c29740 1 -- start start 2024-04-26T01:57:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.703+0000 7fa884a00640 1 -- 172.21.15.151:0/3567382975 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7fa870010430 con 0x7fa880108490 2024-04-26T01:57:52.702 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.703+0000 7fa888c29740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561d2e34b9b0 0x561d2e34bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:52.702 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.703+0000 7fa888c29740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d2e34c2d0 0x561d2e356650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:52.702 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.703+0000 7fa888c29740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d2e356b90 0x561d2e358f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:52.702 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.703+0000 7fa888c29740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x561d2e2b1900 con 0x561d2e34c2d0 2024-04-26T01:57:52.702 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.703+0000 7fa888c29740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561d2e257ef0 con 0x561d2e34b9b0 2024-04-26T01:57:52.702 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.703+0000 7fa888c29740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561d2e2ad8c0 con 0x561d2e356b90 2024-04-26T01:57:52.702 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.703+0000 7fa87f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561d2e34b9b0 0x561d2e34bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:52.702 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.703+0000 7fa87e000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d2e356b90 0x561d2e358f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:52.702 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.703+0000 7fa87f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561d2e34b9b0 0x561d2e34bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:46996/0 (socket says 172.21.15.151:46996) 2024-04-26T01:57:52.702 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.703+0000 7fa87e000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d2e356b90 0x561d2e358f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:36480/0 (socket says 172.21.15.151:36480) 2024-04-26T01:57:52.702 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.703+0000 7fa87f400640 1 -- 172.21.15.151:0/2125758303 learned_addr learned my addr 172.21.15.151:0/2125758303 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:57:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.704+0000 7fa87ea00640 1 --2- 172.21.15.151:0/2125758303 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d2e34c2d0 0x561d2e356650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.704+0000 7fa87f400640 1 -- 172.21.15.151:0/2125758303 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d2e356b90 msgr2=0x561d2e358f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.704+0000 7fa87f400640 1 --2- 172.21.15.151:0/2125758303 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d2e356b90 0x561d2e358f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.704+0000 7fa87f400640 1 -- 172.21.15.151:0/2125758303 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d2e34c2d0 msgr2=0x561d2e356650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.704+0000 7fa87f400640 1 --2- 172.21.15.151:0/2125758303 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d2e34c2d0 0x561d2e356650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.704+0000 7fa87f400640 1 -- 172.21.15.151:0/2125758303 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d2dfc4210 con 0x561d2e34b9b0 2024-04-26T01:57:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.704+0000 7fa87e000640 1 --2- 172.21.15.151:0/2125758303 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d2e356b90 0x561d2e358f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:57:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.704+0000 7fa87f400640 1 --2- 172.21.15.151:0/2125758303 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561d2e34b9b0 0x561d2e34bd90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa878005fb0 tx=0x7fa8780093e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d2a4dc9b4094a5f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.705+0000 7fa884a00640 1 --2- 172.21.15.151:0/3567382975 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fa8640a3d10 0x7fa8640a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.705+0000 7fa86be00640 1 -- 172.21.15.151:0/2125758303 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa878048070 con 0x561d2e34b9b0 2024-04-26T01:57:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.705+0000 7fa86be00640 1 -- 172.21.15.151:0/2125758303 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa878036070 con 0x561d2e34b9b0 2024-04-26T01:57:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.705+0000 7fa86be00640 1 -- 172.21.15.151:0/2125758303 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa878004830 con 0x561d2e34b9b0 2024-04-26T01:57:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.705+0000 7fa884a00640 1 -- 172.21.15.151:0/3567382975 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fa8700d0a10 con 0x7fa880108490 2024-04-26T01:57:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.705+0000 7fa87ea00640 1 --2- 172.21.15.151:0/3567382975 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fa8640a3d10 0x7fa8640a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.706+0000 7fa888c29740 1 -- 172.21.15.151:0/2125758303 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561d2e34b9b0 msgr2=0x561d2e34bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.706+0000 7fa888c29740 1 --2- 172.21.15.151:0/2125758303 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561d2e34b9b0 0x561d2e34bd90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa878005fb0 tx=0x7fa8780093e0 comp rx=0 tx=0).stop 2024-04-26T01:57:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.706+0000 7fa87ea00640 1 --2- 172.21.15.151:0/3567382975 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fa8640a3d10 0x7fa8640a61d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa8700ce280 tx=0x7fa870009700 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.706+0000 7fa888c29740 1 -- 172.21.15.151:0/2125758303 shutdown_connections 2024-04-26T01:57:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.706+0000 7fa888c29740 1 --2- 172.21.15.151:0/2125758303 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d2e356b90 0x561d2e358f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.706+0000 7fa888c29740 1 --2- 172.21.15.151:0/2125758303 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d2e34c2d0 0x561d2e356650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.706+0000 7fa888c29740 1 --2- 172.21.15.151:0/2125758303 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561d2e34b9b0 0x561d2e34bd90 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.706+0000 7fa888c29740 1 -- 172.21.15.151:0/2125758303 >> 172.21.15.151:0/2125758303 conn(0x561d2e347ec0 msgr2=0x561d2e3482a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.706+0000 7fa888c29740 1 -- 172.21.15.151:0/2125758303 shutdown_connections 2024-04-26T01:57:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.706+0000 7fa888c29740 1 -- 172.21.15.151:0/2125758303 wait complete. 2024-04-26T01:57:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.706+0000 7fa888c29740 1 -- start start 2024-04-26T01:57:52.708 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.707+0000 7fa888c29740 1 Processor -- start 2024-04-26T01:57:52.708 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.707+0000 7fa888c29740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d2e34b9b0 0x561d2e353bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:52.708 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.707+0000 7fa888c29740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561d2e34c2d0 0x561d2e3540f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:52.708 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.707+0000 7fa888c29740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d2e356b90 0x561d2e354630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:52.708 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.707+0000 7fa888c29740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x561d2e359760 con 0x561d2e34b9b0 2024-04-26T01:57:52.708 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.707+0000 7fa888c29740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561d2e26b310 con 0x561d2e34c2d0 2024-04-26T01:57:52.708 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.707+0000 7fa888c29740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561d2e3598e0 con 0x561d2e356b90 2024-04-26T01:57:52.708 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.707+0000 7fa888c29740 10 client.0 _ll_register_callbacks cb 0x561d2e239cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:57:52.708 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.707+0000 7fa87e000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561d2e34c2d0 0x561d2e3540f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:52.708 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.708+0000 7fa87e000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561d2e34c2d0 0x561d2e3540f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:47010/0 (socket says 172.21.15.151:47010) 2024-04-26T01:57:52.708 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.708+0000 7fa87e000640 1 -- 172.21.15.151:0/3229348107 learned_addr learned my addr 172.21.15.151:0/3229348107 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:57:52.708 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.708+0000 7fa87f400640 1 --2- 172.21.15.151:0/3229348107 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d2e34b9b0 0x561d2e353bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:52.708 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.708+0000 7fa87f400640 1 --2- 172.21.15.151:0/3229348107 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d2e356b90 0x561d2e354630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:52.708 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.708+0000 7fa87e000640 1 -- 172.21.15.151:0/3229348107 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d2e356b90 msgr2=0x561d2e354630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:52.709 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.708+0000 7fa87e000640 1 --2- 172.21.15.151:0/3229348107 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d2e356b90 0x561d2e354630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:52.709 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.708+0000 7fa87e000640 1 -- 172.21.15.151:0/3229348107 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d2e34b9b0 msgr2=0x561d2e353bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:52.709 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.708+0000 7fa87e000640 1 --2- 172.21.15.151:0/3229348107 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d2e34b9b0 0x561d2e353bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:52.709 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.708+0000 7fa87e000640 1 -- 172.21.15.151:0/3229348107 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d2e090620 con 0x561d2e34c2d0 2024-04-26T01:57:52.709 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.708+0000 7fa87e000640 1 --2- 172.21.15.151:0/3229348107 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561d2e34c2d0 0x561d2e3540f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa86c012700 tx=0x7fa86c012b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e1f4e0059f886d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:52.709 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.708+0000 7fa868c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T01:57:52.709 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.708+0000 7fa868c00640 1 -- 172.21.15.151:0/3229348107 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86c004030 con 0x561d2e34c2d0 2024-04-26T01:57:52.709 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.709+0000 7fa868c00640 1 -- 172.21.15.151:0/3229348107 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fa86c0041d0 con 0x561d2e34c2d0 2024-04-26T01:57:52.709 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.709+0000 7fa888c29740 1 -- 172.21.15.151:0/3229348107 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa878036070 con 0x561d2e34c2d0 2024-04-26T01:57:52.711 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.709+0000 7fa868c00640 1 -- 172.21.15.151:0/3229348107 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa86c0043d0 con 0x561d2e34c2d0 2024-04-26T01:57:52.711 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.709+0000 7fa888c29740 10 client.25676 fetch_fsmap learned FSMap version 29 2024-04-26T01:57:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.709+0000 7fa888c29740 1 -- 172.21.15.151:0/3229348107 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561d2e365ab0 con 0x561d2e34c2d0 2024-04-26T01:57:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.709+0000 7fa868c00640 1 -- 172.21.15.151:0/3229348107 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa878036070 con 0x561d2e34c2d0 2024-04-26T01:57:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.709+0000 7fa868c00640 1 -- 172.21.15.151:0/3229348107 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 29) ==== 8342+0+0 (secure 0 0 0) 0x7fa86c00f7f0 con 0x561d2e34c2d0 2024-04-26T01:57:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.710+0000 7fa888c29740 10 client.25676 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T01:57:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.710+0000 7fa888c29740 10 client.25676 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:57:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.710+0000 7fa888c29740 20 client.25676 populate_metadata read hostname 'smithi151' 2024-04-26T01:57:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.710+0000 7fa888c29740 1 --2- 172.21.15.151:0/3229348107 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x561d2e366f20 0x561d2e369370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.710+0000 7fa888c29740 4 client.25676 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:57:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.710+0000 7fa888c29740 1 -- 172.21.15.151:0/3229348107 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d2e2c58b0 con 0x561d2e366f20 2024-04-26T01:57:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.710+0000 7fa87f400640 1 --2- 172.21.15.151:0/3229348107 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x561d2e366f20 0x561d2e369370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.711+0000 7fa868c00640 10 client.25676 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:57:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.711+0000 7fa87f400640 1 --2- 172.21.15.151:0/3229348107 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x561d2e366f20 0x561d2e369370 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6302087428eb7b55 server_cookie=58831b04acbb433 in_seq=0 out_seq=0 2024-04-26T01:57:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.711+0000 7fa868c00640 1 -- 172.21.15.151:0/3229348107 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561d2e2c58b0 con 0x561d2e366f20 2024-04-26T01:57:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.711+0000 7fa868c00640 10 client.25676 handle_command_reply: tid=0 2024-04-26T01:57:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.711+0000 7fa888c29740 1 client.25676 shutdown 2024-04-26T01:57:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.712+0000 7fa888c29740 1 -- 172.21.15.151:0/3229348107 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561d2e34c2d0 msgr2=0x561d2e3540f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.712+0000 7fa888c29740 1 --2- 172.21.15.151:0/3229348107 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561d2e34c2d0 0x561d2e3540f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa86c012700 tx=0x7fa86c012b00 comp rx=0 tx=0).stop 2024-04-26T01:57:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.712+0000 7fa888c29740 1 -- 172.21.15.151:0/3229348107 shutdown_connections 2024-04-26T01:57:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.712+0000 7fa888c29740 1 --2- 172.21.15.151:0/3229348107 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x561d2e366f20 0x561d2e369370 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.712+0000 7fa888c29740 1 --2- 172.21.15.151:0/3229348107 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d2e356b90 0x561d2e354630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.712+0000 7fa888c29740 1 --2- 172.21.15.151:0/3229348107 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561d2e34c2d0 0x561d2e3540f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.712+0000 7fa888c29740 1 --2- 172.21.15.151:0/3229348107 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d2e34b9b0 0x561d2e353bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.712+0000 7fa868c00640 0 client.25676 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:57:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.712+0000 7fa888c29740 1 -- 172.21.15.151:0/3229348107 >> 172.21.15.151:0/3229348107 conn(0x561d2e347ec0 msgr2=0x561d2e34dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.712+0000 7fa888c29740 1 -- 172.21.15.151:0/3229348107 shutdown_connections 2024-04-26T01:57:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.712+0000 7fa888c29740 1 -- 172.21.15.151:0/3229348107 wait complete. 2024-04-26T01:57:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.712+0000 7fa888c29740 20 client.25676 trim_cache size 0 max 16384 2024-04-26T01:57:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.731+0000 7fa888c29740 1 Processor -- start 2024-04-26T01:57:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.732+0000 7fa888c29740 1 -- start start 2024-04-26T01:57:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.732+0000 7fa888c29740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561d2e34b9b0 0x561d2e362620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.732+0000 7fa888c29740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d2e34c2d0 0x561d2e34dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.732+0000 7fa888c29740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d2e356b90 0x561d2e36ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.732+0000 7fa888c29740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa8801b66e0 con 0x561d2e356b90 2024-04-26T01:57:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.732+0000 7fa888c29740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa8801b3fb0 con 0x561d2e34b9b0 2024-04-26T01:57:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.732+0000 7fa888c29740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa8801b1e00 con 0x561d2e34c2d0 2024-04-26T01:57:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.732+0000 7fa87e000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d2e34c2d0 0x561d2e34dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.732+0000 7fa87e000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d2e34c2d0 0x561d2e34dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:36498/0 (socket says 172.21.15.151:36498) 2024-04-26T01:57:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.732+0000 7fa87e000640 1 -- 172.21.15.151:0/673456063 learned_addr learned my addr 172.21.15.151:0/673456063 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:57:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.732+0000 7fa87f400640 1 --2- 172.21.15.151:0/673456063 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561d2e34b9b0 0x561d2e362620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.732+0000 7fa87f400640 1 --2- 172.21.15.151:0/673456063 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d2e356b90 0x561d2e36ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.733+0000 7fa87e000640 1 -- 172.21.15.151:0/673456063 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561d2e34b9b0 msgr2=0x561d2e362620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.733+0000 7fa87e000640 1 --2- 172.21.15.151:0/673456063 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561d2e34b9b0 0x561d2e362620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.733+0000 7fa87e000640 1 -- 172.21.15.151:0/673456063 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d2e356b90 msgr2=0x561d2e36ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.733+0000 7fa87e000640 1 --2- 172.21.15.151:0/673456063 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d2e356b90 0x561d2e36ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.733+0000 7fa87e000640 1 -- 172.21.15.151:0/673456063 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d2e342490 con 0x561d2e34c2d0 2024-04-26T01:57:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.733+0000 7fa87e000640 1 --2- 172.21.15.151:0/673456063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d2e34c2d0 0x561d2e34dac0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa86c000c00 tx=0x7fa86c0104d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b23e96f98d5c14a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.733+0000 7fa86b400640 1 -- 172.21.15.151:0/673456063 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86c009e70 con 0x561d2e34c2d0 2024-04-26T01:57:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.733+0000 7fa86b400640 1 -- 172.21.15.151:0/673456063 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa86c005030 con 0x561d2e34c2d0 2024-04-26T01:57:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.733+0000 7fa86b400640 1 -- 172.21.15.151:0/673456063 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86c0047d0 con 0x561d2e34c2d0 2024-04-26T01:57:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.734+0000 7fa888c29740 1 -- 172.21.15.151:0/673456063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d2e34c2d0 msgr2=0x561d2e34dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.734+0000 7fa888c29740 1 --2- 172.21.15.151:0/673456063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d2e34c2d0 0x561d2e34dac0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa86c000c00 tx=0x7fa86c0104d0 comp rx=0 tx=0).stop 2024-04-26T01:57:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.734+0000 7fa888c29740 1 -- 172.21.15.151:0/673456063 shutdown_connections 2024-04-26T01:57:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.734+0000 7fa888c29740 1 --2- 172.21.15.151:0/673456063 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d2e356b90 0x561d2e36ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.734+0000 7fa888c29740 1 --2- 172.21.15.151:0/673456063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d2e34c2d0 0x561d2e34dac0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.734+0000 7fa888c29740 1 --2- 172.21.15.151:0/673456063 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561d2e34b9b0 0x561d2e362620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.734+0000 7fa888c29740 1 -- 172.21.15.151:0/673456063 >> 172.21.15.151:0/673456063 conn(0x561d2e347ec0 msgr2=0x561d2e34d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.734+0000 7fa888c29740 1 -- 172.21.15.151:0/673456063 shutdown_connections 2024-04-26T01:57:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.734+0000 7fa888c29740 1 -- 172.21.15.151:0/673456063 wait complete. 2024-04-26T01:57:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.735+0000 7fa888c29740 1 -- start start 2024-04-26T01:57:52.734 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.735+0000 7fa888c29740 1 Processor -- start 2024-04-26T01:57:52.736 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.735+0000 7fa888c29740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561d2e34b9b0 0x561d2e35cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:52.736 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.736+0000 7fa888c29740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d2e34c2d0 0x561d2e35d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:52.736 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.736+0000 7fa888c29740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d2e356b90 0x561d2e3c64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:52.736 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.736+0000 7fa888c29740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa8801b2470 con 0x561d2e34c2d0 2024-04-26T01:57:52.736 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.736+0000 7fa888c29740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa8801b38e0 con 0x561d2e34b9b0 2024-04-26T01:57:52.736 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.736+0000 7fa888c29740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa8801b2b40 con 0x561d2e356b90 2024-04-26T01:57:52.736 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.736+0000 7fa888c29740 10 client.0 _ll_register_callbacks cb 0x7fa86c009dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:57:52.736 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.736+0000 7fa87f400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d2e356b90 0x561d2e3c64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:52.736 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.736+0000 7fa87f400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d2e356b90 0x561d2e3c64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:36514/0 (socket says 172.21.15.151:36514) 2024-04-26T01:57:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.736+0000 7fa87f400640 1 -- 172.21.15.151:0/3627716189 learned_addr learned my addr 172.21.15.151:0/3627716189 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:57:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.736+0000 7fa87f400640 1 --2- 172.21.15.151:0/3627716189 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561d2e34b9b0 0x561d2e35cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.736+0000 7fa87e000640 1 --2- 172.21.15.151:0/3627716189 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d2e34c2d0 0x561d2e35d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.736+0000 7fa87f400640 1 -- 172.21.15.151:0/3627716189 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561d2e34b9b0 msgr2=0x561d2e35cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.736+0000 7fa87f400640 1 --2- 172.21.15.151:0/3627716189 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561d2e34b9b0 0x561d2e35cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.736+0000 7fa87f400640 1 -- 172.21.15.151:0/3627716189 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d2e34c2d0 msgr2=0x561d2e35d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.736+0000 7fa87f400640 1 --2- 172.21.15.151:0/3627716189 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d2e34c2d0 0x561d2e35d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.736+0000 7fa87f400640 1 -- 172.21.15.151:0/3627716189 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d2e3c6ac0 con 0x561d2e356b90 2024-04-26T01:57:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.737+0000 7fa87f400640 1 --2- 172.21.15.151:0/3627716189 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d2e356b90 0x561d2e3c64d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa878004360 tx=0x7fa878005d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3d2f0974c1a8c79 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.737+0000 7fa868c00640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-26T01:57:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.737+0000 7fa868c00640 1 -- 172.21.15.151:0/3627716189 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa878048040 con 0x561d2e356b90 2024-04-26T01:57:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.737+0000 7fa888c29740 1 -- 172.21.15.151:0/3627716189 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa86c005030 con 0x561d2e356b90 2024-04-26T01:57:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.737+0000 7fa868c00640 1 -- 172.21.15.151:0/3627716189 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fa878004780 con 0x561d2e356b90 2024-04-26T01:57:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.737+0000 7fa868c00640 1 -- 172.21.15.151:0/3627716189 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa878069050 con 0x561d2e356b90 2024-04-26T01:57:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.737+0000 7fa888c29740 10 client.25714 fetch_fsmap learned FSMap version 29 2024-04-26T01:57:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.737+0000 7fa888c29740 1 -- 172.21.15.151:0/3627716189 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561d2e3cde50 con 0x561d2e356b90 2024-04-26T01:57:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.737+0000 7fa868c00640 1 -- 172.21.15.151:0/3627716189 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa86c005030 con 0x561d2e356b90 2024-04-26T01:57:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.738+0000 7fa868c00640 1 -- 172.21.15.151:0/3627716189 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8342+0+0 (secure 0 0 0) 0x7fa87804e040 con 0x561d2e356b90 2024-04-26T01:57:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.738+0000 7fa888c29740 10 client.25714 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T01:57:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.738+0000 7fa888c29740 10 client.25714 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:57:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.738+0000 7fa888c29740 20 client.25714 populate_metadata read hostname 'smithi151' 2024-04-26T01:57:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.738+0000 7fa888c29740 1 --2- 172.21.15.151:0/3627716189 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x561d2e3ce140 0x561d2e3d0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.738+0000 7fa888c29740 4 client.25714 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:57:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.738+0000 7fa888c29740 1 -- 172.21.15.151:0/3627716189 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561d2e347d10 con 0x561d2e3ce140 2024-04-26T01:57:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.738+0000 7fa87e000640 1 --2- 172.21.15.151:0/3627716189 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x561d2e3ce140 0x561d2e3d0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:52.739 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.739+0000 7fa87e000640 1 --2- 172.21.15.151:0/3627716189 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x561d2e3ce140 0x561d2e3d0530 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b87211a630041804 server_cookie=ab2e4a0b2b47c4f5 in_seq=0 out_seq=0 2024-04-26T01:57:52.739 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.739+0000 7fa868c00640 10 client.25714 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:57:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.749+0000 7fa868c00640 1 -- 172.21.15.151:0/3627716189 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561d2e347d10 con 0x561d2e3ce140 2024-04-26T01:57:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.749+0000 7fa868c00640 10 client.25714 handle_command_reply: tid=0 2024-04-26T01:57:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.749+0000 7fa888c29740 1 client.25714 shutdown 2024-04-26T01:57:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.750+0000 7fa888c29740 1 -- 172.21.15.151:0/3627716189 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d2e356b90 msgr2=0x561d2e3c64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.750+0000 7fa888c29740 1 --2- 172.21.15.151:0/3627716189 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d2e356b90 0x561d2e3c64d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa878004360 tx=0x7fa878005d80 comp rx=0 tx=0).stop 2024-04-26T01:57:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.750+0000 7fa888c29740 1 -- 172.21.15.151:0/3627716189 shutdown_connections 2024-04-26T01:57:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.750+0000 7fa888c29740 1 --2- 172.21.15.151:0/3627716189 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x561d2e3ce140 0x561d2e3d0530 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.750+0000 7fa888c29740 1 --2- 172.21.15.151:0/3627716189 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d2e356b90 0x561d2e3c64d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.750+0000 7fa888c29740 1 --2- 172.21.15.151:0/3627716189 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d2e34c2d0 0x561d2e35d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.750+0000 7fa868c00640 0 client.25714 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:57:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.750+0000 7fa888c29740 1 --2- 172.21.15.151:0/3627716189 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561d2e34b9b0 0x561d2e35cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:52.749 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.750+0000 7fa888c29740 1 -- 172.21.15.151:0/3627716189 >> 172.21.15.151:0/3627716189 conn(0x561d2e347ec0 msgr2=0x561d2e353b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:52.749 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.750+0000 7fa888c29740 1 -- 172.21.15.151:0/3627716189 shutdown_connections 2024-04-26T01:57:52.749 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.750+0000 7fa888c29740 1 -- 172.21.15.151:0/3627716189 wait complete. 2024-04-26T01:57:52.749 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.750+0000 7fa888c29740 20 client.25714 trim_cache size 0 max 16384 2024-04-26T01:57:52.749 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:{ 2024-04-26T01:57:52.749 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "message": "", 2024-04-26T01:57:52.749 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "return_code": 0 2024-04-26T01:57:52.749 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:} 2024-04-26T01:57:52.750 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.751+0000 7fa87fe00640 1 -- 172.21.15.151:0/3567382975 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fa8640a3d10 msgr2=0x7fa8640a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:52.750 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.751+0000 7fa87fe00640 1 --2- 172.21.15.151:0/3567382975 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fa8640a3d10 0x7fa8640a61d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa8700ce280 tx=0x7fa870009700 comp rx=0 tx=0).stop 2024-04-26T01:57:52.750 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.751+0000 7fa87fe00640 1 -- 172.21.15.151:0/3567382975 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa880108490 msgr2=0x7fa8801a4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:52.750 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.751+0000 7fa87fe00640 1 --2- 172.21.15.151:0/3567382975 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa880108490 0x7fa8801a4780 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa87000e9f0 tx=0x7fa87000eeb0 comp rx=0 tx=0).stop 2024-04-26T01:57:52.750 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.751+0000 7fa87fe00640 1 -- 172.21.15.151:0/3567382975 shutdown_connections 2024-04-26T01:57:52.750 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.751+0000 7fa87fe00640 1 --2- 172.21.15.151:0/3567382975 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fa8640a3d10 0x7fa8640a61d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:52.750 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.751+0000 7fa87fe00640 1 --2- 172.21.15.151:0/3567382975 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa88010e420 0x7fa88019e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:52.750 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.751+0000 7fa87fe00640 1 --2- 172.21.15.151:0/3567382975 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa880108e10 0x7fa8801a4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:52.750 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.751+0000 7fa87fe00640 1 --2- 172.21.15.151:0/3567382975 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa880108490 0x7fa8801a4780 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:52.750 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.751+0000 7fa87fe00640 1 -- 172.21.15.151:0/3567382975 >> 172.21.15.151:0/3567382975 conn(0x7fa880077d30 msgr2=0x7fa880078ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:52.750 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.751+0000 7fa87fe00640 1 -- 172.21.15.151:0/3567382975 shutdown_connections 2024-04-26T01:57:52.750 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:57:52.751+0000 7fa87fe00640 1 -- 172.21.15.151:0/3567382975 wait complete. 2024-04-26T01:57:52.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:57:52 smithi172 ceph-mon[32785]: pgmap v130: 97 pgs: 97 active+clean; 39 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 221 op/s 2024-04-26T01:57:52.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:57:52 smithi165 ceph-mon[32648]: pgmap v130: 97 pgs: 97 active+clean; 39 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 221 op/s 2024-04-26T01:57:54.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:57:54 smithi151 ceph-mon[28057]: pgmap v131: 97 pgs: 97 active+clean; 40 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 233 op/s 2024-04-26T01:57:54.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:57:54 smithi151 ceph-mon[28057]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T01:57:54.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:57:54 smithi172 ceph-mon[32785]: pgmap v131: 97 pgs: 97 active+clean; 40 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 233 op/s 2024-04-26T01:57:54.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:57:54 smithi172 ceph-mon[32785]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T01:57:54.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:57:54 smithi165 ceph-mon[32648]: pgmap v131: 97 pgs: 97 active+clean; 40 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 233 op/s 2024-04-26T01:57:54.847 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:57:54 smithi165 ceph-mon[32648]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T01:57:56.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:57:56 smithi172 ceph-mon[32785]: pgmap v132: 97 pgs: 97 active+clean; 37 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 404 op/s 2024-04-26T01:57:56.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:57:56 smithi165 ceph-mon[32648]: pgmap v132: 97 pgs: 97 active+clean; 37 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 404 op/s 2024-04-26T01:57:56.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:57:56 smithi151 ceph-mon[28057]: pgmap v132: 97 pgs: 97 active+clean; 37 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 404 op/s 2024-04-26T01:57:58.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:57:58 smithi172 ceph-mon[32785]: pgmap v133: 97 pgs: 97 active+clean; 37 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.6 MiB/s wr, 392 op/s 2024-04-26T01:57:58.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:57:58 smithi165 ceph-mon[32648]: pgmap v133: 97 pgs: 97 active+clean; 37 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.6 MiB/s wr, 392 op/s 2024-04-26T01:57:58.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:57:58 smithi151 ceph-mon[28057]: pgmap v133: 97 pgs: 97 active+clean; 37 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.6 MiB/s wr, 392 op/s 2024-04-26T01:58:00.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:58:00 smithi172 ceph-mon[32785]: pgmap v134: 97 pgs: 97 active+clean; 35 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.6 MiB/s wr, 538 op/s 2024-04-26T01:58:00.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:58:00 smithi165 ceph-mon[32648]: pgmap v134: 97 pgs: 97 active+clean; 35 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.6 MiB/s wr, 538 op/s 2024-04-26T01:58:00.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:58:00 smithi151 ceph-mon[28057]: pgmap v134: 97 pgs: 97 active+clean; 35 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.6 MiB/s wr, 538 op/s 2024-04-26T01:58:02.675 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:58:02 smithi151 ceph-mon[28057]: pgmap v135: 97 pgs: 97 active+clean; 35 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 334 op/s 2024-04-26T01:58:02.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:58:02 smithi172 ceph-mon[32785]: pgmap v135: 97 pgs: 97 active+clean; 35 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 334 op/s 2024-04-26T01:58:02.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:58:02 smithi165 ceph-mon[32648]: pgmap v135: 97 pgs: 97 active+clean; 35 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 334 op/s 2024-04-26T01:58:04.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:58:04 smithi172 ceph-mon[32785]: pgmap v136: 97 pgs: 97 active+clean; 37 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 377 op/s 2024-04-26T01:58:04.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:58:04 smithi165 ceph-mon[32648]: pgmap v136: 97 pgs: 97 active+clean; 37 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 377 op/s 2024-04-26T01:58:04.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:58:04 smithi151 ceph-mon[28057]: pgmap v136: 97 pgs: 97 active+clean; 37 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 377 op/s 2024-04-26T01:58:05.910 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.911+0000 7ff4e2800640 1 -- 172.21.15.151:0/2834456348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff4dc1072a0 msgr2=0x7ff4dc10ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:05.910 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.911+0000 7ff4e2800640 1 --2- 172.21.15.151:0/2834456348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff4dc1072a0 0x7ff4dc10ee70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4009be0 tx=0x7ff4d402a6c0 comp rx=0 tx=0).stop 2024-04-26T01:58:05.910 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.911+0000 7ff4e2800640 1 -- 172.21.15.151:0/2834456348 shutdown_connections 2024-04-26T01:58:05.910 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.911+0000 7ff4e2800640 1 --2- 172.21.15.151:0/2834456348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff4dc1072a0 0x7ff4dc10ee70 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:05.911 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.911+0000 7ff4e2800640 1 --2- 172.21.15.151:0/2834456348 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff4dc102b80 0x7ff4dc106bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:05.911 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.911+0000 7ff4e2800640 1 --2- 172.21.15.151:0/2834456348 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff4dc1021d0 0x7ff4dc1025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:05.911 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.911+0000 7ff4e2800640 1 -- 172.21.15.151:0/2834456348 >> 172.21.15.151:0/2834456348 conn(0x7ff4dc0793e0 msgr2=0x7ff4dc0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:05.911 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.911+0000 7ff4e2800640 1 -- 172.21.15.151:0/2834456348 shutdown_connections 2024-04-26T01:58:05.911 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.911+0000 7ff4e2800640 1 -- 172.21.15.151:0/2834456348 wait complete. 2024-04-26T01:58:05.913 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.912+0000 7ff4e2800640 1 Processor -- start 2024-04-26T01:58:05.913 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.912+0000 7ff4e2800640 1 -- start start 2024-04-26T01:58:05.913 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.913+0000 7ff4e2800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff4dc1021d0 0x7ff4dc1a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:05.913 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.913+0000 7ff4e2800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff4dc1072a0 0x7ff4dc1a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:05.913 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.913+0000 7ff4e2800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff4dc19a6b0 0x7ff4dc19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:05.913 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.913+0000 7ff4e2800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff4dc112210 con 0x7ff4dc19a6b0 2024-04-26T01:58:05.913 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.913+0000 7ff4e2800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff4dc112090 con 0x7ff4dc1021d0 2024-04-26T01:58:05.913 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.913+0000 7ff4e2800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff4dc112390 con 0x7ff4dc1072a0 2024-04-26T01:58:05.913 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.913+0000 7ff4d2a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff4dc1072a0 0x7ff4dc1a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:05.913 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.913+0000 7ff4db400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff4dc1021d0 0x7ff4dc1a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:05.914 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.913+0000 7ff4d2a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff4dc1072a0 0x7ff4dc1a0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:57426/0 (socket says 172.21.15.151:57426) 2024-04-26T01:58:05.914 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.913+0000 7ff4d2a00640 1 -- 172.21.15.151:0/3890126382 learned_addr learned my addr 172.21.15.151:0/3890126382 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:58:05.914 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.913+0000 7ff4dbe00640 1 --2- 172.21.15.151:0/3890126382 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff4dc19a6b0 0x7ff4dc19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:05.914 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.913+0000 7ff4d2a00640 1 -- 172.21.15.151:0/3890126382 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff4dc1021d0 msgr2=0x7ff4dc1a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:05.914 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.913+0000 7ff4d2a00640 1 --2- 172.21.15.151:0/3890126382 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff4dc1021d0 0x7ff4dc1a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:05.914 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.913+0000 7ff4d2a00640 1 -- 172.21.15.151:0/3890126382 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff4dc19a6b0 msgr2=0x7ff4dc19ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:05.914 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.913+0000 7ff4d2a00640 1 --2- 172.21.15.151:0/3890126382 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff4dc19a6b0 0x7ff4dc19ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:05.914 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.914+0000 7ff4d2a00640 1 -- 172.21.15.151:0/3890126382 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4dc19b2b0 con 0x7ff4dc1072a0 2024-04-26T01:58:05.914 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.914+0000 7ff4dbe00640 1 --2- 172.21.15.151:0/3890126382 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff4dc19a6b0 0x7ff4dc19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:58:05.914 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.914+0000 7ff4d2a00640 1 --2- 172.21.15.151:0/3890126382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff4dc1072a0 0x7ff4dc1a0b30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff4c400e9c0 tx=0x7ff4c400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:05.914 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.914+0000 7ff4d8c00640 1 -- 172.21.15.151:0/3890126382 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c400ccc0 con 0x7ff4dc1072a0 2024-04-26T01:58:05.914 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.914+0000 7ff4d8c00640 1 -- 172.21.15.151:0/3890126382 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4c40054d0 con 0x7ff4dc1072a0 2024-04-26T01:58:05.915 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.914+0000 7ff4e2800640 1 -- 172.21.15.151:0/3890126382 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4dc1a7390 con 0x7ff4dc1072a0 2024-04-26T01:58:05.915 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.914+0000 7ff4d8c00640 1 -- 172.21.15.151:0/3890126382 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c4010430 con 0x7ff4dc1072a0 2024-04-26T01:58:05.915 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.914+0000 7ff4e2800640 1 -- 172.21.15.151:0/3890126382 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4dc1a7740 con 0x7ff4dc1072a0 2024-04-26T01:58:05.916 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.918+0000 7ff4e55f3740 1 Processor -- start 2024-04-26T01:58:05.916 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.918+0000 7ff4e55f3740 1 -- start start 2024-04-26T01:58:05.918 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.918+0000 7ff4e55f3740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5597a6dc4820 0x5597a6dc4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:05.918 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.918+0000 7ff4e55f3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5597a6dc5140 0x5597a6dcf4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:05.918 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.918+0000 7ff4d8c00640 1 -- 172.21.15.151:0/3890126382 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7ff4c4005020 con 0x7ff4dc1072a0 2024-04-26T01:58:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.918+0000 7ff4e55f3740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597a6dcfa00 0x5597a6dd1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.918+0000 7ff4e55f3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5597a6d2a900 con 0x5597a6dc5140 2024-04-26T01:58:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.918+0000 7ff4e55f3740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5597a6cd0ef0 con 0x5597a6dcfa00 2024-04-26T01:58:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.918+0000 7ff4e55f3740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5597a6d268c0 con 0x5597a6dc4820 2024-04-26T01:58:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.918+0000 7ff4dbe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5597a6dc4820 0x5597a6dc4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.919+0000 7ff4dbe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5597a6dc4820 0x5597a6dc4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:57432/0 (socket says 172.21.15.151:57432) 2024-04-26T01:58:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.919+0000 7ff4dbe00640 1 -- 172.21.15.151:0/1701254312 learned_addr learned my addr 172.21.15.151:0/1701254312 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:58:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.919+0000 7ff4db400640 1 --2- 172.21.15.151:0/1701254312 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5597a6dc5140 0x5597a6dcf4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.919+0000 7ff4d2a00640 1 --2- 172.21.15.151:0/1701254312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597a6dcfa00 0x5597a6dd1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.919+0000 7ff4dbe00640 1 -- 172.21.15.151:0/1701254312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597a6dcfa00 msgr2=0x5597a6dd1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.919+0000 7ff4dbe00640 1 --2- 172.21.15.151:0/1701254312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597a6dcfa00 0x5597a6dd1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.919+0000 7ff4dbe00640 1 -- 172.21.15.151:0/1701254312 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5597a6dc5140 msgr2=0x5597a6dcf4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.919+0000 7ff4dbe00640 1 --2- 172.21.15.151:0/1701254312 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5597a6dc5140 0x5597a6dcf4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:05.920 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.919+0000 7ff4dbe00640 1 -- 172.21.15.151:0/1701254312 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597a6a3d210 con 0x5597a6dc4820 2024-04-26T01:58:05.920 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.919+0000 7ff4dbe00640 1 --2- 172.21.15.151:0/1701254312 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5597a6dc4820 0x5597a6dc4c00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4009be0 tx=0x7ff4d4030640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e14de608fd3ed2d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:05.920 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.920+0000 7ff4d0c00640 1 -- 172.21.15.151:0/1701254312 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d4040030 con 0x5597a6dc4820 2024-04-26T01:58:05.920 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.920+0000 7ff4d0c00640 1 -- 172.21.15.151:0/1701254312 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4d4003e20 con 0x5597a6dc4820 2024-04-26T01:58:05.920 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.920+0000 7ff4d0c00640 1 -- 172.21.15.151:0/1701254312 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d4036040 con 0x5597a6dc4820 2024-04-26T01:58:05.924 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.920+0000 7ff4d8c00640 1 --2- 172.21.15.151:0/3890126382 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7ff4c00a3dc0 0x7ff4c00a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:05.924 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.921+0000 7ff4e55f3740 1 -- 172.21.15.151:0/1701254312 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5597a6dc4820 msgr2=0x5597a6dc4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:05.924 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.921+0000 7ff4e55f3740 1 --2- 172.21.15.151:0/1701254312 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5597a6dc4820 0x5597a6dc4c00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4009be0 tx=0x7ff4d4030640 comp rx=0 tx=0).stop 2024-04-26T01:58:05.924 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.921+0000 7ff4d8c00640 1 -- 172.21.15.151:0/3890126382 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7ff4c40d0be0 con 0x7ff4dc1072a0 2024-04-26T01:58:05.924 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.921+0000 7ff4e55f3740 1 -- 172.21.15.151:0/1701254312 shutdown_connections 2024-04-26T01:58:05.924 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.921+0000 7ff4e55f3740 1 --2- 172.21.15.151:0/1701254312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597a6dcfa00 0x5597a6dd1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:05.924 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.921+0000 7ff4e55f3740 1 --2- 172.21.15.151:0/1701254312 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5597a6dc5140 0x5597a6dcf4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:05.924 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.921+0000 7ff4e55f3740 1 --2- 172.21.15.151:0/1701254312 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5597a6dc4820 0x5597a6dc4c00 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:05.924 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.921+0000 7ff4db400640 1 --2- 172.21.15.151:0/3890126382 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7ff4c00a3dc0 0x7ff4c00a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:05.924 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.921+0000 7ff4e55f3740 1 -- 172.21.15.151:0/1701254312 >> 172.21.15.151:0/1701254312 conn(0x5597a6dc0d30 msgr2=0x5597a6dc1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:05.924 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.921+0000 7ff4e55f3740 1 -- 172.21.15.151:0/1701254312 shutdown_connections 2024-04-26T01:58:05.925 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.921+0000 7ff4e55f3740 1 -- 172.21.15.151:0/1701254312 wait complete. 2024-04-26T01:58:05.925 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.922+0000 7ff4db400640 1 --2- 172.21.15.151:0/3890126382 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7ff4c00a3dc0 0x7ff4c00a6280 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x5597a6dc5dc0 tx=0x7ff4cc00b410 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:05.925 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.922+0000 7ff4e55f3740 1 -- start start 2024-04-26T01:58:05.925 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.922+0000 7ff4e55f3740 1 Processor -- start 2024-04-26T01:58:05.925 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.922+0000 7ff4e55f3740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597a6dc4820 0x5597a6dcc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:05.925 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.922+0000 7ff4e55f3740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5597a6dc5140 0x5597a6dccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:05.925 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.922+0000 7ff4e55f3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5597a6dcfa00 0x5597a6dcd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:05.925 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.922+0000 7ff4e55f3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5597a6dd25d0 con 0x5597a6dcfa00 2024-04-26T01:58:05.925 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.922+0000 7ff4e55f3740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5597a6ce4310 con 0x5597a6dc4820 2024-04-26T01:58:05.925 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.922+0000 7ff4e55f3740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5597a6dd2750 con 0x5597a6dc5140 2024-04-26T01:58:05.925 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.922+0000 7ff4e55f3740 10 client.0 _ll_register_callbacks cb 0x5597a6cb2cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:58:05.925 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.923+0000 7ff4d2a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5597a6dc5140 0x5597a6dccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:05.925 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.923+0000 7ff4d2a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5597a6dc5140 0x5597a6dccf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:57442/0 (socket says 172.21.15.151:57442) 2024-04-26T01:58:05.926 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.923+0000 7ff4d2a00640 1 -- 172.21.15.151:0/1667355645 learned_addr learned my addr 172.21.15.151:0/1667355645 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:58:05.926 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.923+0000 7ff4db400640 1 --2- 172.21.15.151:0/1667355645 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597a6dc4820 0x5597a6dcc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:05.926 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.923+0000 7ff4dbe00640 1 --2- 172.21.15.151:0/1667355645 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5597a6dcfa00 0x5597a6dcd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:05.926 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.923+0000 7ff4d2a00640 1 -- 172.21.15.151:0/1667355645 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597a6dc4820 msgr2=0x5597a6dcc9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:05.926 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.923+0000 7ff4d2a00640 1 --2- 172.21.15.151:0/1667355645 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597a6dc4820 0x5597a6dcc9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:05.926 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.923+0000 7ff4d2a00640 1 -- 172.21.15.151:0/1667355645 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5597a6dcfa00 msgr2=0x5597a6dcd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:05.926 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.923+0000 7ff4d2a00640 1 --2- 172.21.15.151:0/1667355645 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5597a6dcfa00 0x5597a6dcd470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:05.926 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.923+0000 7ff4d2a00640 1 -- 172.21.15.151:0/1667355645 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597a6b09620 con 0x5597a6dc5140 2024-04-26T01:58:05.926 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.923+0000 7ff4dbe00640 1 --2- 172.21.15.151:0/1667355645 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5597a6dcfa00 0x5597a6dcd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:58:05.926 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.923+0000 7ff4db400640 1 --2- 172.21.15.151:0/1667355645 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597a6dc4820 0x5597a6dcc9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:58:05.926 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.923+0000 7ff4d2a00640 1 --2- 172.21.15.151:0/1667355645 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5597a6dc5140 0x5597a6dccf30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff4c400cfd0 tx=0x7ff4c400ce60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=197d5ef705d63432 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:05.926 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.923+0000 7ff4c9600640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-26T01:58:05.927 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.923+0000 7ff4c9600640 1 -- 172.21.15.151:0/1667355645 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c409c6d0 con 0x5597a6dc5140 2024-04-26T01:58:05.927 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.923+0000 7ff4c9600640 1 -- 172.21.15.151:0/1667355645 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7ff4c40ce5c0 con 0x5597a6dc5140 2024-04-26T01:58:05.927 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.923+0000 7ff4e55f3740 1 -- 172.21.15.151:0/1667355645 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4d4003e20 con 0x5597a6dc5140 2024-04-26T01:58:05.927 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.924+0000 7ff4c9600640 1 -- 172.21.15.151:0/1667355645 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff4c409c4f0 con 0x5597a6dc5140 2024-04-26T01:58:05.927 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.924+0000 7ff4e55f3740 10 client.25726 fetch_fsmap learned FSMap version 30 2024-04-26T01:58:05.927 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.924+0000 7ff4e55f3740 1 -- 172.21.15.151:0/1667355645 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5597a6dde8f0 con 0x5597a6dc5140 2024-04-26T01:58:05.927 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.924+0000 7ff4c9600640 1 -- 172.21.15.151:0/1667355645 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4d4003e20 con 0x5597a6dc5140 2024-04-26T01:58:05.927 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.924+0000 7ff4c9600640 1 -- 172.21.15.151:0/1667355645 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7ff4c40988e0 con 0x5597a6dc5140 2024-04-26T01:58:05.927 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.924+0000 7ff4e55f3740 10 client.25726 fetch_fsmap finished waiting for FSMap version 30 2024-04-26T01:58:05.927 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.924+0000 7ff4e55f3740 10 client.25726 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:58:05.927 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.924+0000 7ff4e55f3740 20 client.25726 populate_metadata read hostname 'smithi151' 2024-04-26T01:58:05.927 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.924+0000 7ff4e55f3740 1 --2- 172.21.15.151:0/1667355645 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5597a6ddfdb0 0x5597a6de21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:05.927 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.924+0000 7ff4e55f3740 4 client.25726 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:58:05.928 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.924+0000 7ff4e55f3740 1 -- 172.21.15.151:0/1667355645 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597a6d3e8b0 con 0x5597a6ddfdb0 2024-04-26T01:58:05.928 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.925+0000 7ff4db400640 1 --2- 172.21.15.151:0/1667355645 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5597a6ddfdb0 0x5597a6de21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:05.928 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.925+0000 7ff4db400640 1 --2- 172.21.15.151:0/1667355645 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5597a6ddfdb0 0x5597a6de21e0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad76fb0b06ddf14d server_cookie=461035970c1d111d in_seq=0 out_seq=0 2024-04-26T01:58:05.928 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.925+0000 7ff4c9600640 10 client.25726 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:58:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.926+0000 7ff4c9600640 1 -- 172.21.15.151:0/1667355645 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5597a6d3e8b0 con 0x5597a6ddfdb0 2024-04-26T01:58:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.926+0000 7ff4c9600640 10 client.25726 handle_command_reply: tid=0 2024-04-26T01:58:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.926+0000 7ff4e55f3740 1 client.25726 shutdown 2024-04-26T01:58:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.926+0000 7ff4e55f3740 1 -- 172.21.15.151:0/1667355645 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5597a6dc5140 msgr2=0x5597a6dccf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.926+0000 7ff4e55f3740 1 --2- 172.21.15.151:0/1667355645 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5597a6dc5140 0x5597a6dccf30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff4c400cfd0 tx=0x7ff4c400ce60 comp rx=0 tx=0).stop 2024-04-26T01:58:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.927+0000 7ff4e55f3740 1 -- 172.21.15.151:0/1667355645 shutdown_connections 2024-04-26T01:58:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.927+0000 7ff4e55f3740 1 --2- 172.21.15.151:0/1667355645 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5597a6ddfdb0 0x5597a6de21e0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.927+0000 7ff4e55f3740 1 --2- 172.21.15.151:0/1667355645 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5597a6dcfa00 0x5597a6dcd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.927+0000 7ff4e55f3740 1 --2- 172.21.15.151:0/1667355645 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5597a6dc5140 0x5597a6dccf30 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.927+0000 7ff4e55f3740 1 --2- 172.21.15.151:0/1667355645 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597a6dc4820 0x5597a6dcc9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:05.930 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.927+0000 7ff4c9600640 0 client.25726 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:58:05.930 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.927+0000 7ff4e55f3740 1 -- 172.21.15.151:0/1667355645 >> 172.21.15.151:0/1667355645 conn(0x5597a6dc0d30 msgr2=0x5597a6dc6ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:05.930 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.927+0000 7ff4e55f3740 1 -- 172.21.15.151:0/1667355645 shutdown_connections 2024-04-26T01:58:05.930 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.927+0000 7ff4e55f3740 1 -- 172.21.15.151:0/1667355645 wait complete. 2024-04-26T01:58:05.930 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.927+0000 7ff4e55f3740 20 client.25726 trim_cache size 0 max 16384 2024-04-26T01:58:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.944+0000 7ff4e55f3740 1 Processor -- start 2024-04-26T01:58:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.944+0000 7ff4e55f3740 1 -- start start 2024-04-26T01:58:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.944+0000 7ff4e55f3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5597a6dc4820 0x5597a6dc0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.945+0000 7ff4e55f3740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5597a6dc5140 0x5597a6dc68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.945+0000 7ff4e55f3740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597a6dcfa00 0x5597a6de7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.945+0000 7ff4e55f3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff4dc1b2480 con 0x5597a6dc4820 2024-04-26T01:58:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.945+0000 7ff4e55f3740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff4dc1afd50 con 0x5597a6dcfa00 2024-04-26T01:58:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.945+0000 7ff4e55f3740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff4dc1adba0 con 0x5597a6dc5140 2024-04-26T01:58:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.945+0000 7ff4dbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597a6dcfa00 0x5597a6de7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.945+0000 7ff4d2a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5597a6dc5140 0x5597a6dc68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.945+0000 7ff4db400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5597a6dc4820 0x5597a6dc0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.945+0000 7ff4d2a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5597a6dc5140 0x5597a6dc68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:57452/0 (socket says 172.21.15.151:57452) 2024-04-26T01:58:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.945+0000 7ff4dbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597a6dcfa00 0x5597a6de7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:46346/0 (socket says 172.21.15.151:46346) 2024-04-26T01:58:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.945+0000 7ff4d2a00640 1 -- 172.21.15.151:0/3860916241 learned_addr learned my addr 172.21.15.151:0/3860916241 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:58:05.946 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.945+0000 7ff4d2a00640 1 -- 172.21.15.151:0/3860916241 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597a6dcfa00 msgr2=0x5597a6de7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:05.946 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.945+0000 7ff4d2a00640 1 --2- 172.21.15.151:0/3860916241 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597a6dcfa00 0x5597a6de7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:05.946 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.945+0000 7ff4d2a00640 1 -- 172.21.15.151:0/3860916241 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5597a6dc4820 msgr2=0x5597a6dc0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:05.946 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.945+0000 7ff4d2a00640 1 --2- 172.21.15.151:0/3860916241 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5597a6dc4820 0x5597a6dc0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:05.946 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.945+0000 7ff4d2a00640 1 -- 172.21.15.151:0/3860916241 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597a6de2860 con 0x5597a6dc5140 2024-04-26T01:58:05.946 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.945+0000 7ff4dbe00640 1 --2- 172.21.15.151:0/3860916241 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597a6dcfa00 0x5597a6de7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:58:05.946 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.946+0000 7ff4d2a00640 1 --2- 172.21.15.151:0/3860916241 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5597a6dc5140 0x5597a6dc68e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4099560 tx=0x7ff4c409b0a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e32be927230d0f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:05.946 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.946+0000 7ff4cbe00640 1 -- 172.21.15.151:0/3860916241 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c409c6d0 con 0x5597a6dc5140 2024-04-26T01:58:05.946 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.946+0000 7ff4cbe00640 1 -- 172.21.15.151:0/3860916241 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4c4005310 con 0x5597a6dc5140 2024-04-26T01:58:05.947 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.946+0000 7ff4cbe00640 1 -- 172.21.15.151:0/3860916241 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c4098e30 con 0x5597a6dc5140 2024-04-26T01:58:05.947 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.947+0000 7ff4e55f3740 1 -- 172.21.15.151:0/3860916241 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5597a6dc5140 msgr2=0x5597a6dc68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:05.947 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.947+0000 7ff4e55f3740 1 --2- 172.21.15.151:0/3860916241 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5597a6dc5140 0x5597a6dc68e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4099560 tx=0x7ff4c409b0a0 comp rx=0 tx=0).stop 2024-04-26T01:58:05.947 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.947+0000 7ff4e55f3740 1 -- 172.21.15.151:0/3860916241 shutdown_connections 2024-04-26T01:58:05.947 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.947+0000 7ff4e55f3740 1 --2- 172.21.15.151:0/3860916241 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597a6dcfa00 0x5597a6de7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:05.947 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.947+0000 7ff4e55f3740 1 --2- 172.21.15.151:0/3860916241 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5597a6dc5140 0x5597a6dc68e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:05.947 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.947+0000 7ff4e55f3740 1 --2- 172.21.15.151:0/3860916241 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5597a6dc4820 0x5597a6dc0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:05.947 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.947+0000 7ff4e55f3740 1 -- 172.21.15.151:0/3860916241 >> 172.21.15.151:0/3860916241 conn(0x5597a6dc0d30 msgr2=0x5597a6dc61d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:05.947 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.947+0000 7ff4e55f3740 1 -- 172.21.15.151:0/3860916241 shutdown_connections 2024-04-26T01:58:05.947 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.947+0000 7ff4e55f3740 1 -- 172.21.15.151:0/3860916241 wait complete. 2024-04-26T01:58:05.947 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.947+0000 7ff4e55f3740 1 -- start start 2024-04-26T01:58:05.950 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.948+0000 7ff4e55f3740 1 Processor -- start 2024-04-26T01:58:05.950 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.948+0000 7ff4e55f3740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597a6dc4820 0x5597a6dd5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:05.950 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.948+0000 7ff4e55f3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5597a6dc5140 0x5597a6dd5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:05.950 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.948+0000 7ff4e55f3740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5597a6dcfa00 0x5597a6dd60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:05.950 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.948+0000 7ff4e55f3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff4dc1ae210 con 0x5597a6dc5140 2024-04-26T01:58:05.951 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.948+0000 7ff4e55f3740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff4dc1af680 con 0x5597a6dc4820 2024-04-26T01:58:05.951 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.948+0000 7ff4e55f3740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff4dc1ae8e0 con 0x5597a6dcfa00 2024-04-26T01:58:05.951 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.948+0000 7ff4e55f3740 10 client.0 _ll_register_callbacks cb 0x7ff4c40d4380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:58:05.951 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.948+0000 7ff4db400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597a6dc4820 0x5597a6dd5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:05.951 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.948+0000 7ff4d2a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5597a6dc5140 0x5597a6dd5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:05.951 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.949+0000 7ff4d2a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5597a6dc5140 0x5597a6dd5b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37402/0 (socket says 172.21.15.151:37402) 2024-04-26T01:58:05.951 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.949+0000 7ff4d2a00640 1 -- 172.21.15.151:0/2489269419 learned_addr learned my addr 172.21.15.151:0/2489269419 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:58:05.951 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.949+0000 7ff4dbe00640 1 --2- 172.21.15.151:0/2489269419 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5597a6dcfa00 0x5597a6dd60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:05.951 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.949+0000 7ff4d2a00640 1 -- 172.21.15.151:0/2489269419 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5597a6dcfa00 msgr2=0x5597a6dd60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:05.951 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.949+0000 7ff4d2a00640 1 --2- 172.21.15.151:0/2489269419 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5597a6dcfa00 0x5597a6dd60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:05.951 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.949+0000 7ff4d2a00640 1 -- 172.21.15.151:0/2489269419 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597a6dc4820 msgr2=0x5597a6dd5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:05.951 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.949+0000 7ff4d2a00640 1 --2- 172.21.15.151:0/2489269419 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597a6dc4820 0x5597a6dd5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:05.951 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.949+0000 7ff4d2a00640 1 -- 172.21.15.151:0/2489269419 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597a6e45610 con 0x5597a6dc5140 2024-04-26T01:58:05.951 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.949+0000 7ff4db400640 1 --2- 172.21.15.151:0/2489269419 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597a6dc4820 0x5597a6dd5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:58:05.952 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.949+0000 7ff4d2a00640 1 --2- 172.21.15.151:0/2489269419 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5597a6dc5140 0x5597a6dd5b90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4098d10 tx=0x7ff4c40d4210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b501ec974ea2b127 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:05.952 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.949+0000 7ff4c9600640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-04-26T01:58:05.952 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.949+0000 7ff4c9600640 1 -- 172.21.15.151:0/2489269419 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c4098e30 con 0x5597a6dc5140 2024-04-26T01:58:05.952 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.949+0000 7ff4c9600640 1 -- 172.21.15.151:0/2489269419 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7ff4c409c1e0 con 0x5597a6dc5140 2024-04-26T01:58:05.952 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.949+0000 7ff4e55f3740 1 -- 172.21.15.151:0/2489269419 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5597a6d38320 con 0x5597a6dc5140 2024-04-26T01:58:05.952 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.950+0000 7ff4c9600640 1 -- 172.21.15.151:0/2489269419 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff4c409c6d0 con 0x5597a6dc5140 2024-04-26T01:58:05.952 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.950+0000 7ff4e55f3740 10 client.15606 fetch_fsmap learned FSMap version 30 2024-04-26T01:58:05.952 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.950+0000 7ff4e55f3740 1 -- 172.21.15.151:0/2489269419 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5597a6e46900 con 0x5597a6dc5140 2024-04-26T01:58:05.952 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.950+0000 7ff4c9600640 1 -- 172.21.15.151:0/2489269419 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5597a6d38320 con 0x5597a6dc5140 2024-04-26T01:58:05.952 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.950+0000 7ff4c9600640 1 -- 172.21.15.151:0/2489269419 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7ff4c40cea90 con 0x5597a6dc5140 2024-04-26T01:58:05.952 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.950+0000 7ff4e55f3740 10 client.15606 fetch_fsmap finished waiting for FSMap version 30 2024-04-26T01:58:05.952 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.950+0000 7ff4e55f3740 10 client.15606 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:58:05.952 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.950+0000 7ff4e55f3740 20 client.15606 populate_metadata read hostname 'smithi151' 2024-04-26T01:58:05.952 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.950+0000 7ff4e55f3740 1 --2- 172.21.15.151:0/2489269419 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5597a6e46c50 0x5597a6e49070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:05.953 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.950+0000 7ff4e55f3740 4 client.15606 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:58:05.953 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.950+0000 7ff4e55f3740 1 -- 172.21.15.151:0/2489269419 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5597a6d32190 con 0x5597a6e46c50 2024-04-26T01:58:05.953 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.951+0000 7ff4db400640 1 --2- 172.21.15.151:0/2489269419 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5597a6e46c50 0x5597a6e49070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:05.953 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.951+0000 7ff4db400640 1 --2- 172.21.15.151:0/2489269419 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5597a6e46c50 0x5597a6e49070 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d4df684bdb16c49 server_cookie=2d0fecac2cb71f3d in_seq=0 out_seq=0 2024-04-26T01:58:05.953 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.951+0000 7ff4c9600640 10 client.15606 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:58:05.963 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.965+0000 7ff4c9600640 1 -- 172.21.15.151:0/2489269419 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5597a6d32190 con 0x5597a6e46c50 2024-04-26T01:58:05.963 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.965+0000 7ff4c9600640 10 client.15606 handle_command_reply: tid=0 2024-04-26T01:58:05.964 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.965+0000 7ff4e55f3740 1 client.15606 shutdown 2024-04-26T01:58:05.964 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.965+0000 7ff4e55f3740 1 -- 172.21.15.151:0/2489269419 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5597a6dc5140 msgr2=0x5597a6dd5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:05.964 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.965+0000 7ff4e55f3740 1 --2- 172.21.15.151:0/2489269419 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5597a6dc5140 0x5597a6dd5b90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4098d10 tx=0x7ff4c40d4210 comp rx=0 tx=0).stop 2024-04-26T01:58:05.964 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.965+0000 7ff4e55f3740 1 -- 172.21.15.151:0/2489269419 shutdown_connections 2024-04-26T01:58:05.964 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.965+0000 7ff4e55f3740 1 --2- 172.21.15.151:0/2489269419 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5597a6e46c50 0x5597a6e49070 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:05.965 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.965+0000 7ff4e55f3740 1 --2- 172.21.15.151:0/2489269419 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5597a6dcfa00 0x5597a6dd60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:05.965 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.965+0000 7ff4e55f3740 1 --2- 172.21.15.151:0/2489269419 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5597a6dc5140 0x5597a6dd5b90 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:05.965 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.965+0000 7ff4e55f3740 1 --2- 172.21.15.151:0/2489269419 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597a6dc4820 0x5597a6dd5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:05.965 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.965+0000 7ff4e55f3740 1 -- 172.21.15.151:0/2489269419 >> 172.21.15.151:0/2489269419 conn(0x5597a6dc0d30 msgr2=0x5597a6dc6a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:05.965 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.965+0000 7ff4c9600640 0 client.15606 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:58:05.965 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.965+0000 7ff4e55f3740 1 -- 172.21.15.151:0/2489269419 shutdown_connections 2024-04-26T01:58:05.965 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.965+0000 7ff4e55f3740 1 -- 172.21.15.151:0/2489269419 wait complete. 2024-04-26T01:58:05.965 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.966+0000 7ff4e55f3740 20 client.15606 trim_cache size 0 max 16384 2024-04-26T01:58:05.965 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:{ 2024-04-26T01:58:05.965 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "message": "", 2024-04-26T01:58:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "return_code": 0 2024-04-26T01:58:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:} 2024-04-26T01:58:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.966+0000 7ff4e2800640 1 -- 172.21.15.151:0/3890126382 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7ff4c00a3dc0 msgr2=0x7ff4c00a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.966+0000 7ff4e2800640 1 --2- 172.21.15.151:0/3890126382 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7ff4c00a3dc0 0x7ff4c00a6280 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x5597a6dc5dc0 tx=0x7ff4cc00b410 comp rx=0 tx=0).stop 2024-04-26T01:58:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.966+0000 7ff4e2800640 1 -- 172.21.15.151:0/3890126382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff4dc1072a0 msgr2=0x7ff4dc1a0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.966+0000 7ff4e2800640 1 --2- 172.21.15.151:0/3890126382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff4dc1072a0 0x7ff4dc1a0b30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff4c400e9c0 tx=0x7ff4c400ee80 comp rx=0 tx=0).stop 2024-04-26T01:58:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.966+0000 7ff4e2800640 1 -- 172.21.15.151:0/3890126382 shutdown_connections 2024-04-26T01:58:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.966+0000 7ff4e2800640 1 --2- 172.21.15.151:0/3890126382 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7ff4c00a3dc0 0x7ff4c00a6280 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.966+0000 7ff4e2800640 1 --2- 172.21.15.151:0/3890126382 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff4dc19a6b0 0x7ff4dc19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.966+0000 7ff4e2800640 1 --2- 172.21.15.151:0/3890126382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff4dc1072a0 0x7ff4dc1a0b30 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.966+0000 7ff4e2800640 1 --2- 172.21.15.151:0/3890126382 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff4dc1021d0 0x7ff4dc1a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.967+0000 7ff4e2800640 1 -- 172.21.15.151:0/3890126382 >> 172.21.15.151:0/3890126382 conn(0x7ff4dc0793e0 msgr2=0x7ff4dc0fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.967+0000 7ff4e2800640 1 -- 172.21.15.151:0/3890126382 shutdown_connections 2024-04-26T01:58:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:05.967+0000 7ff4e2800640 1 -- 172.21.15.151:0/3890126382 wait complete. 2024-04-26T01:58:06.648 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:58:06 smithi165 ceph-mon[32648]: pgmap v137: 97 pgs: 97 active+clean; 38 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 474 op/s 2024-04-26T01:58:06.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:58:06 smithi172 ceph-mon[32785]: pgmap v137: 97 pgs: 97 active+clean; 38 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 474 op/s 2024-04-26T01:58:06.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:58:06 smithi151 ceph-mon[28057]: pgmap v137: 97 pgs: 97 active+clean; 38 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 474 op/s 2024-04-26T01:58:08.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:58:08 smithi172 ceph-mon[32785]: pgmap v138: 97 pgs: 97 active+clean; 38 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 297 op/s 2024-04-26T01:58:08.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:58:08 smithi165 ceph-mon[32648]: pgmap v138: 97 pgs: 97 active+clean; 38 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 297 op/s 2024-04-26T01:58:08.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:58:08 smithi151 ceph-mon[28057]: pgmap v138: 97 pgs: 97 active+clean; 38 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 297 op/s 2024-04-26T01:58:10.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:58:10 smithi172 ceph-mon[32785]: pgmap v139: 97 pgs: 97 active+clean; 36 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 436 op/s 2024-04-26T01:58:10.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:58:10 smithi165 ceph-mon[32648]: pgmap v139: 97 pgs: 97 active+clean; 36 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 436 op/s 2024-04-26T01:58:10.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:58:10 smithi151 ceph-mon[28057]: pgmap v139: 97 pgs: 97 active+clean; 36 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 436 op/s 2024-04-26T01:58:12.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:58:12 smithi172 ceph-mon[32785]: pgmap v140: 97 pgs: 97 active+clean; 36 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 290 op/s 2024-04-26T01:58:12.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:58:12 smithi165 ceph-mon[32648]: pgmap v140: 97 pgs: 97 active+clean; 36 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 290 op/s 2024-04-26T01:58:12.875 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:58:12 smithi151 ceph-mon[28057]: pgmap v140: 97 pgs: 97 active+clean; 36 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 290 op/s 2024-04-26T01:58:14.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:58:14 smithi172 ceph-mon[32785]: pgmap v141: 97 pgs: 97 active+clean; 36 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 290 op/s 2024-04-26T01:58:14.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:58:14 smithi165 ceph-mon[32648]: pgmap v141: 97 pgs: 97 active+clean; 36 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 290 op/s 2024-04-26T01:58:14.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:58:14 smithi151 ceph-mon[28057]: pgmap v141: 97 pgs: 97 active+clean; 36 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 290 op/s 2024-04-26T01:58:16.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:58:16 smithi172 ceph-mon[32785]: pgmap v142: 97 pgs: 97 active+clean; 38 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 379 op/s 2024-04-26T01:58:16.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:58:16 smithi165 ceph-mon[32648]: pgmap v142: 97 pgs: 97 active+clean; 38 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 379 op/s 2024-04-26T01:58:16.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:58:16 smithi151 ceph-mon[28057]: pgmap v142: 97 pgs: 97 active+clean; 38 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 379 op/s 2024-04-26T01:58:17.234 INFO:tasks.workunit.client.0.smithi151.stdout:30000 trunc from 0x7bb2f to 0xcdf83 2024-04-26T01:58:17.884 INFO:tasks.workunit.client.0.smithi151.stdout:copying to largest ever: 0xffcdd 2024-04-26T01:58:18.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:58:18 smithi172 ceph-mon[32785]: pgmap v143: 97 pgs: 97 active+clean; 38 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 270 op/s 2024-04-26T01:58:18.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:58:18 smithi165 ceph-mon[32648]: pgmap v143: 97 pgs: 97 active+clean; 38 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 270 op/s 2024-04-26T01:58:18.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:58:18 smithi151 ceph-mon[28057]: pgmap v143: 97 pgs: 97 active+clean; 38 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 270 op/s 2024-04-26T01:58:19.119 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.120+0000 7fb00d200640 1 -- 172.21.15.151:0/275233113 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb008103830 msgr2=0x7fb00810bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:19.120 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.120+0000 7fb00d200640 1 --2- 172.21.15.151:0/275233113 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb008103830 0x7fb00810bd10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7faff4009900 tx=0x7faff402a490 comp rx=0 tx=0).stop 2024-04-26T01:58:19.120 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.120+0000 7fb00d200640 1 -- 172.21.15.151:0/275233113 shutdown_connections 2024-04-26T01:58:19.120 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.120+0000 7fb00d200640 1 --2- 172.21.15.151:0/275233113 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb00810c330 0x7fb00810e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:19.120 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.120+0000 7fb00d200640 1 --2- 172.21.15.151:0/275233113 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb008103830 0x7fb00810bd10 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:19.120 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.120+0000 7fb00d200640 1 --2- 172.21.15.151:0/275233113 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb008102f10 0x7fb0081032f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:19.120 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.120+0000 7fb00d200640 1 -- 172.21.15.151:0/275233113 >> 172.21.15.151:0/275233113 conn(0x7fb0080fc770 msgr2=0x7fb0080feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:19.120 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.120+0000 7fb00d200640 1 -- 172.21.15.151:0/275233113 shutdown_connections 2024-04-26T01:58:19.120 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.121+0000 7fb00d200640 1 -- 172.21.15.151:0/275233113 wait complete. 2024-04-26T01:58:19.121 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.122+0000 7fb00d200640 1 Processor -- start 2024-04-26T01:58:19.121 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.122+0000 7fb00d200640 1 -- start start 2024-04-26T01:58:19.121 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.122+0000 7fb00d200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb008102f10 0x7fb008100b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:19.122 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.122+0000 7fb00d200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb008103830 0x7fb0081010a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:19.122 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.122+0000 7fb00d200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb00810c330 0x7fb0081015e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:19.122 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.122+0000 7fb00d200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb008112010 con 0x7fb008103830 2024-04-26T01:58:19.122 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.122+0000 7fb00d200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb008111e90 con 0x7fb008102f10 2024-04-26T01:58:19.122 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.122+0000 7fb00d200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb008112190 con 0x7fb00810c330 2024-04-26T01:58:19.122 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.123+0000 7fb007400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb008102f10 0x7fb008100b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:19.122 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.123+0000 7fb006a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb008103830 0x7fb0081010a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:19.122 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.123+0000 7fb007e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb00810c330 0x7fb0081015e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:19.122 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.123+0000 7fb007400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb008102f10 0x7fb008100b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:47932/0 (socket says 172.21.15.151:47932) 2024-04-26T01:58:19.122 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.123+0000 7fb007400640 1 -- 172.21.15.151:0/1632712541 learned_addr learned my addr 172.21.15.151:0/1632712541 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:58:19.123 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.123+0000 7fb007400640 1 -- 172.21.15.151:0/1632712541 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb00810c330 msgr2=0x7fb0081015e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:19.123 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.123+0000 7fb007400640 1 --2- 172.21.15.151:0/1632712541 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb00810c330 0x7fb0081015e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:19.123 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.123+0000 7fb007400640 1 -- 172.21.15.151:0/1632712541 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb008103830 msgr2=0x7fb0081010a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:19.124 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.123+0000 7fb007400640 1 --2- 172.21.15.151:0/1632712541 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb008103830 0x7fb0081010a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:19.124 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.123+0000 7fb007400640 1 -- 172.21.15.151:0/1632712541 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0081a6a70 con 0x7fb008102f10 2024-04-26T01:58:19.124 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.123+0000 7fb007400640 1 --2- 172.21.15.151:0/1632712541 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb008102f10 0x7fb008100b60 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7faff800e9f0 tx=0x7faff800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:19.124 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.124+0000 7fafffe00640 1 -- 172.21.15.151:0/1632712541 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff800cd80 con 0x7fb008102f10 2024-04-26T01:58:19.124 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.124+0000 7fb00d200640 1 -- 172.21.15.151:0/1632712541 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0081a6d60 con 0x7fb008102f10 2024-04-26T01:58:19.124 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.124+0000 7fafffe00640 1 -- 172.21.15.151:0/1632712541 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faff80054d0 con 0x7fb008102f10 2024-04-26T01:58:19.124 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.124+0000 7fafffe00640 1 -- 172.21.15.151:0/1632712541 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff8010690 con 0x7fb008102f10 2024-04-26T01:58:19.124 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.124+0000 7fb00d200640 1 -- 172.21.15.151:0/1632712541 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0081a71d0 con 0x7fb008102f10 2024-04-26T01:58:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.128+0000 7fb011644740 1 Processor -- start 2024-04-26T01:58:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.128+0000 7fb011644740 1 -- start start 2024-04-26T01:58:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.128+0000 7fafffe00640 1 -- 172.21.15.151:0/1632712541 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7faff8004e50 con 0x7fb008102f10 2024-04-26T01:58:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.128+0000 7fb011644740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561af490d9b0 0x561af490dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.128+0000 7fb011644740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561af490e2d0 0x561af4918650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.128+0000 7fb011644740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561af4918b90 0x561af491af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.128+0000 7fb011644740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x561af4873900 con 0x561af490e2d0 2024-04-26T01:58:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.128+0000 7fb011644740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561af4819ef0 con 0x561af4918b90 2024-04-26T01:58:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.128+0000 7fb011644740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561af486f8c0 con 0x561af490d9b0 2024-04-26T01:58:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.129+0000 7fb007e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561af490d9b0 0x561af490dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.129+0000 7fb007e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561af490d9b0 0x561af490dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:53550/0 (socket says 172.21.15.151:53550) 2024-04-26T01:58:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.129+0000 7fb007e00640 1 -- 172.21.15.151:0/2512347403 learned_addr learned my addr 172.21.15.151:0/2512347403 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:58:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.129+0000 7fb007400640 1 --2- 172.21.15.151:0/2512347403 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561af490e2d0 0x561af4918650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:19.129 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.129+0000 7fb007e00640 1 -- 172.21.15.151:0/2512347403 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561af4918b90 msgr2=0x561af491af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:19.129 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.129+0000 7fb007e00640 1 --2- 172.21.15.151:0/2512347403 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561af4918b90 0x561af491af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:19.129 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.129+0000 7fb007e00640 1 -- 172.21.15.151:0/2512347403 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561af490e2d0 msgr2=0x561af4918650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:19.129 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.129+0000 7fb007e00640 1 --2- 172.21.15.151:0/2512347403 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561af490e2d0 0x561af4918650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:19.129 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.129+0000 7fb007e00640 1 -- 172.21.15.151:0/2512347403 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561af4586210 con 0x561af490d9b0 2024-04-26T01:58:19.130 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.129+0000 7fb007e00640 1 --2- 172.21.15.151:0/2512347403 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561af490d9b0 0x561af490dd90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb000009f70 tx=0x7fb000010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50bd69b743dacef6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.130+0000 7faffcc00640 1 -- 172.21.15.151:0/2512347403 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb000016070 con 0x561af490d9b0 2024-04-26T01:58:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.130+0000 7faffcc00640 1 -- 172.21.15.151:0/2512347403 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb000004d20 con 0x561af490d9b0 2024-04-26T01:58:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.130+0000 7faffcc00640 1 -- 172.21.15.151:0/2512347403 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0000127d0 con 0x561af490d9b0 2024-04-26T01:58:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.130+0000 7fafffe00640 1 --2- 172.21.15.151:0/1632712541 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fafec0a3d40 0x7fafec0a6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.130+0000 7fafffe00640 1 -- 172.21.15.151:0/1632712541 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7faff80d0ba0 con 0x7fb008102f10 2024-04-26T01:58:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.130+0000 7fb007400640 1 --2- 172.21.15.151:0/1632712541 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fafec0a3d40 0x7fafec0a6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.130+0000 7fb011644740 1 -- 172.21.15.151:0/2512347403 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561af490d9b0 msgr2=0x561af490dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.130+0000 7fb011644740 1 --2- 172.21.15.151:0/2512347403 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561af490d9b0 0x561af490dd90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb000009f70 tx=0x7fb000010800 comp rx=0 tx=0).stop 2024-04-26T01:58:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.131+0000 7fb011644740 1 -- 172.21.15.151:0/2512347403 shutdown_connections 2024-04-26T01:58:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.131+0000 7fb011644740 1 --2- 172.21.15.151:0/2512347403 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561af4918b90 0x561af491af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.131+0000 7fb011644740 1 --2- 172.21.15.151:0/2512347403 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561af490e2d0 0x561af4918650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.131+0000 7fb011644740 1 --2- 172.21.15.151:0/2512347403 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561af490d9b0 0x561af490dd90 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.131+0000 7fb011644740 1 -- 172.21.15.151:0/2512347403 >> 172.21.15.151:0/2512347403 conn(0x561af4909ec0 msgr2=0x561af490a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.131+0000 7fb011644740 1 -- 172.21.15.151:0/2512347403 shutdown_connections 2024-04-26T01:58:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.131+0000 7fb011644740 1 -- 172.21.15.151:0/2512347403 wait complete. 2024-04-26T01:58:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.131+0000 7fb007400640 1 --2- 172.21.15.151:0/1632712541 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fafec0a3d40 0x7fafec0a6200 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x561af490ef50 tx=0x7faff8004a90 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.131+0000 7fb011644740 1 -- start start 2024-04-26T01:58:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.132+0000 7fb011644740 1 Processor -- start 2024-04-26T01:58:19.136 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.132+0000 7fb011644740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561af490d9b0 0x561af4915b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:19.136 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.132+0000 7fb011644740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561af490e2d0 0x561af49160d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:19.136 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.132+0000 7fb011644740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561af4918b90 0x561af4916610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:19.136 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.132+0000 7fb011644740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x561af491b7c0 con 0x561af490d9b0 2024-04-26T01:58:19.136 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.133+0000 7fb011644740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561af482d310 con 0x561af4918b90 2024-04-26T01:58:19.136 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.133+0000 7fb011644740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561af491b940 con 0x561af490e2d0 2024-04-26T01:58:19.136 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.133+0000 7fb011644740 10 client.0 _ll_register_callbacks cb 0x561af47fbcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:58:19.136 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.133+0000 7fb006a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561af490e2d0 0x561af49160d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:19.136 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.133+0000 7fb007e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561af490d9b0 0x561af4915b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.133+0000 7fb006a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561af490e2d0 0x561af49160d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:53556/0 (socket says 172.21.15.151:53556) 2024-04-26T01:58:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.133+0000 7fb007e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561af490d9b0 0x561af4915b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:48224/0 (socket says 172.21.15.151:48224) 2024-04-26T01:58:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.133+0000 7fb006a00640 1 -- 172.21.15.151:0/2955019234 learned_addr learned my addr 172.21.15.151:0/2955019234 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:58:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.133+0000 7fb007e00640 1 --2- 172.21.15.151:0/2955019234 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561af4918b90 0x561af4916610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.133+0000 7fb006a00640 1 -- 172.21.15.151:0/2955019234 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561af4918b90 msgr2=0x561af4916610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.133+0000 7fb006a00640 1 --2- 172.21.15.151:0/2955019234 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561af4918b90 0x561af4916610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.133+0000 7fb006a00640 1 -- 172.21.15.151:0/2955019234 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561af490d9b0 msgr2=0x561af4915b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.133+0000 7fb006a00640 1 --2- 172.21.15.151:0/2955019234 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561af490d9b0 0x561af4915b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.133+0000 7fb006a00640 1 -- 172.21.15.151:0/2955019234 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561af4652620 con 0x561af490e2d0 2024-04-26T01:58:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.133+0000 7fb007e00640 1 --2- 172.21.15.151:0/2955019234 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561af4918b90 0x561af4916610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:58:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.134+0000 7fb006a00640 1 --2- 172.21.15.151:0/2955019234 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561af490e2d0 0x561af49160d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7faff4009230 tx=0x7faff4004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd660d4ef852154d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.134+0000 7faff1600640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-26T01:58:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.134+0000 7faff1600640 1 -- 172.21.15.151:0/2955019234 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff4035040 con 0x561af490e2d0 2024-04-26T01:58:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.134+0000 7faff1600640 1 -- 172.21.15.151:0/2955019234 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7faff4005240 con 0x561af490e2d0 2024-04-26T01:58:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.134+0000 7fb011644740 1 -- 172.21.15.151:0/2955019234 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb000004d20 con 0x561af490e2d0 2024-04-26T01:58:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.134+0000 7faff1600640 1 -- 172.21.15.151:0/2955019234 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7faff4005440 con 0x561af490e2d0 2024-04-26T01:58:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.134+0000 7fb011644740 10 client.25744 fetch_fsmap learned FSMap version 30 2024-04-26T01:58:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.134+0000 7fb011644740 1 -- 172.21.15.151:0/2955019234 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561af4927a80 con 0x561af490e2d0 2024-04-26T01:58:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.135+0000 7faff1600640 1 -- 172.21.15.151:0/2955019234 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb000004d20 con 0x561af490e2d0 2024-04-26T01:58:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.135+0000 7faff1600640 1 -- 172.21.15.151:0/2955019234 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7faff404c050 con 0x561af490e2d0 2024-04-26T01:58:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.135+0000 7fb011644740 10 client.25744 fetch_fsmap finished waiting for FSMap version 30 2024-04-26T01:58:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.135+0000 7fb011644740 10 client.25744 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:58:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.135+0000 7fb011644740 20 client.25744 populate_metadata read hostname 'smithi151' 2024-04-26T01:58:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.135+0000 7fb011644740 1 --2- 172.21.15.151:0/2955019234 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x561af4928ef0 0x561af492b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.135+0000 7fb011644740 4 client.25744 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:58:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.135+0000 7fb011644740 1 -- 172.21.15.151:0/2955019234 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561af48878b0 con 0x561af4928ef0 2024-04-26T01:58:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.135+0000 7fb007e00640 1 --2- 172.21.15.151:0/2955019234 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x561af4928ef0 0x561af492b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.136+0000 7faff1600640 10 client.25744 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:58:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.136+0000 7fb007e00640 1 --2- 172.21.15.151:0/2955019234 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x561af4928ef0 0x561af492b2e0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=760951eae39eb040 server_cookie=1d64d148163cf843 in_seq=0 out_seq=0 2024-04-26T01:58:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.136+0000 7faff1600640 1 -- 172.21.15.151:0/2955019234 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561af48878b0 con 0x561af4928ef0 2024-04-26T01:58:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.136+0000 7faff1600640 10 client.25744 handle_command_reply: tid=0 2024-04-26T01:58:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.136+0000 7fb011644740 1 client.25744 shutdown 2024-04-26T01:58:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.137+0000 7fb011644740 1 -- 172.21.15.151:0/2955019234 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561af490e2d0 msgr2=0x561af49160d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.137+0000 7fb011644740 1 --2- 172.21.15.151:0/2955019234 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561af490e2d0 0x561af49160d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7faff4009230 tx=0x7faff4004270 comp rx=0 tx=0).stop 2024-04-26T01:58:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.137+0000 7fb011644740 1 -- 172.21.15.151:0/2955019234 shutdown_connections 2024-04-26T01:58:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.137+0000 7fb011644740 1 --2- 172.21.15.151:0/2955019234 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x561af4928ef0 0x561af492b2e0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.137+0000 7fb011644740 1 --2- 172.21.15.151:0/2955019234 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561af4918b90 0x561af4916610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.137+0000 7fb011644740 1 --2- 172.21.15.151:0/2955019234 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561af490e2d0 0x561af49160d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.137+0000 7fb011644740 1 --2- 172.21.15.151:0/2955019234 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561af490d9b0 0x561af4915b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.137+0000 7faff1600640 0 client.25744 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:58:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.137+0000 7fb011644740 1 -- 172.21.15.151:0/2955019234 >> 172.21.15.151:0/2955019234 conn(0x561af4909ec0 msgr2=0x561af490fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:19.140 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.137+0000 7fb011644740 1 -- 172.21.15.151:0/2955019234 shutdown_connections 2024-04-26T01:58:19.140 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.137+0000 7fb011644740 1 -- 172.21.15.151:0/2955019234 wait complete. 2024-04-26T01:58:19.140 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.137+0000 7fb011644740 20 client.25744 trim_cache size 0 max 16384 2024-04-26T01:58:19.152 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.153+0000 7fb011644740 1 Processor -- start 2024-04-26T01:58:19.152 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.153+0000 7fb011644740 1 -- start start 2024-04-26T01:58:19.152 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.154+0000 7fb011644740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561af490d9b0 0x561af49097a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.154+0000 7fb011644740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561af490e2d0 0x561af4910370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.154+0000 7fb011644740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561af4918b90 0x561af4917ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.154+0000 7fb011644740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb0081b2360 con 0x561af4918b90 2024-04-26T01:58:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.154+0000 7fb011644740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb0081afc30 con 0x561af490d9b0 2024-04-26T01:58:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.154+0000 7fb011644740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb0081ada80 con 0x561af490e2d0 2024-04-26T01:58:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.154+0000 7fb006a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561af490e2d0 0x561af4910370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.154+0000 7fb006a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561af490e2d0 0x561af4910370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:53558/0 (socket says 172.21.15.151:53558) 2024-04-26T01:58:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.154+0000 7fb006a00640 1 -- 172.21.15.151:0/3909742619 learned_addr learned my addr 172.21.15.151:0/3909742619 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:58:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.154+0000 7fb007e00640 1 --2- 172.21.15.151:0/3909742619 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561af490d9b0 0x561af49097a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.154+0000 7fb007e00640 1 --2- 172.21.15.151:0/3909742619 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561af4918b90 0x561af4917ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.155+0000 7fb006a00640 1 -- 172.21.15.151:0/3909742619 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561af490d9b0 msgr2=0x561af49097a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.155+0000 7fb006a00640 1 --2- 172.21.15.151:0/3909742619 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561af490d9b0 0x561af49097a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.155+0000 7fb006a00640 1 -- 172.21.15.151:0/3909742619 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561af4918b90 msgr2=0x561af4917ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.155+0000 7fb006a00640 1 --2- 172.21.15.151:0/3909742619 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561af4918b90 0x561af4917ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.155+0000 7fb006a00640 1 -- 172.21.15.151:0/3909742619 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561af4909ce0 con 0x561af490e2d0 2024-04-26T01:58:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.155+0000 7fb007e00640 1 --2- 172.21.15.151:0/3909742619 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561af4918b90 0x561af4917ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:58:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.155+0000 7fb006a00640 1 --2- 172.21.15.151:0/3909742619 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561af490e2d0 0x561af4910370 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7faff4003a80 tx=0x7faff402a490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac7d8e3f9c185afa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.155+0000 7faff3e00640 1 -- 172.21.15.151:0/3909742619 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff4035030 con 0x561af490e2d0 2024-04-26T01:58:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.155+0000 7faff3e00640 1 -- 172.21.15.151:0/3909742619 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faff4005850 con 0x561af490e2d0 2024-04-26T01:58:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.155+0000 7faff3e00640 1 -- 172.21.15.151:0/3909742619 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff403c440 con 0x561af490e2d0 2024-04-26T01:58:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.156+0000 7fb011644740 1 -- 172.21.15.151:0/3909742619 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561af490e2d0 msgr2=0x561af4910370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.156+0000 7fb011644740 1 --2- 172.21.15.151:0/3909742619 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561af490e2d0 0x561af4910370 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7faff4003a80 tx=0x7faff402a490 comp rx=0 tx=0).stop 2024-04-26T01:58:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.156+0000 7fb011644740 1 -- 172.21.15.151:0/3909742619 shutdown_connections 2024-04-26T01:58:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.156+0000 7fb011644740 1 --2- 172.21.15.151:0/3909742619 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561af4918b90 0x561af4917ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.156+0000 7fb011644740 1 --2- 172.21.15.151:0/3909742619 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561af490e2d0 0x561af4910370 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.156+0000 7fb011644740 1 --2- 172.21.15.151:0/3909742619 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561af490d9b0 0x561af49097a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.156+0000 7fb011644740 1 -- 172.21.15.151:0/3909742619 >> 172.21.15.151:0/3909742619 conn(0x561af4909ec0 msgr2=0x561af49173c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.156+0000 7fb011644740 1 -- 172.21.15.151:0/3909742619 shutdown_connections 2024-04-26T01:58:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.156+0000 7fb011644740 1 -- 172.21.15.151:0/3909742619 wait complete. 2024-04-26T01:58:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.157+0000 7fb011644740 1 -- start start 2024-04-26T01:58:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.157+0000 7fb011644740 1 Processor -- start 2024-04-26T01:58:19.159 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.157+0000 7fb011644740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561af490d9b0 0x561af4920cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:19.159 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.157+0000 7fb011644740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561af490e2d0 0x561af491e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:19.159 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.157+0000 7fb011644740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561af4918b90 0x561af491e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:19.159 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.157+0000 7fb011644740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb0081ae0f0 con 0x561af4918b90 2024-04-26T01:58:19.159 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.157+0000 7fb011644740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb0081af560 con 0x561af490e2d0 2024-04-26T01:58:19.159 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.157+0000 7fb011644740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb0081ae7c0 con 0x561af490d9b0 2024-04-26T01:58:19.159 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.158+0000 7fb011644740 10 client.0 _ll_register_callbacks cb 0x7faff4009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:58:19.159 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.158+0000 7fb006a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561af490e2d0 0x561af491e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:19.159 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.158+0000 7fb006a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561af490e2d0 0x561af491e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:47984/0 (socket says 172.21.15.151:47984) 2024-04-26T01:58:19.159 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.158+0000 7fb006a00640 1 -- 172.21.15.151:0/4284376357 learned_addr learned my addr 172.21.15.151:0/4284376357 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:58:19.159 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.158+0000 7fb007e00640 1 --2- 172.21.15.151:0/4284376357 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561af490d9b0 0x561af4920cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:19.159 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.158+0000 7fb007e00640 1 --2- 172.21.15.151:0/4284376357 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561af4918b90 0x561af491e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:19.159 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.158+0000 7fb006a00640 1 -- 172.21.15.151:0/4284376357 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561af490d9b0 msgr2=0x561af4920cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:19.160 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.158+0000 7fb006a00640 1 --2- 172.21.15.151:0/4284376357 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561af490d9b0 0x561af4920cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:19.160 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.158+0000 7fb006a00640 1 -- 172.21.15.151:0/4284376357 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561af4918b90 msgr2=0x561af491e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:19.160 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.158+0000 7fb006a00640 1 --2- 172.21.15.151:0/4284376357 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561af4918b90 0x561af491e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:19.160 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.158+0000 7fb006a00640 1 -- 172.21.15.151:0/4284376357 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561af491ec10 con 0x561af490e2d0 2024-04-26T01:58:19.160 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.158+0000 7fb007e00640 1 --2- 172.21.15.151:0/4284376357 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561af4918b90 0x561af491e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:58:19.160 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.158+0000 7fb006a00640 1 --2- 172.21.15.151:0/4284376357 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561af490e2d0 0x561af491e160 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7faff4003a00 tx=0x7faff4004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a96538c7c3765d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:19.160 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.158+0000 7faff1600640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T01:58:19.160 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.159+0000 7faff1600640 1 -- 172.21.15.151:0/4284376357 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff4035040 con 0x561af490e2d0 2024-04-26T01:58:19.160 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.159+0000 7faff1600640 1 -- 172.21.15.151:0/4284376357 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7faff40314a0 con 0x561af490e2d0 2024-04-26T01:58:19.160 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.159+0000 7fb011644740 1 -- 172.21.15.151:0/4284376357 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faff4005850 con 0x561af490e2d0 2024-04-26T01:58:19.160 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.159+0000 7faff1600640 1 -- 172.21.15.151:0/4284376357 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7faff40316a0 con 0x561af490e2d0 2024-04-26T01:58:19.160 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.159+0000 7fb011644740 10 client.25715 fetch_fsmap learned FSMap version 30 2024-04-26T01:58:19.160 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.159+0000 7fb011644740 1 -- 172.21.15.151:0/4284376357 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561af498f190 con 0x561af490e2d0 2024-04-26T01:58:19.161 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.159+0000 7faff1600640 1 -- 172.21.15.151:0/4284376357 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faff4005850 con 0x561af490e2d0 2024-04-26T01:58:19.161 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.160+0000 7faff1600640 1 -- 172.21.15.151:0/4284376357 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7faff4066050 con 0x561af490e2d0 2024-04-26T01:58:19.162 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.160+0000 7fb011644740 10 client.25715 fetch_fsmap finished waiting for FSMap version 30 2024-04-26T01:58:19.162 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.160+0000 7fb011644740 10 client.25715 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:58:19.162 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.160+0000 7fb011644740 20 client.25715 populate_metadata read hostname 'smithi151' 2024-04-26T01:58:19.162 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.160+0000 7fb011644740 1 --2- 172.21.15.151:0/4284376357 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x561af498f800 0x561af4991c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:19.162 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.160+0000 7fb011644740 4 client.25715 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:58:19.162 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.160+0000 7fb011644740 1 -- 172.21.15.151:0/4284376357 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561af4881320 con 0x561af498f800 2024-04-26T01:58:19.162 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.160+0000 7fb007e00640 1 --2- 172.21.15.151:0/4284376357 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x561af498f800 0x561af4991c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:19.162 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.161+0000 7fb007e00640 1 --2- 172.21.15.151:0/4284376357 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x561af498f800 0x561af4991c20 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6759ca34d29680d server_cookie=38773daad70711a6 in_seq=0 out_seq=0 2024-04-26T01:58:19.162 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.161+0000 7faff1600640 10 client.25715 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:58:19.175 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.177+0000 7faff1600640 1 -- 172.21.15.151:0/4284376357 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561af4881320 con 0x561af498f800 2024-04-26T01:58:19.176 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.177+0000 7faff1600640 10 client.25715 handle_command_reply: tid=0 2024-04-26T01:58:19.176 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.177+0000 7fb011644740 1 client.25715 shutdown 2024-04-26T01:58:19.176 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.177+0000 7fb011644740 1 -- 172.21.15.151:0/4284376357 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561af490e2d0 msgr2=0x561af491e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:19.176 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.177+0000 7fb011644740 1 --2- 172.21.15.151:0/4284376357 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561af490e2d0 0x561af491e160 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7faff4003a00 tx=0x7faff4004270 comp rx=0 tx=0).stop 2024-04-26T01:58:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.177+0000 7fb011644740 1 -- 172.21.15.151:0/4284376357 shutdown_connections 2024-04-26T01:58:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.177+0000 7fb011644740 1 --2- 172.21.15.151:0/4284376357 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x561af498f800 0x561af4991c20 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.177+0000 7fb011644740 1 --2- 172.21.15.151:0/4284376357 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561af4918b90 0x561af491e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.177+0000 7fb011644740 1 --2- 172.21.15.151:0/4284376357 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561af490e2d0 0x561af491e160 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.177+0000 7fb011644740 1 --2- 172.21.15.151:0/4284376357 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561af490d9b0 0x561af4920cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.177+0000 7faff1600640 0 client.25715 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:58:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.177+0000 7fb011644740 1 -- 172.21.15.151:0/4284376357 >> 172.21.15.151:0/4284376357 conn(0x561af4909ec0 msgr2=0x561af49173c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.177+0000 7fb011644740 1 -- 172.21.15.151:0/4284376357 shutdown_connections 2024-04-26T01:58:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.178+0000 7fb011644740 1 -- 172.21.15.151:0/4284376357 wait complete. 2024-04-26T01:58:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.178+0000 7fb011644740 20 client.25715 trim_cache size 0 max 16384 2024-04-26T01:58:19.178 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:{ 2024-04-26T01:58:19.178 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "message": "", 2024-04-26T01:58:19.178 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "return_code": 0 2024-04-26T01:58:19.178 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:} 2024-04-26T01:58:19.178 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.179+0000 7fb00d200640 1 -- 172.21.15.151:0/1632712541 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fafec0a3d40 msgr2=0x7fafec0a6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:19.178 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.179+0000 7fb00d200640 1 --2- 172.21.15.151:0/1632712541 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fafec0a3d40 0x7fafec0a6200 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x561af490ef50 tx=0x7faff8004a90 comp rx=0 tx=0).stop 2024-04-26T01:58:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.179+0000 7fb00d200640 1 -- 172.21.15.151:0/1632712541 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb008102f10 msgr2=0x7fb008100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.179+0000 7fb00d200640 1 --2- 172.21.15.151:0/1632712541 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb008102f10 0x7fb008100b60 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7faff800e9f0 tx=0x7faff800edf0 comp rx=0 tx=0).stop 2024-04-26T01:58:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.179+0000 7fb00d200640 1 -- 172.21.15.151:0/1632712541 shutdown_connections 2024-04-26T01:58:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.179+0000 7fb00d200640 1 --2- 172.21.15.151:0/1632712541 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fafec0a3d40 0x7fafec0a6200 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.179+0000 7fb00d200640 1 --2- 172.21.15.151:0/1632712541 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb00810c330 0x7fb0081015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.179+0000 7fb00d200640 1 --2- 172.21.15.151:0/1632712541 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb008103830 0x7fb0081010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.179+0000 7fb00d200640 1 --2- 172.21.15.151:0/1632712541 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb008102f10 0x7fb008100b60 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.179+0000 7fb00d200640 1 -- 172.21.15.151:0/1632712541 >> 172.21.15.151:0/1632712541 conn(0x7fb0080fc770 msgr2=0x7fb0080fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.179+0000 7fb00d200640 1 -- 172.21.15.151:0/1632712541 shutdown_connections 2024-04-26T01:58:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:19.179+0000 7fb00d200640 1 -- 172.21.15.151:0/1632712541 wait complete. 2024-04-26T01:58:20.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:58:20 smithi172 ceph-mon[32785]: pgmap v144: 97 pgs: 97 active+clean; 40 MiB data, 723 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 411 op/s 2024-04-26T01:58:20.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:58:20 smithi165 ceph-mon[32648]: pgmap v144: 97 pgs: 97 active+clean; 40 MiB data, 723 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 411 op/s 2024-04-26T01:58:20.853 INFO:tasks.workunit.client.0.smithi151.stdout:copying to largest ever: 0xffd97 2024-04-26T01:58:20.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:58:20 smithi151 ceph-mon[28057]: pgmap v144: 97 pgs: 97 active+clean; 40 MiB data, 723 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 411 op/s 2024-04-26T01:58:22.786 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:58:22 smithi172 ceph-mon[32785]: pgmap v145: 97 pgs: 97 active+clean; 40 MiB data, 723 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 272 op/s 2024-04-26T01:58:22.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:58:22 smithi165 ceph-mon[32648]: pgmap v145: 97 pgs: 97 active+clean; 40 MiB data, 723 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 272 op/s 2024-04-26T01:58:22.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:58:22 smithi151 ceph-mon[28057]: pgmap v145: 97 pgs: 97 active+clean; 40 MiB data, 723 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 272 op/s 2024-04-26T01:58:24.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:58:24 smithi172 ceph-mon[32785]: pgmap v146: 97 pgs: 97 active+clean; 41 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 284 op/s 2024-04-26T01:58:24.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:58:24 smithi165 ceph-mon[32648]: pgmap v146: 97 pgs: 97 active+clean; 41 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 284 op/s 2024-04-26T01:58:24.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:58:24 smithi151 ceph-mon[28057]: pgmap v146: 97 pgs: 97 active+clean; 41 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 284 op/s 2024-04-26T01:58:26.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:58:26 smithi172 ceph-mon[32785]: pgmap v147: 97 pgs: 97 active+clean; 38 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 403 op/s 2024-04-26T01:58:26.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:58:26 smithi165 ceph-mon[32648]: pgmap v147: 97 pgs: 97 active+clean; 38 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 403 op/s 2024-04-26T01:58:26.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:58:26 smithi151 ceph-mon[28057]: pgmap v147: 97 pgs: 97 active+clean; 38 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 403 op/s 2024-04-26T01:58:28.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:58:28 smithi172 ceph-mon[32785]: pgmap v148: 97 pgs: 97 active+clean; 38 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 271 op/s 2024-04-26T01:58:28.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:58:28 smithi165 ceph-mon[32648]: pgmap v148: 97 pgs: 97 active+clean; 38 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 271 op/s 2024-04-26T01:58:28.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:58:28 smithi151 ceph-mon[28057]: pgmap v148: 97 pgs: 97 active+clean; 38 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 271 op/s 2024-04-26T01:58:30.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:58:30 smithi172 ceph-mon[32785]: pgmap v149: 97 pgs: 97 active+clean; 40 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 377 op/s 2024-04-26T01:58:30.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:58:30 smithi165 ceph-mon[32648]: pgmap v149: 97 pgs: 97 active+clean; 40 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 377 op/s 2024-04-26T01:58:30.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:58:30 smithi151 ceph-mon[28057]: pgmap v149: 97 pgs: 97 active+clean; 40 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 377 op/s 2024-04-26T01:58:32.333 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.331+0000 7f112e400640 1 -- 172.21.15.151:0/160740116 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1128107260 msgr2=0x7f112810ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.331+0000 7f112e400640 1 --2- 172.21.15.151:0/160740116 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1128107260 0x7f112810ee50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1120009ca0 tx=0x7f112002a8c0 comp rx=0 tx=0).stop 2024-04-26T01:58:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.331+0000 7f112e400640 1 -- 172.21.15.151:0/160740116 shutdown_connections 2024-04-26T01:58:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.331+0000 7f112e400640 1 --2- 172.21.15.151:0/160740116 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1128107260 0x7f112810ee50 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.331+0000 7f112e400640 1 --2- 172.21.15.151:0/160740116 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f112806b3a0 0x7f1128106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.331+0000 7f112e400640 1 --2- 172.21.15.151:0/160740116 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f112806a9f0 0x7f112806add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.331+0000 7f112e400640 1 -- 172.21.15.151:0/160740116 >> 172.21.15.151:0/160740116 conn(0x7f1128076c50 msgr2=0x7f1128077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.331+0000 7f112e400640 1 -- 172.21.15.151:0/160740116 shutdown_connections 2024-04-26T01:58:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.331+0000 7f112e400640 1 -- 172.21.15.151:0/160740116 wait complete. 2024-04-26T01:58:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.332+0000 7f112e400640 1 Processor -- start 2024-04-26T01:58:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.332+0000 7f112e400640 1 -- start start 2024-04-26T01:58:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.333+0000 7f112e400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f112806a9f0 0x7f1128101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.333+0000 7f112e400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1128107260 0x7f11281017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.333+0000 7f112e400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1128101cf0 0x7f11281a6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.333+0000 7f112e400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1128112230 con 0x7f112806a9f0 2024-04-26T01:58:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.333+0000 7f112e400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f11281120b0 con 0x7f1128101cf0 2024-04-26T01:58:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.333+0000 7f112e400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f11281123b0 con 0x7f1128107260 2024-04-26T01:58:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.333+0000 7f1126a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1128107260 0x7f11281017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.333+0000 7f1127400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f112806a9f0 0x7f1128101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.333+0000 7f1126a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1128107260 0x7f11281017b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:36780/0 (socket says 172.21.15.151:36780) 2024-04-26T01:58:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.333+0000 7f1126a00640 1 -- 172.21.15.151:0/1605730403 learned_addr learned my addr 172.21.15.151:0/1605730403 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:58:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.333+0000 7f1127e00640 1 --2- 172.21.15.151:0/1605730403 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1128101cf0 0x7f11281a6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.334+0000 7f1126a00640 1 -- 172.21.15.151:0/1605730403 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1128101cf0 msgr2=0x7f11281a6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.334+0000 7f1126a00640 1 --2- 172.21.15.151:0/1605730403 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1128101cf0 0x7f11281a6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.334+0000 7f1126a00640 1 -- 172.21.15.151:0/1605730403 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f112806a9f0 msgr2=0x7f1128101270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.334+0000 7f1126a00640 1 --2- 172.21.15.151:0/1605730403 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f112806a9f0 0x7f1128101270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.334+0000 7f1126a00640 1 -- 172.21.15.151:0/1605730403 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11281a6ee0 con 0x7f1128107260 2024-04-26T01:58:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.334+0000 7f1126a00640 1 --2- 172.21.15.151:0/1605730403 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1128107260 0x7f11281017b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f111400e9c0 tx=0x7f111400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.334+0000 7f111fe00640 1 -- 172.21.15.151:0/1605730403 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f111400cd80 con 0x7f1128107260 2024-04-26T01:58:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.334+0000 7f111fe00640 1 -- 172.21.15.151:0/1605730403 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11140054d0 con 0x7f1128107260 2024-04-26T01:58:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.334+0000 7f111fe00640 1 -- 172.21.15.151:0/1605730403 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1114010430 con 0x7f1128107260 2024-04-26T01:58:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.334+0000 7f112e400640 1 -- 172.21.15.151:0/1605730403 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11281a71d0 con 0x7f1128107260 2024-04-26T01:58:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.335+0000 7f112e400640 1 -- 172.21.15.151:0/1605730403 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11281a7720 con 0x7f1128107260 2024-04-26T01:58:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.340+0000 7f1131273740 1 Processor -- start 2024-04-26T01:58:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.340+0000 7f1131273740 1 -- start start 2024-04-26T01:58:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.340+0000 7f111fe00640 1 -- 172.21.15.151:0/1605730403 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f1114004fd0 con 0x7f1128107260 2024-04-26T01:58:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.340+0000 7f1131273740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558be1caa820 0x558be1caac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.340+0000 7f1131273740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558be1cab140 0x558be1cb54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.340+0000 7f1131273740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558be1cb5a00 0x558be1cb7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.340+0000 7f1131273740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558be1c10900 con 0x558be1caa820 2024-04-26T01:58:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.340+0000 7f1131273740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558be1bb6ef0 con 0x558be1cb5a00 2024-04-26T01:58:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.340+0000 7f1131273740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558be1c0c8c0 con 0x558be1cab140 2024-04-26T01:58:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.341+0000 7f1127400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558be1cab140 0x558be1cb54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:32.341 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.341+0000 7f1127400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558be1cab140 0x558be1cb54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:36796/0 (socket says 172.21.15.151:36796) 2024-04-26T01:58:32.341 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.341+0000 7f1127400640 1 -- 172.21.15.151:0/2878699063 learned_addr learned my addr 172.21.15.151:0/2878699063 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:58:32.341 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.341+0000 7f1127e00640 1 --2- 172.21.15.151:0/2878699063 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558be1caa820 0x558be1caac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:32.341 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.341+0000 7f1126a00640 1 --2- 172.21.15.151:0/2878699063 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558be1cb5a00 0x558be1cb7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:32.341 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.341+0000 7f1127400640 1 -- 172.21.15.151:0/2878699063 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558be1cb5a00 msgr2=0x558be1cb7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:32.341 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.341+0000 7f1127400640 1 --2- 172.21.15.151:0/2878699063 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558be1cb5a00 0x558be1cb7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:32.341 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.341+0000 7f1127400640 1 -- 172.21.15.151:0/2878699063 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558be1caa820 msgr2=0x558be1caac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:32.341 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.341+0000 7f1127400640 1 --2- 172.21.15.151:0/2878699063 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558be1caa820 0x558be1caac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:32.341 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.341+0000 7f1127400640 1 -- 172.21.15.151:0/2878699063 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558be1923210 con 0x558be1cab140 2024-04-26T01:58:32.341 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.341+0000 7f1127e00640 1 --2- 172.21.15.151:0/2878699063 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558be1caa820 0x558be1caac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:58:32.341 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.341+0000 7f1127400640 1 --2- 172.21.15.151:0/2878699063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558be1cab140 0x558be1cb54c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1118009cf0 tx=0x7f1118011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd22060394ec985a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:32.345 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.342+0000 7f111cc00640 1 -- 172.21.15.151:0/2878699063 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1118004230 con 0x558be1cab140 2024-04-26T01:58:32.346 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.342+0000 7f111cc00640 1 -- 172.21.15.151:0/2878699063 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11180043d0 con 0x558be1cab140 2024-04-26T01:58:32.346 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.342+0000 7f111cc00640 1 -- 172.21.15.151:0/2878699063 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1118019880 con 0x558be1cab140 2024-04-26T01:58:32.346 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.342+0000 7f111fe00640 1 --2- 172.21.15.151:0/1605730403 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f110c0a3dc0 0x7f110c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:32.346 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.342+0000 7f111fe00640 1 -- 172.21.15.151:0/1605730403 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f11140d0b70 con 0x7f1128107260 2024-04-26T01:58:32.346 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.342+0000 7f1127e00640 1 --2- 172.21.15.151:0/1605730403 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f110c0a3dc0 0x7f110c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:32.346 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.343+0000 7f1131273740 1 -- 172.21.15.151:0/2878699063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558be1cab140 msgr2=0x558be1cb54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:32.346 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.343+0000 7f1131273740 1 --2- 172.21.15.151:0/2878699063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558be1cab140 0x558be1cb54c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1118009cf0 tx=0x7f1118011810 comp rx=0 tx=0).stop 2024-04-26T01:58:32.346 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.343+0000 7f1131273740 1 -- 172.21.15.151:0/2878699063 shutdown_connections 2024-04-26T01:58:32.346 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.343+0000 7f1131273740 1 --2- 172.21.15.151:0/2878699063 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558be1cb5a00 0x558be1cb7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:32.346 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.343+0000 7f1131273740 1 --2- 172.21.15.151:0/2878699063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558be1cab140 0x558be1cb54c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.343+0000 7f1131273740 1 --2- 172.21.15.151:0/2878699063 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558be1caa820 0x558be1caac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.343+0000 7f1131273740 1 -- 172.21.15.151:0/2878699063 >> 172.21.15.151:0/2878699063 conn(0x558be1ca6d30 msgr2=0x558be1ca7110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.343+0000 7f1131273740 1 -- 172.21.15.151:0/2878699063 shutdown_connections 2024-04-26T01:58:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.343+0000 7f1131273740 1 -- 172.21.15.151:0/2878699063 wait complete. 2024-04-26T01:58:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.343+0000 7f1127e00640 1 --2- 172.21.15.151:0/1605730403 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f110c0a3dc0 0x7f110c0a6280 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1120006010 tx=0x7f1120005d80 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.344+0000 7f1131273740 1 -- start start 2024-04-26T01:58:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.344+0000 7f1131273740 1 Processor -- start 2024-04-26T01:58:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.344+0000 7f1131273740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558be1caa820 0x558be1cb2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.344+0000 7f1131273740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558be1cab140 0x558be1cb2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.344+0000 7f1131273740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558be1cb5a00 0x558be1cb33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.344+0000 7f1131273740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558be1cb8630 con 0x558be1cb5a00 2024-04-26T01:58:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.344+0000 7f1131273740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558be1bca310 con 0x558be1cab140 2024-04-26T01:58:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.344+0000 7f1131273740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558be1cb87b0 con 0x558be1caa820 2024-04-26T01:58:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.344+0000 7f1131273740 10 client.0 _ll_register_callbacks cb 0x558be1b98cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:58:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.345+0000 7f1126a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558be1cab140 0x558be1cb2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.345+0000 7f1126a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558be1cab140 0x558be1cb2e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:40512/0 (socket says 172.21.15.151:40512) 2024-04-26T01:58:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.345+0000 7f1127400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558be1caa820 0x558be1cb2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.345+0000 7f1126a00640 1 -- 172.21.15.151:0/954775595 learned_addr learned my addr 172.21.15.151:0/954775595 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:58:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.345+0000 7f1127e00640 1 --2- 172.21.15.151:0/954775595 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558be1cb5a00 0x558be1cb33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.345+0000 7f1126a00640 1 -- 172.21.15.151:0/954775595 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558be1caa820 msgr2=0x558be1cb2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.345+0000 7f1126a00640 1 --2- 172.21.15.151:0/954775595 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558be1caa820 0x558be1cb2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.345+0000 7f1126a00640 1 -- 172.21.15.151:0/954775595 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558be1cb5a00 msgr2=0x558be1cb33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.345+0000 7f1126a00640 1 --2- 172.21.15.151:0/954775595 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558be1cb5a00 0x558be1cb33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.345+0000 7f1126a00640 1 -- 172.21.15.151:0/954775595 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558be19ef620 con 0x558be1cab140 2024-04-26T01:58:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.345+0000 7f1127e00640 1 --2- 172.21.15.151:0/954775595 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558be1cb5a00 0x558be1cb33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:58:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.345+0000 7f1126a00640 1 --2- 172.21.15.151:0/954775595 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558be1cab140 0x558be1cb2e60 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f111409c880 tx=0x7f11140d42b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdfc6cff16c35f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.345+0000 7f1111600640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T01:58:32.349 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.345+0000 7f1111600640 1 -- 172.21.15.151:0/954775595 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f111409c6c0 con 0x558be1cab140 2024-04-26T01:58:32.349 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.345+0000 7f1111600640 1 -- 172.21.15.151:0/954775595 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f11140bd0f0 con 0x558be1cab140 2024-04-26T01:58:32.349 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.345+0000 7f1131273740 1 -- 172.21.15.151:0/954775595 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11180043d0 con 0x558be1cab140 2024-04-26T01:58:32.349 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.346+0000 7f1111600640 1 -- 172.21.15.151:0/954775595 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f111409c4e0 con 0x558be1cab140 2024-04-26T01:58:32.349 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.346+0000 7f1131273740 10 client.25727 fetch_fsmap learned FSMap version 30 2024-04-26T01:58:32.349 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.346+0000 7f1131273740 1 -- 172.21.15.151:0/954775595 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558be1cc4830 con 0x558be1cab140 2024-04-26T01:58:32.349 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.346+0000 7f1111600640 1 -- 172.21.15.151:0/954775595 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11180043d0 con 0x558be1cab140 2024-04-26T01:58:32.349 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.346+0000 7f1111600640 1 -- 172.21.15.151:0/954775595 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f11140ce5b0 con 0x558be1cab140 2024-04-26T01:58:32.349 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.346+0000 7f1131273740 10 client.25727 fetch_fsmap finished waiting for FSMap version 30 2024-04-26T01:58:32.349 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.346+0000 7f1131273740 10 client.25727 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:58:32.349 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.346+0000 7f1131273740 20 client.25727 populate_metadata read hostname 'smithi151' 2024-04-26T01:58:32.349 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.346+0000 7f1131273740 1 --2- 172.21.15.151:0/954775595 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x558be1cc5c90 0x558be1cc80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:32.349 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.346+0000 7f1131273740 4 client.25727 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:58:32.350 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.346+0000 7f1131273740 1 -- 172.21.15.151:0/954775595 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558be1c248b0 con 0x558be1cc5c90 2024-04-26T01:58:32.350 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.347+0000 7f1127400640 1 --2- 172.21.15.151:0/954775595 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x558be1cc5c90 0x558be1cc80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:32.350 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.347+0000 7f1127400640 1 --2- 172.21.15.151:0/954775595 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x558be1cc5c90 0x558be1cc80a0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2951186e70f2491f server_cookie=3b214bc5a166f930 in_seq=0 out_seq=0 2024-04-26T01:58:32.350 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.347+0000 7f1111600640 10 client.25727 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:58:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.348+0000 7f1111600640 1 -- 172.21.15.151:0/954775595 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558be1c248b0 con 0x558be1cc5c90 2024-04-26T01:58:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.348+0000 7f1111600640 10 client.25727 handle_command_reply: tid=0 2024-04-26T01:58:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.348+0000 7f1131273740 1 client.25727 shutdown 2024-04-26T01:58:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.349+0000 7f1131273740 1 -- 172.21.15.151:0/954775595 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558be1cab140 msgr2=0x558be1cb2e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.349+0000 7f1131273740 1 --2- 172.21.15.151:0/954775595 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558be1cab140 0x558be1cb2e60 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f111409c880 tx=0x7f11140d42b0 comp rx=0 tx=0).stop 2024-04-26T01:58:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.349+0000 7f1131273740 1 -- 172.21.15.151:0/954775595 shutdown_connections 2024-04-26T01:58:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.349+0000 7f1131273740 1 --2- 172.21.15.151:0/954775595 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x558be1cc5c90 0x558be1cc80a0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.349+0000 7f1131273740 1 --2- 172.21.15.151:0/954775595 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558be1cb5a00 0x558be1cb33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.349+0000 7f1131273740 1 --2- 172.21.15.151:0/954775595 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558be1cab140 0x558be1cb2e60 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.349+0000 7f1111600640 0 client.25727 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:58:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.349+0000 7f1131273740 1 --2- 172.21.15.151:0/954775595 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558be1caa820 0x558be1cb2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.349+0000 7f1131273740 1 -- 172.21.15.151:0/954775595 >> 172.21.15.151:0/954775595 conn(0x558be1ca6d30 msgr2=0x558be1cacab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.349+0000 7f1131273740 1 -- 172.21.15.151:0/954775595 shutdown_connections 2024-04-26T01:58:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.349+0000 7f1131273740 1 -- 172.21.15.151:0/954775595 wait complete. 2024-04-26T01:58:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.349+0000 7f1131273740 20 client.25727 trim_cache size 0 max 16384 2024-04-26T01:58:32.367 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.367+0000 7f1131273740 1 Processor -- start 2024-04-26T01:58:32.367 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.367+0000 7f1131273740 1 -- start start 2024-04-26T01:58:32.367 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.368+0000 7f1131273740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558be1caa820 0x558be1cc14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:32.367 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.368+0000 7f1131273740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558be1cab140 0x558be1ccd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:32.367 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.368+0000 7f1131273740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558be1cb5a00 0x558be1ccdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:32.367 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.368+0000 7f1131273740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f11281b23d0 con 0x558be1caa820 2024-04-26T01:58:32.367 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.368+0000 7f1131273740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f11281afca0 con 0x558be1cb5a00 2024-04-26T01:58:32.367 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.368+0000 7f1131273740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f11281adb30 con 0x558be1cab140 2024-04-26T01:58:32.367 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.368+0000 7f1126a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558be1cab140 0x558be1ccd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:32.367 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.368+0000 7f1127e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558be1cb5a00 0x558be1ccdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.368+0000 7f1127400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558be1caa820 0x558be1cc14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.368+0000 7f1126a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558be1cab140 0x558be1ccd9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:36810/0 (socket says 172.21.15.151:36810) 2024-04-26T01:58:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.368+0000 7f1126a00640 1 -- 172.21.15.151:0/705070925 learned_addr learned my addr 172.21.15.151:0/705070925 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:58:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.368+0000 7f1127e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558be1cb5a00 0x558be1ccdef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:40522/0 (socket says 172.21.15.151:40522) 2024-04-26T01:58:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.369+0000 7f1127e00640 1 -- 172.21.15.151:0/705070925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558be1cab140 msgr2=0x558be1ccd9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.369+0000 7f1127e00640 1 --2- 172.21.15.151:0/705070925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558be1cab140 0x558be1ccd9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.369+0000 7f1127e00640 1 -- 172.21.15.151:0/705070925 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558be1caa820 msgr2=0x558be1cc14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.369+0000 7f1127e00640 1 --2- 172.21.15.151:0/705070925 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558be1caa820 0x558be1cc14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.369+0000 7f1127e00640 1 -- 172.21.15.151:0/705070925 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558be1cae4f0 con 0x558be1cb5a00 2024-04-26T01:58:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.369+0000 7f1126a00640 1 --2- 172.21.15.151:0/705070925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558be1cab140 0x558be1ccd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:58:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.369+0000 7f1127e00640 1 --2- 172.21.15.151:0/705070925 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558be1cb5a00 0x558be1ccdef0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1120005e30 tx=0x7f1120041000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb540dc5a95c212c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.369+0000 7f1113e00640 1 -- 172.21.15.151:0/705070925 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1120003970 con 0x558be1cb5a00 2024-04-26T01:58:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.369+0000 7f1113e00640 1 -- 172.21.15.151:0/705070925 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1120036050 con 0x558be1cb5a00 2024-04-26T01:58:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.369+0000 7f1113e00640 1 -- 172.21.15.151:0/705070925 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1120004780 con 0x558be1cb5a00 2024-04-26T01:58:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.370+0000 7f1131273740 1 -- 172.21.15.151:0/705070925 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558be1cb5a00 msgr2=0x558be1ccdef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.370+0000 7f1131273740 1 --2- 172.21.15.151:0/705070925 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558be1cb5a00 0x558be1ccdef0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1120005e30 tx=0x7f1120041000 comp rx=0 tx=0).stop 2024-04-26T01:58:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.370+0000 7f1131273740 1 -- 172.21.15.151:0/705070925 shutdown_connections 2024-04-26T01:58:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.370+0000 7f1131273740 1 --2- 172.21.15.151:0/705070925 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558be1cb5a00 0x558be1ccdef0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.370+0000 7f1131273740 1 --2- 172.21.15.151:0/705070925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558be1cab140 0x558be1ccd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.370+0000 7f1131273740 1 --2- 172.21.15.151:0/705070925 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558be1caa820 0x558be1cc14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.370+0000 7f1131273740 1 -- 172.21.15.151:0/705070925 >> 172.21.15.151:0/705070925 conn(0x558be1ca6d30 msgr2=0x558be1ca7110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.370+0000 7f1131273740 1 -- 172.21.15.151:0/705070925 shutdown_connections 2024-04-26T01:58:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.370+0000 7f1131273740 1 -- 172.21.15.151:0/705070925 wait complete. 2024-04-26T01:58:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.371+0000 7f1131273740 1 -- start start 2024-04-26T01:58:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.371+0000 7f1131273740 1 Processor -- start 2024-04-26T01:58:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.371+0000 7f1131273740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558be1caa820 0x558be1cbde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.371+0000 7f1131273740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558be1cab140 0x558be1cbb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.371+0000 7f1131273740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558be1cb5a00 0x558be1cbb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.371+0000 7f1131273740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f11281ae180 con 0x558be1cb5a00 2024-04-26T01:58:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.371+0000 7f1131273740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f11281af5d0 con 0x558be1cab140 2024-04-26T01:58:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.371+0000 7f1131273740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f11281ae830 con 0x558be1caa820 2024-04-26T01:58:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.372+0000 7f1131273740 10 client.0 _ll_register_callbacks cb 0x7f11140048b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:58:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.372+0000 7f1126a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558be1cab140 0x558be1cbb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.372+0000 7f1127400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558be1caa820 0x558be1cbde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.372+0000 7f1126a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558be1cab140 0x558be1cbb1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:40528/0 (socket says 172.21.15.151:40528) 2024-04-26T01:58:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.372+0000 7f1126a00640 1 -- 172.21.15.151:0/404068686 learned_addr learned my addr 172.21.15.151:0/404068686 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:58:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.372+0000 7f1127e00640 1 --2- 172.21.15.151:0/404068686 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558be1cb5a00 0x558be1cbb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.372+0000 7f1127400640 1 -- 172.21.15.151:0/404068686 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558be1cab140 msgr2=0x558be1cbb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.372+0000 7f1127400640 1 --2- 172.21.15.151:0/404068686 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558be1cab140 0x558be1cbb1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.372+0000 7f1127400640 1 -- 172.21.15.151:0/404068686 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558be1cb5a00 msgr2=0x558be1cbb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.372+0000 7f1127400640 1 --2- 172.21.15.151:0/404068686 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558be1cb5a00 0x558be1cbb770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.372+0000 7f1127400640 1 -- 172.21.15.151:0/404068686 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558be1cbbfa0 con 0x558be1caa820 2024-04-26T01:58:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.372+0000 7f1126a00640 1 --2- 172.21.15.151:0/404068686 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558be1cab140 0x558be1cbb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:58:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.372+0000 7f1127400640 1 --2- 172.21.15.151:0/404068686 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558be1caa820 0x558be1cbde60 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1118011d10 tx=0x7f1118009640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=369d594dce2f75de server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.372+0000 7f1111600640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-26T01:58:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.373+0000 7f1111600640 1 -- 172.21.15.151:0/404068686 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1118003f80 con 0x558be1caa820 2024-04-26T01:58:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.373+0000 7f1131273740 1 -- 172.21.15.151:0/404068686 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1120036050 con 0x558be1caa820 2024-04-26T01:58:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.373+0000 7f1111600640 1 -- 172.21.15.151:0/404068686 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f1118004120 con 0x558be1caa820 2024-04-26T01:58:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.373+0000 7f1111600640 1 -- 172.21.15.151:0/404068686 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f111803c050 con 0x558be1caa820 2024-04-26T01:58:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.373+0000 7f1131273740 10 client.25768 fetch_fsmap learned FSMap version 30 2024-04-26T01:58:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.373+0000 7f1131273740 1 -- 172.21.15.151:0/404068686 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558be1d2c2e0 con 0x558be1caa820 2024-04-26T01:58:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.373+0000 7f1111600640 1 -- 172.21.15.151:0/404068686 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1120036050 con 0x558be1caa820 2024-04-26T01:58:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.373+0000 7f1111600640 1 -- 172.21.15.151:0/404068686 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f11180035f0 con 0x558be1caa820 2024-04-26T01:58:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.374+0000 7f1131273740 10 client.25768 fetch_fsmap finished waiting for FSMap version 30 2024-04-26T01:58:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.374+0000 7f1131273740 10 client.25768 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:58:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.374+0000 7f1131273740 20 client.25768 populate_metadata read hostname 'smithi151' 2024-04-26T01:58:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.374+0000 7f1131273740 1 --2- 172.21.15.151:0/404068686 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x558be1d2c870 0x558be1d2ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.374+0000 7f1131273740 4 client.25768 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:58:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.374+0000 7f1131273740 1 -- 172.21.15.151:0/404068686 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558be1c1e320 con 0x558be1d2c870 2024-04-26T01:58:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.374+0000 7f1126a00640 1 --2- 172.21.15.151:0/404068686 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x558be1d2c870 0x558be1d2ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.374+0000 7f1126a00640 1 --2- 172.21.15.151:0/404068686 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x558be1d2c870 0x558be1d2ec60 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ade29a80b8932a32 server_cookie=3611c1f63763ef84 in_seq=0 out_seq=0 2024-04-26T01:58:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.374+0000 7f1111600640 10 client.25768 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:58:32.392 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.392+0000 7f1111600640 1 -- 172.21.15.151:0/404068686 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558be1c1e320 con 0x558be1d2c870 2024-04-26T01:58:32.392 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.392+0000 7f1111600640 10 client.25768 handle_command_reply: tid=0 2024-04-26T01:58:32.392 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.392+0000 7f1131273740 1 client.25768 shutdown 2024-04-26T01:58:32.392 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.393+0000 7f1131273740 1 -- 172.21.15.151:0/404068686 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558be1caa820 msgr2=0x558be1cbde60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.393+0000 7f1131273740 1 --2- 172.21.15.151:0/404068686 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558be1caa820 0x558be1cbde60 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1118011d10 tx=0x7f1118009640 comp rx=0 tx=0).stop 2024-04-26T01:58:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.393+0000 7f1131273740 1 -- 172.21.15.151:0/404068686 shutdown_connections 2024-04-26T01:58:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.393+0000 7f1131273740 1 --2- 172.21.15.151:0/404068686 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x558be1d2c870 0x558be1d2ec60 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.393+0000 7f1131273740 1 --2- 172.21.15.151:0/404068686 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558be1cb5a00 0x558be1cbb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.393+0000 7f1131273740 1 --2- 172.21.15.151:0/404068686 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558be1cab140 0x558be1cbb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.393+0000 7f1111600640 0 client.25768 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:58:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.393+0000 7f1131273740 1 --2- 172.21.15.151:0/404068686 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558be1caa820 0x558be1cbde60 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.393+0000 7f1131273740 1 -- 172.21.15.151:0/404068686 >> 172.21.15.151:0/404068686 conn(0x558be1ca6d30 msgr2=0x558be1cb28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.393+0000 7f1131273740 1 -- 172.21.15.151:0/404068686 shutdown_connections 2024-04-26T01:58:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.393+0000 7f1131273740 1 -- 172.21.15.151:0/404068686 wait complete. 2024-04-26T01:58:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.393+0000 7f1131273740 20 client.25768 trim_cache size 0 max 16384 2024-04-26T01:58:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:{ 2024-04-26T01:58:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "message": "", 2024-04-26T01:58:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "return_code": 0 2024-04-26T01:58:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:} 2024-04-26T01:58:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.394+0000 7f112e400640 1 -- 172.21.15.151:0/1605730403 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f110c0a3dc0 msgr2=0x7f110c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.394+0000 7f112e400640 1 --2- 172.21.15.151:0/1605730403 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f110c0a3dc0 0x7f110c0a6280 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1120006010 tx=0x7f1120005d80 comp rx=0 tx=0).stop 2024-04-26T01:58:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.394+0000 7f112e400640 1 -- 172.21.15.151:0/1605730403 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1128107260 msgr2=0x7f11281017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.394+0000 7f112e400640 1 --2- 172.21.15.151:0/1605730403 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1128107260 0x7f11281017b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f111400e9c0 tx=0x7f111400ee80 comp rx=0 tx=0).stop 2024-04-26T01:58:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.394+0000 7f112e400640 1 -- 172.21.15.151:0/1605730403 shutdown_connections 2024-04-26T01:58:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.394+0000 7f112e400640 1 --2- 172.21.15.151:0/1605730403 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f110c0a3dc0 0x7f110c0a6280 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.394+0000 7f112e400640 1 --2- 172.21.15.151:0/1605730403 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1128101cf0 0x7f11281a6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.394+0000 7f112e400640 1 --2- 172.21.15.151:0/1605730403 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1128107260 0x7f11281017b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.394+0000 7f112e400640 1 --2- 172.21.15.151:0/1605730403 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f112806a9f0 0x7f1128101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.394+0000 7f112e400640 1 -- 172.21.15.151:0/1605730403 >> 172.21.15.151:0/1605730403 conn(0x7f1128076c50 msgr2=0x7f11280ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.394+0000 7f112e400640 1 -- 172.21.15.151:0/1605730403 shutdown_connections 2024-04-26T01:58:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:32.394+0000 7f112e400640 1 -- 172.21.15.151:0/1605730403 wait complete. 2024-04-26T01:58:32.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:58:32 smithi172 ceph-mon[32785]: pgmap v150: 97 pgs: 97 active+clean; 40 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 237 op/s 2024-04-26T01:58:32.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:58:32 smithi165 ceph-mon[32648]: pgmap v150: 97 pgs: 97 active+clean; 40 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 237 op/s 2024-04-26T01:58:32.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:58:32 smithi151 ceph-mon[28057]: pgmap v150: 97 pgs: 97 active+clean; 40 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 237 op/s 2024-04-26T01:58:34.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:58:34 smithi172 ceph-mon[32785]: pgmap v151: 97 pgs: 97 active+clean; 41 MiB data, 758 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 274 op/s 2024-04-26T01:58:34.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:58:34 smithi165 ceph-mon[32648]: pgmap v151: 97 pgs: 97 active+clean; 41 MiB data, 758 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 274 op/s 2024-04-26T01:58:34.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:58:34 smithi151 ceph-mon[28057]: pgmap v151: 97 pgs: 97 active+clean; 41 MiB data, 758 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 274 op/s 2024-04-26T01:58:36.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:58:36 smithi172 ceph-mon[32785]: pgmap v152: 97 pgs: 97 active+clean; 37 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 334 op/s 2024-04-26T01:58:36.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:58:36 smithi165 ceph-mon[32648]: pgmap v152: 97 pgs: 97 active+clean; 37 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 334 op/s 2024-04-26T01:58:36.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:58:36 smithi151 ceph-mon[28057]: pgmap v152: 97 pgs: 97 active+clean; 37 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 334 op/s 2024-04-26T01:58:37.658 INFO:tasks.workunit.client.0.smithi151.stdout:copying to largest ever: 0xffdb8 2024-04-26T01:58:37.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:58:37 smithi172 ceph-mon[32785]: pgmap v153: 97 pgs: 97 active+clean; 37 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 214 op/s 2024-04-26T01:58:37.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:58:37 smithi165 ceph-mon[32648]: pgmap v153: 97 pgs: 97 active+clean; 37 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 214 op/s 2024-04-26T01:58:37.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:58:37 smithi151 ceph-mon[28057]: pgmap v153: 97 pgs: 97 active+clean; 37 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 214 op/s 2024-04-26T01:58:40.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:58:39 smithi151 ceph-mon[28057]: pgmap v154: 97 pgs: 97 active+clean; 39 MiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 330 op/s 2024-04-26T01:58:40.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:58:39 smithi172 ceph-mon[32785]: pgmap v154: 97 pgs: 97 active+clean; 39 MiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 330 op/s 2024-04-26T01:58:40.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:58:39 smithi165 ceph-mon[32648]: pgmap v154: 97 pgs: 97 active+clean; 39 MiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 330 op/s 2024-04-26T01:58:42.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:58:41 smithi172 ceph-mon[32785]: pgmap v155: 97 pgs: 97 active+clean; 39 MiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 224 op/s 2024-04-26T01:58:42.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:58:41 smithi165 ceph-mon[32648]: pgmap v155: 97 pgs: 97 active+clean; 39 MiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 224 op/s 2024-04-26T01:58:42.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:58:41 smithi151 ceph-mon[28057]: pgmap v155: 97 pgs: 97 active+clean; 39 MiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 224 op/s 2024-04-26T01:58:44.286 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:58:43 smithi172 ceph-mon[32785]: pgmap v156: 97 pgs: 97 active+clean; 41 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 267 op/s 2024-04-26T01:58:44.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:58:43 smithi165 ceph-mon[32648]: pgmap v156: 97 pgs: 97 active+clean; 41 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 267 op/s 2024-04-26T01:58:44.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:58:43 smithi151 ceph-mon[28057]: pgmap v156: 97 pgs: 97 active+clean; 41 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 267 op/s 2024-04-26T01:58:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.548+0000 7fde4fe00640 1 -- 172.21.15.151:0/1671139085 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fde50076480 msgr2=0x7fde50076860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:45.548 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.548+0000 7fde4fe00640 1 --2- 172.21.15.151:0/1671139085 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fde50076480 0x7fde50076860 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fde40009930 tx=0x7fde4002a420 comp rx=0 tx=0).stop 2024-04-26T01:58:45.548 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.548+0000 7fde4fe00640 1 -- 172.21.15.151:0/1671139085 shutdown_connections 2024-04-26T01:58:45.548 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.548+0000 7fde4fe00640 1 --2- 172.21.15.151:0/1671139085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde50112c30 0x7fde50115020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:45.548 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.548+0000 7fde4fe00640 1 --2- 172.21.15.151:0/1671139085 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde50076da0 0x7fde501126f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:45.548 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.548+0000 7fde4fe00640 1 --2- 172.21.15.151:0/1671139085 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fde50076480 0x7fde50076860 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:45.548 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.548+0000 7fde4fe00640 1 -- 172.21.15.151:0/1671139085 >> 172.21.15.151:0/1671139085 conn(0x7fde500ffa60 msgr2=0x7fde50101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:45.548 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.549+0000 7fde4fe00640 1 -- 172.21.15.151:0/1671139085 shutdown_connections 2024-04-26T01:58:45.548 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.549+0000 7fde4fe00640 1 -- 172.21.15.151:0/1671139085 wait complete. 2024-04-26T01:58:45.548 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.550+0000 7fde4fe00640 1 Processor -- start 2024-04-26T01:58:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.550+0000 7fde4fe00640 1 -- start start 2024-04-26T01:58:45.550 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.551+0000 7fde4fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fde50076480 0x7fde501a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:45.550 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.551+0000 7fde4fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde50076da0 0x7fde501a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:45.550 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.551+0000 7fde4fe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde50112c30 0x7fde5019ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:45.550 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.551+0000 7fde4fe00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fde501186a0 con 0x7fde50076da0 2024-04-26T01:58:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.551+0000 7fde4fe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fde50118520 con 0x7fde50076480 2024-04-26T01:58:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.551+0000 7fde4fe00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fde50118820 con 0x7fde50112c30 2024-04-26T01:58:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.551+0000 7fde4f400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde50112c30 0x7fde5019ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.551+0000 7fde4e000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde50076da0 0x7fde501a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.551+0000 7fde4f400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde50112c30 0x7fde5019ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:32994/0 (socket says 172.21.15.151:32994) 2024-04-26T01:58:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.551+0000 7fde4f400640 1 -- 172.21.15.151:0/998366796 learned_addr learned my addr 172.21.15.151:0/998366796 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:58:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.551+0000 7fde4ea00640 1 --2- 172.21.15.151:0/998366796 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fde50076480 0x7fde501a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.551+0000 7fde4f400640 1 -- 172.21.15.151:0/998366796 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fde50076480 msgr2=0x7fde501a4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.551+0000 7fde4f400640 1 --2- 172.21.15.151:0/998366796 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fde50076480 0x7fde501a4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.551+0000 7fde4f400640 1 -- 172.21.15.151:0/998366796 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde50076da0 msgr2=0x7fde501a4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.551+0000 7fde4f400640 1 --2- 172.21.15.151:0/998366796 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde50076da0 0x7fde501a4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.551+0000 7fde4f400640 1 -- 172.21.15.151:0/998366796 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde5019f1c0 con 0x7fde50112c30 2024-04-26T01:58:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.551+0000 7fde4ea00640 1 --2- 172.21.15.151:0/998366796 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fde50076480 0x7fde501a4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:58:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.552+0000 7fde4f400640 1 --2- 172.21.15.151:0/998366796 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde50112c30 0x7fde5019ea40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fde4800eb40 tx=0x7fde4800c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:45.552 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.552+0000 7fde47400640 1 -- 172.21.15.151:0/998366796 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde48004b20 con 0x7fde50112c30 2024-04-26T01:58:45.552 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.552+0000 7fde47400640 1 -- 172.21.15.151:0/998366796 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde48004cc0 con 0x7fde50112c30 2024-04-26T01:58:45.552 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.552+0000 7fde4fe00640 1 -- 172.21.15.151:0/998366796 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde5019f4b0 con 0x7fde50112c30 2024-04-26T01:58:45.552 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.552+0000 7fde47400640 1 -- 172.21.15.151:0/998366796 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde48010720 con 0x7fde50112c30 2024-04-26T01:58:45.552 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.552+0000 7fde4fe00640 1 -- 172.21.15.151:0/998366796 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde501ab750 con 0x7fde50112c30 2024-04-26T01:58:45.555 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.557+0000 7fde5887f740 1 Processor -- start 2024-04-26T01:58:45.556 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.557+0000 7fde47400640 1 -- 172.21.15.151:0/998366796 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7fde48005560 con 0x7fde50112c30 2024-04-26T01:58:45.556 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.557+0000 7fde5887f740 1 -- start start 2024-04-26T01:58:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.557+0000 7fde5887f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5614c05c19b0 0x5614c05c1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.557+0000 7fde5887f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614c05c22d0 0x5614c05cc650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.557+0000 7fde5887f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5614c05ccb90 0x5614c05cef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.557+0000 7fde5887f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5614c0527900 con 0x5614c05c19b0 2024-04-26T01:58:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.557+0000 7fde5887f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5614c04cdef0 con 0x5614c05ccb90 2024-04-26T01:58:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.557+0000 7fde5887f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5614c05238c0 con 0x5614c05c22d0 2024-04-26T01:58:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.558+0000 7fde4ea00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614c05c22d0 0x5614c05cc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.558+0000 7fde4ea00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614c05c22d0 0x5614c05cc650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:33010/0 (socket says 172.21.15.151:33010) 2024-04-26T01:58:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.558+0000 7fde4ea00640 1 -- 172.21.15.151:0/1137471930 learned_addr learned my addr 172.21.15.151:0/1137471930 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:58:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.558+0000 7fde4e000640 1 --2- 172.21.15.151:0/1137471930 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5614c05c19b0 0x5614c05c1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.558+0000 7fde4e000640 1 --2- 172.21.15.151:0/1137471930 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5614c05ccb90 0x5614c05cef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.558+0000 7fde4ea00640 1 -- 172.21.15.151:0/1137471930 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5614c05ccb90 msgr2=0x5614c05cef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.558+0000 7fde4ea00640 1 --2- 172.21.15.151:0/1137471930 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5614c05ccb90 0x5614c05cef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.558+0000 7fde4ea00640 1 -- 172.21.15.151:0/1137471930 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5614c05c19b0 msgr2=0x5614c05c1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.558+0000 7fde4ea00640 1 --2- 172.21.15.151:0/1137471930 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5614c05c19b0 0x5614c05c1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.558+0000 7fde4ea00640 1 -- 172.21.15.151:0/1137471930 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614c023a210 con 0x5614c05c22d0 2024-04-26T01:58:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.558+0000 7fde4e000640 1 --2- 172.21.15.151:0/1137471930 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5614c05ccb90 0x5614c05cef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T01:58:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.558+0000 7fde4ea00640 1 --2- 172.21.15.151:0/1137471930 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614c05c22d0 0x5614c05cc650 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fde4002aa10 tx=0x7fde40004140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f47c3031b893515f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.559+0000 7fde3be00640 1 -- 172.21.15.151:0/1137471930 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde40035030 con 0x5614c05c22d0 2024-04-26T01:58:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.559+0000 7fde3be00640 1 -- 172.21.15.151:0/1137471930 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde400054f0 con 0x5614c05c22d0 2024-04-26T01:58:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.559+0000 7fde3be00640 1 -- 172.21.15.151:0/1137471930 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde400035f0 con 0x5614c05c22d0 2024-04-26T01:58:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.559+0000 7fde47400640 1 --2- 172.21.15.151:0/998366796 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fde340a3d90 0x7fde340a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.559+0000 7fde47400640 1 -- 172.21.15.151:0/998366796 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fde480d0ee0 con 0x7fde50112c30 2024-04-26T01:58:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.559+0000 7fde4e000640 1 --2- 172.21.15.151:0/998366796 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fde340a3d90 0x7fde340a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:45.561 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.559+0000 7fde5887f740 1 -- 172.21.15.151:0/1137471930 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614c05c22d0 msgr2=0x5614c05cc650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:45.561 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.559+0000 7fde5887f740 1 --2- 172.21.15.151:0/1137471930 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614c05c22d0 0x5614c05cc650 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fde4002aa10 tx=0x7fde40004140 comp rx=0 tx=0).stop 2024-04-26T01:58:45.561 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.560+0000 7fde5887f740 1 -- 172.21.15.151:0/1137471930 shutdown_connections 2024-04-26T01:58:45.561 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.560+0000 7fde5887f740 1 --2- 172.21.15.151:0/1137471930 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5614c05ccb90 0x5614c05cef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:45.561 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.560+0000 7fde5887f740 1 --2- 172.21.15.151:0/1137471930 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614c05c22d0 0x5614c05cc650 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:45.561 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.560+0000 7fde5887f740 1 --2- 172.21.15.151:0/1137471930 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5614c05c19b0 0x5614c05c1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:45.561 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.560+0000 7fde5887f740 1 -- 172.21.15.151:0/1137471930 >> 172.21.15.151:0/1137471930 conn(0x5614c05bdec0 msgr2=0x5614c05be2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:45.561 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.560+0000 7fde5887f740 1 -- 172.21.15.151:0/1137471930 shutdown_connections 2024-04-26T01:58:45.561 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.560+0000 7fde5887f740 1 -- 172.21.15.151:0/1137471930 wait complete. 2024-04-26T01:58:45.561 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.560+0000 7fde4e000640 1 --2- 172.21.15.151:0/998366796 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fde340a3d90 0x7fde340a6250 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x5614c05bf2e0 tx=0x7fde3c0096c0 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:45.562 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.560+0000 7fde5887f740 1 -- start start 2024-04-26T01:58:45.565 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.560+0000 7fde5887f740 1 Processor -- start 2024-04-26T01:58:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.561+0000 7fde5887f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614c05c19b0 0x5614c05c9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.561+0000 7fde5887f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5614c05c22d0 0x5614c05ca010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.561+0000 7fde5887f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5614c05ccb90 0x5614c05ca550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.561+0000 7fde5887f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5614c05cf760 con 0x5614c05c22d0 2024-04-26T01:58:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.561+0000 7fde5887f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5614c04e1310 con 0x5614c05ccb90 2024-04-26T01:58:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.561+0000 7fde5887f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5614c05cf8e0 con 0x5614c05c19b0 2024-04-26T01:58:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.561+0000 7fde5887f740 10 client.0 _ll_register_callbacks cb 0x5614c04afcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:58:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.561+0000 7fde4ea00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614c05c19b0 0x5614c05c9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.561+0000 7fde4ea00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614c05c19b0 0x5614c05c9ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:33022/0 (socket says 172.21.15.151:33022) 2024-04-26T01:58:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.561+0000 7fde4ea00640 1 -- 172.21.15.151:0/3386542576 learned_addr learned my addr 172.21.15.151:0/3386542576 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:58:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.561+0000 7fde4e000640 1 --2- 172.21.15.151:0/3386542576 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5614c05c22d0 0x5614c05ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:45.567 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.561+0000 7fde4f400640 1 --2- 172.21.15.151:0/3386542576 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5614c05ccb90 0x5614c05ca550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:45.567 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.562+0000 7fde4ea00640 1 -- 172.21.15.151:0/3386542576 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5614c05ccb90 msgr2=0x5614c05ca550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:45.567 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.562+0000 7fde4ea00640 1 --2- 172.21.15.151:0/3386542576 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5614c05ccb90 0x5614c05ca550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:45.567 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.562+0000 7fde4ea00640 1 -- 172.21.15.151:0/3386542576 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5614c05c22d0 msgr2=0x5614c05ca010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:45.567 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.562+0000 7fde4ea00640 1 --2- 172.21.15.151:0/3386542576 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5614c05c22d0 0x5614c05ca010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:45.567 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.562+0000 7fde4ea00640 1 -- 172.21.15.151:0/3386542576 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614c0306620 con 0x5614c05c19b0 2024-04-26T01:58:45.567 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.562+0000 7fde4f400640 1 --2- 172.21.15.151:0/3386542576 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5614c05ccb90 0x5614c05ca550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T01:58:45.567 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.562+0000 7fde4ea00640 1 --2- 172.21.15.151:0/3386542576 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614c05c19b0 0x5614c05c9ad0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fde40000c00 tx=0x7fde400048f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eaea30953e920d2b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:45.567 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.562+0000 7fde38c00640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-26T01:58:45.567 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.562+0000 7fde38c00640 1 -- 172.21.15.151:0/3386542576 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde40051060 con 0x5614c05c19b0 2024-04-26T01:58:45.567 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.562+0000 7fde38c00640 1 -- 172.21.15.151:0/3386542576 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fde400051f0 con 0x5614c05c19b0 2024-04-26T01:58:45.567 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.562+0000 7fde5887f740 1 -- 172.21.15.151:0/3386542576 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde400054f0 con 0x5614c05c19b0 2024-04-26T01:58:45.567 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.562+0000 7fde38c00640 1 -- 172.21.15.151:0/3386542576 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fde40035040 con 0x5614c05c19b0 2024-04-26T01:58:45.568 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.562+0000 7fde5887f740 10 client.25780 fetch_fsmap learned FSMap version 30 2024-04-26T01:58:45.568 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.562+0000 7fde5887f740 1 -- 172.21.15.151:0/3386542576 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5614c05cf760 con 0x5614c05c19b0 2024-04-26T01:58:45.568 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.563+0000 7fde38c00640 1 -- 172.21.15.151:0/3386542576 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde400054f0 con 0x5614c05c19b0 2024-04-26T01:58:45.568 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.563+0000 7fde38c00640 1 -- 172.21.15.151:0/3386542576 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fde40054020 con 0x5614c05c19b0 2024-04-26T01:58:45.568 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.563+0000 7fde5887f740 10 client.25780 fetch_fsmap finished waiting for FSMap version 30 2024-04-26T01:58:45.568 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.563+0000 7fde5887f740 10 client.25780 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:58:45.568 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.563+0000 7fde5887f740 20 client.25780 populate_metadata read hostname 'smithi151' 2024-04-26T01:58:45.568 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.563+0000 7fde5887f740 1 --2- 172.21.15.151:0/3386542576 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5614c05dcad0 0x5614c05deec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:45.568 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.563+0000 7fde5887f740 4 client.25780 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:58:45.568 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.563+0000 7fde5887f740 1 -- 172.21.15.151:0/3386542576 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614c053b8b0 con 0x5614c05dcad0 2024-04-26T01:58:45.568 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.563+0000 7fde4e000640 1 --2- 172.21.15.151:0/3386542576 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5614c05dcad0 0x5614c05deec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:45.568 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.564+0000 7fde4e000640 1 --2- 172.21.15.151:0/3386542576 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5614c05dcad0 0x5614c05deec0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d379162b975c147b server_cookie=64271ca8f6fa72d4 in_seq=0 out_seq=0 2024-04-26T01:58:45.568 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.564+0000 7fde38c00640 10 client.25780 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:58:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.564+0000 7fde38c00640 1 -- 172.21.15.151:0/3386542576 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5614c053b8b0 con 0x5614c05dcad0 2024-04-26T01:58:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.564+0000 7fde38c00640 10 client.25780 handle_command_reply: tid=0 2024-04-26T01:58:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.565+0000 7fde5887f740 1 client.25780 shutdown 2024-04-26T01:58:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.565+0000 7fde5887f740 1 -- 172.21.15.151:0/3386542576 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614c05c19b0 msgr2=0x5614c05c9ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.565+0000 7fde5887f740 1 --2- 172.21.15.151:0/3386542576 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614c05c19b0 0x5614c05c9ad0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fde40000c00 tx=0x7fde400048f0 comp rx=0 tx=0).stop 2024-04-26T01:58:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.565+0000 7fde5887f740 1 -- 172.21.15.151:0/3386542576 shutdown_connections 2024-04-26T01:58:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.565+0000 7fde5887f740 1 --2- 172.21.15.151:0/3386542576 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5614c05dcad0 0x5614c05deec0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.565+0000 7fde5887f740 1 --2- 172.21.15.151:0/3386542576 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5614c05ccb90 0x5614c05ca550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.565+0000 7fde5887f740 1 --2- 172.21.15.151:0/3386542576 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5614c05c22d0 0x5614c05ca010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.565+0000 7fde5887f740 1 --2- 172.21.15.151:0/3386542576 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614c05c19b0 0x5614c05c9ad0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.565+0000 7fde5887f740 1 -- 172.21.15.151:0/3386542576 >> 172.21.15.151:0/3386542576 conn(0x5614c05bdec0 msgr2=0x5614c05c3c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.565+0000 7fde38c00640 0 client.25780 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:58:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.565+0000 7fde5887f740 1 -- 172.21.15.151:0/3386542576 shutdown_connections 2024-04-26T01:58:45.570 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.565+0000 7fde5887f740 1 -- 172.21.15.151:0/3386542576 wait complete. 2024-04-26T01:58:45.570 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.565+0000 7fde5887f740 20 client.25780 trim_cache size 0 max 16384 2024-04-26T01:58:45.581 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.582+0000 7fde5887f740 1 Processor -- start 2024-04-26T01:58:45.581 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.582+0000 7fde5887f740 1 -- start start 2024-04-26T01:58:45.583 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.582+0000 7fde5887f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614c05c19b0 0x5614c05d8650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:45.583 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.583+0000 7fde5887f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5614c05c22d0 0x5614c05e45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:45.583 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.583+0000 7fde5887f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5614c05ccb90 0x5614c05e4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:45.583 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.583+0000 7fde5887f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fde501b6780 con 0x5614c05c22d0 2024-04-26T01:58:45.583 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.583+0000 7fde5887f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fde501b4050 con 0x5614c05ccb90 2024-04-26T01:58:45.583 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.583+0000 7fde5887f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fde501b1ea0 con 0x5614c05c19b0 2024-04-26T01:58:45.583 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.583+0000 7fde4ea00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614c05c19b0 0x5614c05d8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:45.583 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.583+0000 7fde4ea00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614c05c19b0 0x5614c05d8650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:33024/0 (socket says 172.21.15.151:33024) 2024-04-26T01:58:45.583 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.583+0000 7fde4ea00640 1 -- 172.21.15.151:0/2579137051 learned_addr learned my addr 172.21.15.151:0/2579137051 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:58:45.583 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.583+0000 7fde4f400640 1 --2- 172.21.15.151:0/2579137051 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5614c05ccb90 0x5614c05e4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:45.583 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.583+0000 7fde4e000640 1 --2- 172.21.15.151:0/2579137051 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5614c05c22d0 0x5614c05e45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:45.583 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.583+0000 7fde4ea00640 1 -- 172.21.15.151:0/2579137051 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5614c05ccb90 msgr2=0x5614c05e4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:45.584 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.583+0000 7fde4ea00640 1 --2- 172.21.15.151:0/2579137051 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5614c05ccb90 0x5614c05e4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:45.584 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.583+0000 7fde4ea00640 1 -- 172.21.15.151:0/2579137051 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5614c05c22d0 msgr2=0x5614c05e45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:45.584 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.583+0000 7fde4ea00640 1 --2- 172.21.15.151:0/2579137051 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5614c05c22d0 0x5614c05e45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:45.584 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.583+0000 7fde4ea00640 1 -- 172.21.15.151:0/2579137051 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614c05c2100 con 0x5614c05c19b0 2024-04-26T01:58:45.584 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.583+0000 7fde4f400640 1 --2- 172.21.15.151:0/2579137051 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5614c05ccb90 0x5614c05e4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:58:45.584 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.583+0000 7fde4e000640 1 --2- 172.21.15.151:0/2579137051 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5614c05c22d0 0x5614c05e45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:58:45.584 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.584+0000 7fde4ea00640 1 --2- 172.21.15.151:0/2579137051 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614c05c19b0 0x5614c05d8650 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fde400092a0 tx=0x7fde40031400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=316858d52d9f628c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:45.584 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.584+0000 7fde3b400640 1 -- 172.21.15.151:0/2579137051 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde40051040 con 0x5614c05c19b0 2024-04-26T01:58:45.584 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.584+0000 7fde3b400640 1 -- 172.21.15.151:0/2579137051 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde40004a20 con 0x5614c05c19b0 2024-04-26T01:58:45.584 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.584+0000 7fde3b400640 1 -- 172.21.15.151:0/2579137051 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde40003c10 con 0x5614c05c19b0 2024-04-26T01:58:45.588 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.585+0000 7fde5887f740 1 -- 172.21.15.151:0/2579137051 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614c05c19b0 msgr2=0x5614c05d8650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:45.588 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.585+0000 7fde5887f740 1 --2- 172.21.15.151:0/2579137051 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614c05c19b0 0x5614c05d8650 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fde400092a0 tx=0x7fde40031400 comp rx=0 tx=0).stop 2024-04-26T01:58:45.588 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.585+0000 7fde5887f740 1 -- 172.21.15.151:0/2579137051 shutdown_connections 2024-04-26T01:58:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.585+0000 7fde5887f740 1 --2- 172.21.15.151:0/2579137051 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5614c05ccb90 0x5614c05e4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.585+0000 7fde5887f740 1 --2- 172.21.15.151:0/2579137051 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5614c05c22d0 0x5614c05e45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.585+0000 7fde5887f740 1 --2- 172.21.15.151:0/2579137051 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614c05c19b0 0x5614c05d8650 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.585+0000 7fde5887f740 1 -- 172.21.15.151:0/2579137051 >> 172.21.15.151:0/2579137051 conn(0x5614c05bdec0 msgr2=0x5614c05be2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.585+0000 7fde5887f740 1 -- 172.21.15.151:0/2579137051 shutdown_connections 2024-04-26T01:58:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.585+0000 7fde5887f740 1 -- 172.21.15.151:0/2579137051 wait complete. 2024-04-26T01:58:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.585+0000 7fde5887f740 1 -- start start 2024-04-26T01:58:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.586+0000 7fde5887f740 1 Processor -- start 2024-04-26T01:58:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.586+0000 7fde5887f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614c05c19b0 0x5614c05d26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.586+0000 7fde5887f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5614c05c22d0 0x5614c05d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.586+0000 7fde5887f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5614c05ccb90 0x5614c05d3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.586+0000 7fde5887f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fde501b2510 con 0x5614c05ccb90 2024-04-26T01:58:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.586+0000 7fde5887f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fde501b3980 con 0x5614c05c22d0 2024-04-26T01:58:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.586+0000 7fde5887f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fde501b2be0 con 0x5614c05c19b0 2024-04-26T01:58:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.586+0000 7fde5887f740 10 client.0 _ll_register_callbacks cb 0x7fde400056a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:58:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.586+0000 7fde4ea00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614c05c19b0 0x5614c05d26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.586+0000 7fde4e000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5614c05c22d0 0x5614c05d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.586+0000 7fde4e000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5614c05c22d0 0x5614c05d2bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:44918/0 (socket says 172.21.15.151:44918) 2024-04-26T01:58:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.586+0000 7fde4e000640 1 -- 172.21.15.151:0/1873402280 learned_addr learned my addr 172.21.15.151:0/1873402280 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:58:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.586+0000 7fde4ea00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614c05c19b0 0x5614c05d26b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:33036/0 (socket says 172.21.15.151:33036) 2024-04-26T01:58:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.586+0000 7fde4f400640 1 --2- 172.21.15.151:0/1873402280 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5614c05ccb90 0x5614c05d3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.587+0000 7fde4ea00640 1 -- 172.21.15.151:0/1873402280 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5614c05c22d0 msgr2=0x5614c05d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.587+0000 7fde4ea00640 1 --2- 172.21.15.151:0/1873402280 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5614c05c22d0 0x5614c05d2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.587+0000 7fde4ea00640 1 -- 172.21.15.151:0/1873402280 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5614c05ccb90 msgr2=0x5614c05d3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.587+0000 7fde4ea00640 1 --2- 172.21.15.151:0/1873402280 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5614c05ccb90 0x5614c05d3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.587+0000 7fde4ea00640 1 -- 172.21.15.151:0/1873402280 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614c0641f60 con 0x5614c05c19b0 2024-04-26T01:58:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.587+0000 7fde4e000640 1 --2- 172.21.15.151:0/1873402280 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5614c05c22d0 0x5614c05d2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:58:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.587+0000 7fde4f400640 1 --2- 172.21.15.151:0/1873402280 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5614c05ccb90 0x5614c05d3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:58:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.587+0000 7fde4ea00640 1 --2- 172.21.15.151:0/1873402280 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614c05c19b0 0x5614c05d26b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fde400316b0 tx=0x7fde40031a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd011bf033a0f415 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.587+0000 7fde38c00640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-26T01:58:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.587+0000 7fde38c00640 1 -- 172.21.15.151:0/1873402280 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde40051040 con 0x5614c05c19b0 2024-04-26T01:58:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.587+0000 7fde38c00640 1 -- 172.21.15.151:0/1873402280 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fde40003990 con 0x5614c05c19b0 2024-04-26T01:58:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.587+0000 7fde5887f740 1 -- 172.21.15.151:0/1873402280 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5614c0535320 con 0x5614c05c19b0 2024-04-26T01:58:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.587+0000 7fde38c00640 1 -- 172.21.15.151:0/1873402280 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fde400054c0 con 0x5614c05c19b0 2024-04-26T01:58:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.587+0000 7fde5887f740 10 client.25786 fetch_fsmap learned FSMap version 30 2024-04-26T01:58:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.587+0000 7fde5887f740 1 -- 172.21.15.151:0/1873402280 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5614c0643270 con 0x5614c05c19b0 2024-04-26T01:58:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.588+0000 7fde38c00640 1 -- 172.21.15.151:0/1873402280 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5614c0535320 con 0x5614c05c19b0 2024-04-26T01:58:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.588+0000 7fde38c00640 1 -- 172.21.15.151:0/1873402280 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7fde4004b030 con 0x5614c05c19b0 2024-04-26T01:58:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.588+0000 7fde5887f740 10 client.25786 fetch_fsmap finished waiting for FSMap version 30 2024-04-26T01:58:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.588+0000 7fde5887f740 10 client.25786 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:58:45.592 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.588+0000 7fde5887f740 20 client.25786 populate_metadata read hostname 'smithi151' 2024-04-26T01:58:45.592 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.588+0000 7fde5887f740 1 --2- 172.21.15.151:0/1873402280 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5614c06436b0 0x5614c0645ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:45.592 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.588+0000 7fde5887f740 4 client.25786 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:58:45.592 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.588+0000 7fde5887f740 1 -- 172.21.15.151:0/1873402280 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5614c052f190 con 0x5614c06436b0 2024-04-26T01:58:45.592 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.588+0000 7fde4e000640 1 --2- 172.21.15.151:0/1873402280 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5614c06436b0 0x5614c0645ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:45.592 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.589+0000 7fde4e000640 1 --2- 172.21.15.151:0/1873402280 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5614c06436b0 0x5614c0645ad0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c18c1472a0f8195 server_cookie=86b014058926efca in_seq=0 out_seq=0 2024-04-26T01:58:45.592 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.589+0000 7fde38c00640 10 client.25786 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:58:45.642 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.643+0000 7fde38c00640 1 -- 172.21.15.151:0/1873402280 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5614c052f190 con 0x5614c06436b0 2024-04-26T01:58:45.642 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.643+0000 7fde38c00640 10 client.25786 handle_command_reply: tid=0 2024-04-26T01:58:45.642 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.643+0000 7fde5887f740 1 client.25786 shutdown 2024-04-26T01:58:45.643 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.643+0000 7fde5887f740 1 -- 172.21.15.151:0/1873402280 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614c05c19b0 msgr2=0x5614c05d26b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:45.643 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.643+0000 7fde5887f740 1 --2- 172.21.15.151:0/1873402280 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614c05c19b0 0x5614c05d26b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fde400316b0 tx=0x7fde40031a50 comp rx=0 tx=0).stop 2024-04-26T01:58:45.643 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.644+0000 7fde5887f740 1 -- 172.21.15.151:0/1873402280 shutdown_connections 2024-04-26T01:58:45.643 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.644+0000 7fde5887f740 1 --2- 172.21.15.151:0/1873402280 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5614c06436b0 0x5614c0645ad0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:45.643 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.644+0000 7fde5887f740 1 --2- 172.21.15.151:0/1873402280 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5614c05ccb90 0x5614c05d3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:45.643 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.644+0000 7fde5887f740 1 --2- 172.21.15.151:0/1873402280 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5614c05c22d0 0x5614c05d2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:45.643 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.644+0000 7fde5887f740 1 --2- 172.21.15.151:0/1873402280 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614c05c19b0 0x5614c05d26b0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:45.643 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.644+0000 7fde5887f740 1 -- 172.21.15.151:0/1873402280 >> 172.21.15.151:0/1873402280 conn(0x5614c05bdec0 msgr2=0x5614c05bc790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:45.643 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.644+0000 7fde38c00640 0 client.25786 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:58:45.644 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.644+0000 7fde5887f740 1 -- 172.21.15.151:0/1873402280 shutdown_connections 2024-04-26T01:58:45.644 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.644+0000 7fde5887f740 1 -- 172.21.15.151:0/1873402280 wait complete. 2024-04-26T01:58:45.644 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.644+0000 7fde5887f740 20 client.25786 trim_cache size 0 max 16384 2024-04-26T01:58:45.644 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:{ 2024-04-26T01:58:45.644 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "message": "", 2024-04-26T01:58:45.644 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "return_code": 0 2024-04-26T01:58:45.644 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:} 2024-04-26T01:58:45.645 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.645+0000 7fde4fe00640 1 -- 172.21.15.151:0/998366796 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fde340a3d90 msgr2=0x7fde340a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:45.645 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.645+0000 7fde4fe00640 1 --2- 172.21.15.151:0/998366796 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fde340a3d90 0x7fde340a6250 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x5614c05bf2e0 tx=0x7fde3c0096c0 comp rx=0 tx=0).stop 2024-04-26T01:58:45.645 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.645+0000 7fde4fe00640 1 -- 172.21.15.151:0/998366796 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde50112c30 msgr2=0x7fde5019ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:45.645 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.645+0000 7fde4fe00640 1 --2- 172.21.15.151:0/998366796 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde50112c30 0x7fde5019ea40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fde4800eb40 tx=0x7fde4800c2d0 comp rx=0 tx=0).stop 2024-04-26T01:58:45.645 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.645+0000 7fde4fe00640 1 -- 172.21.15.151:0/998366796 shutdown_connections 2024-04-26T01:58:45.645 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.645+0000 7fde4fe00640 1 --2- 172.21.15.151:0/998366796 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fde340a3d90 0x7fde340a6250 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:45.645 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.645+0000 7fde4fe00640 1 --2- 172.21.15.151:0/998366796 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde50112c30 0x7fde5019ea40 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:45.645 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.645+0000 7fde4fe00640 1 --2- 172.21.15.151:0/998366796 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde50076da0 0x7fde501a4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:45.645 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.645+0000 7fde4fe00640 1 --2- 172.21.15.151:0/998366796 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fde50076480 0x7fde501a4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:45.645 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.645+0000 7fde4fe00640 1 -- 172.21.15.151:0/998366796 >> 172.21.15.151:0/998366796 conn(0x7fde500ffa60 msgr2=0x7fde50101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:45.646 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.645+0000 7fde4fe00640 1 -- 172.21.15.151:0/998366796 shutdown_connections 2024-04-26T01:58:45.646 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:45.646+0000 7fde4fe00640 1 -- 172.21.15.151:0/998366796 wait complete. 2024-04-26T01:58:46.132 INFO:journalctl@ceph.mds.f.smithi172.stdout:Apr 26 01:58:45 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-f[63189]: ignoring --setuser ceph since I am not root 2024-04-26T01:58:46.132 INFO:journalctl@ceph.mds.f.smithi172.stdout:Apr 26 01:58:45 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-f[63189]: ignoring --setgroup ceph since I am not root 2024-04-26T01:58:46.133 INFO:journalctl@ceph.mds.f.smithi172.stdout:Apr 26 01:58:45 smithi172 ceph-mds[63193]: -- 172.21.15.172:0/2437710367 <== mon.1 v2:172.21.15.172:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555bd9609a40 con 0x555bd95dd000 2024-04-26T01:58:46.133 INFO:journalctl@ceph.mds.f.smithi172.stdout:Apr 26 01:58:45 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-f[63189]: starting mds.f at 2024-04-26T01:58:46.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:58:46 smithi172 ceph-mon[32785]: pgmap v157: 97 pgs: 97 active+clean; 40 MiB data, 788 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 303 op/s 2024-04-26T01:58:46.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:58:46 smithi172 ceph-mon[32785]: daemon mds.f restarted 2024-04-26T01:58:46.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:58:46 smithi165 ceph-mon[32648]: pgmap v157: 97 pgs: 97 active+clean; 40 MiB data, 788 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 303 op/s 2024-04-26T01:58:46.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:58:46 smithi165 ceph-mon[32648]: daemon mds.f restarted 2024-04-26T01:58:46.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:58:46 smithi151 ceph-mon[28057]: pgmap v157: 97 pgs: 97 active+clean; 40 MiB data, 788 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 303 op/s 2024-04-26T01:58:46.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:58:46 smithi151 ceph-mon[28057]: daemon mds.f restarted 2024-04-26T01:58:47.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:58:47 smithi172 ceph-mon[32785]: osdmap e94: 12 total, 12 up, 12 in 2024-04-26T01:58:47.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:58:47 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T01:58:47.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:58:47 smithi172 ceph-mon[32785]: mds.? [v2:172.21.15.172:6834/1657987703,v1:172.21.15.172:6835/1657987703] up:boot 2024-04-26T01:58:47.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:58:47 smithi172 ceph-mon[32785]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T01:58:47.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:58:47 smithi172 ceph-mon[32785]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T01:58:47.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:58:47 smithi165 ceph-mon[32648]: osdmap e94: 12 total, 12 up, 12 in 2024-04-26T01:58:47.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:58:47 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T01:58:47.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:58:47 smithi165 ceph-mon[32648]: mds.? [v2:172.21.15.172:6834/1657987703,v1:172.21.15.172:6835/1657987703] up:boot 2024-04-26T01:58:47.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:58:47 smithi165 ceph-mon[32648]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T01:58:47.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:58:47 smithi165 ceph-mon[32648]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T01:58:47.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:58:47 smithi151 ceph-mon[28057]: osdmap e94: 12 total, 12 up, 12 in 2024-04-26T01:58:47.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:58:47 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T01:58:47.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:58:47 smithi151 ceph-mon[28057]: mds.? [v2:172.21.15.172:6834/1657987703,v1:172.21.15.172:6835/1657987703] up:boot 2024-04-26T01:58:47.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:58:47 smithi151 ceph-mon[28057]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T01:58:47.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:58:47 smithi151 ceph-mon[28057]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T01:58:48.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:58:48 smithi172 ceph-mon[32785]: pgmap v159: 97 pgs: 97 active+clean; 40 MiB data, 788 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 278 op/s 2024-04-26T01:58:48.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:58:48 smithi165 ceph-mon[32648]: pgmap v159: 97 pgs: 97 active+clean; 40 MiB data, 788 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 278 op/s 2024-04-26T01:58:48.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:58:48 smithi151 ceph-mon[28057]: pgmap v159: 97 pgs: 97 active+clean; 40 MiB data, 788 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 278 op/s 2024-04-26T01:58:50.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:58:50 smithi172 ceph-mon[32785]: pgmap v160: 97 pgs: 97 active+clean; 35 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 270 op/s 2024-04-26T01:58:50.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:58:50 smithi165 ceph-mon[32648]: pgmap v160: 97 pgs: 97 active+clean; 35 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 270 op/s 2024-04-26T01:58:50.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:58:50 smithi151 ceph-mon[28057]: pgmap v160: 97 pgs: 97 active+clean; 35 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 270 op/s 2024-04-26T01:58:52.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:58:52 smithi172 ceph-mon[32785]: pgmap v161: 97 pgs: 97 active+clean; 35 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 270 op/s 2024-04-26T01:58:52.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:58:52 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:58:52.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:58:52 smithi165 ceph-mon[32648]: pgmap v161: 97 pgs: 97 active+clean; 35 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 270 op/s 2024-04-26T01:58:52.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:58:52 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:58:52.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:58:52 smithi151 ceph-mon[28057]: pgmap v161: 97 pgs: 97 active+clean; 35 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 270 op/s 2024-04-26T01:58:52.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:58:52 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:58:53.458 INFO:tasks.workunit.client.0.smithi151.stdout:50000 copy from 0x898b8 to 0x92015, (0x875d bytes) at 0xb142c 2024-04-26T01:58:53.459 INFO:tasks.workunit.client.0.smithi151.stdout:All 50000 operations completed A-OK! 2024-04-26T01:58:53.464 INFO:tasks.workunit.client.0.smithi151.stderr:+ ./fsx -z 10MB -N 50000 -p 10000 -l 10485760 2024-04-26T01:58:53.474 INFO:tasks.workunit.client.0.smithi151.stdout:Seed set to 1 2024-04-26T01:58:53.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:58:53 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:58:53.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:58:53 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:58:53.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:58:53 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:58:53.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:58:53 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:58:53.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:58:53 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:58:53.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:58:53 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:58:53.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:58:53 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:58:53.626 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:58:53 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:58:53.626 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:58:53 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:58:53.633 INFO:tasks.workunit.client.0.smithi151.stderr:main: filesystem does not support fallocate mode 0, disabling! 2024-04-26T01:58:53.634 INFO:tasks.workunit.client.0.smithi151.stderr:main: filesystem does not support fallocate mode FALLOC_FL_KEEP_SIZE, disabling! 2024-04-26T01:58:53.636 INFO:tasks.workunit.client.0.smithi151.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2024-04-26T01:58:53.636 INFO:tasks.workunit.client.0.smithi151.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2024-04-26T01:58:53.636 INFO:tasks.workunit.client.0.smithi151.stderr:main: filesystem does not support clone range, disabling! 2024-04-26T01:58:53.639 INFO:tasks.workunit.client.0.smithi151.stderr:main: filesystem does not support dedupe range, disabling! 2024-04-26T01:58:53.642 INFO:tasks.workunit.client.0.smithi151.stderr:main: filesystem does not support exchange range, disabling! 2024-04-26T01:58:53.642 INFO:tasks.workunit.client.0.smithi151.stdout:skipping zero length punch hole 2024-04-26T01:58:53.642 INFO:tasks.workunit.client.0.smithi151.stdout:skipping zero length punch hole 2024-04-26T01:58:53.643 INFO:tasks.workunit.client.0.smithi151.stdout:skipping zero length punch hole 2024-04-26T01:58:53.643 INFO:tasks.workunit.client.0.smithi151.stdout:skipping zero size read 2024-04-26T01:58:53.643 INFO:tasks.workunit.client.0.smithi151.stdout:skipping zero length punch hole 2024-04-26T01:58:53.643 INFO:tasks.workunit.client.0.smithi151.stdout:skipping zero size read 2024-04-26T01:58:53.643 INFO:tasks.workunit.client.0.smithi151.stdout:skipping zero length punch hole 2024-04-26T01:58:53.643 INFO:tasks.workunit.client.0.smithi151.stdout:truncating to largest ever: 0x4310e9 2024-04-26T01:58:53.654 INFO:tasks.workunit.client.0.smithi151.stdout:copying to largest ever: 0x954a45 2024-04-26T01:58:53.715 INFO:tasks.workunit.client.0.smithi151.stdout:truncating to largest ever: 0x9bc582 2024-04-26T01:58:53.768 INFO:tasks.workunit.client.0.smithi151.stdout:truncating to largest ever: 0x9eecbc 2024-04-26T01:58:54.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:58:54 smithi172 ceph-mon[32785]: pgmap v162: 97 pgs: 97 active+clean; 36 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 264 op/s 2024-04-26T01:58:54.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:58:54 smithi165 ceph-mon[32648]: pgmap v162: 97 pgs: 97 active+clean; 36 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 264 op/s 2024-04-26T01:58:54.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:58:54 smithi151 ceph-mon[28057]: pgmap v162: 97 pgs: 97 active+clean; 36 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 264 op/s 2024-04-26T01:58:56.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:58:56 smithi172 ceph-mon[32785]: pgmap v163: 97 pgs: 97 active+clean; 36 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 256 op/s 2024-04-26T01:58:56.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:58:56 smithi165 ceph-mon[32648]: pgmap v163: 97 pgs: 97 active+clean; 36 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 256 op/s 2024-04-26T01:58:56.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:58:56 smithi151 ceph-mon[28057]: pgmap v163: 97 pgs: 97 active+clean; 36 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 256 op/s 2024-04-26T01:58:58.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:58:58 smithi172 ceph-mon[32785]: pgmap v164: 97 pgs: 97 active+clean; 36 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 240 op/s 2024-04-26T01:58:58.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:58:58 smithi165 ceph-mon[32648]: pgmap v164: 97 pgs: 97 active+clean; 36 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 240 op/s 2024-04-26T01:58:58.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:58:58 smithi151 ceph-mon[28057]: pgmap v164: 97 pgs: 97 active+clean; 36 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 240 op/s 2024-04-26T01:58:58.807 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.807+0000 7f04ec600640 1 -- 172.21.15.151:0/17348671 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f04e4104d70 msgr2=0x7f04e4108db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:58.807 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.807+0000 7f04ec600640 1 --2- 172.21.15.151:0/17348671 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f04e4104d70 0x7f04e4108db0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f04d0009900 tx=0x7f04d002a3f0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.807 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.808+0000 7f04ec600640 1 -- 172.21.15.151:0/17348671 shutdown_connections 2024-04-26T01:58:58.807 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.808+0000 7f04ec600640 1 --2- 172.21.15.151:0/17348671 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04e4109490 0x7f04e4111060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.807 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.808+0000 7f04ec600640 1 --2- 172.21.15.151:0/17348671 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f04e4104d70 0x7f04e4108db0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.807 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.808+0000 7f04ec600640 1 --2- 172.21.15.151:0/17348671 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04e41043c0 0x7f04e41047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.807 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.808+0000 7f04ec600640 1 -- 172.21.15.151:0/17348671 >> 172.21.15.151:0/17348671 conn(0x7f04e40ffc20 msgr2=0x7f04e4102040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:58.807 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.808+0000 7f04ec600640 1 -- 172.21.15.151:0/17348671 shutdown_connections 2024-04-26T01:58:58.807 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.808+0000 7f04ec600640 1 -- 172.21.15.151:0/17348671 wait complete. 2024-04-26T01:58:58.808 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.809+0000 7f04ec600640 1 Processor -- start 2024-04-26T01:58:58.808 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.809+0000 7f04ec600640 1 -- start start 2024-04-26T01:58:58.810 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.810+0000 7f04ec600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04e41043c0 0x7f04e41a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:58.810 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.810+0000 7f04ec600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04e4109490 0x7f04e41a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:58.810 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.810+0000 7f04ec600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f04e419a700 0x7f04e419ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:58.810 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.810+0000 7f04ec600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f04e4114420 con 0x7f04e41043c0 2024-04-26T01:58:58.810 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.810+0000 7f04ec600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f04e41142a0 con 0x7f04e4109490 2024-04-26T01:58:58.810 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.810+0000 7f04ec600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f04e41145a0 con 0x7f04e419a700 2024-04-26T01:58:58.810 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.810+0000 7f04e9200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04e4109490 0x7f04e41a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:58.810 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.810+0000 7f04e9200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04e4109490 0x7f04e41a0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:48294/0 (socket says 172.21.15.151:48294) 2024-04-26T01:58:58.810 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.810+0000 7f04e9200640 1 -- 172.21.15.151:0/778643251 learned_addr learned my addr 172.21.15.151:0/778643251 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:58:58.810 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.810+0000 7f04ea600640 1 --2- 172.21.15.151:0/778643251 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f04e419a700 0x7f04e419ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:58.810 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.810+0000 7f04e9c00640 1 --2- 172.21.15.151:0/778643251 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04e41043c0 0x7f04e41a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:58.810 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.811+0000 7f04e9200640 1 -- 172.21.15.151:0/778643251 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f04e419a700 msgr2=0x7f04e419ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:58.811 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.811+0000 7f04e9200640 1 --2- 172.21.15.151:0/778643251 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f04e419a700 0x7f04e419ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.811 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.811+0000 7f04e9200640 1 -- 172.21.15.151:0/778643251 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04e41043c0 msgr2=0x7f04e41a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:58.811 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.811+0000 7f04e9200640 1 --2- 172.21.15.151:0/778643251 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04e41043c0 0x7f04e41a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.811 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.811+0000 7f04e9200640 1 -- 172.21.15.151:0/778643251 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e419b300 con 0x7f04e4109490 2024-04-26T01:58:58.811 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.811+0000 7f04e9c00640 1 --2- 172.21.15.151:0/778643251 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04e41043c0 0x7f04e41a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:58:58.811 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.811+0000 7f04ea600640 1 --2- 172.21.15.151:0/778643251 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f04e419a700 0x7f04e419ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:58:58.811 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.811+0000 7f04e9200640 1 --2- 172.21.15.151:0/778643251 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04e4109490 0x7f04e41a0b30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f04d0005ab0 tx=0x7f04d0003cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:58.812 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.811+0000 7f04e2000640 1 -- 172.21.15.151:0/778643251 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d003f070 con 0x7f04e4109490 2024-04-26T01:58:58.812 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.811+0000 7f04e2000640 1 -- 172.21.15.151:0/778643251 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04d0031d60 con 0x7f04e4109490 2024-04-26T01:58:58.812 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.811+0000 7f04ec600640 1 -- 172.21.15.151:0/778643251 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04e419b590 con 0x7f04e4109490 2024-04-26T01:58:58.812 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.812+0000 7f04e2000640 1 -- 172.21.15.151:0/778643251 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d0035030 con 0x7f04e4109490 2024-04-26T01:58:58.812 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.812+0000 7f04ec600640 1 -- 172.21.15.151:0/778643251 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04e41a76d0 con 0x7f04e4109490 2024-04-26T01:58:58.815 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.816+0000 7f04e2000640 1 -- 172.21.15.151:0/778643251 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f04d0042070 con 0x7f04e4109490 2024-04-26T01:58:58.815 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.816+0000 7f04ef425740 1 Processor -- start 2024-04-26T01:58:58.815 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.816+0000 7f04ef425740 1 -- start start 2024-04-26T01:58:58.816 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.816+0000 7f04ef425740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f2a4b9820 0x564f2a4b9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:58.816 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.816+0000 7f04ef425740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564f2a4ba140 0x564f2a4c44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:58.816 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.816+0000 7f04ef425740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564f2a4c4a00 0x564f2a4c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:58.816 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.816+0000 7f04ef425740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564f2a41f900 con 0x564f2a4c4a00 2024-04-26T01:58:58.816 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.816+0000 7f04ef425740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564f2a3c5ef0 con 0x564f2a4ba140 2024-04-26T01:58:58.816 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.816+0000 7f04ef425740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564f2a41b8c0 con 0x564f2a4b9820 2024-04-26T01:58:58.816 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.816+0000 7f04e9c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564f2a4ba140 0x564f2a4c44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:58.816 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.816+0000 7f04ea600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f2a4b9820 0x564f2a4b9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:58.816 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.817+0000 7f04e9200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564f2a4c4a00 0x564f2a4c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:58.816 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.817+0000 7f04ea600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f2a4b9820 0x564f2a4b9c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:35302/0 (socket says 172.21.15.151:35302) 2024-04-26T01:58:58.816 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.817+0000 7f04ea600640 1 -- 172.21.15.151:0/3087922521 learned_addr learned my addr 172.21.15.151:0/3087922521 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:58:58.816 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.817+0000 7f04e9c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564f2a4ba140 0x564f2a4c44c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:48304/0 (socket says 172.21.15.151:48304) 2024-04-26T01:58:58.818 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.817+0000 7f04ea600640 1 -- 172.21.15.151:0/3087922521 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564f2a4ba140 msgr2=0x564f2a4c44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:58.818 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.817+0000 7f04ea600640 1 --2- 172.21.15.151:0/3087922521 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564f2a4ba140 0x564f2a4c44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.818 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.817+0000 7f04ea600640 1 -- 172.21.15.151:0/3087922521 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564f2a4c4a00 msgr2=0x564f2a4c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:58.818 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.817+0000 7f04ea600640 1 --2- 172.21.15.151:0/3087922521 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564f2a4c4a00 0x564f2a4c6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.818 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.817+0000 7f04ea600640 1 -- 172.21.15.151:0/3087922521 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f2a132210 con 0x564f2a4b9820 2024-04-26T01:58:58.818 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.817+0000 7f04e9c00640 1 --2- 172.21.15.151:0/3087922521 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564f2a4ba140 0x564f2a4c44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:58:58.818 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.817+0000 7f04ea600640 1 --2- 172.21.15.151:0/3087922521 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f2a4b9820 0x564f2a4b9c00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f04dc010710 tx=0x7f04dc010bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2f57aac78bf13d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:58.819 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.817+0000 7f04d6a00640 1 -- 172.21.15.151:0/3087922521 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04dc00e8d0 con 0x564f2a4b9820 2024-04-26T01:58:58.819 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.817+0000 7f04d6a00640 1 -- 172.21.15.151:0/3087922521 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04dc012450 con 0x564f2a4b9820 2024-04-26T01:58:58.819 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.817+0000 7f04d6a00640 1 -- 172.21.15.151:0/3087922521 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04dc01c670 con 0x564f2a4b9820 2024-04-26T01:58:58.819 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.818+0000 7f04e2000640 1 --2- 172.21.15.151:0/778643251 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f04cc0a3e40 0x7f04cc0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:58.819 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.818+0000 7f04ef425740 1 -- 172.21.15.151:0/3087922521 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f2a4b9820 msgr2=0x564f2a4b9c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:58.819 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.818+0000 7f04ef425740 1 --2- 172.21.15.151:0/3087922521 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f2a4b9820 0x564f2a4b9c00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f04dc010710 tx=0x7f04dc010bd0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.819 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.818+0000 7f04e2000640 1 -- 172.21.15.151:0/778643251 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7f04d00f0f40 con 0x7f04e4109490 2024-04-26T01:58:58.819 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.818+0000 7f04e9c00640 1 --2- 172.21.15.151:0/778643251 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f04cc0a3e40 0x7f04cc0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:58.819 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.818+0000 7f04ef425740 1 -- 172.21.15.151:0/3087922521 shutdown_connections 2024-04-26T01:58:58.819 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.818+0000 7f04ef425740 1 --2- 172.21.15.151:0/3087922521 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564f2a4c4a00 0x564f2a4c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.819 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.818+0000 7f04ef425740 1 --2- 172.21.15.151:0/3087922521 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564f2a4ba140 0x564f2a4c44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.819 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.818+0000 7f04ef425740 1 --2- 172.21.15.151:0/3087922521 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f2a4b9820 0x564f2a4b9c00 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.819 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.818+0000 7f04ef425740 1 -- 172.21.15.151:0/3087922521 >> 172.21.15.151:0/3087922521 conn(0x564f2a4b5d30 msgr2=0x564f2a4b6110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:58.820 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.819+0000 7f04ef425740 1 -- 172.21.15.151:0/3087922521 shutdown_connections 2024-04-26T01:58:58.820 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.819+0000 7f04ef425740 1 -- 172.21.15.151:0/3087922521 wait complete. 2024-04-26T01:58:58.820 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.819+0000 7f04ef425740 1 -- start start 2024-04-26T01:58:58.820 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.819+0000 7f04e9c00640 1 --2- 172.21.15.151:0/778643251 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f04cc0a3e40 0x7f04cc0a6300 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x564f2a4badc0 tx=0x7f04d800b570 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:58.820 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.819+0000 7f04ef425740 1 Processor -- start 2024-04-26T01:58:58.823 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.820+0000 7f04ef425740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f2a4b9820 0x564f2a4c1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:58.823 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.820+0000 7f04ef425740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564f2a4ba140 0x564f2a4c1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:58.823 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.820+0000 7f04ef425740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564f2a4c4a00 0x564f2a4c2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:58.823 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.820+0000 7f04ef425740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564f2a4c7630 con 0x564f2a4ba140 2024-04-26T01:58:58.823 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.820+0000 7f04ef425740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564f2a3d9310 con 0x564f2a4c4a00 2024-04-26T01:58:58.824 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.820+0000 7f04ef425740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564f2a4c77b0 con 0x564f2a4b9820 2024-04-26T01:58:58.824 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.820+0000 7f04ef425740 10 client.0 _ll_register_callbacks cb 0x564f2a3a7cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:58:58.824 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.820+0000 7f04e9200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564f2a4ba140 0x564f2a4c1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:58.824 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.820+0000 7f04e9c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f2a4b9820 0x564f2a4c1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:58.824 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.820+0000 7f04e9c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f2a4b9820 0x564f2a4c1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:35312/0 (socket says 172.21.15.151:35312) 2024-04-26T01:58:58.824 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.820+0000 7f04e9c00640 1 -- 172.21.15.151:0/2896228916 learned_addr learned my addr 172.21.15.151:0/2896228916 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:58:58.824 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.820+0000 7f04ea600640 1 --2- 172.21.15.151:0/2896228916 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564f2a4c4a00 0x564f2a4c2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:58.824 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.821+0000 7f04e9c00640 1 -- 172.21.15.151:0/2896228916 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564f2a4c4a00 msgr2=0x564f2a4c2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:58.824 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.821+0000 7f04e9c00640 1 --2- 172.21.15.151:0/2896228916 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564f2a4c4a00 0x564f2a4c2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.824 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.821+0000 7f04e9c00640 1 -- 172.21.15.151:0/2896228916 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564f2a4ba140 msgr2=0x564f2a4c1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:58.824 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.821+0000 7f04e9c00640 1 --2- 172.21.15.151:0/2896228916 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564f2a4ba140 0x564f2a4c1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.824 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.821+0000 7f04e9c00640 1 -- 172.21.15.151:0/2896228916 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f2a1fe620 con 0x564f2a4b9820 2024-04-26T01:58:58.824 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.821+0000 7f04e9c00640 1 --2- 172.21.15.151:0/2896228916 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f2a4b9820 0x564f2a4c1a00 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f04d8005bc0 tx=0x7f04d80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62daa85e775eb476 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:58.824 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.821+0000 7f04cb400640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-26T01:58:58.825 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.821+0000 7f04cb400640 1 -- 172.21.15.151:0/2896228916 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d801d830 con 0x564f2a4b9820 2024-04-26T01:58:58.825 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.821+0000 7f04cb400640 1 -- 172.21.15.151:0/2896228916 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7f04d801d9d0 con 0x564f2a4b9820 2024-04-26T01:58:58.825 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.821+0000 7f04ef425740 1 -- 172.21.15.151:0/2896228916 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04dc012450 con 0x564f2a4b9820 2024-04-26T01:58:58.825 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.822+0000 7f04cb400640 1 -- 172.21.15.151:0/2896228916 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f04d800be50 con 0x564f2a4b9820 2024-04-26T01:58:58.825 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.822+0000 7f04ef425740 10 client.25804 fetch_fsmap learned FSMap version 32 2024-04-26T01:58:58.825 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.822+0000 7f04ef425740 1 -- 172.21.15.151:0/2896228916 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564f2a4d3950 con 0x564f2a4b9820 2024-04-26T01:58:58.825 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.822+0000 7f04cb400640 1 -- 172.21.15.151:0/2896228916 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04dc012450 con 0x564f2a4b9820 2024-04-26T01:58:58.825 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.822+0000 7f04cb400640 1 -- 172.21.15.151:0/2896228916 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f04d802f050 con 0x564f2a4b9820 2024-04-26T01:58:58.825 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.822+0000 7f04ef425740 10 client.25804 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T01:58:58.825 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.822+0000 7f04ef425740 10 client.25804 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:58:58.825 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.822+0000 7f04ef425740 20 client.25804 populate_metadata read hostname 'smithi151' 2024-04-26T01:58:58.825 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.822+0000 7f04ef425740 1 --2- 172.21.15.151:0/2896228916 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x564f2a4d4e10 0x564f2a4d7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:58.825 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.822+0000 7f04ef425740 4 client.25804 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:58:58.825 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.822+0000 7f04ef425740 1 -- 172.21.15.151:0/2896228916 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f2a4338b0 con 0x564f2a4d4e10 2024-04-26T01:58:58.825 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.823+0000 7f04e9200640 1 --2- 172.21.15.151:0/2896228916 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x564f2a4d4e10 0x564f2a4d7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:58.826 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.823+0000 7f04e9200640 1 --2- 172.21.15.151:0/2896228916 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x564f2a4d4e10 0x564f2a4d7200 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2e556c9d1b7dd35 server_cookie=50eb5c4354408f9d in_seq=0 out_seq=0 2024-04-26T01:58:58.826 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.823+0000 7f04cb400640 10 client.25804 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:58:58.826 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.824+0000 7f04cb400640 1 -- 172.21.15.151:0/2896228916 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564f2a4338b0 con 0x564f2a4d4e10 2024-04-26T01:58:58.826 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.824+0000 7f04cb400640 10 client.25804 handle_command_reply: tid=0 2024-04-26T01:58:58.826 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.824+0000 7f04ef425740 1 client.25804 shutdown 2024-04-26T01:58:58.826 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.824+0000 7f04ef425740 1 -- 172.21.15.151:0/2896228916 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f2a4b9820 msgr2=0x564f2a4c1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:58.826 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.824+0000 7f04ef425740 1 --2- 172.21.15.151:0/2896228916 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f2a4b9820 0x564f2a4c1a00 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f04d8005bc0 tx=0x7f04d80046c0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.826 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.824+0000 7f04ef425740 1 -- 172.21.15.151:0/2896228916 shutdown_connections 2024-04-26T01:58:58.826 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.824+0000 7f04ef425740 1 --2- 172.21.15.151:0/2896228916 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x564f2a4d4e10 0x564f2a4d7200 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.826 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.824+0000 7f04ef425740 1 --2- 172.21.15.151:0/2896228916 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564f2a4c4a00 0x564f2a4c2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.826 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.824+0000 7f04cb400640 0 client.25804 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:58:58.826 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.824+0000 7f04ef425740 1 --2- 172.21.15.151:0/2896228916 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564f2a4ba140 0x564f2a4c1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.826 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.824+0000 7f04ef425740 1 --2- 172.21.15.151:0/2896228916 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f2a4b9820 0x564f2a4c1a00 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.826 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.824+0000 7f04ef425740 1 -- 172.21.15.151:0/2896228916 >> 172.21.15.151:0/2896228916 conn(0x564f2a4b5d30 msgr2=0x564f2a4bba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:58.827 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.824+0000 7f04ef425740 1 -- 172.21.15.151:0/2896228916 shutdown_connections 2024-04-26T01:58:58.827 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.824+0000 7f04ef425740 1 -- 172.21.15.151:0/2896228916 wait complete. 2024-04-26T01:58:58.827 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.825+0000 7f04ef425740 20 client.25804 trim_cache size 0 max 16384 2024-04-26T01:58:58.833 INFO:tasks.workunit.client.0.smithi151.stdout:copying to largest ever: 0x9fdd84 2024-04-26T01:58:58.845 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.844+0000 7f04ef425740 1 Processor -- start 2024-04-26T01:58:58.845 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.845+0000 7f04ef425740 1 -- start start 2024-04-26T01:58:58.845 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.845+0000 7f04ef425740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f2a4b9820 0x564f2a4b5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:58.845 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.845+0000 7f04ef425740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564f2a4ba140 0x564f2a4bc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:58.845 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.845+0000 7f04ef425740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564f2a4c4a00 0x564f2a4dcb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:58.845 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.845+0000 7f04ef425740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f04e41b2490 con 0x564f2a4c4a00 2024-04-26T01:58:58.845 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.845+0000 7f04ef425740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f04e41afd60 con 0x564f2a4ba140 2024-04-26T01:58:58.846 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.845+0000 7f04ef425740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f04e41adc30 con 0x564f2a4b9820 2024-04-26T01:58:58.846 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.845+0000 7f04e9c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f2a4b9820 0x564f2a4b5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:58.846 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.845+0000 7f04e9200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564f2a4ba140 0x564f2a4bc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:58.846 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.845+0000 7f04e9200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564f2a4ba140 0x564f2a4bc1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:48330/0 (socket says 172.21.15.151:48330) 2024-04-26T01:58:58.846 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.845+0000 7f04e9200640 1 -- 172.21.15.151:0/1912703550 learned_addr learned my addr 172.21.15.151:0/1912703550 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:58:58.846 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.845+0000 7f04e9c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f2a4b9820 0x564f2a4b5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:35328/0 (socket says 172.21.15.151:35328) 2024-04-26T01:58:58.846 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.845+0000 7f04ea600640 1 --2- 172.21.15.151:0/1912703550 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564f2a4c4a00 0x564f2a4dcb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:58.846 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.846+0000 7f04e9200640 1 -- 172.21.15.151:0/1912703550 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f2a4b9820 msgr2=0x564f2a4b5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:58.846 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.846+0000 7f04e9200640 1 --2- 172.21.15.151:0/1912703550 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f2a4b9820 0x564f2a4b5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.846 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.846+0000 7f04e9200640 1 -- 172.21.15.151:0/1912703550 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564f2a4c4a00 msgr2=0x564f2a4dcb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:58.846 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.846+0000 7f04e9200640 1 --2- 172.21.15.151:0/1912703550 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564f2a4c4a00 0x564f2a4dcb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.846 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.846+0000 7f04e9200640 1 -- 172.21.15.151:0/1912703550 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f2a4c09d0 con 0x564f2a4ba140 2024-04-26T01:58:58.847 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.846+0000 7f04ea600640 1 --2- 172.21.15.151:0/1912703550 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564f2a4c4a00 0x564f2a4dcb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:58:58.847 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.846+0000 7f04e9200640 1 --2- 172.21.15.151:0/1912703550 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564f2a4ba140 0x564f2a4bc1e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f04d00b9350 tx=0x7f04d0000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a1f8dead2bfba62 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:58.847 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.846+0000 7f04d6000640 1 -- 172.21.15.151:0/1912703550 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d0049040 con 0x564f2a4ba140 2024-04-26T01:58:58.847 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.846+0000 7f04d6000640 1 -- 172.21.15.151:0/1912703550 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04d00bd5f0 con 0x564f2a4ba140 2024-04-26T01:58:58.847 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.846+0000 7f04d6000640 1 -- 172.21.15.151:0/1912703550 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d00bb280 con 0x564f2a4ba140 2024-04-26T01:58:58.849 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.847+0000 7f04ef425740 1 -- 172.21.15.151:0/1912703550 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564f2a4ba140 msgr2=0x564f2a4bc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:58.849 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.847+0000 7f04ef425740 1 --2- 172.21.15.151:0/1912703550 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564f2a4ba140 0x564f2a4bc1e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f04d00b9350 tx=0x7f04d0000bc0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.850 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.847+0000 7f04ef425740 1 -- 172.21.15.151:0/1912703550 shutdown_connections 2024-04-26T01:58:58.850 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.847+0000 7f04ef425740 1 --2- 172.21.15.151:0/1912703550 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564f2a4c4a00 0x564f2a4dcb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.850 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.847+0000 7f04ef425740 1 --2- 172.21.15.151:0/1912703550 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564f2a4ba140 0x564f2a4bc1e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.850 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.847+0000 7f04ef425740 1 --2- 172.21.15.151:0/1912703550 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f2a4b9820 0x564f2a4b5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.850 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.847+0000 7f04ef425740 1 -- 172.21.15.151:0/1912703550 >> 172.21.15.151:0/1912703550 conn(0x564f2a4b5d30 msgr2=0x564f2a4c2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:58.850 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.847+0000 7f04ef425740 1 -- 172.21.15.151:0/1912703550 shutdown_connections 2024-04-26T01:58:58.850 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.847+0000 7f04ef425740 1 -- 172.21.15.151:0/1912703550 wait complete. 2024-04-26T01:58:58.850 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.848+0000 7f04ef425740 1 -- start start 2024-04-26T01:58:58.850 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.848+0000 7f04ef425740 1 Processor -- start 2024-04-26T01:58:58.850 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.848+0000 7f04ef425740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564f2a4b9820 0x564f2a4ca540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:58.850 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.848+0000 7f04ef425740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564f2a4ba140 0x564f2a4caa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:58.850 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.848+0000 7f04ef425740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f2a4c4a00 0x564f2a4cafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:58.851 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.848+0000 7f04ef425740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f04e41ae280 con 0x564f2a4ba140 2024-04-26T01:58:58.851 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.848+0000 7f04ef425740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f04e41af690 con 0x564f2a4b9820 2024-04-26T01:58:58.851 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.848+0000 7f04ef425740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f04e41ae8f0 con 0x564f2a4c4a00 2024-04-26T01:58:58.851 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.848+0000 7f04ef425740 10 client.0 _ll_register_callbacks cb 0x7f04d8004100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:58:58.851 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.848+0000 7f04e9c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564f2a4b9820 0x564f2a4ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:58.851 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.848+0000 7f04e9c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564f2a4b9820 0x564f2a4ca540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:48334/0 (socket says 172.21.15.151:48334) 2024-04-26T01:58:58.851 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.848+0000 7f04e9c00640 1 -- 172.21.15.151:0/1251680906 learned_addr learned my addr 172.21.15.151:0/1251680906 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:58:58.851 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.849+0000 7f04e9200640 1 --2- 172.21.15.151:0/1251680906 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564f2a4ba140 0x564f2a4caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:58.851 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.849+0000 7f04ea600640 1 --2- 172.21.15.151:0/1251680906 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f2a4c4a00 0x564f2a4cafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:58.851 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.849+0000 7f04e9c00640 1 -- 172.21.15.151:0/1251680906 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f2a4c4a00 msgr2=0x564f2a4cafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:58.851 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.849+0000 7f04e9c00640 1 --2- 172.21.15.151:0/1251680906 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f2a4c4a00 0x564f2a4cafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.851 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.849+0000 7f04e9c00640 1 -- 172.21.15.151:0/1251680906 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564f2a4ba140 msgr2=0x564f2a4caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:58.851 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.849+0000 7f04e9c00640 1 --2- 172.21.15.151:0/1251680906 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564f2a4ba140 0x564f2a4caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.852 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.849+0000 7f04e9c00640 1 -- 172.21.15.151:0/1251680906 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f2a53a490 con 0x564f2a4b9820 2024-04-26T01:58:58.852 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.849+0000 7f04e9c00640 1 --2- 172.21.15.151:0/1251680906 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564f2a4b9820 0x564f2a4ca540 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f04d8005b90 tx=0x7f04d8012d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56ccdbab6ba1a1b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:58.852 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.849+0000 7f04cb400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T01:58:58.852 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.849+0000 7f04cb400640 1 -- 172.21.15.151:0/1251680906 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d8027420 con 0x564f2a4b9820 2024-04-26T01:58:58.852 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.850+0000 7f04ef425740 1 -- 172.21.15.151:0/1251680906 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04d00bd5f0 con 0x564f2a4b9820 2024-04-26T01:58:58.852 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.850+0000 7f04cb400640 1 -- 172.21.15.151:0/1251680906 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7f04d80275c0 con 0x564f2a4b9820 2024-04-26T01:58:58.852 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.850+0000 7f04cb400640 1 -- 172.21.15.151:0/1251680906 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f04d8041050 con 0x564f2a4b9820 2024-04-26T01:58:58.852 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.850+0000 7f04ef425740 10 client.25769 fetch_fsmap learned FSMap version 32 2024-04-26T01:58:58.852 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.851+0000 7f04ef425740 1 -- 172.21.15.151:0/1251680906 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564f2a53b7a0 con 0x564f2a4b9820 2024-04-26T01:58:58.853 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.851+0000 7f04cb400640 1 -- 172.21.15.151:0/1251680906 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04d00bd5f0 con 0x564f2a4b9820 2024-04-26T01:58:58.853 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.851+0000 7f04cb400640 1 -- 172.21.15.151:0/1251680906 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f04d802f040 con 0x564f2a4b9820 2024-04-26T01:58:58.853 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.851+0000 7f04ef425740 10 client.25769 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T01:58:58.853 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.851+0000 7f04ef425740 10 client.25769 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:58:58.853 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.852+0000 7f04ef425740 20 client.25769 populate_metadata read hostname 'smithi151' 2024-04-26T01:58:58.853 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.852+0000 7f04ef425740 1 --2- 172.21.15.151:0/1251680906 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x564f2a53be90 0x564f2a53e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:58.853 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.852+0000 7f04ef425740 4 client.25769 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:58:58.853 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.852+0000 7f04ef425740 1 -- 172.21.15.151:0/1251680906 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564f2a42d320 con 0x564f2a53be90 2024-04-26T01:58:58.854 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.852+0000 7f04e9200640 1 --2- 172.21.15.151:0/1251680906 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x564f2a53be90 0x564f2a53e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:58.854 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.853+0000 7f04e9200640 1 --2- 172.21.15.151:0/1251680906 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x564f2a53be90 0x564f2a53e2b0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e4d49114ebb744b server_cookie=3c530bdfef15cfed in_seq=0 out_seq=0 2024-04-26T01:58:58.854 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.853+0000 7f04cb400640 10 client.25769 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:58:58.865 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.866+0000 7f04cb400640 1 -- 172.21.15.151:0/1251680906 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564f2a42d320 con 0x564f2a53be90 2024-04-26T01:58:58.865 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.866+0000 7f04cb400640 10 client.25769 handle_command_reply: tid=0 2024-04-26T01:58:58.865 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.866+0000 7f04ef425740 1 client.25769 shutdown 2024-04-26T01:58:58.866 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.867+0000 7f04ef425740 1 -- 172.21.15.151:0/1251680906 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564f2a4b9820 msgr2=0x564f2a4ca540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:58.866 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.867+0000 7f04ef425740 1 --2- 172.21.15.151:0/1251680906 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564f2a4b9820 0x564f2a4ca540 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f04d8005b90 tx=0x7f04d8012d50 comp rx=0 tx=0).stop 2024-04-26T01:58:58.866 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.867+0000 7f04ef425740 1 -- 172.21.15.151:0/1251680906 shutdown_connections 2024-04-26T01:58:58.866 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.867+0000 7f04ef425740 1 --2- 172.21.15.151:0/1251680906 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x564f2a53be90 0x564f2a53e2b0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.867+0000 7f04ef425740 1 --2- 172.21.15.151:0/1251680906 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f2a4c4a00 0x564f2a4cafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.867+0000 7f04ef425740 1 --2- 172.21.15.151:0/1251680906 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564f2a4ba140 0x564f2a4caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.867+0000 7f04ef425740 1 --2- 172.21.15.151:0/1251680906 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564f2a4b9820 0x564f2a4ca540 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.867+0000 7f04ef425740 1 -- 172.21.15.151:0/1251680906 >> 172.21.15.151:0/1251680906 conn(0x564f2a4b5d30 msgr2=0x564f2a4c3230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.867+0000 7f04cb400640 0 client.25769 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:58:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.867+0000 7f04ef425740 1 -- 172.21.15.151:0/1251680906 shutdown_connections 2024-04-26T01:58:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.867+0000 7f04ef425740 1 -- 172.21.15.151:0/1251680906 wait complete. 2024-04-26T01:58:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.867+0000 7f04ef425740 20 client.25769 trim_cache size 0 max 16384 2024-04-26T01:58:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:{ 2024-04-26T01:58:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "message": "", 2024-04-26T01:58:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "return_code": 0 2024-04-26T01:58:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:} 2024-04-26T01:58:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.868+0000 7f04ec600640 1 -- 172.21.15.151:0/778643251 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f04cc0a3e40 msgr2=0x7f04cc0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.868+0000 7f04ec600640 1 --2- 172.21.15.151:0/778643251 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f04cc0a3e40 0x7f04cc0a6300 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x564f2a4badc0 tx=0x7f04d800b570 comp rx=0 tx=0).stop 2024-04-26T01:58:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.868+0000 7f04ec600640 1 -- 172.21.15.151:0/778643251 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04e4109490 msgr2=0x7f04e41a0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:58.869 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.868+0000 7f04ec600640 1 --2- 172.21.15.151:0/778643251 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04e4109490 0x7f04e41a0b30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f04d0005ab0 tx=0x7f04d0003cd0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.869 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.868+0000 7f04ec600640 1 -- 172.21.15.151:0/778643251 shutdown_connections 2024-04-26T01:58:58.869 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.868+0000 7f04ec600640 1 --2- 172.21.15.151:0/778643251 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f04cc0a3e40 0x7f04cc0a6300 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.869 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.868+0000 7f04ec600640 1 --2- 172.21.15.151:0/778643251 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f04e419a700 0x7f04e419ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.869 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.869+0000 7f04ec600640 1 --2- 172.21.15.151:0/778643251 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04e4109490 0x7f04e41a0b30 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.869 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.869+0000 7f04ec600640 1 --2- 172.21.15.151:0/778643251 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04e41043c0 0x7f04e41a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.869 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.869+0000 7f04ec600640 1 -- 172.21.15.151:0/778643251 >> 172.21.15.151:0/778643251 conn(0x7f04e40ffc20 msgr2=0x7f04e4101f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:58.869 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.869+0000 7f04ec600640 1 -- 172.21.15.151:0/778643251 shutdown_connections 2024-04-26T01:58:58.869 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:58:58.869+0000 7f04ec600640 1 -- 172.21.15.151:0/778643251 wait complete. 2024-04-26T01:59:00.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:59:00 smithi172 ceph-mon[32785]: pgmap v165: 97 pgs: 97 active+clean; 44 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.0 MiB/s wr, 344 op/s 2024-04-26T01:59:00.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:59:00 smithi165 ceph-mon[32648]: pgmap v165: 97 pgs: 97 active+clean; 44 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.0 MiB/s wr, 344 op/s 2024-04-26T01:59:00.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:59:00 smithi151 ceph-mon[28057]: pgmap v165: 97 pgs: 97 active+clean; 44 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.0 MiB/s wr, 344 op/s 2024-04-26T01:59:02.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:59:02 smithi172 ceph-mon[32785]: pgmap v166: 97 pgs: 97 active+clean; 44 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.4 MiB/s wr, 234 op/s 2024-04-26T01:59:02.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:59:02 smithi165 ceph-mon[32648]: pgmap v166: 97 pgs: 97 active+clean; 44 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.4 MiB/s wr, 234 op/s 2024-04-26T01:59:02.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:59:02 smithi151 ceph-mon[28057]: pgmap v166: 97 pgs: 97 active+clean; 44 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.4 MiB/s wr, 234 op/s 2024-04-26T01:59:04.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:59:04 smithi172 ceph-mon[32785]: pgmap v167: 97 pgs: 97 active+clean; 42 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.6 MiB/s wr, 280 op/s 2024-04-26T01:59:04.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:59:04 smithi165 ceph-mon[32648]: pgmap v167: 97 pgs: 97 active+clean; 42 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.6 MiB/s wr, 280 op/s 2024-04-26T01:59:04.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:59:04 smithi151 ceph-mon[28057]: pgmap v167: 97 pgs: 97 active+clean; 42 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.6 MiB/s wr, 280 op/s 2024-04-26T01:59:06.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:59:06 smithi172 ceph-mon[32785]: pgmap v168: 97 pgs: 97 active+clean; 50 MiB data, 848 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.1 MiB/s wr, 359 op/s 2024-04-26T01:59:06.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:59:06 smithi165 ceph-mon[32648]: pgmap v168: 97 pgs: 97 active+clean; 50 MiB data, 848 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.1 MiB/s wr, 359 op/s 2024-04-26T01:59:06.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:59:06 smithi151 ceph-mon[28057]: pgmap v168: 97 pgs: 97 active+clean; 50 MiB data, 848 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.1 MiB/s wr, 359 op/s 2024-04-26T01:59:08.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:59:08 smithi172 ceph-mon[32785]: pgmap v169: 97 pgs: 97 active+clean; 50 MiB data, 848 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.6 MiB/s wr, 292 op/s 2024-04-26T01:59:08.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:59:08 smithi165 ceph-mon[32648]: pgmap v169: 97 pgs: 97 active+clean; 50 MiB data, 848 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.6 MiB/s wr, 292 op/s 2024-04-26T01:59:08.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:59:08 smithi151 ceph-mon[28057]: pgmap v169: 97 pgs: 97 active+clean; 50 MiB data, 848 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.6 MiB/s wr, 292 op/s 2024-04-26T01:59:10.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:59:10 smithi172 ceph-mon[32785]: pgmap v170: 97 pgs: 97 active+clean; 53 MiB data, 834 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.3 MiB/s wr, 420 op/s 2024-04-26T01:59:10.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:59:10 smithi165 ceph-mon[32648]: pgmap v170: 97 pgs: 97 active+clean; 53 MiB data, 834 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.3 MiB/s wr, 420 op/s 2024-04-26T01:59:10.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:59:10 smithi151 ceph-mon[28057]: pgmap v170: 97 pgs: 97 active+clean; 53 MiB data, 834 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.3 MiB/s wr, 420 op/s 2024-04-26T01:59:12.025 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.026+0000 7f7ad6e00640 1 -- 172.21.15.151:0/669530243 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7ad0102b80 msgr2=0x7f7ad0106bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:12.025 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.026+0000 7f7ad6e00640 1 --2- 172.21.15.151:0/669530243 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7ad0102b80 0x7f7ad0106bc0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7abc009900 tx=0x7f7abc02a410 comp rx=0 tx=0).stop 2024-04-26T01:59:12.025 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.026+0000 7f7ad6e00640 1 -- 172.21.15.151:0/669530243 shutdown_connections 2024-04-26T01:59:12.025 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.026+0000 7f7ad6e00640 1 --2- 172.21.15.151:0/669530243 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ad01072a0 0x7f7ad010ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:12.026 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.026+0000 7f7ad6e00640 1 --2- 172.21.15.151:0/669530243 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7ad0102b80 0x7f7ad0106bc0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:12.026 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.026+0000 7f7ad6e00640 1 --2- 172.21.15.151:0/669530243 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ad01021d0 0x7f7ad01025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:12.026 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.026+0000 7f7ad6e00640 1 -- 172.21.15.151:0/669530243 >> 172.21.15.151:0/669530243 conn(0x7f7ad00793e0 msgr2=0x7f7ad00797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:12.026 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.026+0000 7f7ad6e00640 1 -- 172.21.15.151:0/669530243 shutdown_connections 2024-04-26T01:59:12.026 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.026+0000 7f7ad6e00640 1 -- 172.21.15.151:0/669530243 wait complete. 2024-04-26T01:59:12.028 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.027+0000 7f7ad6e00640 1 Processor -- start 2024-04-26T01:59:12.028 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.027+0000 7f7ad6e00640 1 -- start start 2024-04-26T01:59:12.028 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.028+0000 7f7ad6e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ad01021d0 0x7f7ad0198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:12.028 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.028+0000 7f7ad6e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7ad01072a0 0x7f7ad0198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:12.028 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.028+0000 7f7ad6e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ad019cfa0 0x7f7ad019d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:12.028 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.028+0000 7f7ad6e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad0112230 con 0x7f7ad01021d0 2024-04-26T01:59:12.028 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.028+0000 7f7ad6e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad01120b0 con 0x7f7ad01072a0 2024-04-26T01:59:12.028 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.028+0000 7f7ad6e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad01123b0 con 0x7f7ad019cfa0 2024-04-26T01:59:12.028 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.028+0000 7f7acf400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7ad01072a0 0x7f7ad0198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:12.028 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.028+0000 7f7acf400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7ad01072a0 0x7f7ad0198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:55206/0 (socket says 172.21.15.151:55206) 2024-04-26T01:59:12.028 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.028+0000 7f7acf400640 1 -- 172.21.15.151:0/1287064209 learned_addr learned my addr 172.21.15.151:0/1287064209 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:59:12.028 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.028+0000 7f7acfe00640 1 --2- 172.21.15.151:0/1287064209 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ad01021d0 0x7f7ad0198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:12.029 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.028+0000 7f7ad4e00640 1 --2- 172.21.15.151:0/1287064209 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ad019cfa0 0x7f7ad019d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:12.029 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.029+0000 7f7acf400640 1 -- 172.21.15.151:0/1287064209 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ad019cfa0 msgr2=0x7f7ad019d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:12.029 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.029+0000 7f7acf400640 1 --2- 172.21.15.151:0/1287064209 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ad019cfa0 0x7f7ad019d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:12.029 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.029+0000 7f7acf400640 1 -- 172.21.15.151:0/1287064209 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ad01021d0 msgr2=0x7f7ad0198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:12.029 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.029+0000 7f7acf400640 1 --2- 172.21.15.151:0/1287064209 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ad01021d0 0x7f7ad0198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:12.029 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.029+0000 7f7acf400640 1 -- 172.21.15.151:0/1287064209 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ad019da90 con 0x7f7ad01072a0 2024-04-26T01:59:12.029 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.029+0000 7f7acfe00640 1 --2- 172.21.15.151:0/1287064209 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ad01021d0 0x7f7ad0198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:59:12.029 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.029+0000 7f7acf400640 1 --2- 172.21.15.151:0/1287064209 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7ad01072a0 0x7f7ad0198c70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7abc005ab0 tx=0x7f7abc003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:12.030 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.029+0000 7f7accc00640 1 -- 172.21.15.151:0/1287064209 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7abc03f070 con 0x7f7ad01072a0 2024-04-26T01:59:12.030 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.029+0000 7f7accc00640 1 -- 172.21.15.151:0/1287064209 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7abc031b50 con 0x7f7ad01072a0 2024-04-26T01:59:12.030 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.029+0000 7f7accc00640 1 -- 172.21.15.151:0/1287064209 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7abc035030 con 0x7f7ad01072a0 2024-04-26T01:59:12.030 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.029+0000 7f7ad6e00640 1 -- 172.21.15.151:0/1287064209 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ad019dd20 con 0x7f7ad01072a0 2024-04-26T01:59:12.030 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.030+0000 7f7ad6e00640 1 -- 172.21.15.151:0/1287064209 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ad01a5540 con 0x7f7ad01072a0 2024-04-26T01:59:12.032 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.033+0000 7f7accc00640 1 -- 172.21.15.151:0/1287064209 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f7abc042070 con 0x7f7ad01072a0 2024-04-26T01:59:12.032 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.034+0000 7f7ad9c0b740 1 Processor -- start 2024-04-26T01:59:12.032 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.034+0000 7f7ad9c0b740 1 -- start start 2024-04-26T01:59:12.034 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.034+0000 7f7ad9c0b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55686173d820 0x55686173dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:12.034 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.034+0000 7f7ad9c0b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55686173e140 0x5568617484c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:12.034 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.034+0000 7f7ad9c0b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556861748a00 0x55686174adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:12.034 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.034+0000 7f7ad9c0b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5568616a3900 con 0x55686173d820 2024-04-26T01:59:12.034 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.034+0000 7f7ad9c0b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556861649ef0 con 0x556861748a00 2024-04-26T01:59:12.034 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.034+0000 7f7ad9c0b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55686169f8c0 con 0x55686173e140 2024-04-26T01:59:12.034 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.034+0000 7f7acf400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556861748a00 0x55686174adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:12.034 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.035+0000 7f7acf400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556861748a00 0x55686174adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:55216/0 (socket says 172.21.15.151:55216) 2024-04-26T01:59:12.035 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.035+0000 7f7acf400640 1 -- 172.21.15.151:0/175362355 learned_addr learned my addr 172.21.15.151:0/175362355 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:59:12.035 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.035+0000 7f7acfe00640 1 --2- 172.21.15.151:0/175362355 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55686173e140 0x5568617484c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:12.035 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.035+0000 7f7ad4e00640 1 --2- 172.21.15.151:0/175362355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55686173d820 0x55686173dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:12.035 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.035+0000 7f7acf400640 1 -- 172.21.15.151:0/175362355 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55686173e140 msgr2=0x5568617484c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:12.035 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.035+0000 7f7acf400640 1 --2- 172.21.15.151:0/175362355 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55686173e140 0x5568617484c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:12.035 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.035+0000 7f7acf400640 1 -- 172.21.15.151:0/175362355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55686173d820 msgr2=0x55686173dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:12.035 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.035+0000 7f7acf400640 1 --2- 172.21.15.151:0/175362355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55686173d820 0x55686173dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:12.035 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.035+0000 7f7acf400640 1 -- 172.21.15.151:0/175362355 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568613b6210 con 0x556861748a00 2024-04-26T01:59:12.035 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.035+0000 7f7ad4e00640 1 --2- 172.21.15.151:0/175362355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55686173d820 0x55686173dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:59:12.035 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.035+0000 7f7acfe00640 1 --2- 172.21.15.151:0/175362355 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55686173e140 0x5568617484c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:59:12.035 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.035+0000 7f7accc00640 1 --2- 172.21.15.151:0/1287064209 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f7ab40a3e40 0x7f7ab40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:12.035 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.035+0000 7f7acf400640 1 --2- 172.21.15.151:0/175362355 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556861748a00 0x55686174adf0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7abc0ee500 tx=0x7f7abc0f90e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d7c924bc4e09476 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:12.037 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.036+0000 7f7ad4e00640 1 --2- 172.21.15.151:0/1287064209 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f7ab40a3e40 0x7f7ab40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:12.037 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.036+0000 7f7accc00640 1 -- 172.21.15.151:0/1287064209 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7f7abc0f0e60 con 0x7f7ad01072a0 2024-04-26T01:59:12.037 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.036+0000 7f7ac5600640 1 -- 172.21.15.151:0/175362355 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7abc0b90b0 con 0x556861748a00 2024-04-26T01:59:12.037 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.036+0000 7f7ac5600640 1 -- 172.21.15.151:0/175362355 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7abc0fe070 con 0x556861748a00 2024-04-26T01:59:12.037 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.036+0000 7f7ac5600640 1 -- 172.21.15.151:0/175362355 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7abc0bc770 con 0x556861748a00 2024-04-26T01:59:12.037 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.037+0000 7f7ad9c0b740 1 -- 172.21.15.151:0/175362355 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556861748a00 msgr2=0x55686174adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:12.037 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.037+0000 7f7ad9c0b740 1 --2- 172.21.15.151:0/175362355 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556861748a00 0x55686174adf0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7abc0ee500 tx=0x7f7abc0f90e0 comp rx=0 tx=0).stop 2024-04-26T01:59:12.037 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.037+0000 7f7ad4e00640 1 --2- 172.21.15.151:0/1287064209 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f7ab40a3e40 0x7f7ab40a6300 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7ac8005fd0 tx=0x7f7ac8009500 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:12.037 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.037+0000 7f7ad9c0b740 1 -- 172.21.15.151:0/175362355 shutdown_connections 2024-04-26T01:59:12.037 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.037+0000 7f7ad9c0b740 1 --2- 172.21.15.151:0/175362355 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556861748a00 0x55686174adf0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:12.037 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.037+0000 7f7ad9c0b740 1 --2- 172.21.15.151:0/175362355 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55686173e140 0x5568617484c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:12.037 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.037+0000 7f7ad9c0b740 1 --2- 172.21.15.151:0/175362355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55686173d820 0x55686173dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:12.037 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.037+0000 7f7ad9c0b740 1 -- 172.21.15.151:0/175362355 >> 172.21.15.151:0/175362355 conn(0x556861739d30 msgr2=0x55686173a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:12.038 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.037+0000 7f7ad9c0b740 1 -- 172.21.15.151:0/175362355 shutdown_connections 2024-04-26T01:59:12.038 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.037+0000 7f7ad9c0b740 1 -- 172.21.15.151:0/175362355 wait complete. 2024-04-26T01:59:12.038 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.038+0000 7f7ad9c0b740 1 -- start start 2024-04-26T01:59:12.038 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.038+0000 7f7ad9c0b740 1 Processor -- start 2024-04-26T01:59:12.040 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.039+0000 7f7ad9c0b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55686173d820 0x556861745a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:12.040 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.039+0000 7f7ad9c0b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55686173e140 0x556861745f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:12.040 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.039+0000 7f7ad9c0b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556861748a00 0x556861746480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:12.040 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.039+0000 7f7ad9c0b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55686174b3f0 con 0x556861748a00 2024-04-26T01:59:12.040 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.039+0000 7f7ad9c0b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55686165d310 con 0x55686173e140 2024-04-26T01:59:12.040 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.039+0000 7f7ad9c0b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55686174b570 con 0x55686173d820 2024-04-26T01:59:12.040 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.039+0000 7f7ad9c0b740 10 client.0 _ll_register_callbacks cb 0x55686162bcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:59:12.040 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.039+0000 7f7acfe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55686173d820 0x556861745a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:12.040 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.039+0000 7f7ad4e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556861748a00 0x556861746480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:12.040 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.039+0000 7f7acfe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55686173d820 0x556861745a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:41128/0 (socket says 172.21.15.151:41128) 2024-04-26T01:59:12.041 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.039+0000 7f7acfe00640 1 -- 172.21.15.151:0/2088703125 learned_addr learned my addr 172.21.15.151:0/2088703125 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:59:12.041 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.039+0000 7f7acf400640 1 --2- 172.21.15.151:0/2088703125 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55686173e140 0x556861745f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:12.041 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.039+0000 7f7acfe00640 1 -- 172.21.15.151:0/2088703125 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55686173e140 msgr2=0x556861745f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:12.041 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.039+0000 7f7acfe00640 1 --2- 172.21.15.151:0/2088703125 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55686173e140 0x556861745f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:12.041 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.039+0000 7f7acfe00640 1 -- 172.21.15.151:0/2088703125 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556861748a00 msgr2=0x556861746480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:12.041 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.039+0000 7f7acfe00640 1 --2- 172.21.15.151:0/2088703125 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556861748a00 0x556861746480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:12.041 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.039+0000 7f7acfe00640 1 -- 172.21.15.151:0/2088703125 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556861482620 con 0x55686173d820 2024-04-26T01:59:12.041 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.039+0000 7f7acf400640 1 --2- 172.21.15.151:0/2088703125 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55686173e140 0x556861745f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:59:12.041 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.040+0000 7f7acfe00640 1 --2- 172.21.15.151:0/2088703125 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55686173d820 0x556861745a00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0009860 tx=0x7f7ac0009d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94a683b2ad890ec4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:12.041 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.040+0000 7f7aba000640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-26T01:59:12.041 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.040+0000 7f7aba000640 1 -- 172.21.15.151:0/2088703125 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac0022060 con 0x55686173d820 2024-04-26T01:59:12.041 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.040+0000 7f7ad9c0b740 1 -- 172.21.15.151:0/2088703125 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7abc0fe070 con 0x55686173d820 2024-04-26T01:59:12.042 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.040+0000 7f7aba000640 1 -- 172.21.15.151:0/2088703125 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7f7ac000bcb0 con 0x55686173d820 2024-04-26T01:59:12.042 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.040+0000 7f7aba000640 1 -- 172.21.15.151:0/2088703125 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7ac0027440 con 0x55686173d820 2024-04-26T01:59:12.042 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.041+0000 7f7ad9c0b740 10 client.25822 fetch_fsmap learned FSMap version 32 2024-04-26T01:59:12.042 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.041+0000 7f7ad9c0b740 1 -- 172.21.15.151:0/2088703125 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556861757710 con 0x55686173d820 2024-04-26T01:59:12.042 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.041+0000 7f7aba000640 1 -- 172.21.15.151:0/2088703125 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7abc0fe070 con 0x55686173d820 2024-04-26T01:59:12.042 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.041+0000 7f7aba000640 1 -- 172.21.15.151:0/2088703125 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f7ac0025020 con 0x55686173d820 2024-04-26T01:59:12.042 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.041+0000 7f7ad9c0b740 10 client.25822 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T01:59:12.042 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.041+0000 7f7ad9c0b740 10 client.25822 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:59:12.042 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.041+0000 7f7ad9c0b740 20 client.25822 populate_metadata read hostname 'smithi151' 2024-04-26T01:59:12.042 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.041+0000 7f7ad9c0b740 1 --2- 172.21.15.151:0/2088703125 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x556861758b80 0x55686175af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:12.042 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.041+0000 7f7ad9c0b740 4 client.25822 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:59:12.042 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.041+0000 7f7ad9c0b740 1 -- 172.21.15.151:0/2088703125 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568616b78b0 con 0x556861758b80 2024-04-26T01:59:12.043 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.042+0000 7f7acf400640 1 --2- 172.21.15.151:0/2088703125 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x556861758b80 0x55686175af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:12.044 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.042+0000 7f7acf400640 1 --2- 172.21.15.151:0/2088703125 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x556861758b80 0x55686175af70 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee2e9bd6333b544f server_cookie=b9cc16cd24ac8bad in_seq=0 out_seq=0 2024-04-26T01:59:12.044 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.042+0000 7f7aba000640 10 client.25822 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:59:12.044 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.043+0000 7f7aba000640 1 -- 172.21.15.151:0/2088703125 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5568616b78b0 con 0x556861758b80 2024-04-26T01:59:12.044 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.043+0000 7f7aba000640 10 client.25822 handle_command_reply: tid=0 2024-04-26T01:59:12.044 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.043+0000 7f7ad9c0b740 1 client.25822 shutdown 2024-04-26T01:59:12.044 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.043+0000 7f7ad9c0b740 1 -- 172.21.15.151:0/2088703125 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55686173d820 msgr2=0x556861745a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:12.044 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.043+0000 7f7ad9c0b740 1 --2- 172.21.15.151:0/2088703125 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55686173d820 0x556861745a00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0009860 tx=0x7f7ac0009d20 comp rx=0 tx=0).stop 2024-04-26T01:59:12.044 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.043+0000 7f7ad9c0b740 1 -- 172.21.15.151:0/2088703125 shutdown_connections 2024-04-26T01:59:12.044 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.043+0000 7f7ad9c0b740 1 --2- 172.21.15.151:0/2088703125 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x556861758b80 0x55686175af70 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:12.044 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.043+0000 7f7ad9c0b740 1 --2- 172.21.15.151:0/2088703125 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556861748a00 0x556861746480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:12.044 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.043+0000 7f7ad9c0b740 1 --2- 172.21.15.151:0/2088703125 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55686173e140 0x556861745f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:12.044 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.043+0000 7f7ad9c0b740 1 --2- 172.21.15.151:0/2088703125 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55686173d820 0x556861745a00 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:12.044 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.043+0000 7f7aba000640 0 client.25822 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:59:12.044 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.043+0000 7f7ad9c0b740 1 -- 172.21.15.151:0/2088703125 >> 172.21.15.151:0/2088703125 conn(0x556861739d30 msgr2=0x55686173fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:12.045 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.044+0000 7f7ad9c0b740 1 -- 172.21.15.151:0/2088703125 shutdown_connections 2024-04-26T01:59:12.045 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.044+0000 7f7ad9c0b740 1 -- 172.21.15.151:0/2088703125 wait complete. 2024-04-26T01:59:12.045 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.044+0000 7f7ad9c0b740 20 client.25822 trim_cache size 0 max 16384 2024-04-26T01:59:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.065+0000 7f7ad9c0b740 1 Processor -- start 2024-04-26T01:59:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.065+0000 7f7ad9c0b740 1 -- start start 2024-04-26T01:59:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.065+0000 7f7ad9c0b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55686173d820 0x5568617541f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.065+0000 7f7ad9c0b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55686173e140 0x5568617401e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.065+0000 7f7ad9c0b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556861748a00 0x556861738d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.065+0000 7f7ad9c0b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad01ae030 con 0x55686173d820 2024-04-26T01:59:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.065+0000 7f7ad9c0b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad01ab900 con 0x55686173e140 2024-04-26T01:59:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.065+0000 7f7ad9c0b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad01a9790 con 0x556861748a00 2024-04-26T01:59:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.065+0000 7f7acfe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55686173d820 0x5568617541f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.065+0000 7f7acf400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55686173e140 0x5568617401e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.065+0000 7f7ad4e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556861748a00 0x556861738d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.065+0000 7f7ad4e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556861748a00 0x556861738d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:41142/0 (socket says 172.21.15.151:41142) 2024-04-26T01:59:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.065+0000 7f7ad4e00640 1 -- 172.21.15.151:0/3153755399 learned_addr learned my addr 172.21.15.151:0/3153755399 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:59:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.065+0000 7f7acfe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55686173d820 0x5568617541f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42456/0 (socket says 172.21.15.151:42456) 2024-04-26T01:59:12.070 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.066+0000 7f7acf400640 1 -- 172.21.15.151:0/3153755399 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556861748a00 msgr2=0x556861738d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:12.070 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.066+0000 7f7acf400640 1 --2- 172.21.15.151:0/3153755399 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556861748a00 0x556861738d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:12.070 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.066+0000 7f7acf400640 1 -- 172.21.15.151:0/3153755399 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55686173d820 msgr2=0x5568617541f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:12.070 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.066+0000 7f7acf400640 1 --2- 172.21.15.151:0/3153755399 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55686173d820 0x5568617541f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:12.070 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.066+0000 7f7acf400640 1 -- 172.21.15.151:0/3153755399 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55686173df70 con 0x55686173e140 2024-04-26T01:59:12.071 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.066+0000 7f7acf400640 1 --2- 172.21.15.151:0/3153755399 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55686173e140 0x5568617401e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x55686173aec0 tx=0x7f7abc005c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e26ea1d810d00ad6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:12.071 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.066+0000 7f7ac4c00640 1 -- 172.21.15.151:0/3153755399 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7abc049040 con 0x55686173e140 2024-04-26T01:59:12.071 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.066+0000 7f7ac4c00640 1 -- 172.21.15.151:0/3153755399 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7abc0bd510 con 0x55686173e140 2024-04-26T01:59:12.071 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.066+0000 7f7ac4c00640 1 -- 172.21.15.151:0/3153755399 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7abc0bc950 con 0x55686173e140 2024-04-26T01:59:12.071 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.067+0000 7f7ad9c0b740 1 -- 172.21.15.151:0/3153755399 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55686173e140 msgr2=0x5568617401e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:12.071 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.067+0000 7f7ad9c0b740 1 --2- 172.21.15.151:0/3153755399 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55686173e140 0x5568617401e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x55686173aec0 tx=0x7f7abc005c90 comp rx=0 tx=0).stop 2024-04-26T01:59:12.071 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.067+0000 7f7ad9c0b740 1 -- 172.21.15.151:0/3153755399 shutdown_connections 2024-04-26T01:59:12.071 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.067+0000 7f7ad9c0b740 1 --2- 172.21.15.151:0/3153755399 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556861748a00 0x556861738d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:12.071 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.067+0000 7f7ad9c0b740 1 --2- 172.21.15.151:0/3153755399 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55686173e140 0x5568617401e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:12.071 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.067+0000 7f7ad9c0b740 1 --2- 172.21.15.151:0/3153755399 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55686173d820 0x5568617541f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:12.071 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.067+0000 7f7ad9c0b740 1 -- 172.21.15.151:0/3153755399 >> 172.21.15.151:0/3153755399 conn(0x556861739d30 msgr2=0x556861738600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:12.071 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.068+0000 7f7ad9c0b740 1 -- 172.21.15.151:0/3153755399 shutdown_connections 2024-04-26T01:59:12.072 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.068+0000 7f7ad9c0b740 1 -- 172.21.15.151:0/3153755399 wait complete. 2024-04-26T01:59:12.072 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.068+0000 7f7ad9c0b740 1 -- start start 2024-04-26T01:59:12.072 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.068+0000 7f7ad9c0b740 1 Processor -- start 2024-04-26T01:59:12.072 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.069+0000 7f7ad9c0b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55686173d820 0x55686174e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:12.072 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.069+0000 7f7ad9c0b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55686173e140 0x55686174e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:12.072 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.069+0000 7f7ad9c0b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556861748a00 0x55686174ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:12.072 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.069+0000 7f7ad9c0b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad01a9de0 con 0x55686173e140 2024-04-26T01:59:12.072 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.069+0000 7f7ad9c0b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad01ab230 con 0x55686173d820 2024-04-26T01:59:12.072 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.069+0000 7f7ad9c0b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad01aa490 con 0x556861748a00 2024-04-26T01:59:12.072 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.069+0000 7f7acfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55686173d820 0x55686174e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:12.072 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.069+0000 7f7ad9c0b740 10 client.0 _ll_register_callbacks cb 0x7f7ac0011ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:59:12.072 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.069+0000 7f7acfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55686173d820 0x55686174e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:55240/0 (socket says 172.21.15.151:55240) 2024-04-26T01:59:12.072 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.069+0000 7f7acfe00640 1 -- 172.21.15.151:0/159046998 learned_addr learned my addr 172.21.15.151:0/159046998 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:59:12.073 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.069+0000 7f7acf400640 1 --2- 172.21.15.151:0/159046998 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55686173e140 0x55686174e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:12.073 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.069+0000 7f7ad4e00640 1 --2- 172.21.15.151:0/159046998 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556861748a00 0x55686174ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:12.073 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.069+0000 7f7acfe00640 1 -- 172.21.15.151:0/159046998 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556861748a00 msgr2=0x55686174ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:12.073 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.069+0000 7f7acfe00640 1 --2- 172.21.15.151:0/159046998 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556861748a00 0x55686174ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:12.073 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.069+0000 7f7acfe00640 1 -- 172.21.15.151:0/159046998 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55686173e140 msgr2=0x55686174e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:12.073 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.069+0000 7f7acfe00640 1 --2- 172.21.15.151:0/159046998 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55686173e140 0x55686174e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:12.073 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.069+0000 7f7acfe00640 1 -- 172.21.15.151:0/159046998 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568617be330 con 0x55686173d820 2024-04-26T01:59:12.073 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.070+0000 7f7acfe00640 1 --2- 172.21.15.151:0/159046998 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55686173d820 0x55686174e3a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0009830 tx=0x7f7ac0005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3f8c5be531c6a5d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:12.073 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.070+0000 7f7aba000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T01:59:12.073 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.070+0000 7f7aba000640 1 -- 172.21.15.151:0/159046998 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac000b690 con 0x55686173d820 2024-04-26T01:59:12.073 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.070+0000 7f7aba000640 1 -- 172.21.15.151:0/159046998 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7f7ac0004660 con 0x55686173d820 2024-04-26T01:59:12.073 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.070+0000 7f7ad9c0b740 1 -- 172.21.15.151:0/159046998 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7abc0bd510 con 0x55686173d820 2024-04-26T01:59:12.073 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.070+0000 7f7aba000640 1 -- 172.21.15.151:0/159046998 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7ac0038050 con 0x55686173d820 2024-04-26T01:59:12.074 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.070+0000 7f7ad9c0b740 10 client.25787 fetch_fsmap learned FSMap version 32 2024-04-26T01:59:12.074 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.070+0000 7f7ad9c0b740 1 -- 172.21.15.151:0/159046998 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5568617bf600 con 0x55686173d820 2024-04-26T01:59:12.074 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.071+0000 7f7aba000640 1 -- 172.21.15.151:0/159046998 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7abc0bd510 con 0x55686173d820 2024-04-26T01:59:12.074 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.071+0000 7f7aba000640 1 -- 172.21.15.151:0/159046998 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f7ac002f070 con 0x55686173d820 2024-04-26T01:59:12.074 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.071+0000 7f7ad9c0b740 10 client.25787 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T01:59:12.074 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.071+0000 7f7ad9c0b740 10 client.25787 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:59:12.074 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.071+0000 7f7ad9c0b740 20 client.25787 populate_metadata read hostname 'smithi151' 2024-04-26T01:59:12.074 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.071+0000 7f7ad9c0b740 1 --2- 172.21.15.151:0/159046998 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5568617bfcf0 0x5568617c2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:12.074 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.071+0000 7f7ad9c0b740 4 client.25787 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:59:12.074 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.071+0000 7f7ad9c0b740 1 -- 172.21.15.151:0/159046998 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5568616b1320 con 0x5568617bfcf0 2024-04-26T01:59:12.074 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.071+0000 7f7acf400640 1 --2- 172.21.15.151:0/159046998 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5568617bfcf0 0x5568617c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:12.075 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.072+0000 7f7aba000640 10 client.25787 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:59:12.075 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.072+0000 7f7acf400640 1 --2- 172.21.15.151:0/159046998 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5568617bfcf0 0x5568617c2110 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b50639f08388d39 server_cookie=c97a475b4fdf9075 in_seq=0 out_seq=0 2024-04-26T01:59:12.091 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.092+0000 7f7aba000640 1 -- 172.21.15.151:0/159046998 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5568616b1320 con 0x5568617bfcf0 2024-04-26T01:59:12.091 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.092+0000 7f7aba000640 10 client.25787 handle_command_reply: tid=0 2024-04-26T01:59:12.091 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.092+0000 7f7ad9c0b740 1 client.25787 shutdown 2024-04-26T01:59:12.091 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.092+0000 7f7ad9c0b740 1 -- 172.21.15.151:0/159046998 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55686173d820 msgr2=0x55686174e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:12.091 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.092+0000 7f7ad9c0b740 1 --2- 172.21.15.151:0/159046998 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55686173d820 0x55686174e3a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0009830 tx=0x7f7ac0005270 comp rx=0 tx=0).stop 2024-04-26T01:59:12.091 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.092+0000 7f7ad9c0b740 1 -- 172.21.15.151:0/159046998 shutdown_connections 2024-04-26T01:59:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.092+0000 7f7ad9c0b740 1 --2- 172.21.15.151:0/159046998 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5568617bfcf0 0x5568617c2110 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.092+0000 7f7ad9c0b740 1 --2- 172.21.15.151:0/159046998 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556861748a00 0x55686174ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.092+0000 7f7ad9c0b740 1 --2- 172.21.15.151:0/159046998 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55686173e140 0x55686174e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.092+0000 7f7ad9c0b740 1 --2- 172.21.15.151:0/159046998 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55686173d820 0x55686174e3a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.092+0000 7f7aba000640 0 client.25787 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:59:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.092+0000 7f7ad9c0b740 1 -- 172.21.15.151:0/159046998 >> 172.21.15.151:0/159046998 conn(0x556861739d30 msgr2=0x556861738e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.092+0000 7f7ad9c0b740 1 -- 172.21.15.151:0/159046998 shutdown_connections 2024-04-26T01:59:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.092+0000 7f7ad9c0b740 1 -- 172.21.15.151:0/159046998 wait complete. 2024-04-26T01:59:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.093+0000 7f7ad9c0b740 20 client.25787 trim_cache size 0 max 16384 2024-04-26T01:59:12.093 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:{ 2024-04-26T01:59:12.093 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "message": "", 2024-04-26T01:59:12.093 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "return_code": 0 2024-04-26T01:59:12.093 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:} 2024-04-26T01:59:12.093 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.093+0000 7f7ad6e00640 1 -- 172.21.15.151:0/1287064209 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f7ab40a3e40 msgr2=0x7f7ab40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:12.093 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.093+0000 7f7ad6e00640 1 --2- 172.21.15.151:0/1287064209 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f7ab40a3e40 0x7f7ab40a6300 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7ac8005fd0 tx=0x7f7ac8009500 comp rx=0 tx=0).stop 2024-04-26T01:59:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.093+0000 7f7ad6e00640 1 -- 172.21.15.151:0/1287064209 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7ad01072a0 msgr2=0x7f7ad0198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.093+0000 7f7ad6e00640 1 --2- 172.21.15.151:0/1287064209 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7ad01072a0 0x7f7ad0198c70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7abc005ab0 tx=0x7f7abc003b90 comp rx=0 tx=0).stop 2024-04-26T01:59:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.093+0000 7f7ad6e00640 1 -- 172.21.15.151:0/1287064209 shutdown_connections 2024-04-26T01:59:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.093+0000 7f7ad6e00640 1 --2- 172.21.15.151:0/1287064209 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f7ab40a3e40 0x7f7ab40a6300 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.093+0000 7f7ad6e00640 1 --2- 172.21.15.151:0/1287064209 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ad019cfa0 0x7f7ad019d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.093+0000 7f7ad6e00640 1 --2- 172.21.15.151:0/1287064209 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7ad01072a0 0x7f7ad0198c70 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.093+0000 7f7ad6e00640 1 --2- 172.21.15.151:0/1287064209 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ad01021d0 0x7f7ad0198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.093+0000 7f7ad6e00640 1 -- 172.21.15.151:0/1287064209 >> 172.21.15.151:0/1287064209 conn(0x7f7ad00793e0 msgr2=0x7f7ad00ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.094+0000 7f7ad6e00640 1 -- 172.21.15.151:0/1287064209 shutdown_connections 2024-04-26T01:59:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:12.094+0000 7f7ad6e00640 1 -- 172.21.15.151:0/1287064209 wait complete. 2024-04-26T01:59:12.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:59:12 smithi172 ceph-mon[32785]: pgmap v171: 97 pgs: 97 active+clean; 53 MiB data, 834 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.6 MiB/s wr, 290 op/s 2024-04-26T01:59:12.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:59:12 smithi165 ceph-mon[32648]: pgmap v171: 97 pgs: 97 active+clean; 53 MiB data, 834 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.6 MiB/s wr, 290 op/s 2024-04-26T01:59:12.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:59:12 smithi151 ceph-mon[28057]: pgmap v171: 97 pgs: 97 active+clean; 53 MiB data, 834 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.6 MiB/s wr, 290 op/s 2024-04-26T01:59:14.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:59:14 smithi172 ceph-mon[32785]: pgmap v172: 97 pgs: 97 active+clean; 49 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.6 MiB/s wr, 291 op/s 2024-04-26T01:59:14.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:59:14 smithi165 ceph-mon[32648]: pgmap v172: 97 pgs: 97 active+clean; 49 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.6 MiB/s wr, 291 op/s 2024-04-26T01:59:14.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:59:14 smithi151 ceph-mon[28057]: pgmap v172: 97 pgs: 97 active+clean; 49 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.6 MiB/s wr, 291 op/s 2024-04-26T01:59:16.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:59:16 smithi172 ceph-mon[32785]: pgmap v173: 97 pgs: 97 active+clean; 51 MiB data, 853 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 2.2 MiB/s wr, 376 op/s 2024-04-26T01:59:16.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:59:16 smithi165 ceph-mon[32648]: pgmap v173: 97 pgs: 97 active+clean; 51 MiB data, 853 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 2.2 MiB/s wr, 376 op/s 2024-04-26T01:59:16.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:59:16 smithi151 ceph-mon[28057]: pgmap v173: 97 pgs: 97 active+clean; 51 MiB data, 853 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 2.2 MiB/s wr, 376 op/s 2024-04-26T01:59:18.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:59:18 smithi172 ceph-mon[32785]: pgmap v174: 97 pgs: 97 active+clean; 51 MiB data, 853 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.6 MiB/s wr, 260 op/s 2024-04-26T01:59:18.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:59:18 smithi165 ceph-mon[32648]: pgmap v174: 97 pgs: 97 active+clean; 51 MiB data, 853 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.6 MiB/s wr, 260 op/s 2024-04-26T01:59:18.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:59:18 smithi151 ceph-mon[28057]: pgmap v174: 97 pgs: 97 active+clean; 51 MiB data, 853 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.6 MiB/s wr, 260 op/s 2024-04-26T01:59:20.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:59:20 smithi172 ceph-mon[32785]: pgmap v175: 97 pgs: 97 active+clean; 48 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.3 MiB/s wr, 378 op/s 2024-04-26T01:59:20.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:59:20 smithi165 ceph-mon[32648]: pgmap v175: 97 pgs: 97 active+clean; 48 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.3 MiB/s wr, 378 op/s 2024-04-26T01:59:20.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:59:20 smithi151 ceph-mon[28057]: pgmap v175: 97 pgs: 97 active+clean; 48 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.3 MiB/s wr, 378 op/s 2024-04-26T01:59:22.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:59:22 smithi172 ceph-mon[32785]: pgmap v176: 97 pgs: 97 active+clean; 48 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.5 MiB/s wr, 250 op/s 2024-04-26T01:59:22.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:59:22 smithi165 ceph-mon[32648]: pgmap v176: 97 pgs: 97 active+clean; 48 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.5 MiB/s wr, 250 op/s 2024-04-26T01:59:22.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:59:22 smithi151 ceph-mon[28057]: pgmap v176: 97 pgs: 97 active+clean; 48 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.5 MiB/s wr, 250 op/s 2024-04-26T01:59:24.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:59:24 smithi172 ceph-mon[32785]: pgmap v177: 97 pgs: 97 active+clean; 50 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.7 MiB/s wr, 292 op/s 2024-04-26T01:59:24.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:59:24 smithi165 ceph-mon[32648]: pgmap v177: 97 pgs: 97 active+clean; 50 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.7 MiB/s wr, 292 op/s 2024-04-26T01:59:24.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:59:24 smithi151 ceph-mon[28057]: pgmap v177: 97 pgs: 97 active+clean; 50 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.7 MiB/s wr, 292 op/s 2024-04-26T01:59:25.249 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.250+0000 7f30c7200640 1 -- 172.21.15.151:0/2073330065 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30c8101fe0 msgr2=0x7f30c81023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:25.249 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.250+0000 7f30c7200640 1 --2- 172.21.15.151:0/2073330065 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30c8101fe0 0x7f30c81023c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f30b8009900 tx=0x7f30b802a410 comp rx=0 tx=0).stop 2024-04-26T01:59:25.250 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.250+0000 7f30c7200640 1 -- 172.21.15.151:0/2073330065 shutdown_connections 2024-04-26T01:59:25.250 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.250+0000 7f30c7200640 1 --2- 172.21.15.151:0/2073330065 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30c81070b0 0x7f30c810ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:25.250 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.250+0000 7f30c7200640 1 --2- 172.21.15.151:0/2073330065 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30c8102990 0x7f30c81069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:25.250 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.250+0000 7f30c7200640 1 --2- 172.21.15.151:0/2073330065 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30c8101fe0 0x7f30c81023c0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:25.250 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.250+0000 7f30c7200640 1 -- 172.21.15.151:0/2073330065 >> 172.21.15.151:0/2073330065 conn(0x7f30c80fd8a0 msgr2=0x7f30c80ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:25.250 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.250+0000 7f30c7200640 1 -- 172.21.15.151:0/2073330065 shutdown_connections 2024-04-26T01:59:25.250 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.250+0000 7f30c7200640 1 -- 172.21.15.151:0/2073330065 wait complete. 2024-04-26T01:59:25.252 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.251+0000 7f30c7200640 1 Processor -- start 2024-04-26T01:59:25.252 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.252+0000 7f30c7200640 1 -- start start 2024-04-26T01:59:25.252 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.252+0000 7f30c7200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30c8101fe0 0x7f30c8074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:25.252 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.252+0000 7f30c7200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30c81070b0 0x7f30c8074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:25.252 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.252+0000 7f30c7200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30c8074cd0 0x7f30c81a6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:25.252 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.252+0000 7f30c7200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f30c8112040 con 0x7f30c8074cd0 2024-04-26T01:59:25.252 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.252+0000 7f30c7200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f30c8111ec0 con 0x7f30c81070b0 2024-04-26T01:59:25.252 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.252+0000 7f30c7200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f30c81121c0 con 0x7f30c8101fe0 2024-04-26T01:59:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.252+0000 7f30bfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30c81070b0 0x7f30c8074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.253+0000 7f30c6800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30c8074cd0 0x7f30c81a6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.253+0000 7f30bfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30c81070b0 0x7f30c8074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:52904/0 (socket says 172.21.15.151:52904) 2024-04-26T01:59:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.253+0000 7f30bfe00640 1 -- 172.21.15.151:0/2630497745 learned_addr learned my addr 172.21.15.151:0/2630497745 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:59:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.253+0000 7f30c5e00640 1 --2- 172.21.15.151:0/2630497745 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30c8101fe0 0x7f30c8074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.253+0000 7f30bfe00640 1 -- 172.21.15.151:0/2630497745 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30c8101fe0 msgr2=0x7f30c8074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.253+0000 7f30bfe00640 1 --2- 172.21.15.151:0/2630497745 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30c8101fe0 0x7f30c8074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.253+0000 7f30bfe00640 1 -- 172.21.15.151:0/2630497745 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30c8074cd0 msgr2=0x7f30c81a6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.253+0000 7f30bfe00640 1 --2- 172.21.15.151:0/2630497745 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30c8074cd0 0x7f30c81a6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.253+0000 7f30bfe00640 1 -- 172.21.15.151:0/2630497745 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30c81a6e90 con 0x7f30c81070b0 2024-04-26T01:59:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.253+0000 7f30bfe00640 1 --2- 172.21.15.151:0/2630497745 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30c81070b0 0x7f30c8074790 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f30b000e9c0 tx=0x7f30b000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:25.254 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.254+0000 7f30bea00640 1 -- 172.21.15.151:0/2630497745 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b000cd80 con 0x7f30c81070b0 2024-04-26T01:59:25.254 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.254+0000 7f30c7200640 1 -- 172.21.15.151:0/2630497745 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30c81a7180 con 0x7f30c81070b0 2024-04-26T01:59:25.254 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.254+0000 7f30bea00640 1 -- 172.21.15.151:0/2630497745 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30b00054d0 con 0x7f30c81070b0 2024-04-26T01:59:25.254 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.254+0000 7f30bea00640 1 -- 172.21.15.151:0/2630497745 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b0010430 con 0x7f30c81070b0 2024-04-26T01:59:25.254 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.254+0000 7f30c7200640 1 -- 172.21.15.151:0/2630497745 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30c81a76d0 con 0x7f30c81070b0 2024-04-26T01:59:25.256 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.257+0000 7f30cfd7c740 1 Processor -- start 2024-04-26T01:59:25.256 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.257+0000 7f30bea00640 1 -- 172.21.15.151:0/2630497745 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f30b0004f90 con 0x7f30c81070b0 2024-04-26T01:59:25.256 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.257+0000 7f30cfd7c740 1 -- start start 2024-04-26T01:59:25.257 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.258+0000 7f30cfd7c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a0cd2349b0 0x55a0cd234d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:25.258 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.258+0000 7f30cfd7c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a0cd2352d0 0x55a0cd23f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:25.258 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.258+0000 7f30cfd7c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0cd23fb90 0x55a0cd241f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:25.258 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.258+0000 7f30cfd7c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a0cd19a900 con 0x55a0cd2349b0 2024-04-26T01:59:25.258 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.258+0000 7f30cfd7c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a0cd140ef0 con 0x55a0cd2352d0 2024-04-26T01:59:25.258 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.258+0000 7f30cfd7c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a0cd1968c0 con 0x55a0cd23fb90 2024-04-26T01:59:25.258 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.258+0000 7f30c5e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a0cd2352d0 0x55a0cd23f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:25.258 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.258+0000 7f30c5e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a0cd2352d0 0x55a0cd23f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:52906/0 (socket says 172.21.15.151:52906) 2024-04-26T01:59:25.258 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.258+0000 7f30c6800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a0cd2349b0 0x55a0cd234d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:25.258 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.258+0000 7f30c5e00640 1 -- 172.21.15.151:0/1378924501 learned_addr learned my addr 172.21.15.151:0/1378924501 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:59:25.258 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.258+0000 7f30bfe00640 1 --2- 172.21.15.151:0/1378924501 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0cd23fb90 0x55a0cd241f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:25.259 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.259+0000 7f30c5e00640 1 -- 172.21.15.151:0/1378924501 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0cd23fb90 msgr2=0x55a0cd241f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:25.259 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.259+0000 7f30c5e00640 1 --2- 172.21.15.151:0/1378924501 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0cd23fb90 0x55a0cd241f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:25.259 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.259+0000 7f30c5e00640 1 -- 172.21.15.151:0/1378924501 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a0cd2349b0 msgr2=0x55a0cd234d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:25.259 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.259+0000 7f30c5e00640 1 --2- 172.21.15.151:0/1378924501 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a0cd2349b0 0x55a0cd234d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:25.259 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.259+0000 7f30c5e00640 1 -- 172.21.15.151:0/1378924501 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0ccead210 con 0x55a0cd2352d0 2024-04-26T01:59:25.259 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.259+0000 7f30c6800640 1 --2- 172.21.15.151:0/1378924501 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a0cd2349b0 0x55a0cd234d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:59:25.259 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.259+0000 7f30c5e00640 1 --2- 172.21.15.151:0/1378924501 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a0cd2352d0 0x55a0cd23f650 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f30b8000c00 tx=0x7f30b8004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c0390cc2c224811 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.259+0000 7f30b7400640 1 -- 172.21.15.151:0/1378924501 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b8035040 con 0x55a0cd2352d0 2024-04-26T01:59:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.259+0000 7f30b7400640 1 -- 172.21.15.151:0/1378924501 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30b80054f0 con 0x55a0cd2352d0 2024-04-26T01:59:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.259+0000 7f30b7400640 1 -- 172.21.15.151:0/1378924501 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b80035f0 con 0x55a0cd2352d0 2024-04-26T01:59:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.260+0000 7f30bea00640 1 --2- 172.21.15.151:0/2630497745 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f30ac0a3d40 0x7f30ac0a6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.260+0000 7f30cfd7c740 1 -- 172.21.15.151:0/1378924501 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a0cd2352d0 msgr2=0x55a0cd23f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.260+0000 7f30cfd7c740 1 --2- 172.21.15.151:0/1378924501 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a0cd2352d0 0x55a0cd23f650 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f30b8000c00 tx=0x7f30b8004040 comp rx=0 tx=0).stop 2024-04-26T01:59:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.260+0000 7f30bea00640 1 -- 172.21.15.151:0/2630497745 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7f30b00d0b80 con 0x7f30c81070b0 2024-04-26T01:59:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.261+0000 7f30c6800640 1 --2- 172.21.15.151:0/2630497745 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f30ac0a3d40 0x7f30ac0a6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.261+0000 7f30cfd7c740 1 -- 172.21.15.151:0/1378924501 shutdown_connections 2024-04-26T01:59:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.261+0000 7f30cfd7c740 1 --2- 172.21.15.151:0/1378924501 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0cd23fb90 0x55a0cd241f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.261+0000 7f30cfd7c740 1 --2- 172.21.15.151:0/1378924501 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a0cd2352d0 0x55a0cd23f650 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.261+0000 7f30cfd7c740 1 --2- 172.21.15.151:0/1378924501 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a0cd2349b0 0x55a0cd234d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.261+0000 7f30cfd7c740 1 -- 172.21.15.151:0/1378924501 >> 172.21.15.151:0/1378924501 conn(0x55a0cd230ec0 msgr2=0x55a0cd2312a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.261+0000 7f30cfd7c740 1 -- 172.21.15.151:0/1378924501 shutdown_connections 2024-04-26T01:59:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.262+0000 7f30cfd7c740 1 -- 172.21.15.151:0/1378924501 wait complete. 2024-04-26T01:59:25.262 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.262+0000 7f30c6800640 1 --2- 172.21.15.151:0/2630497745 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f30ac0a3d40 0x7f30ac0a6200 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f30c0009d40 tx=0x7f30c0009500 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:25.265 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.262+0000 7f30cfd7c740 1 -- start start 2024-04-26T01:59:25.265 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.262+0000 7f30cfd7c740 1 Processor -- start 2024-04-26T01:59:25.265 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.263+0000 7f30cfd7c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0cd2349b0 0x55a0cd23cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:25.265 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.263+0000 7f30cfd7c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a0cd2352d0 0x55a0cd23d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:25.265 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.263+0000 7f30cfd7c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a0cd23fb90 0x55a0cd23d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:25.265 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.263+0000 7f30cfd7c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a0cd242760 con 0x55a0cd23fb90 2024-04-26T01:59:25.265 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.263+0000 7f30cfd7c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a0cd154310 con 0x55a0cd2352d0 2024-04-26T01:59:25.265 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.263+0000 7f30cfd7c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a0cd2428e0 con 0x55a0cd2349b0 2024-04-26T01:59:25.265 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.263+0000 7f30cfd7c740 10 client.0 _ll_register_callbacks cb 0x55a0cd122cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:59:25.265 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.263+0000 7f30bfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a0cd2352d0 0x55a0cd23d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:25.265 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.263+0000 7f30c5e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0cd2349b0 0x55a0cd23cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.263+0000 7f30c5e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0cd2349b0 0x55a0cd23cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:50114/0 (socket says 172.21.15.151:50114) 2024-04-26T01:59:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.263+0000 7f30c5e00640 1 -- 172.21.15.151:0/129455735 learned_addr learned my addr 172.21.15.151:0/129455735 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:59:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.263+0000 7f30c6800640 1 --2- 172.21.15.151:0/129455735 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a0cd23fb90 0x55a0cd23d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.264+0000 7f30bfe00640 1 -- 172.21.15.151:0/129455735 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0cd2349b0 msgr2=0x55a0cd23cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.264+0000 7f30bfe00640 1 --2- 172.21.15.151:0/129455735 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0cd2349b0 0x55a0cd23cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.264+0000 7f30bfe00640 1 -- 172.21.15.151:0/129455735 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a0cd23fb90 msgr2=0x55a0cd23d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.264+0000 7f30bfe00640 1 --2- 172.21.15.151:0/129455735 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a0cd23fb90 0x55a0cd23d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.264+0000 7f30bfe00640 1 -- 172.21.15.151:0/129455735 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0ccf79620 con 0x55a0cd2352d0 2024-04-26T01:59:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.264+0000 7f30bfe00640 1 --2- 172.21.15.151:0/129455735 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a0cd2352d0 0x55a0cd23d050 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f30b00ce2e0 tx=0x7f30b00d4220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cef6f95f3503173e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.264+0000 7f30abe00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T01:59:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.264+0000 7f30abe00640 1 -- 172.21.15.151:0/129455735 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b009de60 con 0x55a0cd2352d0 2024-04-26T01:59:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.264+0000 7f30abe00640 1 -- 172.21.15.151:0/129455735 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7f30b00bd0c0 con 0x55a0cd2352d0 2024-04-26T01:59:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.264+0000 7f30cfd7c740 1 -- 172.21.15.151:0/129455735 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30b80054f0 con 0x55a0cd2352d0 2024-04-26T01:59:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.265+0000 7f30abe00640 1 -- 172.21.15.151:0/129455735 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f30b009c670 con 0x55a0cd2352d0 2024-04-26T01:59:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.265+0000 7f30cfd7c740 10 client.25799 fetch_fsmap learned FSMap version 32 2024-04-26T01:59:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.265+0000 7f30cfd7c740 1 -- 172.21.15.151:0/129455735 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a0cd24e9a0 con 0x55a0cd2352d0 2024-04-26T01:59:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.265+0000 7f30abe00640 1 -- 172.21.15.151:0/129455735 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30b80054f0 con 0x55a0cd2352d0 2024-04-26T01:59:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.265+0000 7f30abe00640 1 -- 172.21.15.151:0/129455735 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f30b00991a0 con 0x55a0cd2352d0 2024-04-26T01:59:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.265+0000 7f30cfd7c740 10 client.25799 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T01:59:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.265+0000 7f30cfd7c740 10 client.25799 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:59:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.265+0000 7f30cfd7c740 20 client.25799 populate_metadata read hostname 'smithi151' 2024-04-26T01:59:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.265+0000 7f30cfd7c740 1 --2- 172.21.15.151:0/129455735 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55a0cd24feb0 0x55a0cd2522a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.265+0000 7f30cfd7c740 4 client.25799 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:59:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.265+0000 7f30cfd7c740 1 -- 172.21.15.151:0/129455735 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0cd1ae8b0 con 0x55a0cd24feb0 2024-04-26T01:59:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.265+0000 7f30c5e00640 1 --2- 172.21.15.151:0/129455735 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55a0cd24feb0 0x55a0cd2522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.266+0000 7f30c5e00640 1 --2- 172.21.15.151:0/129455735 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55a0cd24feb0 0x55a0cd2522a0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52fc41ba928ecb21 server_cookie=8e77d2c87f81e523 in_seq=0 out_seq=0 2024-04-26T01:59:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.266+0000 7f30abe00640 10 client.25799 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:59:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.267+0000 7f30abe00640 1 -- 172.21.15.151:0/129455735 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a0cd1ae8b0 con 0x55a0cd24feb0 2024-04-26T01:59:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.267+0000 7f30abe00640 10 client.25799 handle_command_reply: tid=0 2024-04-26T01:59:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.267+0000 7f30cfd7c740 1 client.25799 shutdown 2024-04-26T01:59:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.267+0000 7f30cfd7c740 1 -- 172.21.15.151:0/129455735 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a0cd2352d0 msgr2=0x55a0cd23d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.267+0000 7f30cfd7c740 1 --2- 172.21.15.151:0/129455735 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a0cd2352d0 0x55a0cd23d050 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f30b00ce2e0 tx=0x7f30b00d4220 comp rx=0 tx=0).stop 2024-04-26T01:59:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.267+0000 7f30cfd7c740 1 -- 172.21.15.151:0/129455735 shutdown_connections 2024-04-26T01:59:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.267+0000 7f30cfd7c740 1 --2- 172.21.15.151:0/129455735 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55a0cd24feb0 0x55a0cd2522a0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.267+0000 7f30abe00640 0 client.25799 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:59:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.267+0000 7f30cfd7c740 1 --2- 172.21.15.151:0/129455735 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a0cd23fb90 0x55a0cd23d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.267+0000 7f30cfd7c740 1 --2- 172.21.15.151:0/129455735 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a0cd2352d0 0x55a0cd23d050 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.267+0000 7f30cfd7c740 1 --2- 172.21.15.151:0/129455735 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0cd2349b0 0x55a0cd23cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.267+0000 7f30cfd7c740 1 -- 172.21.15.151:0/129455735 >> 172.21.15.151:0/129455735 conn(0x55a0cd230ec0 msgr2=0x55a0cd236c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.267+0000 7f30cfd7c740 1 -- 172.21.15.151:0/129455735 shutdown_connections 2024-04-26T01:59:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.267+0000 7f30cfd7c740 1 -- 172.21.15.151:0/129455735 wait complete. 2024-04-26T01:59:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.268+0000 7f30cfd7c740 20 client.25799 trim_cache size 0 max 16384 2024-04-26T01:59:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.288+0000 7f30cfd7c740 1 Processor -- start 2024-04-26T01:59:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.289+0000 7f30cfd7c740 1 -- start start 2024-04-26T01:59:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.289+0000 7f30cfd7c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0cd2349b0 0x55a0cd24b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.289+0000 7f30cfd7c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a0cd2352d0 0x55a0cd23d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.289+0000 7f30cfd7c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a0cd23fb90 0x55a0cd257b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.289+0000 7f30cfd7c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f30c81b2350 con 0x55a0cd23fb90 2024-04-26T01:59:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.289+0000 7f30cfd7c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f30c81afc20 con 0x55a0cd2352d0 2024-04-26T01:59:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.289+0000 7f30cfd7c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f30c81ada90 con 0x55a0cd2349b0 2024-04-26T01:59:25.291 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.289+0000 7f30c5e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0cd2349b0 0x55a0cd24b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:25.291 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.289+0000 7f30c5e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0cd2349b0 0x55a0cd24b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:50120/0 (socket says 172.21.15.151:50120) 2024-04-26T01:59:25.291 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.289+0000 7f30bfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a0cd2352d0 0x55a0cd23d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:25.291 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.289+0000 7f30c5e00640 1 -- 172.21.15.151:0/1813271931 learned_addr learned my addr 172.21.15.151:0/1813271931 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:59:25.291 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.289+0000 7f30c6800640 1 --2- 172.21.15.151:0/1813271931 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a0cd23fb90 0x55a0cd257b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:25.291 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.290+0000 7f30c5e00640 1 -- 172.21.15.151:0/1813271931 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a0cd2352d0 msgr2=0x55a0cd23d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:25.291 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.290+0000 7f30c5e00640 1 --2- 172.21.15.151:0/1813271931 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a0cd2352d0 0x55a0cd23d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:25.291 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.290+0000 7f30c5e00640 1 -- 172.21.15.151:0/1813271931 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a0cd23fb90 msgr2=0x55a0cd257b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:25.291 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.290+0000 7f30c5e00640 1 --2- 172.21.15.151:0/1813271931 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a0cd23fb90 0x55a0cd257b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:25.292 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.290+0000 7f30c5e00640 1 -- 172.21.15.151:0/1813271931 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0cd235100 con 0x55a0cd2349b0 2024-04-26T01:59:25.292 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.290+0000 7f30c5e00640 1 --2- 172.21.15.151:0/1813271931 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0cd2349b0 0x55a0cd24b620 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f30b80057c0 tx=0x7f30b8005a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fa4804ed335e5f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:25.292 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.290+0000 7f30b6a00640 1 -- 172.21.15.151:0/1813271931 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b8035040 con 0x55a0cd2349b0 2024-04-26T01:59:25.292 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.290+0000 7f30b6a00640 1 -- 172.21.15.151:0/1813271931 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30b8053630 con 0x55a0cd2349b0 2024-04-26T01:59:25.292 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.290+0000 7f30b6a00640 1 -- 172.21.15.151:0/1813271931 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b80038e0 con 0x55a0cd2349b0 2024-04-26T01:59:25.292 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.291+0000 7f30cfd7c740 1 -- 172.21.15.151:0/1813271931 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0cd2349b0 msgr2=0x55a0cd24b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:25.292 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.291+0000 7f30cfd7c740 1 --2- 172.21.15.151:0/1813271931 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0cd2349b0 0x55a0cd24b620 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f30b80057c0 tx=0x7f30b8005a20 comp rx=0 tx=0).stop 2024-04-26T01:59:25.292 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.291+0000 7f30cfd7c740 1 -- 172.21.15.151:0/1813271931 shutdown_connections 2024-04-26T01:59:25.292 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.291+0000 7f30cfd7c740 1 --2- 172.21.15.151:0/1813271931 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a0cd23fb90 0x55a0cd257b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:25.292 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.291+0000 7f30cfd7c740 1 --2- 172.21.15.151:0/1813271931 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a0cd2352d0 0x55a0cd23d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:25.292 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.291+0000 7f30cfd7c740 1 --2- 172.21.15.151:0/1813271931 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0cd2349b0 0x55a0cd24b620 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:25.292 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.291+0000 7f30cfd7c740 1 -- 172.21.15.151:0/1813271931 >> 172.21.15.151:0/1813271931 conn(0x55a0cd230ec0 msgr2=0x55a0cd237dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:25.292 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.291+0000 7f30cfd7c740 1 -- 172.21.15.151:0/1813271931 shutdown_connections 2024-04-26T01:59:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.291+0000 7f30cfd7c740 1 -- 172.21.15.151:0/1813271931 wait complete. 2024-04-26T01:59:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.292+0000 7f30cfd7c740 1 -- start start 2024-04-26T01:59:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.292+0000 7f30cfd7c740 1 Processor -- start 2024-04-26T01:59:25.295 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.292+0000 7f30cfd7c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a0cd2349b0 0x55a0cd244cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:25.295 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.292+0000 7f30cfd7c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a0cd2352d0 0x55a0cd245230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:25.295 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.292+0000 7f30cfd7c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0cd23fb90 0x55a0cd2b43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:25.295 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.293+0000 7f30cfd7c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f30c81ae0e0 con 0x55a0cd2352d0 2024-04-26T01:59:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.293+0000 7f30cfd7c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f30c81af550 con 0x55a0cd2349b0 2024-04-26T01:59:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.293+0000 7f30cfd7c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f30c81ae7b0 con 0x55a0cd23fb90 2024-04-26T01:59:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.293+0000 7f30cfd7c740 10 client.0 _ll_register_callbacks cb 0x7f30b00048b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:59:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.293+0000 7f30c5e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a0cd2349b0 0x55a0cd244cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.293+0000 7f30c5e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a0cd2349b0 0x55a0cd244cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:52938/0 (socket says 172.21.15.151:52938) 2024-04-26T01:59:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.293+0000 7f30bfe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a0cd2352d0 0x55a0cd245230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.293+0000 7f30c5e00640 1 -- 172.21.15.151:0/2971363812 learned_addr learned my addr 172.21.15.151:0/2971363812 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:59:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.293+0000 7f30c6800640 1 --2- 172.21.15.151:0/2971363812 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0cd23fb90 0x55a0cd2b43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.293+0000 7f30c5e00640 1 -- 172.21.15.151:0/2971363812 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0cd23fb90 msgr2=0x55a0cd2b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.293+0000 7f30c5e00640 1 --2- 172.21.15.151:0/2971363812 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0cd23fb90 0x55a0cd2b43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.293+0000 7f30c5e00640 1 -- 172.21.15.151:0/2971363812 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a0cd2352d0 msgr2=0x55a0cd245230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.293+0000 7f30c5e00640 1 --2- 172.21.15.151:0/2971363812 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a0cd2352d0 0x55a0cd245230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.293+0000 7f30c5e00640 1 -- 172.21.15.151:0/2971363812 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0cd2b4920 con 0x55a0cd2349b0 2024-04-26T01:59:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.293+0000 7f30c6800640 1 --2- 172.21.15.151:0/2971363812 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0cd23fb90 0x55a0cd2b43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:59:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.293+0000 7f30bfe00640 1 --2- 172.21.15.151:0/2971363812 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a0cd2352d0 0x55a0cd245230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:59:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.293+0000 7f30c5e00640 1 --2- 172.21.15.151:0/2971363812 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a0cd2349b0 0x55a0cd244cf0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f30b8051040 tx=0x7f30b802a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8929ef281bb67d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.294+0000 7f30abe00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T01:59:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.294+0000 7f30abe00640 1 -- 172.21.15.151:0/2971363812 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b8035040 con 0x55a0cd2349b0 2024-04-26T01:59:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.294+0000 7f30abe00640 1 -- 172.21.15.151:0/2971363812 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7f30b80042f0 con 0x55a0cd2349b0 2024-04-26T01:59:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.294+0000 7f30cfd7c740 1 -- 172.21.15.151:0/2971363812 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30b8053630 con 0x55a0cd2349b0 2024-04-26T01:59:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.294+0000 7f30abe00640 1 -- 172.21.15.151:0/2971363812 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f30b806a050 con 0x55a0cd2349b0 2024-04-26T01:59:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.294+0000 7f30cfd7c740 10 client.25805 fetch_fsmap learned FSMap version 32 2024-04-26T01:59:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.294+0000 7f30cfd7c740 1 -- 172.21.15.151:0/2971363812 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a0cd2b5be0 con 0x55a0cd2349b0 2024-04-26T01:59:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.294+0000 7f30abe00640 1 -- 172.21.15.151:0/2971363812 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30b8053630 con 0x55a0cd2349b0 2024-04-26T01:59:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.295+0000 7f30abe00640 1 -- 172.21.15.151:0/2971363812 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f30b8003ce0 con 0x55a0cd2349b0 2024-04-26T01:59:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.295+0000 7f30cfd7c740 10 client.25805 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T01:59:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.295+0000 7f30cfd7c740 10 client.25805 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:59:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.295+0000 7f30cfd7c740 20 client.25805 populate_metadata read hostname 'smithi151' 2024-04-26T01:59:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.295+0000 7f30cfd7c740 1 --2- 172.21.15.151:0/2971363812 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55a0cd2b5e60 0x55a0cd2b8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.295+0000 7f30cfd7c740 4 client.25805 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:59:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.295+0000 7f30cfd7c740 1 -- 172.21.15.151:0/2971363812 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a0cd1a8320 con 0x55a0cd2b5e60 2024-04-26T01:59:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.295+0000 7f30bfe00640 1 --2- 172.21.15.151:0/2971363812 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55a0cd2b5e60 0x55a0cd2b8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.295+0000 7f30bfe00640 1 --2- 172.21.15.151:0/2971363812 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55a0cd2b5e60 0x55a0cd2b8250 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20efc603b7a3ac98 server_cookie=60ad0cb47f87b516 in_seq=0 out_seq=0 2024-04-26T01:59:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.295+0000 7f30abe00640 10 client.25805 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:59:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.315+0000 7f30abe00640 1 -- 172.21.15.151:0/2971363812 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a0cd1a8320 con 0x55a0cd2b5e60 2024-04-26T01:59:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.315+0000 7f30abe00640 10 client.25805 handle_command_reply: tid=0 2024-04-26T01:59:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.315+0000 7f30cfd7c740 1 client.25805 shutdown 2024-04-26T01:59:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.315+0000 7f30cfd7c740 1 -- 172.21.15.151:0/2971363812 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a0cd2349b0 msgr2=0x55a0cd244cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.315+0000 7f30cfd7c740 1 --2- 172.21.15.151:0/2971363812 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a0cd2349b0 0x55a0cd244cf0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f30b8051040 tx=0x7f30b802a410 comp rx=0 tx=0).stop 2024-04-26T01:59:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.315+0000 7f30cfd7c740 1 -- 172.21.15.151:0/2971363812 shutdown_connections 2024-04-26T01:59:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.315+0000 7f30cfd7c740 1 --2- 172.21.15.151:0/2971363812 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55a0cd2b5e60 0x55a0cd2b8250 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.315+0000 7f30cfd7c740 1 --2- 172.21.15.151:0/2971363812 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0cd23fb90 0x55a0cd2b43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.315+0000 7f30cfd7c740 1 --2- 172.21.15.151:0/2971363812 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a0cd2352d0 0x55a0cd245230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.315+0000 7f30abe00640 0 client.25805 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:59:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.315+0000 7f30cfd7c740 1 --2- 172.21.15.151:0/2971363812 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a0cd2349b0 0x55a0cd244cf0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.315+0000 7f30cfd7c740 1 -- 172.21.15.151:0/2971363812 >> 172.21.15.151:0/2971363812 conn(0x55a0cd230ec0 msgr2=0x55a0cd23e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.315+0000 7f30cfd7c740 1 -- 172.21.15.151:0/2971363812 shutdown_connections 2024-04-26T01:59:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.315+0000 7f30cfd7c740 1 -- 172.21.15.151:0/2971363812 wait complete. 2024-04-26T01:59:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.315+0000 7f30cfd7c740 20 client.25805 trim_cache size 0 max 16384 2024-04-26T01:59:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:{ 2024-04-26T01:59:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "message": "", 2024-04-26T01:59:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "return_code": 0 2024-04-26T01:59:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:} 2024-04-26T01:59:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.316+0000 7f30c7200640 1 -- 172.21.15.151:0/2630497745 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f30ac0a3d40 msgr2=0x7f30ac0a6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.316+0000 7f30c7200640 1 --2- 172.21.15.151:0/2630497745 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f30ac0a3d40 0x7f30ac0a6200 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f30c0009d40 tx=0x7f30c0009500 comp rx=0 tx=0).stop 2024-04-26T01:59:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.316+0000 7f30c7200640 1 -- 172.21.15.151:0/2630497745 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30c81070b0 msgr2=0x7f30c8074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.316+0000 7f30c7200640 1 --2- 172.21.15.151:0/2630497745 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30c81070b0 0x7f30c8074790 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f30b000e9c0 tx=0x7f30b000ee80 comp rx=0 tx=0).stop 2024-04-26T01:59:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.317+0000 7f30c7200640 1 -- 172.21.15.151:0/2630497745 shutdown_connections 2024-04-26T01:59:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.317+0000 7f30c7200640 1 --2- 172.21.15.151:0/2630497745 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f30ac0a3d40 0x7f30ac0a6200 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.317+0000 7f30c7200640 1 --2- 172.21.15.151:0/2630497745 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30c8074cd0 0x7f30c81a6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.317+0000 7f30c7200640 1 --2- 172.21.15.151:0/2630497745 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30c81070b0 0x7f30c8074790 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.317+0000 7f30c7200640 1 --2- 172.21.15.151:0/2630497745 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30c8101fe0 0x7f30c8074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.317+0000 7f30c7200640 1 -- 172.21.15.151:0/2630497745 >> 172.21.15.151:0/2630497745 conn(0x7f30c80fd8a0 msgr2=0x7f30c80fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.317+0000 7f30c7200640 1 -- 172.21.15.151:0/2630497745 shutdown_connections 2024-04-26T01:59:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:25.317+0000 7f30c7200640 1 -- 172.21.15.151:0/2630497745 wait complete. 2024-04-26T01:59:26.414 INFO:tasks.workunit.client.0.smithi151.stdout:copying to largest ever: 0x9ff773 2024-04-26T01:59:26.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:59:26 smithi172 ceph-mon[32785]: pgmap v178: 97 pgs: 97 active+clean; 51 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 2.3 MiB/s wr, 384 op/s 2024-04-26T01:59:26.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:59:26 smithi165 ceph-mon[32648]: pgmap v178: 97 pgs: 97 active+clean; 51 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 2.3 MiB/s wr, 384 op/s 2024-04-26T01:59:26.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:59:26 smithi151 ceph-mon[28057]: pgmap v178: 97 pgs: 97 active+clean; 51 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 2.3 MiB/s wr, 384 op/s 2024-04-26T01:59:27.410 INFO:tasks.workunit.client.0.smithi151.stdout:20000 punch from 0x34ddcb to 0x35324f, (0x5484 bytes) 2024-04-26T01:59:28.536 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:59:28 smithi172 ceph-mon[32785]: pgmap v179: 97 pgs: 97 active+clean; 51 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.6 MiB/s wr, 254 op/s 2024-04-26T01:59:28.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:59:28 smithi165 ceph-mon[32648]: pgmap v179: 97 pgs: 97 active+clean; 51 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.6 MiB/s wr, 254 op/s 2024-04-26T01:59:28.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:59:28 smithi151 ceph-mon[28057]: pgmap v179: 97 pgs: 97 active+clean; 51 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.6 MiB/s wr, 254 op/s 2024-04-26T01:59:30.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:59:30 smithi172 ceph-mon[32785]: pgmap v180: 97 pgs: 97 active+clean; 47 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.4 MiB/s wr, 389 op/s 2024-04-26T01:59:30.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:59:30 smithi165 ceph-mon[32648]: pgmap v180: 97 pgs: 97 active+clean; 47 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.4 MiB/s wr, 389 op/s 2024-04-26T01:59:30.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:59:30 smithi151 ceph-mon[28057]: pgmap v180: 97 pgs: 97 active+clean; 47 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.4 MiB/s wr, 389 op/s 2024-04-26T01:59:32.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:59:32 smithi172 ceph-mon[32785]: pgmap v181: 97 pgs: 97 active+clean; 47 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.7 MiB/s wr, 270 op/s 2024-04-26T01:59:32.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:59:32 smithi165 ceph-mon[32648]: pgmap v181: 97 pgs: 97 active+clean; 47 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.7 MiB/s wr, 270 op/s 2024-04-26T01:59:32.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:59:32 smithi151 ceph-mon[28057]: pgmap v181: 97 pgs: 97 active+clean; 47 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.7 MiB/s wr, 270 op/s 2024-04-26T01:59:34.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:59:34 smithi165 ceph-mon[32648]: pgmap v182: 97 pgs: 97 active+clean; 47 MiB data, 919 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.7 MiB/s wr, 272 op/s 2024-04-26T01:59:34.624 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:59:34 smithi151 ceph-mon[28057]: pgmap v182: 97 pgs: 97 active+clean; 47 MiB data, 919 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.7 MiB/s wr, 272 op/s 2024-04-26T01:59:34.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:59:34 smithi172 ceph-mon[32785]: pgmap v182: 97 pgs: 97 active+clean; 47 MiB data, 919 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.7 MiB/s wr, 272 op/s 2024-04-26T01:59:34.999 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10 2024-04-26T01:59:34.999 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:59:35.503 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:59:36.589 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:59:36 smithi172 ceph-mon[32785]: pgmap v183: 97 pgs: 97 active+clean; 55 MiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 2.1 MiB/s wr, 342 op/s 2024-04-26T01:59:36.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:59:36 smithi165 ceph-mon[32648]: pgmap v183: 97 pgs: 97 active+clean; 55 MiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 2.1 MiB/s wr, 342 op/s 2024-04-26T01:59:36.613 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:59:36 smithi151 ceph-mon[28057]: pgmap v183: 97 pgs: 97 active+clean; 55 MiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 2.1 MiB/s wr, 342 op/s 2024-04-26T01:59:38.373 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:59:38 smithi151 ceph-mon[28057]: pgmap v184: 97 pgs: 97 active+clean; 55 MiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.5 MiB/s wr, 248 op/s 2024-04-26T01:59:38.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:38.425+0000 7f7115a00700 1 -- 172.21.15.151:0/2152788126 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f711006a750 msgr2=0x7f711006ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:38.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:38.425+0000 7f7115a00700 1 --2- 172.21.15.151:0/2152788126 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f711006a750 0x7f711006ab30 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f71000098f0 tx=0x7f7100009c00 comp rx=0 tx=0).stop 2024-04-26T01:59:38.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:38.426+0000 7f7115a00700 1 -- 172.21.15.151:0/2152788126 shutdown_connections 2024-04-26T01:59:38.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:38.426+0000 7f7115a00700 1 --2- 172.21.15.151:0/2152788126 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f711010f4f0 0x7f71101118e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:38.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:38.426+0000 7f7115a00700 1 --2- 172.21.15.151:0/2152788126 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f711006a750 0x7f711006ab30 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:38.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:38.426+0000 7f7115a00700 1 --2- 172.21.15.151:0/2152788126 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f711006b070 0x7f711010efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:38.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:38.426+0000 7f7115a00700 1 -- 172.21.15.151:0/2152788126 >> 172.21.15.151:0/2152788126 conn(0x7f71100788f0 msgr2=0x7f7110078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:38.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:38.426+0000 7f7115a00700 1 -- 172.21.15.151:0/2152788126 shutdown_connections 2024-04-26T01:59:38.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:38.426+0000 7f7115a00700 1 -- 172.21.15.151:0/2152788126 wait complete. 2024-04-26T01:59:38.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:38.427+0000 7f7115a00700 1 Processor -- start 2024-04-26T01:59:38.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:38.427+0000 7f7115a00700 1 -- start start 2024-04-26T01:59:38.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:38.428+0000 7f7115a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f711006a750 0x7f711019b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:38.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:38.428+0000 7f7115a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f711006b070 0x7f711019b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:38.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:38.428+0000 7f7115a00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f711010f4f0 0x7f711019fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:38.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:38.428+0000 7f7115a00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7110114e70 con 0x7f711006b070 2024-04-26T01:59:38.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:38.428+0000 7f7115a00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7110114cf0 con 0x7f711006a750 2024-04-26T01:59:38.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:38.428+0000 7f7115a00700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7110114ff0 con 0x7f711010f4f0 2024-04-26T01:59:38.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:38.428+0000 7f710ea00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f711006a750 0x7f711019b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:38.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:38.428+0000 7f710ea00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f711006a750 0x7f711019b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:40446/0 (socket says 172.21.15.151:40446) 2024-04-26T01:59:38.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:38.428+0000 7f710ea00700 1 -- 172.21.15.151:0/1918497915 learned_addr learned my addr 172.21.15.151:0/1918497915 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:59:38.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:38.428+0000 7f710e000700 1 --2- 172.21.15.151:0/1918497915 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f711006b070 0x7f711019b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:38.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:38.428+0000 7f710f400700 1 --2- 172.21.15.151:0/1918497915 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f711010f4f0 0x7f711019fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:38.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:38.429+0000 7f710ea00700 1 -- 172.21.15.151:0/1918497915 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f711010f4f0 msgr2=0x7f711019fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:38.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:38.429+0000 7f710ea00700 1 --2- 172.21.15.151:0/1918497915 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f711010f4f0 0x7f711019fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:38.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:38.429+0000 7f710ea00700 1 -- 172.21.15.151:0/1918497915 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f711006b070 msgr2=0x7f711019b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:38.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:38.429+0000 7f710ea00700 1 --2- 172.21.15.151:0/1918497915 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f711006b070 0x7f711019b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:38.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:38.429+0000 7f710ea00700 1 -- 172.21.15.151:0/1918497915 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71101a03d0 con 0x7f711006a750 2024-04-26T01:59:38.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:38.429+0000 7f710e000700 1 --2- 172.21.15.151:0/1918497915 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f711006b070 0x7f711019b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:59:38.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:38.429+0000 7f710ea00700 1 --2- 172.21.15.151:0/1918497915 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f711006a750 0x7f711019b3c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7100009230 tx=0x7f710000fa70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:38.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:38.429+0000 7f7107400700 1 -- 172.21.15.151:0/1918497915 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7100005140 con 0x7f711006a750 2024-04-26T01:59:38.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:38.429+0000 7f7107400700 1 -- 172.21.15.151:0/1918497915 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f710000b560 con 0x7f711006a750 2024-04-26T01:59:38.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:38.429+0000 7f7107400700 1 -- 172.21.15.151:0/1918497915 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7100003ae0 con 0x7f711006a750 2024-04-26T01:59:38.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:38.429+0000 7f7115a00700 1 -- 172.21.15.151:0/1918497915 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71101a0660 con 0x7f711006a750 2024-04-26T01:59:38.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:38.430+0000 7f7115a00700 1 -- 172.21.15.151:0/1918497915 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71101a7f40 con 0x7f711006a750 2024-04-26T01:59:38.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:38.431+0000 7f7115a00700 1 -- 172.21.15.151:0/1918497915 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7110101f60 con 0x7f711006a750 2024-04-26T01:59:38.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:38.433+0000 7f7107400700 1 -- 172.21.15.151:0/1918497915 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f7100003c80 con 0x7f711006a750 2024-04-26T01:59:38.433 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:38.435+0000 7f7107400700 1 --2- 172.21.15.151:0/1918497915 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f70f40a3e30 0x7f70f40a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:38.434 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:38.435+0000 7f7107400700 1 -- 172.21.15.151:0/1918497915 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7f71000d1710 con 0x7f711006a750 2024-04-26T01:59:38.434 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:38.435+0000 7f710e000700 1 --2- 172.21.15.151:0/1918497915 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f70f40a3e30 0x7f70f40a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:38.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:38.436+0000 7f710e000700 1 --2- 172.21.15.151:0/1918497915 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f70f40a3e30 0x7f70f40a62f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7110102f80 tx=0x7f70fc009260 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:38.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:38.437+0000 7f7107400700 1 -- 172.21.15.151:0/1918497915 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7100013030 con 0x7f711006a750 2024-04-26T01:59:38.476 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.476+0000 7f16e9c00640 1 -- 172.21.15.151:0/1231802379 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16e4103830 msgr2=0x7f16e410bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:38.476 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.476+0000 7f16e9c00640 1 --2- 172.21.15.151:0/1231802379 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16e4103830 0x7f16e410bd10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f16d0009900 tx=0x7f16d002a480 comp rx=0 tx=0).stop 2024-04-26T01:59:38.476 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.477+0000 7f16e9c00640 1 -- 172.21.15.151:0/1231802379 shutdown_connections 2024-04-26T01:59:38.476 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.477+0000 7f16e9c00640 1 --2- 172.21.15.151:0/1231802379 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f16e410c330 0x7f16e410e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:38.476 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.477+0000 7f16e9c00640 1 --2- 172.21.15.151:0/1231802379 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16e4103830 0x7f16e410bd10 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:38.476 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.477+0000 7f16e9c00640 1 --2- 172.21.15.151:0/1231802379 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16e4102f10 0x7f16e41032f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:38.476 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.477+0000 7f16e9c00640 1 -- 172.21.15.151:0/1231802379 >> 172.21.15.151:0/1231802379 conn(0x7f16e40fc770 msgr2=0x7f16e40feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:38.476 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.477+0000 7f16e9c00640 1 -- 172.21.15.151:0/1231802379 shutdown_connections 2024-04-26T01:59:38.476 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.477+0000 7f16e9c00640 1 -- 172.21.15.151:0/1231802379 wait complete. 2024-04-26T01:59:38.477 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.478+0000 7f16e9c00640 1 Processor -- start 2024-04-26T01:59:38.477 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.478+0000 7f16e9c00640 1 -- start start 2024-04-26T01:59:38.478 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.478+0000 7f16e9c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16e4102f10 0x7f16e41985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:38.478 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.479+0000 7f16e9c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f16e4103830 0x7f16e4198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:38.478 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.479+0000 7f16e9c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16e410c330 0x7f16e419ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:38.478 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.479+0000 7f16e9c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f16e4112010 con 0x7f16e4103830 2024-04-26T01:59:38.478 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.479+0000 7f16e9c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f16e4111e90 con 0x7f16e410c330 2024-04-26T01:59:38.478 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.479+0000 7f16e9c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f16e4112190 con 0x7f16e4102f10 2024-04-26T01:59:38.478 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.479+0000 7f16e3e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16e4102f10 0x7f16e41985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:38.478 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.479+0000 7f16e3e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16e4102f10 0x7f16e41985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:52876/0 (socket says 172.21.15.151:52876) 2024-04-26T01:59:38.478 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.479+0000 7f16e3e00640 1 -- 172.21.15.151:0/4099224617 learned_addr learned my addr 172.21.15.151:0/4099224617 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:59:38.479 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.479+0000 7f16e3400640 1 --2- 172.21.15.151:0/4099224617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f16e4103830 0x7f16e4198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:38.479 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.479+0000 7f16e3e00640 1 -- 172.21.15.151:0/4099224617 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16e410c330 msgr2=0x7f16e419ce60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T01:59:38.479 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.479+0000 7f16e3e00640 1 --2- 172.21.15.151:0/4099224617 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16e410c330 0x7f16e419ce60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:38.479 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.479+0000 7f16e3e00640 1 -- 172.21.15.151:0/4099224617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f16e4103830 msgr2=0x7f16e4198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:38.479 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.479+0000 7f16e3e00640 1 --2- 172.21.15.151:0/4099224617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f16e4103830 0x7f16e4198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:38.479 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.479+0000 7f16e3e00640 1 -- 172.21.15.151:0/4099224617 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16e419d590 con 0x7f16e4102f10 2024-04-26T01:59:38.480 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.480+0000 7f16e3e00640 1 --2- 172.21.15.151:0/4099224617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16e4102f10 0x7f16e41985f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f16d400e9f0 tx=0x7f16d400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:38.480 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.480+0000 7f16e0c00640 1 -- 172.21.15.151:0/4099224617 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d400cd80 con 0x7f16e4102f10 2024-04-26T01:59:38.480 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.480+0000 7f16e0c00640 1 -- 172.21.15.151:0/4099224617 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16d40054d0 con 0x7f16e4102f10 2024-04-26T01:59:38.480 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.480+0000 7f16e9c00640 1 -- 172.21.15.151:0/4099224617 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16e419d880 con 0x7f16e4102f10 2024-04-26T01:59:38.480 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.480+0000 7f16e0c00640 1 -- 172.21.15.151:0/4099224617 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d4010490 con 0x7f16e4102f10 2024-04-26T01:59:38.480 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.480+0000 7f16e9c00640 1 -- 172.21.15.151:0/4099224617 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16e4100a90 con 0x7f16e4102f10 2024-04-26T01:59:38.483 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.484+0000 7f16edfe9740 1 Processor -- start 2024-04-26T01:59:38.484 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.485+0000 7f16e0c00640 1 -- 172.21.15.151:0/4099224617 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f16d40046c0 con 0x7f16e4102f10 2024-04-26T01:59:38.484 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.485+0000 7f16edfe9740 1 -- start start 2024-04-26T01:59:38.484 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.485+0000 7f16edfe9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b2789829b0 0x55b278982d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:38.484 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.485+0000 7f16edfe9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2789832d0 0x55b27898d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:38.484 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.485+0000 7f16edfe9740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b27898db90 0x55b27898ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:38.484 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.485+0000 7f16edfe9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b2788e8900 con 0x55b2789829b0 2024-04-26T01:59:38.484 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.485+0000 7f16edfe9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b27888eef0 con 0x55b2789832d0 2024-04-26T01:59:38.484 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.485+0000 7f16edfe9740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b2788e48c0 con 0x55b27898db90 2024-04-26T01:59:38.485 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.485+0000 7f16e3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2789832d0 0x55b27898d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:38.485 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.486+0000 7f16e3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2789832d0 0x55b27898d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:40470/0 (socket says 172.21.15.151:40470) 2024-04-26T01:59:38.485 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.486+0000 7f16e3e00640 1 -- 172.21.15.151:0/3689261015 learned_addr learned my addr 172.21.15.151:0/3689261015 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:59:38.485 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.486+0000 7f16e3400640 1 --2- 172.21.15.151:0/3689261015 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b27898db90 0x55b27898ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.485+0000 7f16e9200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b2789829b0 0x55b278982d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.486+0000 7f16e3400640 1 -- 172.21.15.151:0/3689261015 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2789832d0 msgr2=0x55b27898d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.486+0000 7f16e3400640 1 --2- 172.21.15.151:0/3689261015 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2789832d0 0x55b27898d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.486+0000 7f16e3400640 1 -- 172.21.15.151:0/3689261015 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b2789829b0 msgr2=0x55b278982d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.486+0000 7f16e3400640 1 --2- 172.21.15.151:0/3689261015 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b2789829b0 0x55b278982d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.486+0000 7f16e3400640 1 -- 172.21.15.151:0/3689261015 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2785fb210 con 0x55b27898db90 2024-04-26T01:59:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.486+0000 7f16e3e00640 1 --2- 172.21.15.151:0/3689261015 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2789832d0 0x55b27898d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:59:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.486+0000 7f16e0c00640 1 --2- 172.21.15.151:0/4099224617 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f16c80a3d90 0x7f16c80a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.487+0000 7f16e3400640 1 --2- 172.21.15.151:0/3689261015 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b27898db90 0x55b27898ff80 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f16d002aa70 tx=0x7f16d00040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b5f30039f17517b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.487+0000 7f16e0c00640 1 -- 172.21.15.151:0/4099224617 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7f16d40d0c10 con 0x7f16e4102f10 2024-04-26T01:59:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.487+0000 7f16d9600640 1 -- 172.21.15.151:0/3689261015 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d0035030 con 0x55b27898db90 2024-04-26T01:59:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.487+0000 7f16d9600640 1 -- 172.21.15.151:0/3689261015 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16d00044b0 con 0x55b27898db90 2024-04-26T01:59:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.487+0000 7f16d9600640 1 -- 172.21.15.151:0/3689261015 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d00035f0 con 0x55b27898db90 2024-04-26T01:59:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.487+0000 7f16e9200640 1 --2- 172.21.15.151:0/4099224617 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f16c80a3d90 0x7f16c80a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.488+0000 7f16edfe9740 1 -- 172.21.15.151:0/3689261015 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b27898db90 msgr2=0x55b27898ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.488+0000 7f16edfe9740 1 --2- 172.21.15.151:0/3689261015 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b27898db90 0x55b27898ff80 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f16d002aa70 tx=0x7f16d00040c0 comp rx=0 tx=0).stop 2024-04-26T01:59:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.488+0000 7f16e9200640 1 --2- 172.21.15.151:0/4099224617 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f16c80a3d90 0x7f16c80a6250 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f16dc005fd0 tx=0x7f16dc009500 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.488+0000 7f16edfe9740 1 -- 172.21.15.151:0/3689261015 shutdown_connections 2024-04-26T01:59:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.489+0000 7f16edfe9740 1 --2- 172.21.15.151:0/3689261015 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b27898db90 0x55b27898ff80 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.489+0000 7f16edfe9740 1 --2- 172.21.15.151:0/3689261015 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2789832d0 0x55b27898d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.489+0000 7f16edfe9740 1 --2- 172.21.15.151:0/3689261015 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b2789829b0 0x55b278982d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.489+0000 7f16edfe9740 1 -- 172.21.15.151:0/3689261015 >> 172.21.15.151:0/3689261015 conn(0x55b27897eec0 msgr2=0x55b27897f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:38.489 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.490+0000 7f16edfe9740 1 -- 172.21.15.151:0/3689261015 shutdown_connections 2024-04-26T01:59:38.489 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.490+0000 7f16edfe9740 1 -- 172.21.15.151:0/3689261015 wait complete. 2024-04-26T01:59:38.489 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.490+0000 7f16edfe9740 1 -- start start 2024-04-26T01:59:38.489 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.490+0000 7f16edfe9740 1 Processor -- start 2024-04-26T01:59:38.490 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.491+0000 7f16edfe9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2789829b0 0x55b27898aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:38.490 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.491+0000 7f16edfe9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b2789832d0 0x55b27898b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:38.490 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.491+0000 7f16edfe9740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b27898db90 0x55b27898b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:38.490 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.491+0000 7f16edfe9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b278990760 con 0x55b2789832d0 2024-04-26T01:59:38.490 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.491+0000 7f16edfe9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b2788a2310 con 0x55b2789829b0 2024-04-26T01:59:38.490 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.491+0000 7f16edfe9740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b2789908e0 con 0x55b27898db90 2024-04-26T01:59:38.492 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.491+0000 7f16edfe9740 10 client.0 _ll_register_callbacks cb 0x55b278870cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:59:38.492 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.491+0000 7f16e3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2789829b0 0x55b27898aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:38.492 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.491+0000 7f16e3400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b2789832d0 0x55b27898b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:38.492 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.491+0000 7f16e9200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b27898db90 0x55b27898b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:38.492 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.491+0000 7f16e9200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b27898db90 0x55b27898b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:52890/0 (socket says 172.21.15.151:52890) 2024-04-26T01:59:38.492 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.491+0000 7f16e3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2789829b0 0x55b27898aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:40476/0 (socket says 172.21.15.151:40476) 2024-04-26T01:59:38.492 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.491+0000 7f16e9200640 1 -- 172.21.15.151:0/134125032 learned_addr learned my addr 172.21.15.151:0/134125032 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:59:38.492 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.492+0000 7f16e9200640 1 -- 172.21.15.151:0/134125032 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2789829b0 msgr2=0x55b27898aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:38.493 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.492+0000 7f16e9200640 1 --2- 172.21.15.151:0/134125032 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2789829b0 0x55b27898aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:38.493 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.492+0000 7f16e9200640 1 -- 172.21.15.151:0/134125032 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b2789832d0 msgr2=0x55b27898b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:38.493 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.492+0000 7f16e9200640 1 --2- 172.21.15.151:0/134125032 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b2789832d0 0x55b27898b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:38.493 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.492+0000 7f16e9200640 1 -- 172.21.15.151:0/134125032 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2786c7620 con 0x55b27898db90 2024-04-26T01:59:38.493 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.492+0000 7f16e3400640 1 --2- 172.21.15.151:0/134125032 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b2789832d0 0x55b27898b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:59:38.493 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.492+0000 7f16e9200640 1 --2- 172.21.15.151:0/134125032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b27898db90 0x55b27898b620 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f16dc005c20 tx=0x7f16dc004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e200b66931902f4f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:38.493 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.492+0000 7f16ce000640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-26T01:59:38.493 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.493+0000 7f16ce000640 1 -- 172.21.15.151:0/134125032 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16dc004d20 con 0x55b27898db90 2024-04-26T01:59:38.493 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.493+0000 7f16edfe9740 1 -- 172.21.15.151:0/134125032 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16d00044b0 con 0x55b27898db90 2024-04-26T01:59:38.493 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.493+0000 7f16ce000640 1 -- 172.21.15.151:0/134125032 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7f16dc00e930 con 0x55b27898db90 2024-04-26T01:59:38.493 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.493+0000 7f16ce000640 1 -- 172.21.15.151:0/134125032 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f16dc00eb30 con 0x55b27898db90 2024-04-26T01:59:38.493 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.493+0000 7f16edfe9740 10 client.25864 fetch_fsmap learned FSMap version 32 2024-04-26T01:59:38.493 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.493+0000 7f16edfe9740 1 -- 172.21.15.151:0/134125032 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b27899cab0 con 0x55b27898db90 2024-04-26T01:59:38.495 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.494+0000 7f16ce000640 1 -- 172.21.15.151:0/134125032 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16d00044b0 con 0x55b27898db90 2024-04-26T01:59:38.495 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.494+0000 7f16ce000640 1 -- 172.21.15.151:0/134125032 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f16dc02e050 con 0x55b27898db90 2024-04-26T01:59:38.495 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.494+0000 7f16edfe9740 10 client.25864 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T01:59:38.496 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.494+0000 7f16edfe9740 10 client.25864 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:59:38.496 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.494+0000 7f16edfe9740 20 client.25864 populate_metadata read hostname 'smithi151' 2024-04-26T01:59:38.496 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.494+0000 7f16edfe9740 1 --2- 172.21.15.151:0/134125032 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55b27899df20 0x55b2789a0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:38.496 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.494+0000 7f16edfe9740 4 client.25864 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:59:38.496 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.494+0000 7f16edfe9740 1 -- 172.21.15.151:0/134125032 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b2788fc8b0 con 0x55b27899df20 2024-04-26T01:59:38.496 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.494+0000 7f16e3e00640 1 --2- 172.21.15.151:0/134125032 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55b27899df20 0x55b2789a0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:38.496 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.495+0000 7f16e3e00640 1 --2- 172.21.15.151:0/134125032 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55b27899df20 0x55b2789a0330 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7052fdd46fe2d863 server_cookie=77183e99d50ca8ec in_seq=0 out_seq=0 2024-04-26T01:59:38.496 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.495+0000 7f16ce000640 10 client.25864 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:59:38.496 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.496+0000 7f16ce000640 1 -- 172.21.15.151:0/134125032 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b2788fc8b0 con 0x55b27899df20 2024-04-26T01:59:38.496 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.496+0000 7f16ce000640 10 client.25864 handle_command_reply: tid=0 2024-04-26T01:59:38.496 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.496+0000 7f16edfe9740 1 client.25864 shutdown 2024-04-26T01:59:38.496 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.496+0000 7f16edfe9740 1 -- 172.21.15.151:0/134125032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b27898db90 msgr2=0x55b27898b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:38.496 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.496+0000 7f16edfe9740 1 --2- 172.21.15.151:0/134125032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b27898db90 0x55b27898b620 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f16dc005c20 tx=0x7f16dc004ab0 comp rx=0 tx=0).stop 2024-04-26T01:59:38.497 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.496+0000 7f16edfe9740 1 -- 172.21.15.151:0/134125032 shutdown_connections 2024-04-26T01:59:38.497 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.496+0000 7f16edfe9740 1 --2- 172.21.15.151:0/134125032 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55b27899df20 0x55b2789a0330 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:38.497 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.496+0000 7f16edfe9740 1 --2- 172.21.15.151:0/134125032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b27898db90 0x55b27898b620 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:38.497 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.496+0000 7f16edfe9740 1 --2- 172.21.15.151:0/134125032 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b2789832d0 0x55b27898b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:38.497 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.496+0000 7f16edfe9740 1 --2- 172.21.15.151:0/134125032 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2789829b0 0x55b27898aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:38.497 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.496+0000 7f16edfe9740 1 -- 172.21.15.151:0/134125032 >> 172.21.15.151:0/134125032 conn(0x55b27897eec0 msgr2=0x55b278984c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:38.497 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.496+0000 7f16ce000640 0 client.25864 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:59:38.497 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.497+0000 7f16edfe9740 1 -- 172.21.15.151:0/134125032 shutdown_connections 2024-04-26T01:59:38.497 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.497+0000 7f16edfe9740 1 -- 172.21.15.151:0/134125032 wait complete. 2024-04-26T01:59:38.497 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.497+0000 7f16edfe9740 20 client.25864 trim_cache size 0 max 16384 2024-04-26T01:59:38.517 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.517+0000 7f16edfe9740 1 Processor -- start 2024-04-26T01:59:38.518 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.517+0000 7f16edfe9740 1 -- start start 2024-04-26T01:59:38.518 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.517+0000 7f16edfe9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2789829b0 0x55b27897e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:38.518 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.518+0000 7f16edfe9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b2789832d0 0x55b2789853c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:38.518 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.518+0000 7f16edfe9740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b27898db90 0x55b27898b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:38.518 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.518+0000 7f16edfe9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f16e41b22f0 con 0x55b2789832d0 2024-04-26T01:59:38.518 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.518+0000 7f16edfe9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f16e41afbc0 con 0x55b2789829b0 2024-04-26T01:59:38.518 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.518+0000 7f16edfe9740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f16e41ada10 con 0x55b27898db90 2024-04-26T01:59:38.518 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.518+0000 7f16e3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2789829b0 0x55b27897e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:38.518 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.518+0000 7f16e3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2789829b0 0x55b27897e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:40490/0 (socket says 172.21.15.151:40490) 2024-04-26T01:59:38.518 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.518+0000 7f16e9200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b27898db90 0x55b27898b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:38.518 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.518+0000 7f16e3e00640 1 -- 172.21.15.151:0/2086395594 learned_addr learned my addr 172.21.15.151:0/2086395594 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:59:38.518 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.518+0000 7f16e3400640 1 --2- 172.21.15.151:0/2086395594 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b2789832d0 0x55b2789853c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.518+0000 7f16e3e00640 1 -- 172.21.15.151:0/2086395594 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b27898db90 msgr2=0x55b27898b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.518+0000 7f16e3e00640 1 --2- 172.21.15.151:0/2086395594 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b27898db90 0x55b27898b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.518+0000 7f16e3e00640 1 -- 172.21.15.151:0/2086395594 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b2789832d0 msgr2=0x55b2789853c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.518+0000 7f16e3e00640 1 --2- 172.21.15.151:0/2086395594 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b2789832d0 0x55b2789853c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.518+0000 7f16e3e00640 1 -- 172.21.15.151:0/2086395594 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2789901b0 con 0x55b2789829b0 2024-04-26T01:59:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.518+0000 7f16e9200640 1 --2- 172.21.15.151:0/2086395594 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b27898db90 0x55b27898b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:59:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.519+0000 7f16e3e00640 1 --2- 172.21.15.151:0/2086395594 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2789829b0 0x55b27897e7a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f16d40995f0 tx=0x7f16d40bd040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f0123b20b9b17be server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.519+0000 7f16d8c00640 1 -- 172.21.15.151:0/2086395594 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d4005680 con 0x55b2789829b0 2024-04-26T01:59:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.519+0000 7f16d8c00640 1 -- 172.21.15.151:0/2086395594 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16d409d320 con 0x55b2789829b0 2024-04-26T01:59:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.519+0000 7f16d8c00640 1 -- 172.21.15.151:0/2086395594 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d409ad50 con 0x55b2789829b0 2024-04-26T01:59:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.520+0000 7f16edfe9740 1 -- 172.21.15.151:0/2086395594 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2789829b0 msgr2=0x55b27897e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.520+0000 7f16edfe9740 1 --2- 172.21.15.151:0/2086395594 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2789829b0 0x55b27897e7a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f16d40995f0 tx=0x7f16d40bd040 comp rx=0 tx=0).stop 2024-04-26T01:59:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.520+0000 7f16edfe9740 1 -- 172.21.15.151:0/2086395594 shutdown_connections 2024-04-26T01:59:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.520+0000 7f16edfe9740 1 --2- 172.21.15.151:0/2086395594 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b27898db90 0x55b27898b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.520+0000 7f16edfe9740 1 --2- 172.21.15.151:0/2086395594 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b2789832d0 0x55b2789853c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.520+0000 7f16edfe9740 1 --2- 172.21.15.151:0/2086395594 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2789829b0 0x55b27897e7a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.520+0000 7f16edfe9740 1 -- 172.21.15.151:0/2086395594 >> 172.21.15.151:0/2086395594 conn(0x55b27897eec0 msgr2=0x55b27898ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:38.520 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.521+0000 7f16edfe9740 1 -- 172.21.15.151:0/2086395594 shutdown_connections 2024-04-26T01:59:38.521 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.521+0000 7f16edfe9740 1 -- 172.21.15.151:0/2086395594 wait complete. 2024-04-26T01:59:38.521 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.522+0000 7f16edfe9740 1 -- start start 2024-04-26T01:59:38.521 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.522+0000 7f16edfe9740 1 Processor -- start 2024-04-26T01:59:38.522 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.523+0000 7f16edfe9740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b2789829b0 0x55b278992e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:38.522 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.523+0000 7f16edfe9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2789832d0 0x55b2789933d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:38.522 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.523+0000 7f16edfe9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b27898db90 0x55b2789fc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:38.522 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.523+0000 7f16edfe9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f16e41ae080 con 0x55b27898db90 2024-04-26T01:59:38.522 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.523+0000 7f16edfe9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f16e41af4f0 con 0x55b2789832d0 2024-04-26T01:59:38.522 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.523+0000 7f16edfe9740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f16e41ae750 con 0x55b2789829b0 2024-04-26T01:59:38.522 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.523+0000 7f16edfe9740 10 client.0 _ll_register_callbacks cb 0x7f16dc003e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:59:38.522 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.523+0000 7f16e3e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b2789829b0 0x55b278992e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:38.522 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.523+0000 7f16e3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2789832d0 0x55b2789933d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:38.522 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.523+0000 7f16e3e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b2789829b0 0x55b278992e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:52906/0 (socket says 172.21.15.151:52906) 2024-04-26T01:59:38.523 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.523+0000 7f16e3e00640 1 -- 172.21.15.151:0/1640359181 learned_addr learned my addr 172.21.15.151:0/1640359181 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:59:38.523 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.524+0000 7f16e9200640 1 --2- 172.21.15.151:0/1640359181 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b27898db90 0x55b2789fc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:38.524 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.524+0000 7f16e3400640 1 -- 172.21.15.151:0/1640359181 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b2789829b0 msgr2=0x55b278992e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:38.524 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.524+0000 7f16e3400640 1 --2- 172.21.15.151:0/1640359181 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b2789829b0 0x55b278992e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:38.524 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.524+0000 7f16e3400640 1 -- 172.21.15.151:0/1640359181 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b27898db90 msgr2=0x55b2789fc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:38.524 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.524+0000 7f16e3400640 1 --2- 172.21.15.151:0/1640359181 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b27898db90 0x55b2789fc4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:38.524 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.524+0000 7f16e3400640 1 -- 172.21.15.151:0/1640359181 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2789fcb60 con 0x55b2789832d0 2024-04-26T01:59:38.524 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.524+0000 7f16e3400640 1 --2- 172.21.15.151:0/1640359181 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2789832d0 0x55b2789933d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f16d0005c40 tx=0x7f16d0003d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae96a5370e19a72c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:38.524 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.524+0000 7f16ce000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T01:59:38.524 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.525+0000 7f16ce000640 1 -- 172.21.15.151:0/1640359181 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d0051020 con 0x55b2789832d0 2024-04-26T01:59:38.524 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.525+0000 7f16ce000640 1 -- 172.21.15.151:0/1640359181 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7f16d0004390 con 0x55b2789832d0 2024-04-26T01:59:38.524 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.525+0000 7f16edfe9740 1 -- 172.21.15.151:0/1640359181 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16d409d320 con 0x55b2789832d0 2024-04-26T01:59:38.524 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.525+0000 7f16ce000640 1 -- 172.21.15.151:0/1640359181 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f16d0048040 con 0x55b2789832d0 2024-04-26T01:59:38.524 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.525+0000 7f16edfe9740 10 client.25826 fetch_fsmap learned FSMap version 32 2024-04-26T01:59:38.525 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.525+0000 7f16edfe9740 1 -- 172.21.15.151:0/1640359181 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b278a03f10 con 0x55b2789832d0 2024-04-26T01:59:38.525 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.525+0000 7f16ce000640 1 -- 172.21.15.151:0/1640359181 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16d409d320 con 0x55b2789832d0 2024-04-26T01:59:38.525 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.526+0000 7f16ce000640 1 -- 172.21.15.151:0/1640359181 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f16d00564d0 con 0x55b2789832d0 2024-04-26T01:59:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.527+0000 7f16edfe9740 10 client.25826 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T01:59:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.527+0000 7f16edfe9740 10 client.25826 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:59:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.527+0000 7f16edfe9740 20 client.25826 populate_metadata read hostname 'smithi151' 2024-04-26T01:59:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.527+0000 7f16edfe9740 1 --2- 172.21.15.151:0/1640359181 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55b278a04200 0x55b278a065f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.527+0000 7f16edfe9740 4 client.25826 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:59:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.527+0000 7f16edfe9740 1 -- 172.21.15.151:0/1640359181 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b2788f6320 con 0x55b278a04200 2024-04-26T01:59:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.527+0000 7f16e3e00640 1 --2- 172.21.15.151:0/1640359181 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55b278a04200 0x55b278a065f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.527+0000 7f16e3e00640 1 --2- 172.21.15.151:0/1640359181 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55b278a04200 0x55b278a065f0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5261a5d148157e29 server_cookie=97acd1a1473e2e4 in_seq=0 out_seq=0 2024-04-26T01:59:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.527+0000 7f16ce000640 10 client.25826 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:59:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.541+0000 7f16ce000640 1 -- 172.21.15.151:0/1640359181 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b2788f6320 con 0x55b278a04200 2024-04-26T01:59:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.541+0000 7f16ce000640 10 client.25826 handle_command_reply: tid=0 2024-04-26T01:59:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.541+0000 7f16edfe9740 1 client.25826 shutdown 2024-04-26T01:59:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.541+0000 7f16edfe9740 1 -- 172.21.15.151:0/1640359181 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2789832d0 msgr2=0x55b2789933d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.541+0000 7f16edfe9740 1 --2- 172.21.15.151:0/1640359181 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2789832d0 0x55b2789933d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f16d0005c40 tx=0x7f16d0003d20 comp rx=0 tx=0).stop 2024-04-26T01:59:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.541+0000 7f16edfe9740 1 -- 172.21.15.151:0/1640359181 shutdown_connections 2024-04-26T01:59:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.541+0000 7f16edfe9740 1 --2- 172.21.15.151:0/1640359181 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55b278a04200 0x55b278a065f0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.541+0000 7f16edfe9740 1 --2- 172.21.15.151:0/1640359181 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b27898db90 0x55b2789fc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.541+0000 7f16edfe9740 1 --2- 172.21.15.151:0/1640359181 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2789832d0 0x55b2789933d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.541+0000 7f16edfe9740 1 --2- 172.21.15.151:0/1640359181 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b2789829b0 0x55b278992e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.542+0000 7f16edfe9740 1 -- 172.21.15.151:0/1640359181 >> 172.21.15.151:0/1640359181 conn(0x55b27897eec0 msgr2=0x55b27898b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.542+0000 7f16ce000640 0 client.25826 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:59:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.542+0000 7f16edfe9740 1 -- 172.21.15.151:0/1640359181 shutdown_connections 2024-04-26T01:59:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.542+0000 7f16edfe9740 1 -- 172.21.15.151:0/1640359181 wait complete. 2024-04-26T01:59:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.542+0000 7f16edfe9740 20 client.25826 trim_cache size 0 max 16384 2024-04-26T01:59:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:{ 2024-04-26T01:59:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "message": "", 2024-04-26T01:59:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "return_code": 0 2024-04-26T01:59:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:} 2024-04-26T01:59:38.543 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.543+0000 7f16e9c00640 1 -- 172.21.15.151:0/4099224617 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f16c80a3d90 msgr2=0x7f16c80a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:38.543 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.543+0000 7f16e9c00640 1 --2- 172.21.15.151:0/4099224617 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f16c80a3d90 0x7f16c80a6250 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f16dc005fd0 tx=0x7f16dc009500 comp rx=0 tx=0).stop 2024-04-26T01:59:38.543 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.543+0000 7f16e9c00640 1 -- 172.21.15.151:0/4099224617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16e4102f10 msgr2=0x7f16e41985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:38.543 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.543+0000 7f16e9c00640 1 --2- 172.21.15.151:0/4099224617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16e4102f10 0x7f16e41985f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f16d400e9f0 tx=0x7f16d400edf0 comp rx=0 tx=0).stop 2024-04-26T01:59:38.543 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.544+0000 7f16e9c00640 1 -- 172.21.15.151:0/4099224617 shutdown_connections 2024-04-26T01:59:38.543 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.544+0000 7f16e9c00640 1 --2- 172.21.15.151:0/4099224617 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f16c80a3d90 0x7f16c80a6250 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:38.543 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.544+0000 7f16e9c00640 1 --2- 172.21.15.151:0/4099224617 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16e410c330 0x7f16e419ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:38.543 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.544+0000 7f16e9c00640 1 --2- 172.21.15.151:0/4099224617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f16e4103830 0x7f16e4198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:38.543 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.544+0000 7f16e9c00640 1 --2- 172.21.15.151:0/4099224617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16e4102f10 0x7f16e41985f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:38.543 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.544+0000 7f16e9c00640 1 -- 172.21.15.151:0/4099224617 >> 172.21.15.151:0/4099224617 conn(0x7f16e40fc770 msgr2=0x7f16e40fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.544+0000 7f16e9c00640 1 -- 172.21.15.151:0/4099224617 shutdown_connections 2024-04-26T01:59:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:38.544+0000 7f16e9c00640 1 -- 172.21.15.151:0/4099224617 wait complete. 2024-04-26T01:59:38.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:59:38 smithi165 ceph-mon[32648]: pgmap v184: 97 pgs: 97 active+clean; 55 MiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.5 MiB/s wr, 248 op/s 2024-04-26T01:59:38.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:59:38 smithi172 ceph-mon[32785]: pgmap v184: 97 pgs: 97 active+clean; 55 MiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.5 MiB/s wr, 248 op/s 2024-04-26T01:59:38.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:38.789+0000 7f7115a00700 1 -- 172.21.15.151:0/1918497915 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f711019c720 con 0x7f711006a750 2024-04-26T01:59:38.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:38.790+0000 7f7107400700 1 -- 172.21.15.151:0/1918497915 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11889 (secure 0 0 0) 0x7f710009de50 con 0x7f711006a750 2024-04-26T01:59:38.790 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:59:38.790 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2937864097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2937864097},{"type":"v1","addr":"172.21.15.151:6835","nonce":2937864097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14373,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/2947090802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2947090802},{"type":"v1","addr":"172.21.15.151:6837","nonce":2947090802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/1356563280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1356563280},{"type":"v1","addr":"172.21.15.151:6839","nonce":1356563280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6841/1075240552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1075240552},{"type":"v1","addr":"172.21.15.151:6841","nonce":1075240552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/504675912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":504675912},{"type":"v1","addr":"172.21.15.165:6833","nonce":504675912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1207580015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1207580015},{"type":"v1","addr":"172.21.15.172:6833","nonce":1207580015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:49:40.746876+0000","modified":"2024-04-26T01:58:46.154595+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24505,"mds_2":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/3598138294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3598138294},{"type":"v1","addr":"172.21.15.165:6835","nonce":3598138294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/1847859054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":1847859054},{"type":"v1","addr":"172.21.15.165:6837","nonce":1847859054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.172:6837/55972537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":55972537},{"type":"v1","addr":"172.21.15.172:6837","nonce":55972537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.172:6839/1242302810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1242302810},{"type":"v1","addr":"172.21.15.172:6839","nonce":1242302810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/1969716254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1969716254},{"type":"v1","addr":"172.21.15.165:6839","nonce":1969716254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25792":{"gid":25792,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/1657987703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":1657987703},{"type":"v1","addr":"172.21.15.172:6835","nonce":1657987703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24493,24505,24509]},"id":1}]} 2024-04-26T01:59:38.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:38.794+0000 7f7115a00700 1 -- 172.21.15.151:0/1918497915 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f70f40a3e30 msgr2=0x7f70f40a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:38.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:38.794+0000 7f7115a00700 1 --2- 172.21.15.151:0/1918497915 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f70f40a3e30 0x7f70f40a62f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7110102f80 tx=0x7f70fc009260 comp rx=0 tx=0).stop 2024-04-26T01:59:38.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:38.794+0000 7f7115a00700 1 -- 172.21.15.151:0/1918497915 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f711006a750 msgr2=0x7f711019b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:38.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:38.794+0000 7f7115a00700 1 --2- 172.21.15.151:0/1918497915 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f711006a750 0x7f711019b3c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7100009230 tx=0x7f710000fa70 comp rx=0 tx=0).stop 2024-04-26T01:59:38.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:38.794+0000 7f7115a00700 1 -- 172.21.15.151:0/1918497915 shutdown_connections 2024-04-26T01:59:38.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:38.794+0000 7f7115a00700 1 --2- 172.21.15.151:0/1918497915 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f70f40a3e30 0x7f70f40a62f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:38.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:38.794+0000 7f7115a00700 1 --2- 172.21.15.151:0/1918497915 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f711010f4f0 0x7f711019fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:38.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:38.794+0000 7f7115a00700 1 --2- 172.21.15.151:0/1918497915 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f711006a750 0x7f711019b3c0 secure :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7100009230 tx=0x7f710000fa70 comp rx=0 tx=0).stop 2024-04-26T01:59:38.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:38.794+0000 7f7115a00700 1 --2- 172.21.15.151:0/1918497915 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f711006b070 0x7f711019b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:38.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:38.794+0000 7f7115a00700 1 -- 172.21.15.151:0/1918497915 >> 172.21.15.151:0/1918497915 conn(0x7f71100788f0 msgr2=0x7f7110110a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:38.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:38.795+0000 7f7115a00700 1 -- 172.21.15.151:0/1918497915 shutdown_connections 2024-04-26T01:59:38.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:38.795+0000 7f7115a00700 1 -- 172.21.15.151:0/1918497915 wait complete. 2024-04-26T01:59:38.795 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 32 2024-04-26T01:59:39.287 INFO:journalctl@ceph.mds.f.smithi172.stdout:Apr 26 01:59:38 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-f[63189]: ignoring --setuser ceph since I am not root 2024-04-26T01:59:39.287 INFO:journalctl@ceph.mds.f.smithi172.stdout:Apr 26 01:59:38 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-f[63189]: ignoring --setgroup ceph since I am not root 2024-04-26T01:59:39.287 INFO:journalctl@ceph.mds.f.smithi172.stdout:Apr 26 01:59:38 smithi172 ceph-mds[63193]: -- 172.21.15.172:0/524288597 <== mon.1 v2:172.21.15.172:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55960b617a40 con 0x55960b5ea800 2024-04-26T01:59:39.287 INFO:journalctl@ceph.mds.f.smithi172.stdout:Apr 26 01:59:38 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-f[63189]: starting mds.f at 2024-04-26T01:59:39.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:59:39 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/1918497915' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:59:39.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:59:39 smithi165 ceph-mon[32648]: daemon mds.f restarted 2024-04-26T01:59:39.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:59:39 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T01:59:39.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:59:39 smithi165 ceph-mon[32648]: osdmap e95: 12 total, 12 up, 12 in 2024-04-26T01:59:39.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:59:39 smithi165 ceph-mon[32648]: mds.? [v2:172.21.15.172:6834/4230821277,v1:172.21.15.172:6835/4230821277] up:boot 2024-04-26T01:59:39.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:59:39 smithi165 ceph-mon[32648]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T01:59:39.623 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:59:39 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/1918497915' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:59:39.623 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:59:39 smithi151 ceph-mon[28057]: daemon mds.f restarted 2024-04-26T01:59:39.623 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:59:39 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T01:59:39.623 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:59:39 smithi151 ceph-mon[28057]: osdmap e95: 12 total, 12 up, 12 in 2024-04-26T01:59:39.623 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:59:39 smithi151 ceph-mon[28057]: mds.? [v2:172.21.15.172:6834/4230821277,v1:172.21.15.172:6835/4230821277] up:boot 2024-04-26T01:59:39.623 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:59:39 smithi151 ceph-mon[28057]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T01:59:39.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:59:39 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/1918497915' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:59:39.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:59:39 smithi172 ceph-mon[32785]: daemon mds.f restarted 2024-04-26T01:59:39.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:59:39 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T01:59:39.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:59:39 smithi172 ceph-mon[32785]: osdmap e95: 12 total, 12 up, 12 in 2024-04-26T01:59:39.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:59:39 smithi172 ceph-mon[32785]: mds.? [v2:172.21.15.172:6834/4230821277,v1:172.21.15.172:6835/4230821277] up:boot 2024-04-26T01:59:39.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:59:39 smithi172 ceph-mon[32785]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T01:59:39.815 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24509 quiesce db / --timeout 60 --expiration 70 --await-for 42.0' 2024-04-26T01:59:39.815 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24509 quiesce db / --timeout 60 --expiration 70 --await-for 42.0 2024-04-26T01:59:40.290 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:59:40.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:59:40 smithi165 ceph-mon[32648]: pgmap v185: 97 pgs: 97 active+clean; 52 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.3 MiB/s wr, 386 op/s 2024-04-26T01:59:40.613 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:59:40 smithi151 ceph-mon[28057]: pgmap v185: 97 pgs: 97 active+clean; 52 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.3 MiB/s wr, 386 op/s 2024-04-26T01:59:40.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:59:40 smithi172 ceph-mon[32785]: pgmap v185: 97 pgs: 97 active+clean; 52 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.3 MiB/s wr, 386 op/s 2024-04-26T01:59:42.495 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:59:42 smithi151 ceph-mon[28057]: pgmap v187: 97 pgs: 97 active+clean; 52 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.7 MiB/s wr, 301 op/s 2024-04-26T01:59:42.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:59:42 smithi165 ceph-mon[32648]: pgmap v187: 97 pgs: 97 active+clean; 52 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.7 MiB/s wr, 301 op/s 2024-04-26T01:59:42.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.700+0000 7f0b2ac00700 1 -- 172.21.15.151:0/265357535 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b241034b0 msgr2=0x7f0b2410f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:42.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.700+0000 7f0b2ac00700 1 --2- 172.21.15.151:0/265357535 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b241034b0 0x7f0b2410f8e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c009fc0 tx=0x7f0b1c00b330 comp rx=0 tx=0).stop 2024-04-26T01:59:42.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.700+0000 7f0b2ac00700 1 -- 172.21.15.151:0/265357535 shutdown_connections 2024-04-26T01:59:42.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.700+0000 7f0b2ac00700 1 --2- 172.21.15.151:0/265357535 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b241034b0 0x7f0b2410f8e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:42.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.700+0000 7f0b2ac00700 1 --2- 172.21.15.151:0/265357535 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b24108a30 0x7f0b24108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:42.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.700+0000 7f0b2ac00700 1 --2- 172.21.15.151:0/265357535 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b24102af0 0x7f0b24102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:42.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.700+0000 7f0b2ac00700 1 -- 172.21.15.151:0/265357535 >> 172.21.15.151:0/265357535 conn(0x7f0b24079b10 msgr2=0x7f0b24079f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:42.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.700+0000 7f0b2ac00700 1 -- 172.21.15.151:0/265357535 shutdown_connections 2024-04-26T01:59:42.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.700+0000 7f0b2ac00700 1 -- 172.21.15.151:0/265357535 wait complete. 2024-04-26T01:59:42.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.701+0000 7f0b2ac00700 1 Processor -- start 2024-04-26T01:59:42.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.702+0000 7f0b2ac00700 1 -- start start 2024-04-26T01:59:42.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.702+0000 7f0b2ac00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b24102af0 0x7f0b24199010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:42.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.702+0000 7f0b2ac00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b241034b0 0x7f0b24199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:42.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.702+0000 7f0b2ac00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b24108a30 0x7f0b2419d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:42.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.702+0000 7f0b2ac00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0b24112c40 con 0x7f0b24108a30 2024-04-26T01:59:42.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.702+0000 7f0b2ac00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0b24112ac0 con 0x7f0b241034b0 2024-04-26T01:59:42.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.702+0000 7f0b2ac00700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0b24112dc0 con 0x7f0b24102af0 2024-04-26T01:59:42.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.702+0000 7f0b23400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b241034b0 0x7f0b24199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:42.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.702+0000 7f0b23e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b24102af0 0x7f0b24199010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:42.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.702+0000 7f0b23400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b241034b0 0x7f0b24199550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:56642/0 (socket says 172.21.15.151:56642) 2024-04-26T01:59:42.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.702+0000 7f0b23400700 1 -- 172.21.15.151:0/471038007 learned_addr learned my addr 172.21.15.151:0/471038007 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:59:42.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.703+0000 7f0b28c00700 1 --2- 172.21.15.151:0/471038007 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b24108a30 0x7f0b2419d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:42.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.703+0000 7f0b23e00700 1 -- 172.21.15.151:0/471038007 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b241034b0 msgr2=0x7f0b24199550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:42.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.703+0000 7f0b23e00700 1 --2- 172.21.15.151:0/471038007 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b241034b0 0x7f0b24199550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:42.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.703+0000 7f0b23e00700 1 -- 172.21.15.151:0/471038007 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b24108a30 msgr2=0x7f0b2419d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:42.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.703+0000 7f0b23e00700 1 --2- 172.21.15.151:0/471038007 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b24108a30 0x7f0b2419d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:42.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.703+0000 7f0b23e00700 1 -- 172.21.15.151:0/471038007 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b2419e020 con 0x7f0b24102af0 2024-04-26T01:59:42.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.703+0000 7f0b28c00700 1 --2- 172.21.15.151:0/471038007 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b24108a30 0x7f0b2419d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T01:59:42.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.703+0000 7f0b23e00700 1 --2- 172.21.15.151:0/471038007 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b24102af0 0x7f0b24199010 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0b1400e9f0 tx=0x7f0b1400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:42.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.704+0000 7f0b20c00700 1 -- 172.21.15.151:0/471038007 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1400cbd0 con 0x7f0b24102af0 2024-04-26T01:59:42.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.704+0000 7f0b20c00700 1 -- 172.21.15.151:0/471038007 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b1400cd70 con 0x7f0b24102af0 2024-04-26T01:59:42.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.704+0000 7f0b2ac00700 1 -- 172.21.15.151:0/471038007 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b2419e310 con 0x7f0b24102af0 2024-04-26T01:59:42.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.704+0000 7f0b2ac00700 1 -- 172.21.15.151:0/471038007 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b241a5b90 con 0x7f0b24102af0 2024-04-26T01:59:42.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.704+0000 7f0b20c00700 1 -- 172.21.15.151:0/471038007 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b14018680 con 0x7f0b24102af0 2024-04-26T01:59:42.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.707+0000 7f0b20c00700 1 -- 172.21.15.151:0/471038007 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f0b14010430 con 0x7f0b24102af0 2024-04-26T01:59:42.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.709+0000 7f0b20c00700 1 --2- 172.21.15.151:0/471038007 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f0b080a3ed0 0x7f0b080a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:42.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.709+0000 7f0b20c00700 1 -- 172.21.15.151:0/471038007 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8411+0+0 (secure 0 0 0) 0x7f0b140d07c0 con 0x7f0b24102af0 2024-04-26T01:59:42.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.710+0000 7f0b23400700 1 --2- 172.21.15.151:0/471038007 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f0b080a3ed0 0x7f0b080a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:42.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.710+0000 7f0b355adb80 1 Processor -- start 2024-04-26T01:59:42.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.711+0000 7f0b355adb80 1 -- start start 2024-04-26T01:59:42.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.711+0000 7f0b23400700 1 --2- 172.21.15.151:0/471038007 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f0b080a3ed0 0x7f0b080a6390 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0b2407a0a0 tx=0x7f0b10009310 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:42.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.711+0000 7f0b355adb80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c5ce27d610 0x55c5ce27d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:42.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.711+0000 7f0b355adb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5ce26ede0 0x55c5ce26f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:42.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.711+0000 7f0b355adb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5ce26f7a0 0x55c5ce2843d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:42.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.711+0000 7f0b355adb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c5ce1c1020 con 0x55c5ce26ede0 2024-04-26T01:59:42.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.711+0000 7f0b355adb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55c5ce1678c0 con 0x55c5ce26f7a0 2024-04-26T01:59:42.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.711+0000 7f0b355adb80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c5ce1bcfe0 con 0x55c5ce27d610 2024-04-26T01:59:42.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.711+0000 7f0b23400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5ce26ede0 0x55c5ce26f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:42.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.711+0000 7f0b23400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5ce26ede0 0x55c5ce26f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:56480/0 (socket says 172.21.15.151:56480) 2024-04-26T01:59:42.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.711+0000 7f0b23400700 1 -- 172.21.15.151:0/651280229 learned_addr learned my addr 172.21.15.151:0/651280229 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:59:42.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.711+0000 7f0b28c00700 1 --2- 172.21.15.151:0/651280229 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5ce26f7a0 0x55c5ce2843d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:42.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.712+0000 7f0b23e00700 1 --2- 172.21.15.151:0/651280229 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c5ce27d610 0x55c5ce27d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:42.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.712+0000 7f0b28c00700 1 -- 172.21.15.151:0/651280229 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c5ce27d610 msgr2=0x55c5ce27d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:42.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.712+0000 7f0b28c00700 1 --2- 172.21.15.151:0/651280229 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c5ce27d610 0x55c5ce27d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:42.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.712+0000 7f0b28c00700 1 -- 172.21.15.151:0/651280229 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5ce26ede0 msgr2=0x55c5ce26f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:42.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.712+0000 7f0b28c00700 1 --2- 172.21.15.151:0/651280229 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5ce26ede0 0x55c5ce26f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:42.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.712+0000 7f0b28c00700 1 -- 172.21.15.151:0/651280229 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5ce15d810 con 0x55c5ce26f7a0 2024-04-26T01:59:42.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.712+0000 7f0b23e00700 1 --2- 172.21.15.151:0/651280229 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c5ce27d610 0x55c5ce27d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:59:42.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.712+0000 7f0b28c00700 1 --2- 172.21.15.151:0/651280229 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5ce26f7a0 0x55c5ce2843d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c015000 tx=0x7f0b1c009510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87f052b945ceeb75 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:42.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.712+0000 7f0b19600700 1 -- 172.21.15.151:0/651280229 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c021070 con 0x55c5ce26f7a0 2024-04-26T01:59:42.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.713+0000 7f0b19600700 1 -- 172.21.15.151:0/651280229 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b1c004c50 con 0x55c5ce26f7a0 2024-04-26T01:59:42.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.713+0000 7f0b19600700 1 -- 172.21.15.151:0/651280229 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c007bb0 con 0x55c5ce26f7a0 2024-04-26T01:59:42.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.713+0000 7f0b355adb80 1 -- 172.21.15.151:0/651280229 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5ce26f7a0 msgr2=0x55c5ce2843d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:42.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.713+0000 7f0b355adb80 1 --2- 172.21.15.151:0/651280229 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5ce26f7a0 0x55c5ce2843d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c015000 tx=0x7f0b1c009510 comp rx=0 tx=0).stop 2024-04-26T01:59:42.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.714+0000 7f0b355adb80 1 -- 172.21.15.151:0/651280229 shutdown_connections 2024-04-26T01:59:42.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.714+0000 7f0b355adb80 1 --2- 172.21.15.151:0/651280229 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5ce26f7a0 0x55c5ce2843d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:42.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.714+0000 7f0b355adb80 1 --2- 172.21.15.151:0/651280229 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c5ce27d610 0x55c5ce27d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:42.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.714+0000 7f0b355adb80 1 --2- 172.21.15.151:0/651280229 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5ce26ede0 0x55c5ce26f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:42.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.714+0000 7f0b355adb80 1 -- 172.21.15.151:0/651280229 >> 172.21.15.151:0/651280229 conn(0x55c5ce19d0e0 msgr2=0x55c5ce269fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:42.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.714+0000 7f0b355adb80 1 -- 172.21.15.151:0/651280229 shutdown_connections 2024-04-26T01:59:42.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.714+0000 7f0b355adb80 1 -- 172.21.15.151:0/651280229 wait complete. 2024-04-26T01:59:42.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.714+0000 7f0b355adb80 1 -- start start 2024-04-26T01:59:42.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.715+0000 7f0b355adb80 1 Processor -- start 2024-04-26T01:59:42.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.715+0000 7f0b355adb80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c5ce26ede0 0x55c5ce283f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:42.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.715+0000 7f0b355adb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5ce26f7a0 0x55c5ce281290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:42.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.715+0000 7f0b355adb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5ce27d610 0x55c5ce2817d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:42.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.715+0000 7f0b355adb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c5ce284910 con 0x55c5ce26f7a0 2024-04-26T01:59:42.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.715+0000 7f0b355adb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55c5ce17a9f0 con 0x55c5ce27d610 2024-04-26T01:59:42.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.715+0000 7f0b355adb80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c5ce284a90 con 0x55c5ce26ede0 2024-04-26T01:59:42.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.715+0000 7f0b355adb80 10 client.0 _ll_register_callbacks cb 0x55c5ce1598b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:59:42.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.716+0000 7f0b28c00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5ce27d610 0x55c5ce2817d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:42.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.716+0000 7f0b28c00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5ce27d610 0x55c5ce2817d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:56654/0 (socket says 172.21.15.151:56654) 2024-04-26T01:59:42.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.716+0000 7f0b28c00700 1 -- 172.21.15.151:0/12850672 learned_addr learned my addr 172.21.15.151:0/12850672 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:59:42.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.716+0000 7f0b23e00700 1 --2- 172.21.15.151:0/12850672 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c5ce26ede0 0x55c5ce283f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:42.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.716+0000 7f0b28c00700 1 -- 172.21.15.151:0/12850672 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c5ce26ede0 msgr2=0x55c5ce283f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:42.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.716+0000 7f0b28c00700 1 --2- 172.21.15.151:0/12850672 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c5ce26ede0 0x55c5ce283f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:42.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.716+0000 7f0b28c00700 1 -- 172.21.15.151:0/12850672 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5ce26f7a0 msgr2=0x55c5ce281290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:42.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.716+0000 7f0b28c00700 1 --2- 172.21.15.151:0/12850672 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5ce26f7a0 0x55c5ce281290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:42.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.716+0000 7f0b28c00700 1 -- 172.21.15.151:0/12850672 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5ce10ea60 con 0x55c5ce27d610 2024-04-26T01:59:42.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.716+0000 7f0b28c00700 1 --2- 172.21.15.151:0/12850672 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5ce27d610 0x55c5ce2817d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c009960 tx=0x7f0b1c007610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66401fa85ec0a1e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:42.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.716+0000 7f0b0e000700 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T01:59:42.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.717+0000 7f0b0e000700 1 -- 172.21.15.151:0/12850672 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c021070 con 0x55c5ce27d610 2024-04-26T01:59:42.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.717+0000 7f0b355adb80 1 -- 172.21.15.151:0/12850672 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b1c004c50 con 0x55c5ce27d610 2024-04-26T01:59:42.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.717+0000 7f0b0e000700 1 -- 172.21.15.151:0/12850672 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8411+0+0 (secure 0 0 0) 0x7f0b1c004a50 con 0x55c5ce27d610 2024-04-26T01:59:42.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.717+0000 7f0b0e000700 1 -- 172.21.15.151:0/12850672 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0b1c004e00 con 0x55c5ce27d610 2024-04-26T01:59:42.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.718+0000 7f0b355adb80 10 client.25844 fetch_fsmap learned FSMap version 33 2024-04-26T01:59:42.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.718+0000 7f0b355adb80 1 -- 172.21.15.151:0/12850672 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c5ce157c80 con 0x55c5ce27d610 2024-04-26T01:59:42.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.718+0000 7f0b0e000700 1 -- 172.21.15.151:0/12850672 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b1c004c50 con 0x55c5ce27d610 2024-04-26T01:59:42.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.718+0000 7f0b0e000700 1 -- 172.21.15.151:0/12850672 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 33) ==== 8338+0+0 (secure 0 0 0) 0x7f0b1c035050 con 0x55c5ce27d610 2024-04-26T01:59:42.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.718+0000 7f0b355adb80 10 client.25844 fetch_fsmap finished waiting for FSMap version 33 2024-04-26T01:59:42.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.718+0000 7f0b355adb80 10 client.25844 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-26T01:59:42.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.718+0000 7f0b355adb80 20 client.25844 populate_metadata read hostname 'smithi151' 2024-04-26T01:59:42.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.718+0000 7f0b355adb80 1 --2- 172.21.15.151:0/12850672 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55c5ce291350 0x55c5ce293740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:42.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.718+0000 7f0b355adb80 4 client.25844 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:59:42.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.718+0000 7f0b355adb80 1 -- 172.21.15.151:0/12850672 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c5ce1ce810 con 0x55c5ce291350 2024-04-26T01:59:42.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.719+0000 7f0b23e00700 1 --2- 172.21.15.151:0/12850672 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55c5ce291350 0x55c5ce293740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:42.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.719+0000 7f0b23e00700 1 --2- 172.21.15.151:0/12850672 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55c5ce291350 0x55c5ce293740 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43a2119cfbddba5b server_cookie=a000dda3b1ad6d13 in_seq=0 out_seq=0 2024-04-26T01:59:42.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.719+0000 7f0b0e000700 10 client.25844 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:59:42.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.720+0000 7f0b0e000700 1 -- 172.21.15.151:0/12850672 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c5ce1ce810 con 0x55c5ce291350 2024-04-26T01:59:42.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.720+0000 7f0b0e000700 10 client.25844 handle_command_reply: tid=0 2024-04-26T01:59:42.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.720+0000 7f0b355adb80 1 client.25844 shutdown 2024-04-26T01:59:42.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.720+0000 7f0b355adb80 1 -- 172.21.15.151:0/12850672 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5ce27d610 msgr2=0x55c5ce2817d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:42.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.720+0000 7f0b355adb80 1 --2- 172.21.15.151:0/12850672 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5ce27d610 0x55c5ce2817d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c009960 tx=0x7f0b1c007610 comp rx=0 tx=0).stop 2024-04-26T01:59:42.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.720+0000 7f0b355adb80 1 -- 172.21.15.151:0/12850672 shutdown_connections 2024-04-26T01:59:42.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.721+0000 7f0b355adb80 1 --2- 172.21.15.151:0/12850672 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55c5ce291350 0x55c5ce293740 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:42.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.721+0000 7f0b0e000700 0 client.25844 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:59:42.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.721+0000 7f0b355adb80 1 --2- 172.21.15.151:0/12850672 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5ce27d610 0x55c5ce2817d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:42.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.721+0000 7f0b355adb80 1 --2- 172.21.15.151:0/12850672 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c5ce26ede0 0x55c5ce283f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:42.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.721+0000 7f0b355adb80 1 --2- 172.21.15.151:0/12850672 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5ce26f7a0 0x55c5ce281290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:42.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.721+0000 7f0b355adb80 1 -- 172.21.15.151:0/12850672 >> 172.21.15.151:0/12850672 conn(0x55c5ce19d0e0 msgr2=0x55c5ce27c990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:42.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.721+0000 7f0b355adb80 1 -- 172.21.15.151:0/12850672 shutdown_connections 2024-04-26T01:59:42.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.721+0000 7f0b355adb80 1 -- 172.21.15.151:0/12850672 wait complete. 2024-04-26T01:59:42.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.721+0000 7f0b355adb80 20 client.25844 trim_cache size 0 max 16384 2024-04-26T01:59:42.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.744+0000 7f0b355adb80 1 Processor -- start 2024-04-26T01:59:42.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.745+0000 7f0b355adb80 1 -- start start 2024-04-26T01:59:42.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.745+0000 7f0b355adb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5ce26ede0 0x55c5ce28d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:42.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.745+0000 7f0b355adb80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c5ce26f7a0 0x55c5ce298d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:42.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.745+0000 7f0b355adb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5ce27d610 0x55c5ce2992b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:42.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.745+0000 7f0b355adb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0400fcc0 con 0x55c5ce27d610 2024-04-26T01:59:42.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.745+0000 7f0b355adb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0400d550 con 0x55c5ce26ede0 2024-04-26T01:59:42.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.745+0000 7f0b355adb80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0400b3a0 con 0x55c5ce26f7a0 2024-04-26T01:59:42.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.745+0000 7f0b23e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5ce26ede0 0x55c5ce28d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:42.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.745+0000 7f0b23400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c5ce26f7a0 0x55c5ce298d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:42.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.745+0000 7f0b23400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c5ce26f7a0 0x55c5ce298d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:49834/0 (socket says 172.21.15.151:49834) 2024-04-26T01:59:42.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.745+0000 7f0b28c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5ce27d610 0x55c5ce2992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:42.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.745+0000 7f0b23400700 1 -- 172.21.15.151:0/484096028 learned_addr learned my addr 172.21.15.151:0/484096028 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:59:42.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.745+0000 7f0b23e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5ce26ede0 0x55c5ce28d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:56658/0 (socket says 172.21.15.151:56658) 2024-04-26T01:59:42.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.746+0000 7f0b23e00700 1 -- 172.21.15.151:0/484096028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c5ce26f7a0 msgr2=0x55c5ce298d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:42.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.746+0000 7f0b23e00700 1 --2- 172.21.15.151:0/484096028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c5ce26f7a0 0x55c5ce298d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:42.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.746+0000 7f0b23e00700 1 -- 172.21.15.151:0/484096028 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5ce27d610 msgr2=0x55c5ce2992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:42.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.746+0000 7f0b23e00700 1 --2- 172.21.15.151:0/484096028 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5ce27d610 0x55c5ce2992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:42.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.746+0000 7f0b23e00700 1 -- 172.21.15.151:0/484096028 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5ce284770 con 0x55c5ce26ede0 2024-04-26T01:59:42.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.747+0000 7f0b23e00700 1 --2- 172.21.15.151:0/484096028 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5ce26ede0 0x55c5ce28d7d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0b140cdf20 tx=0x7f0b14009f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6283d3dbefc3c8e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:42.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.747+0000 7f0b18c00700 1 -- 172.21.15.151:0/484096028 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b140a53e0 con 0x55c5ce26ede0 2024-04-26T01:59:42.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.747+0000 7f0b18c00700 1 -- 172.21.15.151:0/484096028 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b140870b0 con 0x55c5ce26ede0 2024-04-26T01:59:42.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.747+0000 7f0b18c00700 1 -- 172.21.15.151:0/484096028 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b140a51b0 con 0x55c5ce26ede0 2024-04-26T01:59:42.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.748+0000 7f0b355adb80 1 -- 172.21.15.151:0/484096028 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5ce26ede0 msgr2=0x55c5ce28d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:42.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.748+0000 7f0b355adb80 1 --2- 172.21.15.151:0/484096028 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5ce26ede0 0x55c5ce28d7d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0b140cdf20 tx=0x7f0b14009f50 comp rx=0 tx=0).stop 2024-04-26T01:59:42.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.749+0000 7f0b355adb80 1 -- 172.21.15.151:0/484096028 shutdown_connections 2024-04-26T01:59:42.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.749+0000 7f0b355adb80 1 --2- 172.21.15.151:0/484096028 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5ce27d610 0x55c5ce2992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:42.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.749+0000 7f0b355adb80 1 --2- 172.21.15.151:0/484096028 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5ce26ede0 0x55c5ce28d7d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:42.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.749+0000 7f0b355adb80 1 --2- 172.21.15.151:0/484096028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c5ce26f7a0 0x55c5ce298d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:42.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.749+0000 7f0b355adb80 1 -- 172.21.15.151:0/484096028 >> 172.21.15.151:0/484096028 conn(0x55c5ce19d0e0 msgr2=0x55c5ce26e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:42.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.749+0000 7f0b355adb80 1 -- 172.21.15.151:0/484096028 shutdown_connections 2024-04-26T01:59:42.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.749+0000 7f0b355adb80 1 -- 172.21.15.151:0/484096028 wait complete. 2024-04-26T01:59:42.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.750+0000 7f0b355adb80 1 -- start start 2024-04-26T01:59:42.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.750+0000 7f0b355adb80 1 Processor -- start 2024-04-26T01:59:42.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.751+0000 7f0b355adb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5ce26ede0 0x55c5ce28a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:42.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.751+0000 7f0b355adb80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c5ce26f7a0 0x55c5ce27a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:42.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.751+0000 7f0b355adb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5ce27d610 0x55c5ce27a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:42.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.751+0000 7f0b355adb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0400ba10 con 0x55c5ce26ede0 2024-04-26T01:59:42.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.751+0000 7f0b355adb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0400ce80 con 0x55c5ce27d610 2024-04-26T01:59:42.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.751+0000 7f0b355adb80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0400c0e0 con 0x55c5ce26f7a0 2024-04-26T01:59:42.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.751+0000 7f0b355adb80 10 client.0 _ll_register_callbacks cb 0x55c5ce1598b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:59:42.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.751+0000 7f0b23e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5ce26ede0 0x55c5ce28a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:42.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.751+0000 7f0b23400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c5ce26f7a0 0x55c5ce27a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:42.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.751+0000 7f0b23e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5ce26ede0 0x55c5ce28a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:56494/0 (socket says 172.21.15.151:56494) 2024-04-26T01:59:42.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.751+0000 7f0b23e00700 1 -- 172.21.15.151:0/3648714701 learned_addr learned my addr 172.21.15.151:0/3648714701 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:59:42.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.751+0000 7f0b28c00700 1 --2- 172.21.15.151:0/3648714701 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5ce27d610 0x55c5ce27a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:42.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.752+0000 7f0b23400700 1 -- 172.21.15.151:0/3648714701 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5ce27d610 msgr2=0x55c5ce27a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:42.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.752+0000 7f0b23400700 1 --2- 172.21.15.151:0/3648714701 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5ce27d610 0x55c5ce27a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:42.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.752+0000 7f0b23400700 1 -- 172.21.15.151:0/3648714701 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5ce26ede0 msgr2=0x55c5ce28a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:42.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.752+0000 7f0b23400700 1 --2- 172.21.15.151:0/3648714701 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5ce26ede0 0x55c5ce28a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:42.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.752+0000 7f0b23400700 1 -- 172.21.15.151:0/3648714701 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5ce2790a0 con 0x55c5ce26f7a0 2024-04-26T01:59:42.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.752+0000 7f0b28c00700 1 --2- 172.21.15.151:0/3648714701 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5ce27d610 0x55c5ce27a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:59:42.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.752+0000 7f0b23400700 1 --2- 172.21.15.151:0/3648714701 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c5ce26f7a0 0x55c5ce27a180 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x55c5ce234820 tx=0x7f0b10004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f51b2d84c22a0500 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:42.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.752+0000 7f0b0e000700 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-26T01:59:42.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.752+0000 7f0b0e000700 1 -- 172.21.15.151:0/3648714701 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1001f790 con 0x55c5ce26f7a0 2024-04-26T01:59:42.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.752+0000 7f0b355adb80 1 -- 172.21.15.151:0/3648714701 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b140870b0 con 0x55c5ce26f7a0 2024-04-26T01:59:42.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.752+0000 7f0b0e000700 1 -- 172.21.15.151:0/3648714701 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8411+0+0 (secure 0 0 0) 0x7f0b1001f930 con 0x55c5ce26f7a0 2024-04-26T01:59:42.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.753+0000 7f0b0e000700 1 -- 172.21.15.151:0/3648714701 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0b1001fb30 con 0x55c5ce26f7a0 2024-04-26T01:59:42.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.753+0000 7f0b355adb80 10 client.25894 fetch_fsmap learned FSMap version 33 2024-04-26T01:59:42.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.753+0000 7f0b355adb80 1 -- 172.21.15.151:0/3648714701 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c5ce2f8a30 con 0x55c5ce26f7a0 2024-04-26T01:59:42.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.753+0000 7f0b0e000700 1 -- 172.21.15.151:0/3648714701 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b140870b0 con 0x55c5ce26f7a0 2024-04-26T01:59:42.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.753+0000 7f0b0e000700 1 -- 172.21.15.151:0/3648714701 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 33) ==== 8338+0+0 (secure 0 0 0) 0x7f0b10029be0 con 0x55c5ce26f7a0 2024-04-26T01:59:42.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.753+0000 7f0b355adb80 10 client.25894 fetch_fsmap finished waiting for FSMap version 33 2024-04-26T01:59:42.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.753+0000 7f0b355adb80 10 client.25894 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-26T01:59:42.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.753+0000 7f0b355adb80 20 client.25894 populate_metadata read hostname 'smithi151' 2024-04-26T01:59:42.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.753+0000 7f0b355adb80 1 --2- 172.21.15.151:0/3648714701 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55c5ce2f91e0 0x55c5ce2fb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:42.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.753+0000 7f0b355adb80 4 client.25894 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 70.0, "await_for": 42.0}] 2024-04-26T01:59:42.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.753+0000 7f0b355adb80 1 -- 172.21.15.151:0/3648714701 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 70.0, "await_for": 42.0}) -- 0x55c5ce1c88b0 con 0x55c5ce2f91e0 2024-04-26T01:59:42.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.754+0000 7f0b23e00700 1 --2- 172.21.15.151:0/3648714701 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55c5ce2f91e0 0x55c5ce2fb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:42.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.754+0000 7f0b0e000700 10 client.25894 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:59:42.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:42.754+0000 7f0b23e00700 1 --2- 172.21.15.151:0/3648714701 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55c5ce2f91e0 0x55c5ce2fb600 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c68530140a1d7de7 server_cookie=f14b2c80378426a4 in_seq=0 out_seq=0 2024-04-26T01:59:42.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:59:42 smithi172 ceph-mon[32785]: pgmap v187: 97 pgs: 97 active+clean; 52 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.7 MiB/s wr, 301 op/s 2024-04-26T01:59:43.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:43.665+0000 7f0b0e000700 1 -- 172.21.15.151:0/3648714701 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55c5ce1c88b0 con 0x55c5ce2f91e0 2024-04-26T01:59:43.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:43.665+0000 7f0b0e000700 10 client.25894 handle_command_reply: tid=0 2024-04-26T01:59:43.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:43.665+0000 7f0b355adb80 1 client.25894 shutdown 2024-04-26T01:59:43.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:43.665+0000 7f0b355adb80 1 -- 172.21.15.151:0/3648714701 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c5ce26f7a0 msgr2=0x55c5ce27a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:43.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:43.665+0000 7f0b355adb80 1 --2- 172.21.15.151:0/3648714701 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c5ce26f7a0 0x55c5ce27a180 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x55c5ce234820 tx=0x7f0b10004750 comp rx=0 tx=0).stop 2024-04-26T01:59:43.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:43.665+0000 7f0b355adb80 1 -- 172.21.15.151:0/3648714701 shutdown_connections 2024-04-26T01:59:43.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:43.665+0000 7f0b355adb80 1 --2- 172.21.15.151:0/3648714701 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55c5ce2f91e0 0x55c5ce2fb600 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:43.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:43.665+0000 7f0b355adb80 1 --2- 172.21.15.151:0/3648714701 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5ce27d610 0x55c5ce27a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:43.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:43.665+0000 7f0b355adb80 1 --2- 172.21.15.151:0/3648714701 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5ce26ede0 0x55c5ce28a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:43.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:43.665+0000 7f0b355adb80 1 --2- 172.21.15.151:0/3648714701 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c5ce26f7a0 0x55c5ce27a180 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:43.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:43.665+0000 7f0b0e000700 0 client.25894 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:59:43.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:43.665+0000 7f0b355adb80 1 -- 172.21.15.151:0/3648714701 >> 172.21.15.151:0/3648714701 conn(0x55c5ce19d0e0 msgr2=0x55c5ce282ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:43.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:43.666+0000 7f0b355adb80 1 -- 172.21.15.151:0/3648714701 shutdown_connections 2024-04-26T01:59:43.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:43.666+0000 7f0b355adb80 1 -- 172.21.15.151:0/3648714701 wait complete. 2024-04-26T01:59:43.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:43.666+0000 7f0b355adb80 20 client.25894 trim_cache size 0 max 16384 2024-04-26T01:59:43.667 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-26T01:59:43.667 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 33, 2024-04-26T01:59:43.667 INFO:teuthology.orchestra.run.smithi151.stdout: "leader": 24509, 2024-04-26T01:59:43.667 INFO:teuthology.orchestra.run.smithi151.stdout: "set_version": 14, 2024-04-26T01:59:43.667 INFO:teuthology.orchestra.run.smithi151.stdout: "sets": { 2024-04-26T01:59:43.667 INFO:teuthology.orchestra.run.smithi151.stdout: "6d684c64": { 2024-04-26T01:59:43.667 INFO:teuthology.orchestra.run.smithi151.stdout: "version": 14, 2024-04-26T01:59:43.667 INFO:teuthology.orchestra.run.smithi151.stdout: "age_ref": 0.0, 2024-04-26T01:59:43.667 INFO:teuthology.orchestra.run.smithi151.stdout: "state": { 2024-04-26T01:59:43.667 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "QUIESCED", 2024-04-26T01:59:43.667 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 0.0 2024-04-26T01:59:43.667 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T01:59:43.668 INFO:teuthology.orchestra.run.smithi151.stdout: "timeout": 60.0, 2024-04-26T01:59:43.668 INFO:teuthology.orchestra.run.smithi151.stdout: "expiration": 70.0, 2024-04-26T01:59:43.668 INFO:teuthology.orchestra.run.smithi151.stdout: "members": { 2024-04-26T01:59:43.668 INFO:teuthology.orchestra.run.smithi151.stdout: "file:/": { 2024-04-26T01:59:43.668 INFO:teuthology.orchestra.run.smithi151.stdout: "excluded": false, 2024-04-26T01:59:43.668 INFO:teuthology.orchestra.run.smithi151.stdout: "state": { 2024-04-26T01:59:43.668 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "QUIESCED", 2024-04-26T01:59:43.668 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 0.0 2024-04-26T01:59:43.668 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T01:59:43.668 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T01:59:43.668 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T01:59:43.668 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T01:59:43.668 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T01:59:43.669 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-26T01:59:43.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:43.668+0000 7f0b2ac00700 1 -- 172.21.15.151:0/471038007 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f0b080a3ed0 msgr2=0x7f0b080a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:43.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:43.668+0000 7f0b2ac00700 1 --2- 172.21.15.151:0/471038007 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f0b080a3ed0 0x7f0b080a6390 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0b2407a0a0 tx=0x7f0b10009310 comp rx=0 tx=0).stop 2024-04-26T01:59:43.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:43.668+0000 7f0b2ac00700 1 -- 172.21.15.151:0/471038007 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b24102af0 msgr2=0x7f0b24199010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:43.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:43.668+0000 7f0b2ac00700 1 --2- 172.21.15.151:0/471038007 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b24102af0 0x7f0b24199010 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0b1400e9f0 tx=0x7f0b1400ed00 comp rx=0 tx=0).stop 2024-04-26T01:59:43.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:43.668+0000 7f0b2ac00700 1 -- 172.21.15.151:0/471038007 shutdown_connections 2024-04-26T01:59:43.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:43.668+0000 7f0b2ac00700 1 --2- 172.21.15.151:0/471038007 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f0b080a3ed0 0x7f0b080a6390 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:43.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:43.668+0000 7f0b2ac00700 1 --2- 172.21.15.151:0/471038007 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b24108a30 0x7f0b2419d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:43.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:43.668+0000 7f0b2ac00700 1 --2- 172.21.15.151:0/471038007 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b24102af0 0x7f0b24199010 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:43.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:43.668+0000 7f0b2ac00700 1 --2- 172.21.15.151:0/471038007 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b241034b0 0x7f0b24199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:43.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:43.668+0000 7f0b2ac00700 1 -- 172.21.15.151:0/471038007 >> 172.21.15.151:0/471038007 conn(0x7f0b24079b10 msgr2=0x7f0b24101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:43.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:43.668+0000 7f0b2ac00700 1 -- 172.21.15.151:0/471038007 shutdown_connections 2024-04-26T01:59:43.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:43.668+0000 7f0b2ac00700 1 -- 172.21.15.151:0/471038007 wait complete. 2024-04-26T01:59:44.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:59:44 smithi151 ceph-mon[28057]: pgmap v188: 97 pgs: 97 active+clean; 48 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.7 MiB/s wr, 302 op/s 2024-04-26T01:59:44.557 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '6d684c64', quiesce took 9.6 seconds. Will release after: 10 2024-04-26T01:59:44.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:59:44 smithi165 ceph-mon[32648]: pgmap v188: 97 pgs: 97 active+clean; 48 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.7 MiB/s wr, 302 op/s 2024-04-26T01:59:44.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:59:44 smithi172 ceph-mon[32785]: pgmap v188: 97 pgs: 97 active+clean; 48 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.7 MiB/s wr, 302 op/s 2024-04-26T01:59:46.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:59:46 smithi165 ceph-mon[32648]: pgmap v189: 97 pgs: 97 active+clean; 47 MiB data, 956 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.2 MiB/s wr, 359 op/s 2024-04-26T01:59:46.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:59:46 smithi151 ceph-mon[28057]: pgmap v189: 97 pgs: 97 active+clean; 47 MiB data, 956 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.2 MiB/s wr, 359 op/s 2024-04-26T01:59:46.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:59:46 smithi172 ceph-mon[32785]: pgmap v189: 97 pgs: 97 active+clean; 47 MiB data, 956 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.2 MiB/s wr, 359 op/s 2024-04-26T01:59:48.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:59:48 smithi165 ceph-mon[32648]: pgmap v190: 97 pgs: 97 active+clean; 47 MiB data, 956 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.2 MiB/s wr, 359 op/s 2024-04-26T01:59:48.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:59:48 smithi151 ceph-mon[28057]: pgmap v190: 97 pgs: 97 active+clean; 47 MiB data, 956 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.2 MiB/s wr, 359 op/s 2024-04-26T01:59:48.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:59:48 smithi172 ceph-mon[32785]: pgmap v190: 97 pgs: 97 active+clean; 47 MiB data, 956 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.2 MiB/s wr, 359 op/s 2024-04-26T01:59:50.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:59:50 smithi165 ceph-mon[32648]: pgmap v191: 97 pgs: 97 active+clean; 47 MiB data, 956 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.2 MiB/s wr, 198 op/s 2024-04-26T01:59:50.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:59:50 smithi151 ceph-mon[28057]: pgmap v191: 97 pgs: 97 active+clean; 47 MiB data, 956 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.2 MiB/s wr, 198 op/s 2024-04-26T01:59:50.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:59:50 smithi172 ceph-mon[32785]: pgmap v191: 97 pgs: 97 active+clean; 47 MiB data, 956 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.2 MiB/s wr, 198 op/s 2024-04-26T01:59:51.687 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.688+0000 7f37dde00640 1 -- 172.21.15.151:0/3340484052 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37d8104d50 msgr2=0x7f37d8108d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:51.688 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.688+0000 7f37dde00640 1 --2- 172.21.15.151:0/3340484052 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37d8104d50 0x7f37d8108d90 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f37c4009900 tx=0x7f37c402a410 comp rx=0 tx=0).stop 2024-04-26T01:59:51.688 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.688+0000 7f37dde00640 1 -- 172.21.15.151:0/3340484052 shutdown_connections 2024-04-26T01:59:51.688 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.688+0000 7f37dde00640 1 --2- 172.21.15.151:0/3340484052 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f37d8109470 0x7f37d8111040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:51.688 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.688+0000 7f37dde00640 1 --2- 172.21.15.151:0/3340484052 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37d8104d50 0x7f37d8108d90 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:51.688 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.688+0000 7f37dde00640 1 --2- 172.21.15.151:0/3340484052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37d81043a0 0x7f37d8104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:51.688 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.688+0000 7f37dde00640 1 -- 172.21.15.151:0/3340484052 >> 172.21.15.151:0/3340484052 conn(0x7f37d80ffc40 msgr2=0x7f37d8102060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:51.688 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.688+0000 7f37dde00640 1 -- 172.21.15.151:0/3340484052 shutdown_connections 2024-04-26T01:59:51.688 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.689+0000 7f37dde00640 1 -- 172.21.15.151:0/3340484052 wait complete. 2024-04-26T01:59:51.690 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.689+0000 7f37dde00640 1 Processor -- start 2024-04-26T01:59:51.690 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.690+0000 7f37dde00640 1 -- start start 2024-04-26T01:59:51.690 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.690+0000 7f37dde00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37d81043a0 0x7f37d81a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:51.690 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.690+0000 7f37dde00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37d8109470 0x7f37d81a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:51.690 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.690+0000 7f37dde00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f37d819a6c0 0x7f37d819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:51.690 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.690+0000 7f37dde00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f37d8114400 con 0x7f37d81043a0 2024-04-26T01:59:51.690 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.690+0000 7f37dde00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f37d8114280 con 0x7f37d819a6c0 2024-04-26T01:59:51.690 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.690+0000 7f37dde00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f37d8114580 con 0x7f37d8109470 2024-04-26T01:59:51.690 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.690+0000 7f37d6a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37d81043a0 0x7f37d81a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:51.690 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.690+0000 7f37d6a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37d81043a0 0x7f37d81a0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:38336/0 (socket says 172.21.15.151:38336) 2024-04-26T01:59:51.691 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.690+0000 7f37d7400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f37d819a6c0 0x7f37d819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:51.691 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.691+0000 7f37d6a00640 1 -- 172.21.15.151:0/2334683207 learned_addr learned my addr 172.21.15.151:0/2334683207 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:59:51.691 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.691+0000 7f37d6000640 1 --2- 172.21.15.151:0/2334683207 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37d8109470 0x7f37d81a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:51.691 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.691+0000 7f37d7400640 1 -- 172.21.15.151:0/2334683207 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37d8109470 msgr2=0x7f37d81a0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:51.691 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.691+0000 7f37d7400640 1 --2- 172.21.15.151:0/2334683207 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37d8109470 0x7f37d81a0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:51.691 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.691+0000 7f37d7400640 1 -- 172.21.15.151:0/2334683207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37d81043a0 msgr2=0x7f37d81a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:51.691 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.691+0000 7f37d7400640 1 --2- 172.21.15.151:0/2334683207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37d81043a0 0x7f37d81a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:51.691 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.691+0000 7f37d7400640 1 -- 172.21.15.151:0/2334683207 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37d819b2c0 con 0x7f37d819a6c0 2024-04-26T01:59:51.691 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.691+0000 7f37d7400640 1 --2- 172.21.15.151:0/2334683207 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f37d819a6c0 0x7f37d819ab40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f37d0009fd0 tx=0x7f37d000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.691+0000 7f37dca00640 1 -- 172.21.15.151:0/2334683207 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d000ce20 con 0x7f37d819a6c0 2024-04-26T01:59:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.691+0000 7f37dca00640 1 -- 172.21.15.151:0/2334683207 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37d0005020 con 0x7f37d819a6c0 2024-04-26T01:59:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.691+0000 7f37dde00640 1 -- 172.21.15.151:0/2334683207 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37d81a73a0 con 0x7f37d819a6c0 2024-04-26T01:59:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.692+0000 7f37dca00640 1 -- 172.21.15.151:0/2334683207 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d0010690 con 0x7f37d819a6c0 2024-04-26T01:59:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.692+0000 7f37dde00640 1 -- 172.21.15.151:0/2334683207 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37d81a7750 con 0x7f37d819a6c0 2024-04-26T01:59:51.696 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.695+0000 7f37dca00640 1 -- 172.21.15.151:0/2334683207 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f37d00054d0 con 0x7f37d819a6c0 2024-04-26T01:59:51.696 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.695+0000 7f37e0c5d740 1 Processor -- start 2024-04-26T01:59:51.696 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.695+0000 7f37e0c5d740 1 -- start start 2024-04-26T01:59:51.696 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.695+0000 7f37e0c5d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fd07849820 0x55fd07849c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:51.696 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.695+0000 7f37e0c5d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fd0784a140 0x55fd078544c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:51.696 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.695+0000 7f37e0c5d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fd07854a00 0x55fd07856df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:51.696 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.695+0000 7f37e0c5d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55fd077af900 con 0x55fd07849820 2024-04-26T01:59:51.696 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.695+0000 7f37e0c5d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55fd07755ef0 con 0x55fd0784a140 2024-04-26T01:59:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.695+0000 7f37e0c5d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55fd077ab8c0 con 0x55fd07854a00 2024-04-26T01:59:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.696+0000 7f37d6000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fd07849820 0x55fd07849c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.696+0000 7f37d6a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fd0784a140 0x55fd078544c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.696+0000 7f37d6a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fd0784a140 0x55fd078544c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:36634/0 (socket says 172.21.15.151:36634) 2024-04-26T01:59:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.696+0000 7f37d6a00640 1 -- 172.21.15.151:0/1149658540 learned_addr learned my addr 172.21.15.151:0/1149658540 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:59:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.696+0000 7f37d6000640 1 --2- 172.21.15.151:0/1149658540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fd07854a00 0x55fd07856df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.696+0000 7f37d6a00640 1 -- 172.21.15.151:0/1149658540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fd07854a00 msgr2=0x55fd07856df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.696+0000 7f37d6a00640 1 --2- 172.21.15.151:0/1149658540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fd07854a00 0x55fd07856df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.696+0000 7f37d6a00640 1 -- 172.21.15.151:0/1149658540 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fd07849820 msgr2=0x55fd07849c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.696+0000 7f37d6a00640 1 --2- 172.21.15.151:0/1149658540 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fd07849820 0x55fd07849c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.696+0000 7f37d6a00640 1 -- 172.21.15.151:0/1149658540 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd074c2210 con 0x55fd0784a140 2024-04-26T01:59:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.696+0000 7f37dca00640 1 --2- 172.21.15.151:0/2334683207 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f37bc0a3dc0 0x7f37bc0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.697+0000 7f37d6a00640 1 --2- 172.21.15.151:0/1149658540 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fd0784a140 0x55fd078544c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f37c8009e10 tx=0x7f37c8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b7a19584c44af74 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.697+0000 7f37dca00640 1 -- 172.21.15.151:0/2334683207 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8411+0+0 (secure 0 0 0) 0x7f37d00d0dd0 con 0x7f37d819a6c0 2024-04-26T01:59:51.698 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.697+0000 7f37d6000640 1 --2- 172.21.15.151:0/2334683207 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f37bc0a3dc0 0x7f37bc0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:51.698 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.697+0000 7f37c3e00640 1 -- 172.21.15.151:0/1149658540 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c801e070 con 0x55fd0784a140 2024-04-26T01:59:51.698 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.697+0000 7f37c3e00640 1 -- 172.21.15.151:0/1149658540 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37c8019ae0 con 0x55fd0784a140 2024-04-26T01:59:51.698 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.697+0000 7f37c3e00640 1 -- 172.21.15.151:0/1149658540 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c800fad0 con 0x55fd0784a140 2024-04-26T01:59:51.702 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.698+0000 7f37e0c5d740 1 -- 172.21.15.151:0/1149658540 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fd0784a140 msgr2=0x55fd078544c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:51.702 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.698+0000 7f37d6000640 1 --2- 172.21.15.151:0/2334683207 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f37bc0a3dc0 0x7f37bc0a6280 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f37c4031440 tx=0x7f37c4041040 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.698+0000 7f37e0c5d740 1 --2- 172.21.15.151:0/1149658540 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fd0784a140 0x55fd078544c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f37c8009e10 tx=0x7f37c8011810 comp rx=0 tx=0).stop 2024-04-26T01:59:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.698+0000 7f37e0c5d740 1 -- 172.21.15.151:0/1149658540 shutdown_connections 2024-04-26T01:59:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.698+0000 7f37e0c5d740 1 --2- 172.21.15.151:0/1149658540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fd07854a00 0x55fd07856df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.698+0000 7f37e0c5d740 1 --2- 172.21.15.151:0/1149658540 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fd0784a140 0x55fd078544c0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.698+0000 7f37e0c5d740 1 --2- 172.21.15.151:0/1149658540 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fd07849820 0x55fd07849c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.698+0000 7f37e0c5d740 1 -- 172.21.15.151:0/1149658540 >> 172.21.15.151:0/1149658540 conn(0x55fd07845d30 msgr2=0x55fd07846110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.698+0000 7f37e0c5d740 1 -- 172.21.15.151:0/1149658540 shutdown_connections 2024-04-26T01:59:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.698+0000 7f37e0c5d740 1 -- 172.21.15.151:0/1149658540 wait complete. 2024-04-26T01:59:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.698+0000 7f37e0c5d740 1 -- start start 2024-04-26T01:59:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.699+0000 7f37e0c5d740 1 Processor -- start 2024-04-26T01:59:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.699+0000 7f37e0c5d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fd07849820 0x55fd07851900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.699+0000 7f37e0c5d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fd0784a140 0x55fd07851e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.699+0000 7f37e0c5d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fd07854a00 0x55fd07852380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:51.704 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.699+0000 7f37e0c5d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55fd07857630 con 0x55fd07849820 2024-04-26T01:59:51.704 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.699+0000 7f37e0c5d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55fd07769310 con 0x55fd07854a00 2024-04-26T01:59:51.704 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.699+0000 7f37e0c5d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55fd078577b0 con 0x55fd0784a140 2024-04-26T01:59:51.704 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.699+0000 7f37e0c5d740 10 client.0 _ll_register_callbacks cb 0x55fd07737cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:59:51.704 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.699+0000 7f37d6a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fd07849820 0x55fd07851900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:51.704 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.699+0000 7f37d6a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fd07849820 0x55fd07851900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:38360/0 (socket says 172.21.15.151:38360) 2024-04-26T01:59:51.704 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.699+0000 7f37d6a00640 1 -- 172.21.15.151:0/3563258757 learned_addr learned my addr 172.21.15.151:0/3563258757 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:59:51.704 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.699+0000 7f37d6000640 1 --2- 172.21.15.151:0/3563258757 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fd0784a140 0x55fd07851e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:51.704 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.700+0000 7f37d7400640 1 --2- 172.21.15.151:0/3563258757 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fd07854a00 0x55fd07852380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:51.704 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.700+0000 7f37d6a00640 1 -- 172.21.15.151:0/3563258757 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fd0784a140 msgr2=0x55fd07851e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:51.704 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.700+0000 7f37d6a00640 1 --2- 172.21.15.151:0/3563258757 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fd0784a140 0x55fd07851e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:51.704 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.700+0000 7f37d6a00640 1 -- 172.21.15.151:0/3563258757 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fd07854a00 msgr2=0x55fd07852380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:51.704 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.700+0000 7f37d6a00640 1 --2- 172.21.15.151:0/3563258757 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fd07854a00 0x55fd07852380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:51.705 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.700+0000 7f37d6a00640 1 -- 172.21.15.151:0/3563258757 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd0758e620 con 0x55fd07849820 2024-04-26T01:59:51.705 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.700+0000 7f37d7400640 1 --2- 172.21.15.151:0/3563258757 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fd07854a00 0x55fd07852380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:59:51.705 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.700+0000 7f37d6a00640 1 --2- 172.21.15.151:0/3563258757 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fd07849820 0x55fd07851900 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f37c8009810 tx=0x7f37c8009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=319d32954853d5e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:51.705 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.700+0000 7f37c0c00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-04-26T01:59:51.705 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.700+0000 7f37c0c00640 1 -- 172.21.15.151:0/3563258757 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c8027020 con 0x55fd07849820 2024-04-26T01:59:51.705 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.700+0000 7f37c0c00640 1 -- 172.21.15.151:0/3563258757 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8411+0+0 (secure 0 0 0) 0x7f37c800f8b0 con 0x55fd07849820 2024-04-26T01:59:51.705 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.700+0000 7f37e0c5d740 1 -- 172.21.15.151:0/3563258757 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37c8019ae0 con 0x55fd07849820 2024-04-26T01:59:51.705 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.701+0000 7f37c0c00640 1 -- 172.21.15.151:0/3563258757 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f37c8032050 con 0x55fd07849820 2024-04-26T01:59:51.705 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.701+0000 7f37e0c5d740 10 client.15768 fetch_fsmap learned FSMap version 33 2024-04-26T01:59:51.705 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.701+0000 7f37e0c5d740 1 -- 172.21.15.151:0/3563258757 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55fd078637f0 con 0x55fd07849820 2024-04-26T01:59:51.705 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.701+0000 7f37c0c00640 1 -- 172.21.15.151:0/3563258757 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37c8019ae0 con 0x55fd07849820 2024-04-26T01:59:51.705 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.701+0000 7f37c0c00640 1 -- 172.21.15.151:0/3563258757 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 33) ==== 8338+0+0 (secure 0 0 0) 0x7f37c80046c0 con 0x55fd07849820 2024-04-26T01:59:51.705 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.701+0000 7f37e0c5d740 10 client.15768 fetch_fsmap finished waiting for FSMap version 33 2024-04-26T01:59:51.705 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.701+0000 7f37e0c5d740 10 client.15768 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:59:51.706 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.701+0000 7f37e0c5d740 20 client.15768 populate_metadata read hostname 'smithi151' 2024-04-26T01:59:51.706 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.701+0000 7f37e0c5d740 1 --2- 172.21.15.151:0/3563258757 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55fd07864ba0 0x55fd07866f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:51.706 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.701+0000 7f37e0c5d740 4 client.15768 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:59:51.706 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.701+0000 7f37e0c5d740 1 -- 172.21.15.151:0/3563258757 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd077c38b0 con 0x55fd07864ba0 2024-04-26T01:59:51.706 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.702+0000 7f37d6000640 1 --2- 172.21.15.151:0/3563258757 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55fd07864ba0 0x55fd07866f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:51.706 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.702+0000 7f37d6000640 1 --2- 172.21.15.151:0/3563258757 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55fd07864ba0 0x55fd07866f90 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4308eff1735efa05 server_cookie=a4638fd97766972f in_seq=0 out_seq=0 2024-04-26T01:59:51.706 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.702+0000 7f37c0c00640 10 client.15768 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:59:51.706 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.703+0000 7f37c0c00640 1 -- 172.21.15.151:0/3563258757 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fd077c38b0 con 0x55fd07864ba0 2024-04-26T01:59:51.706 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.703+0000 7f37c0c00640 10 client.15768 handle_command_reply: tid=0 2024-04-26T01:59:51.706 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.703+0000 7f37e0c5d740 1 client.15768 shutdown 2024-04-26T01:59:51.706 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.703+0000 7f37e0c5d740 1 -- 172.21.15.151:0/3563258757 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fd07849820 msgr2=0x55fd07851900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:51.706 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.703+0000 7f37e0c5d740 1 --2- 172.21.15.151:0/3563258757 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fd07849820 0x55fd07851900 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f37c8009810 tx=0x7f37c8009b60 comp rx=0 tx=0).stop 2024-04-26T01:59:51.706 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.703+0000 7f37e0c5d740 1 -- 172.21.15.151:0/3563258757 shutdown_connections 2024-04-26T01:59:51.707 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.703+0000 7f37e0c5d740 1 --2- 172.21.15.151:0/3563258757 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55fd07864ba0 0x55fd07866f90 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:51.707 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.703+0000 7f37e0c5d740 1 --2- 172.21.15.151:0/3563258757 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fd07854a00 0x55fd07852380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:51.707 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.703+0000 7f37e0c5d740 1 --2- 172.21.15.151:0/3563258757 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fd0784a140 0x55fd07851e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:51.707 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.703+0000 7f37c0c00640 0 client.15768 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:59:51.707 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.703+0000 7f37e0c5d740 1 --2- 172.21.15.151:0/3563258757 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fd07849820 0x55fd07851900 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:51.707 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.703+0000 7f37e0c5d740 1 -- 172.21.15.151:0/3563258757 >> 172.21.15.151:0/3563258757 conn(0x55fd07845d30 msgr2=0x55fd0784bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:51.707 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.704+0000 7f37e0c5d740 1 -- 172.21.15.151:0/3563258757 shutdown_connections 2024-04-26T01:59:51.707 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.704+0000 7f37e0c5d740 1 -- 172.21.15.151:0/3563258757 wait complete. 2024-04-26T01:59:51.707 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.704+0000 7f37e0c5d740 20 client.15768 trim_cache size 0 max 16384 2024-04-26T01:59:51.719 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.720+0000 7f37e0c5d740 1 Processor -- start 2024-04-26T01:59:51.719 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.720+0000 7f37e0c5d740 1 -- start start 2024-04-26T01:59:51.719 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.720+0000 7f37e0c5d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fd07849820 0x55fd078604f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:51.720 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.720+0000 7f37e0c5d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fd0784a140 0x55fd0784c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:51.720 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.720+0000 7f37e0c5d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fd07854a00 0x55fd0786c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:51.720 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.720+0000 7f37e0c5d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f37d81b2360 con 0x55fd0784a140 2024-04-26T01:59:51.720 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.720+0000 7f37e0c5d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f37d81afc30 con 0x55fd07854a00 2024-04-26T01:59:51.720 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.720+0000 7f37e0c5d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f37d81adae0 con 0x55fd07849820 2024-04-26T01:59:51.720 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.721+0000 7f37d6a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fd07849820 0x55fd078604f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:51.720 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.721+0000 7f37d6a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fd07849820 0x55fd078604f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:36016/0 (socket says 172.21.15.151:36016) 2024-04-26T01:59:51.720 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.721+0000 7f37d6a00640 1 -- 172.21.15.151:0/3990070483 learned_addr learned my addr 172.21.15.151:0/3990070483 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:59:51.720 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.721+0000 7f37d6000640 1 --2- 172.21.15.151:0/3990070483 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fd0784a140 0x55fd0784c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:51.720 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.721+0000 7f37d7400640 1 --2- 172.21.15.151:0/3990070483 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fd07854a00 0x55fd0786c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.721+0000 7f37d6a00640 1 -- 172.21.15.151:0/3990070483 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fd07854a00 msgr2=0x55fd0786c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.721+0000 7f37d6a00640 1 --2- 172.21.15.151:0/3990070483 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fd07854a00 0x55fd0786c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.721+0000 7f37d6a00640 1 -- 172.21.15.151:0/3990070483 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fd0784a140 msgr2=0x55fd0784c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.721+0000 7f37d6a00640 1 --2- 172.21.15.151:0/3990070483 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fd0784a140 0x55fd0784c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.721+0000 7f37d6a00640 1 -- 172.21.15.151:0/3990070483 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd07849f70 con 0x55fd07849820 2024-04-26T01:59:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.721+0000 7f37d7400640 1 --2- 172.21.15.151:0/3990070483 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fd07854a00 0x55fd0786c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:59:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.722+0000 7f37d6a00640 1 --2- 172.21.15.151:0/3990070483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fd07849820 0x55fd078604f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f37c8011850 tx=0x7f37c8009d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5047b9cde8d1787e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.722+0000 7f37c3400640 1 -- 172.21.15.151:0/3990070483 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c8015020 con 0x55fd07849820 2024-04-26T01:59:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.722+0000 7f37c3400640 1 -- 172.21.15.151:0/3990070483 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37c801f040 con 0x55fd07849820 2024-04-26T01:59:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.722+0000 7f37c3400640 1 -- 172.21.15.151:0/3990070483 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c8027070 con 0x55fd07849820 2024-04-26T01:59:51.723 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.722+0000 7f37e0c5d740 1 -- 172.21.15.151:0/3990070483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fd07849820 msgr2=0x55fd078604f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:51.723 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.722+0000 7f37e0c5d740 1 --2- 172.21.15.151:0/3990070483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fd07849820 0x55fd078604f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f37c8011850 tx=0x7f37c8009d40 comp rx=0 tx=0).stop 2024-04-26T01:59:51.723 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.723+0000 7f37e0c5d740 1 -- 172.21.15.151:0/3990070483 shutdown_connections 2024-04-26T01:59:51.723 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.723+0000 7f37e0c5d740 1 --2- 172.21.15.151:0/3990070483 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fd07854a00 0x55fd0786c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:51.723 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.723+0000 7f37e0c5d740 1 --2- 172.21.15.151:0/3990070483 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fd0784a140 0x55fd0784c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:51.723 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.723+0000 7f37e0c5d740 1 --2- 172.21.15.151:0/3990070483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fd07849820 0x55fd078604f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:51.723 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.723+0000 7f37e0c5d740 1 -- 172.21.15.151:0/3990070483 >> 172.21.15.151:0/3990070483 conn(0x55fd07845d30 msgr2=0x55fd0784cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:51.723 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.723+0000 7f37e0c5d740 1 -- 172.21.15.151:0/3990070483 shutdown_connections 2024-04-26T01:59:51.723 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.723+0000 7f37e0c5d740 1 -- 172.21.15.151:0/3990070483 wait complete. 2024-04-26T01:59:51.723 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.723+0000 7f37e0c5d740 1 -- start start 2024-04-26T01:59:51.723 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.723+0000 7f37e0c5d740 1 Processor -- start 2024-04-26T01:59:51.725 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.724+0000 7f37e0c5d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fd07849820 0x55fd078c3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:51.726 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.724+0000 7f37e0c5d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fd0784a140 0x55fd078c35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:51.726 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.724+0000 7f37e0c5d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fd07854a00 0x55fd078c3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:51.726 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.724+0000 7f37e0c5d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f37d81ae130 con 0x55fd07854a00 2024-04-26T01:59:51.726 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.724+0000 7f37e0c5d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f37d81af560 con 0x55fd0784a140 2024-04-26T01:59:51.726 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.724+0000 7f37e0c5d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f37d81ae7c0 con 0x55fd07849820 2024-04-26T01:59:51.726 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.724+0000 7f37e0c5d740 10 client.0 _ll_register_callbacks cb 0x7f37c8009c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:59:51.726 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.724+0000 7f37d6a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fd07849820 0x55fd078c3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:51.726 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.724+0000 7f37d7400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fd07854a00 0x55fd078c3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:51.726 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.724+0000 7f37d6a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fd07849820 0x55fd078c3060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:36020/0 (socket says 172.21.15.151:36020) 2024-04-26T01:59:51.726 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.724+0000 7f37d7400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fd07854a00 0x55fd078c3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:38386/0 (socket says 172.21.15.151:38386) 2024-04-26T01:59:51.726 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.724+0000 7f37d7400640 1 -- 172.21.15.151:0/3385396966 learned_addr learned my addr 172.21.15.151:0/3385396966 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:59:51.726 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.724+0000 7f37d6000640 1 --2- 172.21.15.151:0/3385396966 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fd0784a140 0x55fd078c35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.724+0000 7f37d7400640 1 -- 172.21.15.151:0/3385396966 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fd07849820 msgr2=0x55fd078c3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.724+0000 7f37d7400640 1 --2- 172.21.15.151:0/3385396966 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fd07849820 0x55fd078c3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.724+0000 7f37d7400640 1 -- 172.21.15.151:0/3385396966 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fd0784a140 msgr2=0x55fd078c35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.724+0000 7f37d7400640 1 --2- 172.21.15.151:0/3385396966 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fd0784a140 0x55fd078c35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.724+0000 7f37d7400640 1 -- 172.21.15.151:0/3385396966 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd0785bc20 con 0x55fd07854a00 2024-04-26T01:59:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.724+0000 7f37d6000640 1 --2- 172.21.15.151:0/3385396966 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fd0784a140 0x55fd078c35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:59:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.725+0000 7f37d7400640 1 --2- 172.21.15.151:0/3385396966 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fd07854a00 0x55fd078c3ae0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f37d000c760 tx=0x7f37d009d5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37e3c8bfb7930421 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.725+0000 7f37c0c00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-04-26T01:59:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.725+0000 7f37c0c00640 1 -- 172.21.15.151:0/3385396966 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d00bb060 con 0x55fd07854a00 2024-04-26T01:59:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.725+0000 7f37c0c00640 1 -- 172.21.15.151:0/3385396966 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8411+0+0 (secure 0 0 0) 0x7f37d0099450 con 0x55fd07854a00 2024-04-26T01:59:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.725+0000 7f37e0c5d740 1 -- 172.21.15.151:0/3385396966 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37c801f040 con 0x55fd07854a00 2024-04-26T01:59:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.725+0000 7f37c0c00640 1 -- 172.21.15.151:0/3385396966 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f37d0082050 con 0x55fd07854a00 2024-04-26T01:59:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.725+0000 7f37e0c5d740 10 client.15774 fetch_fsmap learned FSMap version 33 2024-04-26T01:59:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.725+0000 7f37e0c5d740 1 -- 172.21.15.151:0/3385396966 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55fd0785cf90 con 0x55fd07854a00 2024-04-26T01:59:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.725+0000 7f37c0c00640 1 -- 172.21.15.151:0/3385396966 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37c801f040 con 0x55fd07854a00 2024-04-26T01:59:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.726+0000 7f37c0c00640 1 -- 172.21.15.151:0/3385396966 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 33) ==== 8338+0+0 (secure 0 0 0) 0x7f37d00cece0 con 0x55fd07854a00 2024-04-26T01:59:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.726+0000 7f37e0c5d740 10 client.15774 fetch_fsmap finished waiting for FSMap version 33 2024-04-26T01:59:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.726+0000 7f37e0c5d740 10 client.15774 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:59:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.726+0000 7f37e0c5d740 20 client.15774 populate_metadata read hostname 'smithi151' 2024-04-26T01:59:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.726+0000 7f37e0c5d740 1 --2- 172.21.15.151:0/3385396966 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55fd078d3900 0x55fd078d5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.726+0000 7f37e0c5d740 4 client.15774 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:59:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.726+0000 7f37e0c5d740 1 -- 172.21.15.151:0/3385396966 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fd077bd320 con 0x55fd078d3900 2024-04-26T01:59:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.726+0000 7f37d6a00640 1 --2- 172.21.15.151:0/3385396966 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55fd078d3900 0x55fd078d5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.727+0000 7f37c0c00640 10 client.15774 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T01:59:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.727+0000 7f37d6a00640 1 --2- 172.21.15.151:0/3385396966 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55fd078d3900 0x55fd078d5d20 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38b847b831f8e250 server_cookie=209ad4fba7bee7f1 in_seq=0 out_seq=0 2024-04-26T01:59:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.742+0000 7f37c0c00640 1 -- 172.21.15.151:0/3385396966 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fd077bd320 con 0x55fd078d3900 2024-04-26T01:59:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.742+0000 7f37c0c00640 10 client.15774 handle_command_reply: tid=0 2024-04-26T01:59:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.742+0000 7f37e0c5d740 1 client.15774 shutdown 2024-04-26T01:59:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.742+0000 7f37e0c5d740 1 -- 172.21.15.151:0/3385396966 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fd07854a00 msgr2=0x55fd078c3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.742+0000 7f37e0c5d740 1 --2- 172.21.15.151:0/3385396966 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fd07854a00 0x55fd078c3ae0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f37d000c760 tx=0x7f37d009d5d0 comp rx=0 tx=0).stop 2024-04-26T01:59:51.741 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.742+0000 7f37e0c5d740 1 -- 172.21.15.151:0/3385396966 shutdown_connections 2024-04-26T01:59:51.741 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.742+0000 7f37e0c5d740 1 --2- 172.21.15.151:0/3385396966 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55fd078d3900 0x55fd078d5d20 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:51.741 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.742+0000 7f37e0c5d740 1 --2- 172.21.15.151:0/3385396966 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fd07854a00 0x55fd078c3ae0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:51.742 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.742+0000 7f37e0c5d740 1 --2- 172.21.15.151:0/3385396966 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fd0784a140 0x55fd078c35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:51.742 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.742+0000 7f37e0c5d740 1 --2- 172.21.15.151:0/3385396966 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fd07849820 0x55fd078c3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:51.742 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.742+0000 7f37c0c00640 0 client.15774 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T01:59:51.742 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.742+0000 7f37e0c5d740 1 -- 172.21.15.151:0/3385396966 >> 172.21.15.151:0/3385396966 conn(0x55fd07845d30 msgr2=0x55fd078518a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:51.742 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.742+0000 7f37e0c5d740 1 -- 172.21.15.151:0/3385396966 shutdown_connections 2024-04-26T01:59:51.742 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.742+0000 7f37e0c5d740 1 -- 172.21.15.151:0/3385396966 wait complete. 2024-04-26T01:59:51.742 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.742+0000 7f37e0c5d740 20 client.15774 trim_cache size 0 max 16384 2024-04-26T01:59:51.742 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:{ 2024-04-26T01:59:51.742 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "message": "", 2024-04-26T01:59:51.743 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "return_code": 0 2024-04-26T01:59:51.743 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:} 2024-04-26T01:59:51.743 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.743+0000 7f37dde00640 1 -- 172.21.15.151:0/2334683207 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f37bc0a3dc0 msgr2=0x7f37bc0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:51.743 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.743+0000 7f37dde00640 1 --2- 172.21.15.151:0/2334683207 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f37bc0a3dc0 0x7f37bc0a6280 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f37c4031440 tx=0x7f37c4041040 comp rx=0 tx=0).stop 2024-04-26T01:59:51.743 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.743+0000 7f37dde00640 1 -- 172.21.15.151:0/2334683207 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f37d819a6c0 msgr2=0x7f37d819ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:51.744 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.743+0000 7f37dde00640 1 --2- 172.21.15.151:0/2334683207 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f37d819a6c0 0x7f37d819ab40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f37d0009fd0 tx=0x7f37d000ef10 comp rx=0 tx=0).stop 2024-04-26T01:59:51.744 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.743+0000 7f37dde00640 1 -- 172.21.15.151:0/2334683207 shutdown_connections 2024-04-26T01:59:51.744 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.743+0000 7f37dde00640 1 --2- 172.21.15.151:0/2334683207 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f37bc0a3dc0 0x7f37bc0a6280 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:51.744 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.743+0000 7f37dde00640 1 --2- 172.21.15.151:0/2334683207 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f37d819a6c0 0x7f37d819ab40 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:51.744 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.743+0000 7f37dde00640 1 --2- 172.21.15.151:0/2334683207 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37d8109470 0x7f37d81a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:51.744 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.743+0000 7f37dde00640 1 --2- 172.21.15.151:0/2334683207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37d81043a0 0x7f37d81a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:51.744 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.743+0000 7f37dde00640 1 -- 172.21.15.151:0/2334683207 >> 172.21.15.151:0/2334683207 conn(0x7f37d80ffc40 msgr2=0x7f37d8100ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:51.744 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.743+0000 7f37dde00640 1 -- 172.21.15.151:0/2334683207 shutdown_connections 2024-04-26T01:59:51.744 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T01:59:51.743+0000 7f37dde00640 1 -- 172.21.15.151:0/2334683207 wait complete. 2024-04-26T01:59:52.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:59:52 smithi151 ceph-mon[28057]: pgmap v192: 97 pgs: 97 active+clean; 47 MiB data, 956 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.1 MiB/s wr, 169 op/s 2024-04-26T01:59:52.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:59:52 smithi172 ceph-mon[32785]: pgmap v192: 97 pgs: 97 active+clean; 47 MiB data, 956 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.1 MiB/s wr, 169 op/s 2024-04-26T01:59:52.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:59:52 smithi165 ceph-mon[32648]: pgmap v192: 97 pgs: 97 active+clean; 47 MiB data, 956 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.1 MiB/s wr, 169 op/s 2024-04-26T01:59:53.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:59:53 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:59:53.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:59:53 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:59:53.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:59:53 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:59:54.559 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:59:54.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:59:54 smithi151 ceph-mon[28057]: pgmap v193: 97 pgs: 97 active+clean; 47 MiB data, 956 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.0 MiB/s wr, 166 op/s 2024-04-26T01:59:54.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:59:54 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:59:54.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:59:54 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:59:54.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:59:54 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:59:54.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:59:54 smithi172 ceph-mon[32785]: pgmap v193: 97 pgs: 97 active+clean; 47 MiB data, 956 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.0 MiB/s wr, 166 op/s 2024-04-26T01:59:54.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:59:54 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:59:54.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:59:54 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:59:54.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:59:54 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:59:54.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:59:54 smithi165 ceph-mon[32648]: pgmap v193: 97 pgs: 97 active+clean; 47 MiB data, 956 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.0 MiB/s wr, 166 op/s 2024-04-26T01:59:54.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:59:54 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:59:54.847 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:59:54 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:59:54.847 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:59:54 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T01:59:55.009 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:59:56.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:59:56 smithi151 ceph-mon[28057]: pgmap v194: 97 pgs: 97 active+clean; 47 MiB data, 956 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.0 MiB/s wr, 169 op/s 2024-04-26T01:59:56.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:59:56 smithi172 ceph-mon[32785]: pgmap v194: 97 pgs: 97 active+clean; 47 MiB data, 956 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.0 MiB/s wr, 169 op/s 2024-04-26T01:59:56.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:59:56 smithi165 ceph-mon[32648]: pgmap v194: 97 pgs: 97 active+clean; 47 MiB data, 956 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.0 MiB/s wr, 169 op/s 2024-04-26T01:59:57.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:57.244+0000 7f8907600700 1 -- 172.21.15.151:0/3508919003 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8900103450 msgr2=0x7f890010f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:57.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:57.244+0000 7f8907600700 1 --2- 172.21.15.151:0/3508919003 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8900103450 0x7f890010f8b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f88f800b0a0 tx=0x7f88f800b3b0 comp rx=0 tx=0).stop 2024-04-26T01:59:57.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:57.244+0000 7f8907600700 1 -- 172.21.15.151:0/3508919003 shutdown_connections 2024-04-26T01:59:57.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:57.244+0000 7f8907600700 1 --2- 172.21.15.151:0/3508919003 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8900103450 0x7f890010f8b0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:57.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:57.244+0000 7f8907600700 1 --2- 172.21.15.151:0/3508919003 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8900108a50 0x7f8900108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:57.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:57.244+0000 7f8907600700 1 --2- 172.21.15.151:0/3508919003 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8900102a90 0x7f8900102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:57.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:57.244+0000 7f8907600700 1 -- 172.21.15.151:0/3508919003 >> 172.21.15.151:0/3508919003 conn(0x7f8900079ba0 msgr2=0x7f8900079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:57.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:57.244+0000 7f8907600700 1 -- 172.21.15.151:0/3508919003 shutdown_connections 2024-04-26T01:59:57.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:57.244+0000 7f8907600700 1 -- 172.21.15.151:0/3508919003 wait complete. 2024-04-26T01:59:57.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:57.246+0000 7f8907600700 1 Processor -- start 2024-04-26T01:59:57.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:57.246+0000 7f8907600700 1 -- start start 2024-04-26T01:59:57.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:57.246+0000 7f8907600700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8900102a90 0x7f890019b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:57.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:57.246+0000 7f8907600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8900103450 0x7f890019b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:57.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:57.246+0000 7f8907600700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8900108a50 0x7f890019fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:57.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:57.247+0000 7f8907600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8900112c20 con 0x7f8900103450 2024-04-26T01:59:57.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:57.247+0000 7f8907600700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8900112aa0 con 0x7f8900102a90 2024-04-26T01:59:57.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:57.247+0000 7f8907600700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8900112da0 con 0x7f8900108a50 2024-04-26T01:59:57.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:57.247+0000 7f8904c00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8900102a90 0x7f890019b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:57.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:57.247+0000 7f8904c00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8900102a90 0x7f890019b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:36686/0 (socket says 172.21.15.151:36686) 2024-04-26T01:59:57.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:57.247+0000 7f8904c00700 1 -- 172.21.15.151:0/3283185765 learned_addr learned my addr 172.21.15.151:0/3283185765 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T01:59:57.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:57.247+0000 7f88ffe00700 1 --2- 172.21.15.151:0/3283185765 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8900103450 0x7f890019b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:57.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:57.247+0000 7f8905600700 1 --2- 172.21.15.151:0/3283185765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8900108a50 0x7f890019fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:57.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:57.247+0000 7f8904c00700 1 -- 172.21.15.151:0/3283185765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8900108a50 msgr2=0x7f890019fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:57.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:57.247+0000 7f8904c00700 1 --2- 172.21.15.151:0/3283185765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8900108a50 0x7f890019fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:57.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:57.247+0000 7f8904c00700 1 -- 172.21.15.151:0/3283185765 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8900103450 msgr2=0x7f890019b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:57.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:57.247+0000 7f8904c00700 1 --2- 172.21.15.151:0/3283185765 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8900103450 0x7f890019b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:57.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:57.247+0000 7f8904c00700 1 -- 172.21.15.151:0/3283185765 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89001a02e0 con 0x7f8900102a90 2024-04-26T01:59:57.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:57.247+0000 7f8905600700 1 --2- 172.21.15.151:0/3283185765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8900108a50 0x7f890019fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T01:59:57.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:57.247+0000 7f88ffe00700 1 --2- 172.21.15.151:0/3283185765 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8900103450 0x7f890019b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:59:57.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:57.248+0000 7f8904c00700 1 --2- 172.21.15.151:0/3283185765 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8900102a90 0x7f890019b2d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f88f400e9a0 tx=0x7f88f400ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:57.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:57.248+0000 7f88fd600700 1 -- 172.21.15.151:0/3283185765 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f400cbd0 con 0x7f8900102a90 2024-04-26T01:59:57.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:57.248+0000 7f88fd600700 1 -- 172.21.15.151:0/3283185765 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88f400cd70 con 0x7f8900102a90 2024-04-26T01:59:57.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:57.248+0000 7f8907600700 1 -- 172.21.15.151:0/3283185765 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89001a05d0 con 0x7f8900102a90 2024-04-26T01:59:57.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:57.248+0000 7f88fd600700 1 -- 172.21.15.151:0/3283185765 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f4018680 con 0x7f8900102a90 2024-04-26T01:59:57.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:57.248+0000 7f8907600700 1 -- 172.21.15.151:0/3283185765 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89001a7e50 con 0x7f8900102a90 2024-04-26T01:59:57.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:57.249+0000 7f8907600700 1 -- 172.21.15.151:0/3283185765 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8900069730 con 0x7f8900102a90 2024-04-26T01:59:57.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:57.251+0000 7f88fd600700 1 -- 172.21.15.151:0/3283185765 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f88f4010970 con 0x7f8900102a90 2024-04-26T01:59:57.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:57.253+0000 7f88fd600700 1 --2- 172.21.15.151:0/3283185765 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f88e80a3e30 0x7f88e80a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:57.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:57.253+0000 7f88fd600700 1 -- 172.21.15.151:0/3283185765 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8411+0+0 (secure 0 0 0) 0x7f88f40d0a70 con 0x7f8900102a90 2024-04-26T01:59:57.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:57.253+0000 7f88ffe00700 1 --2- 172.21.15.151:0/3283185765 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f88e80a3e30 0x7f88e80a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:57.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:57.254+0000 7f88ffe00700 1 --2- 172.21.15.151:0/3283185765 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f88e80a3e30 0x7f88e80a62f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f890007a4f0 tx=0x7f88f0009260 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:57.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:57.257+0000 7f88fd600700 1 -- 172.21.15.151:0/3283185765 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f88f409d120 con 0x7f8900102a90 2024-04-26T01:59:57.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:57.532+0000 7f8907600700 1 -- 172.21.15.151:0/3283185765 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f89001a80c0 con 0x7f8900102a90 2024-04-26T01:59:57.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:57.534+0000 7f88fd600700 1 -- 172.21.15.151:0/3283185765 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11889 (secure 0 0 0) 0x7f88f409ca20 con 0x7f8900102a90 2024-04-26T01:59:57.533 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T01:59:57.533 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2937864097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2937864097},{"type":"v1","addr":"172.21.15.151:6835","nonce":2937864097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14373,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/2947090802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2947090802},{"type":"v1","addr":"172.21.15.151:6837","nonce":2947090802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/1356563280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1356563280},{"type":"v1","addr":"172.21.15.151:6839","nonce":1356563280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6841/1075240552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1075240552},{"type":"v1","addr":"172.21.15.151:6841","nonce":1075240552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/504675912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":504675912},{"type":"v1","addr":"172.21.15.165:6833","nonce":504675912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1207580015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1207580015},{"type":"v1","addr":"172.21.15.172:6833","nonce":1207580015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:49:40.746876+0000","modified":"2024-04-26T01:59:39.148772+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24505,"mds_2":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/3598138294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3598138294},{"type":"v1","addr":"172.21.15.165:6835","nonce":3598138294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/1847859054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":1847859054},{"type":"v1","addr":"172.21.15.165:6837","nonce":1847859054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.172:6837/55972537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":55972537},{"type":"v1","addr":"172.21.15.172:6837","nonce":55972537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.172:6839/1242302810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1242302810},{"type":"v1","addr":"172.21.15.172:6839","nonce":1242302810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/1969716254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1969716254},{"type":"v1","addr":"172.21.15.165:6839","nonce":1969716254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25876":{"gid":25876,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4230821277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4230821277},{"type":"v1","addr":"172.21.15.172:6835","nonce":4230821277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24509,24493,24505]},"id":1}]} 2024-04-26T01:59:57.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:57.537+0000 7f8907600700 1 -- 172.21.15.151:0/3283185765 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f88e80a3e30 msgr2=0x7f88e80a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:57.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:57.537+0000 7f8907600700 1 --2- 172.21.15.151:0/3283185765 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f88e80a3e30 0x7f88e80a62f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f890007a4f0 tx=0x7f88f0009260 comp rx=0 tx=0).stop 2024-04-26T01:59:57.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:57.537+0000 7f8907600700 1 -- 172.21.15.151:0/3283185765 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8900102a90 msgr2=0x7f890019b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:57.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:57.537+0000 7f8907600700 1 --2- 172.21.15.151:0/3283185765 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8900102a90 0x7f890019b2d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f88f400e9a0 tx=0x7f88f400ed60 comp rx=0 tx=0).stop 2024-04-26T01:59:57.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:57.537+0000 7f8907600700 1 -- 172.21.15.151:0/3283185765 shutdown_connections 2024-04-26T01:59:57.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:57.537+0000 7f8907600700 1 --2- 172.21.15.151:0/3283185765 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f88e80a3e30 0x7f88e80a62f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:57.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:57.537+0000 7f8907600700 1 --2- 172.21.15.151:0/3283185765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8900108a50 0x7f890019fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:57.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:57.537+0000 7f8907600700 1 --2- 172.21.15.151:0/3283185765 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8900102a90 0x7f890019b2d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:57.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:57.537+0000 7f8907600700 1 --2- 172.21.15.151:0/3283185765 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8900103450 0x7f890019b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:57.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:57.537+0000 7f8907600700 1 -- 172.21.15.151:0/3283185765 >> 172.21.15.151:0/3283185765 conn(0x7f8900079ba0 msgr2=0x7f890010df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:57.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:57.538+0000 7f8907600700 1 -- 172.21.15.151:0/3283185765 shutdown_connections 2024-04-26T01:59:57.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T01:59:57.538+0000 7f8907600700 1 -- 172.21.15.151:0/3283185765 wait complete. 2024-04-26T01:59:57.538 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 33 2024-04-26T01:59:58.340 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24509 quiesce db --set-id 6d684c64 --release --await' 2024-04-26T01:59:58.340 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24509 quiesce db --set-id 6d684c64 --release --await 2024-04-26T01:59:58.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:59:58 smithi151 ceph-mon[28057]: pgmap v195: 97 pgs: 97 active+clean; 47 MiB data, 956 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 11 KiB/s wr, 8 op/s 2024-04-26T01:59:58.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 01:59:58 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/3283185765' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:59:58.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:59:58 smithi172 ceph-mon[32785]: pgmap v195: 97 pgs: 97 active+clean; 47 MiB data, 956 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 11 KiB/s wr, 8 op/s 2024-04-26T01:59:58.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 01:59:58 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/3283185765' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:59:58.792 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:59:58.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:59:58 smithi165 ceph-mon[32648]: pgmap v195: 97 pgs: 97 active+clean; 47 MiB data, 956 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 11 KiB/s wr, 8 op/s 2024-04-26T01:59:58.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 01:59:58 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/3283185765' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:00:00.595 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:00:00 smithi151 ceph-mon[28057]: pgmap v196: 97 pgs: 97 active+clean; 48 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 11 op/s 2024-04-26T02:00:00.596 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:00:00 smithi151 ceph-mon[28057]: overall HEALTH_OK 2024-04-26T02:00:00.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:00:00 smithi172 ceph-mon[32785]: pgmap v196: 97 pgs: 97 active+clean; 48 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 11 op/s 2024-04-26T02:00:00.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:00:00 smithi172 ceph-mon[32785]: overall HEALTH_OK 2024-04-26T02:00:00.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:00:00 smithi165 ceph-mon[32648]: pgmap v196: 97 pgs: 97 active+clean; 48 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 11 op/s 2024-04-26T02:00:00.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:00:00 smithi165 ceph-mon[32648]: overall HEALTH_OK 2024-04-26T02:00:00.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.899+0000 7f36e3600700 1 -- 172.21.15.151:0/1958757989 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36dc108a80 msgr2=0x7f36dc108e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:00.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.899+0000 7f36e3600700 1 --2- 172.21.15.151:0/1958757989 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36dc108a80 0x7f36dc108e60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f36d0009990 tx=0x7f36d0009ca0 comp rx=0 tx=0).stop 2024-04-26T02:00:00.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.899+0000 7f36e3600700 1 -- 172.21.15.151:0/1958757989 shutdown_connections 2024-04-26T02:00:00.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.899+0000 7f36e3600700 1 --2- 172.21.15.151:0/1958757989 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f36dc103450 0x7f36dc10f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:00.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.899+0000 7f36e3600700 1 --2- 172.21.15.151:0/1958757989 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36dc108a80 0x7f36dc108e60 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:00.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.899+0000 7f36e3600700 1 --2- 172.21.15.151:0/1958757989 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f36dc102a90 0x7f36dc102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:00.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.899+0000 7f36e3600700 1 -- 172.21.15.151:0/1958757989 >> 172.21.15.151:0/1958757989 conn(0x7f36dc079bf0 msgr2=0x7f36dc07a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:00.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.899+0000 7f36e3600700 1 -- 172.21.15.151:0/1958757989 shutdown_connections 2024-04-26T02:00:00.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.900+0000 7f36e3600700 1 -- 172.21.15.151:0/1958757989 wait complete. 2024-04-26T02:00:00.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.901+0000 7f36e3600700 1 Processor -- start 2024-04-26T02:00:00.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.901+0000 7f36e3600700 1 -- start start 2024-04-26T02:00:00.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.901+0000 7f36e3600700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36dc102a90 0x7f36dc074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:00.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.901+0000 7f36e3600700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f36dc103450 0x7f36dc075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:00.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.901+0000 7f36e3600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f36dc108a80 0x7f36dc0757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:00.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.901+0000 7f36e3600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f36dc112c70 con 0x7f36dc108a80 2024-04-26T02:00:00.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.901+0000 7f36e3600700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f36dc112af0 con 0x7f36dc103450 2024-04-26T02:00:00.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.901+0000 7f36e3600700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f36dc112df0 con 0x7f36dc102a90 2024-04-26T02:00:00.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.902+0000 7f36dbe00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f36dc103450 0x7f36dc075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:00.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.902+0000 7f36dbe00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f36dc103450 0x7f36dc075280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:44880/0 (socket says 172.21.15.151:44880) 2024-04-26T02:00:00.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.902+0000 7f36e1600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f36dc108a80 0x7f36dc0757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:00.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.902+0000 7f36e0c00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36dc102a90 0x7f36dc074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:00.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.902+0000 7f36dbe00700 1 -- 172.21.15.151:0/1217703956 learned_addr learned my addr 172.21.15.151:0/1217703956 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:00:00.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.902+0000 7f36e0c00700 1 -- 172.21.15.151:0/1217703956 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f36dc103450 msgr2=0x7f36dc075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:00.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.902+0000 7f36e0c00700 1 --2- 172.21.15.151:0/1217703956 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f36dc103450 0x7f36dc075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:00.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.902+0000 7f36e0c00700 1 -- 172.21.15.151:0/1217703956 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f36dc108a80 msgr2=0x7f36dc0757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:00.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.902+0000 7f36e0c00700 1 --2- 172.21.15.151:0/1217703956 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f36dc108a80 0x7f36dc0757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:00.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.902+0000 7f36e0c00700 1 -- 172.21.15.151:0/1217703956 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36dc1a7880 con 0x7f36dc102a90 2024-04-26T02:00:00.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.903+0000 7f36e0c00700 1 --2- 172.21.15.151:0/1217703956 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36dc102a90 0x7f36dc074d40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f36d0000c00 tx=0x7f36d00047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:00.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.903+0000 7f36d9600700 1 -- 172.21.15.151:0/1217703956 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d0026070 con 0x7f36dc102a90 2024-04-26T02:00:00.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.903+0000 7f36e3600700 1 -- 172.21.15.151:0/1217703956 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36dc1a7b10 con 0x7f36dc102a90 2024-04-26T02:00:00.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.903+0000 7f36d9600700 1 -- 172.21.15.151:0/1217703956 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36d0021410 con 0x7f36dc102a90 2024-04-26T02:00:00.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.903+0000 7f36d9600700 1 -- 172.21.15.151:0/1217703956 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d0017650 con 0x7f36dc102a90 2024-04-26T02:00:00.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.903+0000 7f36e3600700 1 -- 172.21.15.151:0/1217703956 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36dc1a7ff0 con 0x7f36dc102a90 2024-04-26T02:00:00.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.908+0000 7f36d9600700 1 -- 172.21.15.151:0/1217703956 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f36d0029070 con 0x7f36dc102a90 2024-04-26T02:00:00.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.909+0000 7f36edf70b80 1 Processor -- start 2024-04-26T02:00:00.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.909+0000 7f36edf70b80 1 -- start start 2024-04-26T02:00:00.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.910+0000 7f36d9600700 1 --2- 172.21.15.151:0/1217703956 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f36c40a3db0 0x7f36c40a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:00.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.910+0000 7f36edf70b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564aa199c920 0x564aa199cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:00.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.910+0000 7f36edf70b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564aa199d240 0x564aa19a7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:00.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.910+0000 7f36edf70b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564aa19a80f0 0x564aa19aa4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:00.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.910+0000 7f36d9600700 1 -- 172.21.15.151:0/1217703956 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8411+0+0 (secure 0 0 0) 0x7f36d00d1440 con 0x7f36dc102a90 2024-04-26T02:00:00.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.910+0000 7f36edf70b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564aa18eff90 con 0x564aa19a80f0 2024-04-26T02:00:00.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.910+0000 7f36edf70b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564aa1896830 con 0x564aa199c920 2024-04-26T02:00:00.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.910+0000 7f36edf70b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564aa18ebf50 con 0x564aa199d240 2024-04-26T02:00:00.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.910+0000 7f36e1600700 1 --2- 172.21.15.151:0/1217703956 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f36c40a3db0 0x7f36c40a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:00.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.910+0000 7f36e1600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564aa19a80f0 0x564aa19aa4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:00.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.910+0000 7f36dbe00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564aa199d240 0x564aa19a7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:00.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.910+0000 7f36dbe00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564aa199d240 0x564aa19a7bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:53388/0 (socket says 172.21.15.151:53388) 2024-04-26T02:00:00.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.910+0000 7f36dbe00700 1 -- 172.21.15.151:0/455094546 learned_addr learned my addr 172.21.15.151:0/455094546 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:00:00.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.910+0000 7f36e0c00700 1 --2- 172.21.15.151:0/455094546 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564aa199c920 0x564aa199cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:00.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.911+0000 7f36dbe00700 1 -- 172.21.15.151:0/455094546 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564aa199c920 msgr2=0x564aa199cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:00.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.911+0000 7f36dbe00700 1 --2- 172.21.15.151:0/455094546 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564aa199c920 0x564aa199cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:00.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.911+0000 7f36dbe00700 1 -- 172.21.15.151:0/455094546 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564aa19a80f0 msgr2=0x564aa19aa4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:00.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.911+0000 7f36dbe00700 1 --2- 172.21.15.151:0/455094546 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564aa19a80f0 0x564aa19aa4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:00.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.911+0000 7f36dbe00700 1 -- 172.21.15.151:0/455094546 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564aa188c780 con 0x564aa199d240 2024-04-26T02:00:00.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.911+0000 7f36dbe00700 1 --2- 172.21.15.151:0/455094546 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564aa199d240 0x564aa19a7bb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f36c8009d90 tx=0x7f36c8011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb422b48fd49ab5f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:00.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.911+0000 7f36e1600700 1 --2- 172.21.15.151:0/1217703956 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f36c40a3db0 0x7f36c40a6270 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f36d4009da0 tx=0x7f36d4009920 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:00.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.911+0000 7f36ce000700 1 -- 172.21.15.151:0/455094546 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c8004220 con 0x564aa199d240 2024-04-26T02:00:00.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.912+0000 7f36ce000700 1 -- 172.21.15.151:0/455094546 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36c80043c0 con 0x564aa199d240 2024-04-26T02:00:00.912 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.912+0000 7f36ce000700 1 -- 172.21.15.151:0/455094546 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c8005670 con 0x564aa199d240 2024-04-26T02:00:00.912 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.912+0000 7f36edf70b80 1 -- 172.21.15.151:0/455094546 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564aa199d240 msgr2=0x564aa19a7bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:00.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.912+0000 7f36edf70b80 1 --2- 172.21.15.151:0/455094546 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564aa199d240 0x564aa19a7bb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f36c8009d90 tx=0x7f36c8011720 comp rx=0 tx=0).stop 2024-04-26T02:00:00.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.913+0000 7f36edf70b80 1 -- 172.21.15.151:0/455094546 shutdown_connections 2024-04-26T02:00:00.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.913+0000 7f36edf70b80 1 --2- 172.21.15.151:0/455094546 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564aa19a80f0 0x564aa19aa4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:00.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.913+0000 7f36edf70b80 1 --2- 172.21.15.151:0/455094546 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564aa199c920 0x564aa199cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:00.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.913+0000 7f36edf70b80 1 --2- 172.21.15.151:0/455094546 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564aa199d240 0x564aa19a7bb0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:00.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.913+0000 7f36edf70b80 1 -- 172.21.15.151:0/455094546 >> 172.21.15.151:0/455094546 conn(0x564aa18cb6a0 msgr2=0x564aa1998dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:00.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.913+0000 7f36edf70b80 1 -- 172.21.15.151:0/455094546 shutdown_connections 2024-04-26T02:00:00.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.913+0000 7f36edf70b80 1 -- 172.21.15.151:0/455094546 wait complete. 2024-04-26T02:00:00.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.913+0000 7f36edf70b80 1 -- start start 2024-04-26T02:00:00.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.913+0000 7f36edf70b80 1 Processor -- start 2024-04-26T02:00:00.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.914+0000 7f36edf70b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564aa199c920 0x564aa19a7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:00.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.914+0000 7f36edf70b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564aa199d240 0x564aa19a4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:00.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.914+0000 7f36edf70b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564aa19a80f0 0x564aa19a5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:00.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.914+0000 7f36edf70b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564aa19aac10 con 0x564aa199c920 2024-04-26T02:00:00.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.914+0000 7f36edf70b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564aa18a9960 con 0x564aa19a80f0 2024-04-26T02:00:00.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.914+0000 7f36edf70b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564aa19aad90 con 0x564aa199d240 2024-04-26T02:00:00.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.914+0000 7f36edf70b80 10 client.0 _ll_register_callbacks cb 0x564aa1888820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:00:00.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.915+0000 7f36e1600700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564aa19a80f0 0x564aa19a5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:00.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.915+0000 7f36e1600700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564aa19a80f0 0x564aa19a5010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:44896/0 (socket says 172.21.15.151:44896) 2024-04-26T02:00:00.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.915+0000 7f36e1600700 1 -- 172.21.15.151:0/3228696888 learned_addr learned my addr 172.21.15.151:0/3228696888 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:00:00.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.915+0000 7f36e0c00700 1 --2- 172.21.15.151:0/3228696888 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564aa199c920 0x564aa19a7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:00.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.915+0000 7f36dbe00700 1 --2- 172.21.15.151:0/3228696888 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564aa199d240 0x564aa19a4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:00.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.915+0000 7f36e1600700 1 -- 172.21.15.151:0/3228696888 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564aa199d240 msgr2=0x564aa19a4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:00.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.915+0000 7f36e1600700 1 --2- 172.21.15.151:0/3228696888 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564aa199d240 0x564aa19a4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:00.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.915+0000 7f36e1600700 1 -- 172.21.15.151:0/3228696888 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564aa199c920 msgr2=0x564aa19a7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:00.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.915+0000 7f36e1600700 1 --2- 172.21.15.151:0/3228696888 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564aa199c920 0x564aa19a7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:00.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.915+0000 7f36e1600700 1 -- 172.21.15.151:0/3228696888 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564aa183d9d0 con 0x564aa19a80f0 2024-04-26T02:00:00.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.915+0000 7f36e0c00700 1 --2- 172.21.15.151:0/3228696888 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564aa199c920 0x564aa19a7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T02:00:00.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.916+0000 7f36e1600700 1 --2- 172.21.15.151:0/3228696888 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564aa19a80f0 0x564aa19a5010 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f36d401add0 tx=0x7f36d4004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f9e0d1c84660378 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:00.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.916+0000 7f36c2a00700 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T02:00:00.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.916+0000 7f36c2a00700 1 -- 172.21.15.151:0/3228696888 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d40253f0 con 0x564aa19a80f0 2024-04-26T02:00:00.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.916+0000 7f36c2a00700 1 -- 172.21.15.151:0/3228696888 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8411+0+0 (secure 0 0 0) 0x7f36d4025590 con 0x564aa19a80f0 2024-04-26T02:00:00.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.916+0000 7f36edf70b80 1 -- 172.21.15.151:0/3228696888 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36c80043c0 con 0x564aa19a80f0 2024-04-26T02:00:00.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.917+0000 7f36c2a00700 1 -- 172.21.15.151:0/3228696888 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f36d4033050 con 0x564aa19a80f0 2024-04-26T02:00:00.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.917+0000 7f36edf70b80 10 client.25886 fetch_fsmap learned FSMap version 33 2024-04-26T02:00:00.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.917+0000 7f36edf70b80 1 -- 172.21.15.151:0/3228696888 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564aa1886bf0 con 0x564aa19a80f0 2024-04-26T02:00:00.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.917+0000 7f36c2a00700 1 -- 172.21.15.151:0/3228696888 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36c80043c0 con 0x564aa19a80f0 2024-04-26T02:00:00.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.918+0000 7f36c2a00700 1 -- 172.21.15.151:0/3228696888 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 33) ==== 8338+0+0 (secure 0 0 0) 0x7f36d4018790 con 0x564aa19a80f0 2024-04-26T02:00:00.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.918+0000 7f36edf70b80 10 client.25886 fetch_fsmap finished waiting for FSMap version 33 2024-04-26T02:00:00.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.918+0000 7f36edf70b80 10 client.25886 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-26T02:00:00.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.918+0000 7f36edf70b80 20 client.25886 populate_metadata read hostname 'smithi151' 2024-04-26T02:00:00.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.918+0000 7f36edf70b80 1 --2- 172.21.15.151:0/3228696888 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x564aa19b7920 0x564aa19b9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:00.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.918+0000 7f36edf70b80 4 client.25886 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:00:00.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.918+0000 7f36edf70b80 1 -- 172.21.15.151:0/3228696888 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564aa18fd780 con 0x564aa19b7920 2024-04-26T02:00:00.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.918+0000 7f36e0c00700 1 --2- 172.21.15.151:0/3228696888 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x564aa19b7920 0x564aa19b9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:00.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.919+0000 7f36e0c00700 1 --2- 172.21.15.151:0/3228696888 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x564aa19b7920 0x564aa19b9d30 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38ac70739f649af7 server_cookie=b37a99249dd0b757 in_seq=0 out_seq=0 2024-04-26T02:00:00.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.919+0000 7f36c2a00700 10 client.25886 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:00:00.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.919+0000 7f36c2a00700 1 -- 172.21.15.151:0/3228696888 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564aa18fd780 con 0x564aa19b7920 2024-04-26T02:00:00.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.919+0000 7f36c2a00700 10 client.25886 handle_command_reply: tid=0 2024-04-26T02:00:00.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.919+0000 7f36edf70b80 1 client.25886 shutdown 2024-04-26T02:00:00.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.920+0000 7f36edf70b80 1 -- 172.21.15.151:0/3228696888 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564aa19a80f0 msgr2=0x564aa19a5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:00.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.920+0000 7f36edf70b80 1 --2- 172.21.15.151:0/3228696888 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564aa19a80f0 0x564aa19a5010 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f36d401add0 tx=0x7f36d4004b40 comp rx=0 tx=0).stop 2024-04-26T02:00:00.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.920+0000 7f36edf70b80 1 -- 172.21.15.151:0/3228696888 shutdown_connections 2024-04-26T02:00:00.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.920+0000 7f36edf70b80 1 --2- 172.21.15.151:0/3228696888 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x564aa19b7920 0x564aa19b9d30 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:00.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.920+0000 7f36edf70b80 1 --2- 172.21.15.151:0/3228696888 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564aa19a80f0 0x564aa19a5010 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:00.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.920+0000 7f36edf70b80 1 --2- 172.21.15.151:0/3228696888 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564aa199c920 0x564aa19a7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:00.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.920+0000 7f36c2a00700 0 client.25886 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:00:00.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.920+0000 7f36edf70b80 1 --2- 172.21.15.151:0/3228696888 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564aa199d240 0x564aa19a4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:00.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.920+0000 7f36edf70b80 1 -- 172.21.15.151:0/3228696888 >> 172.21.15.151:0/3228696888 conn(0x564aa18cb6a0 msgr2=0x564aa19989c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:00.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.920+0000 7f36edf70b80 1 -- 172.21.15.151:0/3228696888 shutdown_connections 2024-04-26T02:00:00.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.920+0000 7f36edf70b80 1 -- 172.21.15.151:0/3228696888 wait complete. 2024-04-26T02:00:00.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.920+0000 7f36edf70b80 20 client.25886 trim_cache size 0 max 16384 2024-04-26T02:00:00.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.944+0000 7f36edf70b80 1 Processor -- start 2024-04-26T02:00:00.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.944+0000 7f36edf70b80 1 -- start start 2024-04-26T02:00:00.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.944+0000 7f36edf70b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564aa199c920 0x564aa19b3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:00.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.944+0000 7f36edf70b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564aa199d240 0x564aa19a6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:00.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.944+0000 7f36edf70b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564aa19a80f0 0x564aa19bf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:00.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.944+0000 7f36edf70b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f36dc1b1d80 con 0x564aa199c920 2024-04-26T02:00:00.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.944+0000 7f36edf70b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f36dc1af610 con 0x564aa199d240 2024-04-26T02:00:00.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.944+0000 7f36edf70b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f36dc1ad480 con 0x564aa19a80f0 2024-04-26T02:00:00.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.945+0000 7f36e0c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564aa199c920 0x564aa19b3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:00.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.945+0000 7f36e1600700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564aa19a80f0 0x564aa19bf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:00.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.945+0000 7f36e0c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564aa199c920 0x564aa19b3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53472/0 (socket says 172.21.15.151:53472) 2024-04-26T02:00:00.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.945+0000 7f36e0c00700 1 -- 172.21.15.151:0/2856065186 learned_addr learned my addr 172.21.15.151:0/2856065186 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:00:00.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.945+0000 7f36dbe00700 1 --2- 172.21.15.151:0/2856065186 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564aa199d240 0x564aa19a6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:00.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.945+0000 7f36e1600700 1 -- 172.21.15.151:0/2856065186 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564aa199d240 msgr2=0x564aa19a6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:00.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.945+0000 7f36e1600700 1 --2- 172.21.15.151:0/2856065186 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564aa199d240 0x564aa19a6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:00.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.945+0000 7f36e1600700 1 -- 172.21.15.151:0/2856065186 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564aa199c920 msgr2=0x564aa19b3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:00.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.945+0000 7f36e1600700 1 --2- 172.21.15.151:0/2856065186 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564aa199c920 0x564aa19b3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:00.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.945+0000 7f36e1600700 1 -- 172.21.15.151:0/2856065186 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564aa19a0750 con 0x564aa19a80f0 2024-04-26T02:00:00.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.946+0000 7f36e1600700 1 --2- 172.21.15.151:0/2856065186 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564aa19a80f0 0x564aa19bf600 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f36d401ada0 tx=0x7f36d4009a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da42a256b058e285 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:00.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.946+0000 7f36cd600700 1 -- 172.21.15.151:0/2856065186 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d403d020 con 0x564aa19a80f0 2024-04-26T02:00:00.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.946+0000 7f36cd600700 1 -- 172.21.15.151:0/2856065186 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36d402a070 con 0x564aa19a80f0 2024-04-26T02:00:00.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.946+0000 7f36cd600700 1 -- 172.21.15.151:0/2856065186 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d402f450 con 0x564aa19a80f0 2024-04-26T02:00:00.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.947+0000 7f36edf70b80 1 -- 172.21.15.151:0/2856065186 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564aa19a80f0 msgr2=0x564aa19bf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:00.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.947+0000 7f36edf70b80 1 --2- 172.21.15.151:0/2856065186 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564aa19a80f0 0x564aa19bf600 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f36d401ada0 tx=0x7f36d4009a10 comp rx=0 tx=0).stop 2024-04-26T02:00:00.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.947+0000 7f36edf70b80 1 -- 172.21.15.151:0/2856065186 shutdown_connections 2024-04-26T02:00:00.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.947+0000 7f36edf70b80 1 --2- 172.21.15.151:0/2856065186 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564aa19a80f0 0x564aa19bf600 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:00.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.947+0000 7f36edf70b80 1 --2- 172.21.15.151:0/2856065186 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564aa199c920 0x564aa19b3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:00.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.947+0000 7f36edf70b80 1 --2- 172.21.15.151:0/2856065186 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564aa199d240 0x564aa19a6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:00.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.947+0000 7f36edf70b80 1 -- 172.21.15.151:0/2856065186 >> 172.21.15.151:0/2856065186 conn(0x564aa18cb6a0 msgr2=0x564aa19a6300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:00.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.947+0000 7f36edf70b80 1 -- 172.21.15.151:0/2856065186 shutdown_connections 2024-04-26T02:00:00.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.947+0000 7f36edf70b80 1 -- 172.21.15.151:0/2856065186 wait complete. 2024-04-26T02:00:00.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.947+0000 7f36edf70b80 1 -- start start 2024-04-26T02:00:00.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.948+0000 7f36edf70b80 1 Processor -- start 2024-04-26T02:00:00.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.948+0000 7f36edf70b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564aa199c920 0x564aa19b0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:00.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.948+0000 7f36edf70b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564aa199d240 0x564aa19a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:00.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.948+0000 7f36edf70b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564aa19a80f0 0x564aa19a12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:00.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.948+0000 7f36edf70b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f36dc1adad0 con 0x564aa199d240 2024-04-26T02:00:00.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.948+0000 7f36edf70b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f36dc1aef40 con 0x564aa19a80f0 2024-04-26T02:00:00.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.948+0000 7f36edf70b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f36dc1ae1a0 con 0x564aa199c920 2024-04-26T02:00:00.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.948+0000 7f36edf70b80 10 client.0 _ll_register_callbacks cb 0x564aa1888820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:00:00.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.948+0000 7f36e0c00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564aa199c920 0x564aa19b0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:00.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.949+0000 7f36e0c00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564aa199c920 0x564aa19b0230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:53426/0 (socket says 172.21.15.151:53426) 2024-04-26T02:00:00.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.949+0000 7f36dbe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564aa199d240 0x564aa19a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:00.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.949+0000 7f36e0c00700 1 -- 172.21.15.151:0/22083953 learned_addr learned my addr 172.21.15.151:0/22083953 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:00:00.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.949+0000 7f36e1600700 1 --2- 172.21.15.151:0/22083953 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564aa19a80f0 0x564aa19a12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:00.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.949+0000 7f36dbe00700 1 -- 172.21.15.151:0/22083953 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564aa199c920 msgr2=0x564aa19b0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:00.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.949+0000 7f36dbe00700 1 --2- 172.21.15.151:0/22083953 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564aa199c920 0x564aa19b0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:00.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.949+0000 7f36dbe00700 1 -- 172.21.15.151:0/22083953 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564aa19a80f0 msgr2=0x564aa19a12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:00.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.949+0000 7f36dbe00700 1 --2- 172.21.15.151:0/22083953 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564aa19a80f0 0x564aa19a12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:00.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.949+0000 7f36dbe00700 1 -- 172.21.15.151:0/22083953 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564aa19a1820 con 0x564aa199d240 2024-04-26T02:00:00.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.949+0000 7f36e0c00700 1 --2- 172.21.15.151:0/22083953 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564aa199c920 0x564aa19b0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:00:00.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.949+0000 7f36e1600700 1 --2- 172.21.15.151:0/22083953 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564aa19a80f0 0x564aa19a12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:00:00.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.950+0000 7f36dbe00700 1 --2- 172.21.15.151:0/22083953 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564aa199d240 0x564aa19a0da0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f36c8005170 tx=0x7f36c8004940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=766dd0e4eda552eb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:00.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.950+0000 7f36c2a00700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-04-26T02:00:00.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.950+0000 7f36c2a00700 1 -- 172.21.15.151:0/22083953 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c802a030 con 0x564aa199d240 2024-04-26T02:00:00.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.950+0000 7f36edf70b80 1 -- 172.21.15.151:0/22083953 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36d402a070 con 0x564aa199d240 2024-04-26T02:00:00.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.950+0000 7f36c2a00700 1 -- 172.21.15.151:0/22083953 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8411+0+0 (secure 0 0 0) 0x7f36c8004cc0 con 0x564aa199d240 2024-04-26T02:00:00.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.950+0000 7f36c2a00700 1 -- 172.21.15.151:0/22083953 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f36c8037050 con 0x564aa199d240 2024-04-26T02:00:00.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.950+0000 7f36edf70b80 10 client.15795 fetch_fsmap learned FSMap version 33 2024-04-26T02:00:00.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.950+0000 7f36edf70b80 1 -- 172.21.15.151:0/22083953 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564aa19a2b30 con 0x564aa199d240 2024-04-26T02:00:00.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.951+0000 7f36c2a00700 1 -- 172.21.15.151:0/22083953 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36d402a070 con 0x564aa199d240 2024-04-26T02:00:00.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.951+0000 7f36c2a00700 1 -- 172.21.15.151:0/22083953 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 33) ==== 8338+0+0 (secure 0 0 0) 0x7f36c8023050 con 0x564aa199d240 2024-04-26T02:00:00.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.951+0000 7f36edf70b80 10 client.15795 fetch_fsmap finished waiting for FSMap version 33 2024-04-26T02:00:00.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.951+0000 7f36edf70b80 10 client.15795 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-26T02:00:00.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.951+0000 7f36edf70b80 20 client.15795 populate_metadata read hostname 'smithi151' 2024-04-26T02:00:00.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.951+0000 7f36edf70b80 1 --2- 172.21.15.151:0/22083953 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x564aa1a1f150 0x564aa1a21570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:00.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.951+0000 7f36edf70b80 4 client.15795 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "6d684c64", "release": true, "await": true}] 2024-04-26T02:00:00.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.951+0000 7f36edf70b80 1 -- 172.21.15.151:0/22083953 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "quiesce db", "set_id": "6d684c64", "release": true, "await": true}) -- 0x564aa18f7820 con 0x564aa1a1f150 2024-04-26T02:00:00.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.952+0000 7f36e0c00700 1 --2- 172.21.15.151:0/22083953 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x564aa1a1f150 0x564aa1a21570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:00.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.952+0000 7f36e0c00700 1 --2- 172.21.15.151:0/22083953 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x564aa1a1f150 0x564aa1a21570 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71557174fbc36751 server_cookie=b56407d29eedc199 in_seq=0 out_seq=0 2024-04-26T02:00:00.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:00.952+0000 7f36c2a00700 10 client.15795 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:00:01.135 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:01.135+0000 7f36c2a00700 1 -- 172.21.15.151:0/22083953 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x564aa18f7820 con 0x564aa1a1f150 2024-04-26T02:00:01.135 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:01.135+0000 7f36c2a00700 10 client.15795 handle_command_reply: tid=0 2024-04-26T02:00:01.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:01.136+0000 7f36edf70b80 1 client.15795 shutdown 2024-04-26T02:00:01.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:01.136+0000 7f36edf70b80 1 -- 172.21.15.151:0/22083953 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564aa199d240 msgr2=0x564aa19a0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:01.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:01.136+0000 7f36edf70b80 1 --2- 172.21.15.151:0/22083953 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564aa199d240 0x564aa19a0da0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f36c8005170 tx=0x7f36c8004940 comp rx=0 tx=0).stop 2024-04-26T02:00:01.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:01.136+0000 7f36edf70b80 1 -- 172.21.15.151:0/22083953 shutdown_connections 2024-04-26T02:00:01.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:01.136+0000 7f36edf70b80 1 --2- 172.21.15.151:0/22083953 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x564aa1a1f150 0x564aa1a21570 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:01.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:01.136+0000 7f36edf70b80 1 --2- 172.21.15.151:0/22083953 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564aa19a80f0 0x564aa19a12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:01.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:01.136+0000 7f36edf70b80 1 --2- 172.21.15.151:0/22083953 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564aa199c920 0x564aa19b0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:01.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:01.136+0000 7f36edf70b80 1 --2- 172.21.15.151:0/22083953 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564aa199d240 0x564aa19a0da0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:01.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:01.136+0000 7f36edf70b80 1 -- 172.21.15.151:0/22083953 >> 172.21.15.151:0/22083953 conn(0x564aa18cb6a0 msgr2=0x564aa19a6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:01.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:01.136+0000 7f36c2a00700 0 client.15795 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:00:01.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:01.137+0000 7f36edf70b80 1 -- 172.21.15.151:0/22083953 shutdown_connections 2024-04-26T02:00:01.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:01.137+0000 7f36edf70b80 1 -- 172.21.15.151:0/22083953 wait complete. 2024-04-26T02:00:01.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:01.137+0000 7f36edf70b80 20 client.15795 trim_cache size 0 max 16384 2024-04-26T02:00:01.137 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-26T02:00:01.138 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 33, 2024-04-26T02:00:01.138 INFO:teuthology.orchestra.run.smithi151.stdout: "leader": 24509, 2024-04-26T02:00:01.138 INFO:teuthology.orchestra.run.smithi151.stdout: "set_version": 16, 2024-04-26T02:00:01.138 INFO:teuthology.orchestra.run.smithi151.stdout: "sets": { 2024-04-26T02:00:01.138 INFO:teuthology.orchestra.run.smithi151.stdout: "6d684c64": { 2024-04-26T02:00:01.138 INFO:teuthology.orchestra.run.smithi151.stdout: "version": 16, 2024-04-26T02:00:01.138 INFO:teuthology.orchestra.run.smithi151.stdout: "age_ref": 0.0, 2024-04-26T02:00:01.138 INFO:teuthology.orchestra.run.smithi151.stdout: "state": { 2024-04-26T02:00:01.138 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "RELEASED", 2024-04-26T02:00:01.138 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 0.0 2024-04-26T02:00:01.138 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:00:01.138 INFO:teuthology.orchestra.run.smithi151.stdout: "timeout": 60.0, 2024-04-26T02:00:01.138 INFO:teuthology.orchestra.run.smithi151.stdout: "expiration": 70.0, 2024-04-26T02:00:01.139 INFO:teuthology.orchestra.run.smithi151.stdout: "members": { 2024-04-26T02:00:01.139 INFO:teuthology.orchestra.run.smithi151.stdout: "file:/": { 2024-04-26T02:00:01.139 INFO:teuthology.orchestra.run.smithi151.stdout: "excluded": false, 2024-04-26T02:00:01.139 INFO:teuthology.orchestra.run.smithi151.stdout: "state": { 2024-04-26T02:00:01.139 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "RELEASED", 2024-04-26T02:00:01.139 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 0.0 2024-04-26T02:00:01.139 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:00:01.139 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:00:01.139 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:00:01.139 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:00:01.139 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:00:01.139 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-26T02:00:01.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:01.138+0000 7f36e3600700 1 -- 172.21.15.151:0/1217703956 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f36c40a3db0 msgr2=0x7f36c40a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:01.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:01.138+0000 7f36e3600700 1 --2- 172.21.15.151:0/1217703956 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f36c40a3db0 0x7f36c40a6270 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f36d4009da0 tx=0x7f36d4009920 comp rx=0 tx=0).stop 2024-04-26T02:00:01.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:01.138+0000 7f36e3600700 1 -- 172.21.15.151:0/1217703956 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36dc102a90 msgr2=0x7f36dc074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:01.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:01.138+0000 7f36e3600700 1 --2- 172.21.15.151:0/1217703956 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36dc102a90 0x7f36dc074d40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f36d0000c00 tx=0x7f36d00047b0 comp rx=0 tx=0).stop 2024-04-26T02:00:01.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:01.139+0000 7f36e3600700 1 -- 172.21.15.151:0/1217703956 shutdown_connections 2024-04-26T02:00:01.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:01.139+0000 7f36e3600700 1 --2- 172.21.15.151:0/1217703956 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f36c40a3db0 0x7f36c40a6270 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:01.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:01.139+0000 7f36e3600700 1 --2- 172.21.15.151:0/1217703956 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f36dc108a80 0x7f36dc0757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:01.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:01.139+0000 7f36e3600700 1 --2- 172.21.15.151:0/1217703956 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36dc102a90 0x7f36dc074d40 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:01.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:01.139+0000 7f36e3600700 1 --2- 172.21.15.151:0/1217703956 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f36dc103450 0x7f36dc075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:01.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:01.139+0000 7f36e3600700 1 -- 172.21.15.151:0/1217703956 >> 172.21.15.151:0/1217703956 conn(0x7f36dc079bf0 msgr2=0x7f36dc1018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:01.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:01.139+0000 7f36e3600700 1 -- 172.21.15.151:0/1217703956 shutdown_connections 2024-04-26T02:00:01.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:01.139+0000 7f36e3600700 1 -- 172.21.15.151:0/1217703956 wait complete. 2024-04-26T02:00:01.964 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '6d684c64', seconds elapsed: 27.0 2024-04-26T02:00:02.390 INFO:tasks.workunit.client.0.smithi151.stdout:30000 trunc from 0x7a5444 to 0x143a5 2024-04-26T02:00:02.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:00:02 smithi172 ceph-mon[32785]: pgmap v197: 97 pgs: 97 active+clean; 48 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 35 KiB/s wr, 9 op/s 2024-04-26T02:00:02.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:00:02 smithi165 ceph-mon[32648]: pgmap v197: 97 pgs: 97 active+clean; 48 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 35 KiB/s wr, 9 op/s 2024-04-26T02:00:02.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:00:02 smithi151 ceph-mon[28057]: pgmap v197: 97 pgs: 97 active+clean; 48 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 35 KiB/s wr, 9 op/s 2024-04-26T02:00:04.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:00:04 smithi172 ceph-mon[32785]: pgmap v198: 97 pgs: 97 active+clean; 49 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 137 KiB/s wr, 27 op/s 2024-04-26T02:00:04.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:00:04 smithi172 ceph-mon[32785]: osdmap e96: 12 total, 12 up, 12 in 2024-04-26T02:00:04.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:00:04 smithi165 ceph-mon[32648]: pgmap v198: 97 pgs: 97 active+clean; 49 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 137 KiB/s wr, 27 op/s 2024-04-26T02:00:04.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:00:04 smithi165 ceph-mon[32648]: osdmap e96: 12 total, 12 up, 12 in 2024-04-26T02:00:04.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:00:04 smithi151 ceph-mon[28057]: pgmap v198: 97 pgs: 97 active+clean; 49 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 137 KiB/s wr, 27 op/s 2024-04-26T02:00:04.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:00:04 smithi151 ceph-mon[28057]: osdmap e96: 12 total, 12 up, 12 in 2024-04-26T02:00:04.887 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.888+0000 7f84d1400640 1 -- 172.21.15.151:0/389246562 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84cc0742f0 msgr2=0x7f84cc0746d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:04.887 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.888+0000 7f84d1400640 1 --2- 172.21.15.151:0/389246562 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84cc0742f0 0x7f84cc0746d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f84bc009900 tx=0x7f84bc02a480 comp rx=0 tx=0).stop 2024-04-26T02:00:04.887 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.888+0000 7f84d1400640 1 -- 172.21.15.151:0/389246562 shutdown_connections 2024-04-26T02:00:04.887 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.888+0000 7f84d1400640 1 --2- 172.21.15.151:0/389246562 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84cc112f40 0x7f84cc115330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:04.887 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.888+0000 7f84d1400640 1 --2- 172.21.15.151:0/389246562 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f84cc074c10 0x7f84cc112a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:04.887 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.888+0000 7f84d1400640 1 --2- 172.21.15.151:0/389246562 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84cc0742f0 0x7f84cc0746d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:04.888 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.888+0000 7f84d1400640 1 -- 172.21.15.151:0/389246562 >> 172.21.15.151:0/389246562 conn(0x7f84cc0fda00 msgr2=0x7f84cc0ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:04.888 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.888+0000 7f84d1400640 1 -- 172.21.15.151:0/389246562 shutdown_connections 2024-04-26T02:00:04.888 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.888+0000 7f84d1400640 1 -- 172.21.15.151:0/389246562 wait complete. 2024-04-26T02:00:04.889 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.889+0000 7f84d1400640 1 Processor -- start 2024-04-26T02:00:04.889 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.890+0000 7f84d1400640 1 -- start start 2024-04-26T02:00:04.889 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.890+0000 7f84d1400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f84cc0742f0 0x7f84cc1a4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:04.889 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.890+0000 7f84d1400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84cc074c10 0x7f84cc1a4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:04.889 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.890+0000 7f84d1400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84cc112f40 0x7f84cc19eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:04.889 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.890+0000 7f84d1400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f84cc118810 con 0x7f84cc0742f0 2024-04-26T02:00:04.889 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.890+0000 7f84d1400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f84cc118690 con 0x7f84cc112f40 2024-04-26T02:00:04.889 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.890+0000 7f84d1400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f84cc118990 con 0x7f84cc074c10 2024-04-26T02:00:04.890 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.890+0000 7f84ca800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f84cc0742f0 0x7f84cc1a4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:04.891 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.890+0000 7f84ca800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f84cc0742f0 0x7f84cc1a4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53500/0 (socket says 172.21.15.151:53500) 2024-04-26T02:00:04.891 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.890+0000 7f84cb200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84cc112f40 0x7f84cc19eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:04.891 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.890+0000 7f84ca800640 1 -- 172.21.15.151:0/772422046 learned_addr learned my addr 172.21.15.151:0/772422046 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:00:04.891 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.890+0000 7f84c9e00640 1 --2- 172.21.15.151:0/772422046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84cc074c10 0x7f84cc1a4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:04.891 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.891+0000 7f84cb200640 1 -- 172.21.15.151:0/772422046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84cc074c10 msgr2=0x7f84cc1a4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:04.891 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.891+0000 7f84cb200640 1 --2- 172.21.15.151:0/772422046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84cc074c10 0x7f84cc1a4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:04.891 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.891+0000 7f84cb200640 1 -- 172.21.15.151:0/772422046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f84cc0742f0 msgr2=0x7f84cc1a4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:04.891 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.891+0000 7f84cb200640 1 --2- 172.21.15.151:0/772422046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f84cc0742f0 0x7f84cc1a4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:04.891 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.891+0000 7f84cb200640 1 -- 172.21.15.151:0/772422046 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84cc19f270 con 0x7f84cc112f40 2024-04-26T02:00:04.891 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.891+0000 7f84cb200640 1 --2- 172.21.15.151:0/772422046 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84cc112f40 0x7f84cc19eaf0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f84c4005fd0 tx=0x7f84c400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:04.891 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.892+0000 7f84c3400640 1 -- 172.21.15.151:0/772422046 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c400cdf0 con 0x7f84cc112f40 2024-04-26T02:00:04.891 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.892+0000 7f84c3400640 1 -- 172.21.15.151:0/772422046 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84c4005020 con 0x7f84cc112f40 2024-04-26T02:00:04.892 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.892+0000 7f84c3400640 1 -- 172.21.15.151:0/772422046 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c4010690 con 0x7f84cc112f40 2024-04-26T02:00:04.892 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.892+0000 7f84d1400640 1 -- 172.21.15.151:0/772422046 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84cc19f560 con 0x7f84cc112f40 2024-04-26T02:00:04.892 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.892+0000 7f84d1400640 1 -- 172.21.15.151:0/772422046 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84cc1ab800 con 0x7f84cc112f40 2024-04-26T02:00:04.894 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.896+0000 7f84d41fb740 1 Processor -- start 2024-04-26T02:00:04.895 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.896+0000 7f84c3400640 1 -- 172.21.15.151:0/772422046 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f84c4005460 con 0x7f84cc112f40 2024-04-26T02:00:04.895 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.897+0000 7f84d41fb740 1 -- start start 2024-04-26T02:00:04.896 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.897+0000 7f84d41fb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627b814b820 0x5627b814bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:04.896 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.897+0000 7f84d41fb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5627b814c140 0x5627b81564c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:04.896 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.897+0000 7f84d41fb740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5627b8156a00 0x5627b8158df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:04.896 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.897+0000 7f84d41fb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5627b80b1900 con 0x5627b814c140 2024-04-26T02:00:04.896 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.897+0000 7f84d41fb740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5627b8057ef0 con 0x5627b8156a00 2024-04-26T02:00:04.896 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.897+0000 7f84d41fb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5627b80ad8c0 con 0x5627b814b820 2024-04-26T02:00:04.898 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.897+0000 7f84c9e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5627b8156a00 0x5627b8158df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:04.898 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.898+0000 7f84c9e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5627b8156a00 0x5627b8158df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:44940/0 (socket says 172.21.15.151:44940) 2024-04-26T02:00:04.898 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.898+0000 7f84c9e00640 1 -- 172.21.15.151:0/140145825 learned_addr learned my addr 172.21.15.151:0/140145825 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:00:04.898 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.898+0000 7f84c9e00640 1 --2- 172.21.15.151:0/140145825 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627b814b820 0x5627b814bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:04.898 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.898+0000 7f84ca800640 1 --2- 172.21.15.151:0/140145825 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5627b814c140 0x5627b81564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:04.899 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.898+0000 7f84c9e00640 1 -- 172.21.15.151:0/140145825 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627b814b820 msgr2=0x5627b814bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:04.899 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.898+0000 7f84c9e00640 1 --2- 172.21.15.151:0/140145825 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627b814b820 0x5627b814bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:04.899 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.898+0000 7f84c9e00640 1 -- 172.21.15.151:0/140145825 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5627b814c140 msgr2=0x5627b81564c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:04.899 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.898+0000 7f84c9e00640 1 --2- 172.21.15.151:0/140145825 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5627b814c140 0x5627b81564c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:04.899 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.898+0000 7f84c9e00640 1 -- 172.21.15.151:0/140145825 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627b7dc4210 con 0x5627b8156a00 2024-04-26T02:00:04.899 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.898+0000 7f84ca800640 1 --2- 172.21.15.151:0/140145825 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5627b814c140 0x5627b81564c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T02:00:04.899 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.898+0000 7f84c3400640 1 --2- 172.21.15.151:0/772422046 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f84b00a3e40 0x7f84b00a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:04.899 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.898+0000 7f84c9e00640 1 --2- 172.21.15.151:0/140145825 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5627b8156a00 0x5627b8158df0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f84b80058c0 tx=0x7f84b8005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67a49a1cb072d44b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:04.899 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.898+0000 7f84c3400640 1 -- 172.21.15.151:0/772422046 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8499+0+0 (secure 0 0 0) 0x7f84c40d0df0 con 0x7f84cc112f40 2024-04-26T02:00:04.899 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.898+0000 7f84b7400640 1 -- 172.21.15.151:0/140145825 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b8004030 con 0x5627b8156a00 2024-04-26T02:00:04.899 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.898+0000 7f84b7400640 1 -- 172.21.15.151:0/140145825 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84b8005020 con 0x5627b8156a00 2024-04-26T02:00:04.900 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.898+0000 7f84b7400640 1 -- 172.21.15.151:0/140145825 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b8004ba0 con 0x5627b8156a00 2024-04-26T02:00:04.900 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.899+0000 7f84ca800640 1 --2- 172.21.15.151:0/772422046 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f84b00a3e40 0x7f84b00a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:04.900 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.899+0000 7f84d41fb740 1 -- 172.21.15.151:0/140145825 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5627b8156a00 msgr2=0x5627b8158df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:04.900 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.899+0000 7f84d41fb740 1 --2- 172.21.15.151:0/140145825 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5627b8156a00 0x5627b8158df0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f84b80058c0 tx=0x7f84b8005850 comp rx=0 tx=0).stop 2024-04-26T02:00:04.900 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.899+0000 7f84d41fb740 1 -- 172.21.15.151:0/140145825 shutdown_connections 2024-04-26T02:00:04.900 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.899+0000 7f84d41fb740 1 --2- 172.21.15.151:0/140145825 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5627b8156a00 0x5627b8158df0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:04.900 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.899+0000 7f84d41fb740 1 --2- 172.21.15.151:0/140145825 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5627b814c140 0x5627b81564c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:04.900 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.899+0000 7f84d41fb740 1 --2- 172.21.15.151:0/140145825 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627b814b820 0x5627b814bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:04.900 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.899+0000 7f84d41fb740 1 -- 172.21.15.151:0/140145825 >> 172.21.15.151:0/140145825 conn(0x5627b8147d30 msgr2=0x5627b8148110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:04.900 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.899+0000 7f84d41fb740 1 -- 172.21.15.151:0/140145825 shutdown_connections 2024-04-26T02:00:04.900 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.899+0000 7f84d41fb740 1 -- 172.21.15.151:0/140145825 wait complete. 2024-04-26T02:00:04.900 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.899+0000 7f84ca800640 1 --2- 172.21.15.151:0/772422046 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f84b00a3e40 0x7f84b00a6300 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x5627b814cdc0 tx=0x7f84bc005a20 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:04.903 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.900+0000 7f84d41fb740 1 -- start start 2024-04-26T02:00:04.903 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.900+0000 7f84d41fb740 1 Processor -- start 2024-04-26T02:00:04.903 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.901+0000 7f84d41fb740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5627b814b820 0x5627b8153910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:04.903 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.901+0000 7f84d41fb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5627b814c140 0x5627b8153e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:04.903 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.901+0000 7f84d41fb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627b8156a00 0x5627b8154390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:04.903 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.901+0000 7f84d41fb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5627b8159630 con 0x5627b814c140 2024-04-26T02:00:04.903 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.901+0000 7f84d41fb740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5627b806b310 con 0x5627b814b820 2024-04-26T02:00:04.903 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.901+0000 7f84d41fb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5627b81597b0 con 0x5627b8156a00 2024-04-26T02:00:04.903 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.901+0000 7f84d41fb740 10 client.0 _ll_register_callbacks cb 0x5627b8039cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:00:04.903 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.901+0000 7f84c9e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5627b814c140 0x5627b8153e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:04.904 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.901+0000 7f84ca800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5627b814b820 0x5627b8153910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:04.904 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.901+0000 7f84ca800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5627b814b820 0x5627b8153910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:44942/0 (socket says 172.21.15.151:44942) 2024-04-26T02:00:04.904 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.901+0000 7f84ca800640 1 -- 172.21.15.151:0/3072071048 learned_addr learned my addr 172.21.15.151:0/3072071048 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:00:04.904 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.901+0000 7f84cb200640 1 --2- 172.21.15.151:0/3072071048 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627b8156a00 0x5627b8154390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:04.904 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.901+0000 7f84ca800640 1 -- 172.21.15.151:0/3072071048 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627b8156a00 msgr2=0x5627b8154390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:04.904 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.901+0000 7f84ca800640 1 --2- 172.21.15.151:0/3072071048 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627b8156a00 0x5627b8154390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:04.904 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.901+0000 7f84ca800640 1 -- 172.21.15.151:0/3072071048 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5627b814c140 msgr2=0x5627b8153e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:04.904 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.901+0000 7f84ca800640 1 --2- 172.21.15.151:0/3072071048 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5627b814c140 0x5627b8153e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:04.904 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.901+0000 7f84ca800640 1 -- 172.21.15.151:0/3072071048 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627b7e90620 con 0x5627b814b820 2024-04-26T02:00:04.904 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.902+0000 7f84cb200640 1 --2- 172.21.15.151:0/3072071048 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627b8156a00 0x5627b8154390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:00:04.904 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.902+0000 7f84ca800640 1 --2- 172.21.15.151:0/3072071048 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5627b814b820 0x5627b8153910 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f84bc003870 tx=0x7f84bc009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e879aca5768858f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:04.904 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.902+0000 7f84abe00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T02:00:04.904 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.902+0000 7f84abe00640 1 -- 172.21.15.151:0/3072071048 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84bc04d070 con 0x5627b814b820 2024-04-26T02:00:04.905 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.902+0000 7f84abe00640 1 -- 172.21.15.151:0/3072071048 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8499+0+0 (secure 0 0 0) 0x7f84bc0318b0 con 0x5627b814b820 2024-04-26T02:00:04.905 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.902+0000 7f84d41fb740 1 -- 172.21.15.151:0/3072071048 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84b8005020 con 0x5627b814b820 2024-04-26T02:00:04.905 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.902+0000 7f84abe00640 1 -- 172.21.15.151:0/3072071048 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f84bc059050 con 0x5627b814b820 2024-04-26T02:00:04.905 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.902+0000 7f84d41fb740 10 client.25904 fetch_fsmap learned FSMap version 33 2024-04-26T02:00:04.905 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.902+0000 7f84d41fb740 1 -- 172.21.15.151:0/3072071048 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5627b81657c0 con 0x5627b814b820 2024-04-26T02:00:04.905 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.903+0000 7f84abe00640 1 -- 172.21.15.151:0/3072071048 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84b8005020 con 0x5627b814b820 2024-04-26T02:00:04.905 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.903+0000 7f84abe00640 1 -- 172.21.15.151:0/3072071048 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 33) ==== 8338+0+0 (secure 0 0 0) 0x7f84bc049a50 con 0x5627b814b820 2024-04-26T02:00:04.905 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.903+0000 7f84d41fb740 10 client.25904 fetch_fsmap finished waiting for FSMap version 33 2024-04-26T02:00:04.905 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.903+0000 7f84d41fb740 10 client.25904 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:00:04.905 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.903+0000 7f84d41fb740 20 client.25904 populate_metadata read hostname 'smithi151' 2024-04-26T02:00:04.905 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.903+0000 7f84d41fb740 1 --2- 172.21.15.151:0/3072071048 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5627b8166c80 0x5627b81690b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:04.905 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.903+0000 7f84d41fb740 4 client.25904 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:00:04.905 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.903+0000 7f84d41fb740 1 -- 172.21.15.151:0/3072071048 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5627b80c58b0 con 0x5627b8166c80 2024-04-26T02:00:04.905 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.903+0000 7f84c9e00640 1 --2- 172.21.15.151:0/3072071048 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5627b8166c80 0x5627b81690b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:04.906 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.904+0000 7f84c9e00640 1 --2- 172.21.15.151:0/3072071048 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5627b8166c80 0x5627b81690b0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cd30de0f51cd751 server_cookie=f91868e87ec11e5a in_seq=0 out_seq=0 2024-04-26T02:00:04.906 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.904+0000 7f84abe00640 10 client.25904 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:00:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.905+0000 7f84abe00640 1 -- 172.21.15.151:0/3072071048 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5627b80c58b0 con 0x5627b8166c80 2024-04-26T02:00:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.905+0000 7f84abe00640 10 client.25904 handle_command_reply: tid=0 2024-04-26T02:00:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.905+0000 7f84d41fb740 1 client.25904 shutdown 2024-04-26T02:00:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.905+0000 7f84d41fb740 1 -- 172.21.15.151:0/3072071048 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5627b814b820 msgr2=0x5627b8153910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.905+0000 7f84d41fb740 1 --2- 172.21.15.151:0/3072071048 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5627b814b820 0x5627b8153910 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f84bc003870 tx=0x7f84bc009260 comp rx=0 tx=0).stop 2024-04-26T02:00:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.906+0000 7f84d41fb740 1 -- 172.21.15.151:0/3072071048 shutdown_connections 2024-04-26T02:00:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.906+0000 7f84d41fb740 1 --2- 172.21.15.151:0/3072071048 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5627b8166c80 0x5627b81690b0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.906+0000 7f84abe00640 0 client.25904 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:00:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.906+0000 7f84d41fb740 1 --2- 172.21.15.151:0/3072071048 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627b8156a00 0x5627b8154390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.906+0000 7f84d41fb740 1 --2- 172.21.15.151:0/3072071048 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5627b814c140 0x5627b8153e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.906+0000 7f84d41fb740 1 --2- 172.21.15.151:0/3072071048 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5627b814b820 0x5627b8153910 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.906+0000 7f84d41fb740 1 -- 172.21.15.151:0/3072071048 >> 172.21.15.151:0/3072071048 conn(0x5627b8147d30 msgr2=0x5627b814dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.906+0000 7f84d41fb740 1 -- 172.21.15.151:0/3072071048 shutdown_connections 2024-04-26T02:00:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.906+0000 7f84d41fb740 1 -- 172.21.15.151:0/3072071048 wait complete. 2024-04-26T02:00:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.906+0000 7f84d41fb740 20 client.25904 trim_cache size 0 max 16384 2024-04-26T02:00:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.927+0000 7f84d41fb740 1 Processor -- start 2024-04-26T02:00:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.927+0000 7f84d41fb740 1 -- start start 2024-04-26T02:00:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.927+0000 7f84d41fb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5627b814b820 0x5627b81624f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.927+0000 7f84d41fb740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5627b814c140 0x5627b814e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.927+0000 7f84d41fb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627b8156a00 0x5627b8146d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.927+0000 7f84d41fb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f84cc1b6930 con 0x5627b814b820 2024-04-26T02:00:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.927+0000 7f84d41fb740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f84cc1b4200 con 0x5627b814c140 2024-04-26T02:00:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.927+0000 7f84d41fb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f84cc1b2070 con 0x5627b8156a00 2024-04-26T02:00:04.927 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.927+0000 7f84c9e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5627b814c140 0x5627b814e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:04.928 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.927+0000 7f84ca800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5627b814b820 0x5627b81624f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:04.928 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.928+0000 7f84c9e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5627b814c140 0x5627b814e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:44956/0 (socket says 172.21.15.151:44956) 2024-04-26T02:00:04.928 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.928+0000 7f84ca800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5627b814b820 0x5627b81624f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53514/0 (socket says 172.21.15.151:53514) 2024-04-26T02:00:04.928 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.928+0000 7f84ca800640 1 -- 172.21.15.151:0/1873161505 learned_addr learned my addr 172.21.15.151:0/1873161505 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:00:04.928 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.928+0000 7f84c9e00640 1 -- 172.21.15.151:0/1873161505 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627b8156a00 msgr2=0x5627b8146d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:04.928 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.928+0000 7f84c9e00640 1 --2- 172.21.15.151:0/1873161505 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627b8156a00 0x5627b8146d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:04.928 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.928+0000 7f84c9e00640 1 -- 172.21.15.151:0/1873161505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5627b814b820 msgr2=0x5627b81624f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:04.928 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.928+0000 7f84c9e00640 1 --2- 172.21.15.151:0/1873161505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5627b814b820 0x5627b81624f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:04.928 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.928+0000 7f84c9e00640 1 -- 172.21.15.151:0/1873161505 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627b8148f60 con 0x5627b814c140 2024-04-26T02:00:04.928 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.928+0000 7f84c9e00640 1 --2- 172.21.15.151:0/1873161505 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5627b814c140 0x5627b814e1e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f84b8009530 tx=0x7f84b8009c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3d1de151f525ad5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:04.928 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.928+0000 7f84b6a00640 1 -- 172.21.15.151:0/1873161505 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b800f790 con 0x5627b814c140 2024-04-26T02:00:04.928 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.928+0000 7f84b6a00640 1 -- 172.21.15.151:0/1873161505 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84b800fde0 con 0x5627b814c140 2024-04-26T02:00:04.928 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.928+0000 7f84b6a00640 1 -- 172.21.15.151:0/1873161505 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b802d020 con 0x5627b814c140 2024-04-26T02:00:04.930 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.929+0000 7f84d41fb740 1 -- 172.21.15.151:0/1873161505 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5627b814c140 msgr2=0x5627b814e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:04.930 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.929+0000 7f84d41fb740 1 --2- 172.21.15.151:0/1873161505 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5627b814c140 0x5627b814e1e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f84b8009530 tx=0x7f84b8009c70 comp rx=0 tx=0).stop 2024-04-26T02:00:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.929+0000 7f84d41fb740 1 -- 172.21.15.151:0/1873161505 shutdown_connections 2024-04-26T02:00:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.929+0000 7f84d41fb740 1 --2- 172.21.15.151:0/1873161505 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627b8156a00 0x5627b8146d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.929+0000 7f84d41fb740 1 --2- 172.21.15.151:0/1873161505 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5627b814c140 0x5627b814e1e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.929+0000 7f84d41fb740 1 --2- 172.21.15.151:0/1873161505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5627b814b820 0x5627b81624f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.929+0000 7f84d41fb740 1 -- 172.21.15.151:0/1873161505 >> 172.21.15.151:0/1873161505 conn(0x5627b8147d30 msgr2=0x5627b8146600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.930+0000 7f84d41fb740 1 -- 172.21.15.151:0/1873161505 shutdown_connections 2024-04-26T02:00:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.930+0000 7f84d41fb740 1 -- 172.21.15.151:0/1873161505 wait complete. 2024-04-26T02:00:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.930+0000 7f84d41fb740 1 -- start start 2024-04-26T02:00:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.930+0000 7f84d41fb740 1 Processor -- start 2024-04-26T02:00:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.931+0000 7f84d41fb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5627b814b820 0x5627b815ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.931+0000 7f84d41fb740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5627b814c140 0x5627b815c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.931+0000 7f84d41fb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627b8156a00 0x5627b815ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.931+0000 7f84d41fb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f84cc1b26c0 con 0x5627b814b820 2024-04-26T02:00:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.931+0000 7f84d41fb740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f84cc1b3b30 con 0x5627b814c140 2024-04-26T02:00:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.931+0000 7f84d41fb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f84cc1b2d90 con 0x5627b8156a00 2024-04-26T02:00:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.931+0000 7f84d41fb740 10 client.0 _ll_register_callbacks cb 0x7f84bc004430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:00:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.931+0000 7f84c9e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5627b814c140 0x5627b815c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.931+0000 7f84c9e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5627b814c140 0x5627b815c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:44972/0 (socket says 172.21.15.151:44972) 2024-04-26T02:00:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.931+0000 7f84c9e00640 1 -- 172.21.15.151:0/2522817643 learned_addr learned my addr 172.21.15.151:0/2522817643 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:00:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.931+0000 7f84ca800640 1 --2- 172.21.15.151:0/2522817643 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5627b814b820 0x5627b815ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.931+0000 7f84cb200640 1 --2- 172.21.15.151:0/2522817643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627b8156a00 0x5627b815ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.932+0000 7f84c9e00640 1 -- 172.21.15.151:0/2522817643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627b8156a00 msgr2=0x5627b815ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.932+0000 7f84c9e00640 1 --2- 172.21.15.151:0/2522817643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627b8156a00 0x5627b815ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.932+0000 7f84c9e00640 1 -- 172.21.15.151:0/2522817643 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5627b814b820 msgr2=0x5627b815ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.932+0000 7f84c9e00640 1 --2- 172.21.15.151:0/2522817643 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5627b814b820 0x5627b815ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.932+0000 7f84c9e00640 1 -- 172.21.15.151:0/2522817643 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627b81cc240 con 0x5627b814c140 2024-04-26T02:00:04.933 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.932+0000 7f84c9e00640 1 --2- 172.21.15.151:0/2522817643 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5627b814c140 0x5627b815c300 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f84b8009560 tx=0x7f84b8009ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee14e8c1852a12c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:04.933 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.932+0000 7f84abe00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T02:00:04.934 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.932+0000 7f84abe00640 1 -- 172.21.15.151:0/2522817643 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b8004dd0 con 0x5627b814c140 2024-04-26T02:00:04.934 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.932+0000 7f84abe00640 1 -- 172.21.15.151:0/2522817643 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8499+0+0 (secure 0 0 0) 0x7f84b8004f70 con 0x5627b814c140 2024-04-26T02:00:04.934 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.932+0000 7f84d41fb740 1 -- 172.21.15.151:0/2522817643 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84b800fde0 con 0x5627b814c140 2024-04-26T02:00:04.934 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.933+0000 7f84abe00640 1 -- 172.21.15.151:0/2522817643 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f84b8044050 con 0x5627b814c140 2024-04-26T02:00:04.934 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.933+0000 7f84d41fb740 10 client.25910 fetch_fsmap learned FSMap version 33 2024-04-26T02:00:04.934 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.933+0000 7f84d41fb740 1 -- 172.21.15.151:0/2522817643 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5627b81cd3b0 con 0x5627b814c140 2024-04-26T02:00:04.934 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.933+0000 7f84abe00640 1 -- 172.21.15.151:0/2522817643 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84b800fde0 con 0x5627b814c140 2024-04-26T02:00:04.934 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.933+0000 7f84abe00640 1 -- 172.21.15.151:0/2522817643 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 33) ==== 8338+0+0 (secure 0 0 0) 0x7f84b803f030 con 0x5627b814c140 2024-04-26T02:00:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.933+0000 7f84d41fb740 10 client.25910 fetch_fsmap finished waiting for FSMap version 33 2024-04-26T02:00:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.933+0000 7f84d41fb740 10 client.25910 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:00:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.933+0000 7f84d41fb740 20 client.25910 populate_metadata read hostname 'smithi151' 2024-04-26T02:00:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.933+0000 7f84d41fb740 1 --2- 172.21.15.151:0/2522817643 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5627b81cd930 0x5627b81cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.933+0000 7f84d41fb740 4 client.25910 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T02:00:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.933+0000 7f84d41fb740 1 -- 172.21.15.151:0/2522817643 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5627b80bf320 con 0x5627b81cd930 2024-04-26T02:00:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.934+0000 7f84ca800640 1 --2- 172.21.15.151:0/2522817643 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5627b81cd930 0x5627b81cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.934+0000 7f84ca800640 1 --2- 172.21.15.151:0/2522817643 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5627b81cd930 0x5627b81cfd50 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=319dabaf066ee3c7 server_cookie=43ab230f226a1b58 in_seq=0 out_seq=0 2024-04-26T02:00:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.934+0000 7f84abe00640 10 client.25910 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:00:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.950+0000 7f84abe00640 1 -- 172.21.15.151:0/2522817643 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5627b80bf320 con 0x5627b81cd930 2024-04-26T02:00:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.950+0000 7f84abe00640 10 client.25910 handle_command_reply: tid=0 2024-04-26T02:00:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.950+0000 7f84d41fb740 1 client.25910 shutdown 2024-04-26T02:00:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.950+0000 7f84d41fb740 1 -- 172.21.15.151:0/2522817643 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5627b814c140 msgr2=0x5627b815c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.950+0000 7f84d41fb740 1 --2- 172.21.15.151:0/2522817643 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5627b814c140 0x5627b815c300 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f84b8009560 tx=0x7f84b8009ce0 comp rx=0 tx=0).stop 2024-04-26T02:00:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.950+0000 7f84d41fb740 1 -- 172.21.15.151:0/2522817643 shutdown_connections 2024-04-26T02:00:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.950+0000 7f84d41fb740 1 --2- 172.21.15.151:0/2522817643 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5627b81cd930 0x5627b81cfd50 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.950+0000 7f84d41fb740 1 --2- 172.21.15.151:0/2522817643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627b8156a00 0x5627b815ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.950+0000 7f84d41fb740 1 --2- 172.21.15.151:0/2522817643 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5627b814c140 0x5627b815c300 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.950+0000 7f84abe00640 0 client.25910 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:00:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.950+0000 7f84d41fb740 1 --2- 172.21.15.151:0/2522817643 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5627b814b820 0x5627b815ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.950+0000 7f84d41fb740 1 -- 172.21.15.151:0/2522817643 >> 172.21.15.151:0/2522817643 conn(0x5627b8147d30 msgr2=0x5627b8146e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.950+0000 7f84d41fb740 1 -- 172.21.15.151:0/2522817643 shutdown_connections 2024-04-26T02:00:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.950+0000 7f84d41fb740 1 -- 172.21.15.151:0/2522817643 wait complete. 2024-04-26T02:00:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.951+0000 7f84d41fb740 20 client.25910 trim_cache size 0 max 16384 2024-04-26T02:00:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:{ 2024-04-26T02:00:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "message": "", 2024-04-26T02:00:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "return_code": 0 2024-04-26T02:00:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:} 2024-04-26T02:00:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.951+0000 7f84d1400640 1 -- 172.21.15.151:0/772422046 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f84b00a3e40 msgr2=0x7f84b00a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.951+0000 7f84d1400640 1 --2- 172.21.15.151:0/772422046 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f84b00a3e40 0x7f84b00a6300 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x5627b814cdc0 tx=0x7f84bc005a20 comp rx=0 tx=0).stop 2024-04-26T02:00:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.951+0000 7f84d1400640 1 -- 172.21.15.151:0/772422046 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84cc112f40 msgr2=0x7f84cc19eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.951+0000 7f84d1400640 1 --2- 172.21.15.151:0/772422046 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84cc112f40 0x7f84cc19eaf0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f84c4005fd0 tx=0x7f84c400ef10 comp rx=0 tx=0).stop 2024-04-26T02:00:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.951+0000 7f84d1400640 1 -- 172.21.15.151:0/772422046 shutdown_connections 2024-04-26T02:00:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.951+0000 7f84d1400640 1 --2- 172.21.15.151:0/772422046 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f84b00a3e40 0x7f84b00a6300 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.951+0000 7f84d1400640 1 --2- 172.21.15.151:0/772422046 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84cc112f40 0x7f84cc19eaf0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.951+0000 7f84d1400640 1 --2- 172.21.15.151:0/772422046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84cc074c10 0x7f84cc1a4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.951+0000 7f84d1400640 1 --2- 172.21.15.151:0/772422046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f84cc0742f0 0x7f84cc1a4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.951+0000 7f84d1400640 1 -- 172.21.15.151:0/772422046 >> 172.21.15.151:0/772422046 conn(0x7f84cc0fda00 msgr2=0x7f84cc0ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.951+0000 7f84d1400640 1 -- 172.21.15.151:0/772422046 shutdown_connections 2024-04-26T02:00:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:04.952+0000 7f84d1400640 1 -- 172.21.15.151:0/772422046 wait complete. 2024-04-26T02:00:05.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:00:05 smithi172 ceph-mon[32785]: daemon mds.f restarted 2024-04-26T02:00:05.787 INFO:journalctl@ceph.mds.f.smithi172.stdout:Apr 26 02:00:05 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-f[63189]: ignoring --setuser ceph since I am not root 2024-04-26T02:00:05.787 INFO:journalctl@ceph.mds.f.smithi172.stdout:Apr 26 02:00:05 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-f[63189]: ignoring --setgroup ceph since I am not root 2024-04-26T02:00:05.787 INFO:journalctl@ceph.mds.f.smithi172.stdout:Apr 26 02:00:05 smithi172 ceph-mds[63193]: -- 172.21.15.172:0/2037813860 <== mon.1 v2:172.21.15.172:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c145441a40 con 0x55c145414c00 2024-04-26T02:00:05.787 INFO:journalctl@ceph.mds.f.smithi172.stdout:Apr 26 02:00:05 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-f[63189]: starting mds.f at 2024-04-26T02:00:05.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:00:05 smithi165 ceph-mon[32648]: daemon mds.f restarted 2024-04-26T02:00:05.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:00:05 smithi151 ceph-mon[28057]: daemon mds.f restarted 2024-04-26T02:00:06.688 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:00:06 smithi172 ceph-mon[32785]: pgmap v200: 97 pgs: 97 active+clean; 53 MiB data, 971 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 425 KiB/s wr, 85 op/s 2024-04-26T02:00:06.688 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:00:06 smithi172 ceph-mon[32785]: osdmap e97: 12 total, 12 up, 12 in 2024-04-26T02:00:06.688 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:00:06 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T02:00:06.688 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:00:06 smithi172 ceph-mon[32785]: mds.? [v2:172.21.15.172:6834/3656630171,v1:172.21.15.172:6835/3656630171] up:boot 2024-04-26T02:00:06.688 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:00:06 smithi172 ceph-mon[32785]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T02:00:06.688 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:00:06 smithi172 ceph-mon[32785]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T02:00:06.746 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:00:06 smithi165 ceph-mon[32648]: pgmap v200: 97 pgs: 97 active+clean; 53 MiB data, 971 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 425 KiB/s wr, 85 op/s 2024-04-26T02:00:06.747 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:00:06 smithi165 ceph-mon[32648]: osdmap e97: 12 total, 12 up, 12 in 2024-04-26T02:00:06.747 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:00:06 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T02:00:06.747 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:00:06 smithi165 ceph-mon[32648]: mds.? [v2:172.21.15.172:6834/3656630171,v1:172.21.15.172:6835/3656630171] up:boot 2024-04-26T02:00:06.747 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:00:06 smithi165 ceph-mon[32648]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T02:00:06.747 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:00:06 smithi165 ceph-mon[32648]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T02:00:06.873 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:00:06 smithi151 ceph-mon[28057]: pgmap v200: 97 pgs: 97 active+clean; 53 MiB data, 971 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 425 KiB/s wr, 85 op/s 2024-04-26T02:00:06.873 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:00:06 smithi151 ceph-mon[28057]: osdmap e97: 12 total, 12 up, 12 in 2024-04-26T02:00:06.873 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:00:06 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T02:00:06.873 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:00:06 smithi151 ceph-mon[28057]: mds.? [v2:172.21.15.172:6834/3656630171,v1:172.21.15.172:6835/3656630171] up:boot 2024-04-26T02:00:06.873 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:00:06 smithi151 ceph-mon[28057]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T02:00:06.873 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:00:06 smithi151 ceph-mon[28057]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T02:00:08.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:00:08 smithi172 ceph-mon[32785]: pgmap v202: 97 pgs: 97 active+clean; 53 MiB data, 971 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 494 KiB/s wr, 102 op/s 2024-04-26T02:00:08.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:00:08 smithi165 ceph-mon[32648]: pgmap v202: 97 pgs: 97 active+clean; 53 MiB data, 971 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 494 KiB/s wr, 102 op/s 2024-04-26T02:00:08.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:00:08 smithi151 ceph-mon[28057]: pgmap v202: 97 pgs: 97 active+clean; 53 MiB data, 971 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 494 KiB/s wr, 102 op/s 2024-04-26T02:00:10.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:00:10 smithi172 ceph-mon[32785]: pgmap v203: 97 pgs: 97 active+clean; 53 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.7 MiB/s wr, 289 op/s 2024-04-26T02:00:10.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:00:10 smithi165 ceph-mon[32648]: pgmap v203: 97 pgs: 97 active+clean; 53 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.7 MiB/s wr, 289 op/s 2024-04-26T02:00:10.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:00:10 smithi151 ceph-mon[28057]: pgmap v203: 97 pgs: 97 active+clean; 53 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.7 MiB/s wr, 289 op/s 2024-04-26T02:00:11.968 INFO:tasks.workunit.client.0.smithi151.stdout:truncating to largest ever: 0x9ffb01 2024-04-26T02:00:12.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:00:12 smithi172 ceph-mon[32785]: pgmap v204: 97 pgs: 97 active+clean; 53 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 1.6 MiB/s wr, 262 op/s 2024-04-26T02:00:12.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:00:12 smithi165 ceph-mon[32648]: pgmap v204: 97 pgs: 97 active+clean; 53 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 1.6 MiB/s wr, 262 op/s 2024-04-26T02:00:12.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:00:12 smithi151 ceph-mon[28057]: pgmap v204: 97 pgs: 97 active+clean; 53 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 1.6 MiB/s wr, 262 op/s 2024-04-26T02:00:14.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:00:14 smithi172 ceph-mon[32785]: pgmap v205: 97 pgs: 97 active+clean; 55 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.3 MiB/s wr, 204 op/s 2024-04-26T02:00:14.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:00:14 smithi172 ceph-mon[32785]: osdmap e98: 12 total, 12 up, 12 in 2024-04-26T02:00:14.807 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:00:14 smithi151 ceph-mon[28057]: pgmap v205: 97 pgs: 97 active+clean; 55 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.3 MiB/s wr, 204 op/s 2024-04-26T02:00:14.807 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:00:14 smithi151 ceph-mon[28057]: osdmap e98: 12 total, 12 up, 12 in 2024-04-26T02:00:14.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:00:14 smithi165 ceph-mon[32648]: pgmap v205: 97 pgs: 97 active+clean; 55 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.3 MiB/s wr, 204 op/s 2024-04-26T02:00:14.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:00:14 smithi165 ceph-mon[32648]: osdmap e98: 12 total, 12 up, 12 in 2024-04-26T02:00:16.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:00:16 smithi172 ceph-mon[32785]: pgmap v207: 97 pgs: 97 active+clean; 52 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 2.0 MiB/s wr, 307 op/s 2024-04-26T02:00:16.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:00:16 smithi165 ceph-mon[32648]: pgmap v207: 97 pgs: 97 active+clean; 52 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 2.0 MiB/s wr, 307 op/s 2024-04-26T02:00:16.873 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:00:16 smithi151 ceph-mon[28057]: pgmap v207: 97 pgs: 97 active+clean; 52 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 2.0 MiB/s wr, 307 op/s 2024-04-26T02:00:18.104 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.105+0000 7eff5d400640 1 -- 172.21.15.151:0/150799089 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7eff58103850 msgr2=0x7eff5810bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:18.105 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.105+0000 7eff5d400640 1 --2- 172.21.15.151:0/150799089 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7eff58103850 0x7eff5810bd30 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7eff40009860 tx=0x7eff4002a340 comp rx=0 tx=0).stop 2024-04-26T02:00:18.105 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.105+0000 7eff5d400640 1 -- 172.21.15.151:0/150799089 shutdown_connections 2024-04-26T02:00:18.105 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.105+0000 7eff5d400640 1 --2- 172.21.15.151:0/150799089 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff5810c3a0 0x7eff5810e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:18.105 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.105+0000 7eff5d400640 1 --2- 172.21.15.151:0/150799089 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7eff58103850 0x7eff5810bd30 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:18.105 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.105+0000 7eff5d400640 1 --2- 172.21.15.151:0/150799089 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff58102f30 0x7eff58103310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:18.105 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.105+0000 7eff5d400640 1 -- 172.21.15.151:0/150799089 >> 172.21.15.151:0/150799089 conn(0x7eff580fc770 msgr2=0x7eff580feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:18.105 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.105+0000 7eff5d400640 1 -- 172.21.15.151:0/150799089 shutdown_connections 2024-04-26T02:00:18.105 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.106+0000 7eff5d400640 1 -- 172.21.15.151:0/150799089 wait complete. 2024-04-26T02:00:18.105 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.107+0000 7eff5d400640 1 Processor -- start 2024-04-26T02:00:18.106 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.107+0000 7eff5d400640 1 -- start start 2024-04-26T02:00:18.107 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.108+0000 7eff5d400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff58102f30 0x7eff58198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:18.108 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.108+0000 7eff5d400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff58103850 0x7eff58198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:18.108 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.108+0000 7eff5d400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7eff5810c3a0 0x7eff5819ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:18.108 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.108+0000 7eff5d400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7eff58111fd0 con 0x7eff58102f30 2024-04-26T02:00:18.108 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.108+0000 7eff5d400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7eff58111e50 con 0x7eff5810c3a0 2024-04-26T02:00:18.108 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.108+0000 7eff5d400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7eff58112150 con 0x7eff58103850 2024-04-26T02:00:18.108 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.108+0000 7eff57e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff58102f30 0x7eff58198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:18.108 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.108+0000 7eff57e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff58102f30 0x7eff58198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41174/0 (socket says 172.21.15.151:41174) 2024-04-26T02:00:18.108 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.108+0000 7eff57e00640 1 -- 172.21.15.151:0/1834377134 learned_addr learned my addr 172.21.15.151:0/1834377134 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:00:18.108 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.108+0000 7eff5ca00640 1 --2- 172.21.15.151:0/1834377134 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7eff5810c3a0 0x7eff5819ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:18.108 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.108+0000 7eff4f400640 1 --2- 172.21.15.151:0/1834377134 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff58103850 0x7eff58198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:18.108 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.109+0000 7eff5ca00640 1 -- 172.21.15.151:0/1834377134 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff58103850 msgr2=0x7eff58198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:18.108 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.109+0000 7eff5ca00640 1 --2- 172.21.15.151:0/1834377134 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff58103850 0x7eff58198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:18.108 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.109+0000 7eff5ca00640 1 -- 172.21.15.151:0/1834377134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff58102f30 msgr2=0x7eff58198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:18.109 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.109+0000 7eff5ca00640 1 --2- 172.21.15.151:0/1834377134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff58102f30 0x7eff58198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:18.109 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.109+0000 7eff5ca00640 1 -- 172.21.15.151:0/1834377134 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff5819d580 con 0x7eff5810c3a0 2024-04-26T02:00:18.109 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.109+0000 7eff57e00640 1 --2- 172.21.15.151:0/1834377134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff58102f30 0x7eff58198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:00:18.109 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.109+0000 7eff5ca00640 1 --2- 172.21.15.151:0/1834377134 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7eff5810c3a0 0x7eff5819ce50 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7eff5000e970 tx=0x7eff5000ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:18.109 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.109+0000 7eff55600640 1 -- 172.21.15.151:0/1834377134 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5000ccc0 con 0x7eff5810c3a0 2024-04-26T02:00:18.109 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.109+0000 7eff55600640 1 -- 172.21.15.151:0/1834377134 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff50005020 con 0x7eff5810c3a0 2024-04-26T02:00:18.110 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.109+0000 7eff5d400640 1 -- 172.21.15.151:0/1834377134 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff5819d870 con 0x7eff5810c3a0 2024-04-26T02:00:18.110 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.109+0000 7eff55600640 1 -- 172.21.15.151:0/1834377134 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff50004a00 con 0x7eff5810c3a0 2024-04-26T02:00:18.110 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.109+0000 7eff5d400640 1 -- 172.21.15.151:0/1834377134 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff581a5100 con 0x7eff5810c3a0 2024-04-26T02:00:18.112 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.113+0000 7eff55600640 1 -- 172.21.15.151:0/1834377134 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7eff50004ba0 con 0x7eff5810c3a0 2024-04-26T02:00:18.112 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.113+0000 7eff6178f740 1 Processor -- start 2024-04-26T02:00:18.112 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.113+0000 7eff6178f740 1 -- start start 2024-04-26T02:00:18.114 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.114+0000 7eff6178f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee459929b0 0x55ee45992d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:18.114 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.114+0000 7eff6178f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee459932d0 0x55ee4599d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.114+0000 7eff6178f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee4599db90 0x55ee4599ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.114+0000 7eff6178f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ee458f8900 con 0x55ee459932d0 2024-04-26T02:00:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.114+0000 7eff6178f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ee4589eef0 con 0x55ee4599db90 2024-04-26T02:00:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.114+0000 7eff6178f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ee458f48c0 con 0x55ee459929b0 2024-04-26T02:00:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.114+0000 7eff4f400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee459929b0 0x55ee45992d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.114+0000 7eff57e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee459932d0 0x55ee4599d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.114+0000 7eff4f400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee459929b0 0x55ee45992d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:48436/0 (socket says 172.21.15.151:48436) 2024-04-26T02:00:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.114+0000 7eff4f400640 1 -- 172.21.15.151:0/2526837121 learned_addr learned my addr 172.21.15.151:0/2526837121 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:00:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.114+0000 7eff4f400640 1 --2- 172.21.15.151:0/2526837121 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee4599db90 0x55ee4599ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.114+0000 7eff4f400640 1 -- 172.21.15.151:0/2526837121 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee4599db90 msgr2=0x55ee4599ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.114+0000 7eff4f400640 1 --2- 172.21.15.151:0/2526837121 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee4599db90 0x55ee4599ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.114+0000 7eff4f400640 1 -- 172.21.15.151:0/2526837121 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee459932d0 msgr2=0x55ee4599d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.114+0000 7eff4f400640 1 --2- 172.21.15.151:0/2526837121 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee459932d0 0x55ee4599d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.114+0000 7eff4f400640 1 -- 172.21.15.151:0/2526837121 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee4560b210 con 0x55ee459929b0 2024-04-26T02:00:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.115+0000 7eff57e00640 1 --2- 172.21.15.151:0/2526837121 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee459932d0 0x55ee4599d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:00:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.115+0000 7eff4f400640 1 --2- 172.21.15.151:0/2526837121 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee459929b0 0x55ee45992d90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7eff40005b60 tx=0x7eff40000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de35146264c9c064 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.115+0000 7eff55600640 1 --2- 172.21.15.151:0/1834377134 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7eff3c0a3d90 0x7eff3c0a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.115+0000 7eff4d200640 1 -- 172.21.15.151:0/2526837121 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff40037040 con 0x55ee459929b0 2024-04-26T02:00:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.115+0000 7eff4d200640 1 -- 172.21.15.151:0/2526837121 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff40030900 con 0x55ee459929b0 2024-04-26T02:00:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.115+0000 7eff4d200640 1 -- 172.21.15.151:0/2526837121 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff40003b80 con 0x55ee459929b0 2024-04-26T02:00:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.115+0000 7eff55600640 1 -- 172.21.15.151:0/1834377134 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8455+0+0 (secure 0 0 0) 0x7eff500d0680 con 0x7eff5810c3a0 2024-04-26T02:00:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.116+0000 7eff57e00640 1 --2- 172.21.15.151:0/1834377134 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7eff3c0a3d90 0x7eff3c0a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.116+0000 7eff6178f740 1 -- 172.21.15.151:0/2526837121 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee459929b0 msgr2=0x55ee45992d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.116+0000 7eff57e00640 1 --2- 172.21.15.151:0/1834377134 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7eff3c0a3d90 0x7eff3c0a6250 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x55ee45993f50 tx=0x7eff480095c0 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.116+0000 7eff6178f740 1 --2- 172.21.15.151:0/2526837121 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee459929b0 0x55ee45992d90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7eff40005b60 tx=0x7eff40000bc0 comp rx=0 tx=0).stop 2024-04-26T02:00:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.116+0000 7eff6178f740 1 -- 172.21.15.151:0/2526837121 shutdown_connections 2024-04-26T02:00:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.116+0000 7eff6178f740 1 --2- 172.21.15.151:0/2526837121 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee4599db90 0x55ee4599ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.116+0000 7eff6178f740 1 --2- 172.21.15.151:0/2526837121 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee459932d0 0x55ee4599d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.116+0000 7eff6178f740 1 --2- 172.21.15.151:0/2526837121 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee459929b0 0x55ee45992d90 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.116+0000 7eff6178f740 1 -- 172.21.15.151:0/2526837121 >> 172.21.15.151:0/2526837121 conn(0x55ee4598eec0 msgr2=0x55ee4598f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.116+0000 7eff6178f740 1 -- 172.21.15.151:0/2526837121 shutdown_connections 2024-04-26T02:00:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.117+0000 7eff6178f740 1 -- 172.21.15.151:0/2526837121 wait complete. 2024-04-26T02:00:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.117+0000 7eff6178f740 1 -- start start 2024-04-26T02:00:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.117+0000 7eff6178f740 1 Processor -- start 2024-04-26T02:00:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.118+0000 7eff6178f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee459929b0 0x55ee4599abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.118+0000 7eff6178f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee459932d0 0x55ee4599b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.118+0000 7eff6178f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee4599db90 0x55ee4599b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.118+0000 7eff6178f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ee459a0760 con 0x55ee4599db90 2024-04-26T02:00:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.118+0000 7eff6178f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ee458b2310 con 0x55ee459929b0 2024-04-26T02:00:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.118+0000 7eff6178f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ee459a08e0 con 0x55ee459932d0 2024-04-26T02:00:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.118+0000 7eff6178f740 10 client.0 _ll_register_callbacks cb 0x55ee45880cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:00:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.118+0000 7eff57e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee459929b0 0x55ee4599abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.118+0000 7eff57e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee459929b0 0x55ee4599abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:53404/0 (socket says 172.21.15.151:53404) 2024-04-26T02:00:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.118+0000 7eff57e00640 1 -- 172.21.15.151:0/3513104192 learned_addr learned my addr 172.21.15.151:0/3513104192 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:00:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.118+0000 7eff5ca00640 1 --2- 172.21.15.151:0/3513104192 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee4599db90 0x55ee4599b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.118+0000 7eff4f400640 1 --2- 172.21.15.151:0/3513104192 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee459932d0 0x55ee4599b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.119+0000 7eff57e00640 1 -- 172.21.15.151:0/3513104192 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee459932d0 msgr2=0x55ee4599b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.119+0000 7eff57e00640 1 --2- 172.21.15.151:0/3513104192 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee459932d0 0x55ee4599b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.119+0000 7eff57e00640 1 -- 172.21.15.151:0/3513104192 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee4599db90 msgr2=0x55ee4599b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.119+0000 7eff57e00640 1 --2- 172.21.15.151:0/3513104192 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee4599db90 0x55ee4599b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.119+0000 7eff57e00640 1 -- 172.21.15.151:0/3513104192 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee456d7620 con 0x55ee459929b0 2024-04-26T02:00:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.119+0000 7eff4f400640 1 --2- 172.21.15.151:0/3513104192 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee459932d0 0x55ee4599b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T02:00:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.119+0000 7eff57e00640 1 --2- 172.21.15.151:0/3513104192 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee459929b0 0x55ee4599abf0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7eff4801b6b0 tx=0x7eff48004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d0d5b917bb9b513 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.119+0000 7eff45600640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T02:00:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.119+0000 7eff45600640 1 -- 172.21.15.151:0/3513104192 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff48029020 con 0x55ee459929b0 2024-04-26T02:00:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.119+0000 7eff45600640 1 -- 172.21.15.151:0/3513104192 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8455+0+0 (secure 0 0 0) 0x7eff480255f0 con 0x55ee459929b0 2024-04-26T02:00:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.119+0000 7eff6178f740 1 -- 172.21.15.151:0/3513104192 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff40030900 con 0x55ee459929b0 2024-04-26T02:00:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.120+0000 7eff45600640 1 -- 172.21.15.151:0/3513104192 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7eff48034050 con 0x55ee459929b0 2024-04-26T02:00:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.120+0000 7eff6178f740 10 client.25928 fetch_fsmap learned FSMap version 35 2024-04-26T02:00:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.120+0000 7eff6178f740 1 -- 172.21.15.151:0/3513104192 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ee459acb00 con 0x55ee459929b0 2024-04-26T02:00:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.120+0000 7eff45600640 1 -- 172.21.15.151:0/3513104192 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff40030900 con 0x55ee459929b0 2024-04-26T02:00:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.120+0000 7eff45600640 1 -- 172.21.15.151:0/3513104192 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 35) ==== 8338+0+0 (secure 0 0 0) 0x7eff48024030 con 0x55ee459929b0 2024-04-26T02:00:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.120+0000 7eff6178f740 10 client.25928 fetch_fsmap finished waiting for FSMap version 35 2024-04-26T02:00:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.120+0000 7eff6178f740 10 client.25928 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:00:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.120+0000 7eff6178f740 20 client.25928 populate_metadata read hostname 'smithi151' 2024-04-26T02:00:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.120+0000 7eff6178f740 1 --2- 172.21.15.151:0/3513104192 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55ee459ae060 0x55ee459b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.120+0000 7eff6178f740 4 client.25928 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:00:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.120+0000 7eff6178f740 1 -- 172.21.15.151:0/3513104192 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee4590c8b0 con 0x55ee459ae060 2024-04-26T02:00:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.121+0000 7eff4f400640 1 --2- 172.21.15.151:0/3513104192 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55ee459ae060 0x55ee459b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.122+0000 7eff4f400640 1 --2- 172.21.15.151:0/3513104192 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55ee459ae060 0x55ee459b04d0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=929700f9252814e2 server_cookie=bbe899bd57eb3737 in_seq=0 out_seq=0 2024-04-26T02:00:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.122+0000 7eff45600640 10 client.25928 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:00:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.122+0000 7eff45600640 1 -- 172.21.15.151:0/3513104192 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ee4590c8b0 con 0x55ee459ae060 2024-04-26T02:00:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.122+0000 7eff45600640 10 client.25928 handle_command_reply: tid=0 2024-04-26T02:00:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.123+0000 7eff6178f740 1 client.25928 shutdown 2024-04-26T02:00:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.123+0000 7eff6178f740 1 -- 172.21.15.151:0/3513104192 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee459929b0 msgr2=0x55ee4599abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.123+0000 7eff6178f740 1 --2- 172.21.15.151:0/3513104192 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee459929b0 0x55ee4599abf0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7eff4801b6b0 tx=0x7eff48004ab0 comp rx=0 tx=0).stop 2024-04-26T02:00:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.123+0000 7eff6178f740 1 -- 172.21.15.151:0/3513104192 shutdown_connections 2024-04-26T02:00:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.123+0000 7eff6178f740 1 --2- 172.21.15.151:0/3513104192 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55ee459ae060 0x55ee459b04d0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.123+0000 7eff6178f740 1 --2- 172.21.15.151:0/3513104192 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee4599db90 0x55ee4599b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.123+0000 7eff6178f740 1 --2- 172.21.15.151:0/3513104192 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee459932d0 0x55ee4599b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.123+0000 7eff45600640 0 client.25928 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:00:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.123+0000 7eff6178f740 1 --2- 172.21.15.151:0/3513104192 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee459929b0 0x55ee4599abf0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.123+0000 7eff6178f740 1 -- 172.21.15.151:0/3513104192 >> 172.21.15.151:0/3513104192 conn(0x55ee4598eec0 msgr2=0x55ee45994c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.123+0000 7eff6178f740 1 -- 172.21.15.151:0/3513104192 shutdown_connections 2024-04-26T02:00:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.123+0000 7eff6178f740 1 -- 172.21.15.151:0/3513104192 wait complete. 2024-04-26T02:00:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.123+0000 7eff6178f740 20 client.25928 trim_cache size 0 max 16384 2024-04-26T02:00:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.142+0000 7eff6178f740 1 Processor -- start 2024-04-26T02:00:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.142+0000 7eff6178f740 1 -- start start 2024-04-26T02:00:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.142+0000 7eff6178f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee459929b0 0x55ee459a9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.143+0000 7eff6178f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee459932d0 0x55ee4599b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.143+0000 7eff6178f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee4599db90 0x55ee459b5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.143+0000 7eff6178f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7eff581b01c0 con 0x55ee459929b0 2024-04-26T02:00:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.143+0000 7eff6178f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7eff581ada90 con 0x55ee4599db90 2024-04-26T02:00:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.143+0000 7eff6178f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7eff581ab900 con 0x55ee459932d0 2024-04-26T02:00:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.143+0000 7eff4f400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee459932d0 0x55ee4599b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.143+0000 7eff4f400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee459932d0 0x55ee4599b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:48456/0 (socket says 172.21.15.151:48456) 2024-04-26T02:00:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.143+0000 7eff4f400640 1 -- 172.21.15.151:0/3412831119 learned_addr learned my addr 172.21.15.151:0/3412831119 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:00:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.143+0000 7eff57e00640 1 --2- 172.21.15.151:0/3412831119 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee459929b0 0x55ee459a9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.143+0000 7eff5ca00640 1 --2- 172.21.15.151:0/3412831119 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee4599db90 0x55ee459b5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.143+0000 7eff4f400640 1 -- 172.21.15.151:0/3412831119 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee4599db90 msgr2=0x55ee459b5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.143+0000 7eff4f400640 1 --2- 172.21.15.151:0/3412831119 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee4599db90 0x55ee459b5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.143+0000 7eff4f400640 1 -- 172.21.15.151:0/3412831119 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee459929b0 msgr2=0x55ee459a9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.143+0000 7eff4f400640 1 --2- 172.21.15.151:0/3412831119 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee459929b0 0x55ee459a9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.143+0000 7eff4f400640 1 -- 172.21.15.151:0/3412831119 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee45989490 con 0x55ee459932d0 2024-04-26T02:00:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.143+0000 7eff57e00640 1 --2- 172.21.15.151:0/3412831119 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee459929b0 0x55ee459a9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:00:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.144+0000 7eff4f400640 1 --2- 172.21.15.151:0/3412831119 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee459932d0 0x55ee4599b2c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7eff40009230 tx=0x7eff40030c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0d3ac16ce42fac1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.144+0000 7eff47e00640 1 -- 172.21.15.151:0/3412831119 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff40047040 con 0x55ee459932d0 2024-04-26T02:00:18.146 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.144+0000 7eff47e00640 1 -- 172.21.15.151:0/3412831119 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff40030450 con 0x55ee459932d0 2024-04-26T02:00:18.146 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.144+0000 7eff47e00640 1 -- 172.21.15.151:0/3412831119 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff400038c0 con 0x55ee459932d0 2024-04-26T02:00:18.146 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.145+0000 7eff6178f740 1 -- 172.21.15.151:0/3412831119 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee459932d0 msgr2=0x55ee4599b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:18.146 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.145+0000 7eff6178f740 1 --2- 172.21.15.151:0/3412831119 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee459932d0 0x55ee4599b2c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7eff40009230 tx=0x7eff40030c80 comp rx=0 tx=0).stop 2024-04-26T02:00:18.146 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.145+0000 7eff6178f740 1 -- 172.21.15.151:0/3412831119 shutdown_connections 2024-04-26T02:00:18.146 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.145+0000 7eff6178f740 1 --2- 172.21.15.151:0/3412831119 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee4599db90 0x55ee459b5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:18.146 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.145+0000 7eff6178f740 1 --2- 172.21.15.151:0/3412831119 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee459932d0 0x55ee4599b2c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:18.146 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.145+0000 7eff6178f740 1 --2- 172.21.15.151:0/3412831119 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee459929b0 0x55ee459a9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:18.147 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.145+0000 7eff6178f740 1 -- 172.21.15.151:0/3412831119 >> 172.21.15.151:0/3412831119 conn(0x55ee4598eec0 msgr2=0x55ee4599abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:18.147 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.145+0000 7eff6178f740 1 -- 172.21.15.151:0/3412831119 shutdown_connections 2024-04-26T02:00:18.147 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.145+0000 7eff6178f740 1 -- 172.21.15.151:0/3412831119 wait complete. 2024-04-26T02:00:18.147 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.145+0000 7eff6178f740 1 -- start start 2024-04-26T02:00:18.147 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.146+0000 7eff6178f740 1 Processor -- start 2024-04-26T02:00:18.147 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.146+0000 7eff6178f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee459929b0 0x55ee459a3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:18.147 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.146+0000 7eff6178f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee459932d0 0x55ee459a42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:18.147 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.146+0000 7eff6178f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee4599db90 0x55ee45a13660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:18.147 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.146+0000 7eff6178f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7eff581abf50 con 0x55ee459932d0 2024-04-26T02:00:18.147 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.146+0000 7eff6178f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7eff581ad3c0 con 0x55ee4599db90 2024-04-26T02:00:18.147 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.146+0000 7eff6178f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7eff581ac620 con 0x55ee459929b0 2024-04-26T02:00:18.147 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.146+0000 7eff6178f740 10 client.0 _ll_register_callbacks cb 0x7eff4800bd40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:00:18.147 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.147+0000 7eff4f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee459932d0 0x55ee459a42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:18.149 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.147+0000 7eff4f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee459932d0 0x55ee459a42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41220/0 (socket says 172.21.15.151:41220) 2024-04-26T02:00:18.149 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.147+0000 7eff4f400640 1 -- 172.21.15.151:0/4203854391 learned_addr learned my addr 172.21.15.151:0/4203854391 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:00:18.149 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.147+0000 7eff57e00640 1 --2- 172.21.15.151:0/4203854391 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee459929b0 0x55ee459a3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:18.149 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.147+0000 7eff5ca00640 1 --2- 172.21.15.151:0/4203854391 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee4599db90 0x55ee45a13660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:18.151 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.147+0000 7eff57e00640 1 -- 172.21.15.151:0/4203854391 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee4599db90 msgr2=0x55ee45a13660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:18.152 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.147+0000 7eff57e00640 1 --2- 172.21.15.151:0/4203854391 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee4599db90 0x55ee45a13660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:18.152 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.147+0000 7eff57e00640 1 -- 172.21.15.151:0/4203854391 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee459932d0 msgr2=0x55ee459a42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:18.152 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.147+0000 7eff57e00640 1 --2- 172.21.15.151:0/4203854391 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee459932d0 0x55ee459a42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:18.152 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.147+0000 7eff57e00640 1 -- 172.21.15.151:0/4203854391 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee45a13c70 con 0x55ee459929b0 2024-04-26T02:00:18.152 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.147+0000 7eff5ca00640 1 --2- 172.21.15.151:0/4203854391 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee4599db90 0x55ee45a13660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:00:18.152 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.147+0000 7eff4f400640 1 --2- 172.21.15.151:0/4203854391 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee459932d0 0x55ee459a42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:00:18.152 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.147+0000 7eff57e00640 1 --2- 172.21.15.151:0/4203854391 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee459929b0 0x55ee459a3d70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7eff4802d910 tx=0x7eff4800fa10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3453200da9a0e6e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:18.152 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.148+0000 7eff45600640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-26T02:00:18.152 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.148+0000 7eff45600640 1 -- 172.21.15.151:0/4203854391 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff48029040 con 0x55ee459929b0 2024-04-26T02:00:18.152 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.148+0000 7eff45600640 1 -- 172.21.15.151:0/4203854391 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8455+0+0 (secure 0 0 0) 0x7eff48003970 con 0x55ee459929b0 2024-04-26T02:00:18.152 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.148+0000 7eff6178f740 1 -- 172.21.15.151:0/4203854391 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff40030450 con 0x55ee459929b0 2024-04-26T02:00:18.153 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.148+0000 7eff45600640 1 -- 172.21.15.151:0/4203854391 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7eff48021040 con 0x55ee459929b0 2024-04-26T02:00:18.153 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.148+0000 7eff6178f740 10 client.25969 fetch_fsmap learned FSMap version 35 2024-04-26T02:00:18.153 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.148+0000 7eff6178f740 1 -- 172.21.15.151:0/4203854391 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ee45a15010 con 0x55ee459929b0 2024-04-26T02:00:18.153 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.148+0000 7eff45600640 1 -- 172.21.15.151:0/4203854391 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff40030450 con 0x55ee459929b0 2024-04-26T02:00:18.153 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.149+0000 7eff45600640 1 -- 172.21.15.151:0/4203854391 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 35) ==== 8338+0+0 (secure 0 0 0) 0x7eff48024040 con 0x55ee459929b0 2024-04-26T02:00:18.153 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.149+0000 7eff6178f740 10 client.25969 fetch_fsmap finished waiting for FSMap version 35 2024-04-26T02:00:18.153 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.149+0000 7eff6178f740 10 client.25969 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:00:18.153 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.149+0000 7eff6178f740 20 client.25969 populate_metadata read hostname 'smithi151' 2024-04-26T02:00:18.153 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.149+0000 7eff6178f740 1 --2- 172.21.15.151:0/4203854391 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55ee45a157e0 0x55ee45a17c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:18.153 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.149+0000 7eff6178f740 4 client.25969 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T02:00:18.153 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.149+0000 7eff6178f740 1 -- 172.21.15.151:0/4203854391 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ee4598ed10 con 0x55ee45a157e0 2024-04-26T02:00:18.153 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.149+0000 7eff4f400640 1 --2- 172.21.15.151:0/4203854391 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55ee45a157e0 0x55ee45a17c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:18.154 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.150+0000 7eff4f400640 1 --2- 172.21.15.151:0/4203854391 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55ee45a157e0 0x55ee45a17c00 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fd0a5b7171e46d6 server_cookie=56c6ee9fe19e39dc in_seq=0 out_seq=0 2024-04-26T02:00:18.154 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.150+0000 7eff45600640 10 client.25969 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:00:18.163 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.163+0000 7eff45600640 1 -- 172.21.15.151:0/4203854391 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ee4598ed10 con 0x55ee45a157e0 2024-04-26T02:00:18.163 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.163+0000 7eff45600640 10 client.25969 handle_command_reply: tid=0 2024-04-26T02:00:18.163 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.163+0000 7eff6178f740 1 client.25969 shutdown 2024-04-26T02:00:18.164 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.164+0000 7eff6178f740 1 -- 172.21.15.151:0/4203854391 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee459929b0 msgr2=0x55ee459a3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:18.164 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.164+0000 7eff6178f740 1 --2- 172.21.15.151:0/4203854391 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee459929b0 0x55ee459a3d70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7eff4802d910 tx=0x7eff4800fa10 comp rx=0 tx=0).stop 2024-04-26T02:00:18.164 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.164+0000 7eff6178f740 1 -- 172.21.15.151:0/4203854391 shutdown_connections 2024-04-26T02:00:18.164 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.164+0000 7eff6178f740 1 --2- 172.21.15.151:0/4203854391 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55ee45a157e0 0x55ee45a17c00 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:18.164 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.164+0000 7eff6178f740 1 --2- 172.21.15.151:0/4203854391 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee4599db90 0x55ee45a13660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:18.164 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.164+0000 7eff6178f740 1 --2- 172.21.15.151:0/4203854391 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee459932d0 0x55ee459a42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:18.164 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.164+0000 7eff6178f740 1 --2- 172.21.15.151:0/4203854391 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee459929b0 0x55ee459a3d70 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:18.164 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.164+0000 7eff6178f740 1 -- 172.21.15.151:0/4203854391 >> 172.21.15.151:0/4203854391 conn(0x55ee4598eec0 msgr2=0x55ee4599b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:18.164 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.164+0000 7eff45600640 0 client.25969 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:00:18.164 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.164+0000 7eff6178f740 1 -- 172.21.15.151:0/4203854391 shutdown_connections 2024-04-26T02:00:18.164 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.164+0000 7eff6178f740 1 -- 172.21.15.151:0/4203854391 wait complete. 2024-04-26T02:00:18.164 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.164+0000 7eff6178f740 20 client.25969 trim_cache size 0 max 16384 2024-04-26T02:00:18.165 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:{ 2024-04-26T02:00:18.165 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "message": "", 2024-04-26T02:00:18.165 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "return_code": 0 2024-04-26T02:00:18.165 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:} 2024-04-26T02:00:18.166 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.165+0000 7eff5d400640 1 -- 172.21.15.151:0/1834377134 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7eff3c0a3d90 msgr2=0x7eff3c0a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:18.166 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.165+0000 7eff5d400640 1 --2- 172.21.15.151:0/1834377134 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7eff3c0a3d90 0x7eff3c0a6250 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x55ee45993f50 tx=0x7eff480095c0 comp rx=0 tx=0).stop 2024-04-26T02:00:18.166 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.166+0000 7eff5d400640 1 -- 172.21.15.151:0/1834377134 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7eff5810c3a0 msgr2=0x7eff5819ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:18.166 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.166+0000 7eff5d400640 1 --2- 172.21.15.151:0/1834377134 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7eff5810c3a0 0x7eff5819ce50 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7eff5000e970 tx=0x7eff5000ee30 comp rx=0 tx=0).stop 2024-04-26T02:00:18.166 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.166+0000 7eff5d400640 1 -- 172.21.15.151:0/1834377134 shutdown_connections 2024-04-26T02:00:18.166 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.166+0000 7eff5d400640 1 --2- 172.21.15.151:0/1834377134 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7eff3c0a3d90 0x7eff3c0a6250 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:18.167 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.166+0000 7eff5d400640 1 --2- 172.21.15.151:0/1834377134 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7eff5810c3a0 0x7eff5819ce50 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:18.167 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.166+0000 7eff5d400640 1 --2- 172.21.15.151:0/1834377134 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff58103850 0x7eff58198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:18.167 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.166+0000 7eff5d400640 1 --2- 172.21.15.151:0/1834377134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff58102f30 0x7eff58198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:18.167 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.166+0000 7eff5d400640 1 -- 172.21.15.151:0/1834377134 >> 172.21.15.151:0/1834377134 conn(0x7eff580fc770 msgr2=0x7eff580fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:18.167 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.166+0000 7eff5d400640 1 -- 172.21.15.151:0/1834377134 shutdown_connections 2024-04-26T02:00:18.167 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:18.166+0000 7eff5d400640 1 -- 172.21.15.151:0/1834377134 wait complete. 2024-04-26T02:00:18.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:00:18 smithi172 ceph-mon[32785]: pgmap v208: 97 pgs: 97 active+clean; 52 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 1.9 MiB/s wr, 290 op/s 2024-04-26T02:00:18.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:00:18 smithi165 ceph-mon[32648]: pgmap v208: 97 pgs: 97 active+clean; 52 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 1.9 MiB/s wr, 290 op/s 2024-04-26T02:00:18.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:00:18 smithi151 ceph-mon[28057]: pgmap v208: 97 pgs: 97 active+clean; 52 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 1.9 MiB/s wr, 290 op/s 2024-04-26T02:00:20.537 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 15.9 2024-04-26T02:00:20.537 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T02:00:20.678 INFO:tasks.workunit.client.0.smithi151.stdout:40000 mapwrite 0x1becad thru 0x1c6a92 (0x7de6 bytes) 2024-04-26T02:00:20.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:00:20 smithi172 ceph-mon[32785]: pgmap v209: 97 pgs: 97 active+clean; 54 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.8 MiB/s wr, 291 op/s 2024-04-26T02:00:20.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:00:20 smithi165 ceph-mon[32648]: pgmap v209: 97 pgs: 97 active+clean; 54 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.8 MiB/s wr, 291 op/s 2024-04-26T02:00:20.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:00:20 smithi151 ceph-mon[28057]: pgmap v209: 97 pgs: 97 active+clean; 54 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.8 MiB/s wr, 291 op/s 2024-04-26T02:00:21.026 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:00:22.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:00:22 smithi172 ceph-mon[32785]: pgmap v210: 97 pgs: 97 active+clean; 54 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.8 MiB/s wr, 291 op/s 2024-04-26T02:00:22.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:00:22 smithi165 ceph-mon[32648]: pgmap v210: 97 pgs: 97 active+clean; 54 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.8 MiB/s wr, 291 op/s 2024-04-26T02:00:22.873 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:00:22 smithi151 ceph-mon[28057]: pgmap v210: 97 pgs: 97 active+clean; 54 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.8 MiB/s wr, 291 op/s 2024-04-26T02:00:23.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:23.514+0000 7fb31d000700 1 -- 172.21.15.151:0/3714054182 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb318108a20 msgr2=0x7fb318108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:23.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:23.514+0000 7fb31d000700 1 --2- 172.21.15.151:0/3714054182 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb318108a20 0x7fb318108e00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb308009990 tx=0x7fb308009ca0 comp rx=0 tx=0).stop 2024-04-26T02:00:23.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:23.514+0000 7fb31d000700 1 -- 172.21.15.151:0/3714054182 shutdown_connections 2024-04-26T02:00:23.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:23.514+0000 7fb31d000700 1 --2- 172.21.15.151:0/3714054182 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb31806b980 0x7fb31810f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:23.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:23.514+0000 7fb31d000700 1 --2- 172.21.15.151:0/3714054182 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb318108a20 0x7fb318108e00 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:23.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:23.514+0000 7fb31d000700 1 --2- 172.21.15.151:0/3714054182 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb31806afe0 0x7fb31806b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:23.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:23.514+0000 7fb31d000700 1 -- 172.21.15.151:0/3714054182 >> 172.21.15.151:0/3714054182 conn(0x7fb3180788b0 msgr2=0x7fb318078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:23.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:23.516+0000 7fb31d000700 1 -- 172.21.15.151:0/3714054182 shutdown_connections 2024-04-26T02:00:23.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:23.516+0000 7fb31d000700 1 -- 172.21.15.151:0/3714054182 wait complete. 2024-04-26T02:00:23.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:23.517+0000 7fb31d000700 1 Processor -- start 2024-04-26T02:00:23.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:23.517+0000 7fb31d000700 1 -- start start 2024-04-26T02:00:23.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:23.517+0000 7fb31d000700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb31806afe0 0x7fb31819b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:23.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:23.517+0000 7fb31d000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb31806b980 0x7fb31819b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:23.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:23.517+0000 7fb31d000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb318108a20 0x7fb31819fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:23.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:23.517+0000 7fb31d000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb318112bf0 con 0x7fb31806b980 2024-04-26T02:00:23.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:23.517+0000 7fb31d000700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb318112a70 con 0x7fb318108a20 2024-04-26T02:00:23.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:23.517+0000 7fb31d000700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb318112d70 con 0x7fb31806afe0 2024-04-26T02:00:23.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:23.518+0000 7fb315e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb31806afe0 0x7fb31819b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:23.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:23.518+0000 7fb315e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb31806afe0 0x7fb31819b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:46602/0 (socket says 172.21.15.151:46602) 2024-04-26T02:00:23.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:23.518+0000 7fb315e00700 1 -- 172.21.15.151:0/2757084161 learned_addr learned my addr 172.21.15.151:0/2757084161 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:00:23.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:23.518+0000 7fb316800700 1 --2- 172.21.15.151:0/2757084161 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb318108a20 0x7fb31819fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:23.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:23.518+0000 7fb315400700 1 --2- 172.21.15.151:0/2757084161 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb31806b980 0x7fb31819b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:23.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:23.518+0000 7fb316800700 1 -- 172.21.15.151:0/2757084161 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb31806afe0 msgr2=0x7fb31819b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:23.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:23.518+0000 7fb316800700 1 --2- 172.21.15.151:0/2757084161 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb31806afe0 0x7fb31819b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:23.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:23.518+0000 7fb316800700 1 -- 172.21.15.151:0/2757084161 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb31806b980 msgr2=0x7fb31819b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:23.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:23.518+0000 7fb316800700 1 --2- 172.21.15.151:0/2757084161 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb31806b980 0x7fb31819b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:23.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:23.518+0000 7fb316800700 1 -- 172.21.15.151:0/2757084161 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3181a0360 con 0x7fb318108a20 2024-04-26T02:00:23.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:23.518+0000 7fb315400700 1 --2- 172.21.15.151:0/2757084161 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb31806b980 0x7fb31819b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:00:23.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:23.518+0000 7fb315e00700 1 --2- 172.21.15.151:0/2757084161 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb31806afe0 0x7fb31819b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:00:23.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:23.519+0000 7fb316800700 1 --2- 172.21.15.151:0/2757084161 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb318108a20 0x7fb31819fc20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb31000efb0 tx=0x7fb31000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:23.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:23.519+0000 7fb30ea00700 1 -- 172.21.15.151:0/2757084161 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb31000ca50 con 0x7fb318108a20 2024-04-26T02:00:23.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:23.519+0000 7fb30ea00700 1 -- 172.21.15.151:0/2757084161 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb31000cbf0 con 0x7fb318108a20 2024-04-26T02:00:23.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:23.519+0000 7fb31d000700 1 -- 172.21.15.151:0/2757084161 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3181a0650 con 0x7fb318108a20 2024-04-26T02:00:23.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:23.519+0000 7fb30ea00700 1 -- 172.21.15.151:0/2757084161 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb310004890 con 0x7fb318108a20 2024-04-26T02:00:23.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:23.519+0000 7fb31d000700 1 -- 172.21.15.151:0/2757084161 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3181a7ed0 con 0x7fb318108a20 2024-04-26T02:00:23.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:23.520+0000 7fb31d000700 1 -- 172.21.15.151:0/2757084161 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb318069730 con 0x7fb318108a20 2024-04-26T02:00:23.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:23.525+0000 7fb30ea00700 1 -- 172.21.15.151:0/2757084161 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7fb310012050 con 0x7fb318108a20 2024-04-26T02:00:23.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:23.527+0000 7fb30ea00700 1 --2- 172.21.15.151:0/2757084161 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fb2fc0a3eb0 0x7fb2fc0a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:23.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:23.527+0000 7fb30ea00700 1 -- 172.21.15.151:0/2757084161 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8455+0+0 (secure 0 0 0) 0x7fb3100d00d0 con 0x7fb318108a20 2024-04-26T02:00:23.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:23.528+0000 7fb30ea00700 1 -- 172.21.15.151:0/2757084161 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb310022060 con 0x7fb318108a20 2024-04-26T02:00:23.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:23.528+0000 7fb315e00700 1 --2- 172.21.15.151:0/2757084161 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fb2fc0a3eb0 0x7fb2fc0a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:23.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:23.529+0000 7fb315e00700 1 --2- 172.21.15.151:0/2757084161 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fb2fc0a3eb0 0x7fb2fc0a6370 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb308006010 tx=0x7fb30800b280 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:23.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:23.862+0000 7fb31d000700 1 -- 172.21.15.151:0/2757084161 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb31802df80 con 0x7fb318108a20 2024-04-26T02:00:23.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:23.863+0000 7fb30ea00700 1 -- 172.21.15.151:0/2757084161 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11889 (secure 0 0 0) 0x7fb310010040 con 0x7fb318108a20 2024-04-26T02:00:23.863 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T02:00:23.863 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2937864097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2937864097},{"type":"v1","addr":"172.21.15.151:6835","nonce":2937864097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14373,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/2947090802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2947090802},{"type":"v1","addr":"172.21.15.151:6837","nonce":2947090802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/1356563280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1356563280},{"type":"v1","addr":"172.21.15.151:6839","nonce":1356563280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6841/1075240552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1075240552},{"type":"v1","addr":"172.21.15.151:6841","nonce":1075240552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/504675912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":504675912},{"type":"v1","addr":"172.21.15.165:6833","nonce":504675912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1207580015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1207580015},{"type":"v1","addr":"172.21.15.172:6833","nonce":1207580015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:49:40.746876+0000","modified":"2024-04-26T02:00:05.413774+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24505,"mds_2":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/3598138294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3598138294},{"type":"v1","addr":"172.21.15.165:6835","nonce":3598138294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/1847859054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":1847859054},{"type":"v1","addr":"172.21.15.165:6837","nonce":1847859054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.172:6837/55972537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":55972537},{"type":"v1","addr":"172.21.15.172:6837","nonce":55972537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.172:6839/1242302810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1242302810},{"type":"v1","addr":"172.21.15.172:6839","nonce":1242302810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/1969716254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1969716254},{"type":"v1","addr":"172.21.15.165:6839","nonce":1969716254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25954":{"gid":25954,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/3656630171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3656630171},{"type":"v1","addr":"172.21.15.172:6835","nonce":3656630171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24493,24505,24509]},"id":1}]} 2024-04-26T02:00:23.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:23.866+0000 7fb31d000700 1 -- 172.21.15.151:0/2757084161 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fb2fc0a3eb0 msgr2=0x7fb2fc0a6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:23.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:23.866+0000 7fb31d000700 1 --2- 172.21.15.151:0/2757084161 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fb2fc0a3eb0 0x7fb2fc0a6370 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb308006010 tx=0x7fb30800b280 comp rx=0 tx=0).stop 2024-04-26T02:00:23.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:23.866+0000 7fb31d000700 1 -- 172.21.15.151:0/2757084161 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb318108a20 msgr2=0x7fb31819fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:23.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:23.866+0000 7fb31d000700 1 --2- 172.21.15.151:0/2757084161 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb318108a20 0x7fb31819fc20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb31000efb0 tx=0x7fb31000c3a0 comp rx=0 tx=0).stop 2024-04-26T02:00:23.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:23.866+0000 7fb31d000700 1 -- 172.21.15.151:0/2757084161 shutdown_connections 2024-04-26T02:00:23.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:23.866+0000 7fb31d000700 1 --2- 172.21.15.151:0/2757084161 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fb2fc0a3eb0 0x7fb2fc0a6370 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:23.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:23.866+0000 7fb31d000700 1 --2- 172.21.15.151:0/2757084161 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb318108a20 0x7fb31819fc20 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:23.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:23.866+0000 7fb31d000700 1 --2- 172.21.15.151:0/2757084161 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb31806afe0 0x7fb31819b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:23.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:23.866+0000 7fb31d000700 1 --2- 172.21.15.151:0/2757084161 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb31806b980 0x7fb31819b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:23.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:23.866+0000 7fb31d000700 1 -- 172.21.15.151:0/2757084161 >> 172.21.15.151:0/2757084161 conn(0x7fb3180788b0 msgr2=0x7fb31810dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:23.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:23.866+0000 7fb31d000700 1 -- 172.21.15.151:0/2757084161 shutdown_connections 2024-04-26T02:00:23.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:23.866+0000 7fb31d000700 1 -- 172.21.15.151:0/2757084161 wait complete. 2024-04-26T02:00:23.870 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 35 2024-04-26T02:00:24.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:00:24 smithi151 ceph-mon[28057]: pgmap v211: 97 pgs: 97 active+clean; 52 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.8 MiB/s wr, 299 op/s 2024-04-26T02:00:24.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:00:24 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/2757084161' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:00:24.742 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24509 quiesce db / --timeout 60 --expiration 75.9 --await-for 42.0' 2024-04-26T02:00:24.742 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24509 quiesce db / --timeout 60 --expiration 75.9 --await-for 42.0 2024-04-26T02:00:24.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:00:24 smithi172 ceph-mon[32785]: pgmap v211: 97 pgs: 97 active+clean; 52 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.8 MiB/s wr, 299 op/s 2024-04-26T02:00:24.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:00:24 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/2757084161' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:00:24.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:00:24 smithi165 ceph-mon[32648]: pgmap v211: 97 pgs: 97 active+clean; 52 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.8 MiB/s wr, 299 op/s 2024-04-26T02:00:24.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:00:24 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/2757084161' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:00:25.221 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:00:26.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:00:26 smithi151 ceph-mon[28057]: pgmap v212: 97 pgs: 97 active+clean; 52 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.7 MiB/s wr, 300 op/s 2024-04-26T02:00:26.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:00:26 smithi172 ceph-mon[32785]: pgmap v212: 97 pgs: 97 active+clean; 52 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.7 MiB/s wr, 300 op/s 2024-04-26T02:00:26.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:00:26 smithi165 ceph-mon[32648]: pgmap v212: 97 pgs: 97 active+clean; 52 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.7 MiB/s wr, 300 op/s 2024-04-26T02:00:27.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.552+0000 7f2402400700 1 -- 172.21.15.151:0/2248495523 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23fc06aff0 msgr2=0x7f23fc06b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:27.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.552+0000 7f2402400700 1 --2- 172.21.15.151:0/2248495523 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23fc06aff0 0x7f23fc06b450 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f23e4009990 tx=0x7f23e4009ca0 comp rx=0 tx=0).stop 2024-04-26T02:00:27.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.552+0000 7f2402400700 1 -- 172.21.15.151:0/2248495523 shutdown_connections 2024-04-26T02:00:27.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.552+0000 7f2402400700 1 --2- 172.21.15.151:0/2248495523 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23fc06b990 0x7f23fc10f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:27.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.552+0000 7f2402400700 1 --2- 172.21.15.151:0/2248495523 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23fc108a50 0x7f23fc108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:27.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.552+0000 7f2402400700 1 --2- 172.21.15.151:0/2248495523 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23fc06aff0 0x7f23fc06b450 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:27.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.553+0000 7f2402400700 1 -- 172.21.15.151:0/2248495523 >> 172.21.15.151:0/2248495523 conn(0x7f23fc078980 msgr2=0x7f23fc078d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:27.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.553+0000 7f2402400700 1 -- 172.21.15.151:0/2248495523 shutdown_connections 2024-04-26T02:00:27.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.553+0000 7f2402400700 1 -- 172.21.15.151:0/2248495523 wait complete. 2024-04-26T02:00:27.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.554+0000 7f2402400700 1 Processor -- start 2024-04-26T02:00:27.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.554+0000 7f2402400700 1 -- start start 2024-04-26T02:00:27.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.555+0000 7f2402400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23fc06aff0 0x7f23fc101a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:27.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.555+0000 7f2402400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23fc06b990 0x7f23fc101f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:27.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.555+0000 7f2402400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23fc108a50 0x7f23fc102480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:27.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.555+0000 7f2402400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f23fc112bf0 con 0x7f23fc06aff0 2024-04-26T02:00:27.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.555+0000 7f2402400700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f23fc112a70 con 0x7f23fc06b990 2024-04-26T02:00:27.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.555+0000 7f2402400700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f23fc112d70 con 0x7f23fc108a50 2024-04-26T02:00:27.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.555+0000 7f23fbe00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23fc108a50 0x7f23fc102480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:27.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.555+0000 7f23fb400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23fc06aff0 0x7f23fc101a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:27.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.555+0000 7f23fbe00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23fc108a50 0x7f23fc102480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:46620/0 (socket says 172.21.15.151:46620) 2024-04-26T02:00:27.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.555+0000 7f23fbe00700 1 -- 172.21.15.151:0/3862992809 learned_addr learned my addr 172.21.15.151:0/3862992809 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:00:27.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.555+0000 7f23f3e00700 1 --2- 172.21.15.151:0/3862992809 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23fc06b990 0x7f23fc101f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:27.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.556+0000 7f23fbe00700 1 -- 172.21.15.151:0/3862992809 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23fc06b990 msgr2=0x7f23fc101f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:27.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.556+0000 7f23fbe00700 1 --2- 172.21.15.151:0/3862992809 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23fc06b990 0x7f23fc101f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:27.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.556+0000 7f23fbe00700 1 -- 172.21.15.151:0/3862992809 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23fc06aff0 msgr2=0x7f23fc101a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:27.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.556+0000 7f23fbe00700 1 --2- 172.21.15.151:0/3862992809 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23fc06aff0 0x7f23fc101a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:27.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.556+0000 7f23fbe00700 1 -- 172.21.15.151:0/3862992809 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23fc1a7510 con 0x7f23fc108a50 2024-04-26T02:00:27.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.556+0000 7f23fb400700 1 --2- 172.21.15.151:0/3862992809 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23fc06aff0 0x7f23fc101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:00:27.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.556+0000 7f23fbe00700 1 --2- 172.21.15.151:0/3862992809 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23fc108a50 0x7f23fc102480 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f23f400ef30 tx=0x7f23f400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:27.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.556+0000 7f23f8c00700 1 -- 172.21.15.151:0/3862992809 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f40092e0 con 0x7f23fc108a50 2024-04-26T02:00:27.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.556+0000 7f23f8c00700 1 -- 172.21.15.151:0/3862992809 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23f4004bb0 con 0x7f23fc108a50 2024-04-26T02:00:27.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.556+0000 7f23f8c00700 1 -- 172.21.15.151:0/3862992809 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f4004280 con 0x7f23fc108a50 2024-04-26T02:00:27.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.556+0000 7f2402400700 1 -- 172.21.15.151:0/3862992809 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23fc1a77a0 con 0x7f23fc108a50 2024-04-26T02:00:27.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.557+0000 7f2402400700 1 -- 172.21.15.151:0/3862992809 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23fc1a7d00 con 0x7f23fc108a50 2024-04-26T02:00:27.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.560+0000 7f23f8c00700 1 -- 172.21.15.151:0/3862992809 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f23f401b030 con 0x7f23fc108a50 2024-04-26T02:00:27.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.562+0000 7f240ce39b80 1 Processor -- start 2024-04-26T02:00:27.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.562+0000 7f240ce39b80 1 -- start start 2024-04-26T02:00:27.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.562+0000 7f23f8c00700 1 --2- 172.21.15.151:0/3862992809 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f23e00ac6e0 0x7f23e00aeba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:27.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.562+0000 7f240ce39b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5625f4a429b0 0x5625f4a42d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:27.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.562+0000 7f240ce39b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5625f4a432d0 0x5625f4a4dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:27.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.562+0000 7f240ce39b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5625f4a4e180 0x5625f4a50570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:27.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.562+0000 7f240ce39b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5625f4996020 con 0x5625f4a4e180 2024-04-26T02:00:27.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.562+0000 7f240ce39b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5625f493c8c0 con 0x5625f4a432d0 2024-04-26T02:00:27.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.563+0000 7f23f3e00700 1 --2- 172.21.15.151:0/3862992809 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f23e00ac6e0 0x7f23e00aeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:27.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.563+0000 7f240ce39b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5625f4991fe0 con 0x5625f4a429b0 2024-04-26T02:00:27.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.563+0000 7f23f8c00700 1 -- 172.21.15.151:0/3862992809 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8455+0+0 (secure 0 0 0) 0x7f23f40d0240 con 0x7f23fc108a50 2024-04-26T02:00:27.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.563+0000 7f23fb400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5625f4a429b0 0x5625f4a42d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:27.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.563+0000 7f23fb400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5625f4a429b0 0x5625f4a42d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:46632/0 (socket says 172.21.15.151:46632) 2024-04-26T02:00:27.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.563+0000 7f23fb400700 1 -- 172.21.15.151:0/343131135 learned_addr learned my addr 172.21.15.151:0/343131135 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:00:27.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.563+0000 7f23f3e00700 1 --2- 172.21.15.151:0/343131135 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5625f4a432d0 0x5625f4a4dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:27.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.563+0000 7f23fbe00700 1 --2- 172.21.15.151:0/343131135 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5625f4a4e180 0x5625f4a50570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:27.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.563+0000 7f23fb400700 1 -- 172.21.15.151:0/343131135 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5625f4a432d0 msgr2=0x5625f4a4dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:27.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.563+0000 7f23fb400700 1 --2- 172.21.15.151:0/343131135 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5625f4a432d0 0x5625f4a4dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:27.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.563+0000 7f23fb400700 1 -- 172.21.15.151:0/343131135 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5625f4a4e180 msgr2=0x5625f4a50570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:27.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.563+0000 7f23fb400700 1 --2- 172.21.15.151:0/343131135 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5625f4a4e180 0x5625f4a50570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:27.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.563+0000 7f23fb400700 1 -- 172.21.15.151:0/343131135 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625f4932810 con 0x5625f4a429b0 2024-04-26T02:00:27.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.564+0000 7f23f3e00700 1 --2- 172.21.15.151:0/3862992809 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f23e00ac6e0 0x7f23e00aeba0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f23e40098e0 tx=0x7f23e4009620 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:27.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.564+0000 7f23fb400700 1 --2- 172.21.15.151:0/343131135 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5625f4a429b0 0x5625f4a42d90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f23ec009dc0 tx=0x7f23ec0106b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4d90a56f70adc8e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:27.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.564+0000 7f23f0c00700 1 -- 172.21.15.151:0/343131135 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ec010d90 con 0x5625f4a429b0 2024-04-26T02:00:27.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.564+0000 7f23f0c00700 1 -- 172.21.15.151:0/343131135 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23ec01c930 con 0x5625f4a429b0 2024-04-26T02:00:27.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.564+0000 7f23f0c00700 1 -- 172.21.15.151:0/343131135 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ec011870 con 0x5625f4a429b0 2024-04-26T02:00:27.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.565+0000 7f240ce39b80 1 -- 172.21.15.151:0/343131135 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5625f4a429b0 msgr2=0x5625f4a42d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:27.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.565+0000 7f240ce39b80 1 --2- 172.21.15.151:0/343131135 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5625f4a429b0 0x5625f4a42d90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f23ec009dc0 tx=0x7f23ec0106b0 comp rx=0 tx=0).stop 2024-04-26T02:00:27.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.565+0000 7f240ce39b80 1 -- 172.21.15.151:0/343131135 shutdown_connections 2024-04-26T02:00:27.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.565+0000 7f240ce39b80 1 --2- 172.21.15.151:0/343131135 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5625f4a4e180 0x5625f4a50570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:27.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.565+0000 7f240ce39b80 1 --2- 172.21.15.151:0/343131135 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5625f4a429b0 0x5625f4a42d90 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:27.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.565+0000 7f240ce39b80 1 --2- 172.21.15.151:0/343131135 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5625f4a432d0 0x5625f4a4dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:27.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.565+0000 7f240ce39b80 1 -- 172.21.15.151:0/343131135 >> 172.21.15.151:0/343131135 conn(0x5625f4971730 msgr2=0x5625f4a3ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:27.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.565+0000 7f240ce39b80 1 -- 172.21.15.151:0/343131135 shutdown_connections 2024-04-26T02:00:27.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.566+0000 7f240ce39b80 1 -- 172.21.15.151:0/343131135 wait complete. 2024-04-26T02:00:27.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.566+0000 7f240ce39b80 1 -- start start 2024-04-26T02:00:27.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.566+0000 7f240ce39b80 1 Processor -- start 2024-04-26T02:00:27.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.567+0000 7f240ce39b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5625f4a429b0 0x5625f4a4d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:27.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.567+0000 7f240ce39b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5625f4a432d0 0x5625f4a4aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:27.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.567+0000 7f240ce39b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5625f4a4e180 0x5625f4a4b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:27.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.567+0000 7f240ce39b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5625f4a61cb0 con 0x5625f4a429b0 2024-04-26T02:00:27.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.567+0000 7f240ce39b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5625f494f9f0 con 0x5625f4a4e180 2024-04-26T02:00:27.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.567+0000 7f240ce39b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5625f4a61e30 con 0x5625f4a432d0 2024-04-26T02:00:27.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.567+0000 7f240ce39b80 10 client.0 _ll_register_callbacks cb 0x5625f492e8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:00:27.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.567+0000 7f23f3e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5625f4a432d0 0x5625f4a4aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:27.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.568+0000 7f23f3e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5625f4a432d0 0x5625f4a4aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:46634/0 (socket says 172.21.15.151:46634) 2024-04-26T02:00:27.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.568+0000 7f23f3e00700 1 -- 172.21.15.151:0/656296353 learned_addr learned my addr 172.21.15.151:0/656296353 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:00:27.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.568+0000 7f23fbe00700 1 --2- 172.21.15.151:0/656296353 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5625f4a4e180 0x5625f4a4b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:27.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.568+0000 7f23fb400700 1 --2- 172.21.15.151:0/656296353 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5625f4a429b0 0x5625f4a4d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:27.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.568+0000 7f23f3e00700 1 -- 172.21.15.151:0/656296353 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5625f4a4e180 msgr2=0x5625f4a4b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:27.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.568+0000 7f23f3e00700 1 --2- 172.21.15.151:0/656296353 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5625f4a4e180 0x5625f4a4b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:27.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.568+0000 7f23f3e00700 1 -- 172.21.15.151:0/656296353 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5625f4a429b0 msgr2=0x5625f4a4d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:27.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.568+0000 7f23f3e00700 1 --2- 172.21.15.151:0/656296353 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5625f4a429b0 0x5625f4a4d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:27.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.568+0000 7f23f3e00700 1 -- 172.21.15.151:0/656296353 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625f48e3a60 con 0x5625f4a432d0 2024-04-26T02:00:27.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.568+0000 7f23fbe00700 1 --2- 172.21.15.151:0/656296353 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5625f4a4e180 0x5625f4a4b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:00:27.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.568+0000 7f23f3e00700 1 --2- 172.21.15.151:0/656296353 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5625f4a432d0 0x5625f4a4aba0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f23e4015000 tx=0x7f23e4005960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=651c911ed8863e46 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:27.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.568+0000 7f23e9600700 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-26T02:00:27.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.569+0000 7f23e9600700 1 -- 172.21.15.151:0/656296353 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e4003b10 con 0x5625f4a432d0 2024-04-26T02:00:27.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.569+0000 7f23e9600700 1 -- 172.21.15.151:0/656296353 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8455+0+0 (secure 0 0 0) 0x7f23e4003cb0 con 0x5625f4a432d0 2024-04-26T02:00:27.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.569+0000 7f240ce39b80 1 -- 172.21.15.151:0/656296353 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23ec01c930 con 0x5625f4a432d0 2024-04-26T02:00:27.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.569+0000 7f23e9600700 1 -- 172.21.15.151:0/656296353 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f23e4028460 con 0x5625f4a432d0 2024-04-26T02:00:27.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.569+0000 7f240ce39b80 10 client.25987 fetch_fsmap learned FSMap version 35 2024-04-26T02:00:27.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.569+0000 7f240ce39b80 1 -- 172.21.15.151:0/656296353 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5625f492cc80 con 0x5625f4a432d0 2024-04-26T02:00:27.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.570+0000 7f23e9600700 1 -- 172.21.15.151:0/656296353 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23ec01c930 con 0x5625f4a432d0 2024-04-26T02:00:27.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.571+0000 7f23e9600700 1 -- 172.21.15.151:0/656296353 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 35) ==== 8338+0+0 (secure 0 0 0) 0x7f23e4030070 con 0x5625f4a432d0 2024-04-26T02:00:27.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.571+0000 7f240ce39b80 10 client.25987 fetch_fsmap finished waiting for FSMap version 35 2024-04-26T02:00:27.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.571+0000 7f240ce39b80 10 client.25987 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-26T02:00:27.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.571+0000 7f240ce39b80 20 client.25987 populate_metadata read hostname 'smithi151' 2024-04-26T02:00:27.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.571+0000 7f240ce39b80 1 --2- 172.21.15.151:0/656296353 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5625f4a6e8b0 0x5625f4a70cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:27.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.571+0000 7f240ce39b80 4 client.25987 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:00:27.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.571+0000 7f240ce39b80 1 -- 172.21.15.151:0/656296353 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625f49a3810 con 0x5625f4a6e8b0 2024-04-26T02:00:27.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.572+0000 7f23fb400700 1 --2- 172.21.15.151:0/656296353 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5625f4a6e8b0 0x5625f4a70cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:27.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.573+0000 7f23fb400700 1 --2- 172.21.15.151:0/656296353 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5625f4a6e8b0 0x5625f4a70cf0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b85f91fbb356f128 server_cookie=ce54d7dd0effe104 in_seq=0 out_seq=0 2024-04-26T02:00:27.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.573+0000 7f23e9600700 10 client.25987 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:00:27.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.573+0000 7f23e9600700 1 -- 172.21.15.151:0/656296353 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5625f49a3810 con 0x5625f4a6e8b0 2024-04-26T02:00:27.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.573+0000 7f23e9600700 10 client.25987 handle_command_reply: tid=0 2024-04-26T02:00:27.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.574+0000 7f240ce39b80 1 client.25987 shutdown 2024-04-26T02:00:27.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.574+0000 7f240ce39b80 1 -- 172.21.15.151:0/656296353 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5625f4a432d0 msgr2=0x5625f4a4aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:27.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.574+0000 7f240ce39b80 1 --2- 172.21.15.151:0/656296353 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5625f4a432d0 0x5625f4a4aba0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f23e4015000 tx=0x7f23e4005960 comp rx=0 tx=0).stop 2024-04-26T02:00:27.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.574+0000 7f240ce39b80 1 -- 172.21.15.151:0/656296353 shutdown_connections 2024-04-26T02:00:27.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.574+0000 7f240ce39b80 1 --2- 172.21.15.151:0/656296353 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5625f4a6e8b0 0x5625f4a70cf0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:27.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.574+0000 7f240ce39b80 1 --2- 172.21.15.151:0/656296353 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5625f4a4e180 0x5625f4a4b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:27.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.574+0000 7f23e9600700 0 client.25987 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:00:27.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.574+0000 7f240ce39b80 1 --2- 172.21.15.151:0/656296353 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5625f4a429b0 0x5625f4a4d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:27.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.574+0000 7f240ce39b80 1 --2- 172.21.15.151:0/656296353 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5625f4a432d0 0x5625f4a4aba0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:27.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.574+0000 7f240ce39b80 1 -- 172.21.15.151:0/656296353 >> 172.21.15.151:0/656296353 conn(0x5625f4971730 msgr2=0x5625f4a3ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:27.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.574+0000 7f240ce39b80 1 -- 172.21.15.151:0/656296353 shutdown_connections 2024-04-26T02:00:27.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.574+0000 7f240ce39b80 1 -- 172.21.15.151:0/656296353 wait complete. 2024-04-26T02:00:27.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.574+0000 7f240ce39b80 20 client.25987 trim_cache size 0 max 16384 2024-04-26T02:00:27.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.600+0000 7f240ce39b80 1 Processor -- start 2024-04-26T02:00:27.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.600+0000 7f240ce39b80 1 -- start start 2024-04-26T02:00:27.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.600+0000 7f240ce39b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5625f4a429b0 0x5625f4a6abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:27.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.600+0000 7f240ce39b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5625f4a432d0 0x5625f4a4cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:27.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.600+0000 7f240ce39b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5625f4a4e180 0x5625f4a670c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:27.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.600+0000 7f240ce39b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f23fc1ad870 con 0x5625f4a432d0 2024-04-26T02:00:27.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.600+0000 7f240ce39b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f23fc1ab100 con 0x5625f4a429b0 2024-04-26T02:00:27.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.600+0000 7f240ce39b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f23fc19cf10 con 0x5625f4a4e180 2024-04-26T02:00:27.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.601+0000 7f23fb400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5625f4a429b0 0x5625f4a6abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:27.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.601+0000 7f23fbe00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5625f4a4e180 0x5625f4a670c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:27.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.601+0000 7f23fbe00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5625f4a4e180 0x5625f4a670c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:46650/0 (socket says 172.21.15.151:46650) 2024-04-26T02:00:27.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.601+0000 7f23fb400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5625f4a429b0 0x5625f4a6abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:40922/0 (socket says 172.21.15.151:40922) 2024-04-26T02:00:27.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.601+0000 7f23fbe00700 1 -- 172.21.15.151:0/972274283 learned_addr learned my addr 172.21.15.151:0/972274283 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:00:27.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.601+0000 7f23f3e00700 1 --2- 172.21.15.151:0/972274283 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5625f4a432d0 0x5625f4a4cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:27.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.601+0000 7f23fbe00700 1 -- 172.21.15.151:0/972274283 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5625f4a429b0 msgr2=0x5625f4a6abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:27.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.601+0000 7f23fbe00700 1 --2- 172.21.15.151:0/972274283 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5625f4a429b0 0x5625f4a6abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:27.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.601+0000 7f23fbe00700 1 -- 172.21.15.151:0/972274283 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5625f4a432d0 msgr2=0x5625f4a4cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:27.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.601+0000 7f23fbe00700 1 --2- 172.21.15.151:0/972274283 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5625f4a432d0 0x5625f4a4cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:27.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.601+0000 7f23fbe00700 1 -- 172.21.15.151:0/972274283 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625f4a436b0 con 0x5625f4a4e180 2024-04-26T02:00:27.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.601+0000 7f23fbe00700 1 --2- 172.21.15.151:0/972274283 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5625f4a4e180 0x5625f4a670c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f23f4097f90 tx=0x7f23f400ca70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56843ebaae553266 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:27.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.602+0000 7f23ebe00700 1 -- 172.21.15.151:0/972274283 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f400ce60 con 0x5625f4a4e180 2024-04-26T02:00:27.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.602+0000 7f23ebe00700 1 -- 172.21.15.151:0/972274283 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23f409c9b0 con 0x5625f4a4e180 2024-04-26T02:00:27.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.602+0000 7f23ebe00700 1 -- 172.21.15.151:0/972274283 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f40a0170 con 0x5625f4a4e180 2024-04-26T02:00:27.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.603+0000 7f240ce39b80 1 -- 172.21.15.151:0/972274283 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5625f4a4e180 msgr2=0x5625f4a670c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:27.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.603+0000 7f240ce39b80 1 --2- 172.21.15.151:0/972274283 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5625f4a4e180 0x5625f4a670c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f23f4097f90 tx=0x7f23f400ca70 comp rx=0 tx=0).stop 2024-04-26T02:00:27.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.603+0000 7f240ce39b80 1 -- 172.21.15.151:0/972274283 shutdown_connections 2024-04-26T02:00:27.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.603+0000 7f240ce39b80 1 --2- 172.21.15.151:0/972274283 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5625f4a4e180 0x5625f4a670c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:27.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.603+0000 7f240ce39b80 1 --2- 172.21.15.151:0/972274283 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5625f4a429b0 0x5625f4a6abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:27.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.603+0000 7f240ce39b80 1 --2- 172.21.15.151:0/972274283 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5625f4a432d0 0x5625f4a4cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:27.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.603+0000 7f240ce39b80 1 -- 172.21.15.151:0/972274283 >> 172.21.15.151:0/972274283 conn(0x5625f4971730 msgr2=0x5625f4a4c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:27.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.603+0000 7f240ce39b80 1 -- 172.21.15.151:0/972274283 shutdown_connections 2024-04-26T02:00:27.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.603+0000 7f240ce39b80 1 -- 172.21.15.151:0/972274283 wait complete. 2024-04-26T02:00:27.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.604+0000 7f240ce39b80 1 -- start start 2024-04-26T02:00:27.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.604+0000 7f240ce39b80 1 Processor -- start 2024-04-26T02:00:27.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.604+0000 7f240ce39b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5625f4a429b0 0x5625f4a65d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:27.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.605+0000 7f240ce39b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5625f4a432d0 0x5625f4a66260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:27.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.605+0000 7f240ce39b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5625f4a4e180 0x5625f4a667a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:27.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.605+0000 7f240ce39b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f23fc1a95e0 con 0x5625f4a429b0 2024-04-26T02:00:27.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.605+0000 7f240ce39b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f23fc1aaa30 con 0x5625f4a4e180 2024-04-26T02:00:27.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.605+0000 7f240ce39b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f23fc1a9c90 con 0x5625f4a432d0 2024-04-26T02:00:27.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.605+0000 7f240ce39b80 10 client.0 _ll_register_callbacks cb 0x5625f492e8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:00:27.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.605+0000 7f23f3e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5625f4a432d0 0x5625f4a66260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:27.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.605+0000 7f23f3e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5625f4a432d0 0x5625f4a66260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:46658/0 (socket says 172.21.15.151:46658) 2024-04-26T02:00:27.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.605+0000 7f23f3e00700 1 -- 172.21.15.151:0/2914026389 learned_addr learned my addr 172.21.15.151:0/2914026389 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:00:27.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.605+0000 7f23fbe00700 1 --2- 172.21.15.151:0/2914026389 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5625f4a4e180 0x5625f4a667a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:27.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.605+0000 7f23fb400700 1 --2- 172.21.15.151:0/2914026389 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5625f4a429b0 0x5625f4a65d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:27.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.605+0000 7f23fbe00700 1 -- 172.21.15.151:0/2914026389 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5625f4a432d0 msgr2=0x5625f4a66260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:27.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.605+0000 7f23fbe00700 1 --2- 172.21.15.151:0/2914026389 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5625f4a432d0 0x5625f4a66260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:27.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.605+0000 7f23fbe00700 1 -- 172.21.15.151:0/2914026389 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5625f4a429b0 msgr2=0x5625f4a65d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:27.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.605+0000 7f23fbe00700 1 --2- 172.21.15.151:0/2914026389 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5625f4a429b0 0x5625f4a65d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:27.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.605+0000 7f23fbe00700 1 -- 172.21.15.151:0/2914026389 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625f4a71060 con 0x5625f4a4e180 2024-04-26T02:00:27.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.606+0000 7f23fbe00700 1 --2- 172.21.15.151:0/2914026389 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5625f4a4e180 0x5625f4a667a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f23f4098c80 tx=0x7f23f409a7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3b3a17e5d94362e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:27.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.606+0000 7f23e9600700 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T02:00:27.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.606+0000 7f23e9600700 1 -- 172.21.15.151:0/2914026389 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f40b4070 con 0x5625f4a4e180 2024-04-26T02:00:27.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.606+0000 7f23e9600700 1 -- 172.21.15.151:0/2914026389 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8455+0+0 (secure 0 0 0) 0x7f23f40a0170 con 0x5625f4a4e180 2024-04-26T02:00:27.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.606+0000 7f240ce39b80 1 -- 172.21.15.151:0/2914026389 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23f409c9b0 con 0x5625f4a4e180 2024-04-26T02:00:27.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.607+0000 7f23e9600700 1 -- 172.21.15.151:0/2914026389 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f23f409c2b0 con 0x5625f4a4e180 2024-04-26T02:00:27.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.607+0000 7f240ce39b80 10 client.25958 fetch_fsmap learned FSMap version 35 2024-04-26T02:00:27.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.607+0000 7f240ce39b80 1 -- 172.21.15.151:0/2914026389 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5625f4ad6170 con 0x5625f4a4e180 2024-04-26T02:00:27.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.607+0000 7f23e9600700 1 -- 172.21.15.151:0/2914026389 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23f409c9b0 con 0x5625f4a4e180 2024-04-26T02:00:27.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.607+0000 7f23e9600700 1 -- 172.21.15.151:0/2914026389 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 35) ==== 8338+0+0 (secure 0 0 0) 0x7f23f4098000 con 0x5625f4a4e180 2024-04-26T02:00:27.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.607+0000 7f240ce39b80 10 client.25958 fetch_fsmap finished waiting for FSMap version 35 2024-04-26T02:00:27.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.607+0000 7f240ce39b80 10 client.25958 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-26T02:00:27.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.607+0000 7f240ce39b80 20 client.25958 populate_metadata read hostname 'smithi151' 2024-04-26T02:00:27.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.607+0000 7f240ce39b80 1 --2- 172.21.15.151:0/2914026389 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5625f4ad6860 0x5625f4ad8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:27.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.607+0000 7f240ce39b80 4 client.25958 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 75.9, "await_for": 42.0}] 2024-04-26T02:00:27.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.607+0000 7f240ce39b80 1 -- 172.21.15.151:0/2914026389 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 75.9, "await_for": 42.0}) -- 0x5625f499d8b0 con 0x5625f4ad6860 2024-04-26T02:00:27.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.608+0000 7f23fb400700 1 --2- 172.21.15.151:0/2914026389 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5625f4ad6860 0x5625f4ad8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:27.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.609+0000 7f23fb400700 1 --2- 172.21.15.151:0/2914026389 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5625f4ad6860 0x5625f4ad8c80 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a00fb596ce793260 server_cookie=d56c8c623d3a6762 in_seq=0 out_seq=0 2024-04-26T02:00:27.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:27.609+0000 7f23e9600700 10 client.25958 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:00:28.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:28.622+0000 7f23e9600700 1 -- 172.21.15.151:0/2914026389 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5625f499d8b0 con 0x5625f4ad6860 2024-04-26T02:00:28.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:28.622+0000 7f23e9600700 10 client.25958 handle_command_reply: tid=0 2024-04-26T02:00:28.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:28.622+0000 7f240ce39b80 1 client.25958 shutdown 2024-04-26T02:00:28.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:28.623+0000 7f240ce39b80 1 -- 172.21.15.151:0/2914026389 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5625f4a4e180 msgr2=0x5625f4a667a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:28.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:28.623+0000 7f240ce39b80 1 --2- 172.21.15.151:0/2914026389 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5625f4a4e180 0x5625f4a667a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f23f4098c80 tx=0x7f23f409a7c0 comp rx=0 tx=0).stop 2024-04-26T02:00:28.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:28.623+0000 7f240ce39b80 1 -- 172.21.15.151:0/2914026389 shutdown_connections 2024-04-26T02:00:28.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:28.623+0000 7f240ce39b80 1 --2- 172.21.15.151:0/2914026389 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5625f4ad6860 0x5625f4ad8c80 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:28.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:28.623+0000 7f240ce39b80 1 --2- 172.21.15.151:0/2914026389 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5625f4a4e180 0x5625f4a667a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:28.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:28.623+0000 7f240ce39b80 1 --2- 172.21.15.151:0/2914026389 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5625f4a429b0 0x5625f4a65d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:28.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:28.623+0000 7f240ce39b80 1 --2- 172.21.15.151:0/2914026389 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5625f4a432d0 0x5625f4a66260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:28.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:28.623+0000 7f23e9600700 0 client.25958 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:00:28.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:28.623+0000 7f240ce39b80 1 -- 172.21.15.151:0/2914026389 >> 172.21.15.151:0/2914026389 conn(0x5625f4971730 msgr2=0x5625f4a4cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:28.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:28.624+0000 7f240ce39b80 1 -- 172.21.15.151:0/2914026389 shutdown_connections 2024-04-26T02:00:28.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:28.624+0000 7f240ce39b80 1 -- 172.21.15.151:0/2914026389 wait complete. 2024-04-26T02:00:28.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:28.624+0000 7f240ce39b80 20 client.25958 trim_cache size 0 max 16384 2024-04-26T02:00:28.624 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-26T02:00:28.624 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 35, 2024-04-26T02:00:28.624 INFO:teuthology.orchestra.run.smithi151.stdout: "leader": 24509, 2024-04-26T02:00:28.624 INFO:teuthology.orchestra.run.smithi151.stdout: "set_version": 18, 2024-04-26T02:00:28.624 INFO:teuthology.orchestra.run.smithi151.stdout: "sets": { 2024-04-26T02:00:28.624 INFO:teuthology.orchestra.run.smithi151.stdout: "bb158b1e": { 2024-04-26T02:00:28.624 INFO:teuthology.orchestra.run.smithi151.stdout: "version": 18, 2024-04-26T02:00:28.625 INFO:teuthology.orchestra.run.smithi151.stdout: "age_ref": 0.0, 2024-04-26T02:00:28.625 INFO:teuthology.orchestra.run.smithi151.stdout: "state": { 2024-04-26T02:00:28.625 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "QUIESCED", 2024-04-26T02:00:28.625 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 0.0 2024-04-26T02:00:28.625 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:00:28.625 INFO:teuthology.orchestra.run.smithi151.stdout: "timeout": 60.0, 2024-04-26T02:00:28.625 INFO:teuthology.orchestra.run.smithi151.stdout: "expiration": 75.9, 2024-04-26T02:00:28.625 INFO:teuthology.orchestra.run.smithi151.stdout: "members": { 2024-04-26T02:00:28.625 INFO:teuthology.orchestra.run.smithi151.stdout: "file:/": { 2024-04-26T02:00:28.625 INFO:teuthology.orchestra.run.smithi151.stdout: "excluded": false, 2024-04-26T02:00:28.625 INFO:teuthology.orchestra.run.smithi151.stdout: "state": { 2024-04-26T02:00:28.625 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "QUIESCED", 2024-04-26T02:00:28.625 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 0.0 2024-04-26T02:00:28.625 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:00:28.626 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:00:28.626 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:00:28.626 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:00:28.626 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:00:28.626 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-26T02:00:28.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:28.625+0000 7f2402400700 1 -- 172.21.15.151:0/3862992809 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f23e00ac6e0 msgr2=0x7f23e00aeba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:28.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:28.625+0000 7f2402400700 1 --2- 172.21.15.151:0/3862992809 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f23e00ac6e0 0x7f23e00aeba0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f23e40098e0 tx=0x7f23e4009620 comp rx=0 tx=0).stop 2024-04-26T02:00:28.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:28.625+0000 7f2402400700 1 -- 172.21.15.151:0/3862992809 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23fc108a50 msgr2=0x7f23fc102480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:28.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:28.625+0000 7f2402400700 1 --2- 172.21.15.151:0/3862992809 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23fc108a50 0x7f23fc102480 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f23f400ef30 tx=0x7f23f400c3a0 comp rx=0 tx=0).stop 2024-04-26T02:00:28.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:28.626+0000 7f2402400700 1 -- 172.21.15.151:0/3862992809 shutdown_connections 2024-04-26T02:00:28.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:28.626+0000 7f2402400700 1 --2- 172.21.15.151:0/3862992809 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f23e00ac6e0 0x7f23e00aeba0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:28.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:28.626+0000 7f2402400700 1 --2- 172.21.15.151:0/3862992809 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23fc108a50 0x7f23fc102480 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:28.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:28.626+0000 7f2402400700 1 --2- 172.21.15.151:0/3862992809 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23fc06aff0 0x7f23fc101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:28.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:28.626+0000 7f2402400700 1 --2- 172.21.15.151:0/3862992809 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23fc06b990 0x7f23fc101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:28.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:28.626+0000 7f2402400700 1 -- 172.21.15.151:0/3862992809 >> 172.21.15.151:0/3862992809 conn(0x7f23fc078980 msgr2=0x7f23fc0ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:28.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:28.626+0000 7f2402400700 1 -- 172.21.15.151:0/3862992809 shutdown_connections 2024-04-26T02:00:28.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:28.626+0000 7f2402400700 1 -- 172.21.15.151:0/3862992809 wait complete. 2024-04-26T02:00:28.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:00:28 smithi172 ceph-mon[32785]: pgmap v213: 97 pgs: 97 active+clean; 52 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.6 MiB/s wr, 267 op/s 2024-04-26T02:00:28.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:00:28 smithi165 ceph-mon[32648]: pgmap v213: 97 pgs: 97 active+clean; 52 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.6 MiB/s wr, 267 op/s 2024-04-26T02:00:28.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:00:28 smithi151 ceph-mon[28057]: pgmap v213: 97 pgs: 97 active+clean; 52 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.6 MiB/s wr, 267 op/s 2024-04-26T02:00:29.584 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'bb158b1e', quiesce took 9.0 seconds. Will release after: 15.9 2024-04-26T02:00:30.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:00:30 smithi172 ceph-mon[32785]: pgmap v214: 97 pgs: 97 active+clean; 61 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.5 MiB/s wr, 410 op/s 2024-04-26T02:00:30.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:00:30 smithi165 ceph-mon[32648]: pgmap v214: 97 pgs: 97 active+clean; 61 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.5 MiB/s wr, 410 op/s 2024-04-26T02:00:30.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:00:30 smithi151 ceph-mon[28057]: pgmap v214: 97 pgs: 97 active+clean; 61 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.5 MiB/s wr, 410 op/s 2024-04-26T02:00:31.309 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.310+0000 7f33b7200640 1 -- 172.21.15.151:0/1614325796 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f33b8108490 msgr2=0x7f33b81088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:31.309 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.310+0000 7f33b7200640 1 --2- 172.21.15.151:0/1614325796 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f33b8108490 0x7f33b81088d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f33a0009930 tx=0x7f33a002a420 comp rx=0 tx=0).stop 2024-04-26T02:00:31.309 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.310+0000 7f33b7200640 1 -- 172.21.15.151:0/1614325796 shutdown_connections 2024-04-26T02:00:31.309 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.310+0000 7f33b7200640 1 --2- 172.21.15.151:0/1614325796 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33b8108e10 0x7f33b8115250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:31.309 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.310+0000 7f33b7200640 1 --2- 172.21.15.151:0/1614325796 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f33b8108490 0x7f33b81088d0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:31.309 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.310+0000 7f33b7200640 1 --2- 172.21.15.151:0/1614325796 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33b810e420 0x7f33b810e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:31.310 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.310+0000 7f33b7200640 1 -- 172.21.15.151:0/1614325796 >> 172.21.15.151:0/1614325796 conn(0x7f33b8077d30 msgr2=0x7f33b8106d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:31.310 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.310+0000 7f33b7200640 1 -- 172.21.15.151:0/1614325796 shutdown_connections 2024-04-26T02:00:31.310 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.310+0000 7f33b7200640 1 -- 172.21.15.151:0/1614325796 wait complete. 2024-04-26T02:00:31.312 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.311+0000 7f33b7200640 1 Processor -- start 2024-04-26T02:00:31.312 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.311+0000 7f33b7200640 1 -- start start 2024-04-26T02:00:31.312 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.311+0000 7f33b7200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33b8108490 0x7f33b81a47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:31.312 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.311+0000 7f33b7200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33b8108e10 0x7f33b81a4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:31.312 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.311+0000 7f33b7200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f33b810e420 0x7f33b819e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:31.312 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.311+0000 7f33b7200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f33b8118640 con 0x7f33b810e420 2024-04-26T02:00:31.312 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.311+0000 7f33b7200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f33b81184c0 con 0x7f33b8108e10 2024-04-26T02:00:31.312 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.311+0000 7f33b7200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f33b81187c0 con 0x7f33b8108490 2024-04-26T02:00:31.312 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.312+0000 7f33ad400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33b8108e10 0x7f33b81a4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:31.312 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.312+0000 7f33ad400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33b8108e10 0x7f33b81a4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:59744/0 (socket says 172.21.15.151:59744) 2024-04-26T02:00:31.313 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.312+0000 7f33ad400640 1 -- 172.21.15.151:0/905202225 learned_addr learned my addr 172.21.15.151:0/905202225 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:00:31.313 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.312+0000 7f33b6800640 1 --2- 172.21.15.151:0/905202225 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f33b810e420 0x7f33b819e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:31.313 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.312+0000 7f33b5e00640 1 --2- 172.21.15.151:0/905202225 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33b8108490 0x7f33b81a47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:31.313 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.312+0000 7f33ad400640 1 -- 172.21.15.151:0/905202225 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33b8108490 msgr2=0x7f33b81a47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:31.313 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.312+0000 7f33ad400640 1 --2- 172.21.15.151:0/905202225 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33b8108490 0x7f33b81a47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:31.313 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.312+0000 7f33ad400640 1 -- 172.21.15.151:0/905202225 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f33b810e420 msgr2=0x7f33b819e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:31.313 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.312+0000 7f33ad400640 1 --2- 172.21.15.151:0/905202225 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f33b810e420 0x7f33b819e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:31.313 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.312+0000 7f33ad400640 1 -- 172.21.15.151:0/905202225 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33b819f140 con 0x7f33b8108e10 2024-04-26T02:00:31.313 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.312+0000 7f33b6800640 1 --2- 172.21.15.151:0/905202225 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f33b810e420 0x7f33b819e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:00:31.313 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.312+0000 7f33ad400640 1 --2- 172.21.15.151:0/905202225 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33b8108e10 0x7f33b81a4d00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f33a0005ab0 tx=0x7f33a0003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:31.313 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.313+0000 7f33af400640 1 -- 172.21.15.151:0/905202225 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a003f070 con 0x7f33b8108e10 2024-04-26T02:00:31.313 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.313+0000 7f33b7200640 1 -- 172.21.15.151:0/905202225 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33b819f3d0 con 0x7f33b8108e10 2024-04-26T02:00:31.313 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.313+0000 7f33af400640 1 -- 172.21.15.151:0/905202225 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33a0031b50 con 0x7f33b8108e10 2024-04-26T02:00:31.313 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.313+0000 7f33af400640 1 -- 172.21.15.151:0/905202225 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a0035030 con 0x7f33b8108e10 2024-04-26T02:00:31.314 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.313+0000 7f33b7200640 1 -- 172.21.15.151:0/905202225 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33b81ab680 con 0x7f33b8108e10 2024-04-26T02:00:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.316+0000 7f33bf9f4740 1 Processor -- start 2024-04-26T02:00:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.316+0000 7f33af400640 1 -- 172.21.15.151:0/905202225 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f33a0042070 con 0x7f33b8108e10 2024-04-26T02:00:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.316+0000 7f33bf9f4740 1 -- start start 2024-04-26T02:00:31.316 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.317+0000 7f33bf9f4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556712b6b9b0 0x556712b6bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:31.316 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.317+0000 7f33bf9f4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556712b6c2d0 0x556712b76650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.317+0000 7f33bf9f4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556712b76b90 0x556712b78f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.317+0000 7f33bf9f4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x556712ad1900 con 0x556712b76b90 2024-04-26T02:00:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.317+0000 7f33bf9f4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556712a77ef0 con 0x556712b6b9b0 2024-04-26T02:00:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.317+0000 7f33bf9f4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556712acd8c0 con 0x556712b6c2d0 2024-04-26T02:00:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.317+0000 7f33b6800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556712b6b9b0 0x556712b6bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.317+0000 7f33b6800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556712b6b9b0 0x556712b6bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:59760/0 (socket says 172.21.15.151:59760) 2024-04-26T02:00:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.317+0000 7f33b6800640 1 -- 172.21.15.151:0/1163843194 learned_addr learned my addr 172.21.15.151:0/1163843194 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:00:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.317+0000 7f33b5e00640 1 --2- 172.21.15.151:0/1163843194 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556712b6c2d0 0x556712b76650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.317+0000 7f33ad400640 1 --2- 172.21.15.151:0/1163843194 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556712b76b90 0x556712b78f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.317+0000 7f33b6800640 1 -- 172.21.15.151:0/1163843194 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556712b6c2d0 msgr2=0x556712b76650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.317+0000 7f33b6800640 1 --2- 172.21.15.151:0/1163843194 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556712b6c2d0 0x556712b76650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.317+0000 7f33b6800640 1 -- 172.21.15.151:0/1163843194 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556712b76b90 msgr2=0x556712b78f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.317+0000 7f33b6800640 1 --2- 172.21.15.151:0/1163843194 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556712b76b90 0x556712b78f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:31.318 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.317+0000 7f33b6800640 1 -- 172.21.15.151:0/1163843194 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567127e4210 con 0x556712b6b9b0 2024-04-26T02:00:31.318 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.318+0000 7f33b6800640 1 --2- 172.21.15.151:0/1163843194 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556712b6b9b0 0x556712b6bd90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f33b00117e0 tx=0x7f33b0011ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63eb8717e837c207 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:31.318 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.318+0000 7f33af400640 1 --2- 172.21.15.151:0/905202225 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f339c0a3d10 0x7f339c0a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:31.321 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.318+0000 7f33a7000640 1 -- 172.21.15.151:0/1163843194 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b0011310 con 0x556712b6b9b0 2024-04-26T02:00:31.321 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.318+0000 7f33a7000640 1 -- 172.21.15.151:0/1163843194 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33b0003e20 con 0x556712b6b9b0 2024-04-26T02:00:31.321 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.318+0000 7f33a7000640 1 -- 172.21.15.151:0/1163843194 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b00044f0 con 0x556712b6b9b0 2024-04-26T02:00:31.321 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.318+0000 7f33af400640 1 -- 172.21.15.151:0/905202225 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8455+0+0 (secure 0 0 0) 0x7f33a00f0dd0 con 0x7f33b8108e10 2024-04-26T02:00:31.321 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.318+0000 7f33b5e00640 1 --2- 172.21.15.151:0/905202225 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f339c0a3d10 0x7f339c0a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:31.321 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.319+0000 7f33bf9f4740 1 -- 172.21.15.151:0/1163843194 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556712b6b9b0 msgr2=0x556712b6bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:31.321 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.319+0000 7f33bf9f4740 1 --2- 172.21.15.151:0/1163843194 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556712b6b9b0 0x556712b6bd90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f33b00117e0 tx=0x7f33b0011ca0 comp rx=0 tx=0).stop 2024-04-26T02:00:31.322 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.319+0000 7f33b5e00640 1 --2- 172.21.15.151:0/905202225 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f339c0a3d10 0x7f339c0a61d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x556712b6cf50 tx=0x7f33a8005f90 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:31.322 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.319+0000 7f33bf9f4740 1 -- 172.21.15.151:0/1163843194 shutdown_connections 2024-04-26T02:00:31.322 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.319+0000 7f33bf9f4740 1 --2- 172.21.15.151:0/1163843194 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556712b76b90 0x556712b78f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:31.322 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.319+0000 7f33bf9f4740 1 --2- 172.21.15.151:0/1163843194 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556712b6c2d0 0x556712b76650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:31.322 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.319+0000 7f33bf9f4740 1 --2- 172.21.15.151:0/1163843194 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556712b6b9b0 0x556712b6bd90 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:31.322 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.319+0000 7f33bf9f4740 1 -- 172.21.15.151:0/1163843194 >> 172.21.15.151:0/1163843194 conn(0x556712b67ec0 msgr2=0x556712b682a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:31.322 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.319+0000 7f33bf9f4740 1 -- 172.21.15.151:0/1163843194 shutdown_connections 2024-04-26T02:00:31.322 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.319+0000 7f33bf9f4740 1 -- 172.21.15.151:0/1163843194 wait complete. 2024-04-26T02:00:31.322 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.320+0000 7f33bf9f4740 1 -- start start 2024-04-26T02:00:31.322 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.320+0000 7f33bf9f4740 1 Processor -- start 2024-04-26T02:00:31.322 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.320+0000 7f33bf9f4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556712b6b9b0 0x556712b73b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:31.322 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.320+0000 7f33bf9f4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556712b6c2d0 0x556712b740c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:31.323 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.320+0000 7f33bf9f4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556712b76b90 0x556712b74600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:31.323 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.320+0000 7f33bf9f4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x556712b797c0 con 0x556712b6b9b0 2024-04-26T02:00:31.323 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.320+0000 7f33bf9f4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556712a8b310 con 0x556712b76b90 2024-04-26T02:00:31.323 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.320+0000 7f33bf9f4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556712b79940 con 0x556712b6c2d0 2024-04-26T02:00:31.323 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.320+0000 7f33bf9f4740 10 client.0 _ll_register_callbacks cb 0x556712a59cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:00:31.323 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.321+0000 7f33b5e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556712b6b9b0 0x556712b73b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:31.323 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.321+0000 7f33b5e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556712b6b9b0 0x556712b73b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:56312/0 (socket says 172.21.15.151:56312) 2024-04-26T02:00:31.323 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.321+0000 7f33b5e00640 1 -- 172.21.15.151:0/591004729 learned_addr learned my addr 172.21.15.151:0/591004729 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:00:31.323 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.321+0000 7f33ad400640 1 --2- 172.21.15.151:0/591004729 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556712b6c2d0 0x556712b740c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:31.323 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.321+0000 7f33b6800640 1 --2- 172.21.15.151:0/591004729 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556712b76b90 0x556712b74600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:31.323 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.321+0000 7f33b5e00640 1 -- 172.21.15.151:0/591004729 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556712b6c2d0 msgr2=0x556712b740c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:31.323 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.321+0000 7f33b5e00640 1 --2- 172.21.15.151:0/591004729 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556712b6c2d0 0x556712b740c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:31.323 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.321+0000 7f33b5e00640 1 -- 172.21.15.151:0/591004729 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556712b76b90 msgr2=0x556712b74600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:31.324 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.321+0000 7f33b5e00640 1 --2- 172.21.15.151:0/591004729 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556712b76b90 0x556712b74600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:31.324 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.321+0000 7f33b5e00640 1 -- 172.21.15.151:0/591004729 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567128b0620 con 0x556712b6b9b0 2024-04-26T02:00:31.324 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.321+0000 7f33b5e00640 1 --2- 172.21.15.151:0/591004729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556712b6b9b0 0x556712b73b80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f33a8013010 tx=0x7f33a8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1ff23a6c3022b2b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:31.324 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.321+0000 7f339b400640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-04-26T02:00:31.324 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.322+0000 7f339b400640 1 -- 172.21.15.151:0/591004729 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a801f640 con 0x556712b6b9b0 2024-04-26T02:00:31.324 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.322+0000 7f339b400640 1 -- 172.21.15.151:0/591004729 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8455+0+0 (secure 0 0 0) 0x7f33a801f7e0 con 0x556712b6b9b0 2024-04-26T02:00:31.324 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.322+0000 7f33bf9f4740 1 -- 172.21.15.151:0/591004729 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33b0003e20 con 0x556712b6b9b0 2024-04-26T02:00:31.325 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.322+0000 7f339b400640 1 -- 172.21.15.151:0/591004729 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f33a801f9e0 con 0x556712b6b9b0 2024-04-26T02:00:31.325 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.322+0000 7f33bf9f4740 10 client.15864 fetch_fsmap learned FSMap version 35 2024-04-26T02:00:31.325 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.322+0000 7f33bf9f4740 1 -- 172.21.15.151:0/591004729 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556712b85ae0 con 0x556712b6b9b0 2024-04-26T02:00:31.325 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.322+0000 7f339b400640 1 -- 172.21.15.151:0/591004729 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33b0003e20 con 0x556712b6b9b0 2024-04-26T02:00:31.327 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.323+0000 7f339b400640 1 -- 172.21.15.151:0/591004729 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 35) ==== 8338+0+0 (secure 0 0 0) 0x7f33a8027070 con 0x556712b6b9b0 2024-04-26T02:00:31.327 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.323+0000 7f33bf9f4740 10 client.15864 fetch_fsmap finished waiting for FSMap version 35 2024-04-26T02:00:31.327 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.323+0000 7f33bf9f4740 10 client.15864 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:00:31.327 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.323+0000 7f33bf9f4740 20 client.15864 populate_metadata read hostname 'smithi151' 2024-04-26T02:00:31.327 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.323+0000 7f33bf9f4740 1 --2- 172.21.15.151:0/591004729 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x556712b87150 0x556712b895d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:31.327 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.323+0000 7f33bf9f4740 4 client.15864 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:00:31.327 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.323+0000 7f33bf9f4740 1 -- 172.21.15.151:0/591004729 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556712ae58b0 con 0x556712b87150 2024-04-26T02:00:31.327 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.323+0000 7f33ad400640 1 --2- 172.21.15.151:0/591004729 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x556712b87150 0x556712b895d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:31.327 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.324+0000 7f339b400640 10 client.15864 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:00:31.327 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.324+0000 7f33ad400640 1 --2- 172.21.15.151:0/591004729 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x556712b87150 0x556712b895d0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86afc0de5e81afde server_cookie=2568a8cc5ea1d8a3 in_seq=0 out_seq=0 2024-04-26T02:00:31.327 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.324+0000 7f339b400640 1 -- 172.21.15.151:0/591004729 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556712ae58b0 con 0x556712b87150 2024-04-26T02:00:31.327 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.324+0000 7f339b400640 10 client.15864 handle_command_reply: tid=0 2024-04-26T02:00:31.327 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.324+0000 7f33bf9f4740 1 client.15864 shutdown 2024-04-26T02:00:31.328 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.325+0000 7f33bf9f4740 1 -- 172.21.15.151:0/591004729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556712b6b9b0 msgr2=0x556712b73b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:31.328 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.325+0000 7f33bf9f4740 1 --2- 172.21.15.151:0/591004729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556712b6b9b0 0x556712b73b80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f33a8013010 tx=0x7f33a8004730 comp rx=0 tx=0).stop 2024-04-26T02:00:31.328 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.325+0000 7f33bf9f4740 1 -- 172.21.15.151:0/591004729 shutdown_connections 2024-04-26T02:00:31.328 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.325+0000 7f33bf9f4740 1 --2- 172.21.15.151:0/591004729 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x556712b87150 0x556712b895d0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:31.328 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.325+0000 7f33bf9f4740 1 --2- 172.21.15.151:0/591004729 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556712b76b90 0x556712b74600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:31.328 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.325+0000 7f33bf9f4740 1 --2- 172.21.15.151:0/591004729 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556712b6c2d0 0x556712b740c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:31.328 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.325+0000 7f339b400640 0 client.15864 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:00:31.328 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.325+0000 7f33bf9f4740 1 --2- 172.21.15.151:0/591004729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556712b6b9b0 0x556712b73b80 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:31.328 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.325+0000 7f33bf9f4740 1 -- 172.21.15.151:0/591004729 >> 172.21.15.151:0/591004729 conn(0x556712b67ec0 msgr2=0x556712b6dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:31.328 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.325+0000 7f33bf9f4740 1 -- 172.21.15.151:0/591004729 shutdown_connections 2024-04-26T02:00:31.328 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.325+0000 7f33bf9f4740 1 -- 172.21.15.151:0/591004729 wait complete. 2024-04-26T02:00:31.328 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.325+0000 7f33bf9f4740 20 client.15864 trim_cache size 0 max 16384 2024-04-26T02:00:31.340 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.341+0000 7f33bf9f4740 1 Processor -- start 2024-04-26T02:00:31.340 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.341+0000 7f33bf9f4740 1 -- start start 2024-04-26T02:00:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.341+0000 7f33bf9f4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556712b6b9b0 0x556712b82680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.341+0000 7f33bf9f4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556712b6c2d0 0x556712b6e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.342+0000 7f33bf9f4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556712b76b90 0x556712b8eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.342+0000 7f33bf9f4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f33b81b67d0 con 0x556712b76b90 2024-04-26T02:00:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.342+0000 7f33bf9f4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f33b81b40a0 con 0x556712b6c2d0 2024-04-26T02:00:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.342+0000 7f33bf9f4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f33b81b1f10 con 0x556712b6b9b0 2024-04-26T02:00:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.342+0000 7f33ad400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556712b6c2d0 0x556712b6e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.342+0000 7f33b6800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556712b76b90 0x556712b8eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.342+0000 7f33ad400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556712b6c2d0 0x556712b6e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:59772/0 (socket says 172.21.15.151:59772) 2024-04-26T02:00:31.342 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.342+0000 7f33ad400640 1 -- 172.21.15.151:0/779911858 learned_addr learned my addr 172.21.15.151:0/779911858 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:00:31.342 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.342+0000 7f33ad400640 1 -- 172.21.15.151:0/779911858 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556712b6b9b0 msgr2=0x556712b82680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:31.342 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.342+0000 7f33ad400640 1 --2- 172.21.15.151:0/779911858 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556712b6b9b0 0x556712b82680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:31.342 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.342+0000 7f33ad400640 1 -- 172.21.15.151:0/779911858 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556712b76b90 msgr2=0x556712b8eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:31.342 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.342+0000 7f33ad400640 1 --2- 172.21.15.151:0/779911858 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556712b76b90 0x556712b8eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:31.342 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.342+0000 7f33ad400640 1 -- 172.21.15.151:0/779911858 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556712b677a0 con 0x556712b6c2d0 2024-04-26T02:00:31.346 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.342+0000 7f33ad400640 1 --2- 172.21.15.151:0/779911858 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556712b6c2d0 0x556712b6e370 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f33a00b9260 tx=0x7f33a0003f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0555fcb28b553be server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:31.346 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.343+0000 7f33a6600640 1 -- 172.21.15.151:0/779911858 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a0049040 con 0x556712b6c2d0 2024-04-26T02:00:31.346 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.343+0000 7f33a6600640 1 -- 172.21.15.151:0/779911858 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33a00c1100 con 0x556712b6c2d0 2024-04-26T02:00:31.346 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.343+0000 7f33a6600640 1 -- 172.21.15.151:0/779911858 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a00ee780 con 0x556712b6c2d0 2024-04-26T02:00:31.346 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.343+0000 7f33bf9f4740 1 -- 172.21.15.151:0/779911858 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556712b6c2d0 msgr2=0x556712b6e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:31.346 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.343+0000 7f33bf9f4740 1 --2- 172.21.15.151:0/779911858 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556712b6c2d0 0x556712b6e370 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f33a00b9260 tx=0x7f33a0003f90 comp rx=0 tx=0).stop 2024-04-26T02:00:31.346 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.343+0000 7f33bf9f4740 1 -- 172.21.15.151:0/779911858 shutdown_connections 2024-04-26T02:00:31.346 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.343+0000 7f33bf9f4740 1 --2- 172.21.15.151:0/779911858 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556712b76b90 0x556712b8eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:31.346 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.343+0000 7f33bf9f4740 1 --2- 172.21.15.151:0/779911858 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556712b6c2d0 0x556712b6e370 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:31.347 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.343+0000 7f33bf9f4740 1 --2- 172.21.15.151:0/779911858 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556712b6b9b0 0x556712b82680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:31.347 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.343+0000 7f33bf9f4740 1 -- 172.21.15.151:0/779911858 >> 172.21.15.151:0/779911858 conn(0x556712b67ec0 msgr2=0x556712b743d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:31.347 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.343+0000 7f33bf9f4740 1 -- 172.21.15.151:0/779911858 shutdown_connections 2024-04-26T02:00:31.347 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.344+0000 7f33bf9f4740 1 -- 172.21.15.151:0/779911858 wait complete. 2024-04-26T02:00:31.347 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.344+0000 7f33bf9f4740 1 -- start start 2024-04-26T02:00:31.347 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.344+0000 7f33bf9f4740 1 Processor -- start 2024-04-26T02:00:31.347 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.345+0000 7f33bf9f4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556712b6b9b0 0x556712b7edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:31.347 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.345+0000 7f33bf9f4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556712b6c2d0 0x556712b7c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:31.347 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.345+0000 7f33bf9f4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556712b76b90 0x556712b7c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:31.347 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.345+0000 7f33bf9f4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f33b81b2560 con 0x556712b76b90 2024-04-26T02:00:31.347 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.345+0000 7f33bf9f4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f33b81b39d0 con 0x556712b6c2d0 2024-04-26T02:00:31.347 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.345+0000 7f33bf9f4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f33b81b2c30 con 0x556712b6b9b0 2024-04-26T02:00:31.347 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.345+0000 7f33bf9f4740 10 client.0 _ll_register_callbacks cb 0x7f33a80044b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:00:31.348 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.345+0000 7f33b5e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556712b6b9b0 0x556712b7edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:31.348 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.345+0000 7f33b5e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556712b6b9b0 0x556712b7edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:41588/0 (socket says 172.21.15.151:41588) 2024-04-26T02:00:31.348 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.345+0000 7f33b5e00640 1 -- 172.21.15.151:0/363660243 learned_addr learned my addr 172.21.15.151:0/363660243 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:00:31.348 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.345+0000 7f33b6800640 1 --2- 172.21.15.151:0/363660243 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556712b76b90 0x556712b7c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:31.348 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.345+0000 7f33ad400640 1 --2- 172.21.15.151:0/363660243 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556712b6c2d0 0x556712b7c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:31.348 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.345+0000 7f33b5e00640 1 -- 172.21.15.151:0/363660243 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556712b6c2d0 msgr2=0x556712b7c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:31.348 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.345+0000 7f33b5e00640 1 --2- 172.21.15.151:0/363660243 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556712b6c2d0 0x556712b7c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:31.348 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.345+0000 7f33b5e00640 1 -- 172.21.15.151:0/363660243 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556712b76b90 msgr2=0x556712b7c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:31.348 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.345+0000 7f33b5e00640 1 --2- 172.21.15.151:0/363660243 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556712b76b90 0x556712b7c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:31.348 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.345+0000 7f33b5e00640 1 -- 172.21.15.151:0/363660243 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556712b7cbf0 con 0x556712b6b9b0 2024-04-26T02:00:31.348 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.345+0000 7f33b6800640 1 --2- 172.21.15.151:0/363660243 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556712b76b90 0x556712b7c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:00:31.348 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.346+0000 7f33b5e00640 1 --2- 172.21.15.151:0/363660243 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556712b6b9b0 0x556712b7edb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f33a8009710 tx=0x7f33a8003ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38c4b3da10e5488b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:31.348 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.346+0000 7f339b400640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-26T02:00:31.348 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.346+0000 7f339b400640 1 -- 172.21.15.151:0/363660243 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a8036030 con 0x556712b6b9b0 2024-04-26T02:00:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.346+0000 7f339b400640 1 -- 172.21.15.151:0/363660243 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8455+0+0 (secure 0 0 0) 0x7f33a801fb80 con 0x556712b6b9b0 2024-04-26T02:00:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.346+0000 7f33bf9f4740 1 -- 172.21.15.151:0/363660243 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33a00c1100 con 0x556712b6b9b0 2024-04-26T02:00:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.346+0000 7f339b400640 1 -- 172.21.15.151:0/363660243 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f33a8009e90 con 0x556712b6b9b0 2024-04-26T02:00:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.346+0000 7f33bf9f4740 10 client.26008 fetch_fsmap learned FSMap version 35 2024-04-26T02:00:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.346+0000 7f33bf9f4740 1 -- 172.21.15.151:0/363660243 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556712bed4f0 con 0x556712b6b9b0 2024-04-26T02:00:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.346+0000 7f339b400640 1 -- 172.21.15.151:0/363660243 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33a00c1100 con 0x556712b6b9b0 2024-04-26T02:00:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.347+0000 7f339b400640 1 -- 172.21.15.151:0/363660243 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 35) ==== 8338+0+0 (secure 0 0 0) 0x7f33a8027040 con 0x556712b6b9b0 2024-04-26T02:00:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.347+0000 7f33bf9f4740 10 client.26008 fetch_fsmap finished waiting for FSMap version 35 2024-04-26T02:00:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.347+0000 7f33bf9f4740 10 client.26008 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:00:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.347+0000 7f33bf9f4740 20 client.26008 populate_metadata read hostname 'smithi151' 2024-04-26T02:00:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.347+0000 7f33bf9f4740 1 --2- 172.21.15.151:0/363660243 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x556712bedcc0 0x556712bf00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.347+0000 7f33bf9f4740 4 client.26008 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T02:00:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.347+0000 7f33bf9f4740 1 -- 172.21.15.151:0/363660243 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556712adf320 con 0x556712bedcc0 2024-04-26T02:00:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.347+0000 7f33ad400640 1 --2- 172.21.15.151:0/363660243 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x556712bedcc0 0x556712bf00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:31.350 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.348+0000 7f339b400640 10 client.26008 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:00:31.350 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.347+0000 7f33ad400640 1 --2- 172.21.15.151:0/363660243 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x556712bedcc0 0x556712bf00e0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aaf3393de09c0277 server_cookie=adeffe7f169d4d51 in_seq=0 out_seq=0 2024-04-26T02:00:31.362 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.363+0000 7f339b400640 1 -- 172.21.15.151:0/363660243 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556712adf320 con 0x556712bedcc0 2024-04-26T02:00:31.362 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.363+0000 7f339b400640 10 client.26008 handle_command_reply: tid=0 2024-04-26T02:00:31.362 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.363+0000 7f33bf9f4740 1 client.26008 shutdown 2024-04-26T02:00:31.362 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.363+0000 7f33bf9f4740 1 -- 172.21.15.151:0/363660243 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556712b6b9b0 msgr2=0x556712b7edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:31.362 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.363+0000 7f33bf9f4740 1 --2- 172.21.15.151:0/363660243 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556712b6b9b0 0x556712b7edb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f33a8009710 tx=0x7f33a8003ed0 comp rx=0 tx=0).stop 2024-04-26T02:00:31.363 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.363+0000 7f33bf9f4740 1 -- 172.21.15.151:0/363660243 shutdown_connections 2024-04-26T02:00:31.363 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.363+0000 7f33bf9f4740 1 --2- 172.21.15.151:0/363660243 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x556712bedcc0 0x556712bf00e0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:31.363 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.363+0000 7f33bf9f4740 1 --2- 172.21.15.151:0/363660243 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556712b76b90 0x556712b7c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:31.363 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.363+0000 7f33bf9f4740 1 --2- 172.21.15.151:0/363660243 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556712b6c2d0 0x556712b7c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:31.363 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.363+0000 7f33bf9f4740 1 --2- 172.21.15.151:0/363660243 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556712b6b9b0 0x556712b7edb0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:31.363 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.363+0000 7f33bf9f4740 1 -- 172.21.15.151:0/363660243 >> 172.21.15.151:0/363660243 conn(0x556712b67ec0 msgr2=0x556712b73ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:31.363 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.363+0000 7f339b400640 0 client.26008 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:00:31.363 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.363+0000 7f33bf9f4740 1 -- 172.21.15.151:0/363660243 shutdown_connections 2024-04-26T02:00:31.363 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.363+0000 7f33bf9f4740 1 -- 172.21.15.151:0/363660243 wait complete. 2024-04-26T02:00:31.363 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.363+0000 7f33bf9f4740 20 client.26008 trim_cache size 0 max 16384 2024-04-26T02:00:31.363 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:{ 2024-04-26T02:00:31.364 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "message": "", 2024-04-26T02:00:31.364 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "return_code": 0 2024-04-26T02:00:31.364 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:} 2024-04-26T02:00:31.364 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.364+0000 7f33b7200640 1 -- 172.21.15.151:0/905202225 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f339c0a3d10 msgr2=0x7f339c0a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:31.365 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.364+0000 7f33b7200640 1 --2- 172.21.15.151:0/905202225 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f339c0a3d10 0x7f339c0a61d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x556712b6cf50 tx=0x7f33a8005f90 comp rx=0 tx=0).stop 2024-04-26T02:00:31.365 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.364+0000 7f33b7200640 1 -- 172.21.15.151:0/905202225 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33b8108e10 msgr2=0x7f33b81a4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:31.365 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.364+0000 7f33b7200640 1 --2- 172.21.15.151:0/905202225 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33b8108e10 0x7f33b81a4d00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f33a0005ab0 tx=0x7f33a0003b90 comp rx=0 tx=0).stop 2024-04-26T02:00:31.365 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.364+0000 7f33b7200640 1 -- 172.21.15.151:0/905202225 shutdown_connections 2024-04-26T02:00:31.365 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.364+0000 7f33b7200640 1 --2- 172.21.15.151:0/905202225 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f339c0a3d10 0x7f339c0a61d0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:31.365 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.364+0000 7f33b7200640 1 --2- 172.21.15.151:0/905202225 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f33b810e420 0x7f33b819e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:31.365 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.364+0000 7f33b7200640 1 --2- 172.21.15.151:0/905202225 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33b8108e10 0x7f33b81a4d00 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:31.365 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.364+0000 7f33b7200640 1 --2- 172.21.15.151:0/905202225 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33b8108490 0x7f33b81a47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:31.365 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.364+0000 7f33b7200640 1 -- 172.21.15.151:0/905202225 >> 172.21.15.151:0/905202225 conn(0x7f33b8077d30 msgr2=0x7f33b8078ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:31.365 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.365+0000 7f33b7200640 1 -- 172.21.15.151:0/905202225 shutdown_connections 2024-04-26T02:00:31.365 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:31.365+0000 7f33b7200640 1 -- 172.21.15.151:0/905202225 wait complete. 2024-04-26T02:00:32.723 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:00:32 smithi151 ceph-mon[28057]: pgmap v215: 97 pgs: 97 active+clean; 61 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.7 MiB/s wr, 284 op/s 2024-04-26T02:00:32.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:00:32 smithi172 ceph-mon[32785]: pgmap v215: 97 pgs: 97 active+clean; 61 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.7 MiB/s wr, 284 op/s 2024-04-26T02:00:32.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:00:32 smithi165 ceph-mon[32648]: pgmap v215: 97 pgs: 97 active+clean; 61 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.7 MiB/s wr, 284 op/s 2024-04-26T02:00:34.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:00:34 smithi172 ceph-mon[32785]: pgmap v216: 97 pgs: 97 active+clean; 57 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.7 MiB/s wr, 288 op/s 2024-04-26T02:00:34.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:00:34 smithi165 ceph-mon[32648]: pgmap v216: 97 pgs: 97 active+clean; 57 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.7 MiB/s wr, 288 op/s 2024-04-26T02:00:34.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:00:34 smithi151 ceph-mon[28057]: pgmap v216: 97 pgs: 97 active+clean; 57 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.7 MiB/s wr, 288 op/s 2024-04-26T02:00:36.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:00:36 smithi172 ceph-mon[32785]: pgmap v217: 97 pgs: 97 active+clean; 57 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.5 MiB/s wr, 247 op/s 2024-04-26T02:00:36.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:00:36 smithi165 ceph-mon[32648]: pgmap v217: 97 pgs: 97 active+clean; 57 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.5 MiB/s wr, 247 op/s 2024-04-26T02:00:36.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:00:36 smithi151 ceph-mon[28057]: pgmap v217: 97 pgs: 97 active+clean; 57 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.5 MiB/s wr, 247 op/s 2024-04-26T02:00:38.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:00:38 smithi172 ceph-mon[32785]: pgmap v218: 97 pgs: 97 active+clean; 57 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 943 KiB/s wr, 148 op/s 2024-04-26T02:00:38.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:00:38 smithi165 ceph-mon[32648]: pgmap v218: 97 pgs: 97 active+clean; 57 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 943 KiB/s wr, 148 op/s 2024-04-26T02:00:38.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:00:38 smithi151 ceph-mon[28057]: pgmap v218: 97 pgs: 97 active+clean; 57 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 943 KiB/s wr, 148 op/s 2024-04-26T02:00:40.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:00:40 smithi172 ceph-mon[32785]: pgmap v219: 97 pgs: 97 active+clean; 57 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 943 KiB/s wr, 151 op/s 2024-04-26T02:00:40.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:00:40 smithi165 ceph-mon[32648]: pgmap v219: 97 pgs: 97 active+clean; 57 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 943 KiB/s wr, 151 op/s 2024-04-26T02:00:40.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:00:40 smithi151 ceph-mon[28057]: pgmap v219: 97 pgs: 97 active+clean; 57 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 943 KiB/s wr, 151 op/s 2024-04-26T02:00:42.753 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:00:42 smithi151 ceph-mon[28057]: pgmap v220: 97 pgs: 97 active+clean; 57 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 18 KiB/s wr, 8 op/s 2024-04-26T02:00:42.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:00:42 smithi172 ceph-mon[32785]: pgmap v220: 97 pgs: 97 active+clean; 57 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 18 KiB/s wr, 8 op/s 2024-04-26T02:00:42.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:00:42 smithi165 ceph-mon[32648]: pgmap v220: 97 pgs: 97 active+clean; 57 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 18 KiB/s wr, 8 op/s 2024-04-26T02:00:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.509+0000 7fa3f8e00640 1 -- 172.21.15.151:0/1671640183 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa3f4109470 msgr2=0x7fa3f4111040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.509+0000 7fa3f8e00640 1 --2- 172.21.15.151:0/1671640183 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa3f4109470 0x7fa3f4111040 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec009ca0 tx=0x7fa3ec02a8c0 comp rx=0 tx=0).stop 2024-04-26T02:00:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.509+0000 7fa3f8e00640 1 -- 172.21.15.151:0/1671640183 shutdown_connections 2024-04-26T02:00:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.509+0000 7fa3f8e00640 1 --2- 172.21.15.151:0/1671640183 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa3f4109470 0x7fa3f4111040 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.509+0000 7fa3f8e00640 1 --2- 172.21.15.151:0/1671640183 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3f4104d50 0x7fa3f4108d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.509+0000 7fa3f8e00640 1 --2- 172.21.15.151:0/1671640183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3f41043a0 0x7fa3f4104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.509+0000 7fa3f8e00640 1 -- 172.21.15.151:0/1671640183 >> 172.21.15.151:0/1671640183 conn(0x7fa3f40ffc40 msgr2=0x7fa3f4102060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.510+0000 7fa3f8e00640 1 -- 172.21.15.151:0/1671640183 shutdown_connections 2024-04-26T02:00:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.510+0000 7fa3f8e00640 1 -- 172.21.15.151:0/1671640183 wait complete. 2024-04-26T02:00:44.510 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.511+0000 7fa3f8e00640 1 Processor -- start 2024-04-26T02:00:44.510 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.511+0000 7fa3f8e00640 1 -- start start 2024-04-26T02:00:44.510 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.511+0000 7fa3f8e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3f41043a0 0x7fa3f41a0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:44.511 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.511+0000 7fa3f8e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3f4109470 0x7fa3f41a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:44.511 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.511+0000 7fa3f8e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa3f419a640 0x7fa3f419aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:44.511 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.511+0000 7fa3f8e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f41143c0 con 0x7fa3f419a640 2024-04-26T02:00:44.511 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.511+0000 7fa3f8e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f4114240 con 0x7fa3f41043a0 2024-04-26T02:00:44.511 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.511+0000 7fa3f8e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f4114540 con 0x7fa3f4109470 2024-04-26T02:00:44.511 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.512+0000 7fa3f2800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa3f419a640 0x7fa3f419aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:44.511 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.512+0000 7fa3f1e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3f41043a0 0x7fa3f41a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:44.511 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.512+0000 7fa3f2800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa3f419a640 0x7fa3f419aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:55898/0 (socket says 172.21.15.151:55898) 2024-04-26T02:00:44.511 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.512+0000 7fa3f2800640 1 -- 172.21.15.151:0/3299066121 learned_addr learned my addr 172.21.15.151:0/3299066121 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:00:44.511 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.512+0000 7fa3f1400640 1 --2- 172.21.15.151:0/3299066121 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3f4109470 0x7fa3f41a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:44.512 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.512+0000 7fa3f1e00640 1 -- 172.21.15.151:0/3299066121 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3f4109470 msgr2=0x7fa3f41a0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:44.512 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.512+0000 7fa3f1e00640 1 --2- 172.21.15.151:0/3299066121 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3f4109470 0x7fa3f41a0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:44.512 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.512+0000 7fa3f1e00640 1 -- 172.21.15.151:0/3299066121 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa3f419a640 msgr2=0x7fa3f419aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.512+0000 7fa3f1e00640 1 --2- 172.21.15.151:0/3299066121 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa3f419a640 0x7fa3f419aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.512+0000 7fa3f1e00640 1 -- 172.21.15.151:0/3299066121 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3f419b240 con 0x7fa3f41043a0 2024-04-26T02:00:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.512+0000 7fa3f2800640 1 --2- 172.21.15.151:0/3299066121 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa3f419a640 0x7fa3f419aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:00:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.512+0000 7fa3f1e00640 1 --2- 172.21.15.151:0/3299066121 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3f41043a0 0x7fa3f41a0580 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa3e400e9f0 tx=0x7fa3e400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.513+0000 7fa3eaa00640 1 -- 172.21.15.151:0/3299066121 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e400ccc0 con 0x7fa3f41043a0 2024-04-26T02:00:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.513+0000 7fa3f8e00640 1 -- 172.21.15.151:0/3299066121 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3f41a7320 con 0x7fa3f41043a0 2024-04-26T02:00:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.513+0000 7fa3eaa00640 1 -- 172.21.15.151:0/3299066121 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3e40108e0 con 0x7fa3f41043a0 2024-04-26T02:00:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.513+0000 7fa3eaa00640 1 -- 172.21.15.151:0/3299066121 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e4018880 con 0x7fa3f41043a0 2024-04-26T02:00:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.513+0000 7fa3f8e00640 1 -- 172.21.15.151:0/3299066121 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3f41a7720 con 0x7fa3f41043a0 2024-04-26T02:00:44.517 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.517+0000 7fa3fbdd5740 1 Processor -- start 2024-04-26T02:00:44.517 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.517+0000 7fa3fbdd5740 1 -- start start 2024-04-26T02:00:44.517 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.518+0000 7fa3eaa00640 1 -- 172.21.15.151:0/3299066121 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7fa3e4010430 con 0x7fa3f41043a0 2024-04-26T02:00:44.517 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.518+0000 7fa3fbdd5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5645c4ae3820 0x5645c4ae3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:44.518 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.518+0000 7fa3fbdd5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645c4ae4140 0x5645c4aee4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:44.518 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.518+0000 7fa3fbdd5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645c4aeea00 0x5645c4af0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:44.518 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.518+0000 7fa3fbdd5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5645c4a49900 con 0x5645c4ae4140 2024-04-26T02:00:44.518 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.518+0000 7fa3fbdd5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5645c49efef0 con 0x5645c4ae3820 2024-04-26T02:00:44.518 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.518+0000 7fa3fbdd5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5645c4a458c0 con 0x5645c4aeea00 2024-04-26T02:00:44.518 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.518+0000 7fa3f1e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645c4ae4140 0x5645c4aee4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:44.518 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.518+0000 7fa3f1e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645c4ae4140 0x5645c4aee4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:55902/0 (socket says 172.21.15.151:55902) 2024-04-26T02:00:44.518 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.518+0000 7fa3f1e00640 1 -- 172.21.15.151:0/1574451226 learned_addr learned my addr 172.21.15.151:0/1574451226 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:00:44.518 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.518+0000 7fa3f2800640 1 --2- 172.21.15.151:0/1574451226 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5645c4ae3820 0x5645c4ae3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:44.518 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.518+0000 7fa3f1400640 1 --2- 172.21.15.151:0/1574451226 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645c4aeea00 0x5645c4af0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:44.518 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.519+0000 7fa3f1e00640 1 -- 172.21.15.151:0/1574451226 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645c4aeea00 msgr2=0x5645c4af0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:44.518 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.519+0000 7fa3f1e00640 1 --2- 172.21.15.151:0/1574451226 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645c4aeea00 0x5645c4af0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:44.518 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.519+0000 7fa3f1e00640 1 -- 172.21.15.151:0/1574451226 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5645c4ae3820 msgr2=0x5645c4ae3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:44.518 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.519+0000 7fa3f1e00640 1 --2- 172.21.15.151:0/1574451226 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5645c4ae3820 0x5645c4ae3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:44.518 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.519+0000 7fa3f1e00640 1 -- 172.21.15.151:0/1574451226 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645c475c210 con 0x5645c4ae4140 2024-04-26T02:00:44.523 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.519+0000 7fa3f1e00640 1 --2- 172.21.15.151:0/1574451226 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645c4ae4140 0x5645c4aee4c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa3e40ce1c0 tx=0x7fa3e40d9600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=487c3cf1871a8342 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:44.523 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.519+0000 7fa3df400640 1 -- 172.21.15.151:0/1574451226 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e40dd650 con 0x5645c4ae4140 2024-04-26T02:00:44.523 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.519+0000 7fa3df400640 1 -- 172.21.15.151:0/1574451226 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3e40e2070 con 0x5645c4ae4140 2024-04-26T02:00:44.523 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.519+0000 7fa3df400640 1 -- 172.21.15.151:0/1574451226 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e40dc860 con 0x5645c4ae4140 2024-04-26T02:00:44.523 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.519+0000 7fa3eaa00640 1 --2- 172.21.15.151:0/3299066121 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fa3d80a3dc0 0x7fa3d80a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:44.523 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.520+0000 7fa3eaa00640 1 -- 172.21.15.151:0/3299066121 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8455+0+0 (secure 0 0 0) 0x7fa3e40d0ac0 con 0x7fa3f41043a0 2024-04-26T02:00:44.523 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.520+0000 7fa3f2800640 1 --2- 172.21.15.151:0/3299066121 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fa3d80a3dc0 0x7fa3d80a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:44.523 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.520+0000 7fa3fbdd5740 1 -- 172.21.15.151:0/1574451226 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645c4ae4140 msgr2=0x5645c4aee4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:44.523 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.520+0000 7fa3fbdd5740 1 --2- 172.21.15.151:0/1574451226 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645c4ae4140 0x5645c4aee4c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa3e40ce1c0 tx=0x7fa3e40d9600 comp rx=0 tx=0).stop 2024-04-26T02:00:44.523 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.520+0000 7fa3fbdd5740 1 -- 172.21.15.151:0/1574451226 shutdown_connections 2024-04-26T02:00:44.523 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.520+0000 7fa3fbdd5740 1 --2- 172.21.15.151:0/1574451226 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645c4aeea00 0x5645c4af0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:44.524 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.520+0000 7fa3fbdd5740 1 --2- 172.21.15.151:0/1574451226 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645c4ae4140 0x5645c4aee4c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:44.524 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.520+0000 7fa3fbdd5740 1 --2- 172.21.15.151:0/1574451226 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5645c4ae3820 0x5645c4ae3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:44.524 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.520+0000 7fa3fbdd5740 1 -- 172.21.15.151:0/1574451226 >> 172.21.15.151:0/1574451226 conn(0x5645c4adfd30 msgr2=0x5645c4ae0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:44.524 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.520+0000 7fa3fbdd5740 1 -- 172.21.15.151:0/1574451226 shutdown_connections 2024-04-26T02:00:44.524 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.520+0000 7fa3fbdd5740 1 -- 172.21.15.151:0/1574451226 wait complete. 2024-04-26T02:00:44.524 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.521+0000 7fa3f2800640 1 --2- 172.21.15.151:0/3299066121 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fa3d80a3dc0 0x7fa3d80a6280 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec005e30 tx=0x7fa3ec005c50 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:44.524 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.521+0000 7fa3fbdd5740 1 -- start start 2024-04-26T02:00:44.524 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.521+0000 7fa3fbdd5740 1 Processor -- start 2024-04-26T02:00:44.524 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.522+0000 7fa3fbdd5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5645c4ae3820 0x5645c4aeb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:44.524 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.522+0000 7fa3fbdd5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645c4ae4140 0x5645c4aebec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:44.524 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.522+0000 7fa3fbdd5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645c4aeea00 0x5645c4aec400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:44.524 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.522+0000 7fa3fbdd5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5645c4af15d0 con 0x5645c4ae4140 2024-04-26T02:00:44.524 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.522+0000 7fa3fbdd5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5645c4a03310 con 0x5645c4ae3820 2024-04-26T02:00:44.524 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.522+0000 7fa3fbdd5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5645c4af1750 con 0x5645c4aeea00 2024-04-26T02:00:44.525 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.522+0000 7fa3fbdd5740 10 client.0 _ll_register_callbacks cb 0x5645c49d1cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:00:44.525 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.522+0000 7fa3f1e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5645c4ae3820 0x5645c4aeb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:44.525 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.522+0000 7fa3f1400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645c4ae4140 0x5645c4aebec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:44.525 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.522+0000 7fa3f1e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5645c4ae3820 0x5645c4aeb980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:40362/0 (socket says 172.21.15.151:40362) 2024-04-26T02:00:44.525 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.522+0000 7fa3f1e00640 1 -- 172.21.15.151:0/1364251513 learned_addr learned my addr 172.21.15.151:0/1364251513 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:00:44.525 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.522+0000 7fa3f2800640 1 --2- 172.21.15.151:0/1364251513 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645c4aeea00 0x5645c4aec400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:44.525 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.522+0000 7fa3f1e00640 1 -- 172.21.15.151:0/1364251513 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645c4aeea00 msgr2=0x5645c4aec400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:44.525 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.522+0000 7fa3f1e00640 1 --2- 172.21.15.151:0/1364251513 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645c4aeea00 0x5645c4aec400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:44.525 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.522+0000 7fa3f1e00640 1 -- 172.21.15.151:0/1364251513 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645c4ae4140 msgr2=0x5645c4aebec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:44.525 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.522+0000 7fa3f1e00640 1 --2- 172.21.15.151:0/1364251513 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645c4ae4140 0x5645c4aebec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:44.525 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.522+0000 7fa3f1e00640 1 -- 172.21.15.151:0/1364251513 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645c4828620 con 0x5645c4ae3820 2024-04-26T02:00:44.525 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.523+0000 7fa3f1e00640 1 --2- 172.21.15.151:0/1364251513 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5645c4ae3820 0x5645c4aeb980 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa3e40ce300 tx=0x7fa3e40d0dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c8b0844c56dcd01 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:44.525 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.523+0000 7fa3d3e00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T02:00:44.525 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.523+0000 7fa3d3e00640 1 -- 172.21.15.151:0/1364251513 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e4098df0 con 0x5645c4ae3820 2024-04-26T02:00:44.526 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.523+0000 7fa3d3e00640 1 -- 172.21.15.151:0/1364251513 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8455+0+0 (secure 0 0 0) 0x7fa3e4010a90 con 0x5645c4ae3820 2024-04-26T02:00:44.526 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.523+0000 7fa3fbdd5740 1 -- 172.21.15.151:0/1364251513 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3e40e2070 con 0x5645c4ae3820 2024-04-26T02:00:44.526 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.523+0000 7fa3d3e00640 1 -- 172.21.15.151:0/1364251513 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa3e40dc3b0 con 0x5645c4ae3820 2024-04-26T02:00:44.526 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.523+0000 7fa3fbdd5740 10 client.25988 fetch_fsmap learned FSMap version 35 2024-04-26T02:00:44.526 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.523+0000 7fa3fbdd5740 1 -- 172.21.15.151:0/1364251513 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5645c4afd850 con 0x5645c4ae3820 2024-04-26T02:00:44.526 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.523+0000 7fa3d3e00640 1 -- 172.21.15.151:0/1364251513 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3e40e2070 con 0x5645c4ae3820 2024-04-26T02:00:44.526 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.524+0000 7fa3d3e00640 1 -- 172.21.15.151:0/1364251513 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 35) ==== 8338+0+0 (secure 0 0 0) 0x7fa3e40991c0 con 0x5645c4ae3820 2024-04-26T02:00:44.526 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.524+0000 7fa3fbdd5740 10 client.25988 fetch_fsmap finished waiting for FSMap version 35 2024-04-26T02:00:44.526 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.524+0000 7fa3fbdd5740 10 client.25988 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:00:44.526 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.524+0000 7fa3fbdd5740 20 client.25988 populate_metadata read hostname 'smithi151' 2024-04-26T02:00:44.526 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.524+0000 7fa3fbdd5740 1 --2- 172.21.15.151:0/1364251513 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5645c4afeba0 0x5645c4b00f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:44.526 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.524+0000 7fa3fbdd5740 4 client.25988 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:00:44.526 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.524+0000 7fa3fbdd5740 1 -- 172.21.15.151:0/1364251513 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645c4a5d8b0 con 0x5645c4afeba0 2024-04-26T02:00:44.526 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.524+0000 7fa3f1400640 1 --2- 172.21.15.151:0/1364251513 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5645c4afeba0 0x5645c4b00f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:44.527 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.525+0000 7fa3f1400640 1 --2- 172.21.15.151:0/1364251513 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5645c4afeba0 0x5645c4b00f90 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edb1dea63a5d13ef server_cookie=c28157e76961f5c6 in_seq=0 out_seq=0 2024-04-26T02:00:44.527 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.525+0000 7fa3d3e00640 10 client.25988 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:00:44.527 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.525+0000 7fa3d3e00640 1 -- 172.21.15.151:0/1364251513 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5645c4a5d8b0 con 0x5645c4afeba0 2024-04-26T02:00:44.528 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.525+0000 7fa3d3e00640 10 client.25988 handle_command_reply: tid=0 2024-04-26T02:00:44.528 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.525+0000 7fa3fbdd5740 1 client.25988 shutdown 2024-04-26T02:00:44.528 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.526+0000 7fa3fbdd5740 1 -- 172.21.15.151:0/1364251513 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5645c4ae3820 msgr2=0x5645c4aeb980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:44.528 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.526+0000 7fa3fbdd5740 1 --2- 172.21.15.151:0/1364251513 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5645c4ae3820 0x5645c4aeb980 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa3e40ce300 tx=0x7fa3e40d0dc0 comp rx=0 tx=0).stop 2024-04-26T02:00:44.528 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.526+0000 7fa3fbdd5740 1 -- 172.21.15.151:0/1364251513 shutdown_connections 2024-04-26T02:00:44.528 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.526+0000 7fa3fbdd5740 1 --2- 172.21.15.151:0/1364251513 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5645c4afeba0 0x5645c4b00f90 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:44.528 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.526+0000 7fa3fbdd5740 1 --2- 172.21.15.151:0/1364251513 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645c4aeea00 0x5645c4aec400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:44.528 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.526+0000 7fa3fbdd5740 1 --2- 172.21.15.151:0/1364251513 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645c4ae4140 0x5645c4aebec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:44.528 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.526+0000 7fa3d3e00640 0 client.25988 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:00:44.528 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.526+0000 7fa3fbdd5740 1 --2- 172.21.15.151:0/1364251513 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5645c4ae3820 0x5645c4aeb980 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:44.528 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.526+0000 7fa3fbdd5740 1 -- 172.21.15.151:0/1364251513 >> 172.21.15.151:0/1364251513 conn(0x5645c4adfd30 msgr2=0x5645c4ae5ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:44.528 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.526+0000 7fa3fbdd5740 1 -- 172.21.15.151:0/1364251513 shutdown_connections 2024-04-26T02:00:44.528 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.526+0000 7fa3fbdd5740 1 -- 172.21.15.151:0/1364251513 wait complete. 2024-04-26T02:00:44.528 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.526+0000 7fa3fbdd5740 20 client.25988 trim_cache size 0 max 16384 2024-04-26T02:00:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.542+0000 7fa3fbdd5740 1 Processor -- start 2024-04-26T02:00:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.542+0000 7fa3fbdd5740 1 -- start start 2024-04-26T02:00:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.542+0000 7fa3fbdd5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645c4ae3820 0x5645c4afa490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.542+0000 7fa3fbdd5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5645c4ae4140 0x5645c4aec050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.542+0000 7fa3fbdd5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645c4aeea00 0x5645c4b065e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.542+0000 7fa3fbdd5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f41b2560 con 0x5645c4ae3820 2024-04-26T02:00:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.542+0000 7fa3fbdd5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f41afe30 con 0x5645c4ae4140 2024-04-26T02:00:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.542+0000 7fa3fbdd5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f41adca0 con 0x5645c4aeea00 2024-04-26T02:00:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.543+0000 7fa3f1e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645c4ae3820 0x5645c4afa490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.543+0000 7fa3f1e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645c4ae3820 0x5645c4afa490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:55926/0 (socket says 172.21.15.151:55926) 2024-04-26T02:00:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.543+0000 7fa3f1400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5645c4ae4140 0x5645c4aec050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.543+0000 7fa3f1e00640 1 -- 172.21.15.151:0/3653614285 learned_addr learned my addr 172.21.15.151:0/3653614285 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:00:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.543+0000 7fa3f2800640 1 --2- 172.21.15.151:0/3653614285 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645c4aeea00 0x5645c4b065e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.543+0000 7fa3f1e00640 1 -- 172.21.15.151:0/3653614285 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645c4aeea00 msgr2=0x5645c4b065e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.543+0000 7fa3f1e00640 1 --2- 172.21.15.151:0/3653614285 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645c4aeea00 0x5645c4b065e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.543+0000 7fa3f1e00640 1 -- 172.21.15.151:0/3653614285 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5645c4ae4140 msgr2=0x5645c4aec050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.543+0000 7fa3f1e00640 1 --2- 172.21.15.151:0/3653614285 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5645c4ae4140 0x5645c4aec050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.543+0000 7fa3f1e00640 1 -- 172.21.15.151:0/3653614285 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645c4adf860 con 0x5645c4ae3820 2024-04-26T02:00:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.543+0000 7fa3f1e00640 1 --2- 172.21.15.151:0/3653614285 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645c4ae3820 0x5645c4afa490 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa3e40d4c20 tx=0x7fa3e40d0e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2221b8cb9f5033a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.544+0000 7fa3dea00640 1 -- 172.21.15.151:0/3653614285 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e409c4b0 con 0x5645c4ae3820 2024-04-26T02:00:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.544+0000 7fa3dea00640 1 -- 172.21.15.151:0/3653614285 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3e40e2030 con 0x5645c4ae3820 2024-04-26T02:00:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.544+0000 7fa3dea00640 1 -- 172.21.15.151:0/3653614285 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e409c690 con 0x5645c4ae3820 2024-04-26T02:00:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.544+0000 7fa3fbdd5740 1 -- 172.21.15.151:0/3653614285 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645c4ae3820 msgr2=0x5645c4afa490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.544+0000 7fa3fbdd5740 1 --2- 172.21.15.151:0/3653614285 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645c4ae3820 0x5645c4afa490 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa3e40d4c20 tx=0x7fa3e40d0e30 comp rx=0 tx=0).stop 2024-04-26T02:00:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.544+0000 7fa3fbdd5740 1 -- 172.21.15.151:0/3653614285 shutdown_connections 2024-04-26T02:00:44.545 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.544+0000 7fa3fbdd5740 1 --2- 172.21.15.151:0/3653614285 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645c4aeea00 0x5645c4b065e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:44.545 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.544+0000 7fa3fbdd5740 1 --2- 172.21.15.151:0/3653614285 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5645c4ae4140 0x5645c4aec050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:44.545 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.544+0000 7fa3fbdd5740 1 --2- 172.21.15.151:0/3653614285 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645c4ae3820 0x5645c4afa490 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:44.545 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.544+0000 7fa3fbdd5740 1 -- 172.21.15.151:0/3653614285 >> 172.21.15.151:0/3653614285 conn(0x5645c4adfd30 msgr2=0x5645c4aeb940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:44.545 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.545+0000 7fa3fbdd5740 1 -- 172.21.15.151:0/3653614285 shutdown_connections 2024-04-26T02:00:44.545 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.545+0000 7fa3fbdd5740 1 -- 172.21.15.151:0/3653614285 wait complete. 2024-04-26T02:00:44.545 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.545+0000 7fa3fbdd5740 1 -- start start 2024-04-26T02:00:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.545+0000 7fa3fbdd5740 1 Processor -- start 2024-04-26T02:00:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.546+0000 7fa3fbdd5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5645c4ae4140 0x5645c4af4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.546+0000 7fa3fbdd5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645c4aeea00 0x5645c4af4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.546+0000 7fa3fbdd5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645c4af4fa0 0x5645c4b63e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.546+0000 7fa3fbdd5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f41ae2f0 con 0x5645c4aeea00 2024-04-26T02:00:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.546+0000 7fa3fbdd5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f41af760 con 0x5645c4ae4140 2024-04-26T02:00:44.548 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.546+0000 7fa3fbdd5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f41ae9c0 con 0x5645c4af4fa0 2024-04-26T02:00:44.548 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.546+0000 7fa3fbdd5740 10 client.0 _ll_register_callbacks cb 0x7fa3e40d4b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:00:44.548 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.546+0000 7fa3f1e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5645c4ae4140 0x5645c4af4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:44.548 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.546+0000 7fa3f1e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5645c4ae4140 0x5645c4af4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:40392/0 (socket says 172.21.15.151:40392) 2024-04-26T02:00:44.548 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.546+0000 7fa3f1e00640 1 -- 172.21.15.151:0/4185356259 learned_addr learned my addr 172.21.15.151:0/4185356259 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:00:44.548 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.546+0000 7fa3f1400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645c4aeea00 0x5645c4af4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:44.548 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.546+0000 7fa3f2800640 1 --2- 172.21.15.151:0/4185356259 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645c4af4fa0 0x5645c4b63e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:44.548 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.546+0000 7fa3f1e00640 1 -- 172.21.15.151:0/4185356259 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645c4af4fa0 msgr2=0x5645c4b63e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:44.548 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.546+0000 7fa3f1e00640 1 --2- 172.21.15.151:0/4185356259 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645c4af4fa0 0x5645c4b63e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:44.548 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.547+0000 7fa3f1e00640 1 -- 172.21.15.151:0/4185356259 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645c4aeea00 msgr2=0x5645c4af4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:44.548 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.547+0000 7fa3f1e00640 1 --2- 172.21.15.151:0/4185356259 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645c4aeea00 0x5645c4af4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:44.548 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.547+0000 7fa3f1e00640 1 -- 172.21.15.151:0/4185356259 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645c4b64450 con 0x5645c4ae4140 2024-04-26T02:00:44.548 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.547+0000 7fa3f1400640 1 --2- 172.21.15.151:0/4185356259 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645c4aeea00 0x5645c4af4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:00:44.548 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.547+0000 7fa3f1e00640 1 --2- 172.21.15.151:0/4185356259 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5645c4ae4140 0x5645c4af4520 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa3e40ce300 tx=0x7fa3e409cb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e60ec6880fe0b96e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:44.549 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.547+0000 7fa3d3e00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T02:00:44.549 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.547+0000 7fa3d3e00640 1 -- 172.21.15.151:0/4185356259 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e409c920 con 0x5645c4ae4140 2024-04-26T02:00:44.549 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.547+0000 7fa3d3e00640 1 -- 172.21.15.151:0/4185356259 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8455+0+0 (secure 0 0 0) 0x7fa3e40a5480 con 0x5645c4ae4140 2024-04-26T02:00:44.549 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.547+0000 7fa3fbdd5740 1 -- 172.21.15.151:0/4185356259 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3e40e2030 con 0x5645c4ae4140 2024-04-26T02:00:44.549 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.547+0000 7fa3d3e00640 1 -- 172.21.15.151:0/4185356259 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa3e409c690 con 0x5645c4ae4140 2024-04-26T02:00:44.549 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.547+0000 7fa3fbdd5740 10 client.25994 fetch_fsmap learned FSMap version 35 2024-04-26T02:00:44.549 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.547+0000 7fa3fbdd5740 1 -- 172.21.15.151:0/4185356259 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5645c4b64450 con 0x5645c4ae4140 2024-04-26T02:00:44.549 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.547+0000 7fa3d3e00640 1 -- 172.21.15.151:0/4185356259 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3e40e2030 con 0x5645c4ae4140 2024-04-26T02:00:44.549 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.548+0000 7fa3d3e00640 1 -- 172.21.15.151:0/4185356259 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 35) ==== 8338+0+0 (secure 0 0 0) 0x7fa3e409cdb0 con 0x5645c4ae4140 2024-04-26T02:00:44.549 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.548+0000 7fa3fbdd5740 10 client.25994 fetch_fsmap finished waiting for FSMap version 35 2024-04-26T02:00:44.549 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.548+0000 7fa3fbdd5740 10 client.25994 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:00:44.549 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.548+0000 7fa3fbdd5740 20 client.25994 populate_metadata read hostname 'smithi151' 2024-04-26T02:00:44.549 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.548+0000 7fa3fbdd5740 1 --2- 172.21.15.151:0/4185356259 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5645c4b659e0 0x5645c4b67dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:44.549 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.548+0000 7fa3fbdd5740 4 client.25994 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T02:00:44.550 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.548+0000 7fa3fbdd5740 1 -- 172.21.15.151:0/4185356259 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5645c4a57320 con 0x5645c4b659e0 2024-04-26T02:00:44.550 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.548+0000 7fa3f1400640 1 --2- 172.21.15.151:0/4185356259 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5645c4b659e0 0x5645c4b67dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:44.550 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.548+0000 7fa3f1400640 1 --2- 172.21.15.151:0/4185356259 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5645c4b659e0 0x5645c4b67dd0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38b05c6da247bf3f server_cookie=336234901b4999a1 in_seq=0 out_seq=0 2024-04-26T02:00:44.550 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.548+0000 7fa3d3e00640 10 client.25994 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:00:44.559 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.560+0000 7fa3d3e00640 1 -- 172.21.15.151:0/4185356259 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5645c4a57320 con 0x5645c4b659e0 2024-04-26T02:00:44.559 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.560+0000 7fa3d3e00640 10 client.25994 handle_command_reply: tid=0 2024-04-26T02:00:44.559 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.560+0000 7fa3fbdd5740 1 client.25994 shutdown 2024-04-26T02:00:44.559 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.560+0000 7fa3fbdd5740 1 -- 172.21.15.151:0/4185356259 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5645c4ae4140 msgr2=0x5645c4af4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.560+0000 7fa3fbdd5740 1 --2- 172.21.15.151:0/4185356259 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5645c4ae4140 0x5645c4af4520 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa3e40ce300 tx=0x7fa3e409cb50 comp rx=0 tx=0).stop 2024-04-26T02:00:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.560+0000 7fa3fbdd5740 1 -- 172.21.15.151:0/4185356259 shutdown_connections 2024-04-26T02:00:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.560+0000 7fa3fbdd5740 1 --2- 172.21.15.151:0/4185356259 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5645c4b659e0 0x5645c4b67dd0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.560+0000 7fa3fbdd5740 1 --2- 172.21.15.151:0/4185356259 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645c4af4fa0 0x5645c4b63e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.560+0000 7fa3fbdd5740 1 --2- 172.21.15.151:0/4185356259 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645c4aeea00 0x5645c4af4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.560+0000 7fa3fbdd5740 1 --2- 172.21.15.151:0/4185356259 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5645c4ae4140 0x5645c4af4520 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.560+0000 7fa3fbdd5740 1 -- 172.21.15.151:0/4185356259 >> 172.21.15.151:0/4185356259 conn(0x5645c4adfd30 msgr2=0x5645c4aeb940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.560+0000 7fa3d3e00640 0 client.25994 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:00:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.560+0000 7fa3fbdd5740 1 -- 172.21.15.151:0/4185356259 shutdown_connections 2024-04-26T02:00:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.560+0000 7fa3fbdd5740 1 -- 172.21.15.151:0/4185356259 wait complete. 2024-04-26T02:00:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.560+0000 7fa3fbdd5740 20 client.25994 trim_cache size 0 max 16384 2024-04-26T02:00:44.561 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:{ 2024-04-26T02:00:44.561 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "message": "", 2024-04-26T02:00:44.561 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "return_code": 0 2024-04-26T02:00:44.561 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:} 2024-04-26T02:00:44.561 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.561+0000 7fa3f8e00640 1 -- 172.21.15.151:0/3299066121 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fa3d80a3dc0 msgr2=0x7fa3d80a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:44.562 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.561+0000 7fa3f8e00640 1 --2- 172.21.15.151:0/3299066121 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fa3d80a3dc0 0x7fa3d80a6280 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec005e30 tx=0x7fa3ec005c50 comp rx=0 tx=0).stop 2024-04-26T02:00:44.562 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.561+0000 7fa3f8e00640 1 -- 172.21.15.151:0/3299066121 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3f41043a0 msgr2=0x7fa3f41a0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:44.562 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.561+0000 7fa3f8e00640 1 --2- 172.21.15.151:0/3299066121 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3f41043a0 0x7fa3f41a0580 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa3e400e9f0 tx=0x7fa3e400eeb0 comp rx=0 tx=0).stop 2024-04-26T02:00:44.562 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.561+0000 7fa3f8e00640 1 -- 172.21.15.151:0/3299066121 shutdown_connections 2024-04-26T02:00:44.562 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.561+0000 7fa3f8e00640 1 --2- 172.21.15.151:0/3299066121 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fa3d80a3dc0 0x7fa3d80a6280 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:44.562 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.561+0000 7fa3f8e00640 1 --2- 172.21.15.151:0/3299066121 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa3f419a640 0x7fa3f419aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:44.562 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.561+0000 7fa3f8e00640 1 --2- 172.21.15.151:0/3299066121 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3f4109470 0x7fa3f41a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:44.562 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.561+0000 7fa3f8e00640 1 --2- 172.21.15.151:0/3299066121 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3f41043a0 0x7fa3f41a0580 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:44.562 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.561+0000 7fa3f8e00640 1 -- 172.21.15.151:0/3299066121 >> 172.21.15.151:0/3299066121 conn(0x7fa3f40ffc40 msgr2=0x7fa3f4100800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:44.562 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.562+0000 7fa3f8e00640 1 -- 172.21.15.151:0/3299066121 shutdown_connections 2024-04-26T02:00:44.562 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:44.562+0000 7fa3f8e00640 1 -- 172.21.15.151:0/3299066121 wait complete. 2024-04-26T02:00:44.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:00:44 smithi172 ceph-mon[32785]: pgmap v221: 97 pgs: 97 active+clean; 57 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-04-26T02:00:44.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:00:44 smithi165 ceph-mon[32648]: pgmap v221: 97 pgs: 97 active+clean; 57 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-04-26T02:00:44.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:00:44 smithi151 ceph-mon[28057]: pgmap v221: 97 pgs: 97 active+clean; 57 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-04-26T02:00:45.485 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T02:00:45.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:00:45 smithi172 ceph-mon[32785]: pgmap v222: 97 pgs: 97 active+clean; 57 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 12 KiB/s wr, 8 op/s 2024-04-26T02:00:45.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:00:45 smithi165 ceph-mon[32648]: pgmap v222: 97 pgs: 97 active+clean; 57 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 12 KiB/s wr, 8 op/s 2024-04-26T02:00:45.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:00:45 smithi151 ceph-mon[28057]: pgmap v222: 97 pgs: 97 active+clean; 57 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 12 KiB/s wr, 8 op/s 2024-04-26T02:00:45.931 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:00:48.225 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:00:47 smithi151 ceph-mon[28057]: pgmap v223: 97 pgs: 97 active+clean; 57 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.7 KiB/s wr, 5 op/s 2024-04-26T02:00:48.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:48.280+0000 7f2298000700 1 -- 172.21.15.151:0/485208317 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f229006a750 msgr2=0x7f229006ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:48.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:48.280+0000 7f2298000700 1 --2- 172.21.15.151:0/485208317 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f229006a750 0x7f229006ab30 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2284009990 tx=0x7f2284009ca0 comp rx=0 tx=0).stop 2024-04-26T02:00:48.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:48.280+0000 7f2298000700 1 -- 172.21.15.151:0/485208317 shutdown_connections 2024-04-26T02:00:48.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:48.280+0000 7f2298000700 1 --2- 172.21.15.151:0/485208317 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f229010f4f0 0x7f22901118e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:48.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:48.280+0000 7f2298000700 1 --2- 172.21.15.151:0/485208317 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f229006a750 0x7f229006ab30 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:48.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:48.280+0000 7f2298000700 1 --2- 172.21.15.151:0/485208317 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f229006b070 0x7f229010efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:48.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:48.280+0000 7f2298000700 1 -- 172.21.15.151:0/485208317 >> 172.21.15.151:0/485208317 conn(0x7f22900788f0 msgr2=0x7f2290078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:48.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:48.280+0000 7f2298000700 1 -- 172.21.15.151:0/485208317 shutdown_connections 2024-04-26T02:00:48.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:48.280+0000 7f2298000700 1 -- 172.21.15.151:0/485208317 wait complete. 2024-04-26T02:00:48.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:48.281+0000 7f2298000700 1 Processor -- start 2024-04-26T02:00:48.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:48.281+0000 7f2298000700 1 -- start start 2024-04-26T02:00:48.281 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:48.282+0000 7f2298000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f229006a750 0x7f229019b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:48.281 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:48.282+0000 7f2298000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f229006b070 0x7f229019b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:48.281 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:48.282+0000 7f2298000700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f229010f4f0 0x7f229019fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:48.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:48.282+0000 7f2298000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2290114e50 con 0x7f229006b070 2024-04-26T02:00:48.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:48.282+0000 7f2298000700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2290114cd0 con 0x7f229006a750 2024-04-26T02:00:48.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:48.282+0000 7f2298000700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2290114fd0 con 0x7f229010f4f0 2024-04-26T02:00:48.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:48.282+0000 7f2294c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f229006b070 0x7f229019b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:48.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:48.282+0000 7f2295600700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f229006a750 0x7f229019b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:48.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:48.282+0000 7f2294c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f229006b070 0x7f229019b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:55962/0 (socket says 172.21.15.151:55962) 2024-04-26T02:00:48.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:48.282+0000 7f2294c00700 1 -- 172.21.15.151:0/2501524007 learned_addr learned my addr 172.21.15.151:0/2501524007 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:00:48.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:48.283+0000 7f2296000700 1 --2- 172.21.15.151:0/2501524007 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f229010f4f0 0x7f229019fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:48.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:48.283+0000 7f2295600700 1 -- 172.21.15.151:0/2501524007 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f229010f4f0 msgr2=0x7f229019fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:48.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:48.283+0000 7f2295600700 1 --2- 172.21.15.151:0/2501524007 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f229010f4f0 0x7f229019fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:48.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:48.283+0000 7f2295600700 1 -- 172.21.15.151:0/2501524007 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f229006b070 msgr2=0x7f229019b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:48.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:48.283+0000 7f2295600700 1 --2- 172.21.15.151:0/2501524007 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f229006b070 0x7f229019b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:48.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:48.283+0000 7f2295600700 1 -- 172.21.15.151:0/2501524007 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22901a03b0 con 0x7f229006a750 2024-04-26T02:00:48.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:48.283+0000 7f2294c00700 1 --2- 172.21.15.151:0/2501524007 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f229006b070 0x7f229019b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:00:48.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:48.283+0000 7f2295600700 1 --2- 172.21.15.151:0/2501524007 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f229006a750 0x7f229019b3a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2284000c00 tx=0x7f22840047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:48.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:48.284+0000 7f228e000700 1 -- 172.21.15.151:0/2501524007 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2284026070 con 0x7f229006a750 2024-04-26T02:00:48.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:48.284+0000 7f228e000700 1 -- 172.21.15.151:0/2501524007 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2284021410 con 0x7f229006a750 2024-04-26T02:00:48.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:48.284+0000 7f2298000700 1 -- 172.21.15.151:0/2501524007 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22901a0640 con 0x7f229006a750 2024-04-26T02:00:48.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:48.284+0000 7f228e000700 1 -- 172.21.15.151:0/2501524007 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2284017650 con 0x7f229006a750 2024-04-26T02:00:48.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:48.284+0000 7f2298000700 1 -- 172.21.15.151:0/2501524007 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22901a7f20 con 0x7f229006a750 2024-04-26T02:00:48.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:48.285+0000 7f2298000700 1 -- 172.21.15.151:0/2501524007 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2290101f60 con 0x7f229006a750 2024-04-26T02:00:48.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:48.287+0000 7f228e000700 1 -- 172.21.15.151:0/2501524007 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f2284029070 con 0x7f229006a750 2024-04-26T02:00:48.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:00:47 smithi172 ceph-mon[32785]: pgmap v223: 97 pgs: 97 active+clean; 57 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.7 KiB/s wr, 5 op/s 2024-04-26T02:00:48.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:48.288+0000 7f228e000700 1 --2- 172.21.15.151:0/2501524007 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f22780a3db0 0x7f22780a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:48.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:48.289+0000 7f228e000700 1 -- 172.21.15.151:0/2501524007 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8455+0+0 (secure 0 0 0) 0x7f22840d1bc0 con 0x7f229006a750 2024-04-26T02:00:48.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:48.289+0000 7f2294c00700 1 --2- 172.21.15.151:0/2501524007 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f22780a3db0 0x7f22780a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:48.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:48.289+0000 7f2294c00700 1 --2- 172.21.15.151:0/2501524007 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f22780a3db0 0x7f22780a6270 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2290102f00 tx=0x7f227c009260 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:48.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:48.290+0000 7f228e000700 1 -- 172.21.15.151:0/2501524007 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2284030070 con 0x7f229006a750 2024-04-26T02:00:48.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:00:47 smithi165 ceph-mon[32648]: pgmap v223: 97 pgs: 97 active+clean; 57 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.7 KiB/s wr, 5 op/s 2024-04-26T02:00:48.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:48.565+0000 7f2298000700 1 -- 172.21.15.151:0/2501524007 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f229002df80 con 0x7f229006a750 2024-04-26T02:00:48.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:48.566+0000 7f228e000700 1 -- 172.21.15.151:0/2501524007 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11889 (secure 0 0 0) 0x7f2284013070 con 0x7f229006a750 2024-04-26T02:00:48.565 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T02:00:48.565 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2937864097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2937864097},{"type":"v1","addr":"172.21.15.151:6835","nonce":2937864097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14373,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/2947090802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2947090802},{"type":"v1","addr":"172.21.15.151:6837","nonce":2947090802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/1356563280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1356563280},{"type":"v1","addr":"172.21.15.151:6839","nonce":1356563280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6841/1075240552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1075240552},{"type":"v1","addr":"172.21.15.151:6841","nonce":1075240552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/504675912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":504675912},{"type":"v1","addr":"172.21.15.165:6833","nonce":504675912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1207580015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1207580015},{"type":"v1","addr":"172.21.15.172:6833","nonce":1207580015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:49:40.746876+0000","modified":"2024-04-26T02:00:05.413774+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24505,"mds_2":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/3598138294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3598138294},{"type":"v1","addr":"172.21.15.165:6835","nonce":3598138294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/1847859054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":1847859054},{"type":"v1","addr":"172.21.15.165:6837","nonce":1847859054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.172:6837/55972537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":55972537},{"type":"v1","addr":"172.21.15.172:6837","nonce":55972537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.172:6839/1242302810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1242302810},{"type":"v1","addr":"172.21.15.172:6839","nonce":1242302810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/1969716254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1969716254},{"type":"v1","addr":"172.21.15.165:6839","nonce":1969716254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25954":{"gid":25954,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/3656630171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3656630171},{"type":"v1","addr":"172.21.15.172:6835","nonce":3656630171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24493,24505,24509]},"id":1}]} 2024-04-26T02:00:48.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:48.570+0000 7f2298000700 1 -- 172.21.15.151:0/2501524007 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f22780a3db0 msgr2=0x7f22780a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:48.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:48.570+0000 7f2298000700 1 --2- 172.21.15.151:0/2501524007 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f22780a3db0 0x7f22780a6270 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2290102f00 tx=0x7f227c009260 comp rx=0 tx=0).stop 2024-04-26T02:00:48.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:48.570+0000 7f2298000700 1 -- 172.21.15.151:0/2501524007 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f229006a750 msgr2=0x7f229019b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:48.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:48.570+0000 7f2298000700 1 --2- 172.21.15.151:0/2501524007 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f229006a750 0x7f229019b3a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2284000c00 tx=0x7f22840047b0 comp rx=0 tx=0).stop 2024-04-26T02:00:48.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:48.570+0000 7f2298000700 1 -- 172.21.15.151:0/2501524007 shutdown_connections 2024-04-26T02:00:48.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:48.570+0000 7f2298000700 1 --2- 172.21.15.151:0/2501524007 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f22780a3db0 0x7f22780a6270 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:48.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:48.570+0000 7f2298000700 1 --2- 172.21.15.151:0/2501524007 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f229010f4f0 0x7f229019fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:48.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:48.570+0000 7f2298000700 1 --2- 172.21.15.151:0/2501524007 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f229006a750 0x7f229019b3a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:48.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:48.570+0000 7f2298000700 1 --2- 172.21.15.151:0/2501524007 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f229006b070 0x7f229019b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:48.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:48.570+0000 7f2298000700 1 -- 172.21.15.151:0/2501524007 >> 172.21.15.151:0/2501524007 conn(0x7f22900788f0 msgr2=0x7f2290110aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:48.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:48.571+0000 7f2298000700 1 -- 172.21.15.151:0/2501524007 shutdown_connections 2024-04-26T02:00:48.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:48.571+0000 7f2298000700 1 -- 172.21.15.151:0/2501524007 wait complete. 2024-04-26T02:00:48.571 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 35 2024-04-26T02:00:49.273 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:00:48 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/2501524007' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:00:49.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:00:48 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/2501524007' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:00:49.334 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24509 quiesce db --set-id bb158b1e --release --await' 2024-04-26T02:00:49.334 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24509 quiesce db --set-id bb158b1e --release --await 2024-04-26T02:00:49.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:00:48 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/2501524007' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:00:49.791 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:00:50.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:00:49 smithi172 ceph-mon[32785]: pgmap v224: 97 pgs: 97 active+clean; 57 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 9 op/s 2024-04-26T02:00:50.319 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:00:49 smithi151 ceph-mon[28057]: pgmap v224: 97 pgs: 97 active+clean; 57 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 9 op/s 2024-04-26T02:00:50.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:00:49 smithi165 ceph-mon[32648]: pgmap v224: 97 pgs: 97 active+clean; 57 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 9 op/s 2024-04-26T02:00:52.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.072+0000 7f10cc800700 1 -- 172.21.15.151:0/2789149576 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10c41092d0 msgr2=0x7f10c41096b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:52.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.072+0000 7f10cc800700 1 --2- 172.21.15.151:0/2789149576 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10c41092d0 0x7f10c41096b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f10b8009920 tx=0x7f10b8009c30 comp rx=0 tx=0).stop 2024-04-26T02:00:52.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.072+0000 7f10cc800700 1 -- 172.21.15.151:0/2789149576 shutdown_connections 2024-04-26T02:00:52.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.072+0000 7f10cc800700 1 --2- 172.21.15.151:0/2789149576 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10c410e3b0 0x7f10c4115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:52.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.072+0000 7f10cc800700 1 --2- 172.21.15.151:0/2789149576 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10c41092d0 0x7f10c41096b0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:52.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.072+0000 7f10cc800700 1 --2- 172.21.15.151:0/2789149576 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10c4109c80 0x7f10c410dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:52.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.072+0000 7f10cc800700 1 -- 172.21.15.151:0/2789149576 >> 172.21.15.151:0/2789149576 conn(0x7f10c4076210 msgr2=0x7f10c4107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:52.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.072+0000 7f10cc800700 1 -- 172.21.15.151:0/2789149576 shutdown_connections 2024-04-26T02:00:52.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.072+0000 7f10cc800700 1 -- 172.21.15.151:0/2789149576 wait complete. 2024-04-26T02:00:52.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.073+0000 7f10cc800700 1 Processor -- start 2024-04-26T02:00:52.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.073+0000 7f10cc800700 1 -- start start 2024-04-26T02:00:52.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.074+0000 7f10cc800700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10c41092d0 0x7f10c41a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:52.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.074+0000 7f10cc800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10c4109c80 0x7f10c41a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:52.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.074+0000 7f10cc800700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10c410e3b0 0x7f10c419f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:52.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.074+0000 7f10cc800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f10c4119270 con 0x7f10c4109c80 2024-04-26T02:00:52.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.074+0000 7f10cc800700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f10c41190f0 con 0x7f10c410e3b0 2024-04-26T02:00:52.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.074+0000 7f10cc800700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f10c41193f0 con 0x7f10c41092d0 2024-04-26T02:00:52.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.074+0000 7f10c9e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10c41092d0 0x7f10c41a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:52.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.074+0000 7f10c9400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10c4109c80 0x7f10c41a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:52.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.074+0000 7f10c9e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10c41092d0 0x7f10c41a5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:50072/0 (socket says 172.21.15.151:50072) 2024-04-26T02:00:52.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.074+0000 7f10c9e00700 1 -- 172.21.15.151:0/1625224973 learned_addr learned my addr 172.21.15.151:0/1625224973 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:00:52.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.074+0000 7f10ca800700 1 --2- 172.21.15.151:0/1625224973 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10c410e3b0 0x7f10c419f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:52.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.075+0000 7f10c9e00700 1 -- 172.21.15.151:0/1625224973 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10c410e3b0 msgr2=0x7f10c419f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:52.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.075+0000 7f10c9e00700 1 --2- 172.21.15.151:0/1625224973 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10c410e3b0 0x7f10c419f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:52.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.075+0000 7f10c9e00700 1 -- 172.21.15.151:0/1625224973 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10c4109c80 msgr2=0x7f10c41a59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:52.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.075+0000 7f10c9e00700 1 --2- 172.21.15.151:0/1625224973 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10c4109c80 0x7f10c41a59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:52.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.075+0000 7f10c9e00700 1 -- 172.21.15.151:0/1625224973 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10c419fdd0 con 0x7f10c41092d0 2024-04-26T02:00:52.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.075+0000 7f10c9e00700 1 --2- 172.21.15.151:0/1625224973 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10c41092d0 0x7f10c41a5460 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f10b8009230 tx=0x7f10b800fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:52.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.075+0000 7f10bea00700 1 -- 172.21.15.151:0/1625224973 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b8027070 con 0x7f10c41092d0 2024-04-26T02:00:52.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.075+0000 7f10bea00700 1 -- 172.21.15.151:0/1625224973 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10b8005570 con 0x7f10c41092d0 2024-04-26T02:00:52.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.075+0000 7f10cc800700 1 -- 172.21.15.151:0/1625224973 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10c41a0060 con 0x7f10c41092d0 2024-04-26T02:00:52.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.076+0000 7f10bea00700 1 -- 172.21.15.151:0/1625224973 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b8004730 con 0x7f10c41092d0 2024-04-26T02:00:52.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.076+0000 7f10cc800700 1 -- 172.21.15.151:0/1625224973 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10c41ac330 con 0x7f10c41092d0 2024-04-26T02:00:52.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.079+0000 7f10bea00700 1 -- 172.21.15.151:0/1625224973 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f10b802a070 con 0x7f10c41092d0 2024-04-26T02:00:52.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.080+0000 7f10bea00700 1 --2- 172.21.15.151:0/1625224973 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f10ac0a3db0 0x7f10ac0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:52.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.080+0000 7f10bea00700 1 -- 172.21.15.151:0/1625224973 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8455+0+0 (secure 0 0 0) 0x7f10b80d10e0 con 0x7f10c41092d0 2024-04-26T02:00:52.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.081+0000 7f10c9400700 1 --2- 172.21.15.151:0/1625224973 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f10ac0a3db0 0x7f10ac0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:52.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.081+0000 7f10d70b8b80 1 Processor -- start 2024-04-26T02:00:52.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.081+0000 7f10d70b8b80 1 -- start start 2024-04-26T02:00:52.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.081+0000 7f10c9400700 1 --2- 172.21.15.151:0/1625224973 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f10ac0a3db0 0x7f10ac0a6270 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f10b4009750 tx=0x7f10b4009310 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:52.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.081+0000 7f10d70b8b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563ae6393d40 0x563ae6394120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:52.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.081+0000 7f10d70b8b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ae6394660 0x563ae639ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:52.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.081+0000 7f10d70b8b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563ae639f4d0 0x563ae63a18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:52.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.081+0000 7f10d70b8b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563ae62e6f90 con 0x563ae6393d40 2024-04-26T02:00:52.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.081+0000 7f10d70b8b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563ae628d830 con 0x563ae6394660 2024-04-26T02:00:52.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.081+0000 7f10d70b8b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563ae62e2f50 con 0x563ae639f4d0 2024-04-26T02:00:52.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.082+0000 7f10c9e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563ae6393d40 0x563ae6394120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:52.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.082+0000 7f10c9400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ae6394660 0x563ae639ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:52.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.082+0000 7f10c9e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563ae6393d40 0x563ae6394120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:59622/0 (socket says 172.21.15.151:59622) 2024-04-26T02:00:52.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.082+0000 7f10c9400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ae6394660 0x563ae639ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:45380/0 (socket says 172.21.15.151:45380) 2024-04-26T02:00:52.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.082+0000 7f10c9e00700 1 -- 172.21.15.151:0/3702595118 learned_addr learned my addr 172.21.15.151:0/3702595118 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:00:52.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.082+0000 7f10ca800700 1 --2- 172.21.15.151:0/3702595118 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563ae639f4d0 0x563ae63a18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:52.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.082+0000 7f10c9400700 1 -- 172.21.15.151:0/3702595118 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563ae639f4d0 msgr2=0x563ae63a18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:52.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.082+0000 7f10c9400700 1 --2- 172.21.15.151:0/3702595118 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563ae639f4d0 0x563ae63a18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:52.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.082+0000 7f10c9400700 1 -- 172.21.15.151:0/3702595118 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563ae6393d40 msgr2=0x563ae6394120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:52.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.082+0000 7f10c9400700 1 --2- 172.21.15.151:0/3702595118 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563ae6393d40 0x563ae6394120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:52.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.082+0000 7f10c9400700 1 -- 172.21.15.151:0/3702595118 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ae6283780 con 0x563ae6394660 2024-04-26T02:00:52.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.082+0000 7f10c9e00700 1 --2- 172.21.15.151:0/3702595118 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563ae6393d40 0x563ae6394120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:00:52.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.082+0000 7f10ca800700 1 --2- 172.21.15.151:0/3702595118 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563ae639f4d0 0x563ae63a18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:00:52.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.083+0000 7f10c9400700 1 --2- 172.21.15.151:0/3702595118 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ae6394660 0x563ae639ef90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f10c406c260 tx=0x7f10b4014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ec79a2bc998df02 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:52.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.083+0000 7f10b3400700 1 -- 172.21.15.151:0/3702595118 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b4026430 con 0x563ae6394660 2024-04-26T02:00:52.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.083+0000 7f10b3400700 1 -- 172.21.15.151:0/3702595118 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10b401c8c0 con 0x563ae6394660 2024-04-26T02:00:52.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.083+0000 7f10b3400700 1 -- 172.21.15.151:0/3702595118 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b401d810 con 0x563ae6394660 2024-04-26T02:00:52.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.084+0000 7f10d70b8b80 1 -- 172.21.15.151:0/3702595118 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ae6394660 msgr2=0x563ae639ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:52.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.084+0000 7f10d70b8b80 1 --2- 172.21.15.151:0/3702595118 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ae6394660 0x563ae639ef90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f10c406c260 tx=0x7f10b4014000 comp rx=0 tx=0).stop 2024-04-26T02:00:52.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.084+0000 7f10d70b8b80 1 -- 172.21.15.151:0/3702595118 shutdown_connections 2024-04-26T02:00:52.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.084+0000 7f10d70b8b80 1 --2- 172.21.15.151:0/3702595118 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563ae639f4d0 0x563ae63a18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:52.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.084+0000 7f10d70b8b80 1 --2- 172.21.15.151:0/3702595118 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563ae6393d40 0x563ae6394120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:52.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.084+0000 7f10d70b8b80 1 --2- 172.21.15.151:0/3702595118 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ae6394660 0x563ae639ef90 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:52.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.084+0000 7f10d70b8b80 1 -- 172.21.15.151:0/3702595118 >> 172.21.15.151:0/3702595118 conn(0x563ae62c3050 msgr2=0x563ae6390250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:52.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.084+0000 7f10d70b8b80 1 -- 172.21.15.151:0/3702595118 shutdown_connections 2024-04-26T02:00:52.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.084+0000 7f10d70b8b80 1 -- 172.21.15.151:0/3702595118 wait complete. 2024-04-26T02:00:52.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.085+0000 7f10d70b8b80 1 -- start start 2024-04-26T02:00:52.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.085+0000 7f10d70b8b80 1 Processor -- start 2024-04-26T02:00:52.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.085+0000 7f10d70b8b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563ae6393d40 0x563ae639ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:52.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.085+0000 7f10d70b8b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563ae6394660 0x563ae639bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:52.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.085+0000 7f10d70b8b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ae639f4d0 0x563ae639c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:52.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.086+0000 7f10d70b8b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563ae63a1f60 con 0x563ae6393d40 2024-04-26T02:00:52.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.086+0000 7f10d70b8b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563ae62a0960 con 0x563ae639f4d0 2024-04-26T02:00:52.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.086+0000 7f10d70b8b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563ae63a20e0 con 0x563ae6394660 2024-04-26T02:00:52.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.086+0000 7f10d70b8b80 10 client.0 _ll_register_callbacks cb 0x563ae627f820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:00:52.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.086+0000 7f10c9e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563ae6393d40 0x563ae639ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:52.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.086+0000 7f10c9400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563ae6394660 0x563ae639bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:52.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.086+0000 7f10c9400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563ae6394660 0x563ae639bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:50094/0 (socket says 172.21.15.151:50094) 2024-04-26T02:00:52.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.086+0000 7f10c9400700 1 -- 172.21.15.151:0/1728530668 learned_addr learned my addr 172.21.15.151:0/1728530668 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:00:52.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.086+0000 7f10ca800700 1 --2- 172.21.15.151:0/1728530668 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ae639f4d0 0x563ae639c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:52.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.086+0000 7f10c9400700 1 -- 172.21.15.151:0/1728530668 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ae639f4d0 msgr2=0x563ae639c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:52.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.086+0000 7f10c9400700 1 --2- 172.21.15.151:0/1728530668 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ae639f4d0 0x563ae639c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:52.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.086+0000 7f10c9400700 1 -- 172.21.15.151:0/1728530668 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563ae6393d40 msgr2=0x563ae639ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:52.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.086+0000 7f10c9400700 1 --2- 172.21.15.151:0/1728530668 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563ae6393d40 0x563ae639ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:52.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.086+0000 7f10c9400700 1 -- 172.21.15.151:0/1728530668 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ae62349d0 con 0x563ae6394660 2024-04-26T02:00:52.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.087+0000 7f10c9400700 1 --2- 172.21.15.151:0/1728530668 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563ae6394660 0x563ae639bf20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f10b40099d0 tx=0x7f10b400ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4158114cf48791b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:52.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.087+0000 7f10a7e00700 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-26T02:00:52.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.087+0000 7f10a7e00700 1 -- 172.21.15.151:0/1728530668 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b401ca70 con 0x563ae6394660 2024-04-26T02:00:52.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.087+0000 7f10a7e00700 1 -- 172.21.15.151:0/1728530668 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8455+0+0 (secure 0 0 0) 0x7f10b401cc10 con 0x563ae6394660 2024-04-26T02:00:52.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.087+0000 7f10d70b8b80 1 -- 172.21.15.151:0/1728530668 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10b401c8c0 con 0x563ae6394660 2024-04-26T02:00:52.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.087+0000 7f10a7e00700 1 -- 172.21.15.151:0/1728530668 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f10b403c050 con 0x563ae6394660 2024-04-26T02:00:52.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.087+0000 7f10d70b8b80 10 client.26044 fetch_fsmap learned FSMap version 35 2024-04-26T02:00:52.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.087+0000 7f10d70b8b80 1 -- 172.21.15.151:0/1728530668 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x563ae627dbf0 con 0x563ae6394660 2024-04-26T02:00:52.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.088+0000 7f10a7e00700 1 -- 172.21.15.151:0/1728530668 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10b401c8c0 con 0x563ae6394660 2024-04-26T02:00:52.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.088+0000 7f10a7e00700 1 -- 172.21.15.151:0/1728530668 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 35) ==== 8338+0+0 (secure 0 0 0) 0x7f10b4004ae0 con 0x563ae6394660 2024-04-26T02:00:52.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.088+0000 7f10d70b8b80 10 client.26044 fetch_fsmap finished waiting for FSMap version 35 2024-04-26T02:00:52.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.088+0000 7f10d70b8b80 10 client.26044 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-26T02:00:52.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.088+0000 7f10d70b8b80 20 client.26044 populate_metadata read hostname 'smithi151' 2024-04-26T02:00:52.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.088+0000 7f10d70b8b80 1 --2- 172.21.15.151:0/1728530668 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x563ae63aebf0 0x563ae63b0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:52.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.088+0000 7f10d70b8b80 4 client.26044 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:00:52.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.088+0000 7f10d70b8b80 1 -- 172.21.15.151:0/1728530668 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ae62f4780 con 0x563ae63aebf0 2024-04-26T02:00:52.090 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.088+0000 7f10c9e00700 1 --2- 172.21.15.151:0/1728530668 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x563ae63aebf0 0x563ae63b0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:52.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.089+0000 7f10c9e00700 1 --2- 172.21.15.151:0/1728530668 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x563ae63aebf0 0x563ae63b0fe0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a6dbf06cc7b3061 server_cookie=e40a5e193929ba46 in_seq=0 out_seq=0 2024-04-26T02:00:52.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.089+0000 7f10a7e00700 10 client.26044 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:00:52.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.090+0000 7f10a7e00700 1 -- 172.21.15.151:0/1728530668 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563ae62f4780 con 0x563ae63aebf0 2024-04-26T02:00:52.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.090+0000 7f10a7e00700 10 client.26044 handle_command_reply: tid=0 2024-04-26T02:00:52.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.090+0000 7f10d70b8b80 1 client.26044 shutdown 2024-04-26T02:00:52.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.090+0000 7f10d70b8b80 1 -- 172.21.15.151:0/1728530668 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563ae6394660 msgr2=0x563ae639bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:52.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.090+0000 7f10d70b8b80 1 --2- 172.21.15.151:0/1728530668 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563ae6394660 0x563ae639bf20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f10b40099d0 tx=0x7f10b400ee60 comp rx=0 tx=0).stop 2024-04-26T02:00:52.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.090+0000 7f10d70b8b80 1 -- 172.21.15.151:0/1728530668 shutdown_connections 2024-04-26T02:00:52.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.090+0000 7f10d70b8b80 1 --2- 172.21.15.151:0/1728530668 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x563ae63aebf0 0x563ae63b0fe0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:52.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.090+0000 7f10d70b8b80 1 --2- 172.21.15.151:0/1728530668 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ae639f4d0 0x563ae639c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:52.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.090+0000 7f10d70b8b80 1 --2- 172.21.15.151:0/1728530668 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563ae6393d40 0x563ae639ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:52.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.090+0000 7f10d70b8b80 1 --2- 172.21.15.151:0/1728530668 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563ae6394660 0x563ae639bf20 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:52.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.090+0000 7f10a7e00700 0 client.26044 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:00:52.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.090+0000 7f10d70b8b80 1 -- 172.21.15.151:0/1728530668 >> 172.21.15.151:0/1728530668 conn(0x563ae62c3050 msgr2=0x563ae638fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:52.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.091+0000 7f10d70b8b80 1 -- 172.21.15.151:0/1728530668 shutdown_connections 2024-04-26T02:00:52.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.091+0000 7f10d70b8b80 1 -- 172.21.15.151:0/1728530668 wait complete. 2024-04-26T02:00:52.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.091+0000 7f10d70b8b80 20 client.26044 trim_cache size 0 max 16384 2024-04-26T02:00:52.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.112+0000 7f10d70b8b80 1 Processor -- start 2024-04-26T02:00:52.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.112+0000 7f10d70b8b80 1 -- start start 2024-04-26T02:00:52.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.112+0000 7f10d70b8b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563ae6393d40 0x563ae639ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:52.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.112+0000 7f10d70b8b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563ae6394660 0x563ae63903a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:52.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.112+0000 7f10d70b8b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ae639f4d0 0x563ae63aaec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:52.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.112+0000 7f10d70b8b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f10c41b6070 con 0x563ae6394660 2024-04-26T02:00:52.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.112+0000 7f10d70b8b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f10c41b3900 con 0x563ae639f4d0 2024-04-26T02:00:52.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.112+0000 7f10d70b8b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f10c41b1770 con 0x563ae6393d40 2024-04-26T02:00:52.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.113+0000 7f10c9400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563ae6394660 0x563ae63903a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:52.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.113+0000 7f10c9e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563ae6393d40 0x563ae639ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:52.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.113+0000 7f10c9400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563ae6394660 0x563ae63903a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:59650/0 (socket says 172.21.15.151:59650) 2024-04-26T02:00:52.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.113+0000 7f10c9400700 1 -- 172.21.15.151:0/1732859039 learned_addr learned my addr 172.21.15.151:0/1732859039 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:00:52.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.113+0000 7f10ca800700 1 --2- 172.21.15.151:0/1732859039 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ae639f4d0 0x563ae63aaec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:52.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.113+0000 7f10c9400700 1 -- 172.21.15.151:0/1732859039 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563ae6393d40 msgr2=0x563ae639ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:52.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.113+0000 7f10c9400700 1 --2- 172.21.15.151:0/1732859039 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563ae6393d40 0x563ae639ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:52.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.113+0000 7f10c9400700 1 -- 172.21.15.151:0/1732859039 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ae639f4d0 msgr2=0x563ae63aaec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:52.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.113+0000 7f10c9400700 1 --2- 172.21.15.151:0/1732859039 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ae639f4d0 0x563ae63aaec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:52.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.113+0000 7f10c9400700 1 -- 172.21.15.151:0/1732859039 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ae6394ce0 con 0x563ae6394660 2024-04-26T02:00:52.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.113+0000 7f10c9e00700 1 --2- 172.21.15.151:0/1732859039 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563ae6393d40 0x563ae639ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:00:52.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.113+0000 7f10c9400700 1 --2- 172.21.15.151:0/1732859039 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563ae6394660 0x563ae63903a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f10b4000c00 tx=0x7f10b400eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dc9100b5f56e2ce server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:52.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.114+0000 7f10b2a00700 1 -- 172.21.15.151:0/1732859039 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b4026600 con 0x563ae6394660 2024-04-26T02:00:52.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.114+0000 7f10b2a00700 1 -- 172.21.15.151:0/1732859039 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10b4045070 con 0x563ae6394660 2024-04-26T02:00:52.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.114+0000 7f10b2a00700 1 -- 172.21.15.151:0/1732859039 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b4004930 con 0x563ae6394660 2024-04-26T02:00:52.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.115+0000 7f10d70b8b80 1 -- 172.21.15.151:0/1732859039 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563ae6394660 msgr2=0x563ae63903a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:52.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.115+0000 7f10d70b8b80 1 --2- 172.21.15.151:0/1732859039 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563ae6394660 0x563ae63903a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f10b4000c00 tx=0x7f10b400eda0 comp rx=0 tx=0).stop 2024-04-26T02:00:52.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.115+0000 7f10d70b8b80 1 -- 172.21.15.151:0/1732859039 shutdown_connections 2024-04-26T02:00:52.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.115+0000 7f10d70b8b80 1 --2- 172.21.15.151:0/1732859039 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ae639f4d0 0x563ae63aaec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:52.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.115+0000 7f10d70b8b80 1 --2- 172.21.15.151:0/1732859039 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563ae6393d40 0x563ae639ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:52.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.115+0000 7f10d70b8b80 1 --2- 172.21.15.151:0/1732859039 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563ae6394660 0x563ae63903a0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:52.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.115+0000 7f10d70b8b80 1 -- 172.21.15.151:0/1732859039 >> 172.21.15.151:0/1732859039 conn(0x563ae62c3050 msgr2=0x563ae638fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:52.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.115+0000 7f10d70b8b80 1 -- 172.21.15.151:0/1732859039 shutdown_connections 2024-04-26T02:00:52.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.115+0000 7f10d70b8b80 1 -- 172.21.15.151:0/1732859039 wait complete. 2024-04-26T02:00:52.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.115+0000 7f10d70b8b80 1 -- start start 2024-04-26T02:00:52.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.115+0000 7f10d70b8b80 1 Processor -- start 2024-04-26T02:00:52.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.116+0000 7f10d70b8b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ae6393d40 0x563ae63a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:52.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.116+0000 7f10d70b8b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563ae6394660 0x563ae63a48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:52.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.116+0000 7f10d70b8b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563ae639f4d0 0x563ae6414e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:52.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.116+0000 7f10d70b8b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f10c41b1dc0 con 0x563ae6394660 2024-04-26T02:00:52.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.116+0000 7f10d70b8b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f10c41b3230 con 0x563ae6393d40 2024-04-26T02:00:52.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.116+0000 7f10d70b8b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f10c41b2490 con 0x563ae639f4d0 2024-04-26T02:00:52.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.116+0000 7f10d70b8b80 10 client.0 _ll_register_callbacks cb 0x563ae627f820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:00:52.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.116+0000 7f10c9400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563ae6394660 0x563ae63a48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:52.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.116+0000 7f10c9400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563ae6394660 0x563ae63a48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:59656/0 (socket says 172.21.15.151:59656) 2024-04-26T02:00:52.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.116+0000 7f10c9400700 1 -- 172.21.15.151:0/366171966 learned_addr learned my addr 172.21.15.151:0/366171966 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:00:52.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.116+0000 7f10c9e00700 1 --2- 172.21.15.151:0/366171966 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ae6393d40 0x563ae63a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:52.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.116+0000 7f10ca800700 1 --2- 172.21.15.151:0/366171966 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563ae639f4d0 0x563ae6414e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:52.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.117+0000 7f10c9400700 1 -- 172.21.15.151:0/366171966 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563ae639f4d0 msgr2=0x563ae6414e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:52.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.117+0000 7f10c9400700 1 --2- 172.21.15.151:0/366171966 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563ae639f4d0 0x563ae6414e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:52.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.117+0000 7f10c9400700 1 -- 172.21.15.151:0/366171966 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ae6393d40 msgr2=0x563ae63a43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:52.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.117+0000 7f10c9400700 1 --2- 172.21.15.151:0/366171966 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ae6393d40 0x563ae63a43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:52.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.117+0000 7f10c9400700 1 -- 172.21.15.151:0/366171966 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ae63a51a0 con 0x563ae6394660 2024-04-26T02:00:52.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.117+0000 7f10c9400700 1 --2- 172.21.15.151:0/366171966 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563ae6394660 0x563ae63a48e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f10b40099d0 tx=0x7f10b40266b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46ba4172c155d1be server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:52.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.117+0000 7f10a7e00700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-04-26T02:00:52.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.117+0000 7f10a7e00700 1 -- 172.21.15.151:0/366171966 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b4026d30 con 0x563ae6394660 2024-04-26T02:00:52.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.117+0000 7f10a7e00700 1 -- 172.21.15.151:0/366171966 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8455+0+0 (secure 0 0 0) 0x7f10b40046d0 con 0x563ae6394660 2024-04-26T02:00:52.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.117+0000 7f10d70b8b80 1 -- 172.21.15.151:0/366171966 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10b4045070 con 0x563ae6394660 2024-04-26T02:00:52.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.118+0000 7f10a7e00700 1 -- 172.21.15.151:0/366171966 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f10b40048d0 con 0x563ae6394660 2024-04-26T02:00:52.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.118+0000 7f10d70b8b80 10 client.15912 fetch_fsmap learned FSMap version 35 2024-04-26T02:00:52.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.118+0000 7f10d70b8b80 1 -- 172.21.15.151:0/366171966 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x563ae6416510 con 0x563ae6394660 2024-04-26T02:00:52.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.118+0000 7f10a7e00700 1 -- 172.21.15.151:0/366171966 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10b4045070 con 0x563ae6394660 2024-04-26T02:00:52.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.118+0000 7f10a7e00700 1 -- 172.21.15.151:0/366171966 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 35) ==== 8338+0+0 (secure 0 0 0) 0x7f10b4004f30 con 0x563ae6394660 2024-04-26T02:00:52.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.118+0000 7f10d70b8b80 10 client.15912 fetch_fsmap finished waiting for FSMap version 35 2024-04-26T02:00:52.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.118+0000 7f10d70b8b80 10 client.15912 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-26T02:00:52.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.118+0000 7f10d70b8b80 20 client.15912 populate_metadata read hostname 'smithi151' 2024-04-26T02:00:52.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.118+0000 7f10d70b8b80 1 --2- 172.21.15.151:0/366171966 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x563ae6416b60 0x563ae6418f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:52.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.118+0000 7f10d70b8b80 4 client.15912 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "bb158b1e", "release": true, "await": true}] 2024-04-26T02:00:52.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.118+0000 7f10d70b8b80 1 -- 172.21.15.151:0/366171966 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "quiesce db", "set_id": "bb158b1e", "release": true, "await": true}) -- 0x563ae62ee820 con 0x563ae6416b60 2024-04-26T02:00:52.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.119+0000 7f10c9e00700 1 --2- 172.21.15.151:0/366171966 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x563ae6416b60 0x563ae6418f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:52.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.119+0000 7f10c9e00700 1 --2- 172.21.15.151:0/366171966 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x563ae6416b60 0x563ae6418f80 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=610abf210ceace53 server_cookie=de02de43a493fca in_seq=0 out_seq=0 2024-04-26T02:00:52.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.119+0000 7f10a7e00700 10 client.15912 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:00:52.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:00:51 smithi172 ceph-mon[32785]: pgmap v225: 97 pgs: 97 active+clean; 57 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.7 KiB/s wr, 7 op/s 2024-04-26T02:00:52.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.306+0000 7f10a7e00700 1 -- 172.21.15.151:0/366171966 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x563ae62ee820 con 0x563ae6416b60 2024-04-26T02:00:52.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.306+0000 7f10a7e00700 10 client.15912 handle_command_reply: tid=0 2024-04-26T02:00:52.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.307+0000 7f10d70b8b80 1 client.15912 shutdown 2024-04-26T02:00:52.306 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:00:51 smithi151 ceph-mon[28057]: pgmap v225: 97 pgs: 97 active+clean; 57 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.7 KiB/s wr, 7 op/s 2024-04-26T02:00:52.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.307+0000 7f10d70b8b80 1 -- 172.21.15.151:0/366171966 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563ae6394660 msgr2=0x563ae63a48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:52.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.307+0000 7f10d70b8b80 1 --2- 172.21.15.151:0/366171966 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563ae6394660 0x563ae63a48e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f10b40099d0 tx=0x7f10b40266b0 comp rx=0 tx=0).stop 2024-04-26T02:00:52.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.308+0000 7f10d70b8b80 1 -- 172.21.15.151:0/366171966 shutdown_connections 2024-04-26T02:00:52.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.308+0000 7f10d70b8b80 1 --2- 172.21.15.151:0/366171966 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563ae639f4d0 0x563ae6414e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:52.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.308+0000 7f10d70b8b80 1 --2- 172.21.15.151:0/366171966 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x563ae6416b60 0x563ae6418f80 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:52.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.308+0000 7f10d70b8b80 1 --2- 172.21.15.151:0/366171966 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ae6393d40 0x563ae63a43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:52.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.308+0000 7f10d70b8b80 1 --2- 172.21.15.151:0/366171966 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563ae6394660 0x563ae63a48e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:52.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.308+0000 7f10d70b8b80 1 -- 172.21.15.151:0/366171966 >> 172.21.15.151:0/366171966 conn(0x563ae62c3050 msgr2=0x563ae63a9eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:52.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.308+0000 7f10a7e00700 0 client.15912 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:00:52.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.308+0000 7f10d70b8b80 1 -- 172.21.15.151:0/366171966 shutdown_connections 2024-04-26T02:00:52.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.308+0000 7f10d70b8b80 1 -- 172.21.15.151:0/366171966 wait complete. 2024-04-26T02:00:52.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.309+0000 7f10d70b8b80 20 client.15912 trim_cache size 0 max 16384 2024-04-26T02:00:52.309 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-26T02:00:52.309 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 35, 2024-04-26T02:00:52.309 INFO:teuthology.orchestra.run.smithi151.stdout: "leader": 24509, 2024-04-26T02:00:52.309 INFO:teuthology.orchestra.run.smithi151.stdout: "set_version": 20, 2024-04-26T02:00:52.309 INFO:teuthology.orchestra.run.smithi151.stdout: "sets": { 2024-04-26T02:00:52.310 INFO:teuthology.orchestra.run.smithi151.stdout: "bb158b1e": { 2024-04-26T02:00:52.310 INFO:teuthology.orchestra.run.smithi151.stdout: "version": 20, 2024-04-26T02:00:52.310 INFO:teuthology.orchestra.run.smithi151.stdout: "age_ref": 0.0, 2024-04-26T02:00:52.310 INFO:teuthology.orchestra.run.smithi151.stdout: "state": { 2024-04-26T02:00:52.310 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "RELEASED", 2024-04-26T02:00:52.310 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 0.0 2024-04-26T02:00:52.310 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:00:52.310 INFO:teuthology.orchestra.run.smithi151.stdout: "timeout": 60.0, 2024-04-26T02:00:52.310 INFO:teuthology.orchestra.run.smithi151.stdout: "expiration": 75.9, 2024-04-26T02:00:52.310 INFO:teuthology.orchestra.run.smithi151.stdout: "members": { 2024-04-26T02:00:52.310 INFO:teuthology.orchestra.run.smithi151.stdout: "file:/": { 2024-04-26T02:00:52.311 INFO:teuthology.orchestra.run.smithi151.stdout: "excluded": false, 2024-04-26T02:00:52.311 INFO:teuthology.orchestra.run.smithi151.stdout: "state": { 2024-04-26T02:00:52.311 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "RELEASED", 2024-04-26T02:00:52.311 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 0.0 2024-04-26T02:00:52.311 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:00:52.311 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:00:52.311 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:00:52.311 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:00:52.311 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:00:52.311 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-26T02:00:52.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.310+0000 7f10cc800700 1 -- 172.21.15.151:0/1625224973 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f10ac0a3db0 msgr2=0x7f10ac0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:52.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.310+0000 7f10cc800700 1 --2- 172.21.15.151:0/1625224973 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f10ac0a3db0 0x7f10ac0a6270 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f10b4009750 tx=0x7f10b4009310 comp rx=0 tx=0).stop 2024-04-26T02:00:52.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.310+0000 7f10cc800700 1 -- 172.21.15.151:0/1625224973 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10c41092d0 msgr2=0x7f10c41a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:52.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.310+0000 7f10cc800700 1 --2- 172.21.15.151:0/1625224973 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10c41092d0 0x7f10c41a5460 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f10b8009230 tx=0x7f10b800fa00 comp rx=0 tx=0).stop 2024-04-26T02:00:52.313 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.310+0000 7f10cc800700 1 -- 172.21.15.151:0/1625224973 shutdown_connections 2024-04-26T02:00:52.313 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.310+0000 7f10cc800700 1 --2- 172.21.15.151:0/1625224973 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f10ac0a3db0 0x7f10ac0a6270 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:52.313 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.310+0000 7f10cc800700 1 --2- 172.21.15.151:0/1625224973 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10c410e3b0 0x7f10c419f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:52.313 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.310+0000 7f10cc800700 1 --2- 172.21.15.151:0/1625224973 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10c41092d0 0x7f10c41a5460 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:52.313 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.310+0000 7f10cc800700 1 --2- 172.21.15.151:0/1625224973 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10c4109c80 0x7f10c41a59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:52.313 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.310+0000 7f10cc800700 1 -- 172.21.15.151:0/1625224973 >> 172.21.15.151:0/1625224973 conn(0x7f10c4076210 msgr2=0x7f10c41086f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:52.313 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.311+0000 7f10cc800700 1 -- 172.21.15.151:0/1625224973 shutdown_connections 2024-04-26T02:00:52.313 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:00:52.311+0000 7f10cc800700 1 -- 172.21.15.151:0/1625224973 wait complete. 2024-04-26T02:00:52.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:00:51 smithi165 ceph-mon[32648]: pgmap v225: 97 pgs: 97 active+clean; 57 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.7 KiB/s wr, 7 op/s 2024-04-26T02:00:53.262 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'bb158b1e', seconds elapsed: 32.7 2024-04-26T02:00:54.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:00:53 smithi172 ceph-mon[32785]: pgmap v226: 97 pgs: 97 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 31 KiB/s wr, 12 op/s 2024-04-26T02:00:54.287 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:00:53 smithi151 ceph-mon[28057]: pgmap v226: 97 pgs: 97 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 31 KiB/s wr, 12 op/s 2024-04-26T02:00:54.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:00:53 smithi165 ceph-mon[32648]: pgmap v226: 97 pgs: 97 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 31 KiB/s wr, 12 op/s 2024-04-26T02:00:55.072 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:00:54 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T02:00:55.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:00:54 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T02:00:55.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:00:54 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T02:00:56.253 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:00:55 smithi151 ceph-mon[28057]: pgmap v227: 97 pgs: 97 active+clean; 55 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 129 KiB/s wr, 23 op/s 2024-04-26T02:00:56.253 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:00:55 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:00:56.253 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:00:55 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:00:56.253 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:00:55 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T02:00:56.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:00:55 smithi172 ceph-mon[32785]: pgmap v227: 97 pgs: 97 active+clean; 55 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 129 KiB/s wr, 23 op/s 2024-04-26T02:00:56.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:00:55 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:00:56.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:00:55 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:00:56.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:00:55 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T02:00:56.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:00:55 smithi165 ceph-mon[32648]: pgmap v227: 97 pgs: 97 active+clean; 55 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 129 KiB/s wr, 23 op/s 2024-04-26T02:00:56.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:00:55 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:00:56.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:00:55 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:00:56.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:00:55 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T02:00:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.722+0000 7f8240e00640 1 -- 172.21.15.151:0/43002058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f823c1043e0 msgr2=0x7f823c1047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.722+0000 7f8240e00640 1 --2- 172.21.15.151:0/43002058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f823c1043e0 0x7f823c1047c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f822c009900 tx=0x7f822c02a3f0 comp rx=0 tx=0).stop 2024-04-26T02:00:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.722+0000 7f8240e00640 1 -- 172.21.15.151:0/43002058 shutdown_connections 2024-04-26T02:00:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.722+0000 7f8240e00640 1 --2- 172.21.15.151:0/43002058 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f823c1094b0 0x7f823c111080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.722+0000 7f8240e00640 1 --2- 172.21.15.151:0/43002058 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f823c104d90 0x7f823c108dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.722+0000 7f8240e00640 1 --2- 172.21.15.151:0/43002058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f823c1043e0 0x7f823c1047c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.722+0000 7f8240e00640 1 -- 172.21.15.151:0/43002058 >> 172.21.15.151:0/43002058 conn(0x7f823c0ffc60 msgr2=0x7f823c102080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.722+0000 7f8240e00640 1 -- 172.21.15.151:0/43002058 shutdown_connections 2024-04-26T02:00:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.722+0000 7f8240e00640 1 -- 172.21.15.151:0/43002058 wait complete. 2024-04-26T02:00:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.723+0000 7f8240e00640 1 Processor -- start 2024-04-26T02:00:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.723+0000 7f8240e00640 1 -- start start 2024-04-26T02:00:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.724+0000 7f8240e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f823c1043e0 0x7f823c1a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.724+0000 7f8240e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f823c1094b0 0x7f823c1a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.724+0000 7f8240e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f823c19a780 0x7f823c19ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.724+0000 7f8240e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f823c114460 con 0x7f823c1043e0 2024-04-26T02:00:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.724+0000 7f8240e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f823c1142e0 con 0x7f823c19a780 2024-04-26T02:00:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.724+0000 7f8240e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f823c1145e0 con 0x7f823c1094b0 2024-04-26T02:00:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.724+0000 7f8239400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f823c1094b0 0x7f823c1a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.724+0000 7f8239e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f823c1043e0 0x7f823c1a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.724+0000 7f8239400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f823c1094b0 0x7f823c1a0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:50124/0 (socket says 172.21.15.151:50124) 2024-04-26T02:00:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.724+0000 7f8239400640 1 -- 172.21.15.151:0/678659587 learned_addr learned my addr 172.21.15.151:0/678659587 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:00:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.724+0000 7f823a800640 1 --2- 172.21.15.151:0/678659587 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f823c19a780 0x7f823c19ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.725+0000 7f8239400640 1 -- 172.21.15.151:0/678659587 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f823c19a780 msgr2=0x7f823c19ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.725+0000 7f8239400640 1 --2- 172.21.15.151:0/678659587 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f823c19a780 0x7f823c19ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.725+0000 7f8239400640 1 -- 172.21.15.151:0/678659587 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f823c1043e0 msgr2=0x7f823c1a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.725+0000 7f8239400640 1 --2- 172.21.15.151:0/678659587 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f823c1043e0 0x7f823c1a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.725+0000 7f8239400640 1 -- 172.21.15.151:0/678659587 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f823c19b310 con 0x7f823c1094b0 2024-04-26T02:00:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.725+0000 7f8239400640 1 --2- 172.21.15.151:0/678659587 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f823c1094b0 0x7f823c1a0b50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f822800e9f0 tx=0x7f822800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:57.726 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.725+0000 7f8232a00640 1 -- 172.21.15.151:0/678659587 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822800ccc0 con 0x7f823c1094b0 2024-04-26T02:00:57.726 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.725+0000 7f8240e00640 1 -- 172.21.15.151:0/678659587 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f823c19b600 con 0x7f823c1094b0 2024-04-26T02:00:57.726 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.725+0000 7f8232a00640 1 -- 172.21.15.151:0/678659587 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82280054d0 con 0x7f823c1094b0 2024-04-26T02:00:57.726 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.725+0000 7f8232a00640 1 -- 172.21.15.151:0/678659587 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8228010430 con 0x7f823c1094b0 2024-04-26T02:00:57.726 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.726+0000 7f8240e00640 1 -- 172.21.15.151:0/678659587 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f823c1a7790 con 0x7f823c1094b0 2024-04-26T02:00:57.728 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.729+0000 7f8232a00640 1 -- 172.21.15.151:0/678659587 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f82280046c0 con 0x7f823c1094b0 2024-04-26T02:00:57.728 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.729+0000 7f8243d6a740 1 Processor -- start 2024-04-26T02:00:57.728 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.729+0000 7f8243d6a740 1 -- start start 2024-04-26T02:00:57.730 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.729+0000 7f8243d6a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561835ec7820 0x561835ec7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:57.730 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.729+0000 7f8243d6a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561835ec8140 0x561835ed24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:57.730 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.729+0000 7f8243d6a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561835ed2a00 0x561835ed4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:57.730 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.730+0000 7f8243d6a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x561835e2d900 con 0x561835ed2a00 2024-04-26T02:00:57.730 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.730+0000 7f8243d6a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561835dd3ef0 con 0x561835ec7820 2024-04-26T02:00:57.730 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.730+0000 7f8243d6a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561835e298c0 con 0x561835ec8140 2024-04-26T02:00:57.730 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.730+0000 7f823a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561835ec7820 0x561835ec7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:57.730 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.730+0000 7f823a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561835ec7820 0x561835ec7c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:45428/0 (socket says 172.21.15.151:45428) 2024-04-26T02:00:57.731 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.730+0000 7f823a800640 1 -- 172.21.15.151:0/545902002 learned_addr learned my addr 172.21.15.151:0/545902002 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:00:57.731 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.730+0000 7f8239400640 1 --2- 172.21.15.151:0/545902002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561835ed2a00 0x561835ed4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:57.731 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.730+0000 7f8239e00640 1 --2- 172.21.15.151:0/545902002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561835ec8140 0x561835ed24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:57.731 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.730+0000 7f823a800640 1 -- 172.21.15.151:0/545902002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561835ec8140 msgr2=0x561835ed24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:57.731 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.730+0000 7f823a800640 1 --2- 172.21.15.151:0/545902002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561835ec8140 0x561835ed24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:57.731 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.730+0000 7f823a800640 1 -- 172.21.15.151:0/545902002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561835ed2a00 msgr2=0x561835ed4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:57.731 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.730+0000 7f823a800640 1 --2- 172.21.15.151:0/545902002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561835ed2a00 0x561835ed4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:57.731 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.730+0000 7f823a800640 1 -- 172.21.15.151:0/545902002 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561835b40210 con 0x561835ec7820 2024-04-26T02:00:57.731 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.730+0000 7f8239e00640 1 --2- 172.21.15.151:0/545902002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561835ec8140 0x561835ed24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:00:57.731 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.731+0000 7f823a800640 1 --2- 172.21.15.151:0/545902002 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561835ec7820 0x561835ec7c00 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8234009f60 tx=0x7f8234009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ff0902e3553967c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:57.731 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.731+0000 7f8232a00640 1 --2- 172.21.15.151:0/678659587 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f82200a3dc0 0x7f82200a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:57.731 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.731+0000 7f8227400640 1 -- 172.21.15.151:0/545902002 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823401e070 con 0x561835ec7820 2024-04-26T02:00:57.731 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.731+0000 7f8227400640 1 -- 172.21.15.151:0/545902002 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8234003c30 con 0x561835ec7820 2024-04-26T02:00:57.732 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.731+0000 7f8227400640 1 -- 172.21.15.151:0/545902002 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823400f840 con 0x561835ec7820 2024-04-26T02:00:57.732 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.731+0000 7f8232a00640 1 -- 172.21.15.151:0/678659587 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8455+0+0 (secure 0 0 0) 0x7f82280d0b60 con 0x7f823c1094b0 2024-04-26T02:00:57.732 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.731+0000 7f8239e00640 1 --2- 172.21.15.151:0/678659587 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f82200a3dc0 0x7f82200a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:57.734 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.732+0000 7f8243d6a740 1 -- 172.21.15.151:0/545902002 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561835ec7820 msgr2=0x561835ec7c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:57.735 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.732+0000 7f8243d6a740 1 --2- 172.21.15.151:0/545902002 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561835ec7820 0x561835ec7c00 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8234009f60 tx=0x7f8234009f90 comp rx=0 tx=0).stop 2024-04-26T02:00:57.735 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.732+0000 7f8239e00640 1 --2- 172.21.15.151:0/678659587 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f82200a3dc0 0x7f82200a6280 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f822c005ab0 tx=0x7f822c005a20 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:57.735 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.732+0000 7f8243d6a740 1 -- 172.21.15.151:0/545902002 shutdown_connections 2024-04-26T02:00:57.735 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.732+0000 7f8243d6a740 1 --2- 172.21.15.151:0/545902002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561835ed2a00 0x561835ed4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:57.735 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.732+0000 7f8243d6a740 1 --2- 172.21.15.151:0/545902002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561835ec8140 0x561835ed24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:57.735 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.732+0000 7f8243d6a740 1 --2- 172.21.15.151:0/545902002 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561835ec7820 0x561835ec7c00 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:57.735 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.732+0000 7f8243d6a740 1 -- 172.21.15.151:0/545902002 >> 172.21.15.151:0/545902002 conn(0x561835ec3d30 msgr2=0x561835ec4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:57.735 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.732+0000 7f8243d6a740 1 -- 172.21.15.151:0/545902002 shutdown_connections 2024-04-26T02:00:57.735 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.732+0000 7f8243d6a740 1 -- 172.21.15.151:0/545902002 wait complete. 2024-04-26T02:00:57.735 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.733+0000 7f8243d6a740 1 -- start start 2024-04-26T02:00:57.735 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.733+0000 7f8243d6a740 1 Processor -- start 2024-04-26T02:00:57.735 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.733+0000 7f8243d6a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561835ec7820 0x561835ecf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:57.736 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.733+0000 7f8243d6a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561835ec8140 0x561835ecff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:57.736 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.733+0000 7f8243d6a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561835ed2a00 0x561835ed0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:57.736 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.734+0000 7f8239e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561835ec7820 0x561835ecf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:57.736 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.734+0000 7f8239e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561835ec7820 0x561835ecf9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:45436/0 (socket says 172.21.15.151:45436) 2024-04-26T02:00:57.736 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.734+0000 7f8243d6a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x561835ed5630 con 0x561835ec8140 2024-04-26T02:00:57.736 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.734+0000 7f8243d6a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561835de7310 con 0x561835ec7820 2024-04-26T02:00:57.736 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.734+0000 7f8243d6a740 1 -- 172.21.15.151:0/2856361705 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561835ed57b0 con 0x561835ed2a00 2024-04-26T02:00:57.736 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.734+0000 7f8239e00640 1 -- 172.21.15.151:0/2856361705 learned_addr learned my addr 172.21.15.151:0/2856361705 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:00:57.736 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.734+0000 7f8239400640 1 --2- 172.21.15.151:0/2856361705 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561835ec8140 0x561835ecff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:57.736 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.734+0000 7f8243d6a740 10 client.0 _ll_register_callbacks cb 0x561835db5cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:00:57.736 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.734+0000 7f823a800640 1 --2- 172.21.15.151:0/2856361705 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561835ed2a00 0x561835ed0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:57.736 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.734+0000 7f8239e00640 1 -- 172.21.15.151:0/2856361705 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561835ed2a00 msgr2=0x561835ed0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.734+0000 7f8239e00640 1 --2- 172.21.15.151:0/2856361705 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561835ed2a00 0x561835ed0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.734+0000 7f8239e00640 1 -- 172.21.15.151:0/2856361705 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561835ec8140 msgr2=0x561835ecff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.734+0000 7f8239e00640 1 --2- 172.21.15.151:0/2856361705 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561835ec8140 0x561835ecff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.734+0000 7f8239e00640 1 -- 172.21.15.151:0/2856361705 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561835c0c620 con 0x561835ec7820 2024-04-26T02:00:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.735+0000 7f8239e00640 1 --2- 172.21.15.151:0/2856361705 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561835ec7820 0x561835ecf9c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f822c033000 tx=0x7f822c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7b9c2e96a7341c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.735+0000 7f821be00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T02:00:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.735+0000 7f821be00640 1 -- 172.21.15.151:0/2856361705 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822c04d070 con 0x561835ec7820 2024-04-26T02:00:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.735+0000 7f821be00640 1 -- 172.21.15.151:0/2856361705 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8455+0+0 (secure 0 0 0) 0x7f822c004c40 con 0x561835ec7820 2024-04-26T02:00:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.735+0000 7f8243d6a740 1 -- 172.21.15.151:0/2856361705 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8234003c30 con 0x561835ec7820 2024-04-26T02:00:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.735+0000 7f821be00640 1 -- 172.21.15.151:0/2856361705 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f822c057050 con 0x561835ec7820 2024-04-26T02:00:57.738 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.735+0000 7f8243d6a740 10 client.26030 fetch_fsmap learned FSMap version 35 2024-04-26T02:00:57.738 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.736+0000 7f8243d6a740 1 -- 172.21.15.151:0/2856361705 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x561835ee18a0 con 0x561835ec7820 2024-04-26T02:00:57.738 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.736+0000 7f821be00640 1 -- 172.21.15.151:0/2856361705 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8234003c30 con 0x561835ec7820 2024-04-26T02:00:57.738 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.736+0000 7f821be00640 1 -- 172.21.15.151:0/2856361705 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 35) ==== 8338+0+0 (secure 0 0 0) 0x7f822c032540 con 0x561835ec7820 2024-04-26T02:00:57.739 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.736+0000 7f8243d6a740 10 client.26030 fetch_fsmap finished waiting for FSMap version 35 2024-04-26T02:00:57.740 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.736+0000 7f8243d6a740 10 client.26030 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:00:57.740 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.736+0000 7f8243d6a740 20 client.26030 populate_metadata read hostname 'smithi151' 2024-04-26T02:00:57.740 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.736+0000 7f8243d6a740 1 --2- 172.21.15.151:0/2856361705 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x561835ee2db0 0x561835ee5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:57.740 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.736+0000 7f8243d6a740 4 client.26030 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:00:57.740 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.736+0000 7f8243d6a740 1 -- 172.21.15.151:0/2856361705 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561835e418b0 con 0x561835ee2db0 2024-04-26T02:00:57.740 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.737+0000 7f8239400640 1 --2- 172.21.15.151:0/2856361705 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x561835ee2db0 0x561835ee5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:57.740 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.737+0000 7f8239400640 1 --2- 172.21.15.151:0/2856361705 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x561835ee2db0 0x561835ee5240 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab8e083b4547d29 server_cookie=2aaff858e2f403d0 in_seq=0 out_seq=0 2024-04-26T02:00:57.740 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.737+0000 7f821be00640 10 client.26030 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:00:57.740 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.738+0000 7f821be00640 1 -- 172.21.15.151:0/2856361705 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561835e418b0 con 0x561835ee2db0 2024-04-26T02:00:57.740 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.738+0000 7f821be00640 10 client.26030 handle_command_reply: tid=0 2024-04-26T02:00:57.740 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.738+0000 7f8243d6a740 1 client.26030 shutdown 2024-04-26T02:00:57.740 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.738+0000 7f8243d6a740 1 -- 172.21.15.151:0/2856361705 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561835ec7820 msgr2=0x561835ecf9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:57.740 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.738+0000 7f8243d6a740 1 --2- 172.21.15.151:0/2856361705 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561835ec7820 0x561835ecf9c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f822c033000 tx=0x7f822c009260 comp rx=0 tx=0).stop 2024-04-26T02:00:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.738+0000 7f8243d6a740 1 -- 172.21.15.151:0/2856361705 shutdown_connections 2024-04-26T02:00:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.738+0000 7f8243d6a740 1 --2- 172.21.15.151:0/2856361705 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x561835ee2db0 0x561835ee5240 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.738+0000 7f8243d6a740 1 --2- 172.21.15.151:0/2856361705 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561835ed2a00 0x561835ed0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.738+0000 7f8243d6a740 1 --2- 172.21.15.151:0/2856361705 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561835ec8140 0x561835ecff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.738+0000 7f8243d6a740 1 --2- 172.21.15.151:0/2856361705 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561835ec7820 0x561835ecf9c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.738+0000 7f821be00640 0 client.26030 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:00:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.738+0000 7f8243d6a740 1 -- 172.21.15.151:0/2856361705 >> 172.21.15.151:0/2856361705 conn(0x561835ec3d30 msgr2=0x561835ec9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.738+0000 7f8243d6a740 1 -- 172.21.15.151:0/2856361705 shutdown_connections 2024-04-26T02:00:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.738+0000 7f8243d6a740 1 -- 172.21.15.151:0/2856361705 wait complete. 2024-04-26T02:00:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.739+0000 7f8243d6a740 20 client.26030 trim_cache size 0 max 16384 2024-04-26T02:00:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.759+0000 7f8243d6a740 1 Processor -- start 2024-04-26T02:00:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.759+0000 7f8243d6a740 1 -- start start 2024-04-26T02:00:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.760+0000 7f8243d6a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561835ec7820 0x561835ede4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.760+0000 7f8243d6a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561835ec8140 0x561835ec2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.760+0000 7f8243d6a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561835ed2a00 0x561835eeab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.760+0000 7f8243d6a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f823c1b25c0 con 0x561835ed2a00 2024-04-26T02:00:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.760+0000 7f8243d6a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f823c1afe90 con 0x561835ec7820 2024-04-26T02:00:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.760+0000 7f8243d6a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f823c1add00 con 0x561835ec8140 2024-04-26T02:00:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.760+0000 7f823a800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561835ed2a00 0x561835eeab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.760+0000 7f8239400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561835ec8140 0x561835ec2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.760+0000 7f8239e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561835ec7820 0x561835ede4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.760+0000 7f823a800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561835ed2a00 0x561835eeab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:59706/0 (socket says 172.21.15.151:59706) 2024-04-26T02:00:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.760+0000 7f823a800640 1 -- 172.21.15.151:0/167502910 learned_addr learned my addr 172.21.15.151:0/167502910 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:00:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.760+0000 7f8239400640 1 -- 172.21.15.151:0/167502910 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561835ec7820 msgr2=0x561835ede4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.760+0000 7f8239400640 1 --2- 172.21.15.151:0/167502910 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561835ec7820 0x561835ede4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.760+0000 7f8239400640 1 -- 172.21.15.151:0/167502910 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561835ed2a00 msgr2=0x561835eeab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.760+0000 7f8239400640 1 --2- 172.21.15.151:0/167502910 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561835ed2a00 0x561835eeab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.760+0000 7f8239400640 1 -- 172.21.15.151:0/167502910 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561835ed4df0 con 0x561835ec8140 2024-04-26T02:00:57.762 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.761+0000 7f8239400640 1 --2- 172.21.15.151:0/167502910 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561835ec8140 0x561835ec2d10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f82280ce290 tx=0x7f822800ce60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c510d33492f4454e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:57.762 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.761+0000 7f8226a00640 1 -- 172.21.15.151:0/167502910 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822809d1e0 con 0x561835ec8140 2024-04-26T02:00:57.762 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.761+0000 7f8226a00640 1 -- 172.21.15.151:0/167502910 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f822809cae0 con 0x561835ec8140 2024-04-26T02:00:57.762 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.761+0000 7f8226a00640 1 -- 172.21.15.151:0/167502910 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8228099150 con 0x561835ec8140 2024-04-26T02:00:57.762 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.762+0000 7f8243d6a740 1 -- 172.21.15.151:0/167502910 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561835ec8140 msgr2=0x561835ec2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:57.762 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.762+0000 7f8243d6a740 1 --2- 172.21.15.151:0/167502910 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561835ec8140 0x561835ec2d10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f82280ce290 tx=0x7f822800ce60 comp rx=0 tx=0).stop 2024-04-26T02:00:57.762 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.762+0000 7f8243d6a740 1 -- 172.21.15.151:0/167502910 shutdown_connections 2024-04-26T02:00:57.762 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.762+0000 7f8243d6a740 1 --2- 172.21.15.151:0/167502910 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561835ed2a00 0x561835eeab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:57.762 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.762+0000 7f8243d6a740 1 --2- 172.21.15.151:0/167502910 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561835ec8140 0x561835ec2d10 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:57.762 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.762+0000 7f8243d6a740 1 --2- 172.21.15.151:0/167502910 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561835ec7820 0x561835ede4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:57.762 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.762+0000 7f8243d6a740 1 -- 172.21.15.151:0/167502910 >> 172.21.15.151:0/167502910 conn(0x561835ec3d30 msgr2=0x561835ec2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:57.762 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.762+0000 7f8243d6a740 1 -- 172.21.15.151:0/167502910 shutdown_connections 2024-04-26T02:00:57.762 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.762+0000 7f8243d6a740 1 -- 172.21.15.151:0/167502910 wait complete. 2024-04-26T02:00:57.763 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.762+0000 7f8243d6a740 1 -- start start 2024-04-26T02:00:57.763 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.763+0000 7f8243d6a740 1 Processor -- start 2024-04-26T02:00:57.765 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.763+0000 7f8243d6a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561835ec7820 0x561835ed8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:57.766 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.763+0000 7f8243d6a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561835ec8140 0x561835ed8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:57.766 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.763+0000 7f8243d6a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561835ed2a00 0x561835f483d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:57.766 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.763+0000 7f8243d6a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f823c1ae350 con 0x561835ed2a00 2024-04-26T02:00:57.766 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.763+0000 7f8243d6a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f823c1af7c0 con 0x561835ec8140 2024-04-26T02:00:57.766 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.763+0000 7f8243d6a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f823c1aea20 con 0x561835ec7820 2024-04-26T02:00:57.766 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.763+0000 7f8243d6a740 10 client.0 _ll_register_callbacks cb 0x7f822c004480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:00:57.766 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.763+0000 7f8239400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561835ec8140 0x561835ed8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:57.766 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.763+0000 7f8239e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561835ec7820 0x561835ed8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:57.766 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.763+0000 7f823a800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561835ed2a00 0x561835f483d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:57.766 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.763+0000 7f8239e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561835ec7820 0x561835ed8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:50170/0 (socket says 172.21.15.151:50170) 2024-04-26T02:00:57.766 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.763+0000 7f8239400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561835ec8140 0x561835ed8d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:45440/0 (socket says 172.21.15.151:45440) 2024-04-26T02:00:57.767 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.763+0000 7f8239e00640 1 -- 172.21.15.151:0/3223650084 learned_addr learned my addr 172.21.15.151:0/3223650084 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:00:57.767 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.764+0000 7f8239e00640 1 -- 172.21.15.151:0/3223650084 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561835ec8140 msgr2=0x561835ed8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:57.767 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.764+0000 7f8239e00640 1 --2- 172.21.15.151:0/3223650084 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561835ec8140 0x561835ed8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:57.767 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.764+0000 7f8239e00640 1 -- 172.21.15.151:0/3223650084 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561835ed2a00 msgr2=0x561835f483d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:57.767 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.764+0000 7f8239e00640 1 --2- 172.21.15.151:0/3223650084 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561835ed2a00 0x561835f483d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:57.767 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.764+0000 7f8239e00640 1 -- 172.21.15.151:0/3223650084 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561835f48910 con 0x561835ec7820 2024-04-26T02:00:57.767 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.764+0000 7f8239400640 1 --2- 172.21.15.151:0/3223650084 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561835ec8140 0x561835ed8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:00:57.767 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.764+0000 7f8239e00640 1 --2- 172.21.15.151:0/3223650084 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561835ec7820 0x561835ed8830 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f822c02a9e0 tx=0x7f822c004610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e329115fadc8293 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:57.767 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.764+0000 7f821be00640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-26T02:00:57.768 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.765+0000 7f821be00640 1 -- 172.21.15.151:0/3223650084 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822c04d040 con 0x561835ec7820 2024-04-26T02:00:57.768 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.765+0000 7f821be00640 1 -- 172.21.15.151:0/3223650084 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8455+0+0 (secure 0 0 0) 0x7f822c032de0 con 0x561835ec7820 2024-04-26T02:00:57.768 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.765+0000 7f8243d6a740 1 -- 172.21.15.151:0/3223650084 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f822809cae0 con 0x561835ec7820 2024-04-26T02:00:57.768 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.765+0000 7f821be00640 1 -- 172.21.15.151:0/3223650084 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f822c035040 con 0x561835ec7820 2024-04-26T02:00:57.768 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.765+0000 7f8243d6a740 10 client.26068 fetch_fsmap learned FSMap version 35 2024-04-26T02:00:57.768 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.765+0000 7f8243d6a740 1 -- 172.21.15.151:0/3223650084 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x561835f49ac0 con 0x561835ec7820 2024-04-26T02:00:57.768 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.765+0000 7f821be00640 1 -- 172.21.15.151:0/3223650084 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f822809cae0 con 0x561835ec7820 2024-04-26T02:00:57.768 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.765+0000 7f821be00640 1 -- 172.21.15.151:0/3223650084 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 35) ==== 8338+0+0 (secure 0 0 0) 0x7f822c003b60 con 0x561835ec7820 2024-04-26T02:00:57.768 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.765+0000 7f8243d6a740 10 client.26068 fetch_fsmap finished waiting for FSMap version 35 2024-04-26T02:00:57.768 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.766+0000 7f8243d6a740 10 client.26068 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:00:57.768 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.766+0000 7f8243d6a740 20 client.26068 populate_metadata read hostname 'smithi151' 2024-04-26T02:00:57.769 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.766+0000 7f8243d6a740 1 --2- 172.21.15.151:0/3223650084 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x561835f4a230 0x561835f4c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:57.769 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.766+0000 7f8243d6a740 4 client.26068 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T02:00:57.769 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.766+0000 7f8243d6a740 1 -- 172.21.15.151:0/3223650084 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561835e3b320 con 0x561835f4a230 2024-04-26T02:00:57.769 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.766+0000 7f8239400640 1 --2- 172.21.15.151:0/3223650084 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x561835f4a230 0x561835f4c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:57.769 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.766+0000 7f821be00640 10 client.26068 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:00:57.769 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.766+0000 7f8239400640 1 --2- 172.21.15.151:0/3223650084 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x561835f4a230 0x561835f4c650 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a75605ef63e9fe91 server_cookie=529d37c4d4fd4949 in_seq=0 out_seq=0 2024-04-26T02:00:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.776+0000 7f821be00640 1 -- 172.21.15.151:0/3223650084 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561835e3b320 con 0x561835f4a230 2024-04-26T02:00:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.776+0000 7f821be00640 10 client.26068 handle_command_reply: tid=0 2024-04-26T02:00:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.776+0000 7f8243d6a740 1 client.26068 shutdown 2024-04-26T02:00:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.776+0000 7f8243d6a740 1 -- 172.21.15.151:0/3223650084 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561835ec7820 msgr2=0x561835ed8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.776+0000 7f8243d6a740 1 --2- 172.21.15.151:0/3223650084 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561835ec7820 0x561835ed8830 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f822c02a9e0 tx=0x7f822c004610 comp rx=0 tx=0).stop 2024-04-26T02:00:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.776+0000 7f8243d6a740 1 -- 172.21.15.151:0/3223650084 shutdown_connections 2024-04-26T02:00:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.777+0000 7f8243d6a740 1 --2- 172.21.15.151:0/3223650084 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x561835f4a230 0x561835f4c650 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.777+0000 7f8243d6a740 1 --2- 172.21.15.151:0/3223650084 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561835ed2a00 0x561835f483d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.777+0000 7f8243d6a740 1 --2- 172.21.15.151:0/3223650084 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561835ec8140 0x561835ed8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.777+0000 7f8243d6a740 1 --2- 172.21.15.151:0/3223650084 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561835ec7820 0x561835ed8830 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.777+0000 7f8243d6a740 1 -- 172.21.15.151:0/3223650084 >> 172.21.15.151:0/3223650084 conn(0x561835ec3d30 msgr2=0x561835ec2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.777+0000 7f821be00640 0 client.26068 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:00:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.777+0000 7f8243d6a740 1 -- 172.21.15.151:0/3223650084 shutdown_connections 2024-04-26T02:00:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.777+0000 7f8243d6a740 1 -- 172.21.15.151:0/3223650084 wait complete. 2024-04-26T02:00:57.777 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.777+0000 7f8243d6a740 20 client.26068 trim_cache size 0 max 16384 2024-04-26T02:00:57.777 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:{ 2024-04-26T02:00:57.777 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "message": "", 2024-04-26T02:00:57.777 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "return_code": 0 2024-04-26T02:00:57.777 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:} 2024-04-26T02:00:57.778 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.778+0000 7f8240e00640 1 -- 172.21.15.151:0/678659587 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f82200a3dc0 msgr2=0x7f82200a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:57.778 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.778+0000 7f8240e00640 1 --2- 172.21.15.151:0/678659587 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f82200a3dc0 0x7f82200a6280 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f822c005ab0 tx=0x7f822c005a20 comp rx=0 tx=0).stop 2024-04-26T02:00:57.778 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.778+0000 7f8240e00640 1 -- 172.21.15.151:0/678659587 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f823c1094b0 msgr2=0x7f823c1a0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:57.778 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.778+0000 7f8240e00640 1 --2- 172.21.15.151:0/678659587 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f823c1094b0 0x7f823c1a0b50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f822800e9f0 tx=0x7f822800edf0 comp rx=0 tx=0).stop 2024-04-26T02:00:57.778 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.778+0000 7f8240e00640 1 -- 172.21.15.151:0/678659587 shutdown_connections 2024-04-26T02:00:57.778 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.778+0000 7f8240e00640 1 --2- 172.21.15.151:0/678659587 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f82200a3dc0 0x7f82200a6280 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:57.778 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.778+0000 7f8240e00640 1 --2- 172.21.15.151:0/678659587 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f823c19a780 0x7f823c19ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:57.778 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.778+0000 7f8240e00640 1 --2- 172.21.15.151:0/678659587 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f823c1094b0 0x7f823c1a0b50 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:57.779 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.778+0000 7f8240e00640 1 --2- 172.21.15.151:0/678659587 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f823c1043e0 0x7f823c1a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:57.779 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.778+0000 7f8240e00640 1 -- 172.21.15.151:0/678659587 >> 172.21.15.151:0/678659587 conn(0x7f823c0ffc60 msgr2=0x7f823c101fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:57.779 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.779+0000 7f8240e00640 1 -- 172.21.15.151:0/678659587 shutdown_connections 2024-04-26T02:00:57.779 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:00:57.779+0000 7f8240e00640 1 -- 172.21.15.151:0/678659587 wait complete. 2024-04-26T02:00:58.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:00:57 smithi172 ceph-mon[32785]: pgmap v228: 97 pgs: 97 active+clean; 55 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 129 KiB/s wr, 22 op/s 2024-04-26T02:00:58.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:00:57 smithi165 ceph-mon[32648]: pgmap v228: 97 pgs: 97 active+clean; 55 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 129 KiB/s wr, 22 op/s 2024-04-26T02:00:58.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:00:57 smithi151 ceph-mon[28057]: pgmap v228: 97 pgs: 97 active+clean; 55 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 129 KiB/s wr, 22 op/s 2024-04-26T02:01:00.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:00:59 smithi172 ceph-mon[32785]: pgmap v229: 97 pgs: 97 active+clean; 58 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 977 KiB/s wr, 148 op/s 2024-04-26T02:01:00.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:00:59 smithi165 ceph-mon[32648]: pgmap v229: 97 pgs: 97 active+clean; 58 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 977 KiB/s wr, 148 op/s 2024-04-26T02:01:00.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:00:59 smithi151 ceph-mon[28057]: pgmap v229: 97 pgs: 97 active+clean; 58 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 977 KiB/s wr, 148 op/s 2024-04-26T02:01:02.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:01:01 smithi172 ceph-mon[32785]: pgmap v230: 97 pgs: 97 active+clean; 58 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 970 KiB/s wr, 144 op/s 2024-04-26T02:01:02.347 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:01:01 smithi165 ceph-mon[32648]: pgmap v230: 97 pgs: 97 active+clean; 58 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 970 KiB/s wr, 144 op/s 2024-04-26T02:01:02.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:01:01 smithi151 ceph-mon[28057]: pgmap v230: 97 pgs: 97 active+clean; 58 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 970 KiB/s wr, 144 op/s 2024-04-26T02:01:03.094 INFO:tasks.workunit.client.0.smithi151.stdout:50000 write 0x5304f4 thru 0x53fa11 (0xf51e bytes) 2024-04-26T02:01:03.095 INFO:tasks.workunit.client.0.smithi151.stdout:All 50000 operations completed A-OK! 2024-04-26T02:01:03.109 INFO:tasks.workunit.client.0.smithi151.stderr:+ ./fsx -z 100MB -N 50000 -p 10000 -l 104857600 2024-04-26T02:01:03.117 INFO:tasks.workunit.client.0.smithi151.stdout:Seed set to 1 2024-04-26T02:01:04.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:01:03 smithi172 ceph-mon[32785]: pgmap v231: 97 pgs: 97 active+clean; 54 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 993 KiB/s wr, 147 op/s 2024-04-26T02:01:04.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:01:03 smithi165 ceph-mon[32648]: pgmap v231: 97 pgs: 97 active+clean; 54 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 993 KiB/s wr, 147 op/s 2024-04-26T02:01:04.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:01:03 smithi151 ceph-mon[28057]: pgmap v231: 97 pgs: 97 active+clean; 54 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 993 KiB/s wr, 147 op/s 2024-04-26T02:01:04.764 INFO:tasks.workunit.client.0.smithi151.stderr:main: filesystem does not support fallocate mode 0, disabling! 2024-04-26T02:01:04.764 INFO:tasks.workunit.client.0.smithi151.stderr:main: filesystem does not support fallocate mode FALLOC_FL_KEEP_SIZE, disabling! 2024-04-26T02:01:04.766 INFO:tasks.workunit.client.0.smithi151.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2024-04-26T02:01:04.767 INFO:tasks.workunit.client.0.smithi151.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2024-04-26T02:01:04.767 INFO:tasks.workunit.client.0.smithi151.stderr:main: filesystem does not support clone range, disabling! 2024-04-26T02:01:04.769 INFO:tasks.workunit.client.0.smithi151.stderr:main: filesystem does not support dedupe range, disabling! 2024-04-26T02:01:04.776 INFO:tasks.workunit.client.0.smithi151.stdout:skipping zero size read 2024-04-26T02:01:04.777 INFO:tasks.workunit.client.0.smithi151.stderr:main: filesystem does not support exchange range, disabling! 2024-04-26T02:01:04.802 INFO:tasks.workunit.client.0.smithi151.stdout:truncating to largest ever: 0x1c37b90 2024-04-26T02:01:04.811 INFO:tasks.workunit.client.0.smithi151.stdout:truncating to largest ever: 0x5bd330d 2024-04-26T02:01:04.847 INFO:tasks.workunit.client.0.smithi151.stdout:copying to largest ever: 0x637bf99 2024-04-26T02:01:05.261 INFO:tasks.workunit.client.0.smithi151.stdout:copying to largest ever: 0x63d9b9c 2024-04-26T02:01:05.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:01:05 smithi172 ceph-mon[32785]: osdmap e99: 12 total, 12 up, 12 in 2024-04-26T02:01:05.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:01:05 smithi165 ceph-mon[32648]: osdmap e99: 12 total, 12 up, 12 in 2024-04-26T02:01:05.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:01:05 smithi151 ceph-mon[28057]: osdmap e99: 12 total, 12 up, 12 in 2024-04-26T02:01:06.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:01:06 smithi172 ceph-mon[32785]: pgmap v233: 97 pgs: 97 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.8 MiB/s wr, 300 op/s 2024-04-26T02:01:06.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:01:06 smithi165 ceph-mon[32648]: pgmap v233: 97 pgs: 97 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.8 MiB/s wr, 300 op/s 2024-04-26T02:01:06.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:01:06 smithi151 ceph-mon[28057]: pgmap v233: 97 pgs: 97 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.8 MiB/s wr, 300 op/s 2024-04-26T02:01:08.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:01:08 smithi172 ceph-mon[32785]: pgmap v234: 97 pgs: 97 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.8 MiB/s wr, 300 op/s 2024-04-26T02:01:08.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:01:08 smithi165 ceph-mon[32648]: pgmap v234: 97 pgs: 97 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.8 MiB/s wr, 300 op/s 2024-04-26T02:01:08.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:01:08 smithi151 ceph-mon[28057]: pgmap v234: 97 pgs: 97 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.8 MiB/s wr, 300 op/s 2024-04-26T02:01:10.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:01:10 smithi172 ceph-mon[32785]: pgmap v235: 97 pgs: 97 active+clean; 80 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.3 MiB/s wr, 270 op/s 2024-04-26T02:01:10.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:01:10 smithi172 ceph-mon[32785]: osdmap e100: 12 total, 12 up, 12 in 2024-04-26T02:01:10.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:01:10 smithi165 ceph-mon[32648]: pgmap v235: 97 pgs: 97 active+clean; 80 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.3 MiB/s wr, 270 op/s 2024-04-26T02:01:10.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:01:10 smithi165 ceph-mon[32648]: osdmap e100: 12 total, 12 up, 12 in 2024-04-26T02:01:10.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:01:10 smithi151 ceph-mon[28057]: pgmap v235: 97 pgs: 97 active+clean; 80 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.3 MiB/s wr, 270 op/s 2024-04-26T02:01:10.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:01:10 smithi151 ceph-mon[28057]: osdmap e100: 12 total, 12 up, 12 in 2024-04-26T02:01:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.929+0000 7fb939800640 1 -- 172.21.15.151:0/245889412 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb9341080e0 msgr2=0x7fb9341084c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.929+0000 7fb939800640 1 --2- 172.21.15.151:0/245889412 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb9341080e0 0x7fb9341084c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb924009900 tx=0x7fb92402a480 comp rx=0 tx=0).stop 2024-04-26T02:01:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.929+0000 7fb939800640 1 -- 172.21.15.151:0/245889412 shutdown_connections 2024-04-26T02:01:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.929+0000 7fb939800640 1 --2- 172.21.15.151:0/245889412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb93406b120 0x7fb93410edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.929+0000 7fb939800640 1 --2- 172.21.15.151:0/245889412 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb93406a780 0x7fb93406abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.929+0000 7fb939800640 1 --2- 172.21.15.151:0/245889412 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb9341080e0 0x7fb9341084c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.929+0000 7fb939800640 1 -- 172.21.15.151:0/245889412 >> 172.21.15.151:0/245889412 conn(0x7fb934076c50 msgr2=0x7fb934077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.929+0000 7fb939800640 1 -- 172.21.15.151:0/245889412 shutdown_connections 2024-04-26T02:01:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.929+0000 7fb939800640 1 -- 172.21.15.151:0/245889412 wait complete. 2024-04-26T02:01:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.930+0000 7fb939800640 1 Processor -- start 2024-04-26T02:01:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.930+0000 7fb939800640 1 -- start start 2024-04-26T02:01:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.931+0000 7fb939800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb93406a780 0x7fb9341a0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.931+0000 7fb939800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb93406b120 0x7fb9341a0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.931+0000 7fb939800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb9341080e0 0x7fb93419a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.931+0000 7fb939800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb934112200 con 0x7fb9341080e0 2024-04-26T02:01:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.931+0000 7fb939800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb934112080 con 0x7fb93406b120 2024-04-26T02:01:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.931+0000 7fb939800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb934112380 con 0x7fb93406a780 2024-04-26T02:01:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.931+0000 7fb932000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb93406b120 0x7fb9341a0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.931+0000 7fb932a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb93406a780 0x7fb9341a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.931+0000 7fb933400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb9341080e0 0x7fb93419a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.931+0000 7fb932000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb93406b120 0x7fb9341a0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:39194/0 (socket says 172.21.15.151:39194) 2024-04-26T02:01:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.931+0000 7fb932000640 1 -- 172.21.15.151:0/1289737156 learned_addr learned my addr 172.21.15.151:0/1289737156 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:01:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.931+0000 7fb932a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb93406a780 0x7fb9341a0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:50110/0 (socket says 172.21.15.151:50110) 2024-04-26T02:01:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.932+0000 7fb932000640 1 -- 172.21.15.151:0/1289737156 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb93406a780 msgr2=0x7fb9341a0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.932+0000 7fb932000640 1 --2- 172.21.15.151:0/1289737156 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb93406a780 0x7fb9341a0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.932+0000 7fb932000640 1 -- 172.21.15.151:0/1289737156 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb9341080e0 msgr2=0x7fb93419a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.932+0000 7fb932000640 1 --2- 172.21.15.151:0/1289737156 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb9341080e0 0x7fb93419a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.932+0000 7fb932000640 1 -- 172.21.15.151:0/1289737156 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb93419af10 con 0x7fb93406b120 2024-04-26T02:01:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.932+0000 7fb932a00640 1 --2- 172.21.15.151:0/1289737156 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb93406a780 0x7fb9341a0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:01:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.932+0000 7fb932000640 1 --2- 172.21.15.151:0/1289737156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb93406b120 0x7fb9341a0ad0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb92000e9f0 tx=0x7fb92000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.932+0000 7fb92b400640 1 -- 172.21.15.151:0/1289737156 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92000cd80 con 0x7fb93406b120 2024-04-26T02:01:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.932+0000 7fb92b400640 1 -- 172.21.15.151:0/1289737156 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9200054d0 con 0x7fb93406b120 2024-04-26T02:01:10.933 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.932+0000 7fb939800640 1 -- 172.21.15.151:0/1289737156 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb93419b200 con 0x7fb93406b120 2024-04-26T02:01:10.933 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.932+0000 7fb92b400640 1 -- 172.21.15.151:0/1289737156 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb920010690 con 0x7fb93406b120 2024-04-26T02:01:10.933 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.932+0000 7fb939800640 1 -- 172.21.15.151:0/1289737156 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9341a7510 con 0x7fb93406b120 2024-04-26T02:01:10.935 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.936+0000 7fb93c5e8740 1 Processor -- start 2024-04-26T02:01:10.936 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.936+0000 7fb93c5e8740 1 -- start start 2024-04-26T02:01:10.936 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.936+0000 7fb92b400640 1 -- 172.21.15.151:0/1289737156 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7fb920004e90 con 0x7fb93406b120 2024-04-26T02:01:10.936 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.937+0000 7fb93c5e8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f763c87820 0x55f763c87c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:10.936 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.937+0000 7fb93c5e8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f763c88140 0x55f763c924c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:10.936 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.937+0000 7fb93c5e8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f763c92a00 0x55f763c94df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:10.936 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.937+0000 7fb93c5e8740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f763bed900 con 0x55f763c92a00 2024-04-26T02:01:10.936 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.937+0000 7fb93c5e8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f763b93ef0 con 0x55f763c87820 2024-04-26T02:01:10.936 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.937+0000 7fb93c5e8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f763be98c0 con 0x55f763c88140 2024-04-26T02:01:10.936 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.937+0000 7fb932a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f763c88140 0x55f763c924c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:10.936 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.937+0000 7fb932a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f763c88140 0x55f763c924c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:50118/0 (socket says 172.21.15.151:50118) 2024-04-26T02:01:10.936 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.937+0000 7fb932a00640 1 -- 172.21.15.151:0/435036220 learned_addr learned my addr 172.21.15.151:0/435036220 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:01:10.936 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.937+0000 7fb933400640 1 --2- 172.21.15.151:0/435036220 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f763c87820 0x55f763c87c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:10.937 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.937+0000 7fb932000640 1 --2- 172.21.15.151:0/435036220 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f763c92a00 0x55f763c94df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:10.938 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.937+0000 7fb932a00640 1 -- 172.21.15.151:0/435036220 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f763c87820 msgr2=0x55f763c87c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:10.938 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.937+0000 7fb932a00640 1 --2- 172.21.15.151:0/435036220 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f763c87820 0x55f763c87c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:10.938 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.937+0000 7fb932a00640 1 -- 172.21.15.151:0/435036220 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f763c92a00 msgr2=0x55f763c94df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.937+0000 7fb932a00640 1 --2- 172.21.15.151:0/435036220 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f763c92a00 0x55f763c94df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.937+0000 7fb932a00640 1 -- 172.21.15.151:0/435036220 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f763900210 con 0x55f763c88140 2024-04-26T02:01:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.938+0000 7fb932a00640 1 --2- 172.21.15.151:0/435036220 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f763c88140 0x55f763c924c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb924009230 tx=0x7fb924005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58e366a90f4589a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.938+0000 7fb91fe00640 1 -- 172.21.15.151:0/435036220 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb924035040 con 0x55f763c88140 2024-04-26T02:01:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.938+0000 7fb91fe00640 1 -- 172.21.15.151:0/435036220 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb924004770 con 0x55f763c88140 2024-04-26T02:01:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.938+0000 7fb91fe00640 1 -- 172.21.15.151:0/435036220 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb924003910 con 0x55f763c88140 2024-04-26T02:01:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.938+0000 7fb92b400640 1 --2- 172.21.15.151:0/1289737156 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fb9180a3e40 0x7fb9180a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.939+0000 7fb92b400640 1 -- 172.21.15.151:0/1289737156 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8455+0+0 (secure 0 0 0) 0x7fb9200d0bb0 con 0x7fb93406b120 2024-04-26T02:01:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.939+0000 7fb933400640 1 --2- 172.21.15.151:0/1289737156 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fb9180a3e40 0x7fb9180a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.939+0000 7fb93c5e8740 1 -- 172.21.15.151:0/435036220 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f763c88140 msgr2=0x55f763c924c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.939+0000 7fb93c5e8740 1 --2- 172.21.15.151:0/435036220 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f763c88140 0x55f763c924c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb924009230 tx=0x7fb924005c00 comp rx=0 tx=0).stop 2024-04-26T02:01:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.939+0000 7fb93c5e8740 1 -- 172.21.15.151:0/435036220 shutdown_connections 2024-04-26T02:01:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.939+0000 7fb93c5e8740 1 --2- 172.21.15.151:0/435036220 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f763c92a00 0x55f763c94df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.939+0000 7fb93c5e8740 1 --2- 172.21.15.151:0/435036220 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f763c88140 0x55f763c924c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.939+0000 7fb93c5e8740 1 --2- 172.21.15.151:0/435036220 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f763c87820 0x55f763c87c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.939+0000 7fb93c5e8740 1 -- 172.21.15.151:0/435036220 >> 172.21.15.151:0/435036220 conn(0x55f763c83d30 msgr2=0x55f763c84110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.939+0000 7fb93c5e8740 1 -- 172.21.15.151:0/435036220 shutdown_connections 2024-04-26T02:01:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.939+0000 7fb93c5e8740 1 -- 172.21.15.151:0/435036220 wait complete. 2024-04-26T02:01:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.939+0000 7fb933400640 1 --2- 172.21.15.151:0/1289737156 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fb9180a3e40 0x7fb9180a6300 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb92c005fd0 tx=0x7fb92c009500 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:10.942 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.940+0000 7fb93c5e8740 1 -- start start 2024-04-26T02:01:10.942 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.940+0000 7fb93c5e8740 1 Processor -- start 2024-04-26T02:01:10.942 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.941+0000 7fb93c5e8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f763c87820 0x55f763c8f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.941+0000 7fb93c5e8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f763c88140 0x55f763c8fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.941+0000 7fb93c5e8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f763c92a00 0x55f763c90320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.941+0000 7fb93c5e8740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f763c955d0 con 0x55f763c92a00 2024-04-26T02:01:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.941+0000 7fb93c5e8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f763ba7310 con 0x55f763c88140 2024-04-26T02:01:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.941+0000 7fb93c5e8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f763c95750 con 0x55f763c87820 2024-04-26T02:01:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.941+0000 7fb932a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f763c87820 0x55f763c8f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.941+0000 7fb932000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f763c88140 0x55f763c8fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.941+0000 7fb93c5e8740 10 client.0 _ll_register_callbacks cb 0x55f763b75cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:01:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.941+0000 7fb932a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f763c87820 0x55f763c8f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:50130/0 (socket says 172.21.15.151:50130) 2024-04-26T02:01:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.941+0000 7fb932a00640 1 -- 172.21.15.151:0/3232673651 learned_addr learned my addr 172.21.15.151:0/3232673651 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:01:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.941+0000 7fb933400640 1 --2- 172.21.15.151:0/3232673651 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f763c92a00 0x55f763c90320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.941+0000 7fb932a00640 1 -- 172.21.15.151:0/3232673651 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f763c88140 msgr2=0x55f763c8fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.942+0000 7fb932a00640 1 --2- 172.21.15.151:0/3232673651 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f763c88140 0x55f763c8fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.942+0000 7fb932a00640 1 -- 172.21.15.151:0/3232673651 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f763c92a00 msgr2=0x55f763c90320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.942+0000 7fb932a00640 1 --2- 172.21.15.151:0/3232673651 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f763c92a00 0x55f763c90320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.942+0000 7fb932a00640 1 -- 172.21.15.151:0/3232673651 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7639cc620 con 0x55f763c87820 2024-04-26T02:01:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.942+0000 7fb932000640 1 --2- 172.21.15.151:0/3232673651 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f763c88140 0x55f763c8fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:01:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.942+0000 7fb932a00640 1 --2- 172.21.15.151:0/3232673651 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f763c87820 0x55f763c8f8a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb92402a9a0 tx=0x7fb924009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f58ab763f5f4fce8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.942+0000 7fb91cc00640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-26T02:01:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.942+0000 7fb91cc00640 1 -- 172.21.15.151:0/3232673651 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb924053060 con 0x55f763c87820 2024-04-26T02:01:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.942+0000 7fb93c5e8740 1 -- 172.21.15.151:0/3232673651 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb924004770 con 0x55f763c87820 2024-04-26T02:01:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.942+0000 7fb91cc00640 1 -- 172.21.15.151:0/3232673651 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8455+0+0 (secure 0 0 0) 0x7fb924005100 con 0x55f763c87820 2024-04-26T02:01:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.943+0000 7fb91cc00640 1 -- 172.21.15.151:0/3232673651 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb924035050 con 0x55f763c87820 2024-04-26T02:01:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.943+0000 7fb93c5e8740 10 client.26080 fetch_fsmap learned FSMap version 35 2024-04-26T02:01:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.943+0000 7fb93c5e8740 1 -- 172.21.15.151:0/3232673651 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f763c955d0 con 0x55f763c87820 2024-04-26T02:01:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.943+0000 7fb91cc00640 1 -- 172.21.15.151:0/3232673651 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb924004770 con 0x55f763c87820 2024-04-26T02:01:11.006 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.944+0000 7fb91cc00640 1 -- 172.21.15.151:0/3232673651 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 35) ==== 8338+0+0 (secure 0 0 0) 0x7fb9240314a0 con 0x55f763c87820 2024-04-26T02:01:11.006 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.944+0000 7fb93c5e8740 10 client.26080 fetch_fsmap finished waiting for FSMap version 35 2024-04-26T02:01:11.006 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.944+0000 7fb93c5e8740 10 client.26080 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:01:11.006 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.944+0000 7fb93c5e8740 20 client.26080 populate_metadata read hostname 'smithi151' 2024-04-26T02:01:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.944+0000 7fb93c5e8740 1 --2- 172.21.15.151:0/3232673651 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55f763ca28f0 0x55f763ca4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.944+0000 7fb93c5e8740 4 client.26080 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:01:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.944+0000 7fb93c5e8740 1 -- 172.21.15.151:0/3232673651 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f763c018b0 con 0x55f763ca28f0 2024-04-26T02:01:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.945+0000 7fb932000640 1 --2- 172.21.15.151:0/3232673651 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55f763ca28f0 0x55f763ca4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.945+0000 7fb932000640 1 --2- 172.21.15.151:0/3232673651 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55f763ca28f0 0x55f763ca4ce0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22f08736aeb8a064 server_cookie=4bd0304d72e6f344 in_seq=0 out_seq=0 2024-04-26T02:01:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.945+0000 7fb91cc00640 10 client.26080 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:01:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.947+0000 7fb91cc00640 1 -- 172.21.15.151:0/3232673651 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f763c018b0 con 0x55f763ca28f0 2024-04-26T02:01:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.947+0000 7fb91cc00640 10 client.26080 handle_command_reply: tid=0 2024-04-26T02:01:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.947+0000 7fb93c5e8740 1 client.26080 shutdown 2024-04-26T02:01:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.947+0000 7fb93c5e8740 1 -- 172.21.15.151:0/3232673651 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f763c87820 msgr2=0x55f763c8f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.947+0000 7fb93c5e8740 1 --2- 172.21.15.151:0/3232673651 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f763c87820 0x55f763c8f8a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb92402a9a0 tx=0x7fb924009260 comp rx=0 tx=0).stop 2024-04-26T02:01:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.947+0000 7fb93c5e8740 1 -- 172.21.15.151:0/3232673651 shutdown_connections 2024-04-26T02:01:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.947+0000 7fb93c5e8740 1 --2- 172.21.15.151:0/3232673651 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55f763ca28f0 0x55f763ca4ce0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.947+0000 7fb93c5e8740 1 --2- 172.21.15.151:0/3232673651 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f763c92a00 0x55f763c90320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.947+0000 7fb91cc00640 0 client.26080 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:01:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.947+0000 7fb93c5e8740 1 --2- 172.21.15.151:0/3232673651 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f763c88140 0x55f763c8fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.947+0000 7fb93c5e8740 1 --2- 172.21.15.151:0/3232673651 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f763c87820 0x55f763c8f8a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.947+0000 7fb93c5e8740 1 -- 172.21.15.151:0/3232673651 >> 172.21.15.151:0/3232673651 conn(0x55f763c83d30 msgr2=0x55f763c89a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.948+0000 7fb93c5e8740 1 -- 172.21.15.151:0/3232673651 shutdown_connections 2024-04-26T02:01:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.948+0000 7fb93c5e8740 1 -- 172.21.15.151:0/3232673651 wait complete. 2024-04-26T02:01:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.948+0000 7fb93c5e8740 20 client.26080 trim_cache size 0 max 16384 2024-04-26T02:01:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.969+0000 7fb93c5e8740 1 Processor -- start 2024-04-26T02:01:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.969+0000 7fb93c5e8740 1 -- start start 2024-04-26T02:01:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.970+0000 7fb93c5e8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f763c87820 0x55f763c9e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.970+0000 7fb93c5e8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f763c88140 0x55f763c8f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.970+0000 7fb93c5e8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f763c92a00 0x55f763c84110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.970+0000 7fb93c5e8740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb9341b2430 con 0x55f763c87820 2024-04-26T02:01:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.970+0000 7fb93c5e8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb9341afd00 con 0x55f763c92a00 2024-04-26T02:01:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.970+0000 7fb93c5e8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb9341adb70 con 0x55f763c88140 2024-04-26T02:01:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.970+0000 7fb932000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f763c88140 0x55f763c8f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.970+0000 7fb933400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f763c92a00 0x55f763c84110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.970+0000 7fb932000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f763c88140 0x55f763c8f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:50136/0 (socket says 172.21.15.151:50136) 2024-04-26T02:01:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.970+0000 7fb932000640 1 -- 172.21.15.151:0/3235983649 learned_addr learned my addr 172.21.15.151:0/3235983649 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:01:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.970+0000 7fb932a00640 1 --2- 172.21.15.151:0/3235983649 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f763c87820 0x55f763c9e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.970+0000 7fb932000640 1 -- 172.21.15.151:0/3235983649 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f763c92a00 msgr2=0x55f763c84110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.970+0000 7fb932000640 1 --2- 172.21.15.151:0/3235983649 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f763c92a00 0x55f763c84110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.970+0000 7fb932000640 1 -- 172.21.15.151:0/3235983649 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f763c87820 msgr2=0x55f763c9e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.970+0000 7fb932000640 1 --2- 172.21.15.151:0/3235983649 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f763c87820 0x55f763c9e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.970+0000 7fb932000640 1 -- 172.21.15.151:0/3235983649 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f763c8ddf0 con 0x55f763c88140 2024-04-26T02:01:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.971+0000 7fb932000640 1 --2- 172.21.15.151:0/3235983649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f763c88140 0x55f763c8f180 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb920004a40 tx=0x7fb9200bd040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a40262e9a150c96b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.971+0000 7fb91f400640 1 -- 172.21.15.151:0/3235983649 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92009de60 con 0x55f763c88140 2024-04-26T02:01:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.971+0000 7fb91f400640 1 -- 172.21.15.151:0/3235983649 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb920005680 con 0x55f763c88140 2024-04-26T02:01:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.971+0000 7fb91f400640 1 -- 172.21.15.151:0/3235983649 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9200046c0 con 0x55f763c88140 2024-04-26T02:01:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.972+0000 7fb93c5e8740 1 -- 172.21.15.151:0/3235983649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f763c88140 msgr2=0x55f763c8f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.972+0000 7fb93c5e8740 1 --2- 172.21.15.151:0/3235983649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f763c88140 0x55f763c8f180 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb920004a40 tx=0x7fb9200bd040 comp rx=0 tx=0).stop 2024-04-26T02:01:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.972+0000 7fb93c5e8740 1 -- 172.21.15.151:0/3235983649 shutdown_connections 2024-04-26T02:01:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.972+0000 7fb93c5e8740 1 --2- 172.21.15.151:0/3235983649 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f763c92a00 0x55f763c84110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.972+0000 7fb93c5e8740 1 --2- 172.21.15.151:0/3235983649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f763c88140 0x55f763c8f180 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.972+0000 7fb93c5e8740 1 --2- 172.21.15.151:0/3235983649 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f763c87820 0x55f763c9e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.972+0000 7fb93c5e8740 1 -- 172.21.15.151:0/3235983649 >> 172.21.15.151:0/3235983649 conn(0x55f763c83d30 msgr2=0x55f763c89a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.972+0000 7fb93c5e8740 1 -- 172.21.15.151:0/3235983649 shutdown_connections 2024-04-26T02:01:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.972+0000 7fb93c5e8740 1 -- 172.21.15.151:0/3235983649 wait complete. 2024-04-26T02:01:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.972+0000 7fb93c5e8740 1 -- start start 2024-04-26T02:01:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.973+0000 7fb93c5e8740 1 Processor -- start 2024-04-26T02:01:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.973+0000 7fb93c5e8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f763c87820 0x55f763c9a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.973+0000 7fb93c5e8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f763c88140 0x55f763c9ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.973+0000 7fb93c5e8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f763c92a00 0x55f763c95d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.973+0000 7fb93c5e8740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb9341ae1c0 con 0x55f763c87820 2024-04-26T02:01:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.973+0000 7fb93c5e8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb9341af630 con 0x55f763c88140 2024-04-26T02:01:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.973+0000 7fb93c5e8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb9341ae890 con 0x55f763c92a00 2024-04-26T02:01:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.973+0000 7fb93c5e8740 10 client.0 _ll_register_callbacks cb 0x7fb924009520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:01:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.973+0000 7fb933400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f763c92a00 0x55f763c95d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.973+0000 7fb932000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f763c88140 0x55f763c9ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.973+0000 7fb932a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f763c87820 0x55f763c9a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.973+0000 7fb933400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f763c92a00 0x55f763c95d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:50142/0 (socket says 172.21.15.151:50142) 2024-04-26T02:01:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.973+0000 7fb933400640 1 -- 172.21.15.151:0/3502157482 learned_addr learned my addr 172.21.15.151:0/3502157482 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:01:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.974+0000 7fb932000640 1 -- 172.21.15.151:0/3502157482 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f763c92a00 msgr2=0x55f763c95d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.974+0000 7fb932000640 1 --2- 172.21.15.151:0/3502157482 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f763c92a00 0x55f763c95d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.974+0000 7fb932000640 1 -- 172.21.15.151:0/3502157482 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f763c87820 msgr2=0x55f763c9a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.974+0000 7fb932000640 1 --2- 172.21.15.151:0/3502157482 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f763c87820 0x55f763c9a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.974+0000 7fb932000640 1 -- 172.21.15.151:0/3502157482 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f763c96550 con 0x55f763c88140 2024-04-26T02:01:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.974+0000 7fb932000640 1 --2- 172.21.15.151:0/3502157482 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f763c88140 0x55f763c9ac80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb9200ce340 tx=0x7fb9200d42b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9995d69c922487d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.974+0000 7fb91cc00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T02:01:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.974+0000 7fb91cc00640 1 -- 172.21.15.151:0/3502157482 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92009de60 con 0x55f763c88140 2024-04-26T02:01:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.974+0000 7fb91cc00640 1 -- 172.21.15.151:0/3502157482 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8455+0+0 (secure 0 0 0) 0x7fb9200046c0 con 0x55f763c88140 2024-04-26T02:01:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.974+0000 7fb93c5e8740 1 -- 172.21.15.151:0/3502157482 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb920005680 con 0x55f763c88140 2024-04-26T02:01:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.975+0000 7fb91cc00640 1 -- 172.21.15.151:0/3502157482 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb9200a0ee0 con 0x55f763c88140 2024-04-26T02:01:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.975+0000 7fb93c5e8740 10 client.26054 fetch_fsmap learned FSMap version 35 2024-04-26T02:01:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.975+0000 7fb93c5e8740 1 -- 172.21.15.151:0/3502157482 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f763d10fd0 con 0x55f763c88140 2024-04-26T02:01:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.975+0000 7fb91cc00640 1 -- 172.21.15.151:0/3502157482 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb920005680 con 0x55f763c88140 2024-04-26T02:01:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.975+0000 7fb91cc00640 1 -- 172.21.15.151:0/3502157482 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 35) ==== 8338+0+0 (secure 0 0 0) 0x7fb9200b1180 con 0x55f763c88140 2024-04-26T02:01:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.975+0000 7fb93c5e8740 10 client.26054 fetch_fsmap finished waiting for FSMap version 35 2024-04-26T02:01:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.975+0000 7fb93c5e8740 10 client.26054 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:01:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.975+0000 7fb93c5e8740 20 client.26054 populate_metadata read hostname 'smithi151' 2024-04-26T02:01:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.975+0000 7fb93c5e8740 1 --2- 172.21.15.151:0/3502157482 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55f763ca28f0 0x55f763d13660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.975+0000 7fb93c5e8740 4 client.26054 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T02:01:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.975+0000 7fb93c5e8740 1 -- 172.21.15.151:0/3502157482 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f763bfb320 con 0x55f763ca28f0 2024-04-26T02:01:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.976+0000 7fb932a00640 1 --2- 172.21.15.151:0/3502157482 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55f763ca28f0 0x55f763d13660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.976+0000 7fb932a00640 1 --2- 172.21.15.151:0/3502157482 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55f763ca28f0 0x55f763d13660 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f52e69351f4455f3 server_cookie=f60bcb9d289e428c in_seq=0 out_seq=0 2024-04-26T02:01:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.976+0000 7fb91cc00640 10 client.26054 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:01:11.014 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.991+0000 7fb91cc00640 1 -- 172.21.15.151:0/3502157482 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f763bfb320 con 0x55f763ca28f0 2024-04-26T02:01:11.014 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.991+0000 7fb91cc00640 10 client.26054 handle_command_reply: tid=0 2024-04-26T02:01:11.014 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.991+0000 7fb93c5e8740 1 client.26054 shutdown 2024-04-26T02:01:11.014 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.991+0000 7fb93c5e8740 1 -- 172.21.15.151:0/3502157482 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f763c88140 msgr2=0x55f763c9ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:11.014 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.991+0000 7fb93c5e8740 1 --2- 172.21.15.151:0/3502157482 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f763c88140 0x55f763c9ac80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb9200ce340 tx=0x7fb9200d42b0 comp rx=0 tx=0).stop 2024-04-26T02:01:11.014 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.992+0000 7fb93c5e8740 1 -- 172.21.15.151:0/3502157482 shutdown_connections 2024-04-26T02:01:11.014 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.992+0000 7fb93c5e8740 1 --2- 172.21.15.151:0/3502157482 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55f763ca28f0 0x55f763d13660 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:11.014 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.992+0000 7fb93c5e8740 1 --2- 172.21.15.151:0/3502157482 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f763c92a00 0x55f763c95d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:11.014 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.992+0000 7fb93c5e8740 1 --2- 172.21.15.151:0/3502157482 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f763c88140 0x55f763c9ac80 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:11.014 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.992+0000 7fb93c5e8740 1 --2- 172.21.15.151:0/3502157482 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f763c87820 0x55f763c9a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:11.014 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.992+0000 7fb91cc00640 0 client.26054 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:01:11.014 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.992+0000 7fb93c5e8740 1 -- 172.21.15.151:0/3502157482 >> 172.21.15.151:0/3502157482 conn(0x55f763c83d30 msgr2=0x55f763c82600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:11.014 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.992+0000 7fb93c5e8740 1 -- 172.21.15.151:0/3502157482 shutdown_connections 2024-04-26T02:01:11.015 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.992+0000 7fb93c5e8740 1 -- 172.21.15.151:0/3502157482 wait complete. 2024-04-26T02:01:11.015 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.992+0000 7fb93c5e8740 20 client.26054 trim_cache size 0 max 16384 2024-04-26T02:01:11.015 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:{ 2024-04-26T02:01:11.015 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "message": "", 2024-04-26T02:01:11.015 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "return_code": 0 2024-04-26T02:01:11.015 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:} 2024-04-26T02:01:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.993+0000 7fb939800640 1 -- 172.21.15.151:0/1289737156 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fb9180a3e40 msgr2=0x7fb9180a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.993+0000 7fb939800640 1 --2- 172.21.15.151:0/1289737156 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fb9180a3e40 0x7fb9180a6300 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb92c005fd0 tx=0x7fb92c009500 comp rx=0 tx=0).stop 2024-04-26T02:01:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.993+0000 7fb939800640 1 -- 172.21.15.151:0/1289737156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb93406b120 msgr2=0x7fb9341a0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.993+0000 7fb939800640 1 --2- 172.21.15.151:0/1289737156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb93406b120 0x7fb9341a0ad0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb92000e9f0 tx=0x7fb92000edf0 comp rx=0 tx=0).stop 2024-04-26T02:01:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.993+0000 7fb939800640 1 -- 172.21.15.151:0/1289737156 shutdown_connections 2024-04-26T02:01:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.993+0000 7fb939800640 1 --2- 172.21.15.151:0/1289737156 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fb9180a3e40 0x7fb9180a6300 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.993+0000 7fb939800640 1 --2- 172.21.15.151:0/1289737156 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb9341080e0 0x7fb93419a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.993+0000 7fb939800640 1 --2- 172.21.15.151:0/1289737156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb93406b120 0x7fb9341a0ad0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.993+0000 7fb939800640 1 --2- 172.21.15.151:0/1289737156 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb93406a780 0x7fb9341a0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.993+0000 7fb939800640 1 -- 172.21.15.151:0/1289737156 >> 172.21.15.151:0/1289737156 conn(0x7fb934076c50 msgr2=0x7fb93410d140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.994+0000 7fb939800640 1 -- 172.21.15.151:0/1289737156 shutdown_connections 2024-04-26T02:01:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:10.994+0000 7fb939800640 1 -- 172.21.15.151:0/1289737156 wait complete. 2024-04-26T02:01:12.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:01:12 smithi172 ceph-mon[32785]: pgmap v237: 97 pgs: 97 active+clean; 80 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.5 MiB/s wr, 333 op/s 2024-04-26T02:01:12.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:01:12 smithi165 ceph-mon[32648]: pgmap v237: 97 pgs: 97 active+clean; 80 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.5 MiB/s wr, 333 op/s 2024-04-26T02:01:12.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:01:12 smithi151 ceph-mon[28057]: pgmap v237: 97 pgs: 97 active+clean; 80 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.5 MiB/s wr, 333 op/s 2024-04-26T02:01:14.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:01:14 smithi172 ceph-mon[32785]: pgmap v238: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 855 KiB/s wr, 250 op/s 2024-04-26T02:01:14.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:01:14 smithi165 ceph-mon[32648]: pgmap v238: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 855 KiB/s wr, 250 op/s 2024-04-26T02:01:14.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:01:14 smithi151 ceph-mon[28057]: pgmap v238: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 855 KiB/s wr, 250 op/s 2024-04-26T02:01:16.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:01:16 smithi172 ceph-mon[32785]: pgmap v239: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.1 MiB/s wr, 337 op/s 2024-04-26T02:01:16.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:01:16 smithi165 ceph-mon[32648]: pgmap v239: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.1 MiB/s wr, 337 op/s 2024-04-26T02:01:16.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:01:16 smithi151 ceph-mon[28057]: pgmap v239: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.1 MiB/s wr, 337 op/s 2024-04-26T02:01:18.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:01:18 smithi172 ceph-mon[32785]: pgmap v240: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.1 MiB/s wr, 337 op/s 2024-04-26T02:01:18.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:01:18 smithi165 ceph-mon[32648]: pgmap v240: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.1 MiB/s wr, 337 op/s 2024-04-26T02:01:18.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:01:18 smithi151 ceph-mon[28057]: pgmap v240: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.1 MiB/s wr, 337 op/s 2024-04-26T02:01:20.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:01:20 smithi172 ceph-mon[32785]: pgmap v241: 97 pgs: 97 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.2 MiB/s wr, 443 op/s 2024-04-26T02:01:20.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:01:20 smithi165 ceph-mon[32648]: pgmap v241: 97 pgs: 97 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.2 MiB/s wr, 443 op/s 2024-04-26T02:01:20.624 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:01:20 smithi151 ceph-mon[28057]: pgmap v241: 97 pgs: 97 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.2 MiB/s wr, 443 op/s 2024-04-26T02:01:22.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:01:22 smithi172 ceph-mon[32785]: pgmap v242: 97 pgs: 97 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 1.0 MiB/s wr, 379 op/s 2024-04-26T02:01:22.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:01:22 smithi165 ceph-mon[32648]: pgmap v242: 97 pgs: 97 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 1.0 MiB/s wr, 379 op/s 2024-04-26T02:01:22.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:01:22 smithi151 ceph-mon[28057]: pgmap v242: 97 pgs: 97 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 1.0 MiB/s wr, 379 op/s 2024-04-26T02:01:22.792 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.9 2024-04-26T02:01:22.792 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T02:01:23.285 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:01:24.150 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.150+0000 7f9d8a800640 1 -- 172.21.15.151:0/1536360739 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9d841031e0 msgr2=0x7f9d841035c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:24.150 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.150+0000 7f9d8a800640 1 --2- 172.21.15.151:0/1536360739 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9d841031e0 0x7f9d841035c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9d74009ca0 tx=0x7f9d7402a9e0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.150 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.150+0000 7f9d8a800640 1 -- 172.21.15.151:0/1536360739 shutdown_connections 2024-04-26T02:01:24.150 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.150+0000 7f9d8a800640 1 --2- 172.21.15.151:0/1536360739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d8410c520 0x7f9d8410e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.150 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.150+0000 7f9d8a800640 1 --2- 172.21.15.151:0/1536360739 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9d84103b00 0x7f9d8410bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.150 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.150+0000 7f9d8a800640 1 --2- 172.21.15.151:0/1536360739 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9d841031e0 0x7f9d841035c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.150 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.150+0000 7f9d8a800640 1 -- 172.21.15.151:0/1536360739 >> 172.21.15.151:0/1536360739 conn(0x7f9d84077fd0 msgr2=0x7f9d840783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:24.150 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.151+0000 7f9d8a800640 1 -- 172.21.15.151:0/1536360739 shutdown_connections 2024-04-26T02:01:24.150 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.151+0000 7f9d8a800640 1 -- 172.21.15.151:0/1536360739 wait complete. 2024-04-26T02:01:24.152 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.152+0000 7f9d8a800640 1 Processor -- start 2024-04-26T02:01:24.152 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.152+0000 7f9d8a800640 1 -- start start 2024-04-26T02:01:24.152 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.152+0000 7f9d8a800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d841031e0 0x7f9d841a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:24.152 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.152+0000 7f9d8a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9d84103b00 0x7f9d841a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:24.153 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.152+0000 7f9d8a800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9d8410c520 0x7f9d8419a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:24.153 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.152+0000 7f9d8a800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9d841121b0 con 0x7f9d841031e0 2024-04-26T02:01:24.153 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.152+0000 7f9d8a800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9d84112030 con 0x7f9d84103b00 2024-04-26T02:01:24.153 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.152+0000 7f9d8a800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9d84112330 con 0x7f9d8410c520 2024-04-26T02:01:24.153 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.152+0000 7f9d7aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9d84103b00 0x7f9d841a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:24.153 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.152+0000 7f9d7aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9d84103b00 0x7f9d841a0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:41996/0 (socket says 172.21.15.151:41996) 2024-04-26T02:01:24.153 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.152+0000 7f9d7aa00640 1 -- 172.21.15.151:0/1478937000 learned_addr learned my addr 172.21.15.151:0/1478937000 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:01:24.153 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.152+0000 7f9d83e00640 1 --2- 172.21.15.151:0/1478937000 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9d8410c520 0x7f9d8419a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:24.153 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.152+0000 7f9d83400640 1 --2- 172.21.15.151:0/1478937000 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d841031e0 0x7f9d841a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:24.153 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.153+0000 7f9d7aa00640 1 -- 172.21.15.151:0/1478937000 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9d8410c520 msgr2=0x7f9d8419a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:24.153 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.153+0000 7f9d7aa00640 1 --2- 172.21.15.151:0/1478937000 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9d8410c520 0x7f9d8419a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.153 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.153+0000 7f9d7aa00640 1 -- 172.21.15.151:0/1478937000 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d841031e0 msgr2=0x7f9d841a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:24.153 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.153+0000 7f9d7aa00640 1 --2- 172.21.15.151:0/1478937000 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d841031e0 0x7f9d841a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.153 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.153+0000 7f9d7aa00640 1 -- 172.21.15.151:0/1478937000 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d8419b030 con 0x7f9d84103b00 2024-04-26T02:01:24.154 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.153+0000 7f9d7aa00640 1 --2- 172.21.15.151:0/1478937000 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9d84103b00 0x7f9d841a0ba0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c00e9f0 tx=0x7f9d6c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:24.154 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.153+0000 7f9d80c00640 1 -- 172.21.15.151:0/1478937000 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c00cd80 con 0x7f9d84103b00 2024-04-26T02:01:24.154 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.153+0000 7f9d80c00640 1 -- 172.21.15.151:0/1478937000 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d6c0054d0 con 0x7f9d84103b00 2024-04-26T02:01:24.154 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.153+0000 7f9d8a800640 1 -- 172.21.15.151:0/1478937000 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d8419b320 con 0x7f9d84103b00 2024-04-26T02:01:24.154 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.153+0000 7f9d80c00640 1 -- 172.21.15.151:0/1478937000 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c010690 con 0x7f9d84103b00 2024-04-26T02:01:24.154 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.154+0000 7f9d8a800640 1 -- 172.21.15.151:0/1478937000 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d841a75e0 con 0x7f9d84103b00 2024-04-26T02:01:24.156 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.157+0000 7f9d80c00640 1 -- 172.21.15.151:0/1478937000 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f9d6c004e90 con 0x7f9d84103b00 2024-04-26T02:01:24.156 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.157+0000 7f9d8d5cf740 1 Processor -- start 2024-04-26T02:01:24.156 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.157+0000 7f9d8d5cf740 1 -- start start 2024-04-26T02:01:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.157+0000 7f9d8d5cf740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56293e8c3820 0x56293e8c3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.158+0000 7f9d8d5cf740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56293e8c4140 0x56293e8ce4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.158+0000 7f9d8d5cf740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56293e8cea00 0x56293e8d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.158+0000 7f9d8d5cf740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56293e829900 con 0x56293e8c4140 2024-04-26T02:01:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.158+0000 7f9d8d5cf740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56293e7cfef0 con 0x56293e8cea00 2024-04-26T02:01:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.158+0000 7f9d8d5cf740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56293e8258c0 con 0x56293e8c3820 2024-04-26T02:01:24.158 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.158+0000 7f9d83e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56293e8c3820 0x56293e8c3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.158+0000 7f9d83400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56293e8c4140 0x56293e8ce4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.158+0000 7f9d7aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56293e8cea00 0x56293e8d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.158+0000 7f9d83e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56293e8c3820 0x56293e8c3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:35304/0 (socket says 172.21.15.151:35304) 2024-04-26T02:01:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.158+0000 7f9d83e00640 1 -- 172.21.15.151:0/2918287168 learned_addr learned my addr 172.21.15.151:0/2918287168 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:01:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.158+0000 7f9d83e00640 1 -- 172.21.15.151:0/2918287168 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56293e8cea00 msgr2=0x56293e8d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.158+0000 7f9d83e00640 1 --2- 172.21.15.151:0/2918287168 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56293e8cea00 0x56293e8d0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.158+0000 7f9d83e00640 1 -- 172.21.15.151:0/2918287168 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56293e8c4140 msgr2=0x56293e8ce4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.158+0000 7f9d83e00640 1 --2- 172.21.15.151:0/2918287168 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56293e8c4140 0x56293e8ce4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.158+0000 7f9d80c00640 1 --2- 172.21.15.151:0/1478937000 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f9d680a3e40 0x7f9d680a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.158+0000 7f9d83e00640 1 -- 172.21.15.151:0/2918287168 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56293e53c210 con 0x56293e8c3820 2024-04-26T02:01:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.158+0000 7f9d7aa00640 1 --2- 172.21.15.151:0/2918287168 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56293e8cea00 0x56293e8d0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:01:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.159+0000 7f9d80c00640 1 -- 172.21.15.151:0/1478937000 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8455+0+0 (secure 0 0 0) 0x7f9d6c0d0b90 con 0x7f9d84103b00 2024-04-26T02:01:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.159+0000 7f9d83e00640 1 --2- 172.21.15.151:0/2918287168 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56293e8c3820 0x56293e8c3c00 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c004620 tx=0x7f9d7c00bb10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ec3e10dc0c9a9ab server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.159+0000 7f9d7aa00640 1 --2- 172.21.15.151:0/1478937000 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f9d680a3e40 0x7f9d680a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.159+0000 7f9d78c00640 1 -- 172.21.15.151:0/2918287168 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c003ba0 con 0x56293e8c3820 2024-04-26T02:01:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.159+0000 7f9d78c00640 1 -- 172.21.15.151:0/2918287168 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d7c015070 con 0x56293e8c3820 2024-04-26T02:01:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.159+0000 7f9d78c00640 1 -- 172.21.15.151:0/2918287168 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c0116a0 con 0x56293e8c3820 2024-04-26T02:01:24.162 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.160+0000 7f9d8d5cf740 1 -- 172.21.15.151:0/2918287168 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56293e8c3820 msgr2=0x56293e8c3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:24.162 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.160+0000 7f9d7aa00640 1 --2- 172.21.15.151:0/1478937000 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f9d680a3e40 0x7f9d680a6300 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x56293e8c0ec0 tx=0x7f9d6c0d4640 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:24.163 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.160+0000 7f9d8d5cf740 1 --2- 172.21.15.151:0/2918287168 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56293e8c3820 0x56293e8c3c00 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c004620 tx=0x7f9d7c00bb10 comp rx=0 tx=0).stop 2024-04-26T02:01:24.163 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.160+0000 7f9d8d5cf740 1 -- 172.21.15.151:0/2918287168 shutdown_connections 2024-04-26T02:01:24.163 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.160+0000 7f9d8d5cf740 1 --2- 172.21.15.151:0/2918287168 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56293e8cea00 0x56293e8d0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.163 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.160+0000 7f9d8d5cf740 1 --2- 172.21.15.151:0/2918287168 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56293e8c4140 0x56293e8ce4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.163 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.160+0000 7f9d8d5cf740 1 --2- 172.21.15.151:0/2918287168 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56293e8c3820 0x56293e8c3c00 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.163 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.160+0000 7f9d8d5cf740 1 -- 172.21.15.151:0/2918287168 >> 172.21.15.151:0/2918287168 conn(0x56293e8bfd30 msgr2=0x56293e8c0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:24.163 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.160+0000 7f9d8d5cf740 1 -- 172.21.15.151:0/2918287168 shutdown_connections 2024-04-26T02:01:24.163 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.160+0000 7f9d8d5cf740 1 -- 172.21.15.151:0/2918287168 wait complete. 2024-04-26T02:01:24.163 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.161+0000 7f9d8d5cf740 1 -- start start 2024-04-26T02:01:24.163 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.161+0000 7f9d8d5cf740 1 Processor -- start 2024-04-26T02:01:24.163 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.161+0000 7f9d8d5cf740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56293e8c3820 0x56293e8cba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:24.163 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.161+0000 7f9d8d5cf740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56293e8c4140 0x56293e8cbf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:24.164 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.162+0000 7f9d8d5cf740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56293e8cea00 0x56293e8cc490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:24.164 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.162+0000 7f9d8d5cf740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56293e8d1630 con 0x56293e8c3820 2024-04-26T02:01:24.164 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.162+0000 7f9d8d5cf740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56293e7e3310 con 0x56293e8c4140 2024-04-26T02:01:24.164 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.162+0000 7f9d8d5cf740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56293e8d17b0 con 0x56293e8cea00 2024-04-26T02:01:24.164 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.162+0000 7f9d8d5cf740 10 client.0 _ll_register_callbacks cb 0x56293e7b1cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:01:24.164 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.162+0000 7f9d83e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56293e8c3820 0x56293e8cba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:24.164 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.162+0000 7f9d83400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56293e8c4140 0x56293e8cbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:24.164 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.162+0000 7f9d83e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56293e8c3820 0x56293e8cba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43140/0 (socket says 172.21.15.151:43140) 2024-04-26T02:01:24.164 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.162+0000 7f9d83e00640 1 -- 172.21.15.151:0/2793101267 learned_addr learned my addr 172.21.15.151:0/2793101267 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:01:24.164 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.162+0000 7f9d83400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56293e8c4140 0x56293e8cbf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:42018/0 (socket says 172.21.15.151:42018) 2024-04-26T02:01:24.164 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.162+0000 7f9d83e00640 1 --2- 172.21.15.151:0/2793101267 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56293e8cea00 0x56293e8cc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:24.164 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.162+0000 7f9d83e00640 1 -- 172.21.15.151:0/2793101267 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56293e8cea00 msgr2=0x56293e8cc490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:24.164 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.162+0000 7f9d83e00640 1 --2- 172.21.15.151:0/2793101267 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56293e8cea00 0x56293e8cc490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.164 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.162+0000 7f9d83e00640 1 -- 172.21.15.151:0/2793101267 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56293e8c4140 msgr2=0x56293e8cbf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:24.165 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.162+0000 7f9d83e00640 1 --2- 172.21.15.151:0/2793101267 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56293e8c4140 0x56293e8cbf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.165 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.162+0000 7f9d83e00640 1 -- 172.21.15.151:0/2793101267 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56293e608620 con 0x56293e8c3820 2024-04-26T02:01:24.165 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.163+0000 7f9d83e00640 1 --2- 172.21.15.151:0/2793101267 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56293e8c3820 0x56293e8cba10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x56293e8bf750 tx=0x7f9d7c01acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=312d0660414b44a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:24.165 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.163+0000 7f9d71600640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-04-26T02:01:24.165 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.163+0000 7f9d71600640 1 -- 172.21.15.151:0/2793101267 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c025070 con 0x56293e8c3820 2024-04-26T02:01:24.165 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.163+0000 7f9d71600640 1 -- 172.21.15.151:0/2793101267 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8455+0+0 (secure 0 0 0) 0x7f9d7c01a880 con 0x56293e8c3820 2024-04-26T02:01:24.165 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.163+0000 7f9d8d5cf740 1 -- 172.21.15.151:0/2793101267 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d7c015070 con 0x56293e8c3820 2024-04-26T02:01:24.167 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.164+0000 7f9d71600640 1 -- 172.21.15.151:0/2793101267 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f9d7c032050 con 0x56293e8c3820 2024-04-26T02:01:24.168 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.164+0000 7f9d8d5cf740 10 client.15960 fetch_fsmap learned FSMap version 35 2024-04-26T02:01:24.168 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.164+0000 7f9d8d5cf740 1 -- 172.21.15.151:0/2793101267 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56293e8dd920 con 0x56293e8c3820 2024-04-26T02:01:24.168 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.165+0000 7f9d71600640 1 -- 172.21.15.151:0/2793101267 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d7c015070 con 0x56293e8c3820 2024-04-26T02:01:24.168 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.165+0000 7f9d71600640 1 -- 172.21.15.151:0/2793101267 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 35) ==== 8338+0+0 (secure 0 0 0) 0x7f9d7c010440 con 0x56293e8c3820 2024-04-26T02:01:24.168 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.165+0000 7f9d8d5cf740 10 client.15960 fetch_fsmap finished waiting for FSMap version 35 2024-04-26T02:01:24.168 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.165+0000 7f9d8d5cf740 10 client.15960 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:01:24.168 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.165+0000 7f9d8d5cf740 20 client.15960 populate_metadata read hostname 'smithi151' 2024-04-26T02:01:24.168 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.165+0000 7f9d8d5cf740 1 --2- 172.21.15.151:0/2793101267 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x56293e8decd0 0x56293e8e10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:24.168 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.165+0000 7f9d8d5cf740 4 client.15960 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:01:24.168 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.165+0000 7f9d8d5cf740 1 -- 172.21.15.151:0/2793101267 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56293e83d8b0 con 0x56293e8decd0 2024-04-26T02:01:24.168 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.165+0000 7f9d83400640 1 --2- 172.21.15.151:0/2793101267 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x56293e8decd0 0x56293e8e10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:24.169 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.166+0000 7f9d83400640 1 --2- 172.21.15.151:0/2793101267 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x56293e8decd0 0x56293e8e10c0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=928dae90dd4cd66 server_cookie=68400b016826a61d in_seq=0 out_seq=0 2024-04-26T02:01:24.169 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.166+0000 7f9d71600640 10 client.15960 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:01:24.169 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.167+0000 7f9d71600640 1 -- 172.21.15.151:0/2793101267 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56293e83d8b0 con 0x56293e8decd0 2024-04-26T02:01:24.169 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.167+0000 7f9d71600640 10 client.15960 handle_command_reply: tid=0 2024-04-26T02:01:24.169 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.167+0000 7f9d8d5cf740 1 client.15960 shutdown 2024-04-26T02:01:24.169 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.167+0000 7f9d8d5cf740 1 -- 172.21.15.151:0/2793101267 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56293e8c3820 msgr2=0x56293e8cba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:24.169 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.167+0000 7f9d8d5cf740 1 --2- 172.21.15.151:0/2793101267 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56293e8c3820 0x56293e8cba10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x56293e8bf750 tx=0x7f9d7c01acb0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.169 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.167+0000 7f9d8d5cf740 1 -- 172.21.15.151:0/2793101267 shutdown_connections 2024-04-26T02:01:24.169 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.167+0000 7f9d8d5cf740 1 --2- 172.21.15.151:0/2793101267 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x56293e8decd0 0x56293e8e10c0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.169 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.167+0000 7f9d8d5cf740 1 --2- 172.21.15.151:0/2793101267 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56293e8cea00 0x56293e8cc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.169 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.167+0000 7f9d8d5cf740 1 --2- 172.21.15.151:0/2793101267 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56293e8c4140 0x56293e8cbf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.169 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.167+0000 7f9d8d5cf740 1 --2- 172.21.15.151:0/2793101267 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56293e8c3820 0x56293e8cba10 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.169 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.167+0000 7f9d71600640 0 client.15960 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:01:24.169 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.167+0000 7f9d8d5cf740 1 -- 172.21.15.151:0/2793101267 >> 172.21.15.151:0/2793101267 conn(0x56293e8bfd30 msgr2=0x56293e8c5ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:24.170 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.167+0000 7f9d8d5cf740 1 -- 172.21.15.151:0/2793101267 shutdown_connections 2024-04-26T02:01:24.170 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.168+0000 7f9d8d5cf740 1 -- 172.21.15.151:0/2793101267 wait complete. 2024-04-26T02:01:24.170 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.168+0000 7f9d8d5cf740 20 client.15960 trim_cache size 0 max 16384 2024-04-26T02:01:24.186 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.187+0000 7f9d8d5cf740 1 Processor -- start 2024-04-26T02:01:24.186 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.187+0000 7f9d8d5cf740 1 -- start start 2024-04-26T02:01:24.186 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.187+0000 7f9d8d5cf740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56293e8c3820 0x56293e8da4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:24.186 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.187+0000 7f9d8d5cf740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56293e8c4140 0x56293e8c5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:24.186 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.187+0000 7f9d8d5cf740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56293e8cea00 0x56293e8cb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:24.187 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.187+0000 7f9d8d5cf740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9d841b25b0 con 0x56293e8c4140 2024-04-26T02:01:24.187 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.187+0000 7f9d8d5cf740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9d841afe80 con 0x56293e8c3820 2024-04-26T02:01:24.187 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.187+0000 7f9d8d5cf740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9d841adcf0 con 0x56293e8cea00 2024-04-26T02:01:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.188+0000 7f9d83e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56293e8c3820 0x56293e8da4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.188+0000 7f9d83e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56293e8c3820 0x56293e8da4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:42022/0 (socket says 172.21.15.151:42022) 2024-04-26T02:01:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.188+0000 7f9d83e00640 1 -- 172.21.15.151:0/3142266760 learned_addr learned my addr 172.21.15.151:0/3142266760 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:01:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.188+0000 7f9d83e00640 1 --2- 172.21.15.151:0/3142266760 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56293e8cea00 0x56293e8cb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.188+0000 7f9d83400640 1 --2- 172.21.15.151:0/3142266760 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56293e8c4140 0x56293e8c5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.188+0000 7f9d83e00640 1 -- 172.21.15.151:0/3142266760 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56293e8cea00 msgr2=0x56293e8cb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.188+0000 7f9d83e00640 1 --2- 172.21.15.151:0/3142266760 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56293e8cea00 0x56293e8cb1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.188+0000 7f9d83e00640 1 -- 172.21.15.151:0/3142266760 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56293e8c4140 msgr2=0x56293e8c5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.188+0000 7f9d83e00640 1 --2- 172.21.15.151:0/3142266760 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56293e8c4140 0x56293e8c5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.188+0000 7f9d83e00640 1 -- 172.21.15.151:0/3142266760 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56293e8c48e0 con 0x56293e8c3820 2024-04-26T02:01:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.188+0000 7f9d83e00640 1 --2- 172.21.15.151:0/3142266760 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56293e8c3820 0x56293e8da4f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c0056a0 tx=0x7f9d7c004370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4491573031a210ce server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.189+0000 7f9d73e00640 1 -- 172.21.15.151:0/3142266760 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c025040 con 0x56293e8c3820 2024-04-26T02:01:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.189+0000 7f9d73e00640 1 -- 172.21.15.151:0/3142266760 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d7c015070 con 0x56293e8c3820 2024-04-26T02:01:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.189+0000 7f9d73e00640 1 -- 172.21.15.151:0/3142266760 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c02c410 con 0x56293e8c3820 2024-04-26T02:01:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.189+0000 7f9d8d5cf740 1 -- 172.21.15.151:0/3142266760 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56293e8c3820 msgr2=0x56293e8da4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.189+0000 7f9d8d5cf740 1 --2- 172.21.15.151:0/3142266760 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56293e8c3820 0x56293e8da4f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c0056a0 tx=0x7f9d7c004370 comp rx=0 tx=0).stop 2024-04-26T02:01:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.190+0000 7f9d8d5cf740 1 -- 172.21.15.151:0/3142266760 shutdown_connections 2024-04-26T02:01:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.190+0000 7f9d8d5cf740 1 --2- 172.21.15.151:0/3142266760 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56293e8cea00 0x56293e8cb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.190+0000 7f9d8d5cf740 1 --2- 172.21.15.151:0/3142266760 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56293e8c4140 0x56293e8c5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.190+0000 7f9d8d5cf740 1 --2- 172.21.15.151:0/3142266760 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56293e8c3820 0x56293e8da4f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.190+0000 7f9d8d5cf740 1 -- 172.21.15.151:0/3142266760 >> 172.21.15.151:0/3142266760 conn(0x56293e8bfd30 msgr2=0x56293e8bee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.190+0000 7f9d8d5cf740 1 -- 172.21.15.151:0/3142266760 shutdown_connections 2024-04-26T02:01:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.190+0000 7f9d8d5cf740 1 -- 172.21.15.151:0/3142266760 wait complete. 2024-04-26T02:01:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.190+0000 7f9d8d5cf740 1 -- start start 2024-04-26T02:01:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.191+0000 7f9d8d5cf740 1 Processor -- start 2024-04-26T02:01:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.191+0000 7f9d8d5cf740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56293e8c3820 0x56293e8d3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.191+0000 7f9d8d5cf740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56293e8c4140 0x56293e8d4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:24.196 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.191+0000 7f9d8d5cf740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56293e8cea00 0x56293e942f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:24.196 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.191+0000 7f9d8d5cf740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9d841ae340 con 0x56293e8cea00 2024-04-26T02:01:24.196 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.191+0000 7f9d8d5cf740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9d841af7b0 con 0x56293e8c3820 2024-04-26T02:01:24.196 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.191+0000 7f9d8d5cf740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9d841aea10 con 0x56293e8c4140 2024-04-26T02:01:24.196 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.191+0000 7f9d8d5cf740 10 client.0 _ll_register_callbacks cb 0x7f9d7c003ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:01:24.196 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.191+0000 7f9d83400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56293e8c4140 0x56293e8d4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:24.197 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.191+0000 7f9d83e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56293e8c3820 0x56293e8d3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:24.197 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.191+0000 7f9d83400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56293e8c4140 0x56293e8d4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:35334/0 (socket says 172.21.15.151:35334) 2024-04-26T02:01:24.197 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.191+0000 7f9d83400640 1 -- 172.21.15.151:0/2875450655 learned_addr learned my addr 172.21.15.151:0/2875450655 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:01:24.197 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.191+0000 7f9d83e00640 1 --2- 172.21.15.151:0/2875450655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56293e8cea00 0x56293e942f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:24.197 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.192+0000 7f9d83400640 1 -- 172.21.15.151:0/2875450655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56293e8c3820 msgr2=0x56293e8d3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:24.197 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.192+0000 7f9d83400640 1 --2- 172.21.15.151:0/2875450655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56293e8c3820 0x56293e8d3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.198 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.192+0000 7f9d83400640 1 -- 172.21.15.151:0/2875450655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56293e8cea00 msgr2=0x56293e942f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:24.198 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.192+0000 7f9d83400640 1 --2- 172.21.15.151:0/2875450655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56293e8cea00 0x56293e942f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.198 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.192+0000 7f9d83400640 1 -- 172.21.15.151:0/2875450655 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56293e9434f0 con 0x56293e8c4140 2024-04-26T02:01:24.198 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.192+0000 7f9d83400640 1 --2- 172.21.15.151:0/2875450655 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56293e8c4140 0x56293e8d4160 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9d74006010 tx=0x7f9d74009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9052c00c1034070d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:24.198 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.192+0000 7f9d71600640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-26T02:01:24.198 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.192+0000 7f9d71600640 1 -- 172.21.15.151:0/2875450655 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7404c070 con 0x56293e8c4140 2024-04-26T02:01:24.199 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.192+0000 7f9d71600640 1 -- 172.21.15.151:0/2875450655 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8455+0+0 (secure 0 0 0) 0x7f9d74003b70 con 0x56293e8c4140 2024-04-26T02:01:24.199 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.192+0000 7f9d8d5cf740 1 -- 172.21.15.151:0/2875450655 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d7c015070 con 0x56293e8c4140 2024-04-26T02:01:24.199 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.193+0000 7f9d71600640 1 -- 172.21.15.151:0/2875450655 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f9d74058050 con 0x56293e8c4140 2024-04-26T02:01:24.199 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.193+0000 7f9d8d5cf740 10 client.26104 fetch_fsmap learned FSMap version 35 2024-04-26T02:01:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.193+0000 7f9d8d5cf740 1 -- 172.21.15.151:0/2875450655 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56293e944890 con 0x56293e8c4140 2024-04-26T02:01:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.193+0000 7f9d71600640 1 -- 172.21.15.151:0/2875450655 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d7c015070 con 0x56293e8c4140 2024-04-26T02:01:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.193+0000 7f9d71600640 1 -- 172.21.15.151:0/2875450655 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 35) ==== 8338+0+0 (secure 0 0 0) 0x7f9d7402fb10 con 0x56293e8c4140 2024-04-26T02:01:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.193+0000 7f9d8d5cf740 10 client.26104 fetch_fsmap finished waiting for FSMap version 35 2024-04-26T02:01:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.193+0000 7f9d8d5cf740 10 client.26104 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:01:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.193+0000 7f9d8d5cf740 20 client.26104 populate_metadata read hostname 'smithi151' 2024-04-26T02:01:24.201 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.193+0000 7f9d8d5cf740 1 --2- 172.21.15.151:0/2875450655 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x56293e944ff0 0x56293e947410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:24.201 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.193+0000 7f9d8d5cf740 4 client.26104 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T02:01:24.201 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.193+0000 7f9d8d5cf740 1 -- 172.21.15.151:0/2875450655 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56293e837320 con 0x56293e944ff0 2024-04-26T02:01:24.201 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.194+0000 7f9d83e00640 1 --2- 172.21.15.151:0/2875450655 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x56293e944ff0 0x56293e947410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:24.201 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.194+0000 7f9d83e00640 1 --2- 172.21.15.151:0/2875450655 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x56293e944ff0 0x56293e947410 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67fc2a2298758861 server_cookie=d451a6f7a80e5451 in_seq=0 out_seq=0 2024-04-26T02:01:24.201 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.194+0000 7f9d71600640 10 client.26104 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:01:24.206 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.206+0000 7f9d71600640 1 -- 172.21.15.151:0/2875450655 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56293e837320 con 0x56293e944ff0 2024-04-26T02:01:24.206 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.206+0000 7f9d71600640 10 client.26104 handle_command_reply: tid=0 2024-04-26T02:01:24.206 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.206+0000 7f9d8d5cf740 1 client.26104 shutdown 2024-04-26T02:01:24.206 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.206+0000 7f9d8d5cf740 1 -- 172.21.15.151:0/2875450655 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56293e8c4140 msgr2=0x56293e8d4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:24.206 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.206+0000 7f9d8d5cf740 1 --2- 172.21.15.151:0/2875450655 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56293e8c4140 0x56293e8d4160 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9d74006010 tx=0x7f9d74009c20 comp rx=0 tx=0).stop 2024-04-26T02:01:24.206 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.206+0000 7f9d8d5cf740 1 -- 172.21.15.151:0/2875450655 shutdown_connections 2024-04-26T02:01:24.206 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.206+0000 7f9d8d5cf740 1 --2- 172.21.15.151:0/2875450655 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x56293e944ff0 0x56293e947410 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.206 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.206+0000 7f9d8d5cf740 1 --2- 172.21.15.151:0/2875450655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56293e8cea00 0x56293e942f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.206 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.206+0000 7f9d8d5cf740 1 --2- 172.21.15.151:0/2875450655 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56293e8c4140 0x56293e8d4160 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.206 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.206+0000 7f9d71600640 0 client.26104 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:01:24.206 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.206+0000 7f9d8d5cf740 1 --2- 172.21.15.151:0/2875450655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56293e8c3820 0x56293e8d3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.207 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.206+0000 7f9d8d5cf740 1 -- 172.21.15.151:0/2875450655 >> 172.21.15.151:0/2875450655 conn(0x56293e8bfd30 msgr2=0x56293e8be600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:24.207 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.207+0000 7f9d8d5cf740 1 -- 172.21.15.151:0/2875450655 shutdown_connections 2024-04-26T02:01:24.207 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.207+0000 7f9d8d5cf740 1 -- 172.21.15.151:0/2875450655 wait complete. 2024-04-26T02:01:24.207 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.207+0000 7f9d8d5cf740 20 client.26104 trim_cache size 0 max 16384 2024-04-26T02:01:24.207 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:{ 2024-04-26T02:01:24.207 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "message": "", 2024-04-26T02:01:24.207 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "return_code": 0 2024-04-26T02:01:24.207 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:} 2024-04-26T02:01:24.208 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.207+0000 7f9d8a800640 1 -- 172.21.15.151:0/1478937000 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f9d680a3e40 msgr2=0x7f9d680a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:24.208 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.207+0000 7f9d8a800640 1 --2- 172.21.15.151:0/1478937000 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f9d680a3e40 0x7f9d680a6300 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x56293e8c0ec0 tx=0x7f9d6c0d4640 comp rx=0 tx=0).stop 2024-04-26T02:01:24.208 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.207+0000 7f9d8a800640 1 -- 172.21.15.151:0/1478937000 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9d84103b00 msgr2=0x7f9d841a0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:24.208 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.207+0000 7f9d8a800640 1 --2- 172.21.15.151:0/1478937000 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9d84103b00 0x7f9d841a0ba0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c00e9f0 tx=0x7f9d6c00edf0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.208 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.208+0000 7f9d8a800640 1 -- 172.21.15.151:0/1478937000 shutdown_connections 2024-04-26T02:01:24.208 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.208+0000 7f9d8a800640 1 --2- 172.21.15.151:0/1478937000 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f9d680a3e40 0x7f9d680a6300 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.208 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.208+0000 7f9d8a800640 1 --2- 172.21.15.151:0/1478937000 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9d8410c520 0x7f9d8419a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.208 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.208+0000 7f9d8a800640 1 --2- 172.21.15.151:0/1478937000 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9d84103b00 0x7f9d841a0ba0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.208 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.208+0000 7f9d8a800640 1 --2- 172.21.15.151:0/1478937000 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d841031e0 0x7f9d841a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.208+0000 7f9d8a800640 1 -- 172.21.15.151:0/1478937000 >> 172.21.15.151:0/1478937000 conn(0x7f9d84077fd0 msgr2=0x7f9d841084d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.208+0000 7f9d8a800640 1 -- 172.21.15.151:0/1478937000 shutdown_connections 2024-04-26T02:01:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:24.208+0000 7f9d8a800640 1 -- 172.21.15.151:0/1478937000 wait complete. 2024-04-26T02:01:24.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:01:24 smithi151 ceph-mon[28057]: pgmap v243: 97 pgs: 97 active+clean; 73 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.2 MiB/s wr, 425 op/s 2024-04-26T02:01:24.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:01:24 smithi172 ceph-mon[32785]: pgmap v243: 97 pgs: 97 active+clean; 73 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.2 MiB/s wr, 425 op/s 2024-04-26T02:01:24.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:01:24 smithi165 ceph-mon[32648]: pgmap v243: 97 pgs: 97 active+clean; 73 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.2 MiB/s wr, 425 op/s 2024-04-26T02:01:25.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:25.775+0000 7f8661e00700 1 -- 172.21.15.151:0/2805415322 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f865c10f1b0 msgr2=0x7f865c111620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:25.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:25.775+0000 7f8661e00700 1 --2- 172.21.15.151:0/2805415322 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f865c10f1b0 0x7f865c111620 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f865400b0a0 tx=0x7f865400b3b0 comp rx=0 tx=0).stop 2024-04-26T02:01:25.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:25.775+0000 7f8661e00700 1 -- 172.21.15.151:0/2805415322 shutdown_connections 2024-04-26T02:01:25.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:25.775+0000 7f8661e00700 1 --2- 172.21.15.151:0/2805415322 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f865c10f1b0 0x7f865c111620 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:25.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:25.775+0000 7f8661e00700 1 --2- 172.21.15.151:0/2805415322 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f865c074e70 0x7f865c075250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:25.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:25.775+0000 7f8661e00700 1 --2- 172.21.15.151:0/2805415322 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f865c075790 0x7f865c10ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:25.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:25.775+0000 7f8661e00700 1 -- 172.21.15.151:0/2805415322 >> 172.21.15.151:0/2805415322 conn(0x7f865c0fe4c0 msgr2=0x7f865c1008e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:25.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:25.775+0000 7f8661e00700 1 -- 172.21.15.151:0/2805415322 shutdown_connections 2024-04-26T02:01:25.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:25.775+0000 7f8661e00700 1 -- 172.21.15.151:0/2805415322 wait complete. 2024-04-26T02:01:25.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:25.776+0000 7f8661e00700 1 Processor -- start 2024-04-26T02:01:25.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:25.776+0000 7f8661e00700 1 -- start start 2024-04-26T02:01:25.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:25.777+0000 7f8661e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f865c074e70 0x7f865c19b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:25.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:25.777+0000 7f8661e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f865c075790 0x7f865c19b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:25.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:25.777+0000 7f8661e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f865c10f1b0 0x7f865c19faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:25.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:25.777+0000 7f8661e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f865c114d40 con 0x7f865c10f1b0 2024-04-26T02:01:25.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:25.777+0000 7f8661e00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f865c114bc0 con 0x7f865c075790 2024-04-26T02:01:25.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:25.777+0000 7f8661e00700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f865c114ec0 con 0x7f865c074e70 2024-04-26T02:01:25.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:25.777+0000 7f8660a00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f865c074e70 0x7f865c19b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:25.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:25.777+0000 7f8661400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f865c10f1b0 0x7f865c19faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:25.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:25.777+0000 7f8660a00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f865c074e70 0x7f865c19b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:35358/0 (socket says 172.21.15.151:35358) 2024-04-26T02:01:25.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:25.777+0000 7f8660a00700 1 -- 172.21.15.151:0/3778459360 learned_addr learned my addr 172.21.15.151:0/3778459360 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:01:25.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:25.778+0000 7f8660a00700 1 -- 172.21.15.151:0/3778459360 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f865c075790 msgr2=0x7f865c19b710 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T02:01:25.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:25.778+0000 7f8660a00700 1 --2- 172.21.15.151:0/3778459360 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f865c075790 0x7f865c19b710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:25.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:25.778+0000 7f8660a00700 1 -- 172.21.15.151:0/3778459360 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f865c10f1b0 msgr2=0x7f865c19faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:25.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:25.778+0000 7f8660a00700 1 --2- 172.21.15.151:0/3778459360 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f865c10f1b0 0x7f865c19faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:25.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:25.778+0000 7f8660a00700 1 -- 172.21.15.151:0/3778459360 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f865c1a01e0 con 0x7f865c074e70 2024-04-26T02:01:25.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:25.778+0000 7f8660a00700 1 --2- 172.21.15.151:0/3778459360 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f865c074e70 0x7f865c19b1d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f865000e9a0 tx=0x7f865000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:25.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:25.778+0000 7f8659600700 1 -- 172.21.15.151:0/3778459360 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865000cbd0 con 0x7f865c074e70 2024-04-26T02:01:25.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:25.779+0000 7f8659600700 1 -- 172.21.15.151:0/3778459360 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f865000cd70 con 0x7f865c074e70 2024-04-26T02:01:25.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:25.779+0000 7f8661e00700 1 -- 172.21.15.151:0/3778459360 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f865c1a04d0 con 0x7f865c074e70 2024-04-26T02:01:25.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:25.779+0000 7f8659600700 1 -- 172.21.15.151:0/3778459360 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8650018680 con 0x7f865c074e70 2024-04-26T02:01:25.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:25.779+0000 7f8661e00700 1 -- 172.21.15.151:0/3778459360 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f865c1a7d50 con 0x7f865c074e70 2024-04-26T02:01:25.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:25.780+0000 7f8661e00700 1 -- 172.21.15.151:0/3778459360 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f863c005330 con 0x7f865c074e70 2024-04-26T02:01:25.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:25.783+0000 7f8659600700 1 -- 172.21.15.151:0/3778459360 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f8650010430 con 0x7f865c074e70 2024-04-26T02:01:25.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:25.785+0000 7f8659600700 1 --2- 172.21.15.151:0/3778459360 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f86440a3e30 0x7f86440a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:25.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:25.785+0000 7f8659600700 1 -- 172.21.15.151:0/3778459360 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8455+0+0 (secure 0 0 0) 0x7f86500d09e0 con 0x7f865c074e70 2024-04-26T02:01:25.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:25.785+0000 7f865be00700 1 --2- 172.21.15.151:0/3778459360 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f86440a3e30 0x7f86440a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:25.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:25.786+0000 7f865be00700 1 --2- 172.21.15.151:0/3778459360 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f86440a3e30 0x7f86440a62f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8648005fd0 tx=0x7f8648005f60 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:25.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:25.789+0000 7f8659600700 1 -- 172.21.15.151:0/3778459360 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f865009d060 con 0x7f865c074e70 2024-04-26T02:01:26.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:26.113+0000 7f8661e00700 1 -- 172.21.15.151:0/3778459360 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f863c0058f0 con 0x7f865c074e70 2024-04-26T02:01:26.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:26.114+0000 7f8659600700 1 -- 172.21.15.151:0/3778459360 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11889 (secure 0 0 0) 0x7f865009c960 con 0x7f865c074e70 2024-04-26T02:01:26.114 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T02:01:26.114 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2937864097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2937864097},{"type":"v1","addr":"172.21.15.151:6835","nonce":2937864097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14373,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/2947090802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2947090802},{"type":"v1","addr":"172.21.15.151:6837","nonce":2947090802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/1356563280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1356563280},{"type":"v1","addr":"172.21.15.151:6839","nonce":1356563280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6841/1075240552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1075240552},{"type":"v1","addr":"172.21.15.151:6841","nonce":1075240552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/504675912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":504675912},{"type":"v1","addr":"172.21.15.165:6833","nonce":504675912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1207580015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1207580015},{"type":"v1","addr":"172.21.15.172:6833","nonce":1207580015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:49:40.746876+0000","modified":"2024-04-26T02:00:05.413774+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24505,"mds_2":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/3598138294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3598138294},{"type":"v1","addr":"172.21.15.165:6835","nonce":3598138294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/1847859054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":1847859054},{"type":"v1","addr":"172.21.15.165:6837","nonce":1847859054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.172:6837/55972537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":55972537},{"type":"v1","addr":"172.21.15.172:6837","nonce":55972537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.172:6839/1242302810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1242302810},{"type":"v1","addr":"172.21.15.172:6839","nonce":1242302810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/1969716254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1969716254},{"type":"v1","addr":"172.21.15.165:6839","nonce":1969716254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25954":{"gid":25954,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/3656630171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3656630171},{"type":"v1","addr":"172.21.15.172:6835","nonce":3656630171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24493,24505,24509]},"id":1}]} 2024-04-26T02:01:26.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:26.118+0000 7f8661e00700 1 -- 172.21.15.151:0/3778459360 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f86440a3e30 msgr2=0x7f86440a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:26.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:26.118+0000 7f8661e00700 1 --2- 172.21.15.151:0/3778459360 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f86440a3e30 0x7f86440a62f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8648005fd0 tx=0x7f8648005f60 comp rx=0 tx=0).stop 2024-04-26T02:01:26.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:26.118+0000 7f8661e00700 1 -- 172.21.15.151:0/3778459360 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f865c074e70 msgr2=0x7f865c19b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:26.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:26.118+0000 7f8661e00700 1 --2- 172.21.15.151:0/3778459360 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f865c074e70 0x7f865c19b1d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f865000e9a0 tx=0x7f865000ecb0 comp rx=0 tx=0).stop 2024-04-26T02:01:26.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:26.118+0000 7f8661e00700 1 -- 172.21.15.151:0/3778459360 shutdown_connections 2024-04-26T02:01:26.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:26.118+0000 7f8661e00700 1 --2- 172.21.15.151:0/3778459360 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f86440a3e30 0x7f86440a62f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:26.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:26.118+0000 7f8661e00700 1 --2- 172.21.15.151:0/3778459360 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f865c10f1b0 0x7f865c19faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:26.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:26.118+0000 7f8661e00700 1 --2- 172.21.15.151:0/3778459360 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f865c074e70 0x7f865c19b1d0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:26.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:26.118+0000 7f8661e00700 1 --2- 172.21.15.151:0/3778459360 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f865c075790 0x7f865c19b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:26.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:26.118+0000 7f8661e00700 1 -- 172.21.15.151:0/3778459360 >> 172.21.15.151:0/3778459360 conn(0x7f865c0fe4c0 msgr2=0x7f865c0ff130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:26.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:26.119+0000 7f8661e00700 1 -- 172.21.15.151:0/3778459360 shutdown_connections 2024-04-26T02:01:26.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:26.119+0000 7f8661e00700 1 -- 172.21.15.151:0/3778459360 wait complete. 2024-04-26T02:01:26.119 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 35 2024-04-26T02:01:26.375 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:01:26 smithi151 ceph-mon[28057]: pgmap v244: 97 pgs: 97 active+clean; 72 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 1.2 MiB/s wr, 463 op/s 2024-04-26T02:01:26.376 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:01:26 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/3778459360' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:01:26.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:01:26 smithi172 ceph-mon[32785]: pgmap v244: 97 pgs: 97 active+clean; 72 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 1.2 MiB/s wr, 463 op/s 2024-04-26T02:01:26.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:01:26 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/3778459360' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:01:26.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:01:26 smithi165 ceph-mon[32648]: pgmap v244: 97 pgs: 97 active+clean; 72 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 1.2 MiB/s wr, 463 op/s 2024-04-26T02:01:26.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:01:26 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/3778459360' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:01:27.253 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24509 quiesce db / --timeout 60 --expiration 82.9 --await-for 42.0' 2024-04-26T02:01:27.253 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24509 quiesce db / --timeout 60 --expiration 82.9 --await-for 42.0 2024-04-26T02:01:27.708 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:01:28.406 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:01:28 smithi151 ceph-mon[28057]: pgmap v245: 97 pgs: 97 active+clean; 72 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 993 KiB/s wr, 368 op/s 2024-04-26T02:01:28.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:01:28 smithi172 ceph-mon[32785]: pgmap v245: 97 pgs: 97 active+clean; 72 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 993 KiB/s wr, 368 op/s 2024-04-26T02:01:28.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:01:28 smithi165 ceph-mon[32648]: pgmap v245: 97 pgs: 97 active+clean; 72 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 993 KiB/s wr, 368 op/s 2024-04-26T02:01:30.018 INFO:tasks.workunit.client.0.smithi151.stdout:10000 mapread 0x33cc4e3 thru 0x33cec16 (0x2734 bytes) 2024-04-26T02:01:30.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.276+0000 7f6caac00700 1 -- 172.21.15.151:0/1812073345 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ca4102af0 msgr2=0x7f6ca4102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:30.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.276+0000 7f6caac00700 1 --2- 172.21.15.151:0/1812073345 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ca4102af0 0x7f6ca4102f70 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6c900098f0 tx=0x7f6c90009c00 comp rx=0 tx=0).stop 2024-04-26T02:01:30.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.276+0000 7f6caac00700 1 -- 172.21.15.151:0/1812073345 shutdown_connections 2024-04-26T02:01:30.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.276+0000 7f6caac00700 1 --2- 172.21.15.151:0/1812073345 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6ca41034b0 0x7f6ca410f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:30.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.276+0000 7f6caac00700 1 --2- 172.21.15.151:0/1812073345 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ca4108a30 0x7f6ca4108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:30.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.276+0000 7f6caac00700 1 --2- 172.21.15.151:0/1812073345 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ca4102af0 0x7f6ca4102f70 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:30.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.276+0000 7f6caac00700 1 -- 172.21.15.151:0/1812073345 >> 172.21.15.151:0/1812073345 conn(0x7f6ca4079b10 msgr2=0x7f6ca4079f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:30.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.277+0000 7f6caac00700 1 -- 172.21.15.151:0/1812073345 shutdown_connections 2024-04-26T02:01:30.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.277+0000 7f6caac00700 1 -- 172.21.15.151:0/1812073345 wait complete. 2024-04-26T02:01:30.278 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.278+0000 7f6caac00700 1 Processor -- start 2024-04-26T02:01:30.278 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.278+0000 7f6caac00700 1 -- start start 2024-04-26T02:01:30.278 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.279+0000 7f6caac00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ca4102af0 0x7f6ca419b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:30.278 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.279+0000 7f6caac00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ca41034b0 0x7f6ca419b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:30.278 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.279+0000 7f6caac00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6ca4108a30 0x7f6ca419fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:30.278 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.279+0000 7f6caac00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca4112c60 con 0x7f6ca41034b0 2024-04-26T02:01:30.278 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.279+0000 7f6caac00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca4112ae0 con 0x7f6ca4108a30 2024-04-26T02:01:30.278 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.279+0000 7f6caac00700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca4112de0 con 0x7f6ca4102af0 2024-04-26T02:01:30.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.279+0000 7f6ca3e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ca4102af0 0x7f6ca419b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:30.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.279+0000 7f6ca3e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ca4102af0 0x7f6ca419b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:56256/0 (socket says 172.21.15.151:56256) 2024-04-26T02:01:30.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.279+0000 7f6ca3e00700 1 -- 172.21.15.151:0/2482758823 learned_addr learned my addr 172.21.15.151:0/2482758823 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:01:30.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.279+0000 7f6ca3400700 1 --2- 172.21.15.151:0/2482758823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ca41034b0 0x7f6ca419b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:30.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.279+0000 7f6ca8c00700 1 --2- 172.21.15.151:0/2482758823 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6ca4108a30 0x7f6ca419fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:30.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.279+0000 7f6ca3e00700 1 -- 172.21.15.151:0/2482758823 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6ca4108a30 msgr2=0x7f6ca419fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:30.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.280+0000 7f6ca3e00700 1 --2- 172.21.15.151:0/2482758823 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6ca4108a30 0x7f6ca419fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:30.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.280+0000 7f6ca3e00700 1 -- 172.21.15.151:0/2482758823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ca41034b0 msgr2=0x7f6ca419b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:30.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.280+0000 7f6ca3e00700 1 --2- 172.21.15.151:0/2482758823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ca41034b0 0x7f6ca419b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:30.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.280+0000 7f6ca3e00700 1 -- 172.21.15.151:0/2482758823 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ca41a0330 con 0x7f6ca4102af0 2024-04-26T02:01:30.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.280+0000 7f6ca3400700 1 --2- 172.21.15.151:0/2482758823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ca41034b0 0x7f6ca419b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:01:30.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.280+0000 7f6ca3e00700 1 --2- 172.21.15.151:0/2482758823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ca4102af0 0x7f6ca419b320 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6c9400ec10 tx=0x7f6c9400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:30.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.280+0000 7f6ca0c00700 1 -- 172.21.15.151:0/2482758823 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9400cd40 con 0x7f6ca4102af0 2024-04-26T02:01:30.281 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.280+0000 7f6ca0c00700 1 -- 172.21.15.151:0/2482758823 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c940044d0 con 0x7f6ca4102af0 2024-04-26T02:01:30.281 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.280+0000 7f6ca0c00700 1 -- 172.21.15.151:0/2482758823 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c94010690 con 0x7f6ca4102af0 2024-04-26T02:01:30.281 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.280+0000 7f6caac00700 1 -- 172.21.15.151:0/2482758823 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ca41a0620 con 0x7f6ca4102af0 2024-04-26T02:01:30.281 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.281+0000 7f6caac00700 1 -- 172.21.15.151:0/2482758823 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ca41a7ea0 con 0x7f6ca4102af0 2024-04-26T02:01:30.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.284+0000 7f6ca0c00700 1 -- 172.21.15.151:0/2482758823 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f6c94004020 con 0x7f6ca4102af0 2024-04-26T02:01:30.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.285+0000 7f6ca0c00700 1 --2- 172.21.15.151:0/2482758823 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f6c880a3fb0 0x7f6c880a6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:30.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.286+0000 7f6ca0c00700 1 -- 172.21.15.151:0/2482758823 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8455+0+0 (secure 0 0 0) 0x7f6c940d09f0 con 0x7f6ca4102af0 2024-04-26T02:01:30.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.286+0000 7f6ca3400700 1 --2- 172.21.15.151:0/2482758823 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f6c880a3fb0 0x7f6c880a6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:30.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.286+0000 7f6cb562db80 1 Processor -- start 2024-04-26T02:01:30.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.286+0000 7f6cb562db80 1 -- start start 2024-04-26T02:01:30.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.287+0000 7f6ca3400700 1 --2- 172.21.15.151:0/2482758823 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f6c880a3fb0 0x7f6c880a6470 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4101bd0 tx=0x7f6c90019040 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:30.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.287+0000 7f6cb562db80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560d343abdd0 0x560d343ac1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:30.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.287+0000 7f6cb562db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560d343ac6f0 0x560d343bf830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:30.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.287+0000 7f6cb562db80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d343bfd70 0x560d343c2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:30.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.287+0000 7f6cb562db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560d342ff020 con 0x560d343ac6f0 2024-04-26T02:01:30.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.287+0000 7f6cb562db80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560d342a58c0 con 0x560d343abdd0 2024-04-26T02:01:30.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.287+0000 7f6cb562db80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560d342fafe0 con 0x560d343bfd70 2024-04-26T02:01:30.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.287+0000 7f6ca3e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560d343abdd0 0x560d343ac1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:30.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.288+0000 7f6ca3e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560d343abdd0 0x560d343ac1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:41970/0 (socket says 172.21.15.151:41970) 2024-04-26T02:01:30.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.288+0000 7f6ca3e00700 1 -- 172.21.15.151:0/3474917895 learned_addr learned my addr 172.21.15.151:0/3474917895 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:01:30.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.288+0000 7f6ca3400700 1 --2- 172.21.15.151:0/3474917895 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560d343ac6f0 0x560d343bf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:30.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.288+0000 7f6ca3e00700 1 -- 172.21.15.151:0/3474917895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d343bfd70 msgr2=0x560d343c2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:30.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.288+0000 7f6ca3e00700 1 --2- 172.21.15.151:0/3474917895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d343bfd70 0x560d343c2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:30.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.288+0000 7f6ca3e00700 1 -- 172.21.15.151:0/3474917895 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560d343ac6f0 msgr2=0x560d343bf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:30.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.288+0000 7f6ca3e00700 1 --2- 172.21.15.151:0/3474917895 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560d343ac6f0 0x560d343bf830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:30.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.288+0000 7f6ca3e00700 1 -- 172.21.15.151:0/3474917895 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d3429b810 con 0x560d343abdd0 2024-04-26T02:01:30.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.288+0000 7f6ca3e00700 1 --2- 172.21.15.151:0/3474917895 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560d343abdd0 0x560d343ac1b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6c940a50b0 tx=0x7f6c9409c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30039120b44c4971 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:30.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.288+0000 7f6c99600700 1 -- 172.21.15.151:0/3474917895 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c94004310 con 0x560d343abdd0 2024-04-26T02:01:30.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.288+0000 7f6c99600700 1 -- 172.21.15.151:0/3474917895 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c9409d070 con 0x560d343abdd0 2024-04-26T02:01:30.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.288+0000 7f6c99600700 1 -- 172.21.15.151:0/3474917895 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9409cbd0 con 0x560d343abdd0 2024-04-26T02:01:30.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.289+0000 7f6cb562db80 1 -- 172.21.15.151:0/3474917895 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560d343abdd0 msgr2=0x560d343ac1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:30.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.289+0000 7f6cb562db80 1 --2- 172.21.15.151:0/3474917895 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560d343abdd0 0x560d343ac1b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6c940a50b0 tx=0x7f6c9409c740 comp rx=0 tx=0).stop 2024-04-26T02:01:30.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.290+0000 7f6cb562db80 1 -- 172.21.15.151:0/3474917895 shutdown_connections 2024-04-26T02:01:30.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.290+0000 7f6cb562db80 1 --2- 172.21.15.151:0/3474917895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d343bfd70 0x560d343c2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:30.290 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.290+0000 7f6cb562db80 1 --2- 172.21.15.151:0/3474917895 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560d343abdd0 0x560d343ac1b0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:30.290 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.290+0000 7f6cb562db80 1 --2- 172.21.15.151:0/3474917895 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560d343ac6f0 0x560d343bf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:30.290 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.290+0000 7f6cb562db80 1 -- 172.21.15.151:0/3474917895 >> 172.21.15.151:0/3474917895 conn(0x560d342db0e0 msgr2=0x560d343a82e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:30.290 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.290+0000 7f6cb562db80 1 -- 172.21.15.151:0/3474917895 shutdown_connections 2024-04-26T02:01:30.290 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.290+0000 7f6cb562db80 1 -- 172.21.15.151:0/3474917895 wait complete. 2024-04-26T02:01:30.290 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.290+0000 7f6cb562db80 1 -- start start 2024-04-26T02:01:30.290 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.291+0000 7f6cb562db80 1 Processor -- start 2024-04-26T02:01:30.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.291+0000 7f6cb562db80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560d343abdd0 0x560d343bf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:30.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.291+0000 7f6cb562db80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d343ac6f0 0x560d343bc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:30.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.291+0000 7f6cb562db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560d343bfd70 0x560d343bcdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:30.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.291+0000 7f6cb562db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560d343c2800 con 0x560d343bfd70 2024-04-26T02:01:30.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.291+0000 7f6cb562db80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560d342b89f0 con 0x560d343abdd0 2024-04-26T02:01:30.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.291+0000 7f6cb562db80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560d343c2980 con 0x560d343ac6f0 2024-04-26T02:01:30.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.291+0000 7f6cb562db80 10 client.0 _ll_register_callbacks cb 0x560d342978b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:01:30.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.292+0000 7f6ca3400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d343ac6f0 0x560d343bc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:30.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.292+0000 7f6ca3400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d343ac6f0 0x560d343bc890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:56272/0 (socket says 172.21.15.151:56272) 2024-04-26T02:01:30.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.292+0000 7f6ca3400700 1 -- 172.21.15.151:0/159530409 learned_addr learned my addr 172.21.15.151:0/159530409 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:01:30.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.293+0000 7f6ca8c00700 1 --2- 172.21.15.151:0/159530409 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560d343bfd70 0x560d343bcdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:30.294 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.295+0000 7f6ca3e00700 1 --2- 172.21.15.151:0/159530409 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560d343abdd0 0x560d343bf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:30.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.295+0000 7f6ca3400700 1 -- 172.21.15.151:0/159530409 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560d343abdd0 msgr2=0x560d343bf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:30.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.295+0000 7f6ca3400700 1 --2- 172.21.15.151:0/159530409 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560d343abdd0 0x560d343bf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:30.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.295+0000 7f6ca3400700 1 -- 172.21.15.151:0/159530409 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560d343bfd70 msgr2=0x560d343bcdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:30.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.295+0000 7f6ca3400700 1 --2- 172.21.15.151:0/159530409 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560d343bfd70 0x560d343bcdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:30.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.295+0000 7f6ca3400700 1 -- 172.21.15.151:0/159530409 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d3424ca60 con 0x560d343ac6f0 2024-04-26T02:01:30.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.296+0000 7f6ca8c00700 1 --2- 172.21.15.151:0/159530409 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560d343bfd70 0x560d343bcdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:01:30.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.296+0000 7f6ca3400700 1 --2- 172.21.15.151:0/159530409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d343ac6f0 0x560d343bc890 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6c90009230 tx=0x7f6c90009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f0f890a0d9e6e76 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:30.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.297+0000 7f6c8e000700 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-26T02:01:30.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.297+0000 7f6c8e000700 1 -- 172.21.15.151:0/159530409 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c90003b10 con 0x560d343ac6f0 2024-04-26T02:01:30.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.297+0000 7f6c8e000700 1 -- 172.21.15.151:0/159530409 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8455+0+0 (secure 0 0 0) 0x7f6c90003cb0 con 0x560d343ac6f0 2024-04-26T02:01:30.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.297+0000 7f6cb562db80 1 -- 172.21.15.151:0/159530409 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c9409d070 con 0x560d343ac6f0 2024-04-26T02:01:30.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.298+0000 7f6c8e000700 1 -- 172.21.15.151:0/159530409 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f6c9000b560 con 0x560d343ac6f0 2024-04-26T02:01:30.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.298+0000 7f6cb562db80 10 client.26119 fetch_fsmap learned FSMap version 35 2024-04-26T02:01:30.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.298+0000 7f6cb562db80 1 -- 172.21.15.151:0/159530409 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x560d34295c80 con 0x560d343ac6f0 2024-04-26T02:01:30.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.299+0000 7f6c8e000700 1 -- 172.21.15.151:0/159530409 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c9409d070 con 0x560d343ac6f0 2024-04-26T02:01:30.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.299+0000 7f6c8e000700 1 -- 172.21.15.151:0/159530409 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 35) ==== 8338+0+0 (secure 0 0 0) 0x7f6c9002c070 con 0x560d343ac6f0 2024-04-26T02:01:30.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.299+0000 7f6cb562db80 10 client.26119 fetch_fsmap finished waiting for FSMap version 35 2024-04-26T02:01:30.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.299+0000 7f6cb562db80 10 client.26119 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-26T02:01:30.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.299+0000 7f6cb562db80 20 client.26119 populate_metadata read hostname 'smithi151' 2024-04-26T02:01:30.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.299+0000 7f6cb562db80 1 --2- 172.21.15.151:0/159530409 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x560d343cf610 0x560d343d1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:30.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.299+0000 7f6cb562db80 4 client.26119 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:01:30.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.299+0000 7f6cb562db80 1 -- 172.21.15.151:0/159530409 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d3430c810 con 0x560d343cf610 2024-04-26T02:01:30.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.300+0000 7f6ca3e00700 1 --2- 172.21.15.151:0/159530409 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x560d343cf610 0x560d343d1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:30.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.300+0000 7f6ca3e00700 1 --2- 172.21.15.151:0/159530409 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x560d343cf610 0x560d343d1a00 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b833fc5b592c8f16 server_cookie=84559ab50eebaa9b in_seq=0 out_seq=0 2024-04-26T02:01:30.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.300+0000 7f6c8e000700 10 client.26119 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:01:30.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.302+0000 7f6c8e000700 1 -- 172.21.15.151:0/159530409 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560d3430c810 con 0x560d343cf610 2024-04-26T02:01:30.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.302+0000 7f6c8e000700 10 client.26119 handle_command_reply: tid=0 2024-04-26T02:01:30.304 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.302+0000 7f6cb562db80 1 client.26119 shutdown 2024-04-26T02:01:30.304 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.302+0000 7f6cb562db80 1 -- 172.21.15.151:0/159530409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d343ac6f0 msgr2=0x560d343bc890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:30.304 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.302+0000 7f6cb562db80 1 --2- 172.21.15.151:0/159530409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d343ac6f0 0x560d343bc890 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6c90009230 tx=0x7f6c90009260 comp rx=0 tx=0).stop 2024-04-26T02:01:30.304 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.302+0000 7f6cb562db80 1 -- 172.21.15.151:0/159530409 shutdown_connections 2024-04-26T02:01:30.304 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.302+0000 7f6cb562db80 1 --2- 172.21.15.151:0/159530409 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560d343bfd70 0x560d343bcdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:30.304 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.302+0000 7f6cb562db80 1 --2- 172.21.15.151:0/159530409 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x560d343cf610 0x560d343d1a00 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:30.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.302+0000 7f6cb562db80 1 --2- 172.21.15.151:0/159530409 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560d343abdd0 0x560d343bf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:30.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.302+0000 7f6cb562db80 1 --2- 172.21.15.151:0/159530409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d343ac6f0 0x560d343bc890 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:30.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.302+0000 7f6cb562db80 1 -- 172.21.15.151:0/159530409 >> 172.21.15.151:0/159530409 conn(0x560d342db0e0 msgr2=0x560d343a7ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:30.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.302+0000 7f6c8e000700 0 client.26119 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:01:30.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.303+0000 7f6cb562db80 1 -- 172.21.15.151:0/159530409 shutdown_connections 2024-04-26T02:01:30.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.303+0000 7f6cb562db80 1 -- 172.21.15.151:0/159530409 wait complete. 2024-04-26T02:01:30.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.303+0000 7f6cb562db80 20 client.26119 trim_cache size 0 max 16384 2024-04-26T02:01:30.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.327+0000 7f6cb562db80 1 Processor -- start 2024-04-26T02:01:30.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.327+0000 7f6cb562db80 1 -- start start 2024-04-26T02:01:30.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.327+0000 7f6cb562db80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560d343abdd0 0x560d343cb700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:30.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.327+0000 7f6cb562db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560d343ac6f0 0x560d343be7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:30.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.327+0000 7f6cb562db80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d343bfd70 0x560d343d7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:30.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.328+0000 7f6cb562db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca41ad9b0 con 0x560d343ac6f0 2024-04-26T02:01:30.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.328+0000 7f6cb562db80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca41ab240 con 0x560d343abdd0 2024-04-26T02:01:30.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.328+0000 7f6cb562db80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca410da90 con 0x560d343bfd70 2024-04-26T02:01:30.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.328+0000 7f6ca3e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560d343abdd0 0x560d343cb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:30.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.328+0000 7f6ca3e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560d343abdd0 0x560d343cb700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:41986/0 (socket says 172.21.15.151:41986) 2024-04-26T02:01:30.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.328+0000 7f6ca3e00700 1 -- 172.21.15.151:0/1676523068 learned_addr learned my addr 172.21.15.151:0/1676523068 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:01:30.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.328+0000 7f6ca8c00700 1 --2- 172.21.15.151:0/1676523068 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d343bfd70 0x560d343d7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:30.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.328+0000 7f6ca3400700 1 --2- 172.21.15.151:0/1676523068 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560d343ac6f0 0x560d343be7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:30.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.328+0000 7f6ca3e00700 1 -- 172.21.15.151:0/1676523068 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d343bfd70 msgr2=0x560d343d7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:30.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.328+0000 7f6ca3e00700 1 --2- 172.21.15.151:0/1676523068 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d343bfd70 0x560d343d7290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:30.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.328+0000 7f6ca3e00700 1 -- 172.21.15.151:0/1676523068 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560d343ac6f0 msgr2=0x560d343be7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:30.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.328+0000 7f6ca3e00700 1 --2- 172.21.15.151:0/1676523068 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560d343ac6f0 0x560d343be7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:30.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.328+0000 7f6ca3e00700 1 -- 172.21.15.151:0/1676523068 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d343d2060 con 0x560d343abdd0 2024-04-26T02:01:30.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.328+0000 7f6ca3400700 1 --2- 172.21.15.151:0/1676523068 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560d343ac6f0 0x560d343be7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:01:30.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.329+0000 7f6ca3e00700 1 --2- 172.21.15.151:0/1676523068 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560d343abdd0 0x560d343cb700 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6c94018000 tx=0x7f6c9400cee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b56ec4222064df1f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:30.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.329+0000 7f6c98c00700 1 -- 172.21.15.151:0/1676523068 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9409c970 con 0x560d343abdd0 2024-04-26T02:01:30.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.329+0000 7f6c98c00700 1 -- 172.21.15.151:0/1676523068 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c9409d070 con 0x560d343abdd0 2024-04-26T02:01:30.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.329+0000 7f6c98c00700 1 -- 172.21.15.151:0/1676523068 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c94003880 con 0x560d343abdd0 2024-04-26T02:01:30.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.330+0000 7f6cb562db80 1 -- 172.21.15.151:0/1676523068 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560d343abdd0 msgr2=0x560d343cb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:30.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.330+0000 7f6cb562db80 1 --2- 172.21.15.151:0/1676523068 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560d343abdd0 0x560d343cb700 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6c94018000 tx=0x7f6c9400cee0 comp rx=0 tx=0).stop 2024-04-26T02:01:30.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.330+0000 7f6cb562db80 1 -- 172.21.15.151:0/1676523068 shutdown_connections 2024-04-26T02:01:30.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.330+0000 7f6cb562db80 1 --2- 172.21.15.151:0/1676523068 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d343bfd70 0x560d343d7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:30.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.330+0000 7f6cb562db80 1 --2- 172.21.15.151:0/1676523068 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560d343abdd0 0x560d343cb700 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:30.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.330+0000 7f6cb562db80 1 --2- 172.21.15.151:0/1676523068 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560d343ac6f0 0x560d343be7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:30.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.330+0000 7f6cb562db80 1 -- 172.21.15.151:0/1676523068 >> 172.21.15.151:0/1676523068 conn(0x560d342db0e0 msgr2=0x560d343be0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:30.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.330+0000 7f6cb562db80 1 -- 172.21.15.151:0/1676523068 shutdown_connections 2024-04-26T02:01:30.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.330+0000 7f6cb562db80 1 -- 172.21.15.151:0/1676523068 wait complete. 2024-04-26T02:01:30.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.331+0000 7f6cb562db80 1 -- start start 2024-04-26T02:01:30.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.331+0000 7f6cb562db80 1 Processor -- start 2024-04-26T02:01:30.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.331+0000 7f6cb562db80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d343abdd0 0x560d343c4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:30.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.331+0000 7f6cb562db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560d343ac6f0 0x560d343c5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:30.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.331+0000 7f6cb562db80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560d343bfd70 0x560d343c5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:30.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.331+0000 7f6cb562db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca41a9750 con 0x560d343ac6f0 2024-04-26T02:01:30.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.331+0000 7f6cb562db80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca41aab70 con 0x560d343bfd70 2024-04-26T02:01:30.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.331+0000 7f6cb562db80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca41a9e20 con 0x560d343abdd0 2024-04-26T02:01:30.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.331+0000 7f6cb562db80 10 client.0 _ll_register_callbacks cb 0x560d342978b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:01:30.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.332+0000 7f6ca8c00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560d343bfd70 0x560d343c5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:30.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.332+0000 7f6ca8c00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560d343bfd70 0x560d343c5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:41990/0 (socket says 172.21.15.151:41990) 2024-04-26T02:01:30.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.332+0000 7f6ca3e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d343abdd0 0x560d343c4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:30.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.332+0000 7f6ca8c00700 1 -- 172.21.15.151:0/2449583957 learned_addr learned my addr 172.21.15.151:0/2449583957 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:01:30.333 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.332+0000 7f6ca3400700 1 --2- 172.21.15.151:0/2449583957 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560d343ac6f0 0x560d343c5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:30.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.332+0000 7f6ca3e00700 1 -- 172.21.15.151:0/2449583957 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560d343bfd70 msgr2=0x560d343c5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:30.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.332+0000 7f6ca3e00700 1 --2- 172.21.15.151:0/2449583957 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560d343bfd70 0x560d343c5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:30.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.332+0000 7f6ca3e00700 1 -- 172.21.15.151:0/2449583957 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560d343ac6f0 msgr2=0x560d343c5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:30.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.332+0000 7f6ca3e00700 1 --2- 172.21.15.151:0/2449583957 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560d343ac6f0 0x560d343c5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:30.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.332+0000 7f6ca3e00700 1 -- 172.21.15.151:0/2449583957 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d343a8db0 con 0x560d343abdd0 2024-04-26T02:01:30.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.332+0000 7f6ca3e00700 1 --2- 172.21.15.151:0/2449583957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d343abdd0 0x560d343c4be0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x560d343a8fc0 tx=0x7f6c9409c3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a2f8be226551c8d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:30.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.332+0000 7f6c8e000700 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-26T02:01:30.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.333+0000 7f6c8e000700 1 -- 172.21.15.151:0/2449583957 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c940a0d20 con 0x560d343abdd0 2024-04-26T02:01:30.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.333+0000 7f6c8e000700 1 -- 172.21.15.151:0/2449583957 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8455+0+0 (secure 0 0 0) 0x7f6c94003780 con 0x560d343abdd0 2024-04-26T02:01:30.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.333+0000 7f6cb562db80 1 -- 172.21.15.151:0/2449583957 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c9409d070 con 0x560d343abdd0 2024-04-26T02:01:30.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.333+0000 7f6c8e000700 1 -- 172.21.15.151:0/2449583957 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f6c9409c970 con 0x560d343abdd0 2024-04-26T02:01:30.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.333+0000 7f6cb562db80 10 client.26125 fetch_fsmap learned FSMap version 35 2024-04-26T02:01:30.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.333+0000 7f6cb562db80 1 -- 172.21.15.151:0/2449583957 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x560d34436d10 con 0x560d343abdd0 2024-04-26T02:01:30.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.333+0000 7f6c8e000700 1 -- 172.21.15.151:0/2449583957 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c9409d070 con 0x560d343abdd0 2024-04-26T02:01:30.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.333+0000 7f6c8e000700 1 -- 172.21.15.151:0/2449583957 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 35) ==== 8338+0+0 (secure 0 0 0) 0x7f6c9409cc60 con 0x560d343abdd0 2024-04-26T02:01:30.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.333+0000 7f6cb562db80 10 client.26125 fetch_fsmap finished waiting for FSMap version 35 2024-04-26T02:01:30.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.333+0000 7f6cb562db80 10 client.26125 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-26T02:01:30.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.333+0000 7f6cb562db80 20 client.26125 populate_metadata read hostname 'smithi151' 2024-04-26T02:01:30.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.333+0000 7f6cb562db80 1 --2- 172.21.15.151:0/2449583957 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x560d34436f90 0x560d34439380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:30.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.333+0000 7f6cb562db80 4 client.26125 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 82.9, "await_for": 42.0}] 2024-04-26T02:01:30.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.333+0000 7f6cb562db80 1 -- 172.21.15.151:0/2449583957 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 82.9, "await_for": 42.0}) -- 0x560d343068b0 con 0x560d34436f90 2024-04-26T02:01:30.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.334+0000 7f6ca3400700 1 --2- 172.21.15.151:0/2449583957 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x560d34436f90 0x560d34439380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:30.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.334+0000 7f6ca3400700 1 --2- 172.21.15.151:0/2449583957 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x560d34436f90 0x560d34439380 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=336659bb3b819ab9 server_cookie=de65b57f554a29f8 in_seq=0 out_seq=0 2024-04-26T02:01:30.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:30.334+0000 7f6c8e000700 10 client.26125 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:01:30.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:01:30 smithi172 ceph-mon[32785]: pgmap v246: 97 pgs: 97 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.8 MiB/s wr, 583 op/s 2024-04-26T02:01:30.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:01:30 smithi165 ceph-mon[32648]: pgmap v246: 97 pgs: 97 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.8 MiB/s wr, 583 op/s 2024-04-26T02:01:30.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:01:30 smithi151 ceph-mon[28057]: pgmap v246: 97 pgs: 97 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.8 MiB/s wr, 583 op/s 2024-04-26T02:01:31.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:31.251+0000 7f6c8e000700 1 -- 172.21.15.151:0/2449583957 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x560d343068b0 con 0x560d34436f90 2024-04-26T02:01:31.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:31.251+0000 7f6c8e000700 10 client.26125 handle_command_reply: tid=0 2024-04-26T02:01:31.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:31.251+0000 7f6cb562db80 1 client.26125 shutdown 2024-04-26T02:01:31.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:31.251+0000 7f6cb562db80 1 -- 172.21.15.151:0/2449583957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d343abdd0 msgr2=0x560d343c4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:31.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:31.251+0000 7f6cb562db80 1 --2- 172.21.15.151:0/2449583957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d343abdd0 0x560d343c4be0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x560d343a8fc0 tx=0x7f6c9409c3f0 comp rx=0 tx=0).stop 2024-04-26T02:01:31.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:31.252+0000 7f6cb562db80 1 -- 172.21.15.151:0/2449583957 shutdown_connections 2024-04-26T02:01:31.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:31.252+0000 7f6cb562db80 1 --2- 172.21.15.151:0/2449583957 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x560d34436f90 0x560d34439380 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:31.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:31.252+0000 7f6cb562db80 1 --2- 172.21.15.151:0/2449583957 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560d343bfd70 0x560d343c5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:31.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:31.252+0000 7f6cb562db80 1 --2- 172.21.15.151:0/2449583957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d343abdd0 0x560d343c4be0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:31.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:31.252+0000 7f6cb562db80 1 --2- 172.21.15.151:0/2449583957 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560d343ac6f0 0x560d343c5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:31.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:31.252+0000 7f6c8e000700 0 client.26125 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:01:31.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:31.252+0000 7f6cb562db80 1 -- 172.21.15.151:0/2449583957 >> 172.21.15.151:0/2449583957 conn(0x560d342db0e0 msgr2=0x560d343be950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:31.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:31.252+0000 7f6cb562db80 1 -- 172.21.15.151:0/2449583957 shutdown_connections 2024-04-26T02:01:31.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:31.252+0000 7f6cb562db80 1 -- 172.21.15.151:0/2449583957 wait complete. 2024-04-26T02:01:31.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:31.252+0000 7f6cb562db80 20 client.26125 trim_cache size 0 max 16384 2024-04-26T02:01:31.252 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-26T02:01:31.253 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 35, 2024-04-26T02:01:31.253 INFO:teuthology.orchestra.run.smithi151.stdout: "leader": 24509, 2024-04-26T02:01:31.253 INFO:teuthology.orchestra.run.smithi151.stdout: "set_version": 22, 2024-04-26T02:01:31.253 INFO:teuthology.orchestra.run.smithi151.stdout: "sets": { 2024-04-26T02:01:31.253 INFO:teuthology.orchestra.run.smithi151.stdout: "f9d9e07c": { 2024-04-26T02:01:31.253 INFO:teuthology.orchestra.run.smithi151.stdout: "version": 22, 2024-04-26T02:01:31.253 INFO:teuthology.orchestra.run.smithi151.stdout: "age_ref": 0.0, 2024-04-26T02:01:31.253 INFO:teuthology.orchestra.run.smithi151.stdout: "state": { 2024-04-26T02:01:31.253 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "QUIESCED", 2024-04-26T02:01:31.253 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 0.0 2024-04-26T02:01:31.253 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:01:31.253 INFO:teuthology.orchestra.run.smithi151.stdout: "timeout": 60.0, 2024-04-26T02:01:31.253 INFO:teuthology.orchestra.run.smithi151.stdout: "expiration": 82.9, 2024-04-26T02:01:31.254 INFO:teuthology.orchestra.run.smithi151.stdout: "members": { 2024-04-26T02:01:31.254 INFO:teuthology.orchestra.run.smithi151.stdout: "file:/": { 2024-04-26T02:01:31.254 INFO:teuthology.orchestra.run.smithi151.stdout: "excluded": false, 2024-04-26T02:01:31.254 INFO:teuthology.orchestra.run.smithi151.stdout: "state": { 2024-04-26T02:01:31.254 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "QUIESCED", 2024-04-26T02:01:31.254 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 0.0 2024-04-26T02:01:31.254 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:01:31.254 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:01:31.254 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:01:31.254 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:01:31.254 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:01:31.254 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-26T02:01:31.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:31.253+0000 7f6caac00700 1 -- 172.21.15.151:0/2482758823 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f6c880a3fb0 msgr2=0x7f6c880a6470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:31.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:31.253+0000 7f6caac00700 1 --2- 172.21.15.151:0/2482758823 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f6c880a3fb0 0x7f6c880a6470 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4101bd0 tx=0x7f6c90019040 comp rx=0 tx=0).stop 2024-04-26T02:01:31.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:31.253+0000 7f6caac00700 1 -- 172.21.15.151:0/2482758823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ca4102af0 msgr2=0x7f6ca419b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:31.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:31.253+0000 7f6caac00700 1 --2- 172.21.15.151:0/2482758823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ca4102af0 0x7f6ca419b320 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6c9400ec10 tx=0x7f6c9400ef20 comp rx=0 tx=0).stop 2024-04-26T02:01:31.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:31.254+0000 7f6caac00700 1 -- 172.21.15.151:0/2482758823 shutdown_connections 2024-04-26T02:01:31.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:31.254+0000 7f6caac00700 1 --2- 172.21.15.151:0/2482758823 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f6c880a3fb0 0x7f6c880a6470 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:31.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:31.254+0000 7f6caac00700 1 --2- 172.21.15.151:0/2482758823 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6ca4108a30 0x7f6ca419fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:31.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:31.254+0000 7f6caac00700 1 --2- 172.21.15.151:0/2482758823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ca4102af0 0x7f6ca419b320 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:31.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:31.254+0000 7f6caac00700 1 --2- 172.21.15.151:0/2482758823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ca41034b0 0x7f6ca419b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:31.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:31.254+0000 7f6caac00700 1 -- 172.21.15.151:0/2482758823 >> 172.21.15.151:0/2482758823 conn(0x7f6ca4079b10 msgr2=0x7f6ca4074f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:31.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:31.254+0000 7f6caac00700 1 -- 172.21.15.151:0/2482758823 shutdown_connections 2024-04-26T02:01:31.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:31.254+0000 7f6caac00700 1 -- 172.21.15.151:0/2482758823 wait complete. 2024-04-26T02:01:32.165 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'f9d9e07c', quiesce took 9.4 seconds. Will release after: 22.9 2024-04-26T02:01:32.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:01:32 smithi151 ceph-mon[28057]: pgmap v247: 97 pgs: 97 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 1.3 MiB/s wr, 393 op/s 2024-04-26T02:01:32.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:01:32 smithi172 ceph-mon[32785]: pgmap v247: 97 pgs: 97 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 1.3 MiB/s wr, 393 op/s 2024-04-26T02:01:32.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:01:32 smithi165 ceph-mon[32648]: pgmap v247: 97 pgs: 97 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 1.3 MiB/s wr, 393 op/s 2024-04-26T02:01:34.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:01:34 smithi172 ceph-mon[32785]: pgmap v248: 97 pgs: 97 active+clean; 80 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 1.4 MiB/s wr, 426 op/s 2024-04-26T02:01:34.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:01:34 smithi165 ceph-mon[32648]: pgmap v248: 97 pgs: 97 active+clean; 80 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 1.4 MiB/s wr, 426 op/s 2024-04-26T02:01:34.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:01:34 smithi151 ceph-mon[28057]: pgmap v248: 97 pgs: 97 active+clean; 80 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 1.4 MiB/s wr, 426 op/s 2024-04-26T02:01:36.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:01:36 smithi172 ceph-mon[32785]: pgmap v249: 97 pgs: 97 active+clean; 86 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 1.5 MiB/s wr, 439 op/s 2024-04-26T02:01:36.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:01:36 smithi165 ceph-mon[32648]: pgmap v249: 97 pgs: 97 active+clean; 86 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 1.5 MiB/s wr, 439 op/s 2024-04-26T02:01:36.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:01:36 smithi151 ceph-mon[28057]: pgmap v249: 97 pgs: 97 active+clean; 86 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 1.5 MiB/s wr, 439 op/s 2024-04-26T02:01:37.352 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.352+0000 7fd5d8c00640 1 -- 172.21.15.151:0/3496298953 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd5d4104d50 msgr2=0x7fd5d4108d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:37.352 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.352+0000 7fd5d8c00640 1 --2- 172.21.15.151:0/3496298953 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd5d4104d50 0x7fd5d4108d90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0009900 tx=0x7fd5c002a400 comp rx=0 tx=0).stop 2024-04-26T02:01:37.352 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.353+0000 7fd5d8c00640 1 -- 172.21.15.151:0/3496298953 shutdown_connections 2024-04-26T02:01:37.352 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.353+0000 7fd5d8c00640 1 --2- 172.21.15.151:0/3496298953 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5d4109470 0x7fd5d4111040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:37.352 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.353+0000 7fd5d8c00640 1 --2- 172.21.15.151:0/3496298953 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd5d4104d50 0x7fd5d4108d90 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:37.352 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.353+0000 7fd5d8c00640 1 --2- 172.21.15.151:0/3496298953 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5d41043a0 0x7fd5d4104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:37.352 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.353+0000 7fd5d8c00640 1 -- 172.21.15.151:0/3496298953 >> 172.21.15.151:0/3496298953 conn(0x7fd5d40ffc40 msgr2=0x7fd5d4102060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:37.352 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.353+0000 7fd5d8c00640 1 -- 172.21.15.151:0/3496298953 shutdown_connections 2024-04-26T02:01:37.352 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.353+0000 7fd5d8c00640 1 -- 172.21.15.151:0/3496298953 wait complete. 2024-04-26T02:01:37.354 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.354+0000 7fd5d8c00640 1 Processor -- start 2024-04-26T02:01:37.354 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.354+0000 7fd5d8c00640 1 -- start start 2024-04-26T02:01:37.354 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.355+0000 7fd5d8c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5d41043a0 0x7fd5d41a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:37.354 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.355+0000 7fd5d8c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5d4109470 0x7fd5d41a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:37.355 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.355+0000 7fd5d8c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd5d419a7e0 0x7fd5d419ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:37.355 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.355+0000 7fd5d8c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d4114400 con 0x7fd5d4109470 2024-04-26T02:01:37.355 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.355+0000 7fd5d8c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d4114280 con 0x7fd5d419a7e0 2024-04-26T02:01:37.355 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.355+0000 7fd5d8c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d4114580 con 0x7fd5d41043a0 2024-04-26T02:01:37.355 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.355+0000 7fd5d1e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5d41043a0 0x7fd5d41a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:37.355 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.355+0000 7fd5d1400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5d4109470 0x7fd5d41a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:37.355 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.355+0000 7fd5d1e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5d41043a0 0x7fd5d41a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:56308/0 (socket says 172.21.15.151:56308) 2024-04-26T02:01:37.355 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.355+0000 7fd5d1e00640 1 -- 172.21.15.151:0/3875045062 learned_addr learned my addr 172.21.15.151:0/3875045062 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:01:37.355 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.355+0000 7fd5d2800640 1 --2- 172.21.15.151:0/3875045062 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd5d419a7e0 0x7fd5d419ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:37.355 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.355+0000 7fd5d1e00640 1 -- 172.21.15.151:0/3875045062 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd5d419a7e0 msgr2=0x7fd5d419ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:37.355 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.355+0000 7fd5d1e00640 1 --2- 172.21.15.151:0/3875045062 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd5d419a7e0 0x7fd5d419ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:37.355 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.355+0000 7fd5d1e00640 1 -- 172.21.15.151:0/3875045062 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5d4109470 msgr2=0x7fd5d41a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:37.356 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.355+0000 7fd5d1e00640 1 --2- 172.21.15.151:0/3875045062 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5d4109470 0x7fd5d41a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:37.356 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.355+0000 7fd5d1e00640 1 -- 172.21.15.151:0/3875045062 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5d419b370 con 0x7fd5d41043a0 2024-04-26T02:01:37.356 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.356+0000 7fd5d1e00640 1 --2- 172.21.15.151:0/3875045062 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5d41043a0 0x7fd5d41a0620 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd5c400e9f0 tx=0x7fd5c400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:37.356 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.356+0000 7fd5caa00640 1 -- 172.21.15.151:0/3875045062 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c400cd80 con 0x7fd5d41043a0 2024-04-26T02:01:37.357 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.356+0000 7fd5caa00640 1 -- 172.21.15.151:0/3875045062 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5c40054d0 con 0x7fd5d41043a0 2024-04-26T02:01:37.357 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.356+0000 7fd5caa00640 1 -- 172.21.15.151:0/3875045062 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c4010690 con 0x7fd5d41043a0 2024-04-26T02:01:37.357 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.356+0000 7fd5d8c00640 1 -- 172.21.15.151:0/3875045062 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5d419b660 con 0x7fd5d41043a0 2024-04-26T02:01:37.357 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.356+0000 7fd5d8c00640 1 -- 172.21.15.151:0/3875045062 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5d41a7770 con 0x7fd5d41043a0 2024-04-26T02:01:37.361 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.359+0000 7fd5caa00640 1 -- 172.21.15.151:0/3875045062 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7fd5c4004e90 con 0x7fd5d41043a0 2024-04-26T02:01:37.361 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.360+0000 7fd5dba42740 1 Processor -- start 2024-04-26T02:01:37.361 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.360+0000 7fd5dba42740 1 -- start start 2024-04-26T02:01:37.361 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.360+0000 7fd5dba42740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56364bcde820 0x56364bcdec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:37.361 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.360+0000 7fd5dba42740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56364bcdf140 0x56364bce94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:37.361 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.360+0000 7fd5dba42740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56364bce9a00 0x56364bcebdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:37.361 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.360+0000 7fd5dba42740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56364bc44900 con 0x56364bce9a00 2024-04-26T02:01:37.361 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.360+0000 7fd5dba42740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56364bbeaef0 con 0x56364bcdf140 2024-04-26T02:01:37.361 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.360+0000 7fd5dba42740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56364bc408c0 con 0x56364bcde820 2024-04-26T02:01:37.361 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.361+0000 7fd5d2800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56364bcde820 0x56364bcdec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:37.361 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.361+0000 7fd5d1400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56364bce9a00 0x56364bcebdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:37.362 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.361+0000 7fd5d1e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56364bcdf140 0x56364bce94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:37.362 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.361+0000 7fd5d2800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56364bcde820 0x56364bcdec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:56318/0 (socket says 172.21.15.151:56318) 2024-04-26T02:01:37.362 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.361+0000 7fd5d2800640 1 -- 172.21.15.151:0/4213909578 learned_addr learned my addr 172.21.15.151:0/4213909578 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:01:37.362 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.361+0000 7fd5d1400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56364bce9a00 0x56364bcebdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:55014/0 (socket says 172.21.15.151:55014) 2024-04-26T02:01:37.362 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.361+0000 7fd5d2800640 1 -- 172.21.15.151:0/4213909578 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56364bcdf140 msgr2=0x56364bce94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:37.362 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.361+0000 7fd5d2800640 1 --2- 172.21.15.151:0/4213909578 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56364bcdf140 0x56364bce94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:37.362 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.361+0000 7fd5d2800640 1 -- 172.21.15.151:0/4213909578 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56364bce9a00 msgr2=0x56364bcebdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:37.362 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.361+0000 7fd5d2800640 1 --2- 172.21.15.151:0/4213909578 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56364bce9a00 0x56364bcebdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:37.362 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.361+0000 7fd5d2800640 1 -- 172.21.15.151:0/4213909578 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56364b957210 con 0x56364bcde820 2024-04-26T02:01:37.362 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.361+0000 7fd5caa00640 1 --2- 172.21.15.151:0/3875045062 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fd5b80a3dc0 0x7fd5b80a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:37.362 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.361+0000 7fd5caa00640 1 -- 172.21.15.151:0/3875045062 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8455+0+0 (secure 0 0 0) 0x7fd5c40d0c20 con 0x7fd5d41043a0 2024-04-26T02:01:37.362 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.361+0000 7fd5d2800640 1 --2- 172.21.15.151:0/4213909578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56364bcde820 0x56364bcdec00 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc009f50 tx=0x7fd5cc0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13d0bb6f8a4092b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:37.363 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.362+0000 7fd5d1e00640 1 --2- 172.21.15.151:0/3875045062 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fd5b80a3dc0 0x7fd5b80a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:37.363 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.362+0000 7fd5bf000640 1 -- 172.21.15.151:0/4213909578 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc004d20 con 0x56364bcde820 2024-04-26T02:01:37.363 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.362+0000 7fd5bf000640 1 -- 172.21.15.151:0/4213909578 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5cc004050 con 0x56364bcde820 2024-04-26T02:01:37.363 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.362+0000 7fd5bf000640 1 -- 172.21.15.151:0/4213909578 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc005220 con 0x56364bcde820 2024-04-26T02:01:37.368 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.362+0000 7fd5dba42740 1 -- 172.21.15.151:0/4213909578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56364bcde820 msgr2=0x56364bcdec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:37.368 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.362+0000 7fd5dba42740 1 --2- 172.21.15.151:0/4213909578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56364bcde820 0x56364bcdec00 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc009f50 tx=0x7fd5cc0118d0 comp rx=0 tx=0).stop 2024-04-26T02:01:37.368 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.362+0000 7fd5d1e00640 1 --2- 172.21.15.151:0/3875045062 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fd5b80a3dc0 0x7fd5b80a6280 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x56364bcdfdc0 tx=0x7fd5c40d46d0 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:37.368 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.362+0000 7fd5dba42740 1 -- 172.21.15.151:0/4213909578 shutdown_connections 2024-04-26T02:01:37.368 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.362+0000 7fd5dba42740 1 --2- 172.21.15.151:0/4213909578 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56364bce9a00 0x56364bcebdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:37.368 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.362+0000 7fd5dba42740 1 --2- 172.21.15.151:0/4213909578 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56364bcdf140 0x56364bce94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:37.368 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.362+0000 7fd5dba42740 1 --2- 172.21.15.151:0/4213909578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56364bcde820 0x56364bcdec00 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:37.369 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.362+0000 7fd5dba42740 1 -- 172.21.15.151:0/4213909578 >> 172.21.15.151:0/4213909578 conn(0x56364bcdad30 msgr2=0x56364bcdb110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:37.369 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.363+0000 7fd5dba42740 1 -- 172.21.15.151:0/4213909578 shutdown_connections 2024-04-26T02:01:37.369 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.363+0000 7fd5dba42740 1 -- 172.21.15.151:0/4213909578 wait complete. 2024-04-26T02:01:37.369 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.363+0000 7fd5dba42740 1 -- start start 2024-04-26T02:01:37.369 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.363+0000 7fd5dba42740 1 Processor -- start 2024-04-26T02:01:37.370 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.364+0000 7fd5dba42740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56364bcde820 0x56364bce6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:37.370 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.364+0000 7fd5dba42740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56364bcdf140 0x56364bce6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:37.370 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.364+0000 7fd5dba42740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56364bce9a00 0x56364bce73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:37.370 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.364+0000 7fd5dba42740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56364bcec630 con 0x56364bce9a00 2024-04-26T02:01:37.370 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.364+0000 7fd5dba42740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56364bbfe310 con 0x56364bcde820 2024-04-26T02:01:37.370 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.364+0000 7fd5dba42740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56364bcec7b0 con 0x56364bcdf140 2024-04-26T02:01:37.371 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.364+0000 7fd5dba42740 10 client.0 _ll_register_callbacks cb 0x56364bbcccb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:01:37.371 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.364+0000 7fd5d1400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56364bcdf140 0x56364bce6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:37.371 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.364+0000 7fd5d1400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56364bcdf140 0x56364bce6ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:56332/0 (socket says 172.21.15.151:56332) 2024-04-26T02:01:37.371 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.364+0000 7fd5d1400640 1 -- 172.21.15.151:0/635099716 learned_addr learned my addr 172.21.15.151:0/635099716 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:01:37.371 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.364+0000 7fd5d2800640 1 --2- 172.21.15.151:0/635099716 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56364bce9a00 0x56364bce73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:37.371 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.364+0000 7fd5d2800640 1 --2- 172.21.15.151:0/635099716 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56364bcde820 0x56364bce6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:37.371 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.364+0000 7fd5d1400640 1 -- 172.21.15.151:0/635099716 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56364bcde820 msgr2=0x56364bce6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:37.371 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.364+0000 7fd5d1400640 1 --2- 172.21.15.151:0/635099716 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56364bcde820 0x56364bce6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:37.371 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.364+0000 7fd5d1400640 1 -- 172.21.15.151:0/635099716 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56364bce9a00 msgr2=0x56364bce73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:37.371 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.364+0000 7fd5d1400640 1 --2- 172.21.15.151:0/635099716 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56364bce9a00 0x56364bce73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:37.371 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.364+0000 7fd5d1400640 1 -- 172.21.15.151:0/635099716 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56364ba23620 con 0x56364bcdf140 2024-04-26T02:01:37.371 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.364+0000 7fd5d2800640 1 --2- 172.21.15.151:0/635099716 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56364bcde820 0x56364bce6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:01:37.371 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.365+0000 7fd5d1400640 1 --2- 172.21.15.151:0/635099716 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56364bcdf140 0x56364bce6ea0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0000c00 tx=0x7fd5c00043b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ecddb8d2edaf2b18 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:37.372 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.365+0000 7fd5b3400640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-26T02:01:37.372 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.365+0000 7fd5b3400640 1 -- 172.21.15.151:0/635099716 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c0035040 con 0x56364bcdf140 2024-04-26T02:01:37.372 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.365+0000 7fd5dba42740 1 -- 172.21.15.151:0/635099716 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5cc004050 con 0x56364bcdf140 2024-04-26T02:01:37.372 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.365+0000 7fd5b3400640 1 -- 172.21.15.151:0/635099716 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8455+0+0 (secure 0 0 0) 0x7fd5c0005450 con 0x56364bcdf140 2024-04-26T02:01:37.372 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.365+0000 7fd5b3400640 1 -- 172.21.15.151:0/635099716 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fd5c0005650 con 0x56364bcdf140 2024-04-26T02:01:37.372 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.365+0000 7fd5dba42740 10 client.26137 fetch_fsmap learned FSMap version 35 2024-04-26T02:01:37.372 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.365+0000 7fd5dba42740 1 -- 172.21.15.151:0/635099716 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56364bcf88a0 con 0x56364bcdf140 2024-04-26T02:01:37.372 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.366+0000 7fd5b3400640 1 -- 172.21.15.151:0/635099716 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5cc004050 con 0x56364bcdf140 2024-04-26T02:01:37.372 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.366+0000 7fd5b3400640 1 -- 172.21.15.151:0/635099716 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 35) ==== 8338+0+0 (secure 0 0 0) 0x7fd5c0003ae0 con 0x56364bcdf140 2024-04-26T02:01:37.372 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.366+0000 7fd5dba42740 10 client.26137 fetch_fsmap finished waiting for FSMap version 35 2024-04-26T02:01:37.372 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.366+0000 7fd5dba42740 10 client.26137 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:01:37.372 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.366+0000 7fd5dba42740 20 client.26137 populate_metadata read hostname 'smithi151' 2024-04-26T02:01:37.372 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.366+0000 7fd5dba42740 1 --2- 172.21.15.151:0/635099716 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x56364bcf9d10 0x56364bcfc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:37.372 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.366+0000 7fd5dba42740 4 client.26137 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:01:37.373 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.366+0000 7fd5dba42740 1 -- 172.21.15.151:0/635099716 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56364bc588b0 con 0x56364bcf9d10 2024-04-26T02:01:37.373 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.366+0000 7fd5d2800640 1 --2- 172.21.15.151:0/635099716 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x56364bcf9d10 0x56364bcfc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:37.373 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.367+0000 7fd5d2800640 1 --2- 172.21.15.151:0/635099716 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x56364bcf9d10 0x56364bcfc100 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3a6882085cfb68b server_cookie=33eecd2eb6071528 in_seq=0 out_seq=0 2024-04-26T02:01:37.373 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.367+0000 7fd5b3400640 10 client.26137 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:01:37.373 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.368+0000 7fd5b3400640 1 -- 172.21.15.151:0/635099716 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56364bc588b0 con 0x56364bcf9d10 2024-04-26T02:01:37.373 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.368+0000 7fd5b3400640 10 client.26137 handle_command_reply: tid=0 2024-04-26T02:01:37.373 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.368+0000 7fd5dba42740 1 client.26137 shutdown 2024-04-26T02:01:37.373 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.368+0000 7fd5dba42740 1 -- 172.21.15.151:0/635099716 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56364bcdf140 msgr2=0x56364bce6ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:37.373 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.368+0000 7fd5dba42740 1 --2- 172.21.15.151:0/635099716 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56364bcdf140 0x56364bce6ea0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0000c00 tx=0x7fd5c00043b0 comp rx=0 tx=0).stop 2024-04-26T02:01:37.373 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.368+0000 7fd5dba42740 1 -- 172.21.15.151:0/635099716 shutdown_connections 2024-04-26T02:01:37.373 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.368+0000 7fd5dba42740 1 --2- 172.21.15.151:0/635099716 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x56364bcf9d10 0x56364bcfc100 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:37.373 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.368+0000 7fd5dba42740 1 --2- 172.21.15.151:0/635099716 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56364bce9a00 0x56364bce73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:37.373 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.368+0000 7fd5dba42740 1 --2- 172.21.15.151:0/635099716 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56364bcdf140 0x56364bce6ea0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:37.373 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.368+0000 7fd5dba42740 1 --2- 172.21.15.151:0/635099716 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56364bcde820 0x56364bce6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:37.374 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.368+0000 7fd5dba42740 1 -- 172.21.15.151:0/635099716 >> 172.21.15.151:0/635099716 conn(0x56364bcdad30 msgr2=0x56364bce0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:37.374 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.368+0000 7fd5b3400640 0 client.26137 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:01:37.374 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.368+0000 7fd5dba42740 1 -- 172.21.15.151:0/635099716 shutdown_connections 2024-04-26T02:01:37.374 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.368+0000 7fd5dba42740 1 -- 172.21.15.151:0/635099716 wait complete. 2024-04-26T02:01:37.374 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.368+0000 7fd5dba42740 20 client.26137 trim_cache size 0 max 16384 2024-04-26T02:01:37.384 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.384+0000 7fd5dba42740 1 Processor -- start 2024-04-26T02:01:37.384 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.384+0000 7fd5dba42740 1 -- start start 2024-04-26T02:01:37.384 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.385+0000 7fd5dba42740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56364bcde820 0x56364bcf54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:37.384 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.385+0000 7fd5dba42740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56364bcdf140 0x56364bce7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:37.384 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.385+0000 7fd5dba42740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56364bce9a00 0x56364bcd9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:37.384 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.385+0000 7fd5dba42740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d41b2560 con 0x56364bcde820 2024-04-26T02:01:37.384 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.385+0000 7fd5dba42740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d41afe30 con 0x56364bce9a00 2024-04-26T02:01:37.384 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.385+0000 7fd5dba42740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d41adca0 con 0x56364bcdf140 2024-04-26T02:01:37.384 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.385+0000 7fd5d2800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56364bcde820 0x56364bcf54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:37.384 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.385+0000 7fd5d2800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56364bcde820 0x56364bcf54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:55034/0 (socket says 172.21.15.151:55034) 2024-04-26T02:01:37.384 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.385+0000 7fd5d2800640 1 -- 172.21.15.151:0/2052306290 learned_addr learned my addr 172.21.15.151:0/2052306290 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:01:37.384 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.385+0000 7fd5d1400640 1 --2- 172.21.15.151:0/2052306290 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56364bcdf140 0x56364bce7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:37.386 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.385+0000 7fd5d2800640 1 -- 172.21.15.151:0/2052306290 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56364bcdf140 msgr2=0x56364bce7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:37.386 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.385+0000 7fd5d2800640 1 --2- 172.21.15.151:0/2052306290 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56364bcdf140 0x56364bce7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:37.386 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.385+0000 7fd5d2800640 1 -- 172.21.15.151:0/2052306290 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56364bce9a00 msgr2=0x56364bcd9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:37.386 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.385+0000 7fd5d2800640 1 --2- 172.21.15.151:0/2052306290 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56364bce9a00 0x56364bcd9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:37.386 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.385+0000 7fd5d2800640 1 -- 172.21.15.151:0/2052306290 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56364bcfc470 con 0x56364bcde820 2024-04-26T02:01:37.386 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.386+0000 7fd5d2800640 1 --2- 172.21.15.151:0/2052306290 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56364bcde820 0x56364bcf54f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc012fd0 tx=0x7fd5cc009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34c75860c3d9e87a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:37.386 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.386+0000 7fd5be600640 1 -- 172.21.15.151:0/2052306290 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc009a50 con 0x56364bcde820 2024-04-26T02:01:37.386 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.386+0000 7fd5be600640 1 -- 172.21.15.151:0/2052306290 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5cc004050 con 0x56364bcde820 2024-04-26T02:01:37.386 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.386+0000 7fd5be600640 1 -- 172.21.15.151:0/2052306290 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc02c1e0 con 0x56364bcde820 2024-04-26T02:01:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.386+0000 7fd5dba42740 1 -- 172.21.15.151:0/2052306290 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56364bcde820 msgr2=0x56364bcf54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.386+0000 7fd5dba42740 1 --2- 172.21.15.151:0/2052306290 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56364bcde820 0x56364bcf54f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc012fd0 tx=0x7fd5cc009500 comp rx=0 tx=0).stop 2024-04-26T02:01:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.387+0000 7fd5dba42740 1 -- 172.21.15.151:0/2052306290 shutdown_connections 2024-04-26T02:01:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.387+0000 7fd5dba42740 1 --2- 172.21.15.151:0/2052306290 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56364bce9a00 0x56364bcd9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.387+0000 7fd5dba42740 1 --2- 172.21.15.151:0/2052306290 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56364bcdf140 0x56364bce7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.387+0000 7fd5dba42740 1 --2- 172.21.15.151:0/2052306290 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56364bcde820 0x56364bcf54f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.387+0000 7fd5dba42740 1 -- 172.21.15.151:0/2052306290 >> 172.21.15.151:0/2052306290 conn(0x56364bcdad30 msgr2=0x56364bcd9600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.387+0000 7fd5dba42740 1 -- 172.21.15.151:0/2052306290 shutdown_connections 2024-04-26T02:01:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.387+0000 7fd5dba42740 1 -- 172.21.15.151:0/2052306290 wait complete. 2024-04-26T02:01:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.387+0000 7fd5dba42740 1 -- start start 2024-04-26T02:01:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.387+0000 7fd5dba42740 1 Processor -- start 2024-04-26T02:01:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.388+0000 7fd5dba42740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56364bcde820 0x56364bcf1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.388+0000 7fd5dba42740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56364bcdf140 0x56364bcef2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.388+0000 7fd5dba42740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56364bce9a00 0x56364bcef9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.388+0000 7fd5dba42740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d41ae2f0 con 0x56364bcde820 2024-04-26T02:01:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.388+0000 7fd5dba42740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d41af760 con 0x56364bce9a00 2024-04-26T02:01:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.388+0000 7fd5dba42740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d41ae9c0 con 0x56364bcdf140 2024-04-26T02:01:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.388+0000 7fd5dba42740 10 client.0 _ll_register_callbacks cb 0x7fd5c0004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:01:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.388+0000 7fd5d2800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56364bcde820 0x56364bcf1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.388+0000 7fd5d2800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56364bcde820 0x56364bcf1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:55042/0 (socket says 172.21.15.151:55042) 2024-04-26T02:01:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.388+0000 7fd5d2800640 1 -- 172.21.15.151:0/4267334892 learned_addr learned my addr 172.21.15.151:0/4267334892 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:01:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.388+0000 7fd5d1400640 1 --2- 172.21.15.151:0/4267334892 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56364bcdf140 0x56364bcef2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.388+0000 7fd5d2800640 1 -- 172.21.15.151:0/4267334892 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56364bcdf140 msgr2=0x56364bcef2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.388+0000 7fd5d2800640 1 --2- 172.21.15.151:0/4267334892 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56364bcdf140 0x56364bcef2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.388+0000 7fd5d2800640 1 -- 172.21.15.151:0/4267334892 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56364bce9a00 msgr2=0x56364bcef9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.388+0000 7fd5d2800640 1 --2- 172.21.15.151:0/4267334892 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56364bce9a00 0x56364bcef9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.388+0000 7fd5d2800640 1 -- 172.21.15.151:0/4267334892 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56364bd5f270 con 0x56364bcde820 2024-04-26T02:01:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.389+0000 7fd5d2800640 1 --2- 172.21.15.151:0/4267334892 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56364bcde820 0x56364bcf1f10 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc01b040 tx=0x7fd5cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92daef6efbd16544 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.389+0000 7fd5b3400640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-04-26T02:01:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.389+0000 7fd5b3400640 1 -- 172.21.15.151:0/4267334892 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc038020 con 0x56364bcde820 2024-04-26T02:01:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.389+0000 7fd5b3400640 1 -- 172.21.15.151:0/4267334892 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8455+0+0 (secure 0 0 0) 0x7fd5cc004200 con 0x56364bcde820 2024-04-26T02:01:37.392 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.389+0000 7fd5dba42740 1 -- 172.21.15.151:0/4267334892 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5cc004050 con 0x56364bcde820 2024-04-26T02:01:37.392 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.389+0000 7fd5b3400640 1 -- 172.21.15.151:0/4267334892 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fd5cc043050 con 0x56364bcde820 2024-04-26T02:01:37.392 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.389+0000 7fd5dba42740 10 client.16008 fetch_fsmap learned FSMap version 35 2024-04-26T02:01:37.392 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.389+0000 7fd5dba42740 1 -- 172.21.15.151:0/4267334892 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56364bd603c0 con 0x56364bcde820 2024-04-26T02:01:37.392 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.390+0000 7fd5b3400640 1 -- 172.21.15.151:0/4267334892 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5cc004050 con 0x56364bcde820 2024-04-26T02:01:37.392 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.390+0000 7fd5b3400640 1 -- 172.21.15.151:0/4267334892 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 35) ==== 8338+0+0 (secure 0 0 0) 0x7fd5cc02c660 con 0x56364bcde820 2024-04-26T02:01:37.392 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.390+0000 7fd5dba42740 10 client.16008 fetch_fsmap finished waiting for FSMap version 35 2024-04-26T02:01:37.392 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.390+0000 7fd5dba42740 10 client.16008 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:01:37.392 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.390+0000 7fd5dba42740 20 client.16008 populate_metadata read hostname 'smithi151' 2024-04-26T02:01:37.392 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.390+0000 7fd5dba42740 1 --2- 172.21.15.151:0/4267334892 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x56364bd60840 0x56364bd62c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:37.392 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.390+0000 7fd5dba42740 4 client.16008 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T02:01:37.392 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.390+0000 7fd5dba42740 1 -- 172.21.15.151:0/4267334892 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56364bc52320 con 0x56364bd60840 2024-04-26T02:01:37.392 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.390+0000 7fd5d1400640 1 --2- 172.21.15.151:0/4267334892 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x56364bd60840 0x56364bd62c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:37.393 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.391+0000 7fd5d1400640 1 --2- 172.21.15.151:0/4267334892 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x56364bd60840 0x56364bd62c30 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45f9cbe30d8cf0f server_cookie=bd5cd1dd3c3f277 in_seq=0 out_seq=0 2024-04-26T02:01:37.393 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.391+0000 7fd5b3400640 10 client.16008 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:01:37.407 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.408+0000 7fd5b3400640 1 -- 172.21.15.151:0/4267334892 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56364bc52320 con 0x56364bd60840 2024-04-26T02:01:37.408 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.408+0000 7fd5b3400640 10 client.16008 handle_command_reply: tid=0 2024-04-26T02:01:37.408 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.408+0000 7fd5dba42740 1 client.16008 shutdown 2024-04-26T02:01:37.408 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.408+0000 7fd5dba42740 1 -- 172.21.15.151:0/4267334892 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56364bcde820 msgr2=0x56364bcf1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:37.408 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.408+0000 7fd5dba42740 1 --2- 172.21.15.151:0/4267334892 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56364bcde820 0x56364bcf1f10 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc01b040 tx=0x7fd5cc009bb0 comp rx=0 tx=0).stop 2024-04-26T02:01:37.408 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.408+0000 7fd5dba42740 1 -- 172.21.15.151:0/4267334892 shutdown_connections 2024-04-26T02:01:37.408 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.408+0000 7fd5dba42740 1 --2- 172.21.15.151:0/4267334892 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x56364bd60840 0x56364bd62c30 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:37.408 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.408+0000 7fd5dba42740 1 --2- 172.21.15.151:0/4267334892 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56364bce9a00 0x56364bcef9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:37.408 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.408+0000 7fd5dba42740 1 --2- 172.21.15.151:0/4267334892 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56364bcdf140 0x56364bcef2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:37.408 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.408+0000 7fd5dba42740 1 --2- 172.21.15.151:0/4267334892 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56364bcde820 0x56364bcf1f10 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:37.408 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.408+0000 7fd5b3400640 0 client.16008 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:01:37.408 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.408+0000 7fd5dba42740 1 -- 172.21.15.151:0/4267334892 >> 172.21.15.151:0/4267334892 conn(0x56364bcdad30 msgr2=0x56364bcd9e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:37.408 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.409+0000 7fd5dba42740 1 -- 172.21.15.151:0/4267334892 shutdown_connections 2024-04-26T02:01:37.409 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.409+0000 7fd5dba42740 1 -- 172.21.15.151:0/4267334892 wait complete. 2024-04-26T02:01:37.409 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.409+0000 7fd5dba42740 20 client.16008 trim_cache size 0 max 16384 2024-04-26T02:01:37.409 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:{ 2024-04-26T02:01:37.409 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "message": "", 2024-04-26T02:01:37.409 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "return_code": 0 2024-04-26T02:01:37.409 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:} 2024-04-26T02:01:37.410 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.409+0000 7fd5d8c00640 1 -- 172.21.15.151:0/3875045062 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fd5b80a3dc0 msgr2=0x7fd5b80a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:37.410 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.409+0000 7fd5d8c00640 1 --2- 172.21.15.151:0/3875045062 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fd5b80a3dc0 0x7fd5b80a6280 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x56364bcdfdc0 tx=0x7fd5c40d46d0 comp rx=0 tx=0).stop 2024-04-26T02:01:37.410 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.410+0000 7fd5d8c00640 1 -- 172.21.15.151:0/3875045062 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5d41043a0 msgr2=0x7fd5d41a0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:37.410 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.410+0000 7fd5d8c00640 1 --2- 172.21.15.151:0/3875045062 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5d41043a0 0x7fd5d41a0620 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd5c400e9f0 tx=0x7fd5c400edf0 comp rx=0 tx=0).stop 2024-04-26T02:01:37.410 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.410+0000 7fd5d8c00640 1 -- 172.21.15.151:0/3875045062 shutdown_connections 2024-04-26T02:01:37.411 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.410+0000 7fd5d8c00640 1 --2- 172.21.15.151:0/3875045062 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fd5b80a3dc0 0x7fd5b80a6280 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:37.411 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.410+0000 7fd5d8c00640 1 --2- 172.21.15.151:0/3875045062 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd5d419a7e0 0x7fd5d419ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:37.411 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.410+0000 7fd5d8c00640 1 --2- 172.21.15.151:0/3875045062 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5d4109470 0x7fd5d41a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:37.411 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.410+0000 7fd5d8c00640 1 --2- 172.21.15.151:0/3875045062 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5d41043a0 0x7fd5d41a0620 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:37.411 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.410+0000 7fd5d8c00640 1 -- 172.21.15.151:0/3875045062 >> 172.21.15.151:0/3875045062 conn(0x7fd5d40ffc40 msgr2=0x7fd5d4101010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:37.412 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.410+0000 7fd5d8c00640 1 -- 172.21.15.151:0/3875045062 shutdown_connections 2024-04-26T02:01:37.412 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:37.410+0000 7fd5d8c00640 1 -- 172.21.15.151:0/3875045062 wait complete. 2024-04-26T02:01:38.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:01:38 smithi172 ceph-mon[32785]: pgmap v250: 97 pgs: 97 active+clean; 86 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 1.1 MiB/s wr, 317 op/s 2024-04-26T02:01:38.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:01:38 smithi165 ceph-mon[32648]: pgmap v250: 97 pgs: 97 active+clean; 86 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 1.1 MiB/s wr, 317 op/s 2024-04-26T02:01:38.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:01:38 smithi151 ceph-mon[28057]: pgmap v250: 97 pgs: 97 active+clean; 86 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 1.1 MiB/s wr, 317 op/s 2024-04-26T02:01:40.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:01:40 smithi172 ceph-mon[32785]: pgmap v251: 97 pgs: 97 active+clean; 82 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 1.1 MiB/s wr, 323 op/s 2024-04-26T02:01:40.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:01:40 smithi165 ceph-mon[32648]: pgmap v251: 97 pgs: 97 active+clean; 82 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 1.1 MiB/s wr, 323 op/s 2024-04-26T02:01:40.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:01:40 smithi151 ceph-mon[28057]: pgmap v251: 97 pgs: 97 active+clean; 82 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 1.1 MiB/s wr, 323 op/s 2024-04-26T02:01:42.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:01:42 smithi172 ceph-mon[32785]: pgmap v252: 97 pgs: 97 active+clean; 82 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 333 KiB/s wr, 108 op/s 2024-04-26T02:01:42.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:01:42 smithi165 ceph-mon[32648]: pgmap v252: 97 pgs: 97 active+clean; 82 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 333 KiB/s wr, 108 op/s 2024-04-26T02:01:42.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:01:42 smithi151 ceph-mon[28057]: pgmap v252: 97 pgs: 97 active+clean; 82 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 333 KiB/s wr, 108 op/s 2024-04-26T02:01:44.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:01:44 smithi172 ceph-mon[32785]: pgmap v253: 97 pgs: 97 active+clean; 90 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 360 KiB/s wr, 110 op/s 2024-04-26T02:01:44.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:01:44 smithi165 ceph-mon[32648]: pgmap v253: 97 pgs: 97 active+clean; 90 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 360 KiB/s wr, 110 op/s 2024-04-26T02:01:44.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:01:44 smithi151 ceph-mon[28057]: pgmap v253: 97 pgs: 97 active+clean; 90 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 360 KiB/s wr, 110 op/s 2024-04-26T02:01:46.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:01:46 smithi172 ceph-mon[32785]: pgmap v254: 97 pgs: 97 active+clean; 96 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 285 KiB/s wr, 78 op/s 2024-04-26T02:01:46.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:01:46 smithi165 ceph-mon[32648]: pgmap v254: 97 pgs: 97 active+clean; 96 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 285 KiB/s wr, 78 op/s 2024-04-26T02:01:46.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:01:46 smithi151 ceph-mon[28057]: pgmap v254: 97 pgs: 97 active+clean; 96 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 285 KiB/s wr, 78 op/s 2024-04-26T02:01:48.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:01:48 smithi172 ceph-mon[32785]: pgmap v255: 97 pgs: 97 active+clean; 96 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 58 KiB/s wr, 10 op/s 2024-04-26T02:01:48.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:01:48 smithi165 ceph-mon[32648]: pgmap v255: 97 pgs: 97 active+clean; 96 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 58 KiB/s wr, 10 op/s 2024-04-26T02:01:48.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:01:48 smithi151 ceph-mon[28057]: pgmap v255: 97 pgs: 97 active+clean; 96 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 58 KiB/s wr, 10 op/s 2024-04-26T02:01:50.551 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.551+0000 7fa8af600640 1 -- 172.21.15.151:0/2812531758 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8a8104d50 msgr2=0x7fa8a8108d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:50.551 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.551+0000 7fa8af600640 1 --2- 172.21.15.151:0/2812531758 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8a8104d50 0x7fa8a8108d90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa894009860 tx=0x7fa89402a340 comp rx=0 tx=0).stop 2024-04-26T02:01:50.551 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.551+0000 7fa8af600640 1 -- 172.21.15.151:0/2812531758 shutdown_connections 2024-04-26T02:01:50.551 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.551+0000 7fa8af600640 1 --2- 172.21.15.151:0/2812531758 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8a8109470 0x7fa8a8111040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:50.551 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.551+0000 7fa8af600640 1 --2- 172.21.15.151:0/2812531758 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8a8104d50 0x7fa8a8108d90 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:50.551 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.551+0000 7fa8af600640 1 --2- 172.21.15.151:0/2812531758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa8a81043a0 0x7fa8a8104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:50.551 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.551+0000 7fa8af600640 1 -- 172.21.15.151:0/2812531758 >> 172.21.15.151:0/2812531758 conn(0x7fa8a80ffc40 msgr2=0x7fa8a8102060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:50.551 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.551+0000 7fa8af600640 1 -- 172.21.15.151:0/2812531758 shutdown_connections 2024-04-26T02:01:50.551 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.551+0000 7fa8af600640 1 -- 172.21.15.151:0/2812531758 wait complete. 2024-04-26T02:01:50.554 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.552+0000 7fa8af600640 1 Processor -- start 2024-04-26T02:01:50.554 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.553+0000 7fa8af600640 1 -- start start 2024-04-26T02:01:50.554 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.553+0000 7fa8af600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8a81043a0 0x7fa8a81a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:50.554 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.553+0000 7fa8af600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8a8109470 0x7fa8a81a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:50.554 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.553+0000 7fa8af600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa8a819a730 0x7fa8a819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:50.554 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.553+0000 7fa8af600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a81143b0 con 0x7fa8a819a730 2024-04-26T02:01:50.554 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.553+0000 7fa8af600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a8114230 con 0x7fa8a81043a0 2024-04-26T02:01:50.554 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.553+0000 7fa8af600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a8114530 con 0x7fa8a8109470 2024-04-26T02:01:50.554 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.553+0000 7fa8ad600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa8a819a730 0x7fa8a819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:50.554 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.553+0000 7fa89fe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8a8109470 0x7fa8a81a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:50.555 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.553+0000 7fa89fe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8a8109470 0x7fa8a81a0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:49520/0 (socket says 172.21.15.151:49520) 2024-04-26T02:01:50.555 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.553+0000 7fa89fe00640 1 -- 172.21.15.151:0/2111431821 learned_addr learned my addr 172.21.15.151:0/2111431821 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:01:50.555 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.554+0000 7fa8acc00640 1 --2- 172.21.15.151:0/2111431821 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8a81043a0 0x7fa8a81a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:50.555 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.554+0000 7fa89fe00640 1 -- 172.21.15.151:0/2111431821 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8a81043a0 msgr2=0x7fa8a81a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:50.555 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.554+0000 7fa89fe00640 1 --2- 172.21.15.151:0/2111431821 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8a81043a0 0x7fa8a81a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:50.555 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.554+0000 7fa89fe00640 1 -- 172.21.15.151:0/2111431821 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa8a819a730 msgr2=0x7fa8a819abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:50.555 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.554+0000 7fa89fe00640 1 --2- 172.21.15.151:0/2111431821 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa8a819a730 0x7fa8a819abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:50.555 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.554+0000 7fa89fe00640 1 -- 172.21.15.151:0/2111431821 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8a819b330 con 0x7fa8a8109470 2024-04-26T02:01:50.555 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.554+0000 7fa89fe00640 1 --2- 172.21.15.151:0/2111431821 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8a8109470 0x7fa8a81a0b60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa894000c00 tx=0x7fa894032d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:50.555 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.555+0000 7fa8a6000640 1 -- 172.21.15.151:0/2111431821 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa894040020 con 0x7fa8a8109470 2024-04-26T02:01:50.555 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.555+0000 7fa8af600640 1 -- 172.21.15.151:0/2111431821 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8a819b5c0 con 0x7fa8a8109470 2024-04-26T02:01:50.555 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.555+0000 7fa8a6000640 1 -- 172.21.15.151:0/2111431821 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa894030d50 con 0x7fa8a8109470 2024-04-26T02:01:50.556 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.555+0000 7fa8a6000640 1 -- 172.21.15.151:0/2111431821 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa894037040 con 0x7fa8a8109470 2024-04-26T02:01:50.556 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.555+0000 7fa8af600640 1 -- 172.21.15.151:0/2111431821 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8a81a7700 con 0x7fa8a8109470 2024-04-26T02:01:50.558 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.559+0000 7fa8b23c1740 1 Processor -- start 2024-04-26T02:01:50.558 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.559+0000 7fa8b23c1740 1 -- start start 2024-04-26T02:01:50.559 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.560+0000 7fa8b23c1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56423ba33820 0x56423ba33c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:50.560 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.560+0000 7fa8b23c1740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56423ba34140 0x56423ba3e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:50.560 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.560+0000 7fa8b23c1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56423ba3ea00 0x56423ba40df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:50.560 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.560+0000 7fa8b23c1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56423b999900 con 0x56423ba33820 2024-04-26T02:01:50.560 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.560+0000 7fa8b23c1740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56423b93fef0 con 0x56423ba34140 2024-04-26T02:01:50.560 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.560+0000 7fa8b23c1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56423b9958c0 con 0x56423ba3ea00 2024-04-26T02:01:50.560 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.560+0000 7fa8a6000640 1 -- 172.21.15.151:0/2111431821 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7fa894043020 con 0x7fa8a8109470 2024-04-26T02:01:50.560 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.560+0000 7fa8ad600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56423ba33820 0x56423ba33c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:50.560 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.560+0000 7fa8ad600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56423ba33820 0x56423ba33c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:59888/0 (socket says 172.21.15.151:59888) 2024-04-26T02:01:50.560 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.560+0000 7fa8ad600640 1 -- 172.21.15.151:0/4002605679 learned_addr learned my addr 172.21.15.151:0/4002605679 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:01:50.560 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.560+0000 7fa8acc00640 1 --2- 172.21.15.151:0/4002605679 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56423ba34140 0x56423ba3e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:50.560 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.560+0000 7fa8ad600640 1 -- 172.21.15.151:0/4002605679 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56423ba3ea00 msgr2=0x56423ba40df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:50.560 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.561+0000 7fa8ad600640 1 --2- 172.21.15.151:0/4002605679 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56423ba3ea00 0x56423ba40df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:50.560 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.561+0000 7fa8ad600640 1 -- 172.21.15.151:0/4002605679 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56423ba34140 msgr2=0x56423ba3e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:50.561 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.561+0000 7fa8ad600640 1 --2- 172.21.15.151:0/4002605679 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56423ba34140 0x56423ba3e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:50.561 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.561+0000 7fa8ad600640 1 -- 172.21.15.151:0/4002605679 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56423b6ac210 con 0x56423ba33820 2024-04-26T02:01:50.561 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.561+0000 7fa8ad600640 1 --2- 172.21.15.151:0/4002605679 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56423ba33820 0x56423ba33c00 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa8a0009f50 tx=0x7fa8a00118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bebb838916a84e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:50.564 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.561+0000 7fa89dc00640 1 -- 172.21.15.151:0/4002605679 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a0004b20 con 0x56423ba33820 2024-04-26T02:01:50.564 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.561+0000 7fa89dc00640 1 -- 172.21.15.151:0/4002605679 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8a0004cc0 con 0x56423ba33820 2024-04-26T02:01:50.564 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.561+0000 7fa89dc00640 1 -- 172.21.15.151:0/4002605679 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a0005380 con 0x56423ba33820 2024-04-26T02:01:50.564 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.562+0000 7fa8a6000640 1 --2- 172.21.15.151:0/2111431821 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fa88c0a3dc0 0x7fa88c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:50.564 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.562+0000 7fa8b23c1740 1 -- 172.21.15.151:0/4002605679 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56423ba33820 msgr2=0x56423ba33c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:50.564 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.562+0000 7fa8b23c1740 1 --2- 172.21.15.151:0/4002605679 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56423ba33820 0x56423ba33c00 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa8a0009f50 tx=0x7fa8a00118d0 comp rx=0 tx=0).stop 2024-04-26T02:01:50.564 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.562+0000 7fa8a6000640 1 -- 172.21.15.151:0/2111431821 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8455+0+0 (secure 0 0 0) 0x7fa8940f02e0 con 0x7fa8a8109470 2024-04-26T02:01:50.564 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.562+0000 7fa8b23c1740 1 -- 172.21.15.151:0/4002605679 shutdown_connections 2024-04-26T02:01:50.564 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.562+0000 7fa8b23c1740 1 --2- 172.21.15.151:0/4002605679 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56423ba3ea00 0x56423ba40df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:50.565 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.562+0000 7fa8b23c1740 1 --2- 172.21.15.151:0/4002605679 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56423ba34140 0x56423ba3e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:50.565 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.562+0000 7fa8b23c1740 1 --2- 172.21.15.151:0/4002605679 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56423ba33820 0x56423ba33c00 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:50.565 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.562+0000 7fa8b23c1740 1 -- 172.21.15.151:0/4002605679 >> 172.21.15.151:0/4002605679 conn(0x56423ba2fd30 msgr2=0x56423ba30110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:50.565 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.562+0000 7fa8acc00640 1 --2- 172.21.15.151:0/2111431821 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fa88c0a3dc0 0x7fa88c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:50.565 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.562+0000 7fa8b23c1740 1 -- 172.21.15.151:0/4002605679 shutdown_connections 2024-04-26T02:01:50.565 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.562+0000 7fa8b23c1740 1 -- 172.21.15.151:0/4002605679 wait complete. 2024-04-26T02:01:50.565 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.563+0000 7fa8b23c1740 1 -- start start 2024-04-26T02:01:50.565 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.563+0000 7fa8acc00640 1 --2- 172.21.15.151:0/2111431821 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fa88c0a3dc0 0x7fa88c0a6280 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x56423ba34dc0 tx=0x7fa898014040 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:50.565 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.563+0000 7fa8b23c1740 1 Processor -- start 2024-04-26T02:01:50.565 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.563+0000 7fa8b23c1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56423ba33820 0x56423ba3ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:50.565 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.563+0000 7fa8b23c1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56423ba34140 0x56423ba3bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:50.565 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.563+0000 7fa8b23c1740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56423ba3ea00 0x56423ba3c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:50.565 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.563+0000 7fa8b23c1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56423ba41630 con 0x56423ba33820 2024-04-26T02:01:50.566 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.564+0000 7fa8b23c1740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56423b953310 con 0x56423ba3ea00 2024-04-26T02:01:50.566 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.564+0000 7fa8b23c1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56423ba417b0 con 0x56423ba34140 2024-04-26T02:01:50.566 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.564+0000 7fa8b23c1740 10 client.0 _ll_register_callbacks cb 0x56423b921cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:01:50.566 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.564+0000 7fa8acc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56423ba33820 0x56423ba3ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:50.566 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.564+0000 7fa8acc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56423ba33820 0x56423ba3ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:59898/0 (socket says 172.21.15.151:59898) 2024-04-26T02:01:50.566 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.564+0000 7fa8acc00640 1 -- 172.21.15.151:0/3030587954 learned_addr learned my addr 172.21.15.151:0/3030587954 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:01:50.566 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.564+0000 7fa8ad600640 1 --2- 172.21.15.151:0/3030587954 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56423ba3ea00 0x56423ba3c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:50.566 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.564+0000 7fa89fe00640 1 --2- 172.21.15.151:0/3030587954 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56423ba34140 0x56423ba3bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:50.566 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.564+0000 7fa8acc00640 1 -- 172.21.15.151:0/3030587954 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56423ba34140 msgr2=0x56423ba3bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:50.566 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.564+0000 7fa8acc00640 1 --2- 172.21.15.151:0/3030587954 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56423ba34140 0x56423ba3bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:50.566 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.564+0000 7fa8acc00640 1 -- 172.21.15.151:0/3030587954 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56423ba3ea00 msgr2=0x56423ba3c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:50.566 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.564+0000 7fa8acc00640 1 --2- 172.21.15.151:0/3030587954 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56423ba3ea00 0x56423ba3c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:50.567 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.564+0000 7fa8acc00640 1 -- 172.21.15.151:0/3030587954 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56423b778620 con 0x56423ba33820 2024-04-26T02:01:50.567 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.564+0000 7fa8ad600640 1 --2- 172.21.15.151:0/3030587954 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56423ba3ea00 0x56423ba3c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:01:50.567 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.565+0000 7fa8acc00640 1 --2- 172.21.15.151:0/3030587954 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56423ba33820 0x56423ba3ba10 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa898003870 tx=0x7fa898004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da8bbeebaf1b64e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:50.567 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.565+0000 7fa892000640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-04-26T02:01:50.567 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.565+0000 7fa892000640 1 -- 172.21.15.151:0/3030587954 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa898009d40 con 0x56423ba33820 2024-04-26T02:01:50.567 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.565+0000 7fa892000640 1 -- 172.21.15.151:0/3030587954 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8455+0+0 (secure 0 0 0) 0x7fa898003c70 con 0x56423ba33820 2024-04-26T02:01:50.567 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.565+0000 7fa8b23c1740 1 -- 172.21.15.151:0/3030587954 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8a0004cc0 con 0x56423ba33820 2024-04-26T02:01:50.569 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.565+0000 7fa892000640 1 -- 172.21.15.151:0/3030587954 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa89800f940 con 0x56423ba33820 2024-04-26T02:01:50.569 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.565+0000 7fa8b23c1740 10 client.16020 fetch_fsmap learned FSMap version 35 2024-04-26T02:01:50.569 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.565+0000 7fa8b23c1740 1 -- 172.21.15.151:0/3030587954 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56423ba4d980 con 0x56423ba33820 2024-04-26T02:01:50.569 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.566+0000 7fa892000640 1 -- 172.21.15.151:0/3030587954 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8a0004cc0 con 0x56423ba33820 2024-04-26T02:01:50.570 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.566+0000 7fa892000640 1 -- 172.21.15.151:0/3030587954 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 35) ==== 8338+0+0 (secure 0 0 0) 0x7fa89801b070 con 0x56423ba33820 2024-04-26T02:01:50.570 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.566+0000 7fa8b23c1740 10 client.16020 fetch_fsmap finished waiting for FSMap version 35 2024-04-26T02:01:50.570 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.566+0000 7fa8b23c1740 10 client.16020 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:01:50.570 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.566+0000 7fa8b23c1740 20 client.16020 populate_metadata read hostname 'smithi151' 2024-04-26T02:01:50.570 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.566+0000 7fa8b23c1740 1 --2- 172.21.15.151:0/3030587954 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x56423ba4ee90 0x56423ba51280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:50.570 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.566+0000 7fa8b23c1740 4 client.16020 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:01:50.570 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.566+0000 7fa8b23c1740 1 -- 172.21.15.151:0/3030587954 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56423b9ad8b0 con 0x56423ba4ee90 2024-04-26T02:01:50.570 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.567+0000 7fa89fe00640 1 --2- 172.21.15.151:0/3030587954 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x56423ba4ee90 0x56423ba51280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:50.570 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.567+0000 7fa89fe00640 1 --2- 172.21.15.151:0/3030587954 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x56423ba4ee90 0x56423ba51280 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93400724542c20b server_cookie=fb57a4be00a6acdf in_seq=0 out_seq=0 2024-04-26T02:01:50.570 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.567+0000 7fa892000640 10 client.16020 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:01:50.570 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.568+0000 7fa892000640 1 -- 172.21.15.151:0/3030587954 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56423b9ad8b0 con 0x56423ba4ee90 2024-04-26T02:01:50.570 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.568+0000 7fa892000640 10 client.16020 handle_command_reply: tid=0 2024-04-26T02:01:50.571 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.568+0000 7fa8b23c1740 1 client.16020 shutdown 2024-04-26T02:01:50.571 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.568+0000 7fa8b23c1740 1 -- 172.21.15.151:0/3030587954 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56423ba33820 msgr2=0x56423ba3ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:50.571 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.568+0000 7fa8b23c1740 1 --2- 172.21.15.151:0/3030587954 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56423ba33820 0x56423ba3ba10 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa898003870 tx=0x7fa898004730 comp rx=0 tx=0).stop 2024-04-26T02:01:50.571 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.568+0000 7fa8b23c1740 1 -- 172.21.15.151:0/3030587954 shutdown_connections 2024-04-26T02:01:50.571 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.568+0000 7fa8b23c1740 1 --2- 172.21.15.151:0/3030587954 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x56423ba4ee90 0x56423ba51280 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:50.571 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.568+0000 7fa8b23c1740 1 --2- 172.21.15.151:0/3030587954 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56423ba3ea00 0x56423ba3c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:50.571 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.568+0000 7fa8b23c1740 1 --2- 172.21.15.151:0/3030587954 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56423ba34140 0x56423ba3bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:50.571 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.568+0000 7fa892000640 0 client.16020 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:01:50.571 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.568+0000 7fa8b23c1740 1 --2- 172.21.15.151:0/3030587954 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56423ba33820 0x56423ba3ba10 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:50.572 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.568+0000 7fa8b23c1740 1 -- 172.21.15.151:0/3030587954 >> 172.21.15.151:0/3030587954 conn(0x56423ba2fd30 msgr2=0x56423ba35ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:50.572 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.568+0000 7fa8b23c1740 1 -- 172.21.15.151:0/3030587954 shutdown_connections 2024-04-26T02:01:50.572 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.568+0000 7fa8b23c1740 1 -- 172.21.15.151:0/3030587954 wait complete. 2024-04-26T02:01:50.572 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.568+0000 7fa8b23c1740 20 client.16020 trim_cache size 0 max 16384 2024-04-26T02:01:50.584 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.585+0000 7fa8b23c1740 1 Processor -- start 2024-04-26T02:01:50.584 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.585+0000 7fa8b23c1740 1 -- start start 2024-04-26T02:01:50.584 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.585+0000 7fa8b23c1740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56423ba33820 0x56423ba4a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:50.584 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.585+0000 7fa8b23c1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56423ba34140 0x56423ba56b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:50.584 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.585+0000 7fa8b23c1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56423ba3ea00 0x56423ba57090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:50.585 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.585+0000 7fa8b23c1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a81b24f0 con 0x56423ba34140 2024-04-26T02:01:50.585 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.585+0000 7fa8b23c1740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a81afdc0 con 0x56423ba33820 2024-04-26T02:01:50.585 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.585+0000 7fa8b23c1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a81adc30 con 0x56423ba3ea00 2024-04-26T02:01:50.586 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.586+0000 7fa89fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56423ba34140 0x56423ba56b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:50.586 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.586+0000 7fa8acc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56423ba33820 0x56423ba4a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:50.586 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.586+0000 7fa89fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56423ba34140 0x56423ba56b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:59900/0 (socket says 172.21.15.151:59900) 2024-04-26T02:01:50.586 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.586+0000 7fa8acc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56423ba33820 0x56423ba4a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:58094/0 (socket says 172.21.15.151:58094) 2024-04-26T02:01:50.586 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.586+0000 7fa89fe00640 1 -- 172.21.15.151:0/286300983 learned_addr learned my addr 172.21.15.151:0/286300983 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:01:50.586 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.586+0000 7fa8ad600640 1 --2- 172.21.15.151:0/286300983 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56423ba3ea00 0x56423ba57090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:50.586 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.586+0000 7fa8acc00640 1 -- 172.21.15.151:0/286300983 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56423ba3ea00 msgr2=0x56423ba57090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:50.586 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.586+0000 7fa8acc00640 1 --2- 172.21.15.151:0/286300983 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56423ba3ea00 0x56423ba57090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:50.586 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.586+0000 7fa8acc00640 1 -- 172.21.15.151:0/286300983 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56423ba34140 msgr2=0x56423ba56b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:50.586 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.586+0000 7fa8acc00640 1 --2- 172.21.15.151:0/286300983 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56423ba34140 0x56423ba56b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:50.587 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.586+0000 7fa8acc00640 1 -- 172.21.15.151:0/286300983 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56423ba30a00 con 0x56423ba33820 2024-04-26T02:01:50.587 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.587+0000 7fa8acc00640 1 --2- 172.21.15.151:0/286300983 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56423ba33820 0x56423ba4a4f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa898027e40 tx=0x7fa898012db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8346ac211c84f6bd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:50.587 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.587+0000 7fa89d200640 1 -- 172.21.15.151:0/286300983 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa898034070 con 0x56423ba33820 2024-04-26T02:01:50.587 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.587+0000 7fa89d200640 1 -- 172.21.15.151:0/286300983 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa898003c70 con 0x56423ba33820 2024-04-26T02:01:50.587 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.587+0000 7fa89d200640 1 -- 172.21.15.151:0/286300983 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa89801d570 con 0x56423ba33820 2024-04-26T02:01:50.590 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.587+0000 7fa8b23c1740 1 -- 172.21.15.151:0/286300983 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56423ba33820 msgr2=0x56423ba4a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:50.590 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.587+0000 7fa8b23c1740 1 --2- 172.21.15.151:0/286300983 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56423ba33820 0x56423ba4a4f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa898027e40 tx=0x7fa898012db0 comp rx=0 tx=0).stop 2024-04-26T02:01:50.590 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.588+0000 7fa8b23c1740 1 -- 172.21.15.151:0/286300983 shutdown_connections 2024-04-26T02:01:50.591 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.588+0000 7fa8b23c1740 1 --2- 172.21.15.151:0/286300983 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56423ba3ea00 0x56423ba57090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:50.591 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.588+0000 7fa8b23c1740 1 --2- 172.21.15.151:0/286300983 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56423ba34140 0x56423ba56b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:50.591 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.588+0000 7fa8b23c1740 1 --2- 172.21.15.151:0/286300983 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56423ba33820 0x56423ba4a4f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:50.591 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.588+0000 7fa8b23c1740 1 -- 172.21.15.151:0/286300983 >> 172.21.15.151:0/286300983 conn(0x56423ba2fd30 msgr2=0x56423ba36d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:50.591 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.588+0000 7fa8b23c1740 1 -- 172.21.15.151:0/286300983 shutdown_connections 2024-04-26T02:01:50.591 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.588+0000 7fa8b23c1740 1 -- 172.21.15.151:0/286300983 wait complete. 2024-04-26T02:01:50.591 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.588+0000 7fa8b23c1740 1 -- start start 2024-04-26T02:01:50.591 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.588+0000 7fa8b23c1740 1 Processor -- start 2024-04-26T02:01:50.591 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.589+0000 7fa8b23c1740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56423ba33820 0x56423ba46d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:50.591 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.589+0000 7fa8b23c1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56423ba34140 0x56423ba440d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:50.591 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.589+0000 7fa8b23c1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56423ba3ea00 0x56423ba44610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:50.592 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.589+0000 7fa8b23c1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a81ae280 con 0x56423ba3ea00 2024-04-26T02:01:50.592 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.589+0000 7fa8b23c1740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a81af6f0 con 0x56423ba33820 2024-04-26T02:01:50.592 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.589+0000 7fa8b23c1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a81ae950 con 0x56423ba34140 2024-04-26T02:01:50.592 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.589+0000 7fa8b23c1740 10 client.0 _ll_register_callbacks cb 0x7fa8980043e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:01:50.592 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.589+0000 7fa89fe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56423ba34140 0x56423ba440d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:50.592 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.589+0000 7fa89fe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56423ba34140 0x56423ba440d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:49566/0 (socket says 172.21.15.151:49566) 2024-04-26T02:01:50.592 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.589+0000 7fa89fe00640 1 -- 172.21.15.151:0/2084858309 learned_addr learned my addr 172.21.15.151:0/2084858309 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:01:50.592 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.589+0000 7fa8ad600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56423ba3ea00 0x56423ba44610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:50.592 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.589+0000 7fa8acc00640 1 --2- 172.21.15.151:0/2084858309 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56423ba33820 0x56423ba46d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:50.592 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.589+0000 7fa89fe00640 1 -- 172.21.15.151:0/2084858309 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56423ba33820 msgr2=0x56423ba46d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:50.592 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.589+0000 7fa89fe00640 1 --2- 172.21.15.151:0/2084858309 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56423ba33820 0x56423ba46d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:50.592 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.589+0000 7fa89fe00640 1 -- 172.21.15.151:0/2084858309 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56423ba3ea00 msgr2=0x56423ba44610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:50.592 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.589+0000 7fa89fe00640 1 --2- 172.21.15.151:0/2084858309 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56423ba3ea00 0x56423ba44610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:50.592 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.589+0000 7fa89fe00640 1 -- 172.21.15.151:0/2084858309 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56423ba44dc0 con 0x56423ba34140 2024-04-26T02:01:50.593 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.590+0000 7fa89fe00640 1 --2- 172.21.15.151:0/2084858309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56423ba34140 0x56423ba440d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa8940307c0 tx=0x7fa8940bbfb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f6296b5e5a00aa3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:50.593 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.590+0000 7fa892000640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-26T02:01:50.593 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.590+0000 7fa892000640 1 -- 172.21.15.151:0/2084858309 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8940c0760 con 0x56423ba34140 2024-04-26T02:01:50.593 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.590+0000 7fa892000640 1 -- 172.21.15.151:0/2084858309 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8455+0+0 (secure 0 0 0) 0x7fa894030450 con 0x56423ba34140 2024-04-26T02:01:50.593 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.590+0000 7fa8b23c1740 1 -- 172.21.15.151:0/2084858309 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa898003c70 con 0x56423ba34140 2024-04-26T02:01:50.593 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.590+0000 7fa892000640 1 -- 172.21.15.151:0/2084858309 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa8940c5020 con 0x56423ba34140 2024-04-26T02:01:50.593 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.590+0000 7fa8b23c1740 10 client.26158 fetch_fsmap learned FSMap version 35 2024-04-26T02:01:50.593 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.590+0000 7fa8b23c1740 1 -- 172.21.15.151:0/2084858309 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56423bab53c0 con 0x56423ba34140 2024-04-26T02:01:50.593 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.591+0000 7fa892000640 1 -- 172.21.15.151:0/2084858309 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa898003c70 con 0x56423ba34140 2024-04-26T02:01:50.593 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.591+0000 7fa892000640 1 -- 172.21.15.151:0/2084858309 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 35) ==== 8338+0+0 (secure 0 0 0) 0x7fa8940a1ce0 con 0x56423ba34140 2024-04-26T02:01:50.593 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.591+0000 7fa8b23c1740 10 client.26158 fetch_fsmap finished waiting for FSMap version 35 2024-04-26T02:01:50.594 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.591+0000 7fa8b23c1740 10 client.26158 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:01:50.594 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.591+0000 7fa8b23c1740 20 client.26158 populate_metadata read hostname 'smithi151' 2024-04-26T02:01:50.594 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.591+0000 7fa8b23c1740 1 --2- 172.21.15.151:0/2084858309 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x56423bab5b70 0x56423bab7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:50.594 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.591+0000 7fa8b23c1740 4 client.26158 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T02:01:50.594 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.591+0000 7fa8b23c1740 1 -- 172.21.15.151:0/2084858309 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56423b9a7320 con 0x56423bab5b70 2024-04-26T02:01:50.594 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.591+0000 7fa8acc00640 1 --2- 172.21.15.151:0/2084858309 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x56423bab5b70 0x56423bab7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:50.594 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.592+0000 7fa8acc00640 1 --2- 172.21.15.151:0/2084858309 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x56423bab5b70 0x56423bab7f90 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1670ed8e465c6d4 server_cookie=45a8e9fe92922d1e in_seq=0 out_seq=0 2024-04-26T02:01:50.594 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.592+0000 7fa892000640 10 client.26158 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:01:50.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:01:50 smithi165 ceph-mon[32648]: pgmap v256: 97 pgs: 97 active+clean; 96 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 60 KiB/s wr, 12 op/s 2024-04-26T02:01:50.604 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.604+0000 7fa892000640 1 -- 172.21.15.151:0/2084858309 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56423b9a7320 con 0x56423bab5b70 2024-04-26T02:01:50.604 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.604+0000 7fa892000640 10 client.26158 handle_command_reply: tid=0 2024-04-26T02:01:50.604 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.604+0000 7fa8b23c1740 1 client.26158 shutdown 2024-04-26T02:01:50.604 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.604+0000 7fa8b23c1740 1 -- 172.21.15.151:0/2084858309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56423ba34140 msgr2=0x56423ba440d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:50.604 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.604+0000 7fa8b23c1740 1 --2- 172.21.15.151:0/2084858309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56423ba34140 0x56423ba440d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa8940307c0 tx=0x7fa8940bbfb0 comp rx=0 tx=0).stop 2024-04-26T02:01:50.604 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.605+0000 7fa8b23c1740 1 -- 172.21.15.151:0/2084858309 shutdown_connections 2024-04-26T02:01:50.604 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.605+0000 7fa8b23c1740 1 --2- 172.21.15.151:0/2084858309 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x56423bab5b70 0x56423bab7f90 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:50.604 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.605+0000 7fa8b23c1740 1 --2- 172.21.15.151:0/2084858309 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56423ba3ea00 0x56423ba44610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:50.604 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.605+0000 7fa8b23c1740 1 --2- 172.21.15.151:0/2084858309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56423ba34140 0x56423ba440d0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:50.604 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.605+0000 7fa892000640 0 client.26158 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:01:50.604 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.605+0000 7fa8b23c1740 1 --2- 172.21.15.151:0/2084858309 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56423ba33820 0x56423ba46d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:50.604 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.605+0000 7fa8b23c1740 1 -- 172.21.15.151:0/2084858309 >> 172.21.15.151:0/2084858309 conn(0x56423ba2fd30 msgr2=0x56423ba2e600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:50.605 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.605+0000 7fa8b23c1740 1 -- 172.21.15.151:0/2084858309 shutdown_connections 2024-04-26T02:01:50.605 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.605+0000 7fa8b23c1740 1 -- 172.21.15.151:0/2084858309 wait complete. 2024-04-26T02:01:50.605 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.605+0000 7fa8b23c1740 20 client.26158 trim_cache size 0 max 16384 2024-04-26T02:01:50.605 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:{ 2024-04-26T02:01:50.605 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "message": "", 2024-04-26T02:01:50.605 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "return_code": 0 2024-04-26T02:01:50.605 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:} 2024-04-26T02:01:50.606 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.606+0000 7fa8af600640 1 -- 172.21.15.151:0/2111431821 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fa88c0a3dc0 msgr2=0x7fa88c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:50.606 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.606+0000 7fa8af600640 1 --2- 172.21.15.151:0/2111431821 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fa88c0a3dc0 0x7fa88c0a6280 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x56423ba34dc0 tx=0x7fa898014040 comp rx=0 tx=0).stop 2024-04-26T02:01:50.606 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.606+0000 7fa8af600640 1 -- 172.21.15.151:0/2111431821 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8a8109470 msgr2=0x7fa8a81a0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:50.606 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.606+0000 7fa8af600640 1 --2- 172.21.15.151:0/2111431821 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8a8109470 0x7fa8a81a0b60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa894000c00 tx=0x7fa894032d40 comp rx=0 tx=0).stop 2024-04-26T02:01:50.607 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.606+0000 7fa8af600640 1 -- 172.21.15.151:0/2111431821 shutdown_connections 2024-04-26T02:01:50.607 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.606+0000 7fa8af600640 1 --2- 172.21.15.151:0/2111431821 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fa88c0a3dc0 0x7fa88c0a6280 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:50.607 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.606+0000 7fa8af600640 1 --2- 172.21.15.151:0/2111431821 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa8a819a730 0x7fa8a819abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:50.607 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.606+0000 7fa8af600640 1 --2- 172.21.15.151:0/2111431821 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8a8109470 0x7fa8a81a0b60 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:50.607 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.606+0000 7fa8af600640 1 --2- 172.21.15.151:0/2111431821 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8a81043a0 0x7fa8a81a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:50.607 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.606+0000 7fa8af600640 1 -- 172.21.15.151:0/2111431821 >> 172.21.15.151:0/2111431821 conn(0x7fa8a80ffc40 msgr2=0x7fa8a8101ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:50.607 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.606+0000 7fa8af600640 1 -- 172.21.15.151:0/2111431821 shutdown_connections 2024-04-26T02:01:50.607 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:01:50.606+0000 7fa8af600640 1 -- 172.21.15.151:0/2111431821 wait complete. 2024-04-26T02:01:50.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:01:50 smithi151 ceph-mon[28057]: pgmap v256: 97 pgs: 97 active+clean; 96 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 60 KiB/s wr, 12 op/s 2024-04-26T02:01:50.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:01:50 smithi172 ceph-mon[32785]: pgmap v256: 97 pgs: 97 active+clean; 96 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 60 KiB/s wr, 12 op/s 2024-04-26T02:01:52.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:01:52 smithi165 ceph-mon[32648]: pgmap v257: 97 pgs: 97 active+clean; 96 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 47 KiB/s wr, 6 op/s 2024-04-26T02:01:52.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:01:52 smithi151 ceph-mon[28057]: pgmap v257: 97 pgs: 97 active+clean; 96 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 47 KiB/s wr, 6 op/s 2024-04-26T02:01:52.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:01:52 smithi172 ceph-mon[32785]: pgmap v257: 97 pgs: 97 active+clean; 96 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 47 KiB/s wr, 6 op/s 2024-04-26T02:01:54.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:01:54 smithi165 ceph-mon[32648]: pgmap v258: 97 pgs: 97 active+clean; 96 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 48 KiB/s wr, 9 op/s 2024-04-26T02:01:54.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:01:54 smithi151 ceph-mon[28057]: pgmap v258: 97 pgs: 97 active+clean; 96 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 48 KiB/s wr, 9 op/s 2024-04-26T02:01:54.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:01:54 smithi172 ceph-mon[32785]: pgmap v258: 97 pgs: 97 active+clean; 96 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 48 KiB/s wr, 9 op/s 2024-04-26T02:01:55.066 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T02:01:55.507 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:01:56.591 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:01:56 smithi151 ceph-mon[28057]: pgmap v259: 97 pgs: 97 active+clean; 96 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 KiB/s wr, 8 op/s 2024-04-26T02:01:56.592 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:01:56 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T02:01:56.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:01:56 smithi165 ceph-mon[32648]: pgmap v259: 97 pgs: 97 active+clean; 96 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 KiB/s wr, 8 op/s 2024-04-26T02:01:56.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:01:56 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T02:01:56.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:01:56 smithi172 ceph-mon[32785]: pgmap v259: 97 pgs: 97 active+clean; 96 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 KiB/s wr, 8 op/s 2024-04-26T02:01:56.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:01:56 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T02:01:57.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:57.882+0000 7faf97e00700 1 -- 172.21.15.151:0/2705714155 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf90103450 msgr2=0x7faf9010f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:57.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:57.882+0000 7faf97e00700 1 --2- 172.21.15.151:0/2705714155 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf90103450 0x7faf9010f8e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7faf8800b0a0 tx=0x7faf8800b3b0 comp rx=0 tx=0).stop 2024-04-26T02:01:57.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:57.882+0000 7faf97e00700 1 -- 172.21.15.151:0/2705714155 shutdown_connections 2024-04-26T02:01:57.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:57.882+0000 7faf97e00700 1 --2- 172.21.15.151:0/2705714155 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf90103450 0x7faf9010f8e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:57.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:57.882+0000 7faf97e00700 1 --2- 172.21.15.151:0/2705714155 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf90108a80 0x7faf90108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:57.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:57.882+0000 7faf97e00700 1 --2- 172.21.15.151:0/2705714155 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faf90102a90 0x7faf90102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:57.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:57.882+0000 7faf97e00700 1 -- 172.21.15.151:0/2705714155 >> 172.21.15.151:0/2705714155 conn(0x7faf90079bf0 msgr2=0x7faf9007a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:57.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:57.882+0000 7faf97e00700 1 -- 172.21.15.151:0/2705714155 shutdown_connections 2024-04-26T02:01:57.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:57.882+0000 7faf97e00700 1 -- 172.21.15.151:0/2705714155 wait complete. 2024-04-26T02:01:57.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:57.883+0000 7faf97e00700 1 Processor -- start 2024-04-26T02:01:57.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:57.884+0000 7faf97e00700 1 -- start start 2024-04-26T02:01:57.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:57.884+0000 7faf97e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf90102a90 0x7faf90199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:57.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:57.884+0000 7faf97e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf90103450 0x7faf901996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:57.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:57.884+0000 7faf97e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faf90108a80 0x7faf9019da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:57.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:57.884+0000 7faf97e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7faf90112c30 con 0x7faf90102a90 2024-04-26T02:01:57.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:57.884+0000 7faf97e00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7faf90112ab0 con 0x7faf90108a80 2024-04-26T02:01:57.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:57.884+0000 7faf97e00700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7faf90112db0 con 0x7faf90103450 2024-04-26T02:01:57.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:57.884+0000 7faf95400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf90102a90 0x7faf90199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:57.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:57.884+0000 7faf95400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf90102a90 0x7faf90199160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:59926/0 (socket says 172.21.15.151:59926) 2024-04-26T02:01:57.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:57.884+0000 7faf95400700 1 -- 172.21.15.151:0/1558884631 learned_addr learned my addr 172.21.15.151:0/1558884631 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:01:57.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:57.884+0000 7faf94a00700 1 --2- 172.21.15.151:0/1558884631 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf90103450 0x7faf901996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:57.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:57.885+0000 7faf95e00700 1 --2- 172.21.15.151:0/1558884631 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faf90108a80 0x7faf9019da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:57.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:57.885+0000 7faf95400700 1 -- 172.21.15.151:0/1558884631 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf90103450 msgr2=0x7faf901996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:57.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:57.885+0000 7faf95400700 1 --2- 172.21.15.151:0/1558884631 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf90103450 0x7faf901996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:57.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:57.885+0000 7faf95400700 1 -- 172.21.15.151:0/1558884631 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faf90108a80 msgr2=0x7faf9019da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:57.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:57.885+0000 7faf95400700 1 --2- 172.21.15.151:0/1558884631 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faf90108a80 0x7faf9019da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:57.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:57.885+0000 7faf95400700 1 -- 172.21.15.151:0/1558884631 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf9019e170 con 0x7faf90102a90 2024-04-26T02:01:57.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:57.885+0000 7faf95400700 1 --2- 172.21.15.151:0/1558884631 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf90102a90 0x7faf90199160 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7faf8400e9c0 tx=0x7faf8400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:57.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:57.886+0000 7faf8e000700 1 -- 172.21.15.151:0/1558884631 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf8400cbd0 con 0x7faf90102a90 2024-04-26T02:01:57.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:57.886+0000 7faf8e000700 1 -- 172.21.15.151:0/1558884631 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf8400ee60 con 0x7faf90102a90 2024-04-26T02:01:57.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:57.886+0000 7faf97e00700 1 -- 172.21.15.151:0/1558884631 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf9019e460 con 0x7faf90102a90 2024-04-26T02:01:57.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:57.886+0000 7faf8e000700 1 -- 172.21.15.151:0/1558884631 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf840055c0 con 0x7faf90102a90 2024-04-26T02:01:57.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:57.886+0000 7faf97e00700 1 -- 172.21.15.151:0/1558884631 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf901a5ce0 con 0x7faf90102a90 2024-04-26T02:01:57.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:57.887+0000 7faf97e00700 1 -- 172.21.15.151:0/1558884631 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf90069730 con 0x7faf90102a90 2024-04-26T02:01:57.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:57.889+0000 7faf8e000700 1 -- 172.21.15.151:0/1558884631 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7faf84004750 con 0x7faf90102a90 2024-04-26T02:01:57.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:57.890+0000 7faf8e000700 1 --2- 172.21.15.151:0/1558884631 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7faf780a3e30 0x7faf780a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:57.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:57.890+0000 7faf8e000700 1 -- 172.21.15.151:0/1558884631 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8455+0+0 (secure 0 0 0) 0x7faf840d0970 con 0x7faf90102a90 2024-04-26T02:01:57.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:57.891+0000 7faf94a00700 1 --2- 172.21.15.151:0/1558884631 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7faf780a3e30 0x7faf780a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:57.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:57.891+0000 7faf94a00700 1 --2- 172.21.15.151:0/1558884631 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7faf780a3e30 0x7faf780a62f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7faf9007a2a0 tx=0x7faf7c009260 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:57.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:57.896+0000 7faf8e000700 1 -- 172.21.15.151:0/1558884631 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faf8409cff0 con 0x7faf90102a90 2024-04-26T02:01:58.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:58.174+0000 7faf97e00700 1 -- 172.21.15.151:0/1558884631 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faf9002df80 con 0x7faf90102a90 2024-04-26T02:01:58.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:58.175+0000 7faf8e000700 1 -- 172.21.15.151:0/1558884631 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11889 (secure 0 0 0) 0x7faf8409c8f0 con 0x7faf90102a90 2024-04-26T02:01:58.174 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T02:01:58.174 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2937864097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2937864097},{"type":"v1","addr":"172.21.15.151:6835","nonce":2937864097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14373,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/2947090802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2947090802},{"type":"v1","addr":"172.21.15.151:6837","nonce":2947090802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/1356563280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1356563280},{"type":"v1","addr":"172.21.15.151:6839","nonce":1356563280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6841/1075240552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1075240552},{"type":"v1","addr":"172.21.15.151:6841","nonce":1075240552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/504675912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":504675912},{"type":"v1","addr":"172.21.15.165:6833","nonce":504675912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1207580015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1207580015},{"type":"v1","addr":"172.21.15.172:6833","nonce":1207580015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:49:40.746876+0000","modified":"2024-04-26T02:00:05.413774+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24505,"mds_2":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/3598138294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3598138294},{"type":"v1","addr":"172.21.15.165:6835","nonce":3598138294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/1847859054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":1847859054},{"type":"v1","addr":"172.21.15.165:6837","nonce":1847859054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.172:6837/55972537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":55972537},{"type":"v1","addr":"172.21.15.172:6837","nonce":55972537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.172:6839/1242302810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1242302810},{"type":"v1","addr":"172.21.15.172:6839","nonce":1242302810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/1969716254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1969716254},{"type":"v1","addr":"172.21.15.165:6839","nonce":1969716254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25954":{"gid":25954,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/3656630171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3656630171},{"type":"v1","addr":"172.21.15.172:6835","nonce":3656630171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24493,24505,24509]},"id":1}]} 2024-04-26T02:01:58.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:58.179+0000 7faf97e00700 1 -- 172.21.15.151:0/1558884631 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7faf780a3e30 msgr2=0x7faf780a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:58.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:58.179+0000 7faf97e00700 1 --2- 172.21.15.151:0/1558884631 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7faf780a3e30 0x7faf780a62f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7faf9007a2a0 tx=0x7faf7c009260 comp rx=0 tx=0).stop 2024-04-26T02:01:58.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:58.180+0000 7faf97e00700 1 -- 172.21.15.151:0/1558884631 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf90102a90 msgr2=0x7faf90199160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:58.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:58.180+0000 7faf97e00700 1 --2- 172.21.15.151:0/1558884631 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf90102a90 0x7faf90199160 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7faf8400e9c0 tx=0x7faf8400ecd0 comp rx=0 tx=0).stop 2024-04-26T02:01:58.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:58.180+0000 7faf97e00700 1 -- 172.21.15.151:0/1558884631 shutdown_connections 2024-04-26T02:01:58.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:58.180+0000 7faf97e00700 1 --2- 172.21.15.151:0/1558884631 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7faf780a3e30 0x7faf780a62f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:58.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:58.180+0000 7faf97e00700 1 --2- 172.21.15.151:0/1558884631 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faf90108a80 0x7faf9019da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:58.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:58.180+0000 7faf97e00700 1 --2- 172.21.15.151:0/1558884631 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf90102a90 0x7faf90199160 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:58.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:58.180+0000 7faf97e00700 1 --2- 172.21.15.151:0/1558884631 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf90103450 0x7faf901996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:58.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:58.180+0000 7faf97e00700 1 -- 172.21.15.151:0/1558884631 >> 172.21.15.151:0/1558884631 conn(0x7faf90079bf0 msgr2=0x7faf90101770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:58.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:58.180+0000 7faf97e00700 1 -- 172.21.15.151:0/1558884631 shutdown_connections 2024-04-26T02:01:58.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:01:58.180+0000 7faf97e00700 1 -- 172.21.15.151:0/1558884631 wait complete. 2024-04-26T02:01:58.179 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 35 2024-04-26T02:01:58.529 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:01:58 smithi151 ceph-mon[28057]: pgmap v260: 97 pgs: 97 active+clean; 96 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.3 KiB/s wr, 7 op/s 2024-04-26T02:01:58.529 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:01:58 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T02:01:58.529 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:01:58 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T02:01:58.530 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:01:58 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/1558884631' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:01:58.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:01:58 smithi172 ceph-mon[32785]: pgmap v260: 97 pgs: 97 active+clean; 96 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.3 KiB/s wr, 7 op/s 2024-04-26T02:01:58.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:01:58 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T02:01:58.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:01:58 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T02:01:58.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:01:58 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/1558884631' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:01:58.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:01:58 smithi165 ceph-mon[32648]: pgmap v260: 97 pgs: 97 active+clean; 96 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.3 KiB/s wr, 7 op/s 2024-04-26T02:01:58.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:01:58 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T02:01:58.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:01:58 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T02:01:58.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:01:58 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/1558884631' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:01:58.974 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24509 quiesce db --set-id f9d9e07c --release --await' 2024-04-26T02:01:58.975 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24509 quiesce db --set-id f9d9e07c --release --await 2024-04-26T02:01:59.414 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:01:59.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:01:59 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:01:59.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:01:59 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:01:59.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:01:59 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T02:01:59.562 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:01:59 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:01:59.563 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:01:59 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:01:59.563 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:01:59 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T02:01:59.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:01:59 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:01:59.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:01:59 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:01:59.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:01:59 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T02:02:00.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:02:00 smithi172 ceph-mon[32785]: pgmap v261: 97 pgs: 97 active+clean; 96 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.3 KiB/s wr, 9 op/s 2024-04-26T02:02:00.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:02:00 smithi165 ceph-mon[32648]: pgmap v261: 97 pgs: 97 active+clean; 96 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.3 KiB/s wr, 9 op/s 2024-04-26T02:02:00.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:02:00 smithi151 ceph-mon[28057]: pgmap v261: 97 pgs: 97 active+clean; 96 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.3 KiB/s wr, 9 op/s 2024-04-26T02:02:01.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.876+0000 7f6c90e00700 1 -- 172.21.15.151:0/1546563911 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c8c06bab0 msgr2=0x7f6c8c10f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:01.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.876+0000 7f6c90e00700 1 --2- 172.21.15.151:0/1546563911 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c8c06bab0 0x7f6c8c10f920 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6c84009f90 tx=0x7f6c8400b330 comp rx=0 tx=0).stop 2024-04-26T02:02:01.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.876+0000 7f6c90e00700 1 -- 172.21.15.151:0/1546563911 shutdown_connections 2024-04-26T02:02:01.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.876+0000 7f6c90e00700 1 --2- 172.21.15.151:0/1546563911 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c8c06bab0 0x7f6c8c10f920 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:01.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.876+0000 7f6c90e00700 1 --2- 172.21.15.151:0/1546563911 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c8c108a50 0x7f6c8c108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:01.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.876+0000 7f6c90e00700 1 --2- 172.21.15.151:0/1546563911 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c8c06b110 0x7f6c8c06b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:01.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.876+0000 7f6c90e00700 1 -- 172.21.15.151:0/1546563911 >> 172.21.15.151:0/1546563911 conn(0x7f6c8c0788f0 msgr2=0x7f6c8c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:01.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.876+0000 7f6c90e00700 1 -- 172.21.15.151:0/1546563911 shutdown_connections 2024-04-26T02:02:01.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.876+0000 7f6c90e00700 1 -- 172.21.15.151:0/1546563911 wait complete. 2024-04-26T02:02:01.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.877+0000 7f6c90e00700 1 Processor -- start 2024-04-26T02:02:01.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.878+0000 7f6c90e00700 1 -- start start 2024-04-26T02:02:01.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.878+0000 7f6c90e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c8c06b110 0x7f6c8c101df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:01.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.878+0000 7f6c90e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c8c06bab0 0x7f6c8c102330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:01.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.878+0000 7f6c90e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c8c108a50 0x7f6c8c102870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:01.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.878+0000 7f6c90e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6c8c112c50 con 0x7f6c8c108a50 2024-04-26T02:02:01.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.878+0000 7f6c90e00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6c8c112ad0 con 0x7f6c8c06bab0 2024-04-26T02:02:01.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.878+0000 7f6c90e00700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6c8c112dd0 con 0x7f6c8c06b110 2024-04-26T02:02:01.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.878+0000 7f6c8a800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c8c108a50 0x7f6c8c102870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:01.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.879+0000 7f6c89e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c8c06b110 0x7f6c8c101df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:01.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.879+0000 7f6c8a800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c8c108a50 0x7f6c8c102870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:39144/0 (socket says 172.21.15.151:39144) 2024-04-26T02:02:01.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.879+0000 7f6c89400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c8c06bab0 0x7f6c8c102330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:01.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.879+0000 7f6c89e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c8c06b110 0x7f6c8c101df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:47562/0 (socket says 172.21.15.151:47562) 2024-04-26T02:02:01.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.879+0000 7f6c8a800700 1 -- 172.21.15.151:0/1573533781 learned_addr learned my addr 172.21.15.151:0/1573533781 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:02:01.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.879+0000 7f6c89400700 1 -- 172.21.15.151:0/1573533781 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c8c06b110 msgr2=0x7f6c8c101df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:01.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.879+0000 7f6c89400700 1 --2- 172.21.15.151:0/1573533781 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c8c06b110 0x7f6c8c101df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:01.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.879+0000 7f6c89400700 1 -- 172.21.15.151:0/1573533781 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c8c108a50 msgr2=0x7f6c8c102870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:01.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.879+0000 7f6c89400700 1 --2- 172.21.15.151:0/1573533781 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c8c108a50 0x7f6c8c102870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:01.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.879+0000 7f6c89400700 1 -- 172.21.15.151:0/1573533781 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c8c1a76f0 con 0x7f6c8c06bab0 2024-04-26T02:02:01.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.879+0000 7f6c8a800700 1 --2- 172.21.15.151:0/1573533781 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c8c108a50 0x7f6c8c102870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:02:01.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.879+0000 7f6c89400700 1 --2- 172.21.15.151:0/1573533781 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c8c06bab0 0x7f6c8c102330 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6c7800e9c0 tx=0x7f6c7800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:01.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.880+0000 7f6c82a00700 1 -- 172.21.15.151:0/1573533781 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c7800c9c0 con 0x7f6c8c06bab0 2024-04-26T02:02:01.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.880+0000 7f6c82a00700 1 -- 172.21.15.151:0/1573533781 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c7800cb60 con 0x7f6c8c06bab0 2024-04-26T02:02:01.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.880+0000 7f6c90e00700 1 -- 172.21.15.151:0/1573533781 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c8c1a79e0 con 0x7f6c8c06bab0 2024-04-26T02:02:01.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.880+0000 7f6c82a00700 1 -- 172.21.15.151:0/1573533781 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c78018420 con 0x7f6c8c06bab0 2024-04-26T02:02:01.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.880+0000 7f6c90e00700 1 -- 172.21.15.151:0/1573533781 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c8c1a7f20 con 0x7f6c8c06bab0 2024-04-26T02:02:01.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.884+0000 7f6c82a00700 1 -- 172.21.15.151:0/1573533781 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f6c78010430 con 0x7f6c8c06bab0 2024-04-26T02:02:01.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.884+0000 7f6c9b784b80 1 Processor -- start 2024-04-26T02:02:01.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.885+0000 7f6c9b784b80 1 -- start start 2024-04-26T02:02:01.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.885+0000 7f6c9b784b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f5695a920 0x555f5695ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:01.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.885+0000 7f6c9b784b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f5695b240 0x555f56965bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:01.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.885+0000 7f6c9b784b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f569660f0 0x555f569684e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:01.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.885+0000 7f6c9b784b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555f568adf90 con 0x555f5695a920 2024-04-26T02:02:01.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.885+0000 7f6c9b784b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x555f56854830 con 0x555f5695b240 2024-04-26T02:02:01.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.885+0000 7f6c9b784b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555f568a9f50 con 0x555f569660f0 2024-04-26T02:02:01.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.885+0000 7f6c8a800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f5695a920 0x555f5695ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:01.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.885+0000 7f6c82a00700 1 --2- 172.21.15.151:0/1573533781 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f6c700a3db0 0x7f6c700a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:01.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.885+0000 7f6c8a800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f5695a920 0x555f5695ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:39150/0 (socket says 172.21.15.151:39150) 2024-04-26T02:02:01.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.885+0000 7f6c89e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f5695b240 0x555f56965bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:01.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.885+0000 7f6c8a800700 1 -- 172.21.15.151:0/1335963676 learned_addr learned my addr 172.21.15.151:0/1335963676 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:02:01.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.886+0000 7f6c89400700 1 --2- 172.21.15.151:0/1335963676 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f569660f0 0x555f569684e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:01.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.886+0000 7f6c82a00700 1 -- 172.21.15.151:0/1573533781 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8455+0+0 (secure 0 0 0) 0x7f6c780d0940 con 0x7f6c8c06bab0 2024-04-26T02:02:01.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.886+0000 7f6c8a800700 1 --2- 172.21.15.151:0/1573533781 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f6c700a3db0 0x7f6c700a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:01.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.886+0000 7f6c89e00700 1 -- 172.21.15.151:0/1335963676 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f569660f0 msgr2=0x555f569684e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:01.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.886+0000 7f6c89e00700 1 --2- 172.21.15.151:0/1335963676 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f569660f0 0x555f569684e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:01.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.886+0000 7f6c89e00700 1 -- 172.21.15.151:0/1335963676 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f5695a920 msgr2=0x555f5695ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:01.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.886+0000 7f6c89e00700 1 --2- 172.21.15.151:0/1335963676 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f5695a920 0x555f5695ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:01.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.886+0000 7f6c89e00700 1 -- 172.21.15.151:0/1335963676 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f5684a780 con 0x555f5695b240 2024-04-26T02:02:01.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.886+0000 7f6c8a800700 1 --2- 172.21.15.151:0/1335963676 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f5695a920 0x555f5695ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:02:01.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.887+0000 7f6c89e00700 1 --2- 172.21.15.151:0/1335963676 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f5695b240 0x555f56965bb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c009c90 tx=0x7f6c7c0058e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=993da04a54d8ddc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:01.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.887+0000 7f6c8a800700 1 --2- 172.21.15.151:0/1573533781 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f6c700a3db0 0x7f6c700a6270 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6c84009da0 tx=0x7f6c8400b330 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:01.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.887+0000 7f6c77400700 1 -- 172.21.15.151:0/1335963676 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c011c80 con 0x555f5695b240 2024-04-26T02:02:01.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.887+0000 7f6c77400700 1 -- 172.21.15.151:0/1335963676 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c7c011e20 con 0x555f5695b240 2024-04-26T02:02:01.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.887+0000 7f6c77400700 1 -- 172.21.15.151:0/1335963676 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c005280 con 0x555f5695b240 2024-04-26T02:02:01.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.888+0000 7f6c9b784b80 1 -- 172.21.15.151:0/1335963676 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f5695b240 msgr2=0x555f56965bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:01.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.888+0000 7f6c9b784b80 1 --2- 172.21.15.151:0/1335963676 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f5695b240 0x555f56965bb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c009c90 tx=0x7f6c7c0058e0 comp rx=0 tx=0).stop 2024-04-26T02:02:01.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.888+0000 7f6c9b784b80 1 -- 172.21.15.151:0/1335963676 shutdown_connections 2024-04-26T02:02:01.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.888+0000 7f6c9b784b80 1 --2- 172.21.15.151:0/1335963676 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f569660f0 0x555f569684e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:01.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.888+0000 7f6c9b784b80 1 --2- 172.21.15.151:0/1335963676 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f5695a920 0x555f5695ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:01.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.888+0000 7f6c9b784b80 1 --2- 172.21.15.151:0/1335963676 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f5695b240 0x555f56965bb0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:01.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.888+0000 7f6c9b784b80 1 -- 172.21.15.151:0/1335963676 >> 172.21.15.151:0/1335963676 conn(0x555f568896a0 msgr2=0x555f56956dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:01.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.888+0000 7f6c9b784b80 1 -- 172.21.15.151:0/1335963676 shutdown_connections 2024-04-26T02:02:01.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.888+0000 7f6c9b784b80 1 -- 172.21.15.151:0/1335963676 wait complete. 2024-04-26T02:02:01.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.888+0000 7f6c9b784b80 1 -- start start 2024-04-26T02:02:01.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.889+0000 7f6c9b784b80 1 Processor -- start 2024-04-26T02:02:01.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.889+0000 7f6c9b784b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f5695a920 0x555f56965780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:01.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.889+0000 7f6c9b784b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f5695b240 0x555f56962a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:01.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.889+0000 7f6c9b784b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f569660f0 0x555f56962fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:01.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.889+0000 7f6c9b784b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555f56968c10 con 0x555f5695a920 2024-04-26T02:02:01.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.889+0000 7f6c9b784b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x555f56867960 con 0x555f569660f0 2024-04-26T02:02:01.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.889+0000 7f6c9b784b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555f56968d90 con 0x555f5695b240 2024-04-26T02:02:01.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.889+0000 7f6c9b784b80 10 client.0 _ll_register_callbacks cb 0x555f56846820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:02:01.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.889+0000 7f6c89e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f5695a920 0x555f56965780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:01.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.889+0000 7f6c89e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f5695a920 0x555f56965780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:39156/0 (socket says 172.21.15.151:39156) 2024-04-26T02:02:01.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.890+0000 7f6c89e00700 1 -- 172.21.15.151:0/620166992 learned_addr learned my addr 172.21.15.151:0/620166992 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:02:01.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.890+0000 7f6c8a800700 1 --2- 172.21.15.151:0/620166992 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f569660f0 0x555f56962fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:01.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.890+0000 7f6c89400700 1 --2- 172.21.15.151:0/620166992 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f5695b240 0x555f56962a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:01.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.890+0000 7f6c89e00700 1 -- 172.21.15.151:0/620166992 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f5695b240 msgr2=0x555f56962a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:01.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.890+0000 7f6c89e00700 1 --2- 172.21.15.151:0/620166992 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f5695b240 0x555f56962a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:01.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.890+0000 7f6c89e00700 1 -- 172.21.15.151:0/620166992 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f569660f0 msgr2=0x555f56962fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:01.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.890+0000 7f6c89e00700 1 --2- 172.21.15.151:0/620166992 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f569660f0 0x555f56962fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:01.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.890+0000 7f6c89e00700 1 -- 172.21.15.151:0/620166992 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f567fb9d0 con 0x555f5695a920 2024-04-26T02:02:01.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.890+0000 7f6c89400700 1 --2- 172.21.15.151:0/620166992 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f5695b240 0x555f56962a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:02:01.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.890+0000 7f6c89e00700 1 --2- 172.21.15.151:0/620166992 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f5695a920 0x555f56965780 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c009770 tx=0x7f6c7c0048b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e0958ce7726be3f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:01.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.890+0000 7f6c6be00700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-04-26T02:02:01.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.891+0000 7f6c6be00700 1 -- 172.21.15.151:0/620166992 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c026020 con 0x555f5695a920 2024-04-26T02:02:01.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.891+0000 7f6c6be00700 1 -- 172.21.15.151:0/620166992 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8455+0+0 (secure 0 0 0) 0x7f6c7c004050 con 0x555f5695a920 2024-04-26T02:02:01.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.891+0000 7f6c9b784b80 1 -- 172.21.15.151:0/620166992 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c7c011e20 con 0x555f5695a920 2024-04-26T02:02:01.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.891+0000 7f6c6be00700 1 -- 172.21.15.151:0/620166992 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f6c7c031050 con 0x555f5695a920 2024-04-26T02:02:01.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.891+0000 7f6c9b784b80 10 client.16044 fetch_fsmap learned FSMap version 35 2024-04-26T02:02:01.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.891+0000 7f6c9b784b80 1 -- 172.21.15.151:0/620166992 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x555f56844bf0 con 0x555f5695a920 2024-04-26T02:02:01.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.891+0000 7f6c6be00700 1 -- 172.21.15.151:0/620166992 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c7c011e20 con 0x555f5695a920 2024-04-26T02:02:01.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.892+0000 7f6c6be00700 1 -- 172.21.15.151:0/620166992 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 35) ==== 8338+0+0 (secure 0 0 0) 0x7f6c7c018070 con 0x555f5695a920 2024-04-26T02:02:01.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.892+0000 7f6c9b784b80 10 client.16044 fetch_fsmap finished waiting for FSMap version 35 2024-04-26T02:02:01.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.892+0000 7f6c9b784b80 10 client.16044 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-26T02:02:01.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.892+0000 7f6c9b784b80 20 client.16044 populate_metadata read hostname 'smithi151' 2024-04-26T02:02:01.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.892+0000 7f6c9b784b80 1 --2- 172.21.15.151:0/620166992 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x555f569757e0 0x555f56977bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:01.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.892+0000 7f6c9b784b80 4 client.16044 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:02:01.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.892+0000 7f6c9b784b80 1 -- 172.21.15.151:0/620166992 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f568bb780 con 0x555f569757e0 2024-04-26T02:02:01.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.892+0000 7f6c89400700 1 --2- 172.21.15.151:0/620166992 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x555f569757e0 0x555f56977bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:01.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.893+0000 7f6c89400700 1 --2- 172.21.15.151:0/620166992 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x555f569757e0 0x555f56977bd0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b70379edae7d55a server_cookie=3464d57b3e64a13e in_seq=0 out_seq=0 2024-04-26T02:02:01.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.893+0000 7f6c6be00700 10 client.16044 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:02:01.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.893+0000 7f6c6be00700 1 -- 172.21.15.151:0/620166992 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555f568bb780 con 0x555f569757e0 2024-04-26T02:02:01.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.893+0000 7f6c6be00700 10 client.16044 handle_command_reply: tid=0 2024-04-26T02:02:01.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.894+0000 7f6c9b784b80 1 client.16044 shutdown 2024-04-26T02:02:01.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.894+0000 7f6c9b784b80 1 -- 172.21.15.151:0/620166992 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f5695a920 msgr2=0x555f56965780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:01.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.894+0000 7f6c9b784b80 1 --2- 172.21.15.151:0/620166992 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f5695a920 0x555f56965780 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c009770 tx=0x7f6c7c0048b0 comp rx=0 tx=0).stop 2024-04-26T02:02:01.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.894+0000 7f6c9b784b80 1 -- 172.21.15.151:0/620166992 shutdown_connections 2024-04-26T02:02:01.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.894+0000 7f6c9b784b80 1 --2- 172.21.15.151:0/620166992 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x555f569757e0 0x555f56977bd0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:01.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.894+0000 7f6c9b784b80 1 --2- 172.21.15.151:0/620166992 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f569660f0 0x555f56962fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:01.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.894+0000 7f6c9b784b80 1 --2- 172.21.15.151:0/620166992 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f5695a920 0x555f56965780 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:01.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.894+0000 7f6c9b784b80 1 --2- 172.21.15.151:0/620166992 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f5695b240 0x555f56962a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:01.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.894+0000 7f6c9b784b80 1 -- 172.21.15.151:0/620166992 >> 172.21.15.151:0/620166992 conn(0x555f568896a0 msgr2=0x555f569569c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:01.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.894+0000 7f6c6be00700 0 client.16044 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:02:01.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.894+0000 7f6c9b784b80 1 -- 172.21.15.151:0/620166992 shutdown_connections 2024-04-26T02:02:01.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.894+0000 7f6c9b784b80 1 -- 172.21.15.151:0/620166992 wait complete. 2024-04-26T02:02:01.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.894+0000 7f6c9b784b80 20 client.16044 trim_cache size 0 max 16384 2024-04-26T02:02:01.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.915+0000 7f6c9b784b80 1 Processor -- start 2024-04-26T02:02:01.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.915+0000 7f6c9b784b80 1 -- start start 2024-04-26T02:02:01.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.915+0000 7f6c9b784b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f5695a920 0x555f56971b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:01.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.915+0000 7f6c9b784b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f5695b240 0x555f5696e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:01.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.915+0000 7f6c9b784b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f569660f0 0x555f5696e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:01.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.915+0000 7f6c9b784b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6c8c1b1d50 con 0x555f5695b240 2024-04-26T02:02:01.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.915+0000 7f6c9b784b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6c8c1af5e0 con 0x555f569660f0 2024-04-26T02:02:01.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.915+0000 7f6c9b784b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6c8c1ad450 con 0x555f5695a920 2024-04-26T02:02:01.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.916+0000 7f6c89400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f5695b240 0x555f5696e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:01.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.916+0000 7f6c89400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f5695b240 0x555f5696e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:39168/0 (socket says 172.21.15.151:39168) 2024-04-26T02:02:01.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.916+0000 7f6c89400700 1 -- 172.21.15.151:0/4006026445 learned_addr learned my addr 172.21.15.151:0/4006026445 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:02:01.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.916+0000 7f6c8a800700 1 --2- 172.21.15.151:0/4006026445 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f569660f0 0x555f5696e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:01.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.916+0000 7f6c89e00700 1 --2- 172.21.15.151:0/4006026445 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f5695a920 0x555f56971b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:01.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.916+0000 7f6c89400700 1 -- 172.21.15.151:0/4006026445 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f5695a920 msgr2=0x555f56971b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:01.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.916+0000 7f6c89400700 1 --2- 172.21.15.151:0/4006026445 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f5695a920 0x555f56971b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:01.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.916+0000 7f6c89400700 1 -- 172.21.15.151:0/4006026445 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f569660f0 msgr2=0x555f5696e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:01.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.916+0000 7f6c89400700 1 --2- 172.21.15.151:0/4006026445 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f569660f0 0x555f5696e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:01.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.916+0000 7f6c89400700 1 -- 172.21.15.151:0/4006026445 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f56961010 con 0x555f5695b240 2024-04-26T02:02:01.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.916+0000 7f6c89e00700 1 --2- 172.21.15.151:0/4006026445 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f5695a920 0x555f56971b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:02:01.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.917+0000 7f6c89400700 1 --2- 172.21.15.151:0/4006026445 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f5695b240 0x555f5696e080 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f6c780ce0d0 tx=0x7f6c78087040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cd6017e18a0ebc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:01.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.917+0000 7f6c76a00700 1 -- 172.21.15.151:0/4006026445 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c780a5530 con 0x555f5695b240 2024-04-26T02:02:01.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.917+0000 7f6c76a00700 1 -- 172.21.15.151:0/4006026445 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c78010720 con 0x555f5695b240 2024-04-26T02:02:01.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.917+0000 7f6c76a00700 1 -- 172.21.15.151:0/4006026445 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c7809cd80 con 0x555f5695b240 2024-04-26T02:02:01.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.918+0000 7f6c9b784b80 1 -- 172.21.15.151:0/4006026445 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f5695b240 msgr2=0x555f5696e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:01.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.918+0000 7f6c9b784b80 1 --2- 172.21.15.151:0/4006026445 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f5695b240 0x555f5696e080 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f6c780ce0d0 tx=0x7f6c78087040 comp rx=0 tx=0).stop 2024-04-26T02:02:01.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.918+0000 7f6c9b784b80 1 -- 172.21.15.151:0/4006026445 shutdown_connections 2024-04-26T02:02:01.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.918+0000 7f6c9b784b80 1 --2- 172.21.15.151:0/4006026445 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f569660f0 0x555f5696e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:01.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.918+0000 7f6c9b784b80 1 --2- 172.21.15.151:0/4006026445 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f5695a920 0x555f56971b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:01.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.918+0000 7f6c9b784b80 1 --2- 172.21.15.151:0/4006026445 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f5695b240 0x555f5696e080 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:01.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.918+0000 7f6c9b784b80 1 -- 172.21.15.151:0/4006026445 >> 172.21.15.151:0/4006026445 conn(0x555f568896a0 msgr2=0x555f56955f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:01.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.918+0000 7f6c9b784b80 1 -- 172.21.15.151:0/4006026445 shutdown_connections 2024-04-26T02:02:01.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.918+0000 7f6c9b784b80 1 -- 172.21.15.151:0/4006026445 wait complete. 2024-04-26T02:02:01.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.918+0000 7f6c9b784b80 1 -- start start 2024-04-26T02:02:01.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.919+0000 7f6c9b784b80 1 Processor -- start 2024-04-26T02:02:01.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.919+0000 7f6c9b784b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f5695a920 0x555f5696e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:01.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.919+0000 7f6c9b784b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f5695b240 0x555f5696c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:01.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.919+0000 7f6c9b784b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f569660f0 0x555f5696cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:01.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.919+0000 7f6c9b784b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6c8c1adaa0 con 0x555f569660f0 2024-04-26T02:02:01.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.919+0000 7f6c9b784b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6c8c1aef10 con 0x555f5695b240 2024-04-26T02:02:01.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.919+0000 7f6c9b784b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6c8c1ae170 con 0x555f5695a920 2024-04-26T02:02:01.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.919+0000 7f6c9b784b80 10 client.0 _ll_register_callbacks cb 0x555f56846820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:02:01.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.919+0000 7f6c8a800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f569660f0 0x555f5696cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:01.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.920+0000 7f6c89400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f5695b240 0x555f5696c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:01.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.920+0000 7f6c8a800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f569660f0 0x555f5696cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:39182/0 (socket says 172.21.15.151:39182) 2024-04-26T02:02:01.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.920+0000 7f6c8a800700 1 -- 172.21.15.151:0/777321923 learned_addr learned my addr 172.21.15.151:0/777321923 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:02:01.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.920+0000 7f6c89e00700 1 --2- 172.21.15.151:0/777321923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f5695a920 0x555f5696e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:01.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.920+0000 7f6c8a800700 1 -- 172.21.15.151:0/777321923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f5695a920 msgr2=0x555f5696e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:01.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.920+0000 7f6c8a800700 1 --2- 172.21.15.151:0/777321923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f5695a920 0x555f5696e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:01.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.920+0000 7f6c8a800700 1 -- 172.21.15.151:0/777321923 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f5695b240 msgr2=0x555f5696c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:01.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.920+0000 7f6c8a800700 1 --2- 172.21.15.151:0/777321923 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f5695b240 0x555f5696c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:01.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.920+0000 7f6c8a800700 1 -- 172.21.15.151:0/777321923 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f5696d740 con 0x555f569660f0 2024-04-26T02:02:01.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.920+0000 7f6c89400700 1 --2- 172.21.15.151:0/777321923 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f5695b240 0x555f5696c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:02:01.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.920+0000 7f6c8a800700 1 --2- 172.21.15.151:0/777321923 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f569660f0 0x555f5696cf30 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6c84007920 tx=0x7f6c84009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c19192e6f243c12b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:01.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.920+0000 7f6c6be00700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-04-26T02:02:01.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.921+0000 7f6c6be00700 1 -- 172.21.15.151:0/777321923 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c84032070 con 0x555f569660f0 2024-04-26T02:02:01.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.921+0000 7f6c6be00700 1 -- 172.21.15.151:0/777321923 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8455+0+0 (secure 0 0 0) 0x7f6c84020860 con 0x555f569660f0 2024-04-26T02:02:01.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.921+0000 7f6c9b784b80 1 -- 172.21.15.151:0/777321923 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c78010720 con 0x555f569660f0 2024-04-26T02:02:01.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.921+0000 7f6c6be00700 1 -- 172.21.15.151:0/777321923 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f6c84027040 con 0x555f569660f0 2024-04-26T02:02:01.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.921+0000 7f6c9b784b80 10 client.16050 fetch_fsmap learned FSMap version 35 2024-04-26T02:02:01.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.921+0000 7f6c9b784b80 1 -- 172.21.15.151:0/777321923 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x555f569dccf0 con 0x555f569660f0 2024-04-26T02:02:01.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.921+0000 7f6c6be00700 1 -- 172.21.15.151:0/777321923 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c78010720 con 0x555f569660f0 2024-04-26T02:02:01.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.921+0000 7f6c6be00700 1 -- 172.21.15.151:0/777321923 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 35) ==== 8338+0+0 (secure 0 0 0) 0x7f6c84011070 con 0x555f569660f0 2024-04-26T02:02:01.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.922+0000 7f6c9b784b80 10 client.16050 fetch_fsmap finished waiting for FSMap version 35 2024-04-26T02:02:01.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.922+0000 7f6c9b784b80 10 client.16050 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-26T02:02:01.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.922+0000 7f6c9b784b80 20 client.16050 populate_metadata read hostname 'smithi151' 2024-04-26T02:02:01.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.922+0000 7f6c9b784b80 1 --2- 172.21.15.151:0/777321923 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x555f569dd430 0x555f569df850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:01.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.922+0000 7f6c9b784b80 4 client.16050 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "f9d9e07c", "release": true, "await": true}] 2024-04-26T02:02:01.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.922+0000 7f6c9b784b80 1 -- 172.21.15.151:0/777321923 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "quiesce db", "set_id": "f9d9e07c", "release": true, "await": true}) -- 0x555f568b5820 con 0x555f569dd430 2024-04-26T02:02:01.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.922+0000 7f6c89e00700 1 --2- 172.21.15.151:0/777321923 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x555f569dd430 0x555f569df850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:01.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.923+0000 7f6c89e00700 1 --2- 172.21.15.151:0/777321923 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x555f569dd430 0x555f569df850 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=debe7f14de1bf00a server_cookie=dc7eeb88af3aa344 in_seq=0 out_seq=0 2024-04-26T02:02:01.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:01.923+0000 7f6c6be00700 10 client.16050 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:02:02.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:02.107+0000 7f6c6be00700 1 -- 172.21.15.151:0/777321923 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x555f568b5820 con 0x555f569dd430 2024-04-26T02:02:02.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:02.107+0000 7f6c6be00700 10 client.16050 handle_command_reply: tid=0 2024-04-26T02:02:02.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:02.107+0000 7f6c9b784b80 1 client.16050 shutdown 2024-04-26T02:02:02.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:02.108+0000 7f6c9b784b80 1 -- 172.21.15.151:0/777321923 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f569660f0 msgr2=0x555f5696cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:02.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:02.108+0000 7f6c9b784b80 1 --2- 172.21.15.151:0/777321923 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f569660f0 0x555f5696cf30 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6c84007920 tx=0x7f6c84009920 comp rx=0 tx=0).stop 2024-04-26T02:02:02.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:02.108+0000 7f6c9b784b80 1 -- 172.21.15.151:0/777321923 shutdown_connections 2024-04-26T02:02:02.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:02.108+0000 7f6c9b784b80 1 --2- 172.21.15.151:0/777321923 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f569660f0 0x555f5696cf30 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:02.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:02.108+0000 7f6c9b784b80 1 --2- 172.21.15.151:0/777321923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f5695a920 0x555f5696e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:02.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:02.108+0000 7f6c9b784b80 1 --2- 172.21.15.151:0/777321923 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x555f569dd430 0x555f569df850 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:02.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:02.108+0000 7f6c9b784b80 1 --2- 172.21.15.151:0/777321923 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f5695b240 0x555f5696c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:02.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:02.108+0000 7f6c9b784b80 1 -- 172.21.15.151:0/777321923 >> 172.21.15.151:0/777321923 conn(0x555f568896a0 msgr2=0x555f56956790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:02.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:02.108+0000 7f6c6be00700 0 client.16050 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:02:02.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:02.109+0000 7f6c9b784b80 1 -- 172.21.15.151:0/777321923 shutdown_connections 2024-04-26T02:02:02.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:02.110+0000 7f6c9b784b80 1 -- 172.21.15.151:0/777321923 wait complete. 2024-04-26T02:02:02.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:02.111+0000 7f6c9b784b80 20 client.16050 trim_cache size 0 max 16384 2024-04-26T02:02:02.111 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-26T02:02:02.111 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 35, 2024-04-26T02:02:02.111 INFO:teuthology.orchestra.run.smithi151.stdout: "leader": 24509, 2024-04-26T02:02:02.111 INFO:teuthology.orchestra.run.smithi151.stdout: "set_version": 24, 2024-04-26T02:02:02.111 INFO:teuthology.orchestra.run.smithi151.stdout: "sets": { 2024-04-26T02:02:02.111 INFO:teuthology.orchestra.run.smithi151.stdout: "f9d9e07c": { 2024-04-26T02:02:02.111 INFO:teuthology.orchestra.run.smithi151.stdout: "version": 24, 2024-04-26T02:02:02.111 INFO:teuthology.orchestra.run.smithi151.stdout: "age_ref": 0.0, 2024-04-26T02:02:02.111 INFO:teuthology.orchestra.run.smithi151.stdout: "state": { 2024-04-26T02:02:02.111 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "RELEASED", 2024-04-26T02:02:02.111 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 0.0 2024-04-26T02:02:02.111 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:02:02.112 INFO:teuthology.orchestra.run.smithi151.stdout: "timeout": 60.0, 2024-04-26T02:02:02.112 INFO:teuthology.orchestra.run.smithi151.stdout: "expiration": 82.9, 2024-04-26T02:02:02.112 INFO:teuthology.orchestra.run.smithi151.stdout: "members": { 2024-04-26T02:02:02.112 INFO:teuthology.orchestra.run.smithi151.stdout: "file:/": { 2024-04-26T02:02:02.112 INFO:teuthology.orchestra.run.smithi151.stdout: "excluded": false, 2024-04-26T02:02:02.112 INFO:teuthology.orchestra.run.smithi151.stdout: "state": { 2024-04-26T02:02:02.112 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "RELEASED", 2024-04-26T02:02:02.112 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 0.0 2024-04-26T02:02:02.112 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:02:02.112 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:02:02.112 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:02:02.112 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:02:02.112 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:02:02.112 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-26T02:02:02.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:02.113+0000 7f6c90e00700 1 -- 172.21.15.151:0/1573533781 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f6c700a3db0 msgr2=0x7f6c700a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:02.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:02.113+0000 7f6c90e00700 1 --2- 172.21.15.151:0/1573533781 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f6c700a3db0 0x7f6c700a6270 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6c84009da0 tx=0x7f6c8400b330 comp rx=0 tx=0).stop 2024-04-26T02:02:02.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:02.113+0000 7f6c90e00700 1 -- 172.21.15.151:0/1573533781 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c8c06bab0 msgr2=0x7f6c8c102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:02.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:02.113+0000 7f6c90e00700 1 --2- 172.21.15.151:0/1573533781 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c8c06bab0 0x7f6c8c102330 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6c7800e9c0 tx=0x7f6c7800ecd0 comp rx=0 tx=0).stop 2024-04-26T02:02:02.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:02.114+0000 7f6c90e00700 1 -- 172.21.15.151:0/1573533781 shutdown_connections 2024-04-26T02:02:02.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:02.114+0000 7f6c90e00700 1 --2- 172.21.15.151:0/1573533781 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f6c700a3db0 0x7f6c700a6270 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:02.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:02.114+0000 7f6c90e00700 1 --2- 172.21.15.151:0/1573533781 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c8c108a50 0x7f6c8c102870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:02.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:02.114+0000 7f6c90e00700 1 --2- 172.21.15.151:0/1573533781 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c8c06b110 0x7f6c8c101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:02.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:02.114+0000 7f6c90e00700 1 --2- 172.21.15.151:0/1573533781 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c8c06bab0 0x7f6c8c102330 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:02.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:02.114+0000 7f6c90e00700 1 -- 172.21.15.151:0/1573533781 >> 172.21.15.151:0/1573533781 conn(0x7f6c8c0788f0 msgr2=0x7f6c8c0ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:02.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:02.114+0000 7f6c90e00700 1 -- 172.21.15.151:0/1573533781 shutdown_connections 2024-04-26T02:02:02.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:02.114+0000 7f6c90e00700 1 -- 172.21.15.151:0/1573533781 wait complete. 2024-04-26T02:02:02.709 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:02:02 smithi151 ceph-mon[28057]: pgmap v262: 97 pgs: 97 active+clean; 96 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 7.4 KiB/s wr, 7 op/s 2024-04-26T02:02:02.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:02:02 smithi172 ceph-mon[32785]: pgmap v262: 97 pgs: 97 active+clean; 96 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 7.4 KiB/s wr, 7 op/s 2024-04-26T02:02:02.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:02:02 smithi165 ceph-mon[32648]: pgmap v262: 97 pgs: 97 active+clean; 96 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 7.4 KiB/s wr, 7 op/s 2024-04-26T02:02:03.259 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'f9d9e07c', seconds elapsed: 40.5 2024-04-26T02:02:03.782 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.784+0000 7f115e800640 1 -- 172.21.15.151:0/2038604728 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1158109470 msgr2=0x7f1158111040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:03.783 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.784+0000 7f115e800640 1 --2- 172.21.15.151:0/2038604728 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1158109470 0x7f1158111040 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1150009ca0 tx=0x7f115002a8c0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.785 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.787+0000 7f115e800640 1 -- 172.21.15.151:0/2038604728 shutdown_connections 2024-04-26T02:02:03.785 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.787+0000 7f115e800640 1 --2- 172.21.15.151:0/2038604728 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1158109470 0x7f1158111040 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.785 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.787+0000 7f115e800640 1 --2- 172.21.15.151:0/2038604728 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1158104d50 0x7f1158108d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.785 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.787+0000 7f115e800640 1 --2- 172.21.15.151:0/2038604728 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11581043a0 0x7f1158104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.785 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.787+0000 7f115e800640 1 -- 172.21.15.151:0/2038604728 >> 172.21.15.151:0/2038604728 conn(0x7f11580ffc40 msgr2=0x7f1158102060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:03.786 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.788+0000 7f115e800640 1 -- 172.21.15.151:0/2038604728 shutdown_connections 2024-04-26T02:02:03.786 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.789+0000 7f115e800640 1 -- 172.21.15.151:0/2038604728 wait complete. 2024-04-26T02:02:03.788 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.790+0000 7f115e800640 1 Processor -- start 2024-04-26T02:02:03.788 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.790+0000 7f115e800640 1 -- start start 2024-04-26T02:02:03.788 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.790+0000 7f115e800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11581043a0 0x7f11581a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:03.788 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.790+0000 7f115e800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1158109470 0x7f11581a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:03.788 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.791+0000 7f115e800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f115819a6d0 0x7f115819ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:03.788 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.791+0000 7f115e800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1158114430 con 0x7f115819a6d0 2024-04-26T02:02:03.789 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.791+0000 7f115e800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f11581142b0 con 0x7f11581043a0 2024-04-26T02:02:03.789 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.791+0000 7f115e800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f11581145b0 con 0x7f1158109470 2024-04-26T02:02:03.790 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.791+0000 7f1157400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11581043a0 0x7f11581a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:03.790 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.791+0000 7f1157400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11581043a0 0x7f11581a05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:40678/0 (socket says 172.21.15.151:40678) 2024-04-26T02:02:03.790 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.791+0000 7f1157400640 1 -- 172.21.15.151:0/3758705339 learned_addr learned my addr 172.21.15.151:0/3758705339 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:02:03.790 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.791+0000 7f1156a00640 1 --2- 172.21.15.151:0/3758705339 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1158109470 0x7f11581a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:03.790 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.791+0000 7f1157e00640 1 --2- 172.21.15.151:0/3758705339 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f115819a6d0 0x7f115819ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:03.790 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.791+0000 7f1157400640 1 -- 172.21.15.151:0/3758705339 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1158109470 msgr2=0x7f11581a0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:03.791 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.791+0000 7f1157400640 1 --2- 172.21.15.151:0/3758705339 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1158109470 0x7f11581a0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.791 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.791+0000 7f1157400640 1 -- 172.21.15.151:0/3758705339 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f115819a6d0 msgr2=0x7f115819ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:03.791 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.791+0000 7f1157400640 1 --2- 172.21.15.151:0/3758705339 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f115819a6d0 0x7f115819ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.791 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.792+0000 7f1157400640 1 -- 172.21.15.151:0/3758705339 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f115819b2d0 con 0x7f11581043a0 2024-04-26T02:02:03.791 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.792+0000 7f1157400640 1 --2- 172.21.15.151:0/3758705339 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11581043a0 0x7f11581a05c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f114800e9f0 tx=0x7f114800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:03.792 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.792+0000 7f114fe00640 1 -- 172.21.15.151:0/3758705339 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114800cd80 con 0x7f11581043a0 2024-04-26T02:02:03.792 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.792+0000 7f115e800640 1 -- 172.21.15.151:0/3758705339 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11581a7360 con 0x7f11581043a0 2024-04-26T02:02:03.793 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.792+0000 7f114fe00640 1 -- 172.21.15.151:0/3758705339 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11480108e0 con 0x7f11581043a0 2024-04-26T02:02:03.793 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.792+0000 7f114fe00640 1 -- 172.21.15.151:0/3758705339 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1148018980 con 0x7f11581043a0 2024-04-26T02:02:03.793 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.792+0000 7f115e800640 1 -- 172.21.15.151:0/3758705339 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11581a77c0 con 0x7f11581043a0 2024-04-26T02:02:03.793 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.796+0000 7f114fe00640 1 -- 172.21.15.151:0/3758705339 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f1148018b20 con 0x7f11581043a0 2024-04-26T02:02:03.794 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.796+0000 7f116158a740 1 Processor -- start 2024-04-26T02:02:03.794 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.796+0000 7f116158a740 1 -- start start 2024-04-26T02:02:03.797 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.797+0000 7f116158a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4e531820 0x557b4e531c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:03.797 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.797+0000 7f116158a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b4e532140 0x557b4e53c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:03.797 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.797+0000 7f116158a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b4e53ca00 0x557b4e53edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:03.797 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.797+0000 7f116158a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557b4e497900 con 0x557b4e53ca00 2024-04-26T02:02:03.797 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.797+0000 7f116158a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x557b4e43def0 con 0x557b4e531820 2024-04-26T02:02:03.797 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.797+0000 7f116158a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557b4e4938c0 con 0x557b4e532140 2024-04-26T02:02:03.797 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.797+0000 7f1156a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b4e53ca00 0x557b4e53edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:03.797 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.797+0000 7f1157e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4e531820 0x557b4e531c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:03.797 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.797+0000 7f1157400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b4e532140 0x557b4e53c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:03.798 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.797+0000 7f1156a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b4e53ca00 0x557b4e53edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:39226/0 (socket says 172.21.15.151:39226) 2024-04-26T02:02:03.798 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.797+0000 7f1156a00640 1 -- 172.21.15.151:0/3407072171 learned_addr learned my addr 172.21.15.151:0/3407072171 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:02:03.798 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.798+0000 7f1157e00640 1 -- 172.21.15.151:0/3407072171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b4e532140 msgr2=0x557b4e53c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:03.798 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.798+0000 7f1157e00640 1 --2- 172.21.15.151:0/3407072171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b4e532140 0x557b4e53c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.798 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.798+0000 7f1157e00640 1 -- 172.21.15.151:0/3407072171 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b4e53ca00 msgr2=0x557b4e53edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:03.798 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.798+0000 7f1157e00640 1 --2- 172.21.15.151:0/3407072171 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b4e53ca00 0x557b4e53edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.798 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.798+0000 7f1157e00640 1 -- 172.21.15.151:0/3407072171 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b4e1aa210 con 0x557b4e531820 2024-04-26T02:02:03.798 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.798+0000 7f1156a00640 1 --2- 172.21.15.151:0/3407072171 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b4e53ca00 0x557b4e53edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:02:03.798 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.798+0000 7f114fe00640 1 --2- 172.21.15.151:0/3758705339 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f113c0a3e40 0x7f113c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:03.799 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.798+0000 7f1157e00640 1 --2- 172.21.15.151:0/3407072171 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4e531820 0x557b4e531c00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f115002ade0 tx=0x7f1150009870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5786ab6db6a8382d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:03.799 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.798+0000 7f114fe00640 1 -- 172.21.15.151:0/3758705339 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8499+0+0 (secure 0 0 0) 0x7f11480d0b40 con 0x7f11581043a0 2024-04-26T02:02:03.799 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.798+0000 7f1157400640 1 --2- 172.21.15.151:0/3758705339 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f113c0a3e40 0x7f113c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:03.799 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.798+0000 7f114cc00640 1 -- 172.21.15.151:0/3407072171 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1150031cf0 con 0x557b4e531820 2024-04-26T02:02:03.799 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.799+0000 7f114cc00640 1 -- 172.21.15.151:0/3407072171 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1150036040 con 0x557b4e531820 2024-04-26T02:02:03.799 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.799+0000 7f114cc00640 1 -- 172.21.15.151:0/3407072171 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11500042a0 con 0x557b4e531820 2024-04-26T02:02:03.805 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.799+0000 7f116158a740 1 -- 172.21.15.151:0/3407072171 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4e531820 msgr2=0x557b4e531c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:03.805 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.799+0000 7f116158a740 1 --2- 172.21.15.151:0/3407072171 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4e531820 0x557b4e531c00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f115002ade0 tx=0x7f1150009870 comp rx=0 tx=0).stop 2024-04-26T02:02:03.805 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.799+0000 7f1157400640 1 --2- 172.21.15.151:0/3758705339 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f113c0a3e40 0x7f113c0a6300 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f11480ce350 tx=0x7f11480d4690 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:03.805 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.800+0000 7f116158a740 1 -- 172.21.15.151:0/3407072171 shutdown_connections 2024-04-26T02:02:03.805 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.800+0000 7f116158a740 1 --2- 172.21.15.151:0/3407072171 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b4e53ca00 0x557b4e53edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.805 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.800+0000 7f116158a740 1 --2- 172.21.15.151:0/3407072171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b4e532140 0x557b4e53c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.806 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.800+0000 7f116158a740 1 --2- 172.21.15.151:0/3407072171 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4e531820 0x557b4e531c00 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.806 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.800+0000 7f116158a740 1 -- 172.21.15.151:0/3407072171 >> 172.21.15.151:0/3407072171 conn(0x557b4e52dd30 msgr2=0x557b4e52e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:03.806 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.800+0000 7f116158a740 1 -- 172.21.15.151:0/3407072171 shutdown_connections 2024-04-26T02:02:03.806 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.800+0000 7f116158a740 1 -- 172.21.15.151:0/3407072171 wait complete. 2024-04-26T02:02:03.806 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.800+0000 7f116158a740 1 -- start start 2024-04-26T02:02:03.806 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.800+0000 7f116158a740 1 Processor -- start 2024-04-26T02:02:03.806 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.801+0000 7f116158a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4e531820 0x557b4e5399b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:03.806 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.801+0000 7f116158a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b4e532140 0x557b4e539ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:03.806 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.801+0000 7f116158a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b4e53ca00 0x557b4e53a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:03.807 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.801+0000 7f116158a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557b4e53f5d0 con 0x557b4e532140 2024-04-26T02:02:03.807 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.801+0000 7f116158a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x557b4e451310 con 0x557b4e531820 2024-04-26T02:02:03.807 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.801+0000 7f116158a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557b4e53f750 con 0x557b4e53ca00 2024-04-26T02:02:03.807 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.801+0000 7f116158a740 10 client.0 _ll_register_callbacks cb 0x557b4e41fcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:02:03.807 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.801+0000 7f1156a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b4e532140 0x557b4e539ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:03.807 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.801+0000 7f1157e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4e531820 0x557b4e5399b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:03.807 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.801+0000 7f1156a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b4e532140 0x557b4e539ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:39230/0 (socket says 172.21.15.151:39230) 2024-04-26T02:02:03.807 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.801+0000 7f1156a00640 1 -- 172.21.15.151:0/304979911 learned_addr learned my addr 172.21.15.151:0/304979911 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:02:03.807 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.801+0000 7f1157e00640 1 --2- 172.21.15.151:0/304979911 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b4e53ca00 0x557b4e53a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:03.807 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.802+0000 7f1157e00640 1 -- 172.21.15.151:0/304979911 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b4e53ca00 msgr2=0x557b4e53a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:03.808 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.802+0000 7f1157e00640 1 --2- 172.21.15.151:0/304979911 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b4e53ca00 0x557b4e53a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.808 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.802+0000 7f1157e00640 1 -- 172.21.15.151:0/304979911 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b4e532140 msgr2=0x557b4e539ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:03.808 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.802+0000 7f1157e00640 1 --2- 172.21.15.151:0/304979911 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b4e532140 0x557b4e539ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.808 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.802+0000 7f1157e00640 1 -- 172.21.15.151:0/304979911 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b4e276620 con 0x557b4e531820 2024-04-26T02:02:03.808 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.802+0000 7f1156a00640 1 --2- 172.21.15.151:0/304979911 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b4e532140 0x557b4e539ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:02:03.808 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.802+0000 7f1157e00640 1 --2- 172.21.15.151:0/304979911 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4e531820 0x557b4e5399b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1150009620 tx=0x7f115003e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=baf14b990371f22c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:03.808 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.802+0000 7f1141600640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T02:02:03.808 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.802+0000 7f1141600640 1 -- 172.21.15.151:0/304979911 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1150059060 con 0x557b4e531820 2024-04-26T02:02:03.808 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.802+0000 7f1141600640 1 -- 172.21.15.151:0/304979911 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8499+0+0 (secure 0 0 0) 0x7f1150031650 con 0x557b4e531820 2024-04-26T02:02:03.808 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.802+0000 7f116158a740 1 -- 172.21.15.151:0/304979911 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1150036040 con 0x557b4e531820 2024-04-26T02:02:03.808 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.803+0000 7f1141600640 1 -- 172.21.15.151:0/304979911 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f1150031850 con 0x557b4e531820 2024-04-26T02:02:03.809 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.803+0000 7f116158a740 10 client.26159 fetch_fsmap learned FSMap version 35 2024-04-26T02:02:03.809 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.803+0000 7f116158a740 1 -- 172.21.15.151:0/304979911 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x557b4e53f5d0 con 0x557b4e531820 2024-04-26T02:02:03.809 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.803+0000 7f1141600640 1 -- 172.21.15.151:0/304979911 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1150036040 con 0x557b4e531820 2024-04-26T02:02:03.809 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.803+0000 7f1141600640 1 -- 172.21.15.151:0/304979911 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 35) ==== 8338+0+0 (secure 0 0 0) 0x7f115002f610 con 0x557b4e531820 2024-04-26T02:02:03.809 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.803+0000 7f116158a740 10 client.26159 fetch_fsmap finished waiting for FSMap version 35 2024-04-26T02:02:03.809 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.803+0000 7f116158a740 10 client.26159 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:02:03.809 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.803+0000 7f116158a740 20 client.26159 populate_metadata read hostname 'smithi151' 2024-04-26T02:02:03.809 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.803+0000 7f116158a740 1 --2- 172.21.15.151:0/304979911 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x557b4e54c9d0 0x557b4e54edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:03.809 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.803+0000 7f116158a740 4 client.26159 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:02:03.809 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.803+0000 7f116158a740 1 -- 172.21.15.151:0/304979911 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b4e4ab8b0 con 0x557b4e54c9d0 2024-04-26T02:02:03.809 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.803+0000 7f1156a00640 1 --2- 172.21.15.151:0/304979911 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x557b4e54c9d0 0x557b4e54edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:03.809 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.804+0000 7f1156a00640 1 --2- 172.21.15.151:0/304979911 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x557b4e54c9d0 0x557b4e54edc0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c65d60629f539fef server_cookie=69e509ddd042e764 in_seq=0 out_seq=0 2024-04-26T02:02:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.804+0000 7f1141600640 10 client.26159 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:02:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.805+0000 7f1141600640 1 -- 172.21.15.151:0/304979911 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557b4e4ab8b0 con 0x557b4e54c9d0 2024-04-26T02:02:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.805+0000 7f1141600640 10 client.26159 handle_command_reply: tid=0 2024-04-26T02:02:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.805+0000 7f116158a740 1 client.26159 shutdown 2024-04-26T02:02:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.805+0000 7f116158a740 1 -- 172.21.15.151:0/304979911 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4e531820 msgr2=0x557b4e5399b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.805+0000 7f116158a740 1 --2- 172.21.15.151:0/304979911 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4e531820 0x557b4e5399b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1150009620 tx=0x7f115003e040 comp rx=0 tx=0).stop 2024-04-26T02:02:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.805+0000 7f116158a740 1 -- 172.21.15.151:0/304979911 shutdown_connections 2024-04-26T02:02:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.805+0000 7f116158a740 1 --2- 172.21.15.151:0/304979911 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x557b4e54c9d0 0x557b4e54edc0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.805+0000 7f116158a740 1 --2- 172.21.15.151:0/304979911 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b4e53ca00 0x557b4e53a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.805+0000 7f116158a740 1 --2- 172.21.15.151:0/304979911 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b4e532140 0x557b4e539ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.805+0000 7f1141600640 0 client.26159 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:02:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.805+0000 7f116158a740 1 --2- 172.21.15.151:0/304979911 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4e531820 0x557b4e5399b0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.805+0000 7f116158a740 1 -- 172.21.15.151:0/304979911 >> 172.21.15.151:0/304979911 conn(0x557b4e52dd30 msgr2=0x557b4e533a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.805+0000 7f116158a740 1 -- 172.21.15.151:0/304979911 shutdown_connections 2024-04-26T02:02:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.805+0000 7f116158a740 1 -- 172.21.15.151:0/304979911 wait complete. 2024-04-26T02:02:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.805+0000 7f116158a740 20 client.26159 trim_cache size 0 max 16384 2024-04-26T02:02:03.828 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.829+0000 7f116158a740 1 Processor -- start 2024-04-26T02:02:03.828 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.830+0000 7f116158a740 1 -- start start 2024-04-26T02:02:03.828 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.830+0000 7f116158a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b4e531820 0x557b4e548490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:03.828 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.830+0000 7f116158a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b4e532140 0x557b4e52e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:03.828 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.830+0000 7f116158a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4e53ca00 0x557b4e53a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:03.828 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.830+0000 7f116158a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f11581b25b0 con 0x557b4e531820 2024-04-26T02:02:03.828 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.830+0000 7f116158a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f11581afe80 con 0x557b4e53ca00 2024-04-26T02:02:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.830+0000 7f116158a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f11581adcf0 con 0x557b4e532140 2024-04-26T02:02:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.830+0000 7f1156a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b4e532140 0x557b4e52e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.830+0000 7f1156a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b4e532140 0x557b4e52e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:47658/0 (socket says 172.21.15.151:47658) 2024-04-26T02:02:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.830+0000 7f1157e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b4e531820 0x557b4e548490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.830+0000 7f1156a00640 1 -- 172.21.15.151:0/2916528339 learned_addr learned my addr 172.21.15.151:0/2916528339 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:02:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.831+0000 7f1157e00640 1 --2- 172.21.15.151:0/2916528339 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4e53ca00 0x557b4e53a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.831+0000 7f1156a00640 1 -- 172.21.15.151:0/2916528339 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4e53ca00 msgr2=0x557b4e53a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.831+0000 7f1156a00640 1 --2- 172.21.15.151:0/2916528339 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4e53ca00 0x557b4e53a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.831+0000 7f1156a00640 1 -- 172.21.15.151:0/2916528339 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b4e531820 msgr2=0x557b4e548490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.831+0000 7f1156a00640 1 --2- 172.21.15.151:0/2916528339 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b4e531820 0x557b4e548490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.831+0000 7f1156a00640 1 -- 172.21.15.151:0/2916528339 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b4e538960 con 0x557b4e532140 2024-04-26T02:02:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.831+0000 7f1157e00640 1 --2- 172.21.15.151:0/2916528339 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b4e531820 0x557b4e548490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:02:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.831+0000 7f1156a00640 1 --2- 172.21.15.151:0/2916528339 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b4e532140 0x557b4e52e110 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1144005d00 tx=0x7f1144005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ce7805c919c7a40 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:03.831 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.832+0000 7f1143e00640 1 -- 172.21.15.151:0/2916528339 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1144005020 con 0x557b4e532140 2024-04-26T02:02:03.831 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.832+0000 7f1143e00640 1 -- 172.21.15.151:0/2916528339 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11440047a0 con 0x557b4e532140 2024-04-26T02:02:03.831 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.832+0000 7f1143e00640 1 -- 172.21.15.151:0/2916528339 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11440223d0 con 0x557b4e532140 2024-04-26T02:02:03.835 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.832+0000 7f116158a740 1 -- 172.21.15.151:0/2916528339 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b4e532140 msgr2=0x557b4e52e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:03.835 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.832+0000 7f116158a740 1 --2- 172.21.15.151:0/2916528339 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b4e532140 0x557b4e52e110 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1144005d00 tx=0x7f1144005f90 comp rx=0 tx=0).stop 2024-04-26T02:02:03.835 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.833+0000 7f116158a740 1 -- 172.21.15.151:0/2916528339 shutdown_connections 2024-04-26T02:02:03.835 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.833+0000 7f116158a740 1 --2- 172.21.15.151:0/2916528339 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4e53ca00 0x557b4e53a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.835 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.833+0000 7f116158a740 1 --2- 172.21.15.151:0/2916528339 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b4e532140 0x557b4e52e110 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.835 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.833+0000 7f116158a740 1 --2- 172.21.15.151:0/2916528339 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b4e531820 0x557b4e548490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.835 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.833+0000 7f116158a740 1 -- 172.21.15.151:0/2916528339 >> 172.21.15.151:0/2916528339 conn(0x557b4e52dd30 msgr2=0x557b4e52ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:03.835 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.833+0000 7f116158a740 1 -- 172.21.15.151:0/2916528339 shutdown_connections 2024-04-26T02:02:03.835 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.833+0000 7f116158a740 1 -- 172.21.15.151:0/2916528339 wait complete. 2024-04-26T02:02:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.833+0000 7f116158a740 1 -- start start 2024-04-26T02:02:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.833+0000 7f116158a740 1 Processor -- start 2024-04-26T02:02:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.834+0000 7f116158a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4e531820 0x557b4e542410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.834+0000 7f116158a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b4e532140 0x557b4e542950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.834+0000 7f116158a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b4e53ca00 0x557b4e542e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.834+0000 7f116158a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f11581ae340 con 0x557b4e53ca00 2024-04-26T02:02:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.834+0000 7f116158a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f11581af7b0 con 0x557b4e531820 2024-04-26T02:02:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.834+0000 7f116158a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f11581aea10 con 0x557b4e532140 2024-04-26T02:02:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.834+0000 7f116158a740 10 client.0 _ll_register_callbacks cb 0x7f1150009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:02:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.834+0000 7f1157e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4e531820 0x557b4e542410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.834+0000 7f1156a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b4e532140 0x557b4e542950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.834+0000 7f1156a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b4e532140 0x557b4e542950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:47660/0 (socket says 172.21.15.151:47660) 2024-04-26T02:02:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.834+0000 7f1156a00640 1 -- 172.21.15.151:0/4229214685 learned_addr learned my addr 172.21.15.151:0/4229214685 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:02:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.834+0000 7f1157e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4e531820 0x557b4e542410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:40708/0 (socket says 172.21.15.151:40708) 2024-04-26T02:02:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.834+0000 7f1157e00640 1 --2- 172.21.15.151:0/4229214685 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b4e53ca00 0x557b4e542e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.834+0000 7f1156a00640 1 -- 172.21.15.151:0/4229214685 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4e531820 msgr2=0x557b4e542410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.834+0000 7f1156a00640 1 --2- 172.21.15.151:0/4229214685 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4e531820 0x557b4e542410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.835+0000 7f1156a00640 1 -- 172.21.15.151:0/4229214685 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b4e53ca00 msgr2=0x557b4e542e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.835+0000 7f1156a00640 1 --2- 172.21.15.151:0/4229214685 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b4e53ca00 0x557b4e542e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.835+0000 7f1156a00640 1 -- 172.21.15.151:0/4229214685 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b4e531f70 con 0x557b4e532140 2024-04-26T02:02:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.835+0000 7f1156a00640 1 --2- 172.21.15.151:0/4229214685 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b4e532140 0x557b4e542950 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x557b4e53abf0 tx=0x7f11440121d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87c3f204037e57f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.835+0000 7f1141600640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-26T02:02:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.835+0000 7f1141600640 1 -- 172.21.15.151:0/4229214685 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11440054b0 con 0x557b4e532140 2024-04-26T02:02:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.835+0000 7f1141600640 1 -- 172.21.15.151:0/4229214685 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8499+0+0 (secure 0 0 0) 0x7f1144004950 con 0x557b4e532140 2024-04-26T02:02:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.835+0000 7f116158a740 1 -- 172.21.15.151:0/4229214685 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11440047a0 con 0x557b4e532140 2024-04-26T02:02:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.835+0000 7f1141600640 1 -- 172.21.15.151:0/4229214685 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f114403a050 con 0x557b4e532140 2024-04-26T02:02:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.836+0000 7f116158a740 10 client.26200 fetch_fsmap learned FSMap version 35 2024-04-26T02:02:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.836+0000 7f116158a740 1 -- 172.21.15.151:0/4229214685 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x557b4e5b2e40 con 0x557b4e532140 2024-04-26T02:02:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.836+0000 7f1141600640 1 -- 172.21.15.151:0/4229214685 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11440047a0 con 0x557b4e532140 2024-04-26T02:02:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.836+0000 7f1141600640 1 -- 172.21.15.151:0/4229214685 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 35) ==== 8338+0+0 (secure 0 0 0) 0x7f114401f4b0 con 0x557b4e532140 2024-04-26T02:02:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.836+0000 7f116158a740 10 client.26200 fetch_fsmap finished waiting for FSMap version 35 2024-04-26T02:02:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.836+0000 7f116158a740 10 client.26200 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:02:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.836+0000 7f116158a740 20 client.26200 populate_metadata read hostname 'smithi151' 2024-04-26T02:02:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.836+0000 7f116158a740 1 --2- 172.21.15.151:0/4229214685 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x557b4e5b3530 0x557b4e5b5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.836+0000 7f116158a740 4 client.26200 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T02:02:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.836+0000 7f116158a740 1 -- 172.21.15.151:0/4229214685 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557b4e4a5320 con 0x557b4e5b3530 2024-04-26T02:02:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.836+0000 7f1157e00640 1 --2- 172.21.15.151:0/4229214685 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x557b4e5b3530 0x557b4e5b5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.837+0000 7f1157e00640 1 --2- 172.21.15.151:0/4229214685 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x557b4e5b3530 0x557b4e5b5950 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98597c7b54939604 server_cookie=84eb9ab77c48f36e in_seq=0 out_seq=0 2024-04-26T02:02:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.837+0000 7f1141600640 10 client.26200 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:02:03.848 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.849+0000 7f1141600640 1 -- 172.21.15.151:0/4229214685 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557b4e4a5320 con 0x557b4e5b3530 2024-04-26T02:02:03.848 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.849+0000 7f1141600640 10 client.26200 handle_command_reply: tid=0 2024-04-26T02:02:03.848 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.849+0000 7f116158a740 1 client.26200 shutdown 2024-04-26T02:02:03.848 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.850+0000 7f116158a740 1 -- 172.21.15.151:0/4229214685 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b4e532140 msgr2=0x557b4e542950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.850+0000 7f116158a740 1 --2- 172.21.15.151:0/4229214685 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b4e532140 0x557b4e542950 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x557b4e53abf0 tx=0x7f11440121d0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.850+0000 7f116158a740 1 -- 172.21.15.151:0/4229214685 shutdown_connections 2024-04-26T02:02:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.850+0000 7f116158a740 1 --2- 172.21.15.151:0/4229214685 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x557b4e5b3530 0x557b4e5b5950 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.850+0000 7f116158a740 1 --2- 172.21.15.151:0/4229214685 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b4e53ca00 0x557b4e542e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.850+0000 7f116158a740 1 --2- 172.21.15.151:0/4229214685 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b4e532140 0x557b4e542950 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.850+0000 7f116158a740 1 --2- 172.21.15.151:0/4229214685 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4e531820 0x557b4e542410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.850+0000 7f1141600640 0 client.26200 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:02:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.850+0000 7f116158a740 1 -- 172.21.15.151:0/4229214685 >> 172.21.15.151:0/4229214685 conn(0x557b4e52dd30 msgr2=0x557b4e52ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.850+0000 7f116158a740 1 -- 172.21.15.151:0/4229214685 shutdown_connections 2024-04-26T02:02:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.850+0000 7f116158a740 1 -- 172.21.15.151:0/4229214685 wait complete. 2024-04-26T02:02:03.850 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.850+0000 7f116158a740 20 client.26200 trim_cache size 0 max 16384 2024-04-26T02:02:03.850 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:{ 2024-04-26T02:02:03.850 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "message": "", 2024-04-26T02:02:03.850 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "return_code": 0 2024-04-26T02:02:03.850 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:} 2024-04-26T02:02:03.851 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.851+0000 7f115e800640 1 -- 172.21.15.151:0/3758705339 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f113c0a3e40 msgr2=0x7f113c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:03.851 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.851+0000 7f115e800640 1 --2- 172.21.15.151:0/3758705339 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f113c0a3e40 0x7f113c0a6300 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f11480ce350 tx=0x7f11480d4690 comp rx=0 tx=0).stop 2024-04-26T02:02:03.852 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.851+0000 7f115e800640 1 -- 172.21.15.151:0/3758705339 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11581043a0 msgr2=0x7f11581a05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:03.852 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.851+0000 7f115e800640 1 --2- 172.21.15.151:0/3758705339 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11581043a0 0x7f11581a05c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f114800e9f0 tx=0x7f114800eeb0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.852 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.851+0000 7f115e800640 1 -- 172.21.15.151:0/3758705339 shutdown_connections 2024-04-26T02:02:03.852 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.851+0000 7f115e800640 1 --2- 172.21.15.151:0/3758705339 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f113c0a3e40 0x7f113c0a6300 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.852 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.851+0000 7f115e800640 1 --2- 172.21.15.151:0/3758705339 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f115819a6d0 0x7f115819ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.852 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.851+0000 7f115e800640 1 --2- 172.21.15.151:0/3758705339 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1158109470 0x7f11581a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.852 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.851+0000 7f115e800640 1 --2- 172.21.15.151:0/3758705339 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11581043a0 0x7f11581a05c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.852 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.851+0000 7f115e800640 1 -- 172.21.15.151:0/3758705339 >> 172.21.15.151:0/3758705339 conn(0x7f11580ffc40 msgr2=0x7f1158101f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:03.852 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.851+0000 7f115e800640 1 -- 172.21.15.151:0/3758705339 shutdown_connections 2024-04-26T02:02:03.853 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:03.851+0000 7f115e800640 1 -- 172.21.15.151:0/3758705339 wait complete. 2024-04-26T02:02:04.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:02:04 smithi172 ceph-mon[32785]: pgmap v263: 97 pgs: 97 active+clean; 101 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 41 KiB/s wr, 21 op/s 2024-04-26T02:02:04.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:02:04 smithi172 ceph-mon[32785]: osdmap e101: 12 total, 12 up, 12 in 2024-04-26T02:02:04.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:02:04 smithi165 ceph-mon[32648]: pgmap v263: 97 pgs: 97 active+clean; 101 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 41 KiB/s wr, 21 op/s 2024-04-26T02:02:04.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:02:04 smithi165 ceph-mon[32648]: osdmap e101: 12 total, 12 up, 12 in 2024-04-26T02:02:04.873 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:02:04 smithi151 ceph-mon[28057]: pgmap v263: 97 pgs: 97 active+clean; 101 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 41 KiB/s wr, 21 op/s 2024-04-26T02:02:04.874 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:02:04 smithi151 ceph-mon[28057]: osdmap e101: 12 total, 12 up, 12 in 2024-04-26T02:02:06.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:02:06 smithi172 ceph-mon[32785]: pgmap v265: 97 pgs: 97 active+clean; 102 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 168 KiB/s wr, 69 op/s 2024-04-26T02:02:06.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:02:06 smithi165 ceph-mon[32648]: pgmap v265: 97 pgs: 97 active+clean; 102 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 168 KiB/s wr, 69 op/s 2024-04-26T02:02:06.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:02:06 smithi151 ceph-mon[28057]: pgmap v265: 97 pgs: 97 active+clean; 102 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 168 KiB/s wr, 69 op/s 2024-04-26T02:02:08.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:02:08 smithi172 ceph-mon[32785]: pgmap v266: 97 pgs: 97 active+clean; 102 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 168 KiB/s wr, 69 op/s 2024-04-26T02:02:08.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:02:08 smithi165 ceph-mon[32648]: pgmap v266: 97 pgs: 97 active+clean; 102 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 168 KiB/s wr, 69 op/s 2024-04-26T02:02:08.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:02:08 smithi151 ceph-mon[28057]: pgmap v266: 97 pgs: 97 active+clean; 102 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 168 KiB/s wr, 69 op/s 2024-04-26T02:02:10.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:02:10 smithi172 ceph-mon[32785]: pgmap v267: 97 pgs: 97 active+clean; 108 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 805 KiB/s wr, 283 op/s 2024-04-26T02:02:10.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:02:10 smithi165 ceph-mon[32648]: pgmap v267: 97 pgs: 97 active+clean; 108 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 805 KiB/s wr, 283 op/s 2024-04-26T02:02:10.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:02:10 smithi151 ceph-mon[28057]: pgmap v267: 97 pgs: 97 active+clean; 108 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 805 KiB/s wr, 283 op/s 2024-04-26T02:02:12.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:02:12 smithi151 ceph-mon[28057]: pgmap v268: 97 pgs: 97 active+clean; 108 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 805 KiB/s wr, 283 op/s 2024-04-26T02:02:12.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:02:12 smithi172 ceph-mon[32785]: pgmap v268: 97 pgs: 97 active+clean; 108 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 805 KiB/s wr, 283 op/s 2024-04-26T02:02:12.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:02:12 smithi165 ceph-mon[32648]: pgmap v268: 97 pgs: 97 active+clean; 108 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 805 KiB/s wr, 283 op/s 2024-04-26T02:02:14.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:02:14 smithi172 ceph-mon[32785]: pgmap v269: 97 pgs: 97 active+clean; 104 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 946 KiB/s wr, 337 op/s 2024-04-26T02:02:14.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:02:14 smithi172 ceph-mon[32785]: osdmap e102: 12 total, 12 up, 12 in 2024-04-26T02:02:14.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:02:14 smithi165 ceph-mon[32648]: pgmap v269: 97 pgs: 97 active+clean; 104 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 946 KiB/s wr, 337 op/s 2024-04-26T02:02:14.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:02:14 smithi165 ceph-mon[32648]: osdmap e102: 12 total, 12 up, 12 in 2024-04-26T02:02:14.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:02:14 smithi151 ceph-mon[28057]: pgmap v269: 97 pgs: 97 active+clean; 104 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 946 KiB/s wr, 337 op/s 2024-04-26T02:02:14.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:02:14 smithi151 ceph-mon[28057]: osdmap e102: 12 total, 12 up, 12 in 2024-04-26T02:02:16.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:02:16 smithi172 ceph-mon[32785]: pgmap v271: 97 pgs: 97 active+clean; 112 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 1.2 MiB/s wr, 417 op/s 2024-04-26T02:02:16.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:02:16 smithi165 ceph-mon[32648]: pgmap v271: 97 pgs: 97 active+clean; 112 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 1.2 MiB/s wr, 417 op/s 2024-04-26T02:02:16.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:02:16 smithi151 ceph-mon[28057]: pgmap v271: 97 pgs: 97 active+clean; 112 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 1.2 MiB/s wr, 417 op/s 2024-04-26T02:02:17.001 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.002+0000 7f7062c00640 1 -- 172.21.15.151:0/4024410864 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f705c112930 msgr2=0x7f705c112d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:17.002 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.002+0000 7f7062c00640 1 --2- 172.21.15.151:0/4024410864 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f705c112930 0x7f705c112d10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f704c009900 tx=0x7f704c02a400 comp rx=0 tx=0).stop 2024-04-26T02:02:17.002 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.002+0000 7f7062c00640 1 -- 172.21.15.151:0/4024410864 shutdown_connections 2024-04-26T02:02:17.002 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.002+0000 7f7062c00640 1 --2- 172.21.15.151:0/4024410864 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f705c10d330 0x7f705c1197f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:17.002 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.002+0000 7f7062c00640 1 --2- 172.21.15.151:0/4024410864 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f705c10c970 0x7f705c10cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:17.002 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.002+0000 7f7062c00640 1 --2- 172.21.15.151:0/4024410864 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f705c112930 0x7f705c112d10 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:17.002 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.002+0000 7f7062c00640 1 -- 172.21.15.151:0/4024410864 >> 172.21.15.151:0/4024410864 conn(0x7f705c06d1f0 msgr2=0x7f705c10af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:17.002 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.002+0000 7f7062c00640 1 -- 172.21.15.151:0/4024410864 shutdown_connections 2024-04-26T02:02:17.002 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.002+0000 7f7062c00640 1 -- 172.21.15.151:0/4024410864 wait complete. 2024-04-26T02:02:17.003 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.004+0000 7f7062c00640 1 Processor -- start 2024-04-26T02:02:17.003 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.004+0000 7f7062c00640 1 -- start start 2024-04-26T02:02:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.004+0000 7f7062c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f705c10c970 0x7f705c1a3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.004+0000 7f7062c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f705c10d330 0x7f705c1a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.004+0000 7f7062c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f705c112930 0x7f705c1a7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.004+0000 7f7062c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f705c11cc10 con 0x7f705c10c970 2024-04-26T02:02:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.004+0000 7f7062c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f705c11ca90 con 0x7f705c10d330 2024-04-26T02:02:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.004+0000 7f7062c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f705c11cd90 con 0x7f705c112930 2024-04-26T02:02:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.005+0000 7f705be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f705c10c970 0x7f705c1a3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.005+0000 7f705b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f705c10d330 0x7f705c1a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.005+0000 7f705b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f705c10d330 0x7f705c1a35d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:32808/0 (socket says 172.21.15.151:32808) 2024-04-26T02:02:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.005+0000 7f705b400640 1 -- 172.21.15.151:0/2729658047 learned_addr learned my addr 172.21.15.151:0/2729658047 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:02:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.005+0000 7f705be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f705c10c970 0x7f705c1a3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:54462/0 (socket says 172.21.15.151:54462) 2024-04-26T02:02:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.005+0000 7f7060c00640 1 --2- 172.21.15.151:0/2729658047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f705c112930 0x7f705c1a7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.005+0000 7f705b400640 1 -- 172.21.15.151:0/2729658047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f705c112930 msgr2=0x7f705c1a7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.005+0000 7f705b400640 1 --2- 172.21.15.151:0/2729658047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f705c112930 0x7f705c1a7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.005+0000 7f705b400640 1 -- 172.21.15.151:0/2729658047 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f705c10c970 msgr2=0x7f705c1a3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.005+0000 7f705b400640 1 --2- 172.21.15.151:0/2729658047 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f705c10c970 0x7f705c1a3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.005+0000 7f705b400640 1 -- 172.21.15.151:0/2729658047 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f705c1a8030 con 0x7f705c10d330 2024-04-26T02:02:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.005+0000 7f705b400640 1 --2- 172.21.15.151:0/2729658047 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f705c10d330 0x7f705c1a35d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f704400e9f0 tx=0x7f704400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.006+0000 7f7058c00640 1 -- 172.21.15.151:0/2729658047 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704400cd80 con 0x7f705c10d330 2024-04-26T02:02:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.006+0000 7f7062c00640 1 -- 172.21.15.151:0/2729658047 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f705c1a8320 con 0x7f705c10d330 2024-04-26T02:02:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.006+0000 7f7058c00640 1 -- 172.21.15.151:0/2729658047 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70440108e0 con 0x7f705c10d330 2024-04-26T02:02:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.006+0000 7f7058c00640 1 -- 172.21.15.151:0/2729658047 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7044018980 con 0x7f705c10d330 2024-04-26T02:02:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.006+0000 7f7062c00640 1 -- 172.21.15.151:0/2729658047 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f705c1afbb0 con 0x7f705c10d330 2024-04-26T02:02:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.009+0000 7f7065a35740 1 Processor -- start 2024-04-26T02:02:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.009+0000 7f7065a35740 1 -- start start 2024-04-26T02:02:17.010 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.010+0000 7f7065a35740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646f1cfe820 0x5646f1cfec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:17.010 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.010+0000 7f7065a35740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5646f1cff140 0x5646f1d094c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:17.010 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.010+0000 7f7065a35740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646f1d09a00 0x5646f1d0bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:17.010 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.010+0000 7f7065a35740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5646f1c64900 con 0x5646f1cfe820 2024-04-26T02:02:17.010 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.010+0000 7f7065a35740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5646f1c0aef0 con 0x5646f1cff140 2024-04-26T02:02:17.010 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.010+0000 7f7065a35740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5646f1c608c0 con 0x5646f1d09a00 2024-04-26T02:02:17.010 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.010+0000 7f7060c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646f1cfe820 0x5646f1cfec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:17.010 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.010+0000 7f7060c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646f1cfe820 0x5646f1cfec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:54468/0 (socket says 172.21.15.151:54468) 2024-04-26T02:02:17.010 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.010+0000 7f7060c00640 1 -- 172.21.15.151:0/1956375297 learned_addr learned my addr 172.21.15.151:0/1956375297 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:02:17.010 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.010+0000 7f705be00640 1 --2- 172.21.15.151:0/1956375297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5646f1cff140 0x5646f1d094c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.010+0000 7f7058c00640 1 -- 172.21.15.151:0/2729658047 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f7044010430 con 0x7f705c10d330 2024-04-26T02:02:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.011+0000 7f7060c00640 1 -- 172.21.15.151:0/1956375297 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646f1d09a00 msgr2=0x5646f1d0bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.011+0000 7f705b400640 1 --2- 172.21.15.151:0/1956375297 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646f1d09a00 0x5646f1d0bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.011+0000 7f7060c00640 1 --2- 172.21.15.151:0/1956375297 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646f1d09a00 0x5646f1d0bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.011+0000 7f7060c00640 1 -- 172.21.15.151:0/1956375297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5646f1cff140 msgr2=0x5646f1d094c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.011+0000 7f7060c00640 1 --2- 172.21.15.151:0/1956375297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5646f1cff140 0x5646f1d094c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.011+0000 7f7060c00640 1 -- 172.21.15.151:0/1956375297 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646f1977210 con 0x5646f1cfe820 2024-04-26T02:02:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.011+0000 7f705be00640 1 --2- 172.21.15.151:0/1956375297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5646f1cff140 0x5646f1d094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:02:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.011+0000 7f705b400640 1 --2- 172.21.15.151:0/1956375297 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646f1d09a00 0x5646f1d0bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T02:02:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.011+0000 7f7060c00640 1 --2- 172.21.15.151:0/1956375297 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646f1cfe820 0x5646f1cfec00 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7054009f50 tx=0x7f70540118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd7dbdd75e819a7a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:17.014 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.011+0000 7f7051200640 1 -- 172.21.15.151:0/1956375297 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7054004d20 con 0x5646f1cfe820 2024-04-26T02:02:17.014 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.011+0000 7f7051200640 1 -- 172.21.15.151:0/1956375297 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7054004050 con 0x5646f1cfe820 2024-04-26T02:02:17.014 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.011+0000 7f7051200640 1 -- 172.21.15.151:0/1956375297 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7054005220 con 0x5646f1cfe820 2024-04-26T02:02:17.014 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.012+0000 7f7058c00640 1 --2- 172.21.15.151:0/2729658047 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f70400a3dc0 0x7f70400a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:17.014 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.012+0000 7f7058c00640 1 -- 172.21.15.151:0/2729658047 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8455+0+0 (secure 0 0 0) 0x7f70440d0ae0 con 0x7f705c10d330 2024-04-26T02:02:17.014 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.012+0000 7f7065a35740 1 -- 172.21.15.151:0/1956375297 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646f1cfe820 msgr2=0x5646f1cfec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:17.014 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.012+0000 7f7065a35740 1 --2- 172.21.15.151:0/1956375297 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646f1cfe820 0x5646f1cfec00 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7054009f50 tx=0x7f70540118d0 comp rx=0 tx=0).stop 2024-04-26T02:02:17.014 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.012+0000 7f705be00640 1 --2- 172.21.15.151:0/2729658047 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f70400a3dc0 0x7f70400a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:17.014 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.013+0000 7f7065a35740 1 -- 172.21.15.151:0/1956375297 shutdown_connections 2024-04-26T02:02:17.014 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.013+0000 7f7065a35740 1 --2- 172.21.15.151:0/1956375297 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646f1d09a00 0x5646f1d0bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:17.014 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.013+0000 7f7065a35740 1 --2- 172.21.15.151:0/1956375297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5646f1cff140 0x5646f1d094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:17.014 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.013+0000 7f7065a35740 1 --2- 172.21.15.151:0/1956375297 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646f1cfe820 0x5646f1cfec00 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:17.014 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.013+0000 7f7065a35740 1 -- 172.21.15.151:0/1956375297 >> 172.21.15.151:0/1956375297 conn(0x5646f1cfad30 msgr2=0x5646f1cfb110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:17.015 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.013+0000 7f7065a35740 1 -- 172.21.15.151:0/1956375297 shutdown_connections 2024-04-26T02:02:17.015 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.013+0000 7f7065a35740 1 -- 172.21.15.151:0/1956375297 wait complete. 2024-04-26T02:02:17.015 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.013+0000 7f7065a35740 1 -- start start 2024-04-26T02:02:17.015 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.014+0000 7f705be00640 1 --2- 172.21.15.151:0/2729658047 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f70400a3dc0 0x7f70400a6280 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f704c000c00 tx=0x7f704c005e80 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:17.015 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.014+0000 7f7065a35740 1 Processor -- start 2024-04-26T02:02:17.015 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.014+0000 7f7065a35740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5646f1cfe820 0x5646f1d068a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:17.015 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.014+0000 7f7065a35740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646f1cff140 0x5646f1d06de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:17.015 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.014+0000 7f7065a35740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646f1d09a00 0x5646f1d07320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:17.015 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.014+0000 7f7065a35740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5646f1d0c630 con 0x5646f1cff140 2024-04-26T02:02:17.015 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.014+0000 7f7065a35740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5646f1c1e310 con 0x5646f1cfe820 2024-04-26T02:02:17.015 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.014+0000 7f7065a35740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5646f1d0c7b0 con 0x5646f1d09a00 2024-04-26T02:02:17.015 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.014+0000 7f7065a35740 10 client.0 _ll_register_callbacks cb 0x5646f1beccb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:02:17.015 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.015+0000 7f705b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646f1cff140 0x5646f1d06de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:17.016 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.015+0000 7f705be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5646f1cfe820 0x5646f1d068a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:17.016 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.015+0000 7f705be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5646f1cfe820 0x5646f1d068a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:32824/0 (socket says 172.21.15.151:32824) 2024-04-26T02:02:17.016 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.015+0000 7f705be00640 1 -- 172.21.15.151:0/194254702 learned_addr learned my addr 172.21.15.151:0/194254702 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:02:17.016 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.015+0000 7f7060c00640 1 --2- 172.21.15.151:0/194254702 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646f1d09a00 0x5646f1d07320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:17.019 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.015+0000 7f7060c00640 1 -- 172.21.15.151:0/194254702 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5646f1cfe820 msgr2=0x5646f1d068a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:17.019 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.015+0000 7f7060c00640 1 --2- 172.21.15.151:0/194254702 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5646f1cfe820 0x5646f1d068a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:17.019 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.015+0000 7f7060c00640 1 -- 172.21.15.151:0/194254702 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646f1cff140 msgr2=0x5646f1d06de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:17.019 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.015+0000 7f7060c00640 1 --2- 172.21.15.151:0/194254702 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646f1cff140 0x5646f1d06de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:17.019 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.015+0000 7f7060c00640 1 -- 172.21.15.151:0/194254702 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646f1a43620 con 0x5646f1d09a00 2024-04-26T02:02:17.019 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.015+0000 7f705b400640 1 --2- 172.21.15.151:0/194254702 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646f1cff140 0x5646f1d06de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:02:17.019 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.016+0000 7f7060c00640 1 --2- 172.21.15.151:0/194254702 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646f1d09a00 0x5646f1d07320 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f705401b040 tx=0x7f70540044b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e0a711726e4282c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:17.020 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.016+0000 7f7049600640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-26T02:02:17.020 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.016+0000 7f7049600640 1 -- 172.21.15.151:0/194254702 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7054028070 con 0x5646f1d09a00 2024-04-26T02:02:17.020 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.016+0000 7f7049600640 1 -- 172.21.15.151:0/194254702 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8455+0+0 (secure 0 0 0) 0x7f7054005040 con 0x5646f1d09a00 2024-04-26T02:02:17.020 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.016+0000 7f7065a35740 1 -- 172.21.15.151:0/194254702 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7054004050 con 0x5646f1d09a00 2024-04-26T02:02:17.020 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.016+0000 7f7049600640 1 -- 172.21.15.151:0/194254702 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f7054009be0 con 0x5646f1d09a00 2024-04-26T02:02:17.020 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.016+0000 7f7065a35740 10 client.26209 fetch_fsmap learned FSMap version 35 2024-04-26T02:02:17.020 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.016+0000 7f7065a35740 1 -- 172.21.15.151:0/194254702 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5646f1d18720 con 0x5646f1d09a00 2024-04-26T02:02:17.020 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.017+0000 7f7049600640 1 -- 172.21.15.151:0/194254702 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7054004050 con 0x5646f1d09a00 2024-04-26T02:02:17.020 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.017+0000 7f7049600640 1 -- 172.21.15.151:0/194254702 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 35) ==== 8338+0+0 (secure 0 0 0) 0x7f7054024560 con 0x5646f1d09a00 2024-04-26T02:02:17.020 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.017+0000 7f7065a35740 10 client.26209 fetch_fsmap finished waiting for FSMap version 35 2024-04-26T02:02:17.020 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.017+0000 7f7065a35740 10 client.26209 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:02:17.020 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.017+0000 7f7065a35740 20 client.26209 populate_metadata read hostname 'smithi151' 2024-04-26T02:02:17.020 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.017+0000 7f7065a35740 1 --2- 172.21.15.151:0/194254702 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5646f1d19ad0 0x5646f1d1bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:17.020 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.017+0000 7f7065a35740 4 client.26209 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:02:17.021 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.017+0000 7f7065a35740 1 -- 172.21.15.151:0/194254702 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646f1c788b0 con 0x5646f1d19ad0 2024-04-26T02:02:17.021 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.017+0000 7f705be00640 1 --2- 172.21.15.151:0/194254702 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5646f1d19ad0 0x5646f1d1bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:17.021 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.018+0000 7f705be00640 1 --2- 172.21.15.151:0/194254702 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5646f1d19ad0 0x5646f1d1bec0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3274a48145e688fd server_cookie=97c83ce0570ae1a1 in_seq=0 out_seq=0 2024-04-26T02:02:17.021 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.018+0000 7f7049600640 10 client.26209 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:02:17.021 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.019+0000 7f7049600640 1 -- 172.21.15.151:0/194254702 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5646f1c788b0 con 0x5646f1d19ad0 2024-04-26T02:02:17.021 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.019+0000 7f7049600640 10 client.26209 handle_command_reply: tid=0 2024-04-26T02:02:17.021 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.019+0000 7f7065a35740 1 client.26209 shutdown 2024-04-26T02:02:17.021 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.019+0000 7f7065a35740 1 -- 172.21.15.151:0/194254702 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646f1d09a00 msgr2=0x5646f1d07320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:17.021 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.019+0000 7f7065a35740 1 --2- 172.21.15.151:0/194254702 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646f1d09a00 0x5646f1d07320 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f705401b040 tx=0x7f70540044b0 comp rx=0 tx=0).stop 2024-04-26T02:02:17.021 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.019+0000 7f7065a35740 1 -- 172.21.15.151:0/194254702 shutdown_connections 2024-04-26T02:02:17.021 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.019+0000 7f7065a35740 1 --2- 172.21.15.151:0/194254702 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5646f1d19ad0 0x5646f1d1bec0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:17.021 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.019+0000 7f7065a35740 1 --2- 172.21.15.151:0/194254702 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646f1d09a00 0x5646f1d07320 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:17.021 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.019+0000 7f7065a35740 1 --2- 172.21.15.151:0/194254702 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646f1cff140 0x5646f1d06de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:17.022 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.019+0000 7f7065a35740 1 --2- 172.21.15.151:0/194254702 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5646f1cfe820 0x5646f1d068a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:17.022 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.019+0000 7f7049600640 0 client.26209 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:02:17.022 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.019+0000 7f7065a35740 1 -- 172.21.15.151:0/194254702 >> 172.21.15.151:0/194254702 conn(0x5646f1cfad30 msgr2=0x5646f1d00ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:17.022 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.020+0000 7f7065a35740 1 -- 172.21.15.151:0/194254702 shutdown_connections 2024-04-26T02:02:17.022 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.020+0000 7f7065a35740 1 -- 172.21.15.151:0/194254702 wait complete. 2024-04-26T02:02:17.022 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.020+0000 7f7065a35740 20 client.26209 trim_cache size 0 max 16384 2024-04-26T02:02:17.039 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.039+0000 7f7065a35740 1 Processor -- start 2024-04-26T02:02:17.039 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.039+0000 7f7065a35740 1 -- start start 2024-04-26T02:02:17.039 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.040+0000 7f7065a35740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646f1cfad30 0x5646f1d154f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:17.039 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.040+0000 7f7065a35740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646f1cfe820 0x5646f1d05a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:17.039 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.040+0000 7f7065a35740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5646f1cff140 0x5646f1cf9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:17.040 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.040+0000 7f7065a35740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f705c1bacc0 con 0x5646f1cfe820 2024-04-26T02:02:17.040 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.040+0000 7f7065a35740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f705c1b8590 con 0x5646f1cff140 2024-04-26T02:02:17.040 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.040+0000 7f7065a35740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f705c1b6400 con 0x5646f1cfad30 2024-04-26T02:02:17.040 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.040+0000 7f705be00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646f1cfad30 0x5646f1d154f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:17.040 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.040+0000 7f705be00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646f1cfad30 0x5646f1d154f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:50574/0 (socket says 172.21.15.151:50574) 2024-04-26T02:02:17.040 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.040+0000 7f705be00640 1 -- 172.21.15.151:0/4015238175 learned_addr learned my addr 172.21.15.151:0/4015238175 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:02:17.040 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.040+0000 7f705b400640 1 --2- 172.21.15.151:0/4015238175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646f1cfe820 0x5646f1d05a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:17.040 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.040+0000 7f7060c00640 1 --2- 172.21.15.151:0/4015238175 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5646f1cff140 0x5646f1cf9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:17.040 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.040+0000 7f705be00640 1 -- 172.21.15.151:0/4015238175 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5646f1cff140 msgr2=0x5646f1cf9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:17.040 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.040+0000 7f705be00640 1 --2- 172.21.15.151:0/4015238175 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5646f1cff140 0x5646f1cf9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:17.040 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.040+0000 7f705be00640 1 -- 172.21.15.151:0/4015238175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646f1cfe820 msgr2=0x5646f1d05a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:17.040 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.040+0000 7f705be00640 1 --2- 172.21.15.151:0/4015238175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646f1cfe820 0x5646f1d05a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:17.040 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.040+0000 7f705be00640 1 -- 172.21.15.151:0/4015238175 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646f1cfef30 con 0x5646f1cfad30 2024-04-26T02:02:17.041 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.040+0000 7f705b400640 1 --2- 172.21.15.151:0/4015238175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646f1cfe820 0x5646f1d05a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:02:17.044 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.041+0000 7f705be00640 1 --2- 172.21.15.151:0/4015238175 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646f1cfad30 0x5646f1d154f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f704c009230 tx=0x7f704c03f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b52dea46de338ca server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:17.044 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.041+0000 7f704be00640 1 -- 172.21.15.151:0/4015238175 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704c035040 con 0x5646f1cfad30 2024-04-26T02:02:17.044 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.041+0000 7f704be00640 1 -- 172.21.15.151:0/4015238175 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f704c05a070 con 0x5646f1cfad30 2024-04-26T02:02:17.045 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.041+0000 7f704be00640 1 -- 172.21.15.151:0/4015238175 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704c0035f0 con 0x5646f1cfad30 2024-04-26T02:02:17.045 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.042+0000 7f7065a35740 1 -- 172.21.15.151:0/4015238175 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646f1cfad30 msgr2=0x5646f1d154f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:17.045 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.042+0000 7f7065a35740 1 --2- 172.21.15.151:0/4015238175 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646f1cfad30 0x5646f1d154f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f704c009230 tx=0x7f704c03f000 comp rx=0 tx=0).stop 2024-04-26T02:02:17.045 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.042+0000 7f7065a35740 1 -- 172.21.15.151:0/4015238175 shutdown_connections 2024-04-26T02:02:17.045 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.042+0000 7f7065a35740 1 --2- 172.21.15.151:0/4015238175 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5646f1cff140 0x5646f1cf9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:17.045 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.042+0000 7f7065a35740 1 --2- 172.21.15.151:0/4015238175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646f1cfe820 0x5646f1d05a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:17.045 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.042+0000 7f7065a35740 1 --2- 172.21.15.151:0/4015238175 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646f1cfad30 0x5646f1d154f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:17.045 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.042+0000 7f7065a35740 1 -- 172.21.15.151:0/4015238175 >> 172.21.15.151:0/4015238175 conn(0x5646f1cf6260 msgr2=0x5646f1cf9600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:17.045 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.042+0000 7f7065a35740 1 -- 172.21.15.151:0/4015238175 shutdown_connections 2024-04-26T02:02:17.045 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.042+0000 7f7065a35740 1 -- 172.21.15.151:0/4015238175 wait complete. 2024-04-26T02:02:17.045 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.042+0000 7f7065a35740 1 -- start start 2024-04-26T02:02:17.045 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.043+0000 7f7065a35740 1 Processor -- start 2024-04-26T02:02:17.046 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.043+0000 7f7065a35740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646f1cfad30 0x5646f1d0ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:17.046 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.043+0000 7f7065a35740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5646f1cfe820 0x5646f1d0f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:17.046 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.043+0000 7f7065a35740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646f1cff140 0x5646f1d7def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:17.046 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.043+0000 7f7065a35740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f705c1b6a50 con 0x5646f1cfad30 2024-04-26T02:02:17.046 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.043+0000 7f7065a35740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f705c1b7ec0 con 0x5646f1cfe820 2024-04-26T02:02:17.046 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.043+0000 7f7065a35740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f705c1b7120 con 0x5646f1cff140 2024-04-26T02:02:17.046 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.043+0000 7f7065a35740 10 client.0 _ll_register_callbacks cb 0x7f7054003e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:02:17.046 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.043+0000 7f705b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5646f1cfe820 0x5646f1d0f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:17.046 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.043+0000 7f705be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646f1cfad30 0x5646f1d0ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:17.046 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.043+0000 7f705b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5646f1cfe820 0x5646f1d0f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:32836/0 (socket says 172.21.15.151:32836) 2024-04-26T02:02:17.046 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.043+0000 7f705b400640 1 -- 172.21.15.151:0/3530154401 learned_addr learned my addr 172.21.15.151:0/3530154401 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:02:17.046 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.043+0000 7f7060c00640 1 --2- 172.21.15.151:0/3530154401 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646f1cff140 0x5646f1d7def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:17.046 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.044+0000 7f705b400640 1 -- 172.21.15.151:0/3530154401 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646f1cff140 msgr2=0x5646f1d7def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:17.047 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.044+0000 7f705b400640 1 --2- 172.21.15.151:0/3530154401 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646f1cff140 0x5646f1d7def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:17.047 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.044+0000 7f705b400640 1 -- 172.21.15.151:0/3530154401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646f1cfad30 msgr2=0x5646f1d0ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:17.047 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.044+0000 7f705b400640 1 --2- 172.21.15.151:0/3530154401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646f1cfad30 0x5646f1d0ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:17.047 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.044+0000 7f705b400640 1 -- 172.21.15.151:0/3530154401 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646f1d7e4b0 con 0x5646f1cfe820 2024-04-26T02:02:17.047 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.044+0000 7f705b400640 1 --2- 172.21.15.151:0/3530154401 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5646f1cfe820 0x5646f1d0f140 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f704409c810 tx=0x7f704401f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f3500fec4d68107 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:17.047 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.044+0000 7f7049600640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T02:02:17.047 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.044+0000 7f7049600640 1 -- 172.21.15.151:0/3530154401 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704409c160 con 0x5646f1cfe820 2024-04-26T02:02:17.047 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.044+0000 7f7049600640 1 -- 172.21.15.151:0/3530154401 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8455+0+0 (secure 0 0 0) 0x7f7044018b20 con 0x5646f1cfe820 2024-04-26T02:02:17.047 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.044+0000 7f7065a35740 1 -- 172.21.15.151:0/3530154401 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f704c05a070 con 0x5646f1cfe820 2024-04-26T02:02:17.047 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.045+0000 7f7049600640 1 -- 172.21.15.151:0/3530154401 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f70440a0e10 con 0x5646f1cfe820 2024-04-26T02:02:17.047 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.045+0000 7f7065a35740 10 client.26183 fetch_fsmap learned FSMap version 35 2024-04-26T02:02:17.047 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.045+0000 7f7065a35740 1 -- 172.21.15.151:0/3530154401 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5646f1d7f800 con 0x5646f1cfe820 2024-04-26T02:02:17.048 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.045+0000 7f7049600640 1 -- 172.21.15.151:0/3530154401 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f704c05a070 con 0x5646f1cfe820 2024-04-26T02:02:17.048 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.045+0000 7f7049600640 1 -- 172.21.15.151:0/3530154401 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 35) ==== 8338+0+0 (secure 0 0 0) 0x7f704409b9d0 con 0x5646f1cfe820 2024-04-26T02:02:17.048 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.045+0000 7f7065a35740 10 client.26183 fetch_fsmap finished waiting for FSMap version 35 2024-04-26T02:02:17.048 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.045+0000 7f7065a35740 10 client.26183 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:02:17.048 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.045+0000 7f7065a35740 20 client.26183 populate_metadata read hostname 'smithi151' 2024-04-26T02:02:17.048 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.045+0000 7f7065a35740 1 --2- 172.21.15.151:0/3530154401 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5646f1d09a00 0x5646f1d82040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:17.048 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.045+0000 7f7065a35740 4 client.26183 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T02:02:17.048 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.045+0000 7f7065a35740 1 -- 172.21.15.151:0/3530154401 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5646f1c72320 con 0x5646f1d09a00 2024-04-26T02:02:17.048 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.045+0000 7f705be00640 1 --2- 172.21.15.151:0/3530154401 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5646f1d09a00 0x5646f1d82040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:17.049 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.046+0000 7f705be00640 1 --2- 172.21.15.151:0/3530154401 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5646f1d09a00 0x5646f1d82040 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5b452e8ec908da3 server_cookie=3fb73c61aede5efc in_seq=0 out_seq=0 2024-04-26T02:02:17.049 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.046+0000 7f7049600640 10 client.26183 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:02:17.060 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.061+0000 7f7049600640 1 -- 172.21.15.151:0/3530154401 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5646f1c72320 con 0x5646f1d09a00 2024-04-26T02:02:17.060 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.061+0000 7f7049600640 10 client.26183 handle_command_reply: tid=0 2024-04-26T02:02:17.060 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.061+0000 7f7065a35740 1 client.26183 shutdown 2024-04-26T02:02:17.060 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.061+0000 7f7065a35740 1 -- 172.21.15.151:0/3530154401 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5646f1cfe820 msgr2=0x5646f1d0f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:17.060 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.061+0000 7f7065a35740 1 --2- 172.21.15.151:0/3530154401 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5646f1cfe820 0x5646f1d0f140 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f704409c810 tx=0x7f704401f000 comp rx=0 tx=0).stop 2024-04-26T02:02:17.061 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.061+0000 7f7065a35740 1 -- 172.21.15.151:0/3530154401 shutdown_connections 2024-04-26T02:02:17.061 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.061+0000 7f7065a35740 1 --2- 172.21.15.151:0/3530154401 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5646f1d09a00 0x5646f1d82040 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:17.061 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.061+0000 7f7065a35740 1 --2- 172.21.15.151:0/3530154401 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646f1cff140 0x5646f1d7def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:17.061 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.061+0000 7f7065a35740 1 --2- 172.21.15.151:0/3530154401 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5646f1cfe820 0x5646f1d0f140 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:17.061 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.061+0000 7f7065a35740 1 --2- 172.21.15.151:0/3530154401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646f1cfad30 0x5646f1d0ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:17.061 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.061+0000 7f7065a35740 1 -- 172.21.15.151:0/3530154401 >> 172.21.15.151:0/3530154401 conn(0x5646f1cf6260 msgr2=0x5646f1cf9600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:17.061 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.061+0000 7f7049600640 0 client.26183 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:02:17.062 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.062+0000 7f7065a35740 1 -- 172.21.15.151:0/3530154401 shutdown_connections 2024-04-26T02:02:17.062 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.062+0000 7f7065a35740 1 -- 172.21.15.151:0/3530154401 wait complete. 2024-04-26T02:02:17.062 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.062+0000 7f7065a35740 20 client.26183 trim_cache size 0 max 16384 2024-04-26T02:02:17.062 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:{ 2024-04-26T02:02:17.062 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "message": "", 2024-04-26T02:02:17.062 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "return_code": 0 2024-04-26T02:02:17.062 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:} 2024-04-26T02:02:17.063 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.064+0000 7f7062c00640 1 -- 172.21.15.151:0/2729658047 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f70400a3dc0 msgr2=0x7f70400a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:17.063 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.064+0000 7f7062c00640 1 --2- 172.21.15.151:0/2729658047 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f70400a3dc0 0x7f70400a6280 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f704c000c00 tx=0x7f704c005e80 comp rx=0 tx=0).stop 2024-04-26T02:02:17.063 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.064+0000 7f7062c00640 1 -- 172.21.15.151:0/2729658047 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f705c10d330 msgr2=0x7f705c1a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:17.063 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.064+0000 7f7062c00640 1 --2- 172.21.15.151:0/2729658047 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f705c10d330 0x7f705c1a35d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f704400e9f0 tx=0x7f704400eeb0 comp rx=0 tx=0).stop 2024-04-26T02:02:17.063 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.065+0000 7f7062c00640 1 -- 172.21.15.151:0/2729658047 shutdown_connections 2024-04-26T02:02:17.064 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.065+0000 7f7062c00640 1 --2- 172.21.15.151:0/2729658047 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f70400a3dc0 0x7f70400a6280 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:17.064 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.065+0000 7f7062c00640 1 --2- 172.21.15.151:0/2729658047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f705c112930 0x7f705c1a7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:17.064 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.065+0000 7f7062c00640 1 --2- 172.21.15.151:0/2729658047 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f705c10d330 0x7f705c1a35d0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:17.064 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.065+0000 7f7062c00640 1 --2- 172.21.15.151:0/2729658047 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f705c10c970 0x7f705c1a3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:17.064 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.065+0000 7f7062c00640 1 -- 172.21.15.151:0/2729658047 >> 172.21.15.151:0/2729658047 conn(0x7f705c06d1f0 msgr2=0x7f705c10f680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:17.064 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.065+0000 7f7062c00640 1 -- 172.21.15.151:0/2729658047 shutdown_connections 2024-04-26T02:02:17.064 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:17.065+0000 7f7062c00640 1 -- 172.21.15.151:0/2729658047 wait complete. 2024-04-26T02:02:18.352 INFO:tasks.workunit.client.0.smithi151.stdout:truncating to largest ever: 0x63e5d2a 2024-04-26T02:02:18.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:02:18 smithi172 ceph-mon[32785]: pgmap v272: 97 pgs: 97 active+clean; 112 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 1.2 MiB/s wr, 417 op/s 2024-04-26T02:02:18.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:02:18 smithi165 ceph-mon[32648]: pgmap v272: 97 pgs: 97 active+clean; 112 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 1.2 MiB/s wr, 417 op/s 2024-04-26T02:02:18.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:02:18 smithi151 ceph-mon[28057]: pgmap v272: 97 pgs: 97 active+clean; 112 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 1.2 MiB/s wr, 417 op/s 2024-04-26T02:02:20.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:02:20 smithi172 ceph-mon[32785]: pgmap v273: 97 pgs: 97 active+clean; 110 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.2 MiB/s wr, 395 op/s 2024-04-26T02:02:20.847 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:02:20 smithi165 ceph-mon[32648]: pgmap v273: 97 pgs: 97 active+clean; 110 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.2 MiB/s wr, 395 op/s 2024-04-26T02:02:20.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:02:20 smithi151 ceph-mon[28057]: pgmap v273: 97 pgs: 97 active+clean; 110 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.2 MiB/s wr, 395 op/s 2024-04-26T02:02:22.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:02:22 smithi172 ceph-mon[32785]: pgmap v274: 97 pgs: 97 active+clean; 110 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.2 MiB/s wr, 395 op/s 2024-04-26T02:02:22.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:02:22 smithi165 ceph-mon[32648]: pgmap v274: 97 pgs: 97 active+clean; 110 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.2 MiB/s wr, 395 op/s 2024-04-26T02:02:22.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:02:22 smithi151 ceph-mon[28057]: pgmap v274: 97 pgs: 97 active+clean; 110 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.2 MiB/s wr, 395 op/s 2024-04-26T02:02:24.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:02:24 smithi172 ceph-mon[32785]: pgmap v275: 97 pgs: 97 active+clean; 118 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.2 MiB/s wr, 396 op/s 2024-04-26T02:02:24.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:02:24 smithi165 ceph-mon[32648]: pgmap v275: 97 pgs: 97 active+clean; 118 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.2 MiB/s wr, 396 op/s 2024-04-26T02:02:24.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:02:24 smithi151 ceph-mon[28057]: pgmap v275: 97 pgs: 97 active+clean; 118 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.2 MiB/s wr, 396 op/s 2024-04-26T02:02:24.967 INFO:tasks.workunit.client.0.smithi151.stdout:truncating to largest ever: 0x63e9b52 2024-04-26T02:02:26.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:02:26 smithi172 ceph-mon[32785]: pgmap v276: 97 pgs: 97 active+clean; 119 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 1.1 MiB/s wr, 367 op/s 2024-04-26T02:02:26.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:02:26 smithi165 ceph-mon[32648]: pgmap v276: 97 pgs: 97 active+clean; 119 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 1.1 MiB/s wr, 367 op/s 2024-04-26T02:02:26.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:02:26 smithi151 ceph-mon[28057]: pgmap v276: 97 pgs: 97 active+clean; 119 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 1.1 MiB/s wr, 367 op/s 2024-04-26T02:02:28.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:02:28 smithi172 ceph-mon[32785]: pgmap v277: 97 pgs: 97 active+clean; 119 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 1016 KiB/s wr, 327 op/s 2024-04-26T02:02:28.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:02:28 smithi165 ceph-mon[32648]: pgmap v277: 97 pgs: 97 active+clean; 119 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 1016 KiB/s wr, 327 op/s 2024-04-26T02:02:28.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:02:28 smithi151 ceph-mon[28057]: pgmap v277: 97 pgs: 97 active+clean; 119 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 1016 KiB/s wr, 327 op/s 2024-04-26T02:02:29.732 INFO:tasks.workunit.client.0.smithi151.stdout:20000 read 0xa6cbf1 thru 0xa6e358 (0x1768 bytes) 2024-04-26T02:02:30.219 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.220+0000 7f0e29600640 1 -- 172.21.15.151:0/1972304604 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e24109870 msgr2=0x7f0e24114b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:30.219 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.220+0000 7f0e29600640 1 --2- 172.21.15.151:0/1972304604 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e24109870 0x7f0e24114b60 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0e14009930 tx=0x7f0e1402a4d0 comp rx=0 tx=0).stop 2024-04-26T02:02:30.220 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.220+0000 7f0e29600640 1 -- 172.21.15.151:0/1972304604 shutdown_connections 2024-04-26T02:02:30.220 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.220+0000 7f0e29600640 1 --2- 172.21.15.151:0/1972304604 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0e241150a0 0x7f0e24117490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:30.220 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.220+0000 7f0e29600640 1 --2- 172.21.15.151:0/1972304604 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e24109870 0x7f0e24114b60 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:30.220 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.220+0000 7f0e29600640 1 --2- 172.21.15.151:0/1972304604 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0e24108f50 0x7f0e24109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:30.220 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.220+0000 7f0e29600640 1 -- 172.21.15.151:0/1972304604 >> 172.21.15.151:0/1972304604 conn(0x7f0e2406d020 msgr2=0x7f0e2406d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:30.220 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.220+0000 7f0e29600640 1 -- 172.21.15.151:0/1972304604 shutdown_connections 2024-04-26T02:02:30.220 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.220+0000 7f0e29600640 1 -- 172.21.15.151:0/1972304604 wait complete. 2024-04-26T02:02:30.221 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.222+0000 7f0e29600640 1 Processor -- start 2024-04-26T02:02:30.221 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.222+0000 7f0e29600640 1 -- start start 2024-04-26T02:02:30.221 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.222+0000 7f0e29600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0e24108f50 0x7f0e24113f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:30.221 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.222+0000 7f0e29600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0e24109870 0x7f0e24114440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:30.221 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.222+0000 7f0e29600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e241150a0 0x7f0e241abde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:30.221 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.222+0000 7f0e29600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0e2411aa20 con 0x7f0e24109870 2024-04-26T02:02:30.221 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.222+0000 7f0e29600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0e2411a8a0 con 0x7f0e24108f50 2024-04-26T02:02:30.221 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.222+0000 7f0e29600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0e2411aba0 con 0x7f0e241150a0 2024-04-26T02:02:30.223 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.222+0000 7f0e22000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0e24109870 0x7f0e24114440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:30.223 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.222+0000 7f0e22000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0e24109870 0x7f0e24114440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:46902/0 (socket says 172.21.15.151:46902) 2024-04-26T02:02:30.225 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.222+0000 7f0e22000640 1 -- 172.21.15.151:0/512251066 learned_addr learned my addr 172.21.15.151:0/512251066 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:02:30.225 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.222+0000 7f0e23400640 1 --2- 172.21.15.151:0/512251066 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e241150a0 0x7f0e241abde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:30.226 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.223+0000 7f0e22a00640 1 --2- 172.21.15.151:0/512251066 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0e24108f50 0x7f0e24113f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:30.226 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.223+0000 7f0e23400640 1 -- 172.21.15.151:0/512251066 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0e24108f50 msgr2=0x7f0e24113f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:30.226 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.223+0000 7f0e23400640 1 --2- 172.21.15.151:0/512251066 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0e24108f50 0x7f0e24113f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:30.226 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.223+0000 7f0e23400640 1 -- 172.21.15.151:0/512251066 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0e24109870 msgr2=0x7f0e24114440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:30.226 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.223+0000 7f0e23400640 1 --2- 172.21.15.151:0/512251066 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0e24109870 0x7f0e24114440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:30.226 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.223+0000 7f0e23400640 1 -- 172.21.15.151:0/512251066 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e241ac510 con 0x7f0e241150a0 2024-04-26T02:02:30.226 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.223+0000 7f0e22000640 1 --2- 172.21.15.151:0/512251066 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0e24109870 0x7f0e24114440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:02:30.226 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.223+0000 7f0e23400640 1 --2- 172.21.15.151:0/512251066 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e241150a0 0x7f0e241abde0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c00e940 tx=0x7f0e1c00ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:30.226 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.224+0000 7f0e1b400640 1 -- 172.21.15.151:0/512251066 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c004d20 con 0x7f0e241150a0 2024-04-26T02:02:30.226 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.224+0000 7f0e1b400640 1 -- 172.21.15.151:0/512251066 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e1c005020 con 0x7f0e241150a0 2024-04-26T02:02:30.226 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.224+0000 7f0e29600640 1 -- 172.21.15.151:0/512251066 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e241ac800 con 0x7f0e241150a0 2024-04-26T02:02:30.226 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.224+0000 7f0e1b400640 1 -- 172.21.15.151:0/512251066 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c0116d0 con 0x7f0e241150a0 2024-04-26T02:02:30.226 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.224+0000 7f0e29600640 1 -- 172.21.15.151:0/512251066 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e241b4090 con 0x7f0e241150a0 2024-04-26T02:02:30.228 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.229+0000 7f0e1b400640 1 -- 172.21.15.151:0/512251066 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f0e1c005460 con 0x7f0e241150a0 2024-04-26T02:02:30.228 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.229+0000 7f0e2c374740 1 Processor -- start 2024-04-26T02:02:30.228 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.229+0000 7f0e2c374740 1 -- start start 2024-04-26T02:02:30.228 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.230+0000 7f0e2c374740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562de9964820 0x562de9964c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:30.229 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.230+0000 7f0e2c374740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562de9965140 0x562de996f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:30.229 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.230+0000 7f0e2c374740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562de996fa00 0x562de9971df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:30.229 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.230+0000 7f0e2c374740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562de98ca900 con 0x562de9965140 2024-04-26T02:02:30.229 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.230+0000 7f0e2c374740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562de9870ef0 con 0x562de996fa00 2024-04-26T02:02:30.229 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.230+0000 7f0e2c374740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562de98c68c0 con 0x562de9964820 2024-04-26T02:02:30.230 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.230+0000 7f0e22a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562de9965140 0x562de996f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:30.230 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.230+0000 7f0e22000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562de9964820 0x562de9964c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:30.230 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.230+0000 7f0e22000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562de9964820 0x562de9964c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:59980/0 (socket says 172.21.15.151:59980) 2024-04-26T02:02:30.230 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.230+0000 7f0e22a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562de9965140 0x562de996f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:46918/0 (socket says 172.21.15.151:46918) 2024-04-26T02:02:30.230 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.230+0000 7f0e22000640 1 -- 172.21.15.151:0/942899507 learned_addr learned my addr 172.21.15.151:0/942899507 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:02:30.231 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.230+0000 7f0e22000640 1 --2- 172.21.15.151:0/942899507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562de996fa00 0x562de9971df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:30.231 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.231+0000 7f0e22a00640 1 -- 172.21.15.151:0/942899507 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562de9964820 msgr2=0x562de9964c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:30.231 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.231+0000 7f0e1b400640 1 --2- 172.21.15.151:0/512251066 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f0e080a3e40 0x7f0e080a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:30.231 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.231+0000 7f0e22a00640 1 --2- 172.21.15.151:0/942899507 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562de9964820 0x562de9964c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:30.231 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.231+0000 7f0e22a00640 1 -- 172.21.15.151:0/942899507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562de996fa00 msgr2=0x562de9971df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:30.231 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.231+0000 7f0e22a00640 1 --2- 172.21.15.151:0/942899507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562de996fa00 0x562de9971df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:30.231 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.231+0000 7f0e22a00640 1 -- 172.21.15.151:0/942899507 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562de95dd210 con 0x562de9965140 2024-04-26T02:02:30.231 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.231+0000 7f0e1b400640 1 -- 172.21.15.151:0/512251066 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8455+0+0 (secure 0 0 0) 0x7f0e1c0d0830 con 0x7f0e241150a0 2024-04-26T02:02:30.231 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.231+0000 7f0e22a00640 1 --2- 172.21.15.151:0/942899507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562de9965140 0x562de996f4c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0e1000bbe0 tx=0x7f0e1000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e99cf526d8d55264 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:30.233 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.231+0000 7f0e0f400640 1 -- 172.21.15.151:0/942899507 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e10015070 con 0x562de9965140 2024-04-26T02:02:30.233 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.231+0000 7f0e0f400640 1 -- 172.21.15.151:0/942899507 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e100035f0 con 0x562de9965140 2024-04-26T02:02:30.233 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.231+0000 7f0e0f400640 1 -- 172.21.15.151:0/942899507 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e100116a0 con 0x562de9965140 2024-04-26T02:02:30.233 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.232+0000 7f0e23400640 1 --2- 172.21.15.151:0/512251066 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f0e080a3e40 0x7f0e080a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:30.233 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.232+0000 7f0e2c374740 1 -- 172.21.15.151:0/942899507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562de9965140 msgr2=0x562de996f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:30.233 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.232+0000 7f0e2c374740 1 --2- 172.21.15.151:0/942899507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562de9965140 0x562de996f4c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0e1000bbe0 tx=0x7f0e1000f730 comp rx=0 tx=0).stop 2024-04-26T02:02:30.233 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.233+0000 7f0e23400640 1 --2- 172.21.15.151:0/512251066 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f0e080a3e40 0x7f0e080a6300 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c0a4ef0 tx=0x7f0e1c005f50 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:30.233 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.233+0000 7f0e2c374740 1 -- 172.21.15.151:0/942899507 shutdown_connections 2024-04-26T02:02:30.234 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.233+0000 7f0e2c374740 1 --2- 172.21.15.151:0/942899507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562de996fa00 0x562de9971df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:30.234 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.233+0000 7f0e2c374740 1 --2- 172.21.15.151:0/942899507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562de9965140 0x562de996f4c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:30.234 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.233+0000 7f0e2c374740 1 --2- 172.21.15.151:0/942899507 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562de9964820 0x562de9964c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:30.234 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.233+0000 7f0e2c374740 1 -- 172.21.15.151:0/942899507 >> 172.21.15.151:0/942899507 conn(0x562de9960d30 msgr2=0x562de9961110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:30.234 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.233+0000 7f0e2c374740 1 -- 172.21.15.151:0/942899507 shutdown_connections 2024-04-26T02:02:30.234 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.233+0000 7f0e2c374740 1 -- 172.21.15.151:0/942899507 wait complete. 2024-04-26T02:02:30.234 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.233+0000 7f0e2c374740 1 -- start start 2024-04-26T02:02:30.234 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.234+0000 7f0e2c374740 1 Processor -- start 2024-04-26T02:02:30.237 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.234+0000 7f0e2c374740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562de9964820 0x562de996c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:30.237 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.234+0000 7f0e2c374740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562de9965140 0x562de996cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:30.237 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.234+0000 7f0e2c374740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562de996fa00 0x562de996d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:30.237 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.234+0000 7f0e2c374740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562de9972630 con 0x562de996fa00 2024-04-26T02:02:30.237 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.234+0000 7f0e2c374740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562de9884310 con 0x562de9964820 2024-04-26T02:02:30.237 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.234+0000 7f0e2c374740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562de99727b0 con 0x562de9965140 2024-04-26T02:02:30.237 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.234+0000 7f0e2c374740 10 client.0 _ll_register_callbacks cb 0x562de9852cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:02:30.237 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.235+0000 7f0e22000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562de9964820 0x562de996c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:30.237 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.235+0000 7f0e22000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562de9964820 0x562de996c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:46682/0 (socket says 172.21.15.151:46682) 2024-04-26T02:02:30.237 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.235+0000 7f0e22000640 1 -- 172.21.15.151:0/3210564794 learned_addr learned my addr 172.21.15.151:0/3210564794 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:02:30.238 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.235+0000 7f0e22000640 1 --2- 172.21.15.151:0/3210564794 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562de996fa00 0x562de996d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:30.238 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.235+0000 7f0e22000640 1 -- 172.21.15.151:0/3210564794 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562de9965140 msgr2=0x562de996cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:30.238 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.235+0000 7f0e22000640 1 --2- 172.21.15.151:0/3210564794 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562de9965140 0x562de996cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:30.238 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.235+0000 7f0e22000640 1 -- 172.21.15.151:0/3210564794 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562de996fa00 msgr2=0x562de996d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:30.238 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.235+0000 7f0e22000640 1 --2- 172.21.15.151:0/3210564794 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562de996fa00 0x562de996d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:30.238 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.235+0000 7f0e22000640 1 -- 172.21.15.151:0/3210564794 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562de96a9620 con 0x562de9964820 2024-04-26T02:02:30.238 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.235+0000 7f0e22000640 1 --2- 172.21.15.151:0/3210564794 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562de9964820 0x562de996c9f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0e14005ab0 tx=0x7f0e14009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d63dcdfe0e49f1ab server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:30.238 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.236+0000 7f0e03e00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T02:02:30.238 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.236+0000 7f0e03e00640 1 -- 172.21.15.151:0/3210564794 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e14035070 con 0x562de9964820 2024-04-26T02:02:30.238 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.236+0000 7f0e03e00640 1 -- 172.21.15.151:0/3210564794 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8455+0+0 (secure 0 0 0) 0x7f0e14005020 con 0x562de9964820 2024-04-26T02:02:30.238 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.236+0000 7f0e2c374740 1 -- 172.21.15.151:0/3210564794 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e100035f0 con 0x562de9964820 2024-04-26T02:02:30.238 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.236+0000 7f0e03e00640 1 -- 172.21.15.151:0/3210564794 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f0e14005220 con 0x562de9964820 2024-04-26T02:02:30.238 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.236+0000 7f0e2c374740 10 client.26195 fetch_fsmap learned FSMap version 35 2024-04-26T02:02:30.239 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.236+0000 7f0e2c374740 1 -- 172.21.15.151:0/3210564794 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562de997e8f0 con 0x562de9964820 2024-04-26T02:02:30.239 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.236+0000 7f0e03e00640 1 -- 172.21.15.151:0/3210564794 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e100035f0 con 0x562de9964820 2024-04-26T02:02:30.239 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.237+0000 7f0e03e00640 1 -- 172.21.15.151:0/3210564794 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 35) ==== 8338+0+0 (secure 0 0 0) 0x7f0e140316b0 con 0x562de9964820 2024-04-26T02:02:30.239 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.237+0000 7f0e2c374740 10 client.26195 fetch_fsmap finished waiting for FSMap version 35 2024-04-26T02:02:30.239 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.237+0000 7f0e2c374740 10 client.26195 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:02:30.239 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.237+0000 7f0e2c374740 20 client.26195 populate_metadata read hostname 'smithi151' 2024-04-26T02:02:30.239 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.237+0000 7f0e2c374740 1 --2- 172.21.15.151:0/3210564794 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x562de997fdd0 0x562de9982260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:30.239 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.237+0000 7f0e2c374740 4 client.26195 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:02:30.239 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.237+0000 7f0e2c374740 1 -- 172.21.15.151:0/3210564794 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562de98de8b0 con 0x562de997fdd0 2024-04-26T02:02:30.239 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.237+0000 7f0e22a00640 1 --2- 172.21.15.151:0/3210564794 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x562de997fdd0 0x562de9982260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:30.239 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.238+0000 7f0e22a00640 1 --2- 172.21.15.151:0/3210564794 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x562de997fdd0 0x562de9982260 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6dbe54089e065cc5 server_cookie=d9ce8f3128ff28b7 in_seq=0 out_seq=0 2024-04-26T02:02:30.239 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.238+0000 7f0e03e00640 10 client.26195 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:02:30.241 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.239+0000 7f0e03e00640 1 -- 172.21.15.151:0/3210564794 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562de98de8b0 con 0x562de997fdd0 2024-04-26T02:02:30.241 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.239+0000 7f0e03e00640 10 client.26195 handle_command_reply: tid=0 2024-04-26T02:02:30.241 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.239+0000 7f0e2c374740 1 client.26195 shutdown 2024-04-26T02:02:30.241 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.239+0000 7f0e2c374740 1 -- 172.21.15.151:0/3210564794 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562de9964820 msgr2=0x562de996c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:30.241 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.239+0000 7f0e2c374740 1 --2- 172.21.15.151:0/3210564794 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562de9964820 0x562de996c9f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0e14005ab0 tx=0x7f0e14009260 comp rx=0 tx=0).stop 2024-04-26T02:02:30.241 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.239+0000 7f0e2c374740 1 -- 172.21.15.151:0/3210564794 shutdown_connections 2024-04-26T02:02:30.241 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.239+0000 7f0e2c374740 1 --2- 172.21.15.151:0/3210564794 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x562de997fdd0 0x562de9982260 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:30.241 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.239+0000 7f0e2c374740 1 --2- 172.21.15.151:0/3210564794 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562de996fa00 0x562de996d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:30.241 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.239+0000 7f0e2c374740 1 --2- 172.21.15.151:0/3210564794 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562de9965140 0x562de996cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:30.241 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.239+0000 7f0e2c374740 1 --2- 172.21.15.151:0/3210564794 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562de9964820 0x562de996c9f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:30.242 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.239+0000 7f0e03e00640 0 client.26195 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:02:30.242 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.239+0000 7f0e2c374740 1 -- 172.21.15.151:0/3210564794 >> 172.21.15.151:0/3210564794 conn(0x562de9960d30 msgr2=0x562de9966a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:30.242 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.239+0000 7f0e2c374740 1 -- 172.21.15.151:0/3210564794 shutdown_connections 2024-04-26T02:02:30.242 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.239+0000 7f0e2c374740 1 -- 172.21.15.151:0/3210564794 wait complete. 2024-04-26T02:02:30.242 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.239+0000 7f0e2c374740 20 client.26195 trim_cache size 0 max 16384 2024-04-26T02:02:30.257 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.258+0000 7f0e2c374740 1 Processor -- start 2024-04-26T02:02:30.257 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.258+0000 7f0e2c374740 1 -- start start 2024-04-26T02:02:30.257 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.258+0000 7f0e2c374740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562de9964820 0x562de997b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:30.257 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.258+0000 7f0e2c374740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562de9965140 0x562de9987b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.258+0000 7f0e2c374740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562de996fa00 0x562de99880b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.258+0000 7f0e2c374740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0e241bf200 con 0x562de9964820 2024-04-26T02:02:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.258+0000 7f0e2c374740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0e241bcad0 con 0x562de996fa00 2024-04-26T02:02:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.258+0000 7f0e2c374740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0e241ba940 con 0x562de9965140 2024-04-26T02:02:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.259+0000 7f0e22a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562de9965140 0x562de9987b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.259+0000 7f0e22a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562de9965140 0x562de9987b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:59986/0 (socket says 172.21.15.151:59986) 2024-04-26T02:02:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.259+0000 7f0e22000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562de9964820 0x562de997b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.259+0000 7f0e22a00640 1 -- 172.21.15.151:0/663922981 learned_addr learned my addr 172.21.15.151:0/663922981 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:02:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.259+0000 7f0e22000640 1 --2- 172.21.15.151:0/663922981 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562de996fa00 0x562de99880b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.259+0000 7f0e22a00640 1 -- 172.21.15.151:0/663922981 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562de996fa00 msgr2=0x562de99880b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.259+0000 7f0e22a00640 1 --2- 172.21.15.151:0/663922981 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562de996fa00 0x562de99880b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.260+0000 7f0e22a00640 1 -- 172.21.15.151:0/663922981 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562de9964820 msgr2=0x562de997b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.260+0000 7f0e22a00640 1 --2- 172.21.15.151:0/663922981 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562de9964820 0x562de997b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.260+0000 7f0e22a00640 1 -- 172.21.15.151:0/663922981 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562de9964f70 con 0x562de9965140 2024-04-26T02:02:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.261+0000 7f0e22a00640 1 --2- 172.21.15.151:0/663922981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562de9965140 0x562de9987b70 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0e100058c0 tx=0x7f0e10003ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=148c89795af9405a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:30.261 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.262+0000 7f0e0ea00640 1 -- 172.21.15.151:0/663922981 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e10015040 con 0x562de9965140 2024-04-26T02:02:30.261 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.262+0000 7f0e0ea00640 1 -- 172.21.15.151:0/663922981 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e100248b0 con 0x562de9965140 2024-04-26T02:02:30.261 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.262+0000 7f0e0ea00640 1 -- 172.21.15.151:0/663922981 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e100035f0 con 0x562de9965140 2024-04-26T02:02:30.262 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.263+0000 7f0e2c374740 1 -- 172.21.15.151:0/663922981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562de9965140 msgr2=0x562de9987b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.263+0000 7f0e2c374740 1 --2- 172.21.15.151:0/663922981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562de9965140 0x562de9987b70 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0e100058c0 tx=0x7f0e10003ef0 comp rx=0 tx=0).stop 2024-04-26T02:02:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.263+0000 7f0e2c374740 1 -- 172.21.15.151:0/663922981 shutdown_connections 2024-04-26T02:02:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.263+0000 7f0e2c374740 1 --2- 172.21.15.151:0/663922981 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562de996fa00 0x562de99880b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.263+0000 7f0e2c374740 1 --2- 172.21.15.151:0/663922981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562de9965140 0x562de9987b70 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.263+0000 7f0e2c374740 1 --2- 172.21.15.151:0/663922981 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562de9964820 0x562de997b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.263+0000 7f0e2c374740 1 -- 172.21.15.151:0/663922981 >> 172.21.15.151:0/663922981 conn(0x562de9960d30 msgr2=0x562de9967ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.263+0000 7f0e2c374740 1 -- 172.21.15.151:0/663922981 shutdown_connections 2024-04-26T02:02:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.264+0000 7f0e2c374740 1 -- 172.21.15.151:0/663922981 wait complete. 2024-04-26T02:02:30.264 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.264+0000 7f0e2c374740 1 -- start start 2024-04-26T02:02:30.264 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.264+0000 7f0e2c374740 1 Processor -- start 2024-04-26T02:02:30.265 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.265+0000 7f0e2c374740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562de9964820 0x562de9977db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:30.265 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.265+0000 7f0e2c374740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562de9965140 0x562de9975140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:30.265 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.265+0000 7f0e2c374740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562de996fa00 0x562de9975680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:30.265 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.265+0000 7f0e2c374740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0e241baf90 con 0x562de9965140 2024-04-26T02:02:30.265 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.265+0000 7f0e2c374740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0e241bc400 con 0x562de996fa00 2024-04-26T02:02:30.265 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.265+0000 7f0e2c374740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0e241bb660 con 0x562de9964820 2024-04-26T02:02:30.265 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.265+0000 7f0e2c374740 10 client.0 _ll_register_callbacks cb 0x7f0e14003a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:02:30.265 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.265+0000 7f0e22000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562de9964820 0x562de9977db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:30.265 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.265+0000 7f0e22000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562de9964820 0x562de9977db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:59992/0 (socket says 172.21.15.151:59992) 2024-04-26T02:02:30.265 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.265+0000 7f0e22a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562de9965140 0x562de9975140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:30.265 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.265+0000 7f0e22000640 1 -- 172.21.15.151:0/891503582 learned_addr learned my addr 172.21.15.151:0/891503582 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:02:30.265 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.265+0000 7f0e22000640 1 --2- 172.21.15.151:0/891503582 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562de996fa00 0x562de9975680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.266+0000 7f0e22000640 1 -- 172.21.15.151:0/891503582 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562de996fa00 msgr2=0x562de9975680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.266+0000 7f0e22000640 1 --2- 172.21.15.151:0/891503582 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562de996fa00 0x562de9975680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.266+0000 7f0e22000640 1 -- 172.21.15.151:0/891503582 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562de9965140 msgr2=0x562de9975140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.266+0000 7f0e22000640 1 --2- 172.21.15.151:0/891503582 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562de9965140 0x562de9975140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.266+0000 7f0e22000640 1 -- 172.21.15.151:0/891503582 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562de9975eb0 con 0x562de9964820 2024-04-26T02:02:30.267 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.266+0000 7f0e22000640 1 --2- 172.21.15.151:0/891503582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562de9964820 0x562de9977db0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0e1404e4a0 tx=0x7f0e14005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e78924241b3121a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:30.267 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.266+0000 7f0e03e00640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-26T02:02:30.267 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.266+0000 7f0e03e00640 1 -- 172.21.15.151:0/891503582 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e14035040 con 0x562de9964820 2024-04-26T02:02:30.267 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.266+0000 7f0e03e00640 1 -- 172.21.15.151:0/891503582 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8455+0+0 (secure 0 0 0) 0x7f0e1404e110 con 0x562de9964820 2024-04-26T02:02:30.268 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.266+0000 7f0e2c374740 1 -- 172.21.15.151:0/891503582 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e100248b0 con 0x562de9964820 2024-04-26T02:02:30.268 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.267+0000 7f0e03e00640 1 -- 172.21.15.151:0/891503582 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f0e140053c0 con 0x562de9964820 2024-04-26T02:02:30.268 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.267+0000 7f0e2c374740 10 client.26230 fetch_fsmap learned FSMap version 35 2024-04-26T02:02:30.268 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.267+0000 7f0e2c374740 1 -- 172.21.15.151:0/891503582 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562de99e63f0 con 0x562de9964820 2024-04-26T02:02:30.268 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.267+0000 7f0e03e00640 1 -- 172.21.15.151:0/891503582 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e100248b0 con 0x562de9964820 2024-04-26T02:02:30.268 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.267+0000 7f0e03e00640 1 -- 172.21.15.151:0/891503582 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 35) ==== 8338+0+0 (secure 0 0 0) 0x7f0e14004db0 con 0x562de9964820 2024-04-26T02:02:30.268 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.267+0000 7f0e2c374740 10 client.26230 fetch_fsmap finished waiting for FSMap version 35 2024-04-26T02:02:30.268 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.267+0000 7f0e2c374740 10 client.26230 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:02:30.268 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.267+0000 7f0e2c374740 20 client.26230 populate_metadata read hostname 'smithi151' 2024-04-26T02:02:30.268 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.267+0000 7f0e2c374740 1 --2- 172.21.15.151:0/891503582 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x562de99e6a60 0x562de99e8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:30.268 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.267+0000 7f0e2c374740 4 client.26230 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T02:02:30.268 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.267+0000 7f0e2c374740 1 -- 172.21.15.151:0/891503582 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562de98d8320 con 0x562de99e6a60 2024-04-26T02:02:30.268 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.268+0000 7f0e22a00640 1 --2- 172.21.15.151:0/891503582 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x562de99e6a60 0x562de99e8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:30.269 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.269+0000 7f0e22a00640 1 --2- 172.21.15.151:0/891503582 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x562de99e6a60 0x562de99e8e50 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=585ae2700c26259a server_cookie=5cd8fa095eac8915 in_seq=0 out_seq=0 2024-04-26T02:02:30.269 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.269+0000 7f0e03e00640 10 client.26230 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:02:30.281 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.282+0000 7f0e03e00640 1 -- 172.21.15.151:0/891503582 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562de98d8320 con 0x562de99e6a60 2024-04-26T02:02:30.281 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.282+0000 7f0e03e00640 10 client.26230 handle_command_reply: tid=0 2024-04-26T02:02:30.282 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.282+0000 7f0e2c374740 1 client.26230 shutdown 2024-04-26T02:02:30.282 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.282+0000 7f0e2c374740 1 -- 172.21.15.151:0/891503582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562de9964820 msgr2=0x562de9977db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:30.282 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.282+0000 7f0e2c374740 1 --2- 172.21.15.151:0/891503582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562de9964820 0x562de9977db0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0e1404e4a0 tx=0x7f0e14005fd0 comp rx=0 tx=0).stop 2024-04-26T02:02:30.282 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.282+0000 7f0e2c374740 1 -- 172.21.15.151:0/891503582 shutdown_connections 2024-04-26T02:02:30.282 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.282+0000 7f0e2c374740 1 --2- 172.21.15.151:0/891503582 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x562de99e6a60 0x562de99e8e50 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:30.282 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.282+0000 7f0e2c374740 1 --2- 172.21.15.151:0/891503582 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562de996fa00 0x562de9975680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:30.282 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.282+0000 7f0e2c374740 1 --2- 172.21.15.151:0/891503582 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562de9965140 0x562de9975140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:30.282 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.282+0000 7f0e2c374740 1 --2- 172.21.15.151:0/891503582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562de9964820 0x562de9977db0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:30.282 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.282+0000 7f0e03e00640 0 client.26230 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:02:30.282 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.282+0000 7f0e2c374740 1 -- 172.21.15.151:0/891503582 >> 172.21.15.151:0/891503582 conn(0x562de9960d30 msgr2=0x562de99661d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:30.282 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.282+0000 7f0e2c374740 1 -- 172.21.15.151:0/891503582 shutdown_connections 2024-04-26T02:02:30.282 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.282+0000 7f0e2c374740 1 -- 172.21.15.151:0/891503582 wait complete. 2024-04-26T02:02:30.282 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.282+0000 7f0e2c374740 20 client.26230 trim_cache size 0 max 16384 2024-04-26T02:02:30.283 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:{ 2024-04-26T02:02:30.283 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "message": "", 2024-04-26T02:02:30.283 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "return_code": 0 2024-04-26T02:02:30.283 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:} 2024-04-26T02:02:30.284 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.283+0000 7f0e29600640 1 -- 172.21.15.151:0/512251066 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f0e080a3e40 msgr2=0x7f0e080a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:30.284 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.283+0000 7f0e29600640 1 --2- 172.21.15.151:0/512251066 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f0e080a3e40 0x7f0e080a6300 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c0a4ef0 tx=0x7f0e1c005f50 comp rx=0 tx=0).stop 2024-04-26T02:02:30.284 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.283+0000 7f0e29600640 1 -- 172.21.15.151:0/512251066 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e241150a0 msgr2=0x7f0e241abde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:30.284 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.283+0000 7f0e29600640 1 --2- 172.21.15.151:0/512251066 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e241150a0 0x7f0e241abde0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c00e940 tx=0x7f0e1c00ee00 comp rx=0 tx=0).stop 2024-04-26T02:02:30.284 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.283+0000 7f0e29600640 1 -- 172.21.15.151:0/512251066 shutdown_connections 2024-04-26T02:02:30.284 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.283+0000 7f0e29600640 1 --2- 172.21.15.151:0/512251066 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f0e080a3e40 0x7f0e080a6300 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:30.284 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.283+0000 7f0e29600640 1 --2- 172.21.15.151:0/512251066 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e241150a0 0x7f0e241abde0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:30.284 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.283+0000 7f0e29600640 1 --2- 172.21.15.151:0/512251066 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0e24109870 0x7f0e24114440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:30.284 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.283+0000 7f0e29600640 1 --2- 172.21.15.151:0/512251066 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0e24108f50 0x7f0e24113f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:30.285 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.283+0000 7f0e29600640 1 -- 172.21.15.151:0/512251066 >> 172.21.15.151:0/512251066 conn(0x7f0e2406d020 msgr2=0x7f0e2410a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:30.285 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.283+0000 7f0e29600640 1 -- 172.21.15.151:0/512251066 shutdown_connections 2024-04-26T02:02:30.285 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:30.283+0000 7f0e29600640 1 -- 172.21.15.151:0/512251066 wait complete. 2024-04-26T02:02:30.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:02:30 smithi172 ceph-mon[32785]: pgmap v278: 97 pgs: 97 active+clean; 113 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.5 MiB/s wr, 496 op/s 2024-04-26T02:02:30.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:02:30 smithi165 ceph-mon[32648]: pgmap v278: 97 pgs: 97 active+clean; 113 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.5 MiB/s wr, 496 op/s 2024-04-26T02:02:30.873 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:02:30 smithi151 ceph-mon[28057]: pgmap v278: 97 pgs: 97 active+clean; 113 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.5 MiB/s wr, 496 op/s 2024-04-26T02:02:31.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:02:31 smithi165 ceph-mon[32648]: pgmap v279: 97 pgs: 97 active+clean; 113 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 1016 KiB/s wr, 333 op/s 2024-04-26T02:02:31.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:02:31 smithi151 ceph-mon[28057]: pgmap v279: 97 pgs: 97 active+clean; 113 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 1016 KiB/s wr, 333 op/s 2024-04-26T02:02:32.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:02:31 smithi172 ceph-mon[32785]: pgmap v279: 97 pgs: 97 active+clean; 113 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 1016 KiB/s wr, 333 op/s 2024-04-26T02:02:34.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:02:33 smithi172 ceph-mon[32785]: pgmap v280: 97 pgs: 97 active+clean; 108 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 1.3 MiB/s wr, 427 op/s 2024-04-26T02:02:34.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:02:33 smithi165 ceph-mon[32648]: pgmap v280: 97 pgs: 97 active+clean; 108 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 1.3 MiB/s wr, 427 op/s 2024-04-26T02:02:34.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:02:33 smithi151 ceph-mon[28057]: pgmap v280: 97 pgs: 97 active+clean; 108 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 1.3 MiB/s wr, 427 op/s 2024-04-26T02:02:34.581 INFO:tasks.workunit.client.0.smithi151.stdout:copying to largest ever: 0x63ed6b9 2024-04-26T02:02:36.123 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:02:35 smithi151 ceph-mon[28057]: pgmap v281: 97 pgs: 97 active+clean; 108 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.4 MiB/s wr, 446 op/s 2024-04-26T02:02:36.291 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:02:35 smithi172 ceph-mon[32785]: pgmap v281: 97 pgs: 97 active+clean; 108 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.4 MiB/s wr, 446 op/s 2024-04-26T02:02:36.349 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:02:35 smithi165 ceph-mon[32648]: pgmap v281: 97 pgs: 97 active+clean; 108 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.4 MiB/s wr, 446 op/s 2024-04-26T02:02:38.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:02:37 smithi172 ceph-mon[32785]: pgmap v282: 97 pgs: 97 active+clean; 108 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 1.0 MiB/s wr, 342 op/s 2024-04-26T02:02:38.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:02:37 smithi165 ceph-mon[32648]: pgmap v282: 97 pgs: 97 active+clean; 108 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 1.0 MiB/s wr, 342 op/s 2024-04-26T02:02:38.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:02:37 smithi151 ceph-mon[28057]: pgmap v282: 97 pgs: 97 active+clean; 108 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 1.0 MiB/s wr, 342 op/s 2024-04-26T02:02:40.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:02:39 smithi172 ceph-mon[32785]: pgmap v283: 97 pgs: 97 active+clean; 111 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 1.6 MiB/s wr, 514 op/s 2024-04-26T02:02:40.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:02:39 smithi165 ceph-mon[32648]: pgmap v283: 97 pgs: 97 active+clean; 111 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 1.6 MiB/s wr, 514 op/s 2024-04-26T02:02:40.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:02:39 smithi151 ceph-mon[28057]: pgmap v283: 97 pgs: 97 active+clean; 111 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 1.6 MiB/s wr, 514 op/s 2024-04-26T02:02:42.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:02:41 smithi172 ceph-mon[32785]: pgmap v284: 97 pgs: 97 active+clean; 111 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 1.1 MiB/s wr, 345 op/s 2024-04-26T02:02:42.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:02:41 smithi165 ceph-mon[32648]: pgmap v284: 97 pgs: 97 active+clean; 111 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 1.1 MiB/s wr, 345 op/s 2024-04-26T02:02:42.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:02:41 smithi151 ceph-mon[28057]: pgmap v284: 97 pgs: 97 active+clean; 111 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 1.1 MiB/s wr, 345 op/s 2024-04-26T02:02:43.435 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.435+0000 7f0f8b400640 1 -- 172.21.15.151:0/1557921068 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f84103b00 msgr2=0x7f0f8410bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:43.435 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.435+0000 7f0f8b400640 1 --2- 172.21.15.151:0/1557921068 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f84103b00 0x7f0f8410bfe0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0f70009900 tx=0x7f0f7002a520 comp rx=0 tx=0).stop 2024-04-26T02:02:43.435 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.435+0000 7f0f8b400640 1 -- 172.21.15.151:0/1557921068 shutdown_connections 2024-04-26T02:02:43.435 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.435+0000 7f0f8b400640 1 --2- 172.21.15.151:0/1557921068 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f8410c520 0x7f0f8410e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:43.435 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.435+0000 7f0f8b400640 1 --2- 172.21.15.151:0/1557921068 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f84103b00 0x7f0f8410bfe0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:43.435 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.435+0000 7f0f8b400640 1 --2- 172.21.15.151:0/1557921068 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f841031e0 0x7f0f841035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:43.435 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.435+0000 7f0f8b400640 1 -- 172.21.15.151:0/1557921068 >> 172.21.15.151:0/1557921068 conn(0x7f0f84077fd0 msgr2=0x7f0f840783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:43.435 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.435+0000 7f0f8b400640 1 -- 172.21.15.151:0/1557921068 shutdown_connections 2024-04-26T02:02:43.435 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.435+0000 7f0f8b400640 1 -- 172.21.15.151:0/1557921068 wait complete. 2024-04-26T02:02:43.437 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.436+0000 7f0f8b400640 1 Processor -- start 2024-04-26T02:02:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.437+0000 7f0f8b400640 1 -- start start 2024-04-26T02:02:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.437+0000 7f0f8b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f841031e0 0x7f0f84100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.437+0000 7f0f8b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f84103b00 0x7f0f84101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.437+0000 7f0f8b400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f8410c520 0x7f0f841018b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.437+0000 7f0f8b400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0f841121e0 con 0x7f0f841031e0 2024-04-26T02:02:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.437+0000 7f0f8b400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0f84112060 con 0x7f0f84103b00 2024-04-26T02:02:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.437+0000 7f0f8b400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0f84112360 con 0x7f0f8410c520 2024-04-26T02:02:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.437+0000 7f0f83e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f84103b00 0x7f0f84101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.437+0000 7f0f88a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f841031e0 0x7f0f84100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.437+0000 7f0f83e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f84103b00 0x7f0f84101370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:54756/0 (socket says 172.21.15.151:54756) 2024-04-26T02:02:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.437+0000 7f0f83e00640 1 -- 172.21.15.151:0/2066084919 learned_addr learned my addr 172.21.15.151:0/2066084919 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:02:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.437+0000 7f0f89400640 1 --2- 172.21.15.151:0/2066084919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f8410c520 0x7f0f841018b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.438+0000 7f0f88a00640 1 -- 172.21.15.151:0/2066084919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f8410c520 msgr2=0x7f0f841018b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.438+0000 7f0f88a00640 1 --2- 172.21.15.151:0/2066084919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f8410c520 0x7f0f841018b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.438+0000 7f0f88a00640 1 -- 172.21.15.151:0/2066084919 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f84103b00 msgr2=0x7f0f84101370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.438+0000 7f0f88a00640 1 --2- 172.21.15.151:0/2066084919 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f84103b00 0x7f0f84101370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.438+0000 7f0f88a00640 1 -- 172.21.15.151:0/2066084919 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f841a6b20 con 0x7f0f841031e0 2024-04-26T02:02:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.438+0000 7f0f83e00640 1 --2- 172.21.15.151:0/2066084919 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f84103b00 0x7f0f84101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:02:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.438+0000 7f0f89400640 1 --2- 172.21.15.151:0/2066084919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f8410c520 0x7f0f841018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:02:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.438+0000 7f0f88a00640 1 --2- 172.21.15.151:0/2066084919 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f841031e0 0x7f0f84100e30 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0f7800d6d0 tx=0x7f0f7800db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.439+0000 7f0f81600640 1 -- 172.21.15.151:0/2066084919 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f78017070 con 0x7f0f841031e0 2024-04-26T02:02:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.439+0000 7f0f8b400640 1 -- 172.21.15.151:0/2066084919 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f841a6e10 con 0x7f0f841031e0 2024-04-26T02:02:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.439+0000 7f0f81600640 1 -- 172.21.15.151:0/2066084919 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f78009820 con 0x7f0f841031e0 2024-04-26T02:02:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.439+0000 7f0f81600640 1 -- 172.21.15.151:0/2066084919 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f78004c40 con 0x7f0f841031e0 2024-04-26T02:02:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.439+0000 7f0f8b400640 1 -- 172.21.15.151:0/2066084919 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f841a7320 con 0x7f0f841031e0 2024-04-26T02:02:43.441 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.442+0000 7f0f8e364740 1 Processor -- start 2024-04-26T02:02:43.441 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.443+0000 7f0f8e364740 1 -- start start 2024-04-26T02:02:43.442 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.443+0000 7f0f8e364740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559aa1fa7820 0x559aa1fa7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:43.442 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.443+0000 7f0f8e364740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559aa1fa8140 0x559aa1fb24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:43.442 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.443+0000 7f0f8e364740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559aa1fb2a00 0x559aa1fb4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:43.442 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.443+0000 7f0f8e364740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559aa1f0d900 con 0x559aa1fa7820 2024-04-26T02:02:43.442 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.443+0000 7f0f8e364740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559aa1eb3ef0 con 0x559aa1fb2a00 2024-04-26T02:02:43.442 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.443+0000 7f0f8e364740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x559aa1f098c0 con 0x559aa1fa8140 2024-04-26T02:02:43.442 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.443+0000 7f0f81600640 1 -- 172.21.15.151:0/2066084919 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f0f78005220 con 0x7f0f841031e0 2024-04-26T02:02:43.443 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.443+0000 7f0f89400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559aa1fa7820 0x559aa1fa7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:43.443 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.443+0000 7f0f83e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559aa1fb2a00 0x559aa1fb4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:43.443 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.443+0000 7f0f89400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559aa1fa7820 0x559aa1fa7c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:48242/0 (socket says 172.21.15.151:48242) 2024-04-26T02:02:43.443 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.443+0000 7f0f89400640 1 -- 172.21.15.151:0/2510313441 learned_addr learned my addr 172.21.15.151:0/2510313441 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:02:43.444 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.444+0000 7f0f88a00640 1 --2- 172.21.15.151:0/2510313441 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559aa1fa8140 0x559aa1fb24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:43.444 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.444+0000 7f0f83e00640 1 -- 172.21.15.151:0/2510313441 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559aa1fa8140 msgr2=0x559aa1fb24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:43.444 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.444+0000 7f0f83e00640 1 --2- 172.21.15.151:0/2510313441 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559aa1fa8140 0x559aa1fb24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:43.444 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.444+0000 7f0f83e00640 1 -- 172.21.15.151:0/2510313441 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559aa1fa7820 msgr2=0x559aa1fa7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:43.444 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.444+0000 7f0f83e00640 1 --2- 172.21.15.151:0/2510313441 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559aa1fa7820 0x559aa1fa7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:43.444 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.444+0000 7f0f83e00640 1 -- 172.21.15.151:0/2510313441 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559aa1c20210 con 0x559aa1fb2a00 2024-04-26T02:02:43.444 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.444+0000 7f0f83e00640 1 --2- 172.21.15.151:0/2510313441 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559aa1fb2a00 0x559aa1fb4df0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0f70000c00 tx=0x7f0f70003670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5999ab84093182a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:43.445 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.445+0000 7f0f75c00640 1 -- 172.21.15.151:0/2510313441 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f70040070 con 0x559aa1fb2a00 2024-04-26T02:02:43.445 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.445+0000 7f0f75c00640 1 -- 172.21.15.151:0/2510313441 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f70003860 con 0x559aa1fb2a00 2024-04-26T02:02:43.445 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.445+0000 7f0f75c00640 1 -- 172.21.15.151:0/2510313441 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f70035040 con 0x559aa1fb2a00 2024-04-26T02:02:43.445 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.445+0000 7f0f81600640 1 --2- 172.21.15.151:0/2066084919 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f0f6c0a3df0 0x7f0f6c0a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:43.445 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.445+0000 7f0f81600640 1 -- 172.21.15.151:0/2066084919 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8455+0+0 (secure 0 0 0) 0x7f0f780d0200 con 0x7f0f841031e0 2024-04-26T02:02:43.446 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.445+0000 7f0f89400640 1 --2- 172.21.15.151:0/2066084919 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f0f6c0a3df0 0x7f0f6c0a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:43.446 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.446+0000 7f0f8e364740 1 -- 172.21.15.151:0/2510313441 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559aa1fb2a00 msgr2=0x559aa1fb4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:43.446 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.446+0000 7f0f8e364740 1 --2- 172.21.15.151:0/2510313441 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559aa1fb2a00 0x559aa1fb4df0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0f70000c00 tx=0x7f0f70003670 comp rx=0 tx=0).stop 2024-04-26T02:02:43.446 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.446+0000 7f0f8e364740 1 -- 172.21.15.151:0/2510313441 shutdown_connections 2024-04-26T02:02:43.446 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.446+0000 7f0f8e364740 1 --2- 172.21.15.151:0/2510313441 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559aa1fb2a00 0x559aa1fb4df0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:43.446 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.446+0000 7f0f8e364740 1 --2- 172.21.15.151:0/2510313441 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559aa1fa8140 0x559aa1fb24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:43.446 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.446+0000 7f0f8e364740 1 --2- 172.21.15.151:0/2510313441 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559aa1fa7820 0x559aa1fa7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:43.446 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.446+0000 7f0f8e364740 1 -- 172.21.15.151:0/2510313441 >> 172.21.15.151:0/2510313441 conn(0x559aa1fa3d30 msgr2=0x559aa1fa4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:43.446 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.446+0000 7f0f89400640 1 --2- 172.21.15.151:0/2066084919 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f0f6c0a3df0 0x7f0f6c0a62b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c0107e0 tx=0x7f0f7c007520 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:43.446 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.446+0000 7f0f8e364740 1 -- 172.21.15.151:0/2510313441 shutdown_connections 2024-04-26T02:02:43.446 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.446+0000 7f0f8e364740 1 -- 172.21.15.151:0/2510313441 wait complete. 2024-04-26T02:02:43.449 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.447+0000 7f0f8e364740 1 -- start start 2024-04-26T02:02:43.449 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.447+0000 7f0f8e364740 1 Processor -- start 2024-04-26T02:02:43.449 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.448+0000 7f0f8e364740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559aa1fa7820 0x559aa1faf960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:43.449 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.448+0000 7f0f8e364740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559aa1fa8140 0x559aa1fafea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:43.449 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.448+0000 7f0f8e364740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559aa1fb2a00 0x559aa1fb03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:43.449 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.448+0000 7f0f8e364740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559aa1fb55d0 con 0x559aa1fb2a00 2024-04-26T02:02:43.449 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.448+0000 7f0f8e364740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559aa1ec7310 con 0x559aa1fa7820 2024-04-26T02:02:43.449 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.448+0000 7f0f8e364740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x559aa1fb5750 con 0x559aa1fa8140 2024-04-26T02:02:43.449 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.448+0000 7f0f8e364740 10 client.0 _ll_register_callbacks cb 0x559aa1e95cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:02:43.450 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.448+0000 7f0f88a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559aa1fa7820 0x559aa1faf960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:43.450 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.448+0000 7f0f88a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559aa1fa7820 0x559aa1faf960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:54778/0 (socket says 172.21.15.151:54778) 2024-04-26T02:02:43.450 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.448+0000 7f0f88a00640 1 -- 172.21.15.151:0/18678590 learned_addr learned my addr 172.21.15.151:0/18678590 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:02:43.450 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.448+0000 7f0f83e00640 1 --2- 172.21.15.151:0/18678590 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559aa1fa8140 0x559aa1fafea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:43.450 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.448+0000 7f0f89400640 1 --2- 172.21.15.151:0/18678590 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559aa1fb2a00 0x559aa1fb03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:43.450 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.448+0000 7f0f88a00640 1 -- 172.21.15.151:0/18678590 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559aa1fa8140 msgr2=0x559aa1fafea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:43.450 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.448+0000 7f0f88a00640 1 --2- 172.21.15.151:0/18678590 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559aa1fa8140 0x559aa1fafea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:43.450 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.448+0000 7f0f88a00640 1 -- 172.21.15.151:0/18678590 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559aa1fb2a00 msgr2=0x559aa1fb03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:43.450 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.448+0000 7f0f88a00640 1 --2- 172.21.15.151:0/18678590 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559aa1fb2a00 0x559aa1fb03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:43.450 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.448+0000 7f0f88a00640 1 -- 172.21.15.151:0/18678590 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559aa1cec620 con 0x559aa1fa7820 2024-04-26T02:02:43.450 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.448+0000 7f0f88a00640 1 --2- 172.21.15.151:0/18678590 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559aa1fa7820 0x559aa1faf960 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0f78000c00 tx=0x7f0f780985f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed148c7f0bd95f08 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:43.450 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.449+0000 7f0f6a000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T02:02:43.450 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.449+0000 7f0f6a000640 1 -- 172.21.15.151:0/18678590 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f78020040 con 0x559aa1fa7820 2024-04-26T02:02:43.451 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.449+0000 7f0f6a000640 1 -- 172.21.15.151:0/18678590 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8455+0+0 (secure 0 0 0) 0x7f0f78009cd0 con 0x559aa1fa7820 2024-04-26T02:02:43.451 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.449+0000 7f0f8e364740 1 -- 172.21.15.151:0/18678590 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f70003860 con 0x559aa1fa7820 2024-04-26T02:02:43.451 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.449+0000 7f0f6a000640 1 -- 172.21.15.151:0/18678590 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f0f7809bbf0 con 0x559aa1fa7820 2024-04-26T02:02:43.451 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.449+0000 7f0f8e364740 10 client.26210 fetch_fsmap learned FSMap version 35 2024-04-26T02:02:43.451 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.449+0000 7f0f8e364740 1 -- 172.21.15.151:0/18678590 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x559aa1fc1850 con 0x559aa1fa7820 2024-04-26T02:02:43.451 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.449+0000 7f0f6a000640 1 -- 172.21.15.151:0/18678590 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f70003860 con 0x559aa1fa7820 2024-04-26T02:02:43.451 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.449+0000 7f0f6a000640 1 -- 172.21.15.151:0/18678590 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 35) ==== 8338+0+0 (secure 0 0 0) 0x7f0f780cdc10 con 0x559aa1fa7820 2024-04-26T02:02:43.451 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.450+0000 7f0f8e364740 10 client.26210 fetch_fsmap finished waiting for FSMap version 35 2024-04-26T02:02:43.451 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.450+0000 7f0f8e364740 10 client.26210 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:02:43.451 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.450+0000 7f0f8e364740 20 client.26210 populate_metadata read hostname 'smithi151' 2024-04-26T02:02:43.451 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.450+0000 7f0f8e364740 1 --2- 172.21.15.151:0/18678590 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x559aa1fc2d10 0x559aa1fc5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:43.451 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.450+0000 7f0f8e364740 4 client.26210 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:02:43.451 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.450+0000 7f0f8e364740 1 -- 172.21.15.151:0/18678590 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559aa1f218b0 con 0x559aa1fc2d10 2024-04-26T02:02:43.451 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.450+0000 7f0f83e00640 1 --2- 172.21.15.151:0/18678590 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x559aa1fc2d10 0x559aa1fc5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:43.453 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.450+0000 7f0f83e00640 1 --2- 172.21.15.151:0/18678590 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x559aa1fc2d10 0x559aa1fc5160 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d31781a80c40fd2 server_cookie=904c24daace0d637 in_seq=0 out_seq=0 2024-04-26T02:02:43.453 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.450+0000 7f0f6a000640 10 client.26210 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:02:43.453 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.451+0000 7f0f6a000640 1 -- 172.21.15.151:0/18678590 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559aa1f218b0 con 0x559aa1fc2d10 2024-04-26T02:02:43.453 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.451+0000 7f0f6a000640 10 client.26210 handle_command_reply: tid=0 2024-04-26T02:02:43.453 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.451+0000 7f0f8e364740 1 client.26210 shutdown 2024-04-26T02:02:43.453 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.452+0000 7f0f8e364740 1 -- 172.21.15.151:0/18678590 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559aa1fa7820 msgr2=0x559aa1faf960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:43.453 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.452+0000 7f0f8e364740 1 --2- 172.21.15.151:0/18678590 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559aa1fa7820 0x559aa1faf960 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0f78000c00 tx=0x7f0f780985f0 comp rx=0 tx=0).stop 2024-04-26T02:02:43.453 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.452+0000 7f0f8e364740 1 -- 172.21.15.151:0/18678590 shutdown_connections 2024-04-26T02:02:43.453 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.452+0000 7f0f8e364740 1 --2- 172.21.15.151:0/18678590 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x559aa1fc2d10 0x559aa1fc5160 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:43.453 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.452+0000 7f0f8e364740 1 --2- 172.21.15.151:0/18678590 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559aa1fb2a00 0x559aa1fb03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:43.453 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.452+0000 7f0f8e364740 1 --2- 172.21.15.151:0/18678590 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559aa1fa8140 0x559aa1fafea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:43.453 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.452+0000 7f0f6a000640 0 client.26210 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:02:43.454 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.452+0000 7f0f8e364740 1 --2- 172.21.15.151:0/18678590 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559aa1fa7820 0x559aa1faf960 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:43.454 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.452+0000 7f0f8e364740 1 -- 172.21.15.151:0/18678590 >> 172.21.15.151:0/18678590 conn(0x559aa1fa3d30 msgr2=0x559aa1fa9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:43.454 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.452+0000 7f0f8e364740 1 -- 172.21.15.151:0/18678590 shutdown_connections 2024-04-26T02:02:43.454 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.452+0000 7f0f8e364740 1 -- 172.21.15.151:0/18678590 wait complete. 2024-04-26T02:02:43.454 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.452+0000 7f0f8e364740 20 client.26210 trim_cache size 0 max 16384 2024-04-26T02:02:43.473 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.473+0000 7f0f8e364740 1 Processor -- start 2024-04-26T02:02:43.473 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.473+0000 7f0f8e364740 1 -- start start 2024-04-26T02:02:43.474 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.473+0000 7f0f8e364740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559aa1fa7820 0x559aa1fbe490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:43.474 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.473+0000 7f0f8e364740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559aa1fa8140 0x559aa1faa230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:43.474 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.473+0000 7f0f8e364740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559aa1fb2a00 0x559aa1fcaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:43.474 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.474+0000 7f0f8e364740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0f841ae150 con 0x559aa1fa7820 2024-04-26T02:02:43.474 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.474+0000 7f0f8e364740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0f841aba20 con 0x559aa1fa8140 2024-04-26T02:02:43.474 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.474+0000 7f0f8e364740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0f841a98b0 con 0x559aa1fb2a00 2024-04-26T02:02:43.474 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.474+0000 7f0f88a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559aa1fa7820 0x559aa1fbe490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:43.474 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.474+0000 7f0f89400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559aa1fb2a00 0x559aa1fcaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:43.474 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.474+0000 7f0f88a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559aa1fa7820 0x559aa1fbe490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:48264/0 (socket says 172.21.15.151:48264) 2024-04-26T02:02:43.474 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.474+0000 7f0f88a00640 1 -- 172.21.15.151:0/2560293866 learned_addr learned my addr 172.21.15.151:0/2560293866 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:02:43.475 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.474+0000 7f0f83e00640 1 --2- 172.21.15.151:0/2560293866 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559aa1fa8140 0x559aa1faa230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:43.475 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.474+0000 7f0f88a00640 1 -- 172.21.15.151:0/2560293866 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559aa1fb2a00 msgr2=0x559aa1fcaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:43.475 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.474+0000 7f0f88a00640 1 --2- 172.21.15.151:0/2560293866 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559aa1fb2a00 0x559aa1fcaa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:43.475 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.474+0000 7f0f88a00640 1 -- 172.21.15.151:0/2560293866 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559aa1fa8140 msgr2=0x559aa1faa230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:43.475 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.474+0000 7f0f88a00640 1 --2- 172.21.15.151:0/2560293866 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559aa1fa8140 0x559aa1faa230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:43.475 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.474+0000 7f0f88a00640 1 -- 172.21.15.151:0/2560293866 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559aa1fbf900 con 0x559aa1fa7820 2024-04-26T02:02:43.475 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.474+0000 7f0f89400640 1 --2- 172.21.15.151:0/2560293866 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559aa1fb2a00 0x559aa1fcaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:02:43.475 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.474+0000 7f0f83e00640 1 --2- 172.21.15.151:0/2560293866 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559aa1fa8140 0x559aa1faa230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:02:43.475 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.475+0000 7f0f88a00640 1 --2- 172.21.15.151:0/2560293866 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559aa1fa7820 0x559aa1fbe490 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0f7809c220 tx=0x7f0f7809a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49cc9ba408cb3bb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:43.479 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.475+0000 7f0f75200640 1 -- 172.21.15.151:0/2560293866 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f78014020 con 0x559aa1fa7820 2024-04-26T02:02:43.479 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.475+0000 7f0f75200640 1 -- 172.21.15.151:0/2560293866 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f78098a60 con 0x559aa1fa7820 2024-04-26T02:02:43.479 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.475+0000 7f0f75200640 1 -- 172.21.15.151:0/2560293866 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f78020070 con 0x559aa1fa7820 2024-04-26T02:02:43.479 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.476+0000 7f0f8e364740 1 -- 172.21.15.151:0/2560293866 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559aa1fa7820 msgr2=0x559aa1fbe490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:43.479 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.476+0000 7f0f8e364740 1 --2- 172.21.15.151:0/2560293866 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559aa1fa7820 0x559aa1fbe490 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0f7809c220 tx=0x7f0f7809a7a0 comp rx=0 tx=0).stop 2024-04-26T02:02:43.479 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.476+0000 7f0f8e364740 1 -- 172.21.15.151:0/2560293866 shutdown_connections 2024-04-26T02:02:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.476+0000 7f0f8e364740 1 --2- 172.21.15.151:0/2560293866 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559aa1fb2a00 0x559aa1fcaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.476+0000 7f0f8e364740 1 --2- 172.21.15.151:0/2560293866 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559aa1fa8140 0x559aa1faa230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.476+0000 7f0f8e364740 1 --2- 172.21.15.151:0/2560293866 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559aa1fa7820 0x559aa1fbe490 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.476+0000 7f0f8e364740 1 -- 172.21.15.151:0/2560293866 >> 172.21.15.151:0/2560293866 conn(0x559aa1fa3d30 msgr2=0x559aa1fa4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.476+0000 7f0f8e364740 1 -- 172.21.15.151:0/2560293866 shutdown_connections 2024-04-26T02:02:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.476+0000 7f0f8e364740 1 -- 172.21.15.151:0/2560293866 wait complete. 2024-04-26T02:02:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.476+0000 7f0f8e364740 1 -- start start 2024-04-26T02:02:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.477+0000 7f0f8e364740 1 Processor -- start 2024-04-26T02:02:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.477+0000 7f0f8e364740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559aa1fa7820 0x559aa1fbabe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.477+0000 7f0f8e364740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559aa1fa8140 0x559aa1fb7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.477+0000 7f0f8e364740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559aa1fb2a00 0x559aa1fb84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.477+0000 7f0f8e364740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0f841a9f00 con 0x559aa1fb2a00 2024-04-26T02:02:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.477+0000 7f0f8e364740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0f841ab350 con 0x559aa1fa8140 2024-04-26T02:02:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.477+0000 7f0f8e364740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0f841aa5b0 con 0x559aa1fa7820 2024-04-26T02:02:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.477+0000 7f0f8e364740 10 client.0 _ll_register_callbacks cb 0x7f0f780046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:02:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.477+0000 7f0f83e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559aa1fa8140 0x559aa1fb7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.477+0000 7f0f88a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559aa1fa7820 0x559aa1fbabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.477+0000 7f0f83e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559aa1fa8140 0x559aa1fb7f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:54800/0 (socket says 172.21.15.151:54800) 2024-04-26T02:02:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.477+0000 7f0f83e00640 1 -- 172.21.15.151:0/46571592 learned_addr learned my addr 172.21.15.151:0/46571592 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:02:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.478+0000 7f0f89400640 1 --2- 172.21.15.151:0/46571592 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559aa1fb2a00 0x559aa1fb84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.478+0000 7f0f83e00640 1 -- 172.21.15.151:0/46571592 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559aa1fa7820 msgr2=0x559aa1fbabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.478+0000 7f0f83e00640 1 --2- 172.21.15.151:0/46571592 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559aa1fa7820 0x559aa1fbabe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.478+0000 7f0f83e00640 1 -- 172.21.15.151:0/46571592 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559aa1fb2a00 msgr2=0x559aa1fb84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.478+0000 7f0f83e00640 1 --2- 172.21.15.151:0/46571592 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559aa1fb2a00 0x559aa1fb84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.478+0000 7f0f83e00640 1 -- 172.21.15.151:0/46571592 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559aa1fb8b20 con 0x559aa1fa8140 2024-04-26T02:02:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.478+0000 7f0f88a00640 1 --2- 172.21.15.151:0/46571592 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559aa1fa7820 0x559aa1fbabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:02:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.478+0000 7f0f89400640 1 --2- 172.21.15.151:0/46571592 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559aa1fb2a00 0x559aa1fb84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:02:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.478+0000 7f0f83e00640 1 --2- 172.21.15.151:0/46571592 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559aa1fa8140 0x559aa1fb7f70 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0f70004590 tx=0x7f0f70009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=443192f114871936 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.478+0000 7f0f6a000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T02:02:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.478+0000 7f0f6a000640 1 -- 172.21.15.151:0/46571592 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f70040040 con 0x559aa1fa8140 2024-04-26T02:02:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.479+0000 7f0f8e364740 1 -- 172.21.15.151:0/46571592 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f78098a60 con 0x559aa1fa8140 2024-04-26T02:02:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.479+0000 7f0f6a000640 1 -- 172.21.15.151:0/46571592 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8455+0+0 (secure 0 0 0) 0x7f0f7004e5a0 con 0x559aa1fa8140 2024-04-26T02:02:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.479+0000 7f0f6a000640 1 -- 172.21.15.151:0/46571592 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f0f7005e050 con 0x559aa1fa8140 2024-04-26T02:02:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.479+0000 7f0f8e364740 10 client.26216 fetch_fsmap learned FSMap version 35 2024-04-26T02:02:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.479+0000 7f0f8e364740 1 -- 172.21.15.151:0/46571592 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x559aa2029190 con 0x559aa1fa8140 2024-04-26T02:02:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.479+0000 7f0f6a000640 1 -- 172.21.15.151:0/46571592 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f78098a60 con 0x559aa1fa8140 2024-04-26T02:02:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.479+0000 7f0f6a000640 1 -- 172.21.15.151:0/46571592 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 35) ==== 8338+0+0 (secure 0 0 0) 0x7f0f70031b50 con 0x559aa1fa8140 2024-04-26T02:02:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.479+0000 7f0f8e364740 10 client.26216 fetch_fsmap finished waiting for FSMap version 35 2024-04-26T02:02:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.479+0000 7f0f8e364740 10 client.26216 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:02:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.479+0000 7f0f8e364740 20 client.26216 populate_metadata read hostname 'smithi151' 2024-04-26T02:02:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.480+0000 7f0f8e364740 1 --2- 172.21.15.151:0/46571592 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x559aa20294d0 0x559aa202b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.480+0000 7f0f8e364740 4 client.26216 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T02:02:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.480+0000 7f0f8e364740 1 -- 172.21.15.151:0/46571592 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559aa1f1b320 con 0x559aa20294d0 2024-04-26T02:02:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.480+0000 7f0f88a00640 1 --2- 172.21.15.151:0/46571592 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x559aa20294d0 0x559aa202b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.480+0000 7f0f88a00640 1 --2- 172.21.15.151:0/46571592 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x559aa20294d0 0x559aa202b8f0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=271ab775f99b98e server_cookie=810a9388b68fcedd in_seq=0 out_seq=0 2024-04-26T02:02:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.480+0000 7f0f6a000640 10 client.26216 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:02:43.493 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.495+0000 7f0f6a000640 1 -- 172.21.15.151:0/46571592 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559aa1f1b320 con 0x559aa20294d0 2024-04-26T02:02:43.494 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.495+0000 7f0f6a000640 10 client.26216 handle_command_reply: tid=0 2024-04-26T02:02:43.494 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.495+0000 7f0f8e364740 1 client.26216 shutdown 2024-04-26T02:02:43.494 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.495+0000 7f0f8e364740 1 -- 172.21.15.151:0/46571592 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559aa1fa8140 msgr2=0x559aa1fb7f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:43.494 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.495+0000 7f0f8e364740 1 --2- 172.21.15.151:0/46571592 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559aa1fa8140 0x559aa1fb7f70 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0f70004590 tx=0x7f0f70009260 comp rx=0 tx=0).stop 2024-04-26T02:02:43.495 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.495+0000 7f0f8e364740 1 -- 172.21.15.151:0/46571592 shutdown_connections 2024-04-26T02:02:43.495 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.495+0000 7f0f8e364740 1 --2- 172.21.15.151:0/46571592 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x559aa20294d0 0x559aa202b8f0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:43.495 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.495+0000 7f0f8e364740 1 --2- 172.21.15.151:0/46571592 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559aa1fb2a00 0x559aa1fb84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:43.495 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.495+0000 7f0f8e364740 1 --2- 172.21.15.151:0/46571592 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559aa1fa8140 0x559aa1fb7f70 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:43.495 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.495+0000 7f0f8e364740 1 --2- 172.21.15.151:0/46571592 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559aa1fa7820 0x559aa1fbabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:43.495 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.495+0000 7f0f8e364740 1 -- 172.21.15.151:0/46571592 >> 172.21.15.151:0/46571592 conn(0x559aa1fa3d30 msgr2=0x559aa1fa2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:43.495 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.495+0000 7f0f6a000640 0 client.26216 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:02:43.495 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.496+0000 7f0f8e364740 1 -- 172.21.15.151:0/46571592 shutdown_connections 2024-04-26T02:02:43.496 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.496+0000 7f0f8e364740 1 -- 172.21.15.151:0/46571592 wait complete. 2024-04-26T02:02:43.496 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.496+0000 7f0f8e364740 20 client.26216 trim_cache size 0 max 16384 2024-04-26T02:02:43.496 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:{ 2024-04-26T02:02:43.497 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "message": "", 2024-04-26T02:02:43.497 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "return_code": 0 2024-04-26T02:02:43.497 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:} 2024-04-26T02:02:43.497 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.498+0000 7f0f8b400640 1 -- 172.21.15.151:0/2066084919 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f0f6c0a3df0 msgr2=0x7f0f6c0a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:43.497 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.498+0000 7f0f8b400640 1 --2- 172.21.15.151:0/2066084919 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f0f6c0a3df0 0x7f0f6c0a62b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c0107e0 tx=0x7f0f7c007520 comp rx=0 tx=0).stop 2024-04-26T02:02:43.497 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.498+0000 7f0f8b400640 1 -- 172.21.15.151:0/2066084919 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f841031e0 msgr2=0x7f0f84100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:43.498 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.498+0000 7f0f8b400640 1 --2- 172.21.15.151:0/2066084919 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f841031e0 0x7f0f84100e30 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0f7800d6d0 tx=0x7f0f7800db90 comp rx=0 tx=0).stop 2024-04-26T02:02:43.498 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.498+0000 7f0f8b400640 1 -- 172.21.15.151:0/2066084919 shutdown_connections 2024-04-26T02:02:43.498 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.498+0000 7f0f8b400640 1 --2- 172.21.15.151:0/2066084919 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f0f6c0a3df0 0x7f0f6c0a62b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:43.498 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.498+0000 7f0f8b400640 1 --2- 172.21.15.151:0/2066084919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f8410c520 0x7f0f841018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:43.498 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.498+0000 7f0f8b400640 1 --2- 172.21.15.151:0/2066084919 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f84103b00 0x7f0f84101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:43.498 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.498+0000 7f0f8b400640 1 --2- 172.21.15.151:0/2066084919 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f841031e0 0x7f0f84100e30 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:43.498 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.498+0000 7f0f8b400640 1 -- 172.21.15.151:0/2066084919 >> 172.21.15.151:0/2066084919 conn(0x7f0f84077fd0 msgr2=0x7f0f840fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:43.498 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.498+0000 7f0f8b400640 1 -- 172.21.15.151:0/2066084919 shutdown_connections 2024-04-26T02:02:43.498 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:43.498+0000 7f0f8b400640 1 -- 172.21.15.151:0/2066084919 wait complete. 2024-04-26T02:02:44.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:02:43 smithi172 ceph-mon[32785]: pgmap v285: 97 pgs: 97 active+clean; 116 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 1.4 MiB/s wr, 422 op/s 2024-04-26T02:02:44.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:02:43 smithi165 ceph-mon[32648]: pgmap v285: 97 pgs: 97 active+clean; 116 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 1.4 MiB/s wr, 422 op/s 2024-04-26T02:02:44.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:02:43 smithi151 ceph-mon[28057]: pgmap v285: 97 pgs: 97 active+clean; 116 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 1.4 MiB/s wr, 422 op/s 2024-04-26T02:02:45.788 INFO:tasks.quiescer.fs.[cephfs]:Total duration (44.2) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-04-26T02:02:45.788 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.1 2024-04-26T02:02:45.789 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T02:02:46.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:02:45 smithi151 ceph-mon[28057]: pgmap v286: 97 pgs: 97 active+clean; 116 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 1.3 MiB/s wr, 394 op/s 2024-04-26T02:02:46.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:02:45 smithi172 ceph-mon[32785]: pgmap v286: 97 pgs: 97 active+clean; 116 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 1.3 MiB/s wr, 394 op/s 2024-04-26T02:02:46.292 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:02:46.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:02:45 smithi165 ceph-mon[32648]: pgmap v286: 97 pgs: 97 active+clean; 116 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 1.3 MiB/s wr, 394 op/s 2024-04-26T02:02:48.176 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:02:48 smithi151 ceph-mon[28057]: pgmap v287: 97 pgs: 97 active+clean; 116 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 1.1 MiB/s wr, 315 op/s 2024-04-26T02:02:48.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:02:48 smithi172 ceph-mon[32785]: pgmap v287: 97 pgs: 97 active+clean; 116 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 1.1 MiB/s wr, 315 op/s 2024-04-26T02:02:48.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:02:48 smithi165 ceph-mon[32648]: pgmap v287: 97 pgs: 97 active+clean; 116 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 1.1 MiB/s wr, 315 op/s 2024-04-26T02:02:48.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:48.735+0000 7f1872a00700 1 -- 172.21.15.151:0/986362821 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f186c108a30 msgr2=0x7f186c108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:48.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:48.735+0000 7f1872a00700 1 --2- 172.21.15.151:0/986362821 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f186c108a30 0x7f186c108e10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f185c009990 tx=0x7f185c009ca0 comp rx=0 tx=0).stop 2024-04-26T02:02:48.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:48.735+0000 7f1872a00700 1 -- 172.21.15.151:0/986362821 shutdown_connections 2024-04-26T02:02:48.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:48.735+0000 7f1872a00700 1 --2- 172.21.15.151:0/986362821 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f186c06b0f0 0x7f186c10f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:48.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:48.735+0000 7f1872a00700 1 --2- 172.21.15.151:0/986362821 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f186c108a30 0x7f186c108e10 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:48.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:48.735+0000 7f1872a00700 1 --2- 172.21.15.151:0/986362821 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f186c06a750 0x7f186c06abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:48.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:48.735+0000 7f1872a00700 1 -- 172.21.15.151:0/986362821 >> 172.21.15.151:0/986362821 conn(0x7f186c0788f0 msgr2=0x7f186c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:48.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:48.735+0000 7f1872a00700 1 -- 172.21.15.151:0/986362821 shutdown_connections 2024-04-26T02:02:48.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:48.735+0000 7f1872a00700 1 -- 172.21.15.151:0/986362821 wait complete. 2024-04-26T02:02:48.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:48.736+0000 7f1872a00700 1 Processor -- start 2024-04-26T02:02:48.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:48.737+0000 7f1872a00700 1 -- start start 2024-04-26T02:02:48.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:48.737+0000 7f1872a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f186c06a750 0x7f186c101a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:48.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:48.737+0000 7f1872a00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f186c06b0f0 0x7f186c101f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:48.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:48.737+0000 7f1872a00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f186c108a30 0x7f186c102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:48.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:48.737+0000 7f1872a00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f186c112c00 con 0x7f186c06a750 2024-04-26T02:02:48.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:48.737+0000 7f1872a00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f186c112a80 con 0x7f186c108a30 2024-04-26T02:02:48.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:48.737+0000 7f1872a00700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f186c112d80 con 0x7f186c06b0f0 2024-04-26T02:02:48.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:48.737+0000 7f186be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f186c06a750 0x7f186c101a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:48.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:48.737+0000 7f186b400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f186c06b0f0 0x7f186c101f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:48.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:48.737+0000 7f186b400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f186c06b0f0 0x7f186c101f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:50436/0 (socket says 172.21.15.151:50436) 2024-04-26T02:02:48.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:48.737+0000 7f186be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f186c06a750 0x7f186c101a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:48300/0 (socket says 172.21.15.151:48300) 2024-04-26T02:02:48.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:48.737+0000 7f186b400700 1 -- 172.21.15.151:0/1255058980 learned_addr learned my addr 172.21.15.151:0/1255058980 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:02:48.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:48.738+0000 7f1870a00700 1 --2- 172.21.15.151:0/1255058980 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f186c108a30 0x7f186c102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:48.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:48.738+0000 7f186be00700 1 -- 172.21.15.151:0/1255058980 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f186c06b0f0 msgr2=0x7f186c101f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:48.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:48.738+0000 7f186be00700 1 --2- 172.21.15.151:0/1255058980 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f186c06b0f0 0x7f186c101f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:48.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:48.738+0000 7f186be00700 1 -- 172.21.15.151:0/1255058980 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f186c108a30 msgr2=0x7f186c102490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:48.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:48.738+0000 7f186be00700 1 --2- 172.21.15.151:0/1255058980 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f186c108a30 0x7f186c102490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:48.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:48.738+0000 7f186be00700 1 -- 172.21.15.151:0/1255058980 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f186c1a7680 con 0x7f186c06a750 2024-04-26T02:02:48.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:48.738+0000 7f186b400700 1 --2- 172.21.15.151:0/1255058980 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f186c06b0f0 0x7f186c101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:02:48.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:48.738+0000 7f186be00700 1 --2- 172.21.15.151:0/1255058980 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f186c06a750 0x7f186c101a10 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f185c000c00 tx=0x7f185c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:48.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:48.739+0000 7f1868c00700 1 -- 172.21.15.151:0/1255058980 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185c026070 con 0x7f186c06a750 2024-04-26T02:02:48.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:48.739+0000 7f1868c00700 1 -- 172.21.15.151:0/1255058980 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f185c021410 con 0x7f186c06a750 2024-04-26T02:02:48.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:48.739+0000 7f1868c00700 1 -- 172.21.15.151:0/1255058980 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185c017650 con 0x7f186c06a750 2024-04-26T02:02:48.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:48.739+0000 7f1872a00700 1 -- 172.21.15.151:0/1255058980 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f186c1a7910 con 0x7f186c06a750 2024-04-26T02:02:48.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:48.739+0000 7f1872a00700 1 -- 172.21.15.151:0/1255058980 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f186c1a7d10 con 0x7f186c06a750 2024-04-26T02:02:48.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:48.740+0000 7f1872a00700 1 -- 172.21.15.151:0/1255058980 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f186c069730 con 0x7f186c06a750 2024-04-26T02:02:48.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:48.743+0000 7f1868c00700 1 -- 172.21.15.151:0/1255058980 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f185c029070 con 0x7f186c06a750 2024-04-26T02:02:48.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:48.745+0000 7f1868c00700 1 --2- 172.21.15.151:0/1255058980 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f18500a3e30 0x7f18500a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:48.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:48.745+0000 7f1868c00700 1 -- 172.21.15.151:0/1255058980 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8455+0+0 (secure 0 0 0) 0x7f185c0d1440 con 0x7f186c06a750 2024-04-26T02:02:48.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:48.745+0000 7f186b400700 1 --2- 172.21.15.151:0/1255058980 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f18500a3e30 0x7f18500a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:48.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:48.746+0000 7f186b400700 1 --2- 172.21.15.151:0/1255058980 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f18500a3e30 0x7f18500a62f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f186c078f60 tx=0x7f1858009260 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:48.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:48.750+0000 7f1868c00700 1 -- 172.21.15.151:0/1255058980 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f185c05dff0 con 0x7f186c06a750 2024-04-26T02:02:49.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:49.085+0000 7f1872a00700 1 -- 172.21.15.151:0/1255058980 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f186c02e020 con 0x7f186c06a750 2024-04-26T02:02:49.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:49.087+0000 7f1868c00700 1 -- 172.21.15.151:0/1255058980 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11889 (secure 0 0 0) 0x7f185c013070 con 0x7f186c06a750 2024-04-26T02:02:49.086 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T02:02:49.087 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2937864097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2937864097},{"type":"v1","addr":"172.21.15.151:6835","nonce":2937864097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14373,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/2947090802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2947090802},{"type":"v1","addr":"172.21.15.151:6837","nonce":2947090802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/1356563280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1356563280},{"type":"v1","addr":"172.21.15.151:6839","nonce":1356563280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6841/1075240552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1075240552},{"type":"v1","addr":"172.21.15.151:6841","nonce":1075240552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/504675912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":504675912},{"type":"v1","addr":"172.21.15.165:6833","nonce":504675912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1207580015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1207580015},{"type":"v1","addr":"172.21.15.172:6833","nonce":1207580015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:49:40.746876+0000","modified":"2024-04-26T02:00:05.413774+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24505,"mds_2":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/3598138294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3598138294},{"type":"v1","addr":"172.21.15.165:6835","nonce":3598138294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/1847859054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":1847859054},{"type":"v1","addr":"172.21.15.165:6837","nonce":1847859054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.172:6837/55972537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":55972537},{"type":"v1","addr":"172.21.15.172:6837","nonce":55972537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.172:6839/1242302810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1242302810},{"type":"v1","addr":"172.21.15.172:6839","nonce":1242302810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/1969716254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1969716254},{"type":"v1","addr":"172.21.15.165:6839","nonce":1969716254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25954":{"gid":25954,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/3656630171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3656630171},{"type":"v1","addr":"172.21.15.172:6835","nonce":3656630171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24493,24505,24509]},"id":1}]} 2024-04-26T02:02:49.090 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:49.091+0000 7f1872a00700 1 -- 172.21.15.151:0/1255058980 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f18500a3e30 msgr2=0x7f18500a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:49.090 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:49.091+0000 7f1872a00700 1 --2- 172.21.15.151:0/1255058980 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f18500a3e30 0x7f18500a62f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f186c078f60 tx=0x7f1858009260 comp rx=0 tx=0).stop 2024-04-26T02:02:49.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:49.091+0000 7f1872a00700 1 -- 172.21.15.151:0/1255058980 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f186c06a750 msgr2=0x7f186c101a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:49.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:49.091+0000 7f1872a00700 1 --2- 172.21.15.151:0/1255058980 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f186c06a750 0x7f186c101a10 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f185c000c00 tx=0x7f185c0047b0 comp rx=0 tx=0).stop 2024-04-26T02:02:49.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:49.091+0000 7f1872a00700 1 -- 172.21.15.151:0/1255058980 shutdown_connections 2024-04-26T02:02:49.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:49.091+0000 7f1872a00700 1 --2- 172.21.15.151:0/1255058980 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f18500a3e30 0x7f18500a62f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:49.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:49.091+0000 7f1872a00700 1 --2- 172.21.15.151:0/1255058980 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f186c108a30 0x7f186c102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:49.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:49.091+0000 7f1872a00700 1 --2- 172.21.15.151:0/1255058980 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f186c06a750 0x7f186c101a10 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:49.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:49.091+0000 7f1872a00700 1 --2- 172.21.15.151:0/1255058980 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f186c06b0f0 0x7f186c101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:49.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:49.091+0000 7f1872a00700 1 -- 172.21.15.151:0/1255058980 >> 172.21.15.151:0/1255058980 conn(0x7f186c0788f0 msgr2=0x7f186c0ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:49.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:49.091+0000 7f1872a00700 1 -- 172.21.15.151:0/1255058980 shutdown_connections 2024-04-26T02:02:49.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:49.091+0000 7f1872a00700 1 -- 172.21.15.151:0/1255058980 wait complete. 2024-04-26T02:02:49.092 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 35 2024-04-26T02:02:49.920 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24509 quiesce db / --timeout 60 --expiration 82.1 --await-for 42.0' 2024-04-26T02:02:49.920 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24509 quiesce db / --timeout 60 --expiration 82.1 --await-for 42.0 2024-04-26T02:02:50.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:02:50 smithi151 ceph-mon[28057]: pgmap v288: 97 pgs: 97 active+clean; 108 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 1.6 MiB/s wr, 509 op/s 2024-04-26T02:02:50.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:02:50 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/1255058980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:02:50.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:02:50 smithi172 ceph-mon[32785]: pgmap v288: 97 pgs: 97 active+clean; 108 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 1.6 MiB/s wr, 509 op/s 2024-04-26T02:02:50.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:02:50 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/1255058980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:02:50.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:02:50 smithi165 ceph-mon[32648]: pgmap v288: 97 pgs: 97 active+clean; 108 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 1.6 MiB/s wr, 509 op/s 2024-04-26T02:02:50.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:02:50 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/1255058980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:02:50.388 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:02:52.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:02:52 smithi151 ceph-mon[28057]: pgmap v289: 97 pgs: 97 active+clean; 108 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1019 KiB/s wr, 337 op/s 2024-04-26T02:02:52.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:02:52 smithi172 ceph-mon[32785]: pgmap v289: 97 pgs: 97 active+clean; 108 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1019 KiB/s wr, 337 op/s 2024-04-26T02:02:52.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:02:52 smithi165 ceph-mon[32648]: pgmap v289: 97 pgs: 97 active+clean; 108 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1019 KiB/s wr, 337 op/s 2024-04-26T02:02:52.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.766+0000 7ff837400700 1 -- 172.21.15.151:0/322675346 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff83010a360 msgr2=0x7ff830117a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:52.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.766+0000 7ff837400700 1 --2- 172.21.15.151:0/322675346 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff83010a360 0x7ff830117a40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff8200099c0 tx=0x7ff820009cd0 comp rx=0 tx=0).stop 2024-04-26T02:02:52.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.766+0000 7ff837400700 1 -- 172.21.15.151:0/322675346 shutdown_connections 2024-04-26T02:02:52.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.766+0000 7ff837400700 1 --2- 172.21.15.151:0/322675346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff830117f80 0x7ff83011a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:52.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.766+0000 7ff837400700 1 --2- 172.21.15.151:0/322675346 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff830109a40 0x7ff830109e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:52.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.766+0000 7ff837400700 1 --2- 172.21.15.151:0/322675346 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff83010a360 0x7ff830117a40 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:52.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.766+0000 7ff837400700 1 -- 172.21.15.151:0/322675346 >> 172.21.15.151:0/322675346 conn(0x7ff83006dbb0 msgr2=0x7ff83006dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:52.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.767+0000 7ff837400700 1 -- 172.21.15.151:0/322675346 shutdown_connections 2024-04-26T02:02:52.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.767+0000 7ff837400700 1 -- 172.21.15.151:0/322675346 wait complete. 2024-04-26T02:02:52.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.768+0000 7ff837400700 1 Processor -- start 2024-04-26T02:02:52.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.768+0000 7ff837400700 1 -- start start 2024-04-26T02:02:52.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.769+0000 7ff837400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff830109a40 0x7ff8301a9750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:52.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.769+0000 7ff837400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff83010a360 0x7ff8301a9c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:52.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.769+0000 7ff837400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff830117f80 0x7ff8301a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:52.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.769+0000 7ff837400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff83011d690 con 0x7ff830109a40 2024-04-26T02:02:52.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.769+0000 7ff837400700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff83011d510 con 0x7ff83010a360 2024-04-26T02:02:52.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.769+0000 7ff837400700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff83011d810 con 0x7ff830117f80 2024-04-26T02:02:52.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.769+0000 7ff82fe00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff83010a360 0x7ff8301a9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:52.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.769+0000 7ff82fe00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff83010a360 0x7ff8301a9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:53074/0 (socket says 172.21.15.151:53074) 2024-04-26T02:02:52.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.769+0000 7ff82fe00700 1 -- 172.21.15.151:0/4248315204 learned_addr learned my addr 172.21.15.151:0/4248315204 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:02:52.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.769+0000 7ff834a00700 1 --2- 172.21.15.151:0/4248315204 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff830109a40 0x7ff8301a9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:52.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.769+0000 7ff835400700 1 --2- 172.21.15.151:0/4248315204 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff830117f80 0x7ff8301a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:52.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.770+0000 7ff82fe00700 1 -- 172.21.15.151:0/4248315204 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff830117f80 msgr2=0x7ff8301a3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:52.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.770+0000 7ff82fe00700 1 --2- 172.21.15.151:0/4248315204 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff830117f80 0x7ff8301a3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:52.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.770+0000 7ff82fe00700 1 -- 172.21.15.151:0/4248315204 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff830109a40 msgr2=0x7ff8301a9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:52.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.770+0000 7ff82fe00700 1 --2- 172.21.15.151:0/4248315204 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff830109a40 0x7ff8301a9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:52.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.770+0000 7ff82fe00700 1 -- 172.21.15.151:0/4248315204 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8301a4070 con 0x7ff83010a360 2024-04-26T02:02:52.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.770+0000 7ff82fe00700 1 --2- 172.21.15.151:0/4248315204 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff83010a360 0x7ff8301a9c90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff820000c00 tx=0x7ff820005590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:52.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.770+0000 7ff82d600700 1 -- 172.21.15.151:0/4248315204 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff820003ae0 con 0x7ff83010a360 2024-04-26T02:02:52.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.770+0000 7ff82d600700 1 -- 172.21.15.151:0/4248315204 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff820004130 con 0x7ff83010a360 2024-04-26T02:02:52.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.770+0000 7ff837400700 1 -- 172.21.15.151:0/4248315204 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8301a4300 con 0x7ff83010a360 2024-04-26T02:02:52.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.770+0000 7ff82d600700 1 -- 172.21.15.151:0/4248315204 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82000fd50 con 0x7ff83010a360 2024-04-26T02:02:52.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.771+0000 7ff837400700 1 -- 172.21.15.151:0/4248315204 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8301b0620 con 0x7ff83010a360 2024-04-26T02:02:52.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.773+0000 7ff82d600700 1 -- 172.21.15.151:0/4248315204 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7ff820003c80 con 0x7ff83010a360 2024-04-26T02:02:52.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.775+0000 7ff82d600700 1 --2- 172.21.15.151:0/4248315204 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7ff8100a3f20 0x7ff8100a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:52.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.776+0000 7ff82d600700 1 -- 172.21.15.151:0/4248315204 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8455+0+0 (secure 0 0 0) 0x7ff8200d1720 con 0x7ff83010a360 2024-04-26T02:02:52.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.776+0000 7ff834a00700 1 --2- 172.21.15.151:0/4248315204 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7ff8100a3f20 0x7ff8100a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:52.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.777+0000 7ff834a00700 1 --2- 172.21.15.151:0/4248315204 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7ff8100a3f20 0x7ff8100a63e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff818005950 tx=0x7ff81800a300 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:52.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.779+0000 7ff841d32b80 1 Processor -- start 2024-04-26T02:02:52.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.780+0000 7ff841d32b80 1 -- start start 2024-04-26T02:02:52.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.780+0000 7ff841d32b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613afa059b0 0x5613afa05d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:52.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.781+0000 7ff841d32b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5613afa062d0 0x5613afa10c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:52.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.781+0000 7ff841d32b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5613afa11180 0x5613afa13570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:52.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.781+0000 7ff841d32b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5613af959020 con 0x5613afa062d0 2024-04-26T02:02:52.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.781+0000 7ff841d32b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5613af8ff8c0 con 0x5613afa11180 2024-04-26T02:02:52.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.781+0000 7ff841d32b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5613af954fe0 con 0x5613afa059b0 2024-04-26T02:02:52.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.781+0000 7ff82fe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5613afa062d0 0x5613afa10c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:52.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.781+0000 7ff835400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5613afa11180 0x5613afa13570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:52.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.781+0000 7ff834a00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613afa059b0 0x5613afa05d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:52.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.781+0000 7ff835400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5613afa11180 0x5613afa13570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:53088/0 (socket says 172.21.15.151:53088) 2024-04-26T02:02:52.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.781+0000 7ff82fe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5613afa062d0 0x5613afa10c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47534/0 (socket says 172.21.15.151:47534) 2024-04-26T02:02:52.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.781+0000 7ff835400700 1 -- 172.21.15.151:0/1851888004 learned_addr learned my addr 172.21.15.151:0/1851888004 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:02:52.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.781+0000 7ff835400700 1 -- 172.21.15.151:0/1851888004 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613afa059b0 msgr2=0x5613afa05d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:52.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.782+0000 7ff835400700 1 --2- 172.21.15.151:0/1851888004 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613afa059b0 0x5613afa05d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:52.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.782+0000 7ff835400700 1 -- 172.21.15.151:0/1851888004 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5613afa062d0 msgr2=0x5613afa10c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:52.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.782+0000 7ff835400700 1 --2- 172.21.15.151:0/1851888004 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5613afa062d0 0x5613afa10c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:52.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.782+0000 7ff835400700 1 -- 172.21.15.151:0/1851888004 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613af8f5810 con 0x5613afa11180 2024-04-26T02:02:52.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.782+0000 7ff834a00700 1 --2- 172.21.15.151:0/1851888004 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613afa059b0 0x5613afa05d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:02:52.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.782+0000 7ff835400700 1 --2- 172.21.15.151:0/1851888004 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5613afa11180 0x5613afa13570 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff828003fa0 tx=0x7ff828009ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64184e4ddf19f9d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:52.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.782+0000 7ff826000700 1 -- 172.21.15.151:0/1851888004 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff828004bb0 con 0x5613afa11180 2024-04-26T02:02:52.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.782+0000 7ff826000700 1 -- 172.21.15.151:0/1851888004 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff828004d50 con 0x5613afa11180 2024-04-26T02:02:52.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.782+0000 7ff826000700 1 -- 172.21.15.151:0/1851888004 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff828007da0 con 0x5613afa11180 2024-04-26T02:02:52.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.784+0000 7ff841d32b80 1 -- 172.21.15.151:0/1851888004 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5613afa11180 msgr2=0x5613afa13570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:52.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.784+0000 7ff841d32b80 1 --2- 172.21.15.151:0/1851888004 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5613afa11180 0x5613afa13570 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff828003fa0 tx=0x7ff828009ba0 comp rx=0 tx=0).stop 2024-04-26T02:02:52.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.784+0000 7ff841d32b80 1 -- 172.21.15.151:0/1851888004 shutdown_connections 2024-04-26T02:02:52.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.784+0000 7ff841d32b80 1 --2- 172.21.15.151:0/1851888004 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5613afa11180 0x5613afa13570 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:52.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.784+0000 7ff841d32b80 1 --2- 172.21.15.151:0/1851888004 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613afa059b0 0x5613afa05d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:52.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.784+0000 7ff841d32b80 1 --2- 172.21.15.151:0/1851888004 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5613afa062d0 0x5613afa10c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:52.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.784+0000 7ff841d32b80 1 -- 172.21.15.151:0/1851888004 >> 172.21.15.151:0/1851888004 conn(0x5613af934730 msgr2=0x5613afa01e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:52.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.784+0000 7ff841d32b80 1 -- 172.21.15.151:0/1851888004 shutdown_connections 2024-04-26T02:02:52.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.784+0000 7ff841d32b80 1 -- 172.21.15.151:0/1851888004 wait complete. 2024-04-26T02:02:52.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.785+0000 7ff841d32b80 1 -- start start 2024-04-26T02:02:52.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.785+0000 7ff841d32b80 1 Processor -- start 2024-04-26T02:02:52.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.786+0000 7ff841d32b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5613afa059b0 0x5613afa10880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:52.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.786+0000 7ff841d32b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613afa062d0 0x5613afa0db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:52.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.786+0000 7ff841d32b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5613afa11180 0x5613afa0e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:52.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.786+0000 7ff841d32b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5613afa13ca0 con 0x5613afa059b0 2024-04-26T02:02:52.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.786+0000 7ff841d32b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5613af9129f0 con 0x5613afa11180 2024-04-26T02:02:52.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.786+0000 7ff841d32b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5613afa13e20 con 0x5613afa062d0 2024-04-26T02:02:52.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.786+0000 7ff841d32b80 10 client.0 _ll_register_callbacks cb 0x5613af8f18b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:02:52.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.786+0000 7ff834a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5613afa059b0 0x5613afa10880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:52.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.786+0000 7ff82fe00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613afa062d0 0x5613afa0db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:52.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.786+0000 7ff82fe00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613afa062d0 0x5613afa0db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:54034/0 (socket says 172.21.15.151:54034) 2024-04-26T02:02:52.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.786+0000 7ff82fe00700 1 -- 172.21.15.151:0/281511666 learned_addr learned my addr 172.21.15.151:0/281511666 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:02:52.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.787+0000 7ff835400700 1 --2- 172.21.15.151:0/281511666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5613afa11180 0x5613afa0e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:52.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.787+0000 7ff82fe00700 1 -- 172.21.15.151:0/281511666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5613afa11180 msgr2=0x5613afa0e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:52.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.787+0000 7ff82fe00700 1 --2- 172.21.15.151:0/281511666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5613afa11180 0x5613afa0e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:52.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.787+0000 7ff82fe00700 1 -- 172.21.15.151:0/281511666 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5613afa059b0 msgr2=0x5613afa10880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:52.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.787+0000 7ff82fe00700 1 --2- 172.21.15.151:0/281511666 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5613afa059b0 0x5613afa10880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:52.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.787+0000 7ff82fe00700 1 -- 172.21.15.151:0/281511666 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613af8a6a60 con 0x5613afa062d0 2024-04-26T02:02:52.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.787+0000 7ff835400700 1 --2- 172.21.15.151:0/281511666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5613afa11180 0x5613afa0e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:02:52.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.787+0000 7ff834a00700 1 --2- 172.21.15.151:0/281511666 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5613afa059b0 0x5613afa10880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:02:52.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.787+0000 7ff82fe00700 1 --2- 172.21.15.151:0/281511666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613afa062d0 0x5613afa0db80 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff820099bf0 tx=0x7ff82009d350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38a4872290b674df server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:52.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.787+0000 7ff81ea00700 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-26T02:02:52.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.787+0000 7ff81ea00700 1 -- 172.21.15.151:0/281511666 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff820003f70 con 0x5613afa062d0 2024-04-26T02:02:52.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.788+0000 7ff81ea00700 1 -- 172.21.15.151:0/281511666 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8455+0+0 (secure 0 0 0) 0x7ff82000f460 con 0x5613afa062d0 2024-04-26T02:02:52.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.788+0000 7ff841d32b80 1 -- 172.21.15.151:0/281511666 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff828004d50 con 0x5613afa062d0 2024-04-26T02:02:52.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.788+0000 7ff81ea00700 1 -- 172.21.15.151:0/281511666 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff82009de90 con 0x5613afa062d0 2024-04-26T02:02:52.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.788+0000 7ff841d32b80 10 client.26260 fetch_fsmap learned FSMap version 35 2024-04-26T02:02:52.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.788+0000 7ff841d32b80 1 -- 172.21.15.151:0/281511666 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5613af8efc80 con 0x5613afa062d0 2024-04-26T02:02:52.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.788+0000 7ff81ea00700 1 -- 172.21.15.151:0/281511666 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff82009d790 con 0x5613afa062d0 2024-04-26T02:02:52.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.788+0000 7ff81ea00700 1 -- 172.21.15.151:0/281511666 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 35) ==== 8338+0+0 (secure 0 0 0) 0x7ff8200b6060 con 0x5613afa062d0 2024-04-26T02:02:52.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.789+0000 7ff841d32b80 10 client.26260 fetch_fsmap finished waiting for FSMap version 35 2024-04-26T02:02:52.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.789+0000 7ff841d32b80 10 client.26260 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-26T02:02:52.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.789+0000 7ff841d32b80 20 client.26260 populate_metadata read hostname 'smithi151' 2024-04-26T02:02:52.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.789+0000 7ff841d32b80 1 --2- 172.21.15.151:0/281511666 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5613afa20950 0x5613afa22d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:52.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.789+0000 7ff841d32b80 4 client.26260 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:02:52.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.789+0000 7ff841d32b80 1 -- 172.21.15.151:0/281511666 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff828004d50 con 0x5613afa20950 2024-04-26T02:02:52.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.789+0000 7ff834a00700 1 --2- 172.21.15.151:0/281511666 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5613afa20950 0x5613afa22d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:52.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.790+0000 7ff834a00700 1 --2- 172.21.15.151:0/281511666 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5613afa20950 0x5613afa22d40 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13850d933a68bcd0 server_cookie=17ae3bb500eb3b43 in_seq=0 out_seq=0 2024-04-26T02:02:52.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.790+0000 7ff81ea00700 10 client.26260 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:02:52.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.790+0000 7ff81ea00700 1 -- 172.21.15.151:0/281511666 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7ff828004d50 con 0x5613afa20950 2024-04-26T02:02:52.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.790+0000 7ff81ea00700 10 client.26260 handle_command_reply: tid=0 2024-04-26T02:02:52.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.790+0000 7ff841d32b80 1 client.26260 shutdown 2024-04-26T02:02:52.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.791+0000 7ff841d32b80 1 -- 172.21.15.151:0/281511666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613afa062d0 msgr2=0x5613afa0db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:52.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.791+0000 7ff841d32b80 1 --2- 172.21.15.151:0/281511666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613afa062d0 0x5613afa0db80 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff820099bf0 tx=0x7ff82009d350 comp rx=0 tx=0).stop 2024-04-26T02:02:52.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.791+0000 7ff841d32b80 1 -- 172.21.15.151:0/281511666 shutdown_connections 2024-04-26T02:02:52.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.791+0000 7ff841d32b80 1 --2- 172.21.15.151:0/281511666 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5613afa20950 0x5613afa22d40 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:52.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.791+0000 7ff841d32b80 1 --2- 172.21.15.151:0/281511666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5613afa11180 0x5613afa0e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:52.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.791+0000 7ff841d32b80 1 --2- 172.21.15.151:0/281511666 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5613afa059b0 0x5613afa10880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:52.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.791+0000 7ff841d32b80 1 --2- 172.21.15.151:0/281511666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613afa062d0 0x5613afa0db80 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:52.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.791+0000 7ff81ea00700 0 client.26260 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:02:52.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.791+0000 7ff841d32b80 1 -- 172.21.15.151:0/281511666 >> 172.21.15.151:0/281511666 conn(0x5613af934730 msgr2=0x5613afa01a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:52.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.791+0000 7ff841d32b80 1 -- 172.21.15.151:0/281511666 shutdown_connections 2024-04-26T02:02:52.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.791+0000 7ff841d32b80 1 -- 172.21.15.151:0/281511666 wait complete. 2024-04-26T02:02:52.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.791+0000 7ff841d32b80 20 client.26260 trim_cache size 0 max 16384 2024-04-26T02:02:52.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.815+0000 7ff841d32b80 1 Processor -- start 2024-04-26T02:02:52.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.816+0000 7ff841d32b80 1 -- start start 2024-04-26T02:02:52.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.816+0000 7ff841d32b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5613afa059b0 0x5613afa1cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:52.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.816+0000 7ff841d32b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613afa062d0 0x5613afa28610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:52.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.816+0000 7ff841d32b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5613afa11180 0x5613afa28b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:52.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.816+0000 7ff841d32b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff8301ba3c0 con 0x5613afa059b0 2024-04-26T02:02:52.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.816+0000 7ff841d32b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff8301b7c50 con 0x5613afa11180 2024-04-26T02:02:52.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.816+0000 7ff841d32b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff8301b5ac0 con 0x5613afa062d0 2024-04-26T02:02:52.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.816+0000 7ff82fe00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613afa062d0 0x5613afa28610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:52.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.816+0000 7ff835400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5613afa11180 0x5613afa28b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:52.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.816+0000 7ff834a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5613afa059b0 0x5613afa1cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:52.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.816+0000 7ff82fe00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613afa062d0 0x5613afa28610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:54044/0 (socket says 172.21.15.151:54044) 2024-04-26T02:02:52.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.816+0000 7ff835400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5613afa11180 0x5613afa28b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:53094/0 (socket says 172.21.15.151:53094) 2024-04-26T02:02:52.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.816+0000 7ff82fe00700 1 -- 172.21.15.151:0/3341395761 learned_addr learned my addr 172.21.15.151:0/3341395761 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:02:52.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.817+0000 7ff82fe00700 1 -- 172.21.15.151:0/3341395761 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5613afa11180 msgr2=0x5613afa28b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:52.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.817+0000 7ff82fe00700 1 --2- 172.21.15.151:0/3341395761 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5613afa11180 0x5613afa28b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:52.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.817+0000 7ff82fe00700 1 -- 172.21.15.151:0/3341395761 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5613afa059b0 msgr2=0x5613afa1cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:52.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.817+0000 7ff82fe00700 1 --2- 172.21.15.151:0/3341395761 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5613afa059b0 0x5613afa1cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:52.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.817+0000 7ff82fe00700 1 -- 172.21.15.151:0/3341395761 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613af9f9450 con 0x5613afa062d0 2024-04-26T02:02:52.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.817+0000 7ff82fe00700 1 --2- 172.21.15.151:0/3341395761 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613afa062d0 0x5613afa28610 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff8200cf4b0 tx=0x7ff82009ddb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34d9d4d13edf7292 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:52.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.817+0000 7ff825600700 1 -- 172.21.15.151:0/3341395761 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff820003f70 con 0x5613afa062d0 2024-04-26T02:02:52.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.817+0000 7ff825600700 1 -- 172.21.15.151:0/3341395761 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff82009d790 con 0x5613afa062d0 2024-04-26T02:02:52.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.817+0000 7ff825600700 1 -- 172.21.15.151:0/3341395761 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8200a1b40 con 0x5613afa062d0 2024-04-26T02:02:52.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.818+0000 7ff841d32b80 1 -- 172.21.15.151:0/3341395761 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613afa062d0 msgr2=0x5613afa28610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:52.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.818+0000 7ff841d32b80 1 --2- 172.21.15.151:0/3341395761 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613afa062d0 0x5613afa28610 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff8200cf4b0 tx=0x7ff82009ddb0 comp rx=0 tx=0).stop 2024-04-26T02:02:52.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.818+0000 7ff841d32b80 1 -- 172.21.15.151:0/3341395761 shutdown_connections 2024-04-26T02:02:52.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.818+0000 7ff841d32b80 1 --2- 172.21.15.151:0/3341395761 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5613afa11180 0x5613afa28b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:52.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.818+0000 7ff841d32b80 1 --2- 172.21.15.151:0/3341395761 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5613afa059b0 0x5613afa1cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:52.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.818+0000 7ff841d32b80 1 --2- 172.21.15.151:0/3341395761 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613afa062d0 0x5613afa28610 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:52.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.818+0000 7ff841d32b80 1 -- 172.21.15.151:0/3341395761 >> 172.21.15.151:0/3341395761 conn(0x5613af934730 msgr2=0x5613afa01990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:52.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.819+0000 7ff841d32b80 1 -- 172.21.15.151:0/3341395761 shutdown_connections 2024-04-26T02:02:52.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.819+0000 7ff841d32b80 1 -- 172.21.15.151:0/3341395761 wait complete. 2024-04-26T02:02:52.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.819+0000 7ff841d32b80 1 -- start start 2024-04-26T02:02:52.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.821+0000 7ff841d32b80 1 Processor -- start 2024-04-26T02:02:52.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.821+0000 7ff841d32b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5613afa059b0 0x5613afa194b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:52.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.821+0000 7ff841d32b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5613afa062d0 0x5613afa15840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:52.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.821+0000 7ff841d32b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613afa11180 0x5613afa15d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:52.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.821+0000 7ff841d32b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff8301b6110 con 0x5613afa059b0 2024-04-26T02:02:52.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.821+0000 7ff841d32b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff8301b7580 con 0x5613afa062d0 2024-04-26T02:02:52.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.821+0000 7ff841d32b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff8301b67e0 con 0x5613afa11180 2024-04-26T02:02:52.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.822+0000 7ff841d32b80 10 client.0 _ll_register_callbacks cb 0x5613af8f18b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:02:52.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.822+0000 7ff835400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613afa11180 0x5613afa15d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:52.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.822+0000 7ff835400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613afa11180 0x5613afa15d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:54046/0 (socket says 172.21.15.151:54046) 2024-04-26T02:02:52.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.822+0000 7ff835400700 1 -- 172.21.15.151:0/2180705269 learned_addr learned my addr 172.21.15.151:0/2180705269 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:02:52.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.823+0000 7ff834a00700 1 --2- 172.21.15.151:0/2180705269 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5613afa059b0 0x5613afa194b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:52.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.823+0000 7ff82fe00700 1 --2- 172.21.15.151:0/2180705269 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5613afa062d0 0x5613afa15840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:52.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.823+0000 7ff835400700 1 -- 172.21.15.151:0/2180705269 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5613afa062d0 msgr2=0x5613afa15840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:52.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.823+0000 7ff835400700 1 --2- 172.21.15.151:0/2180705269 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5613afa062d0 0x5613afa15840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:52.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.823+0000 7ff835400700 1 -- 172.21.15.151:0/2180705269 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5613afa059b0 msgr2=0x5613afa194b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:52.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.823+0000 7ff835400700 1 --2- 172.21.15.151:0/2180705269 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5613afa059b0 0x5613afa194b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:52.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.823+0000 7ff835400700 1 -- 172.21.15.151:0/2180705269 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613afa16590 con 0x5613afa11180 2024-04-26T02:02:52.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.823+0000 7ff834a00700 1 --2- 172.21.15.151:0/2180705269 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5613afa059b0 0x5613afa194b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T02:02:52.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.823+0000 7ff82fe00700 1 --2- 172.21.15.151:0/2180705269 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5613afa062d0 0x5613afa15840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:02:52.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.824+0000 7ff835400700 1 --2- 172.21.15.151:0/2180705269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613afa11180 0x5613afa15d80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff8280079c0 tx=0x7ff828008ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2abaea36000982a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:52.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.825+0000 7ff81ea00700 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-26T02:02:52.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.825+0000 7ff81ea00700 1 -- 172.21.15.151:0/2180705269 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff828009460 con 0x5613afa11180 2024-04-26T02:02:52.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.825+0000 7ff81ea00700 1 -- 172.21.15.151:0/2180705269 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8455+0+0 (secure 0 0 0) 0x7ff828015690 con 0x5613afa11180 2024-04-26T02:02:52.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.825+0000 7ff841d32b80 1 -- 172.21.15.151:0/2180705269 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff82009d790 con 0x5613afa11180 2024-04-26T02:02:52.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.826+0000 7ff81ea00700 1 -- 172.21.15.151:0/2180705269 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff828015890 con 0x5613afa11180 2024-04-26T02:02:52.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.826+0000 7ff841d32b80 10 client.26266 fetch_fsmap learned FSMap version 35 2024-04-26T02:02:52.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.826+0000 7ff841d32b80 1 -- 172.21.15.151:0/2180705269 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5613afa87eb0 con 0x5613afa11180 2024-04-26T02:02:52.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.826+0000 7ff81ea00700 1 -- 172.21.15.151:0/2180705269 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff82009d790 con 0x5613afa11180 2024-04-26T02:02:52.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.827+0000 7ff81ea00700 1 -- 172.21.15.151:0/2180705269 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 35) ==== 8338+0+0 (secure 0 0 0) 0x7ff828026020 con 0x5613afa11180 2024-04-26T02:02:52.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.827+0000 7ff841d32b80 10 client.26266 fetch_fsmap finished waiting for FSMap version 35 2024-04-26T02:02:52.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.827+0000 7ff841d32b80 10 client.26266 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-26T02:02:52.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.827+0000 7ff841d32b80 20 client.26266 populate_metadata read hostname 'smithi151' 2024-04-26T02:02:52.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.827+0000 7ff841d32b80 1 --2- 172.21.15.151:0/2180705269 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5613afa20950 0x5613afa8a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:52.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.827+0000 7ff841d32b80 4 client.26266 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 82.1, "await_for": 42.0}] 2024-04-26T02:02:52.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.827+0000 7ff841d32b80 1 -- 172.21.15.151:0/2180705269 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 82.1, "await_for": 42.0}) -- 0x5613af966810 con 0x5613afa20950 2024-04-26T02:02:52.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.827+0000 7ff834a00700 1 --2- 172.21.15.151:0/2180705269 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5613afa20950 0x5613afa8a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:52.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.828+0000 7ff834a00700 1 --2- 172.21.15.151:0/2180705269 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5613afa20950 0x5613afa8a430 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3621ff868b966b34 server_cookie=3371b6b40f5da419 in_seq=0 out_seq=0 2024-04-26T02:02:52.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:52.828+0000 7ff81ea00700 10 client.26266 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:02:53.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:53.756+0000 7ff81ea00700 1 -- 172.21.15.151:0/2180705269 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5613af966810 con 0x5613afa20950 2024-04-26T02:02:53.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:53.756+0000 7ff81ea00700 10 client.26266 handle_command_reply: tid=0 2024-04-26T02:02:53.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:53.756+0000 7ff841d32b80 1 client.26266 shutdown 2024-04-26T02:02:53.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:53.757+0000 7ff841d32b80 1 -- 172.21.15.151:0/2180705269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613afa11180 msgr2=0x5613afa15d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:53.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:53.757+0000 7ff841d32b80 1 --2- 172.21.15.151:0/2180705269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613afa11180 0x5613afa15d80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff8280079c0 tx=0x7ff828008ce0 comp rx=0 tx=0).stop 2024-04-26T02:02:53.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:53.757+0000 7ff841d32b80 1 -- 172.21.15.151:0/2180705269 shutdown_connections 2024-04-26T02:02:53.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:53.757+0000 7ff841d32b80 1 --2- 172.21.15.151:0/2180705269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613afa11180 0x5613afa15d80 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:53.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:53.757+0000 7ff841d32b80 1 --2- 172.21.15.151:0/2180705269 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5613afa059b0 0x5613afa194b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:53.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:53.757+0000 7ff841d32b80 1 --2- 172.21.15.151:0/2180705269 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5613afa20950 0x5613afa8a430 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:53.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:53.757+0000 7ff841d32b80 1 --2- 172.21.15.151:0/2180705269 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5613afa062d0 0x5613afa15840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:53.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:53.757+0000 7ff841d32b80 1 -- 172.21.15.151:0/2180705269 >> 172.21.15.151:0/2180705269 conn(0x5613af934730 msgr2=0x5613afa0f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:53.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:53.757+0000 7ff81ea00700 0 client.26266 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:02:53.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:53.757+0000 7ff841d32b80 1 -- 172.21.15.151:0/2180705269 shutdown_connections 2024-04-26T02:02:53.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:53.757+0000 7ff841d32b80 1 -- 172.21.15.151:0/2180705269 wait complete. 2024-04-26T02:02:53.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:53.758+0000 7ff841d32b80 20 client.26266 trim_cache size 0 max 16384 2024-04-26T02:02:53.758 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-26T02:02:53.758 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 35, 2024-04-26T02:02:53.758 INFO:teuthology.orchestra.run.smithi151.stdout: "leader": 24509, 2024-04-26T02:02:53.758 INFO:teuthology.orchestra.run.smithi151.stdout: "set_version": 26, 2024-04-26T02:02:53.758 INFO:teuthology.orchestra.run.smithi151.stdout: "sets": { 2024-04-26T02:02:53.762 INFO:teuthology.orchestra.run.smithi151.stdout: "6face347": { 2024-04-26T02:02:53.762 INFO:teuthology.orchestra.run.smithi151.stdout: "version": 26, 2024-04-26T02:02:53.762 INFO:teuthology.orchestra.run.smithi151.stdout: "age_ref": 0.0, 2024-04-26T02:02:53.762 INFO:teuthology.orchestra.run.smithi151.stdout: "state": { 2024-04-26T02:02:53.762 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "QUIESCED", 2024-04-26T02:02:53.762 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 0.0 2024-04-26T02:02:53.762 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:02:53.763 INFO:teuthology.orchestra.run.smithi151.stdout: "timeout": 60.0, 2024-04-26T02:02:53.763 INFO:teuthology.orchestra.run.smithi151.stdout: "expiration": 82.1, 2024-04-26T02:02:53.763 INFO:teuthology.orchestra.run.smithi151.stdout: "members": { 2024-04-26T02:02:53.763 INFO:teuthology.orchestra.run.smithi151.stdout: "file:/": { 2024-04-26T02:02:53.763 INFO:teuthology.orchestra.run.smithi151.stdout: "excluded": false, 2024-04-26T02:02:53.763 INFO:teuthology.orchestra.run.smithi151.stdout: "state": { 2024-04-26T02:02:53.763 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "QUIESCED", 2024-04-26T02:02:53.763 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 0.0 2024-04-26T02:02:53.763 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:02:53.763 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:02:53.763 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:02:53.763 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:02:53.763 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:02:53.763 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-26T02:02:53.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:53.759+0000 7ff837400700 1 -- 172.21.15.151:0/4248315204 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7ff8100a3f20 msgr2=0x7ff8100a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:53.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:53.759+0000 7ff837400700 1 --2- 172.21.15.151:0/4248315204 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7ff8100a3f20 0x7ff8100a63e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff818005950 tx=0x7ff81800a300 comp rx=0 tx=0).stop 2024-04-26T02:02:53.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:53.759+0000 7ff837400700 1 -- 172.21.15.151:0/4248315204 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff83010a360 msgr2=0x7ff8301a9c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:53.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:53.759+0000 7ff837400700 1 --2- 172.21.15.151:0/4248315204 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff83010a360 0x7ff8301a9c90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff820000c00 tx=0x7ff820005590 comp rx=0 tx=0).stop 2024-04-26T02:02:53.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:53.759+0000 7ff837400700 1 -- 172.21.15.151:0/4248315204 shutdown_connections 2024-04-26T02:02:53.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:53.759+0000 7ff837400700 1 --2- 172.21.15.151:0/4248315204 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7ff8100a3f20 0x7ff8100a63e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:53.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:53.759+0000 7ff837400700 1 --2- 172.21.15.151:0/4248315204 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff830117f80 0x7ff8301a3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:53.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:53.759+0000 7ff837400700 1 --2- 172.21.15.151:0/4248315204 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff830109a40 0x7ff8301a9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:53.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:53.759+0000 7ff837400700 1 --2- 172.21.15.151:0/4248315204 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff83010a360 0x7ff8301a9c90 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:53.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:53.759+0000 7ff837400700 1 -- 172.21.15.151:0/4248315204 >> 172.21.15.151:0/4248315204 conn(0x7ff83006dbb0 msgr2=0x7ff8301184a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:53.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:53.760+0000 7ff837400700 1 -- 172.21.15.151:0/4248315204 shutdown_connections 2024-04-26T02:02:53.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:02:53.760+0000 7ff837400700 1 -- 172.21.15.151:0/4248315204 wait complete. 2024-04-26T02:02:54.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:02:54 smithi172 ceph-mon[32785]: pgmap v290: 97 pgs: 97 active+clean; 107 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 1.4 MiB/s wr, 442 op/s 2024-04-26T02:02:54.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:02:54 smithi165 ceph-mon[32648]: pgmap v290: 97 pgs: 97 active+clean; 107 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 1.4 MiB/s wr, 442 op/s 2024-04-26T02:02:54.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:02:54 smithi151 ceph-mon[28057]: pgmap v290: 97 pgs: 97 active+clean; 107 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 1.4 MiB/s wr, 442 op/s 2024-04-26T02:02:54.566 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '6face347', quiesce took 8.8 seconds. Will release after: 22.1 2024-04-26T02:02:56.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:02:56 smithi172 ceph-mon[32785]: pgmap v291: 97 pgs: 97 active+clean; 107 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 1.3 MiB/s wr, 449 op/s 2024-04-26T02:02:56.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:02:56 smithi165 ceph-mon[32648]: pgmap v291: 97 pgs: 97 active+clean; 107 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 1.3 MiB/s wr, 449 op/s 2024-04-26T02:02:56.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:02:56 smithi151 ceph-mon[28057]: pgmap v291: 97 pgs: 97 active+clean; 107 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 1.3 MiB/s wr, 449 op/s 2024-04-26T02:02:56.643 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.644+0000 7fceb3e00640 1 -- 172.21.15.151:0/903493449 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fceb41038f0 msgr2=0x7fceb410bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:56.644 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.644+0000 7fceb3e00640 1 --2- 172.21.15.151:0/903493449 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fceb41038f0 0x7fceb410bdd0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fce9c009900 tx=0x7fce9c02a480 comp rx=0 tx=0).stop 2024-04-26T02:02:56.644 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.644+0000 7fceb3e00640 1 -- 172.21.15.151:0/903493449 shutdown_connections 2024-04-26T02:02:56.644 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.644+0000 7fceb3e00640 1 --2- 172.21.15.151:0/903493449 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fceb410c3a0 0x7fceb410e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:56.644 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.644+0000 7fceb3e00640 1 --2- 172.21.15.151:0/903493449 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fceb41038f0 0x7fceb410bdd0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:56.644 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.644+0000 7fceb3e00640 1 --2- 172.21.15.151:0/903493449 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fceb4102fd0 0x7fceb41033b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:56.644 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.644+0000 7fceb3e00640 1 -- 172.21.15.151:0/903493449 >> 172.21.15.151:0/903493449 conn(0x7fceb40fc770 msgr2=0x7fceb40feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:56.644 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.644+0000 7fceb3e00640 1 -- 172.21.15.151:0/903493449 shutdown_connections 2024-04-26T02:02:56.644 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.645+0000 7fceb3e00640 1 -- 172.21.15.151:0/903493449 wait complete. 2024-04-26T02:02:56.646 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.645+0000 7fceb3e00640 1 Processor -- start 2024-04-26T02:02:56.646 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.646+0000 7fceb3e00640 1 -- start start 2024-04-26T02:02:56.646 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.646+0000 7fceb3e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fceb4102fd0 0x7fceb41a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:56.646 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.646+0000 7fceb3e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fceb41038f0 0x7fceb41a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:56.646 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.646+0000 7fceb3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fceb410c3a0 0x7fceb419a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:56.646 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.646+0000 7fceb3e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fceb4112010 con 0x7fceb4102fd0 2024-04-26T02:02:56.646 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.646+0000 7fceb3e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fceb4111e90 con 0x7fceb410c3a0 2024-04-26T02:02:56.646 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.646+0000 7fceb3e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fceb4112190 con 0x7fceb41038f0 2024-04-26T02:02:56.646 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.646+0000 7fceb2a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fceb4102fd0 0x7fceb41a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:56.646 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.646+0000 7fceb2a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fceb4102fd0 0x7fceb41a05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47584/0 (socket says 172.21.15.151:47584) 2024-04-26T02:02:56.646 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.646+0000 7fceb2a00640 1 -- 172.21.15.151:0/689531833 learned_addr learned my addr 172.21.15.151:0/689531833 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:02:56.647 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.647+0000 7fceb3400640 1 --2- 172.21.15.151:0/689531833 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fceb410c3a0 0x7fceb419a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:56.647 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.647+0000 7fceabe00640 1 --2- 172.21.15.151:0/689531833 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fceb41038f0 0x7fceb41a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:56.647 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.647+0000 7fceb2a00640 1 -- 172.21.15.151:0/689531833 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fceb41038f0 msgr2=0x7fceb41a0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:56.647 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.647+0000 7fceb2a00640 1 --2- 172.21.15.151:0/689531833 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fceb41038f0 0x7fceb41a0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:56.647 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.647+0000 7fceb2a00640 1 -- 172.21.15.151:0/689531833 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fceb410c3a0 msgr2=0x7fceb419a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:56.647 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.647+0000 7fceb2a00640 1 --2- 172.21.15.151:0/689531833 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fceb410c3a0 0x7fceb419a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:56.647 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.647+0000 7fceb2a00640 1 -- 172.21.15.151:0/689531833 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceb419af30 con 0x7fceb4102fd0 2024-04-26T02:02:56.647 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.647+0000 7fceb3400640 1 --2- 172.21.15.151:0/689531833 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fceb410c3a0 0x7fceb419a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:02:56.647 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.647+0000 7fceabe00640 1 --2- 172.21.15.151:0/689531833 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fceb41038f0 0x7fceb41a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:02:56.648 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.647+0000 7fceb2a00640 1 --2- 172.21.15.151:0/689531833 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fceb4102fd0 0x7fceb41a05b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fcea400e9f0 tx=0x7fcea400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:56.648 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.647+0000 7fceb8a00640 1 -- 172.21.15.151:0/689531833 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea400cd80 con 0x7fceb4102fd0 2024-04-26T02:02:56.648 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.647+0000 7fceb8a00640 1 -- 172.21.15.151:0/689531833 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcea40108e0 con 0x7fceb4102fd0 2024-04-26T02:02:56.648 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.647+0000 7fceb8a00640 1 -- 172.21.15.151:0/689531833 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea4018980 con 0x7fceb4102fd0 2024-04-26T02:02:56.648 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.648+0000 7fceb3e00640 1 -- 172.21.15.151:0/689531833 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fceb419b220 con 0x7fceb4102fd0 2024-04-26T02:02:56.648 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.648+0000 7fceb3e00640 1 -- 172.21.15.151:0/689531833 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fceb41a7530 con 0x7fceb4102fd0 2024-04-26T02:02:56.650 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.651+0000 7fcebcc12740 1 Processor -- start 2024-04-26T02:02:56.650 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.651+0000 7fceb8a00640 1 -- 172.21.15.151:0/689531833 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7fcea4010430 con 0x7fceb4102fd0 2024-04-26T02:02:56.650 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.651+0000 7fcebcc12740 1 -- start start 2024-04-26T02:02:56.651 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.652+0000 7fcebcc12740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e02a1809b0 0x55e02a180d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:56.651 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.652+0000 7fcebcc12740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e02a1812d0 0x55e02a18b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.652+0000 7fcebcc12740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e02a18bb90 0x55e02a18df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.652+0000 7fcebcc12740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e02a0e6900 con 0x55e02a1812d0 2024-04-26T02:02:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.652+0000 7fcebcc12740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e02a08cef0 con 0x55e02a1809b0 2024-04-26T02:02:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.652+0000 7fcebcc12740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e02a0e28c0 con 0x55e02a18bb90 2024-04-26T02:02:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.652+0000 7fceabe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e02a18bb90 0x55e02a18df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.652+0000 7fceb2a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e02a1812d0 0x55e02a18b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.652+0000 7fceabe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e02a18bb90 0x55e02a18df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:54082/0 (socket says 172.21.15.151:54082) 2024-04-26T02:02:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.652+0000 7fceabe00640 1 -- 172.21.15.151:0/3269459458 learned_addr learned my addr 172.21.15.151:0/3269459458 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:02:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.652+0000 7fceb3400640 1 --2- 172.21.15.151:0/3269459458 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e02a1809b0 0x55e02a180d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.652+0000 7fceabe00640 1 -- 172.21.15.151:0/3269459458 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e02a1809b0 msgr2=0x55e02a180d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.652+0000 7fceabe00640 1 --2- 172.21.15.151:0/3269459458 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e02a1809b0 0x55e02a180d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.652+0000 7fceabe00640 1 -- 172.21.15.151:0/3269459458 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e02a1812d0 msgr2=0x55e02a18b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.652+0000 7fceabe00640 1 --2- 172.21.15.151:0/3269459458 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e02a1812d0 0x55e02a18b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.653+0000 7fceabe00640 1 -- 172.21.15.151:0/3269459458 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e029df9210 con 0x55e02a18bb90 2024-04-26T02:02:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.653+0000 7fceabe00640 1 --2- 172.21.15.151:0/3269459458 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e02a18bb90 0x55e02a18df80 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fce9c031440 tx=0x7fce9c0042e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c8eed0d24a0366a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.653+0000 7fcea3e00640 1 -- 172.21.15.151:0/3269459458 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9c035030 con 0x55e02a18bb90 2024-04-26T02:02:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.653+0000 7fcea3e00640 1 -- 172.21.15.151:0/3269459458 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce9c0044d0 con 0x55e02a18bb90 2024-04-26T02:02:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.653+0000 7fcea3e00640 1 -- 172.21.15.151:0/3269459458 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9c0035f0 con 0x55e02a18bb90 2024-04-26T02:02:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.653+0000 7fceb8a00640 1 --2- 172.21.15.151:0/689531833 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fce980a3d90 0x7fce980a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.654+0000 7fceb8a00640 1 -- 172.21.15.151:0/689531833 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8455+0+0 (secure 0 0 0) 0x7fcea40d0bb0 con 0x7fceb4102fd0 2024-04-26T02:02:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.654+0000 7fceb3400640 1 --2- 172.21.15.151:0/689531833 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fce980a3d90 0x7fce980a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.654+0000 7fcebcc12740 1 -- 172.21.15.151:0/3269459458 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e02a18bb90 msgr2=0x55e02a18df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.654+0000 7fcebcc12740 1 --2- 172.21.15.151:0/3269459458 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e02a18bb90 0x55e02a18df80 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fce9c031440 tx=0x7fce9c0042e0 comp rx=0 tx=0).stop 2024-04-26T02:02:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.654+0000 7fcebcc12740 1 -- 172.21.15.151:0/3269459458 shutdown_connections 2024-04-26T02:02:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.654+0000 7fcebcc12740 1 --2- 172.21.15.151:0/3269459458 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e02a18bb90 0x55e02a18df80 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.654+0000 7fcebcc12740 1 --2- 172.21.15.151:0/3269459458 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e02a1812d0 0x55e02a18b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.654+0000 7fcebcc12740 1 --2- 172.21.15.151:0/3269459458 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e02a1809b0 0x55e02a180d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.654+0000 7fcebcc12740 1 -- 172.21.15.151:0/3269459458 >> 172.21.15.151:0/3269459458 conn(0x55e02a17cec0 msgr2=0x55e02a17d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.655+0000 7fceb3400640 1 --2- 172.21.15.151:0/689531833 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fce980a3d90 0x7fce980a6250 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fceb406a6e0 tx=0x7fceac009500 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.655+0000 7fcebcc12740 1 -- 172.21.15.151:0/3269459458 shutdown_connections 2024-04-26T02:02:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.655+0000 7fcebcc12740 1 -- 172.21.15.151:0/3269459458 wait complete. 2024-04-26T02:02:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.655+0000 7fcebcc12740 1 -- start start 2024-04-26T02:02:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.655+0000 7fcebcc12740 1 Processor -- start 2024-04-26T02:02:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.656+0000 7fcebcc12740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e02a1809b0 0x55e02a188a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.656+0000 7fcebcc12740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e02a1812d0 0x55e02a188f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.656+0000 7fcebcc12740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e02a18bb90 0x55e02a1894c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:56.657 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.656+0000 7fcebcc12740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e02a18e760 con 0x55e02a1812d0 2024-04-26T02:02:56.657 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.656+0000 7fcebcc12740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e02a0a0310 con 0x55e02a18bb90 2024-04-26T02:02:56.657 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.656+0000 7fcebcc12740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e02a18e8e0 con 0x55e02a1809b0 2024-04-26T02:02:56.657 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.656+0000 7fcebcc12740 10 client.0 _ll_register_callbacks cb 0x55e02a06ecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:02:56.657 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.656+0000 7fceabe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e02a1812d0 0x55e02a188f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:56.657 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.656+0000 7fceb2a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e02a1809b0 0x55e02a188a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:56.657 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.656+0000 7fceb2a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e02a1809b0 0x55e02a188a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:54096/0 (socket says 172.21.15.151:54096) 2024-04-26T02:02:56.657 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.656+0000 7fceb2a00640 1 -- 172.21.15.151:0/2358849381 learned_addr learned my addr 172.21.15.151:0/2358849381 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:02:56.657 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.656+0000 7fceb3400640 1 --2- 172.21.15.151:0/2358849381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e02a18bb90 0x55e02a1894c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.656+0000 7fceabe00640 1 -- 172.21.15.151:0/2358849381 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e02a1809b0 msgr2=0x55e02a188a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.656+0000 7fceabe00640 1 --2- 172.21.15.151:0/2358849381 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e02a1809b0 0x55e02a188a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.656+0000 7fceabe00640 1 -- 172.21.15.151:0/2358849381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e02a18bb90 msgr2=0x55e02a1894c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.656+0000 7fceabe00640 1 --2- 172.21.15.151:0/2358849381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e02a18bb90 0x55e02a1894c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.656+0000 7fceabe00640 1 -- 172.21.15.151:0/2358849381 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e029ec5620 con 0x55e02a1812d0 2024-04-26T02:02:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.657+0000 7fceabe00640 1 --2- 172.21.15.151:0/2358849381 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e02a1812d0 0x55e02a188f80 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fce9c006010 tx=0x7fce9c004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f2ec3f191caf299 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.657+0000 7fcea0c00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-04-26T02:02:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.657+0000 7fcea0c00640 1 -- 172.21.15.151:0/2358849381 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9c035070 con 0x55e02a1812d0 2024-04-26T02:02:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.657+0000 7fcea0c00640 1 -- 172.21.15.151:0/2358849381 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8455+0+0 (secure 0 0 0) 0x7fce9c0052d0 con 0x55e02a1812d0 2024-04-26T02:02:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.657+0000 7fcebcc12740 1 -- 172.21.15.151:0/2358849381 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce9c0044d0 con 0x55e02a1812d0 2024-04-26T02:02:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.657+0000 7fcea0c00640 1 -- 172.21.15.151:0/2358849381 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fce9c005d50 con 0x55e02a1812d0 2024-04-26T02:02:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.657+0000 7fcebcc12740 10 client.16155 fetch_fsmap learned FSMap version 35 2024-04-26T02:02:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.657+0000 7fcebcc12740 1 -- 172.21.15.151:0/2358849381 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e02a19a920 con 0x55e02a1812d0 2024-04-26T02:02:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.658+0000 7fcea0c00640 1 -- 172.21.15.151:0/2358849381 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce9c0044d0 con 0x55e02a1812d0 2024-04-26T02:02:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.658+0000 7fcea0c00640 1 -- 172.21.15.151:0/2358849381 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 35) ==== 8338+0+0 (secure 0 0 0) 0x7fce9c056440 con 0x55e02a1812d0 2024-04-26T02:02:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.658+0000 7fcebcc12740 10 client.16155 fetch_fsmap finished waiting for FSMap version 35 2024-04-26T02:02:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.658+0000 7fcebcc12740 10 client.16155 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:02:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.658+0000 7fcebcc12740 20 client.16155 populate_metadata read hostname 'smithi151' 2024-04-26T02:02:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.658+0000 7fcebcc12740 1 --2- 172.21.15.151:0/2358849381 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55e02a19bc70 0x55e02a19e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.658+0000 7fcebcc12740 4 client.16155 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:02:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.658+0000 7fcebcc12740 1 -- 172.21.15.151:0/2358849381 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e02a0fa8b0 con 0x55e02a19bc70 2024-04-26T02:02:56.662 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.658+0000 7fceb2a00640 1 --2- 172.21.15.151:0/2358849381 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55e02a19bc70 0x55e02a19e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:56.662 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.659+0000 7fceb2a00640 1 --2- 172.21.15.151:0/2358849381 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55e02a19bc70 0x55e02a19e060 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e4919a669436769 server_cookie=2467967f07f5265a in_seq=0 out_seq=0 2024-04-26T02:02:56.662 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.659+0000 7fcea0c00640 10 client.16155 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:02:56.662 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.660+0000 7fcea0c00640 1 -- 172.21.15.151:0/2358849381 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e02a0fa8b0 con 0x55e02a19bc70 2024-04-26T02:02:56.662 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.660+0000 7fcea0c00640 10 client.16155 handle_command_reply: tid=0 2024-04-26T02:02:56.662 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.660+0000 7fcebcc12740 1 client.16155 shutdown 2024-04-26T02:02:56.662 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.660+0000 7fcebcc12740 1 -- 172.21.15.151:0/2358849381 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e02a1812d0 msgr2=0x55e02a188f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:56.662 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.660+0000 7fcebcc12740 1 --2- 172.21.15.151:0/2358849381 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e02a1812d0 0x55e02a188f80 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fce9c006010 tx=0x7fce9c004ed0 comp rx=0 tx=0).stop 2024-04-26T02:02:56.662 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.660+0000 7fcebcc12740 1 -- 172.21.15.151:0/2358849381 shutdown_connections 2024-04-26T02:02:56.662 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.660+0000 7fcebcc12740 1 --2- 172.21.15.151:0/2358849381 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55e02a19bc70 0x55e02a19e060 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:56.662 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.660+0000 7fcebcc12740 1 --2- 172.21.15.151:0/2358849381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e02a18bb90 0x55e02a1894c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:56.662 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.660+0000 7fcebcc12740 1 --2- 172.21.15.151:0/2358849381 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e02a1812d0 0x55e02a188f80 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:56.662 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.660+0000 7fcebcc12740 1 --2- 172.21.15.151:0/2358849381 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e02a1809b0 0x55e02a188a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:56.663 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.660+0000 7fcea0c00640 0 client.16155 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:02:56.663 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.660+0000 7fcebcc12740 1 -- 172.21.15.151:0/2358849381 >> 172.21.15.151:0/2358849381 conn(0x55e02a17cec0 msgr2=0x55e02a182c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:56.663 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.660+0000 7fcebcc12740 1 -- 172.21.15.151:0/2358849381 shutdown_connections 2024-04-26T02:02:56.663 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.660+0000 7fcebcc12740 1 -- 172.21.15.151:0/2358849381 wait complete. 2024-04-26T02:02:56.663 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.660+0000 7fcebcc12740 20 client.16155 trim_cache size 0 max 16384 2024-04-26T02:02:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.677+0000 7fcebcc12740 1 Processor -- start 2024-04-26T02:02:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.677+0000 7fcebcc12740 1 -- start start 2024-04-26T02:02:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.677+0000 7fcebcc12740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e02a1809b0 0x55e02a197600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.677+0000 7fcebcc12740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e02a1812d0 0x55e02a1833c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.677+0000 7fcebcc12740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e02a18bb90 0x55e02a17b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.677+0000 7fcebcc12740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fceb41b2350 con 0x55e02a1812d0 2024-04-26T02:02:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.677+0000 7fcebcc12740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fceb41afc20 con 0x55e02a18bb90 2024-04-26T02:02:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.677+0000 7fcebcc12740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fceb41ada90 con 0x55e02a1809b0 2024-04-26T02:02:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.677+0000 7fceabe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e02a1812d0 0x55e02a1833c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.677+0000 7fceb2a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e02a1809b0 0x55e02a197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.677+0000 7fceb2a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e02a1809b0 0x55e02a197600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:54100/0 (socket says 172.21.15.151:54100) 2024-04-26T02:02:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.677+0000 7fceb2a00640 1 -- 172.21.15.151:0/2209436506 learned_addr learned my addr 172.21.15.151:0/2209436506 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:02:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.677+0000 7fceabe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e02a1812d0 0x55e02a1833c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47606/0 (socket says 172.21.15.151:47606) 2024-04-26T02:02:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.677+0000 7fceb3400640 1 --2- 172.21.15.151:0/2209436506 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e02a18bb90 0x55e02a17b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.678+0000 7fceb2a00640 1 -- 172.21.15.151:0/2209436506 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e02a18bb90 msgr2=0x55e02a17b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.678+0000 7fceb2a00640 1 --2- 172.21.15.151:0/2209436506 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e02a18bb90 0x55e02a17b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.678+0000 7fceb2a00640 1 -- 172.21.15.151:0/2209436506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e02a1812d0 msgr2=0x55e02a1833c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:56.679 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.678+0000 7fceb2a00640 1 --2- 172.21.15.151:0/2209436506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e02a1812d0 0x55e02a1833c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:56.679 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.678+0000 7fceb2a00640 1 -- 172.21.15.151:0/2209436506 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e02a181100 con 0x55e02a1809b0 2024-04-26T02:02:56.679 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.678+0000 7fceb2a00640 1 --2- 172.21.15.151:0/2209436506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e02a1809b0 0x55e02a197600 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fcea4099540 tx=0x7fcea40bd040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da92b6e54203bf88 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:56.679 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.678+0000 7fcea3400640 1 -- 172.21.15.151:0/2209436506 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea409c4d0 con 0x55e02a1809b0 2024-04-26T02:02:56.679 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.678+0000 7fcea3400640 1 -- 172.21.15.151:0/2209436506 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcea4010720 con 0x55e02a1809b0 2024-04-26T02:02:56.679 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.678+0000 7fcea3400640 1 -- 172.21.15.151:0/2209436506 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea409ac80 con 0x55e02a1809b0 2024-04-26T02:02:56.683 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.679+0000 7fcebcc12740 1 -- 172.21.15.151:0/2209436506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e02a1809b0 msgr2=0x55e02a197600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:56.683 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.679+0000 7fcebcc12740 1 --2- 172.21.15.151:0/2209436506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e02a1809b0 0x55e02a197600 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fcea4099540 tx=0x7fcea40bd040 comp rx=0 tx=0).stop 2024-04-26T02:02:56.683 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.679+0000 7fcebcc12740 1 -- 172.21.15.151:0/2209436506 shutdown_connections 2024-04-26T02:02:56.683 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.679+0000 7fcebcc12740 1 --2- 172.21.15.151:0/2209436506 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e02a18bb90 0x55e02a17b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:56.683 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.679+0000 7fcebcc12740 1 --2- 172.21.15.151:0/2209436506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e02a1812d0 0x55e02a1833c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:56.683 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.679+0000 7fcebcc12740 1 --2- 172.21.15.151:0/2209436506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e02a1809b0 0x55e02a197600 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:56.683 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.679+0000 7fcebcc12740 1 -- 172.21.15.151:0/2209436506 >> 172.21.15.151:0/2209436506 conn(0x55e02a17cec0 msgr2=0x55e02a189290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:56.683 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.679+0000 7fcebcc12740 1 -- 172.21.15.151:0/2209436506 shutdown_connections 2024-04-26T02:02:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.679+0000 7fcebcc12740 1 -- 172.21.15.151:0/2209436506 wait complete. 2024-04-26T02:02:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.679+0000 7fcebcc12740 1 -- start start 2024-04-26T02:02:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.680+0000 7fcebcc12740 1 Processor -- start 2024-04-26T02:02:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.680+0000 7fcebcc12740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e02a1809b0 0x55e02a1a04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.680+0000 7fcebcc12740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e02a1812d0 0x55e02a1a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.680+0000 7fcebcc12740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e02a18bb90 0x55e02a1f9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.680+0000 7fcebcc12740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fceb41ae0e0 con 0x55e02a1809b0 2024-04-26T02:02:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.680+0000 7fcebcc12740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fceb41af550 con 0x55e02a1812d0 2024-04-26T02:02:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.680+0000 7fcebcc12740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fceb41ae7b0 con 0x55e02a18bb90 2024-04-26T02:02:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.680+0000 7fcebcc12740 10 client.0 _ll_register_callbacks cb 0x7fceac0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:02:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.680+0000 7fceb2a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e02a1809b0 0x55e02a1a04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.680+0000 7fceb2a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e02a1809b0 0x55e02a1a04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47608/0 (socket says 172.21.15.151:47608) 2024-04-26T02:02:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.680+0000 7fceb2a00640 1 -- 172.21.15.151:0/1495424315 learned_addr learned my addr 172.21.15.151:0/1495424315 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:02:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.680+0000 7fceabe00640 1 --2- 172.21.15.151:0/1495424315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e02a1812d0 0x55e02a1a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.680+0000 7fceb3400640 1 --2- 172.21.15.151:0/1495424315 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e02a18bb90 0x55e02a1f9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.681+0000 7fceb2a00640 1 -- 172.21.15.151:0/1495424315 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e02a18bb90 msgr2=0x55e02a1f9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.681+0000 7fceb2a00640 1 --2- 172.21.15.151:0/1495424315 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e02a18bb90 0x55e02a1f9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.681+0000 7fceb2a00640 1 -- 172.21.15.151:0/1495424315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e02a1812d0 msgr2=0x55e02a1a0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.681+0000 7fceb2a00640 1 --2- 172.21.15.151:0/1495424315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e02a1812d0 0x55e02a1a0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.681+0000 7fceb2a00640 1 -- 172.21.15.151:0/1495424315 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e02a1fa560 con 0x55e02a1809b0 2024-04-26T02:02:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.681+0000 7fceb3400640 1 --2- 172.21.15.151:0/1495424315 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e02a18bb90 0x55e02a1f9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T02:02:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.681+0000 7fceabe00640 1 --2- 172.21.15.151:0/1495424315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e02a1812d0 0x55e02a1a0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:02:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.681+0000 7fceb2a00640 1 --2- 172.21.15.151:0/1495424315 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e02a1809b0 0x55e02a1a04d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fcea400c6a0 tx=0x7fcea409c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91bd64ab0b44fb77 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.681+0000 7fcea0c00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-04-26T02:02:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.681+0000 7fcea0c00640 1 -- 172.21.15.151:0/1495424315 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea40a57e0 con 0x55e02a1809b0 2024-04-26T02:02:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.681+0000 7fcea0c00640 1 -- 172.21.15.151:0/1495424315 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8455+0+0 (secure 0 0 0) 0x7fcea40bd0b0 con 0x55e02a1809b0 2024-04-26T02:02:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.681+0000 7fcebcc12740 1 -- 172.21.15.151:0/1495424315 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e02a0f4320 con 0x55e02a1809b0 2024-04-26T02:02:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.681+0000 7fcea0c00640 1 -- 172.21.15.151:0/1495424315 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fcea409d270 con 0x55e02a1809b0 2024-04-26T02:02:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.681+0000 7fcebcc12740 10 client.16161 fetch_fsmap learned FSMap version 35 2024-04-26T02:02:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.681+0000 7fcebcc12740 1 -- 172.21.15.151:0/1495424315 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e02a2018f0 con 0x55e02a1809b0 2024-04-26T02:02:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.682+0000 7fcea0c00640 1 -- 172.21.15.151:0/1495424315 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55e02a0f4320 con 0x55e02a1809b0 2024-04-26T02:02:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.682+0000 7fcea0c00640 1 -- 172.21.15.151:0/1495424315 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 35) ==== 8338+0+0 (secure 0 0 0) 0x7fcea40b7020 con 0x55e02a1809b0 2024-04-26T02:02:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.682+0000 7fcebcc12740 10 client.16161 fetch_fsmap finished waiting for FSMap version 35 2024-04-26T02:02:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.682+0000 7fcebcc12740 10 client.16161 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:02:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.682+0000 7fcebcc12740 20 client.16161 populate_metadata read hostname 'smithi151' 2024-04-26T02:02:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.682+0000 7fcebcc12740 1 --2- 172.21.15.151:0/1495424315 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55e02a202040 0x55e02a204460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.682+0000 7fcebcc12740 4 client.16161 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T02:02:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.682+0000 7fcebcc12740 1 -- 172.21.15.151:0/1495424315 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e02a0ee190 con 0x55e02a202040 2024-04-26T02:02:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.682+0000 7fceabe00640 1 --2- 172.21.15.151:0/1495424315 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55e02a202040 0x55e02a204460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.683+0000 7fceabe00640 1 --2- 172.21.15.151:0/1495424315 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55e02a202040 0x55e02a204460 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf770db1646f2f1b server_cookie=65ee809f02c43bc3 in_seq=0 out_seq=0 2024-04-26T02:02:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.683+0000 7fcea0c00640 10 client.16161 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:02:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.698+0000 7fcea0c00640 1 -- 172.21.15.151:0/1495424315 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e02a0ee190 con 0x55e02a202040 2024-04-26T02:02:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.698+0000 7fcea0c00640 10 client.16161 handle_command_reply: tid=0 2024-04-26T02:02:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.698+0000 7fcebcc12740 1 client.16161 shutdown 2024-04-26T02:02:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.698+0000 7fcebcc12740 1 -- 172.21.15.151:0/1495424315 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e02a1809b0 msgr2=0x55e02a1a04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.698+0000 7fcebcc12740 1 --2- 172.21.15.151:0/1495424315 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e02a1809b0 0x55e02a1a04d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fcea400c6a0 tx=0x7fcea409c5d0 comp rx=0 tx=0).stop 2024-04-26T02:02:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.698+0000 7fcebcc12740 1 -- 172.21.15.151:0/1495424315 shutdown_connections 2024-04-26T02:02:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.698+0000 7fcebcc12740 1 --2- 172.21.15.151:0/1495424315 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55e02a202040 0x55e02a204460 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.698+0000 7fcebcc12740 1 --2- 172.21.15.151:0/1495424315 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e02a18bb90 0x55e02a1f9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.698+0000 7fcebcc12740 1 --2- 172.21.15.151:0/1495424315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e02a1812d0 0x55e02a1a0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.698+0000 7fcebcc12740 1 --2- 172.21.15.151:0/1495424315 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e02a1809b0 0x55e02a1a04d0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.698+0000 7fcea0c00640 0 client.16161 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:02:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.698+0000 7fcebcc12740 1 -- 172.21.15.151:0/1495424315 >> 172.21.15.151:0/1495424315 conn(0x55e02a17cec0 msgr2=0x55e02a186f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.699+0000 7fcebcc12740 1 -- 172.21.15.151:0/1495424315 shutdown_connections 2024-04-26T02:02:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.699+0000 7fcebcc12740 1 -- 172.21.15.151:0/1495424315 wait complete. 2024-04-26T02:02:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.699+0000 7fcebcc12740 20 client.16161 trim_cache size 0 max 16384 2024-04-26T02:02:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:{ 2024-04-26T02:02:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "message": "", 2024-04-26T02:02:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "return_code": 0 2024-04-26T02:02:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:} 2024-04-26T02:02:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.700+0000 7fceb3e00640 1 -- 172.21.15.151:0/689531833 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fce980a3d90 msgr2=0x7fce980a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.700+0000 7fceb3e00640 1 --2- 172.21.15.151:0/689531833 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fce980a3d90 0x7fce980a6250 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fceb406a6e0 tx=0x7fceac009500 comp rx=0 tx=0).stop 2024-04-26T02:02:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.700+0000 7fceb3e00640 1 -- 172.21.15.151:0/689531833 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fceb4102fd0 msgr2=0x7fceb41a05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.700+0000 7fceb3e00640 1 --2- 172.21.15.151:0/689531833 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fceb4102fd0 0x7fceb41a05b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fcea400e9f0 tx=0x7fcea400eeb0 comp rx=0 tx=0).stop 2024-04-26T02:02:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.700+0000 7fceb3e00640 1 -- 172.21.15.151:0/689531833 shutdown_connections 2024-04-26T02:02:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.700+0000 7fceb3e00640 1 --2- 172.21.15.151:0/689531833 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fce980a3d90 0x7fce980a6250 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.700+0000 7fceb3e00640 1 --2- 172.21.15.151:0/689531833 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fceb410c3a0 0x7fceb419a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.700+0000 7fceb3e00640 1 --2- 172.21.15.151:0/689531833 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fceb41038f0 0x7fceb41a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.700+0000 7fceb3e00640 1 --2- 172.21.15.151:0/689531833 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fceb4102fd0 0x7fceb41a05b0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.700+0000 7fceb3e00640 1 -- 172.21.15.151:0/689531833 >> 172.21.15.151:0/689531833 conn(0x7fceb40fc770 msgr2=0x7fceb4101d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.700+0000 7fceb3e00640 1 -- 172.21.15.151:0/689531833 shutdown_connections 2024-04-26T02:02:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:02:56.700+0000 7fceb3e00640 1 -- 172.21.15.151:0/689531833 wait complete. 2024-04-26T02:02:58.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:02:58 smithi172 ceph-mon[32785]: pgmap v292: 97 pgs: 97 active+clean; 107 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 1.1 MiB/s wr, 383 op/s 2024-04-26T02:02:58.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:02:58 smithi165 ceph-mon[32648]: pgmap v292: 97 pgs: 97 active+clean; 107 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 1.1 MiB/s wr, 383 op/s 2024-04-26T02:02:58.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:02:58 smithi151 ceph-mon[28057]: pgmap v292: 97 pgs: 97 active+clean; 107 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 1.1 MiB/s wr, 383 op/s 2024-04-26T02:02:59.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:02:59 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T02:02:59.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:02:59 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T02:02:59.371 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:02:59 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T02:03:00.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:03:00 smithi172 ceph-mon[32785]: pgmap v293: 97 pgs: 97 active+clean; 106 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 1.1 MiB/s wr, 390 op/s 2024-04-26T02:03:00.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:03:00 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:03:00.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:03:00 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:03:00.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:03:00 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T02:03:00.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:03:00 smithi165 ceph-mon[32648]: pgmap v293: 97 pgs: 97 active+clean; 106 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 1.1 MiB/s wr, 390 op/s 2024-04-26T02:03:00.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:03:00 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:03:00.347 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:03:00 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:03:00.347 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:03:00 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T02:03:00.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:03:00 smithi151 ceph-mon[28057]: pgmap v293: 97 pgs: 97 active+clean; 106 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 1.1 MiB/s wr, 390 op/s 2024-04-26T02:03:00.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:03:00 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:03:00.373 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:03:00 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:03:00.373 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:03:00 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T02:03:02.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:03:02 smithi165 ceph-mon[32648]: pgmap v294: 97 pgs: 97 active+clean; 106 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 606 KiB/s wr, 195 op/s 2024-04-26T02:03:02.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:03:02 smithi151 ceph-mon[28057]: pgmap v294: 97 pgs: 97 active+clean; 106 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 606 KiB/s wr, 195 op/s 2024-04-26T02:03:02.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:03:02 smithi172 ceph-mon[32785]: pgmap v294: 97 pgs: 97 active+clean; 106 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 606 KiB/s wr, 195 op/s 2024-04-26T02:03:04.347 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:03:04 smithi165 ceph-mon[32648]: pgmap v295: 97 pgs: 97 active+clean; 106 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 629 KiB/s wr, 200 op/s 2024-04-26T02:03:04.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:03:04 smithi151 ceph-mon[28057]: pgmap v295: 97 pgs: 97 active+clean; 106 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 629 KiB/s wr, 200 op/s 2024-04-26T02:03:04.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:03:04 smithi172 ceph-mon[32785]: pgmap v295: 97 pgs: 97 active+clean; 106 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 629 KiB/s wr, 200 op/s 2024-04-26T02:03:06.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:03:06 smithi151 ceph-mon[28057]: pgmap v296: 97 pgs: 97 active+clean; 106 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 248 KiB/s wr, 109 op/s 2024-04-26T02:03:06.339 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:03:06 smithi172 ceph-mon[32785]: pgmap v296: 97 pgs: 97 active+clean; 106 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 248 KiB/s wr, 109 op/s 2024-04-26T02:03:06.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:03:06 smithi165 ceph-mon[32648]: pgmap v296: 97 pgs: 97 active+clean; 106 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 248 KiB/s wr, 109 op/s 2024-04-26T02:03:08.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:03:08 smithi165 ceph-mon[32648]: pgmap v297: 97 pgs: 97 active+clean; 106 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 56 KiB/s wr, 26 op/s 2024-04-26T02:03:08.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:03:08 smithi151 ceph-mon[28057]: pgmap v297: 97 pgs: 97 active+clean; 106 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 56 KiB/s wr, 26 op/s 2024-04-26T02:03:08.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:03:08 smithi172 ceph-mon[32785]: pgmap v297: 97 pgs: 97 active+clean; 106 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 56 KiB/s wr, 26 op/s 2024-04-26T02:03:09.843 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.843+0000 7f992a200640 1 -- 172.21.15.151:0/4132084601 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f992410e620 msgr2=0x7f992410ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:09.843 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.843+0000 7f992a200640 1 --2- 172.21.15.151:0/4132084601 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f992410e620 0x7f992410ea00 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9914009900 tx=0x7f991402a3f0 comp rx=0 tx=0).stop 2024-04-26T02:03:09.843 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.843+0000 7f992a200640 1 -- 172.21.15.151:0/4132084601 shutdown_connections 2024-04-26T02:03:09.843 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.843+0000 7f992a200640 1 --2- 172.21.15.151:0/4132084601 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9924108f20 0x7f99241153e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:09.843 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.843+0000 7f992a200640 1 --2- 172.21.15.151:0/4132084601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9924108560 0x7f99241089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:09.843 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.843+0000 7f992a200640 1 --2- 172.21.15.151:0/4132084601 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f992410e620 0x7f992410ea00 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:09.843 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.843+0000 7f992a200640 1 -- 172.21.15.151:0/4132084601 >> 172.21.15.151:0/4132084601 conn(0x7f99240764e0 msgr2=0x7f9924078900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:09.843 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.843+0000 7f992a200640 1 -- 172.21.15.151:0/4132084601 shutdown_connections 2024-04-26T02:03:09.843 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.843+0000 7f992a200640 1 -- 172.21.15.151:0/4132084601 wait complete. 2024-04-26T02:03:09.846 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.844+0000 7f992a200640 1 Processor -- start 2024-04-26T02:03:09.846 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.844+0000 7f992a200640 1 -- start start 2024-04-26T02:03:09.846 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.845+0000 7f992a200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9924108560 0x7f99241a4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:09.846 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.845+0000 7f992a200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9924108f20 0x7f99241a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:09.846 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.845+0000 7f992a200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f992410e620 0x7f992419ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:09.846 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.845+0000 7f992a200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9924118790 con 0x7f9924108f20 2024-04-26T02:03:09.846 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.845+0000 7f992a200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9924118610 con 0x7f992410e620 2024-04-26T02:03:09.847 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.845+0000 7f992a200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9924118910 con 0x7f9924108560 2024-04-26T02:03:09.847 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.845+0000 7f991be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9924108f20 0x7f99241a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:09.847 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.845+0000 7f9923e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f992410e620 0x7f992419ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:09.847 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.845+0000 7f991be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9924108f20 0x7f99241a4dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:35346/0 (socket says 172.21.15.151:35346) 2024-04-26T02:03:09.847 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.845+0000 7f991be00640 1 -- 172.21.15.151:0/1067442355 learned_addr learned my addr 172.21.15.151:0/1067442355 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:03:09.847 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.845+0000 7f9923400640 1 --2- 172.21.15.151:0/1067442355 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9924108560 0x7f99241a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:09.847 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.846+0000 7f9923e00640 1 -- 172.21.15.151:0/1067442355 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9924108560 msgr2=0x7f99241a4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:09.847 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.846+0000 7f9923e00640 1 --2- 172.21.15.151:0/1067442355 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9924108560 0x7f99241a4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:09.847 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.846+0000 7f9923e00640 1 -- 172.21.15.151:0/1067442355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9924108f20 msgr2=0x7f99241a4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:09.847 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.846+0000 7f9923e00640 1 --2- 172.21.15.151:0/1067442355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9924108f20 0x7f99241a4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:09.847 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.846+0000 7f9923e00640 1 -- 172.21.15.151:0/1067442355 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f992419f1b0 con 0x7f992410e620 2024-04-26T02:03:09.847 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.846+0000 7f9923e00640 1 --2- 172.21.15.151:0/1067442355 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f992410e620 0x7f992419ea30 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f991c00ec50 tx=0x7f991c00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:09.848 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.846+0000 7f9920c00640 1 -- 172.21.15.151:0/1067442355 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991c003b70 con 0x7f992410e620 2024-04-26T02:03:09.848 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.846+0000 7f992a200640 1 -- 172.21.15.151:0/1067442355 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f992419f4a0 con 0x7f992410e620 2024-04-26T02:03:09.848 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.846+0000 7f9920c00640 1 -- 172.21.15.151:0/1067442355 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f991c003d10 con 0x7f992410e620 2024-04-26T02:03:09.848 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.846+0000 7f9920c00640 1 -- 172.21.15.151:0/1067442355 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991c010830 con 0x7f992410e620 2024-04-26T02:03:09.848 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.847+0000 7f992a200640 1 -- 172.21.15.151:0/1067442355 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99241ab740 con 0x7f992410e620 2024-04-26T02:03:09.851 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.850+0000 7f992d190740 1 Processor -- start 2024-04-26T02:03:09.851 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.850+0000 7f9920c00640 1 -- 172.21.15.151:0/1067442355 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f991c003ec0 con 0x7f992410e620 2024-04-26T02:03:09.851 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.850+0000 7f992d190740 1 -- start start 2024-04-26T02:03:09.851 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.850+0000 7f992d190740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d299095820 0x55d299095c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:09.851 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.850+0000 7f992d190740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d299096140 0x55d2990a04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:09.851 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.850+0000 7f992d190740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2990a0a00 0x55d2990a2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:09.852 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.850+0000 7f992d190740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d298ffb900 con 0x55d2990a0a00 2024-04-26T02:03:09.852 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.850+0000 7f992d190740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d298fa1ef0 con 0x55d299095820 2024-04-26T02:03:09.852 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.850+0000 7f992d190740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d298ff78c0 con 0x55d299096140 2024-04-26T02:03:09.852 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.851+0000 7f991be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d299095820 0x55d299095c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:09.852 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.851+0000 7f991be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d299095820 0x55d299095c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:45648/0 (socket says 172.21.15.151:45648) 2024-04-26T02:03:09.852 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.851+0000 7f991be00640 1 -- 172.21.15.151:0/1658956218 learned_addr learned my addr 172.21.15.151:0/1658956218 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:03:09.852 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.851+0000 7f9923400640 1 --2- 172.21.15.151:0/1658956218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d299096140 0x55d2990a04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:09.852 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.851+0000 7f991be00640 1 --2- 172.21.15.151:0/1658956218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2990a0a00 0x55d2990a2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:09.852 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.851+0000 7f9923400640 1 -- 172.21.15.151:0/1658956218 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d299095820 msgr2=0x55d299095c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:09.853 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.851+0000 7f9923400640 1 --2- 172.21.15.151:0/1658956218 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d299095820 0x55d299095c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:09.853 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.851+0000 7f9923400640 1 -- 172.21.15.151:0/1658956218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2990a0a00 msgr2=0x55d2990a2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:09.853 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.851+0000 7f9923400640 1 --2- 172.21.15.151:0/1658956218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2990a0a00 0x55d2990a2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:09.853 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.851+0000 7f9923400640 1 -- 172.21.15.151:0/1658956218 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d298d0e210 con 0x55d299096140 2024-04-26T02:03:09.853 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.851+0000 7f991be00640 1 --2- 172.21.15.151:0/1658956218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2990a0a00 0x55d2990a2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:03:09.853 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.852+0000 7f9923400640 1 --2- 172.21.15.151:0/1658956218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d299096140 0x55d2990a04c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9914005940 tx=0x7f9914005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d82fdd0cfc11a67 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:09.853 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.852+0000 7f9920c00640 1 --2- 172.21.15.151:0/1067442355 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f99040a3dc0 0x7f99040a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:09.853 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.852+0000 7f9920c00640 1 -- 172.21.15.151:0/1067442355 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8499+0+0 (secure 0 0 0) 0x7f991c0d0db0 con 0x7f992410e620 2024-04-26T02:03:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.852+0000 7f9913e00640 1 -- 172.21.15.151:0/1658956218 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9914040070 con 0x55d299096140 2024-04-26T02:03:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.852+0000 7f991be00640 1 --2- 172.21.15.151:0/1067442355 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f99040a3dc0 0x7f99040a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.852+0000 7f9913e00640 1 -- 172.21.15.151:0/1658956218 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9914005020 con 0x55d299096140 2024-04-26T02:03:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.852+0000 7f9913e00640 1 -- 172.21.15.151:0/1658956218 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9914035040 con 0x55d299096140 2024-04-26T02:03:09.859 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.853+0000 7f992d190740 1 -- 172.21.15.151:0/1658956218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d299096140 msgr2=0x55d2990a04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:09.859 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.853+0000 7f991be00640 1 --2- 172.21.15.151:0/1067442355 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f99040a3dc0 0x7f99040a6280 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x55d299093150 tx=0x7f990c013040 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:09.859 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.853+0000 7f992d190740 1 --2- 172.21.15.151:0/1658956218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d299096140 0x55d2990a04c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9914005940 tx=0x7f9914005bd0 comp rx=0 tx=0).stop 2024-04-26T02:03:09.860 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.853+0000 7f992d190740 1 -- 172.21.15.151:0/1658956218 shutdown_connections 2024-04-26T02:03:09.860 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.853+0000 7f992d190740 1 --2- 172.21.15.151:0/1658956218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2990a0a00 0x55d2990a2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:09.860 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.853+0000 7f992d190740 1 --2- 172.21.15.151:0/1658956218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d299096140 0x55d2990a04c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:09.860 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.853+0000 7f992d190740 1 --2- 172.21.15.151:0/1658956218 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d299095820 0x55d299095c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:09.860 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.853+0000 7f992d190740 1 -- 172.21.15.151:0/1658956218 >> 172.21.15.151:0/1658956218 conn(0x55d299091d30 msgr2=0x55d299092110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:09.860 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.853+0000 7f992d190740 1 -- 172.21.15.151:0/1658956218 shutdown_connections 2024-04-26T02:03:09.860 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.853+0000 7f992d190740 1 -- 172.21.15.151:0/1658956218 wait complete. 2024-04-26T02:03:09.860 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.854+0000 7f992d190740 1 -- start start 2024-04-26T02:03:09.860 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.854+0000 7f992d190740 1 Processor -- start 2024-04-26T02:03:09.860 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.854+0000 7f992d190740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d299095820 0x55d29909d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:09.860 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.854+0000 7f992d190740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d299096140 0x55d29909dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:09.860 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.854+0000 7f992d190740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2990a0a00 0x55d29909e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:09.861 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.854+0000 7f992d190740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d2990a35d0 con 0x55d299095820 2024-04-26T02:03:09.861 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.854+0000 7f992d190740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d298fb5310 con 0x55d2990a0a00 2024-04-26T02:03:09.861 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.854+0000 7f992d190740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d2990a3750 con 0x55d299096140 2024-04-26T02:03:09.861 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.854+0000 7f992d190740 10 client.0 _ll_register_callbacks cb 0x55d298f83cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:03:09.861 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.855+0000 7f9923e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2990a0a00 0x55d29909e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:09.861 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.855+0000 7f9923e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2990a0a00 0x55d29909e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:45654/0 (socket says 172.21.15.151:45654) 2024-04-26T02:03:09.861 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.855+0000 7f9923e00640 1 -- 172.21.15.151:0/114440610 learned_addr learned my addr 172.21.15.151:0/114440610 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:03:09.861 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.855+0000 7f9923400640 1 --2- 172.21.15.151:0/114440610 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d299095820 0x55d29909d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:09.862 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.855+0000 7f991be00640 1 --2- 172.21.15.151:0/114440610 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d299096140 0x55d29909dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:09.862 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.855+0000 7f9923e00640 1 -- 172.21.15.151:0/114440610 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d299096140 msgr2=0x55d29909dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:09.862 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.855+0000 7f9923e00640 1 --2- 172.21.15.151:0/114440610 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d299096140 0x55d29909dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:09.862 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.855+0000 7f9923e00640 1 -- 172.21.15.151:0/114440610 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d299095820 msgr2=0x55d29909d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:09.862 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.855+0000 7f9923e00640 1 --2- 172.21.15.151:0/114440610 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d299095820 0x55d29909d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:09.862 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.855+0000 7f9923e00640 1 -- 172.21.15.151:0/114440610 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d298dda620 con 0x55d2990a0a00 2024-04-26T02:03:09.862 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.855+0000 7f9923e00640 1 --2- 172.21.15.151:0/114440610 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2990a0a00 0x55d29909e3e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f991c0996d0 tx=0x7f991c009820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35f0d68f556a2e9c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:09.862 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.855+0000 7f9910c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T02:03:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.855+0000 7f9910c00640 1 -- 172.21.15.151:0/114440610 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991c0b6260 con 0x55d2990a0a00 2024-04-26T02:03:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.856+0000 7f9910c00640 1 -- 172.21.15.151:0/114440610 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8499+0+0 (secure 0 0 0) 0x7f991c0bd0c0 con 0x55d2990a0a00 2024-04-26T02:03:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.856+0000 7f992d190740 1 -- 172.21.15.151:0/114440610 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9914005020 con 0x55d2990a0a00 2024-04-26T02:03:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.856+0000 7f9910c00640 1 -- 172.21.15.151:0/114440610 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f991c09d400 con 0x55d2990a0a00 2024-04-26T02:03:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.856+0000 7f992d190740 10 client.26270 fetch_fsmap learned FSMap version 35 2024-04-26T02:03:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.856+0000 7f992d190740 1 -- 172.21.15.151:0/114440610 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d2990af7f0 con 0x55d2990a0a00 2024-04-26T02:03:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.856+0000 7f9910c00640 1 -- 172.21.15.151:0/114440610 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9914005020 con 0x55d2990a0a00 2024-04-26T02:03:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.856+0000 7f9910c00640 1 -- 172.21.15.151:0/114440610 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 35) ==== 8338+0+0 (secure 0 0 0) 0x7f991c0041b0 con 0x55d2990a0a00 2024-04-26T02:03:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.856+0000 7f992d190740 10 client.26270 fetch_fsmap finished waiting for FSMap version 35 2024-04-26T02:03:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.856+0000 7f992d190740 10 client.26270 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:03:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.856+0000 7f992d190740 20 client.26270 populate_metadata read hostname 'smithi151' 2024-04-26T02:03:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.856+0000 7f992d190740 1 --2- 172.21.15.151:0/114440610 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55d2990b0c60 0x55d2990b3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.856+0000 7f992d190740 4 client.26270 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:03:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.856+0000 7f992d190740 1 -- 172.21.15.151:0/114440610 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d29900f8b0 con 0x55d2990b0c60 2024-04-26T02:03:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.857+0000 7f9923400640 1 --2- 172.21.15.151:0/114440610 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55d2990b0c60 0x55d2990b3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.857+0000 7f9923400640 1 --2- 172.21.15.151:0/114440610 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55d2990b0c60 0x55d2990b3070 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8aecc86953020365 server_cookie=f60f2ff8e5f41782 in_seq=0 out_seq=0 2024-04-26T02:03:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.857+0000 7f9910c00640 10 client.26270 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:03:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.858+0000 7f9910c00640 1 -- 172.21.15.151:0/114440610 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d29900f8b0 con 0x55d2990b0c60 2024-04-26T02:03:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.858+0000 7f9910c00640 10 client.26270 handle_command_reply: tid=0 2024-04-26T02:03:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.858+0000 7f992d190740 1 client.26270 shutdown 2024-04-26T02:03:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.858+0000 7f992d190740 1 -- 172.21.15.151:0/114440610 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2990a0a00 msgr2=0x55d29909e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.858+0000 7f992d190740 1 --2- 172.21.15.151:0/114440610 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2990a0a00 0x55d29909e3e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f991c0996d0 tx=0x7f991c009820 comp rx=0 tx=0).stop 2024-04-26T02:03:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.858+0000 7f992d190740 1 -- 172.21.15.151:0/114440610 shutdown_connections 2024-04-26T02:03:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.858+0000 7f992d190740 1 --2- 172.21.15.151:0/114440610 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55d2990b0c60 0x55d2990b3070 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.858+0000 7f992d190740 1 --2- 172.21.15.151:0/114440610 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2990a0a00 0x55d29909e3e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.858+0000 7f992d190740 1 --2- 172.21.15.151:0/114440610 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d299096140 0x55d29909dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.858+0000 7f992d190740 1 --2- 172.21.15.151:0/114440610 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d299095820 0x55d29909d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.858+0000 7f9910c00640 0 client.26270 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:03:09.866 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.858+0000 7f992d190740 1 -- 172.21.15.151:0/114440610 >> 172.21.15.151:0/114440610 conn(0x55d299091d30 msgr2=0x55d299097a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:09.866 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.859+0000 7f992d190740 1 -- 172.21.15.151:0/114440610 shutdown_connections 2024-04-26T02:03:09.866 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.859+0000 7f992d190740 1 -- 172.21.15.151:0/114440610 wait complete. 2024-04-26T02:03:09.866 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.859+0000 7f992d190740 20 client.26270 trim_cache size 0 max 16384 2024-04-26T02:03:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.875+0000 7f992d190740 1 Processor -- start 2024-04-26T02:03:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.875+0000 7f992d190740 1 -- start start 2024-04-26T02:03:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.875+0000 7f992d190740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d299095820 0x55d2990ac490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.875+0000 7f992d190740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d299096140 0x55d2990981e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.875+0000 7f992d190740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2990a0a00 0x55d2990b8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.875+0000 7f992d190740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f99241b68c0 con 0x55d299095820 2024-04-26T02:03:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.875+0000 7f992d190740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f99241b4190 con 0x55d2990a0a00 2024-04-26T02:03:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.875+0000 7f992d190740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f99241b2000 con 0x55d299096140 2024-04-26T02:03:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.875+0000 7f9923e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2990a0a00 0x55d2990b8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.875+0000 7f991be00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d299096140 0x55d2990981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.875+0000 7f991be00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d299096140 0x55d2990981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:36462/0 (socket says 172.21.15.151:36462) 2024-04-26T02:03:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.875+0000 7f991be00640 1 -- 172.21.15.151:0/3130202678 learned_addr learned my addr 172.21.15.151:0/3130202678 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:03:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.875+0000 7f9923e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2990a0a00 0x55d2990b8980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:45668/0 (socket says 172.21.15.151:45668) 2024-04-26T02:03:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.875+0000 7f9923400640 1 --2- 172.21.15.151:0/3130202678 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d299095820 0x55d2990ac490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.876+0000 7f9923e00640 1 -- 172.21.15.151:0/3130202678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d299096140 msgr2=0x55d2990981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:09.877 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.876+0000 7f9923e00640 1 --2- 172.21.15.151:0/3130202678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d299096140 0x55d2990981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:09.877 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.876+0000 7f9923e00640 1 -- 172.21.15.151:0/3130202678 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d299095820 msgr2=0x55d2990ac490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:09.877 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.876+0000 7f9923e00640 1 --2- 172.21.15.151:0/3130202678 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d299095820 0x55d2990ac490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:09.877 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.876+0000 7f9923e00640 1 -- 172.21.15.151:0/3130202678 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2990b36f0 con 0x55d2990a0a00 2024-04-26T02:03:09.877 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.876+0000 7f9923e00640 1 --2- 172.21.15.151:0/3130202678 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2990a0a00 0x55d2990b8980 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f991c09ca00 tx=0x7f991c004990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c11404b70e5a3f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:09.881 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.876+0000 7f9913400640 1 -- 172.21.15.151:0/3130202678 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991c0b6270 con 0x55d2990a0a00 2024-04-26T02:03:09.881 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.876+0000 7f9913400640 1 -- 172.21.15.151:0/3130202678 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f991c0a5970 con 0x55d2990a0a00 2024-04-26T02:03:09.881 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.876+0000 7f9913400640 1 -- 172.21.15.151:0/3130202678 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991c0bd110 con 0x55d2990a0a00 2024-04-26T02:03:09.881 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.877+0000 7f992d190740 1 -- 172.21.15.151:0/3130202678 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2990a0a00 msgr2=0x55d2990b8980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:09.881 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.877+0000 7f992d190740 1 --2- 172.21.15.151:0/3130202678 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2990a0a00 0x55d2990b8980 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f991c09ca00 tx=0x7f991c004990 comp rx=0 tx=0).stop 2024-04-26T02:03:09.881 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.877+0000 7f992d190740 1 -- 172.21.15.151:0/3130202678 shutdown_connections 2024-04-26T02:03:09.881 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.877+0000 7f992d190740 1 --2- 172.21.15.151:0/3130202678 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2990a0a00 0x55d2990b8980 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:09.882 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.877+0000 7f992d190740 1 --2- 172.21.15.151:0/3130202678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d299096140 0x55d2990981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:09.882 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.877+0000 7f992d190740 1 --2- 172.21.15.151:0/3130202678 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d299095820 0x55d2990ac490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:09.882 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.877+0000 7f992d190740 1 -- 172.21.15.151:0/3130202678 >> 172.21.15.151:0/3130202678 conn(0x55d299091d30 msgr2=0x55d29909e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:09.882 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.877+0000 7f992d190740 1 -- 172.21.15.151:0/3130202678 shutdown_connections 2024-04-26T02:03:09.882 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.877+0000 7f992d190740 1 -- 172.21.15.151:0/3130202678 wait complete. 2024-04-26T02:03:09.882 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.878+0000 7f992d190740 1 -- start start 2024-04-26T02:03:09.882 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.878+0000 7f992d190740 1 Processor -- start 2024-04-26T02:03:09.882 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.879+0000 7f992d190740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d299095820 0x55d2990a8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:09.882 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.879+0000 7f992d190740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d299096140 0x55d2990a6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:09.882 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.879+0000 7f992d190740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2990a0a00 0x55d2990a6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:09.882 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.879+0000 7f992d190740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f99241b2650 con 0x55d299095820 2024-04-26T02:03:09.883 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.879+0000 7f992d190740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f99241b3ac0 con 0x55d299096140 2024-04-26T02:03:09.883 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.879+0000 7f992d190740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f99241b2d20 con 0x55d2990a0a00 2024-04-26T02:03:09.883 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.879+0000 7f992d190740 10 client.0 _ll_register_callbacks cb 0x7f991c09c840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:03:09.883 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.879+0000 7f991be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d299096140 0x55d2990a6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:09.883 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.879+0000 7f9923e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2990a0a00 0x55d2990a6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:09.883 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.879+0000 7f991be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d299096140 0x55d2990a6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:45676/0 (socket says 172.21.15.151:45676) 2024-04-26T02:03:09.883 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.879+0000 7f991be00640 1 -- 172.21.15.151:0/2892543342 learned_addr learned my addr 172.21.15.151:0/2892543342 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:03:09.883 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.879+0000 7f9923400640 1 --2- 172.21.15.151:0/2892543342 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d299095820 0x55d2990a8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:09.884 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.880+0000 7f991be00640 1 -- 172.21.15.151:0/2892543342 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2990a0a00 msgr2=0x55d2990a6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:09.884 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.880+0000 7f991be00640 1 --2- 172.21.15.151:0/2892543342 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2990a0a00 0x55d2990a6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:09.884 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.880+0000 7f991be00640 1 -- 172.21.15.151:0/2892543342 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d299095820 msgr2=0x55d2990a8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:09.884 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.880+0000 7f991be00640 1 --2- 172.21.15.151:0/2892543342 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d299095820 0x55d2990a8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:09.884 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.880+0000 7f991be00640 1 -- 172.21.15.151:0/2892543342 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2990a6cd0 con 0x55d299096140 2024-04-26T02:03:09.884 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.880+0000 7f991be00640 1 --2- 172.21.15.151:0/2892543342 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d299096140 0x55d2990a6050 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f990c005eb0 tx=0x7f990c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afb199f52b21a13a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:09.884 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.880+0000 7f9910c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T02:03:09.884 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.881+0000 7f9910c00640 1 -- 172.21.15.151:0/2892543342 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990c009e20 con 0x55d299096140 2024-04-26T02:03:09.884 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.881+0000 7f9910c00640 1 -- 172.21.15.151:0/2892543342 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8499+0+0 (secure 0 0 0) 0x7f990c01a070 con 0x55d299096140 2024-04-26T02:03:09.884 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.881+0000 7f992d190740 1 -- 172.21.15.151:0/2892543342 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f991c0a5970 con 0x55d299096140 2024-04-26T02:03:09.884 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.881+0000 7f9910c00640 1 -- 172.21.15.151:0/2892543342 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f990c004020 con 0x55d299096140 2024-04-26T02:03:09.884 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.881+0000 7f992d190740 10 client.26276 fetch_fsmap learned FSMap version 35 2024-04-26T02:03:09.885 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.881+0000 7f992d190740 1 -- 172.21.15.151:0/2892543342 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d299117170 con 0x55d299096140 2024-04-26T02:03:09.885 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.881+0000 7f9910c00640 1 -- 172.21.15.151:0/2892543342 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f991c0a5970 con 0x55d299096140 2024-04-26T02:03:09.885 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.882+0000 7f9910c00640 1 -- 172.21.15.151:0/2892543342 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 35) ==== 8338+0+0 (secure 0 0 0) 0x7f990c004830 con 0x55d299096140 2024-04-26T02:03:09.885 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.882+0000 7f992d190740 10 client.26276 fetch_fsmap finished waiting for FSMap version 35 2024-04-26T02:03:09.885 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.882+0000 7f992d190740 10 client.26276 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:03:09.885 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.882+0000 7f992d190740 20 client.26276 populate_metadata read hostname 'smithi151' 2024-04-26T02:03:09.885 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.882+0000 7f992d190740 1 --2- 172.21.15.151:0/2892543342 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55d2991179a0 0x55d299119dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:09.885 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.882+0000 7f992d190740 4 client.26276 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T02:03:09.885 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.882+0000 7f992d190740 1 -- 172.21.15.151:0/2892543342 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d299009320 con 0x55d2991179a0 2024-04-26T02:03:09.886 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.882+0000 7f9923400640 1 --2- 172.21.15.151:0/2892543342 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55d2991179a0 0x55d299119dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:09.886 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.883+0000 7f9910c00640 10 client.26276 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:03:09.886 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.883+0000 7f9923400640 1 --2- 172.21.15.151:0/2892543342 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55d2991179a0 0x55d299119dc0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a9f3628aabcffd0 server_cookie=9bd370cc3dd9abff in_seq=0 out_seq=0 2024-04-26T02:03:09.895 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.895+0000 7f9910c00640 1 -- 172.21.15.151:0/2892543342 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d299009320 con 0x55d2991179a0 2024-04-26T02:03:09.895 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.895+0000 7f9910c00640 10 client.26276 handle_command_reply: tid=0 2024-04-26T02:03:09.895 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.895+0000 7f992d190740 1 client.26276 shutdown 2024-04-26T02:03:09.895 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.896+0000 7f992d190740 1 -- 172.21.15.151:0/2892543342 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d299096140 msgr2=0x55d2990a6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:09.895 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.896+0000 7f992d190740 1 --2- 172.21.15.151:0/2892543342 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d299096140 0x55d2990a6050 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f990c005eb0 tx=0x7f990c005f90 comp rx=0 tx=0).stop 2024-04-26T02:03:09.895 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.896+0000 7f992d190740 1 -- 172.21.15.151:0/2892543342 shutdown_connections 2024-04-26T02:03:09.895 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.896+0000 7f992d190740 1 --2- 172.21.15.151:0/2892543342 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55d2991179a0 0x55d299119dc0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:09.895 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.896+0000 7f992d190740 1 --2- 172.21.15.151:0/2892543342 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2990a0a00 0x55d2990a6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.896+0000 7f992d190740 1 --2- 172.21.15.151:0/2892543342 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d299096140 0x55d2990a6050 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.896+0000 7f992d190740 1 --2- 172.21.15.151:0/2892543342 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d299095820 0x55d2990a8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.896+0000 7f9910c00640 0 client.26276 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:03:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.896+0000 7f992d190740 1 -- 172.21.15.151:0/2892543342 >> 172.21.15.151:0/2892543342 conn(0x55d299091d30 msgr2=0x55d299090600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.896+0000 7f992d190740 1 -- 172.21.15.151:0/2892543342 shutdown_connections 2024-04-26T02:03:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.896+0000 7f992d190740 1 -- 172.21.15.151:0/2892543342 wait complete. 2024-04-26T02:03:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.896+0000 7f992d190740 20 client.26276 trim_cache size 0 max 16384 2024-04-26T02:03:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:{ 2024-04-26T02:03:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "message": "", 2024-04-26T02:03:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "return_code": 0 2024-04-26T02:03:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:} 2024-04-26T02:03:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.897+0000 7f992a200640 1 -- 172.21.15.151:0/1067442355 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f99040a3dc0 msgr2=0x7f99040a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.897+0000 7f992a200640 1 --2- 172.21.15.151:0/1067442355 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f99040a3dc0 0x7f99040a6280 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x55d299093150 tx=0x7f990c013040 comp rx=0 tx=0).stop 2024-04-26T02:03:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.897+0000 7f992a200640 1 -- 172.21.15.151:0/1067442355 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f992410e620 msgr2=0x7f992419ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.897+0000 7f992a200640 1 --2- 172.21.15.151:0/1067442355 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f992410e620 0x7f992419ea30 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f991c00ec50 tx=0x7f991c00c6a0 comp rx=0 tx=0).stop 2024-04-26T02:03:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.897+0000 7f992a200640 1 -- 172.21.15.151:0/1067442355 shutdown_connections 2024-04-26T02:03:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.897+0000 7f992a200640 1 --2- 172.21.15.151:0/1067442355 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f99040a3dc0 0x7f99040a6280 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.897+0000 7f992a200640 1 --2- 172.21.15.151:0/1067442355 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f992410e620 0x7f992419ea30 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.897+0000 7f992a200640 1 --2- 172.21.15.151:0/1067442355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9924108f20 0x7f99241a4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.897+0000 7f992a200640 1 --2- 172.21.15.151:0/1067442355 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9924108560 0x7f99241a4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.897+0000 7f992a200640 1 -- 172.21.15.151:0/1067442355 >> 172.21.15.151:0/1067442355 conn(0x7f99240764e0 msgr2=0x7f9924077bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.897+0000 7f992a200640 1 -- 172.21.15.151:0/1067442355 shutdown_connections 2024-04-26T02:03:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:09.897+0000 7f992a200640 1 -- 172.21.15.151:0/1067442355 wait complete. 2024-04-26T02:03:10.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:03:10 smithi165 ceph-mon[32648]: pgmap v298: 97 pgs: 97 active+clean; 112 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 90 KiB/s wr, 30 op/s 2024-04-26T02:03:10.347 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:03:10 smithi165 ceph-mon[32648]: osdmap e103: 12 total, 12 up, 12 in 2024-04-26T02:03:10.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:03:10 smithi151 ceph-mon[28057]: pgmap v298: 97 pgs: 97 active+clean; 112 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 90 KiB/s wr, 30 op/s 2024-04-26T02:03:10.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:03:10 smithi151 ceph-mon[28057]: osdmap e103: 12 total, 12 up, 12 in 2024-04-26T02:03:10.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:03:10 smithi172 ceph-mon[32785]: pgmap v298: 97 pgs: 97 active+clean; 112 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 90 KiB/s wr, 30 op/s 2024-04-26T02:03:10.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:03:10 smithi172 ceph-mon[32785]: osdmap e103: 12 total, 12 up, 12 in 2024-04-26T02:03:12.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:03:12 smithi165 ceph-mon[32648]: pgmap v300: 97 pgs: 97 active+clean; 112 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 95 KiB/s wr, 28 op/s 2024-04-26T02:03:12.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:03:12 smithi151 ceph-mon[28057]: pgmap v300: 97 pgs: 97 active+clean; 112 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 95 KiB/s wr, 28 op/s 2024-04-26T02:03:12.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:03:12 smithi172 ceph-mon[32785]: pgmap v300: 97 pgs: 97 active+clean; 112 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 95 KiB/s wr, 28 op/s 2024-04-26T02:03:14.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:03:14 smithi165 ceph-mon[32648]: pgmap v301: 97 pgs: 97 active+clean; 112 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 75 KiB/s wr, 26 op/s 2024-04-26T02:03:14.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:03:14 smithi151 ceph-mon[28057]: pgmap v301: 97 pgs: 97 active+clean; 112 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 75 KiB/s wr, 26 op/s 2024-04-26T02:03:14.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:03:14 smithi172 ceph-mon[32785]: pgmap v301: 97 pgs: 97 active+clean; 112 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 75 KiB/s wr, 26 op/s 2024-04-26T02:03:16.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:03:16 smithi165 ceph-mon[32648]: pgmap v302: 97 pgs: 97 active+clean; 112 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 50 KiB/s wr, 10 op/s 2024-04-26T02:03:16.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:03:16 smithi151 ceph-mon[28057]: pgmap v302: 97 pgs: 97 active+clean; 112 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 50 KiB/s wr, 10 op/s 2024-04-26T02:03:16.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:03:16 smithi172 ceph-mon[32785]: pgmap v302: 97 pgs: 97 active+clean; 112 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 50 KiB/s wr, 10 op/s 2024-04-26T02:03:16.668 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T02:03:17.125 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:03:18.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:03:18 smithi151 ceph-mon[28057]: pgmap v303: 97 pgs: 97 active+clean; 112 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 50 KiB/s wr, 10 op/s 2024-04-26T02:03:18.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:03:18 smithi172 ceph-mon[32785]: pgmap v303: 97 pgs: 97 active+clean; 112 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 50 KiB/s wr, 10 op/s 2024-04-26T02:03:18.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:03:18 smithi165 ceph-mon[32648]: pgmap v303: 97 pgs: 97 active+clean; 112 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 50 KiB/s wr, 10 op/s 2024-04-26T02:03:19.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:19.530+0000 7f52cce00700 1 -- 172.21.15.151:0/3241356039 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f52c810f440 msgr2=0x7f52c8111830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:19.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:19.530+0000 7f52cce00700 1 --2- 172.21.15.151:0/3241356039 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f52c810f440 0x7f52c8111830 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f52c000b0a0 tx=0x7f52c000b3b0 comp rx=0 tx=0).stop 2024-04-26T02:03:19.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:19.530+0000 7f52cce00700 1 -- 172.21.15.151:0/3241356039 shutdown_connections 2024-04-26T02:03:19.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:19.530+0000 7f52cce00700 1 --2- 172.21.15.151:0/3241356039 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f52c810f440 0x7f52c8111830 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:19.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:19.530+0000 7f52cce00700 1 --2- 172.21.15.151:0/3241356039 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f52c8074c70 0x7f52c8075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:19.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:19.530+0000 7f52cce00700 1 --2- 172.21.15.151:0/3241356039 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52c8075590 0x7f52c810ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:19.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:19.530+0000 7f52cce00700 1 -- 172.21.15.151:0/3241356039 >> 172.21.15.151:0/3241356039 conn(0x7f52c8079bf0 msgr2=0x7f52c807a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:19.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:19.530+0000 7f52cce00700 1 -- 172.21.15.151:0/3241356039 shutdown_connections 2024-04-26T02:03:19.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:19.530+0000 7f52cce00700 1 -- 172.21.15.151:0/3241356039 wait complete. 2024-04-26T02:03:19.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:19.531+0000 7f52cce00700 1 Processor -- start 2024-04-26T02:03:19.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:19.531+0000 7f52cce00700 1 -- start start 2024-04-26T02:03:19.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:19.532+0000 7f52cce00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f52c8074c70 0x7f52c819b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:19.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:19.532+0000 7f52cce00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52c8075590 0x7f52c819b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:19.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:19.532+0000 7f52cce00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f52c810f440 0x7f52c819fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:19.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:19.532+0000 7f52cce00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f52c8114e30 con 0x7f52c8074c70 2024-04-26T02:03:19.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:19.532+0000 7f52cce00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f52c8114cb0 con 0x7f52c810f440 2024-04-26T02:03:19.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:19.532+0000 7f52cce00700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f52c8114fb0 con 0x7f52c8075590 2024-04-26T02:03:19.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:19.532+0000 7f52c5e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f52c8074c70 0x7f52c819b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:19.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:19.532+0000 7f52c5400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52c8075590 0x7f52c819b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:19.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:19.532+0000 7f52c5e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f52c8074c70 0x7f52c819b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:35404/0 (socket says 172.21.15.151:35404) 2024-04-26T02:03:19.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:19.532+0000 7f52c5400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52c8075590 0x7f52c819b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:36494/0 (socket says 172.21.15.151:36494) 2024-04-26T02:03:19.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:19.532+0000 7f52c5e00700 1 -- 172.21.15.151:0/4097773390 learned_addr learned my addr 172.21.15.151:0/4097773390 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:03:19.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:19.532+0000 7f52c6800700 1 --2- 172.21.15.151:0/4097773390 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f52c810f440 0x7f52c819fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:19.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:19.533+0000 7f52c6800700 1 -- 172.21.15.151:0/4097773390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52c8075590 msgr2=0x7f52c819b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:19.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:19.533+0000 7f52c6800700 1 --2- 172.21.15.151:0/4097773390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52c8075590 0x7f52c819b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:19.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:19.533+0000 7f52c6800700 1 -- 172.21.15.151:0/4097773390 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f52c8074c70 msgr2=0x7f52c819b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:19.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:19.533+0000 7f52c6800700 1 --2- 172.21.15.151:0/4097773390 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f52c8074c70 0x7f52c819b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:19.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:19.533+0000 7f52c6800700 1 -- 172.21.15.151:0/4097773390 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52c81a0380 con 0x7f52c810f440 2024-04-26T02:03:19.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:19.533+0000 7f52c5400700 1 --2- 172.21.15.151:0/4097773390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52c8075590 0x7f52c819b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:03:19.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:19.533+0000 7f52c6800700 1 --2- 172.21.15.151:0/4097773390 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f52c810f440 0x7f52c819fc40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f52c000b070 tx=0x7f52c0004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:19.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:19.533+0000 7f52bea00700 1 -- 172.21.15.151:0/4097773390 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c0021070 con 0x7f52c810f440 2024-04-26T02:03:19.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:19.533+0000 7f52bea00700 1 -- 172.21.15.151:0/4097773390 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52c0007e30 con 0x7f52c810f440 2024-04-26T02:03:19.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:19.533+0000 7f52cce00700 1 -- 172.21.15.151:0/4097773390 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52c81a0610 con 0x7f52c810f440 2024-04-26T02:03:19.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:19.533+0000 7f52bea00700 1 -- 172.21.15.151:0/4097773390 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c00045d0 con 0x7f52c810f440 2024-04-26T02:03:19.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:19.534+0000 7f52cce00700 1 -- 172.21.15.151:0/4097773390 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52c81a7ef0 con 0x7f52c810f440 2024-04-26T02:03:19.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:19.535+0000 7f52cce00700 1 -- 172.21.15.151:0/4097773390 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52c8076390 con 0x7f52c810f440 2024-04-26T02:03:19.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:19.537+0000 7f52bea00700 1 -- 172.21.15.151:0/4097773390 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f52c0003c00 con 0x7f52c810f440 2024-04-26T02:03:19.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:19.538+0000 7f52bea00700 1 --2- 172.21.15.151:0/4097773390 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f52ac0a3db0 0x7f52ac0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:19.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:19.538+0000 7f52bea00700 1 -- 172.21.15.151:0/4097773390 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8455+0+0 (secure 0 0 0) 0x7f52c00d1590 con 0x7f52c810f440 2024-04-26T02:03:19.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:19.538+0000 7f52c5e00700 1 --2- 172.21.15.151:0/4097773390 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f52ac0a3db0 0x7f52ac0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:19.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:19.539+0000 7f52c5e00700 1 --2- 172.21.15.151:0/4097773390 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f52ac0a3db0 0x7f52ac0a6270 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f52b8009d20 tx=0x7f52b8009260 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:19.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:19.542+0000 7f52bea00700 1 -- 172.21.15.151:0/4097773390 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f52c009dca0 con 0x7f52c810f440 2024-04-26T02:03:19.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:19.849+0000 7f52cce00700 1 -- 172.21.15.151:0/4097773390 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f52c819c6d0 con 0x7f52c810f440 2024-04-26T02:03:19.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:19.850+0000 7f52bea00700 1 -- 172.21.15.151:0/4097773390 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11889 (secure 0 0 0) 0x7f52c009d5a0 con 0x7f52c810f440 2024-04-26T02:03:19.849 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T02:03:19.849 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2937864097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2937864097},{"type":"v1","addr":"172.21.15.151:6835","nonce":2937864097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14373,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/2947090802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2947090802},{"type":"v1","addr":"172.21.15.151:6837","nonce":2947090802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/1356563280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1356563280},{"type":"v1","addr":"172.21.15.151:6839","nonce":1356563280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6841/1075240552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1075240552},{"type":"v1","addr":"172.21.15.151:6841","nonce":1075240552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/504675912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":504675912},{"type":"v1","addr":"172.21.15.165:6833","nonce":504675912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1207580015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1207580015},{"type":"v1","addr":"172.21.15.172:6833","nonce":1207580015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:49:40.746876+0000","modified":"2024-04-26T02:00:05.413774+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24505,"mds_2":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/3598138294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3598138294},{"type":"v1","addr":"172.21.15.165:6835","nonce":3598138294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/1847859054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":1847859054},{"type":"v1","addr":"172.21.15.165:6837","nonce":1847859054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.172:6837/55972537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":55972537},{"type":"v1","addr":"172.21.15.172:6837","nonce":55972537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.172:6839/1242302810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1242302810},{"type":"v1","addr":"172.21.15.172:6839","nonce":1242302810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/1969716254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1969716254},{"type":"v1","addr":"172.21.15.165:6839","nonce":1969716254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25954":{"gid":25954,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/3656630171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3656630171},{"type":"v1","addr":"172.21.15.172:6835","nonce":3656630171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24493,24505,24509]},"id":1}]} 2024-04-26T02:03:19.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:19.854+0000 7f52cce00700 1 -- 172.21.15.151:0/4097773390 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f52ac0a3db0 msgr2=0x7f52ac0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:19.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:19.854+0000 7f52cce00700 1 --2- 172.21.15.151:0/4097773390 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f52ac0a3db0 0x7f52ac0a6270 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f52b8009d20 tx=0x7f52b8009260 comp rx=0 tx=0).stop 2024-04-26T02:03:19.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:19.854+0000 7f52cce00700 1 -- 172.21.15.151:0/4097773390 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f52c810f440 msgr2=0x7f52c819fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:19.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:19.854+0000 7f52cce00700 1 --2- 172.21.15.151:0/4097773390 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f52c810f440 0x7f52c819fc40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f52c000b070 tx=0x7f52c0004d00 comp rx=0 tx=0).stop 2024-04-26T02:03:19.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:19.854+0000 7f52cce00700 1 -- 172.21.15.151:0/4097773390 shutdown_connections 2024-04-26T02:03:19.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:19.854+0000 7f52cce00700 1 --2- 172.21.15.151:0/4097773390 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f52ac0a3db0 0x7f52ac0a6270 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:19.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:19.854+0000 7f52cce00700 1 --2- 172.21.15.151:0/4097773390 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f52c810f440 0x7f52c819fc40 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:19.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:19.854+0000 7f52cce00700 1 --2- 172.21.15.151:0/4097773390 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f52c8074c70 0x7f52c819b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:19.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:19.854+0000 7f52cce00700 1 --2- 172.21.15.151:0/4097773390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52c8075590 0x7f52c819b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:19.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:19.854+0000 7f52cce00700 1 -- 172.21.15.151:0/4097773390 >> 172.21.15.151:0/4097773390 conn(0x7f52c8079bf0 msgr2=0x7f52c80ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:19.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:19.854+0000 7f52cce00700 1 -- 172.21.15.151:0/4097773390 shutdown_connections 2024-04-26T02:03:19.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:19.854+0000 7f52cce00700 1 -- 172.21.15.151:0/4097773390 wait complete. 2024-04-26T02:03:19.854 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 35 2024-04-26T02:03:20.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:03:20 smithi172 ceph-mon[32785]: pgmap v304: 97 pgs: 97 active+clean; 112 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 8.8 KiB/s wr, 8 op/s 2024-04-26T02:03:20.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:03:20 smithi172 ceph-mon[32785]: osdmap e104: 12 total, 12 up, 12 in 2024-04-26T02:03:20.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:03:20 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/4097773390' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:03:20.593 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:03:20 smithi151 ceph-mon[28057]: pgmap v304: 97 pgs: 97 active+clean; 112 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 8.8 KiB/s wr, 8 op/s 2024-04-26T02:03:20.594 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:03:20 smithi151 ceph-mon[28057]: osdmap e104: 12 total, 12 up, 12 in 2024-04-26T02:03:20.594 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:03:20 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/4097773390' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:03:20.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:03:20 smithi165 ceph-mon[32648]: pgmap v304: 97 pgs: 97 active+clean; 112 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 8.8 KiB/s wr, 8 op/s 2024-04-26T02:03:20.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:03:20 smithi165 ceph-mon[32648]: osdmap e104: 12 total, 12 up, 12 in 2024-04-26T02:03:20.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:03:20 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/4097773390' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:03:20.651 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24509 quiesce db --set-id 6face347 --release --await' 2024-04-26T02:03:20.651 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24509 quiesce db --set-id 6face347 --release --await 2024-04-26T02:03:21.089 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:03:22.533 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:03:22 smithi151 ceph-mon[28057]: pgmap v306: 97 pgs: 97 active+clean; 112 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 8.8 KiB/s wr, 8 op/s 2024-04-26T02:03:22.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:03:22 smithi172 ceph-mon[32785]: pgmap v306: 97 pgs: 97 active+clean; 112 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 8.8 KiB/s wr, 8 op/s 2024-04-26T02:03:22.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:03:22 smithi165 ceph-mon[32648]: pgmap v306: 97 pgs: 97 active+clean; 112 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 8.8 KiB/s wr, 8 op/s 2024-04-26T02:03:23.043 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.044+0000 7ff6dd000640 1 -- 172.21.15.151:0/5816238 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff6d8074380 msgr2=0x7ff6d8074760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:23.043 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.044+0000 7ff6dd000640 1 --2- 172.21.15.151:0/5816238 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff6d8074380 0x7ff6d8074760 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8009860 tx=0x7ff6c802a3d0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.043 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.044+0000 7ff6dd000640 1 -- 172.21.15.151:0/5816238 shutdown_connections 2024-04-26T02:03:23.043 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.044+0000 7ff6dd000640 1 --2- 172.21.15.151:0/5816238 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6d810e590 0x7ff6d81109e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.043 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.044+0000 7ff6dd000640 1 --2- 172.21.15.151:0/5816238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6d8074ca0 0x7ff6d810e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.043 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.044+0000 7ff6dd000640 1 --2- 172.21.15.151:0/5816238 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff6d8074380 0x7ff6d8074760 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.043 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.044+0000 7ff6dd000640 1 -- 172.21.15.151:0/5816238 >> 172.21.15.151:0/5816238 conn(0x7ff6d80fd8a0 msgr2=0x7ff6d80ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:23.043 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.045+0000 7ff6dd000640 1 -- 172.21.15.151:0/5816238 shutdown_connections 2024-04-26T02:03:23.043 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.045+0000 7ff6dd000640 1 -- 172.21.15.151:0/5816238 wait complete. 2024-04-26T02:03:23.044 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.046+0000 7ff6dd000640 1 Processor -- start 2024-04-26T02:03:23.045 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.046+0000 7ff6dd000640 1 -- start start 2024-04-26T02:03:23.045 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.046+0000 7ff6dd000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6d8074380 0x7ff6d81a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:23.045 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.046+0000 7ff6dd000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6d8074ca0 0x7ff6d81a0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:23.045 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.046+0000 7ff6dd000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff6d810e590 0x7ff6d819a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:23.045 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.046+0000 7ff6dd000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d8114220 con 0x7ff6d8074380 2024-04-26T02:03:23.045 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.046+0000 7ff6dd000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d81140a0 con 0x7ff6d810e590 2024-04-26T02:03:23.045 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.046+0000 7ff6dd000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d81143a0 con 0x7ff6d8074ca0 2024-04-26T02:03:23.045 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.047+0000 7ff6d7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff6d810e590 0x7ff6d819a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:23.045 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.047+0000 7ff6d7400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6d8074380 0x7ff6d81a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:23.045 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.047+0000 7ff6d7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff6d810e590 0x7ff6d819a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:43172/0 (socket says 172.21.15.151:43172) 2024-04-26T02:03:23.045 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.047+0000 7ff6d7e00640 1 -- 172.21.15.151:0/2698284320 learned_addr learned my addr 172.21.15.151:0/2698284320 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:03:23.045 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.047+0000 7ff6cfe00640 1 --2- 172.21.15.151:0/2698284320 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6d8074ca0 0x7ff6d81a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:23.047 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.047+0000 7ff6d7e00640 1 -- 172.21.15.151:0/2698284320 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6d8074ca0 msgr2=0x7ff6d81a0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:23.047 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.047+0000 7ff6d7e00640 1 --2- 172.21.15.151:0/2698284320 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6d8074ca0 0x7ff6d81a0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.047 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.047+0000 7ff6d7e00640 1 -- 172.21.15.151:0/2698284320 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6d8074380 msgr2=0x7ff6d81a0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:23.047 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.047+0000 7ff6d7e00640 1 --2- 172.21.15.151:0/2698284320 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6d8074380 0x7ff6d81a0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.047 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.047+0000 7ff6d7e00640 1 -- 172.21.15.151:0/2698284320 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6d819ad60 con 0x7ff6d810e590 2024-04-26T02:03:23.047 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.047+0000 7ff6d7400640 1 --2- 172.21.15.151:0/2698284320 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6d8074380 0x7ff6d81a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:03:23.047 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.047+0000 7ff6d7e00640 1 --2- 172.21.15.151:0/2698284320 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff6d810e590 0x7ff6d819a5e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0009fd0 tx=0x7ff6d000c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:23.047 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.048+0000 7ff6d4c00640 1 -- 172.21.15.151:0/2698284320 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d0004b20 con 0x7ff6d810e590 2024-04-26T02:03:23.047 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.048+0000 7ff6d4c00640 1 -- 172.21.15.151:0/2698284320 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6d0003e20 con 0x7ff6d810e590 2024-04-26T02:03:23.047 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.048+0000 7ff6dd000640 1 -- 172.21.15.151:0/2698284320 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6d819b050 con 0x7ff6d810e590 2024-04-26T02:03:23.047 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.048+0000 7ff6d4c00640 1 -- 172.21.15.151:0/2698284320 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d0004820 con 0x7ff6d810e590 2024-04-26T02:03:23.047 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.048+0000 7ff6dd000640 1 -- 172.21.15.151:0/2698284320 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6d81a72f0 con 0x7ff6d810e590 2024-04-26T02:03:23.049 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.051+0000 7ff6e143b740 1 Processor -- start 2024-04-26T02:03:23.049 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.051+0000 7ff6d4c00640 1 -- 172.21.15.151:0/2698284320 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7ff6d001d020 con 0x7ff6d810e590 2024-04-26T02:03:23.049 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.052+0000 7ff6e143b740 1 -- start start 2024-04-26T02:03:23.051 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.052+0000 7ff6e143b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5629f43df9b0 0x5629f43dfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:23.052 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.052+0000 7ff6e143b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5629f43e02d0 0x5629f43ea650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:23.052 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.052+0000 7ff6e143b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629f43eab90 0x5629f43ecf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:23.052 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.052+0000 7ff6e143b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5629f4345900 con 0x5629f43e02d0 2024-04-26T02:03:23.052 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.052+0000 7ff6e143b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5629f42ebef0 con 0x5629f43df9b0 2024-04-26T02:03:23.052 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.052+0000 7ff6e143b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5629f43418c0 con 0x5629f43eab90 2024-04-26T02:03:23.052 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.052+0000 7ff6d7400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5629f43e02d0 0x5629f43ea650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:23.052 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.052+0000 7ff6cfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5629f43df9b0 0x5629f43dfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:23.052 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.052+0000 7ff6d7400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5629f43e02d0 0x5629f43ea650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37342/0 (socket says 172.21.15.151:37342) 2024-04-26T02:03:23.052 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.052+0000 7ff6d7400640 1 -- 172.21.15.151:0/3998327469 learned_addr learned my addr 172.21.15.151:0/3998327469 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:03:23.052 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.053+0000 7ff6cfe00640 1 --2- 172.21.15.151:0/3998327469 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629f43eab90 0x5629f43ecf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:23.052 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.053+0000 7ff6d4c00640 1 --2- 172.21.15.151:0/2698284320 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7ff6bc0a3d10 0x7ff6bc0a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:23.052 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.053+0000 7ff6d7400640 1 -- 172.21.15.151:0/3998327469 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629f43eab90 msgr2=0x5629f43ecf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.053+0000 7ff6d7400640 1 --2- 172.21.15.151:0/3998327469 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629f43eab90 0x5629f43ecf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.053+0000 7ff6d7400640 1 -- 172.21.15.151:0/3998327469 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5629f43df9b0 msgr2=0x5629f43dfd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.053+0000 7ff6d7400640 1 --2- 172.21.15.151:0/3998327469 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5629f43df9b0 0x5629f43dfd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.053+0000 7ff6d7400640 1 -- 172.21.15.151:0/3998327469 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629f4058210 con 0x5629f43e02d0 2024-04-26T02:03:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.053+0000 7ff6d4c00640 1 -- 172.21.15.151:0/2698284320 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8455+0+0 (secure 0 0 0) 0x7ff6d00d0760 con 0x7ff6d810e590 2024-04-26T02:03:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.053+0000 7ff6d7e00640 1 --2- 172.21.15.151:0/2698284320 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7ff6bc0a3d10 0x7ff6bc0a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.053+0000 7ff6d7400640 1 --2- 172.21.15.151:0/3998327469 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5629f43e02d0 0x5629f43ea650 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8034000 tx=0x7ff6c8030860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63b937951afcfe9b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.054+0000 7ff6ccc00640 1 -- 172.21.15.151:0/3998327469 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c8004230 con 0x5629f43e02d0 2024-04-26T02:03:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.054+0000 7ff6ccc00640 1 -- 172.21.15.151:0/3998327469 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6c8041030 con 0x5629f43e02d0 2024-04-26T02:03:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.054+0000 7ff6ccc00640 1 -- 172.21.15.151:0/3998327469 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c80046c0 con 0x5629f43e02d0 2024-04-26T02:03:23.056 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.055+0000 7ff6e143b740 1 -- 172.21.15.151:0/3998327469 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5629f43e02d0 msgr2=0x5629f43ea650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:23.056 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.055+0000 7ff6d7e00640 1 --2- 172.21.15.151:0/2698284320 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7ff6bc0a3d10 0x7ff6bc0a61d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff6d000ffd0 tx=0x7ff6d0004a40 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:23.056 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.055+0000 7ff6e143b740 1 --2- 172.21.15.151:0/3998327469 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5629f43e02d0 0x5629f43ea650 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8034000 tx=0x7ff6c8030860 comp rx=0 tx=0).stop 2024-04-26T02:03:23.056 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.055+0000 7ff6e143b740 1 -- 172.21.15.151:0/3998327469 shutdown_connections 2024-04-26T02:03:23.056 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.055+0000 7ff6e143b740 1 --2- 172.21.15.151:0/3998327469 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629f43eab90 0x5629f43ecf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.056 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.055+0000 7ff6e143b740 1 --2- 172.21.15.151:0/3998327469 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5629f43e02d0 0x5629f43ea650 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.056 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.055+0000 7ff6e143b740 1 --2- 172.21.15.151:0/3998327469 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5629f43df9b0 0x5629f43dfd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.056 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.055+0000 7ff6e143b740 1 -- 172.21.15.151:0/3998327469 >> 172.21.15.151:0/3998327469 conn(0x5629f43dbec0 msgr2=0x5629f43dc2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:23.056 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.055+0000 7ff6e143b740 1 -- 172.21.15.151:0/3998327469 shutdown_connections 2024-04-26T02:03:23.056 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.055+0000 7ff6e143b740 1 -- 172.21.15.151:0/3998327469 wait complete. 2024-04-26T02:03:23.057 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.055+0000 7ff6e143b740 1 -- start start 2024-04-26T02:03:23.057 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.056+0000 7ff6e143b740 1 Processor -- start 2024-04-26T02:03:23.057 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.056+0000 7ff6e143b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5629f43df9b0 0x5629f43e7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:23.057 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.056+0000 7ff6e143b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629f43e02d0 0x5629f43e8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:23.057 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.056+0000 7ff6e143b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5629f43eab90 0x5629f43e8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:23.057 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.056+0000 7ff6e143b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5629f43ed760 con 0x5629f43eab90 2024-04-26T02:03:23.057 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.056+0000 7ff6e143b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5629f42ff310 con 0x5629f43df9b0 2024-04-26T02:03:23.057 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.056+0000 7ff6e143b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5629f43ed8e0 con 0x5629f43e02d0 2024-04-26T02:03:23.057 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.056+0000 7ff6e143b740 10 client.0 _ll_register_callbacks cb 0x5629f42cdcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:03:23.057 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.057+0000 7ff6cfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5629f43df9b0 0x5629f43e7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:23.057 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.057+0000 7ff6cfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5629f43df9b0 0x5629f43e7b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:43200/0 (socket says 172.21.15.151:43200) 2024-04-26T02:03:23.057 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.057+0000 7ff6cfe00640 1 -- 172.21.15.151:0/2407150717 learned_addr learned my addr 172.21.15.151:0/2407150717 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:03:23.057 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.057+0000 7ff6cfe00640 1 -- 172.21.15.151:0/2407150717 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629f43e02d0 msgr2=0x5629f43e8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:23.057 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.057+0000 7ff6cfe00640 1 --2- 172.21.15.151:0/2407150717 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629f43e02d0 0x5629f43e8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.058 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.057+0000 7ff6cfe00640 1 -- 172.21.15.151:0/2407150717 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5629f43eab90 msgr2=0x5629f43e8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:23.058 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.057+0000 7ff6cfe00640 1 --2- 172.21.15.151:0/2407150717 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5629f43eab90 0x5629f43e8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.058 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.057+0000 7ff6cfe00640 1 -- 172.21.15.151:0/2407150717 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629f4124620 con 0x5629f43df9b0 2024-04-26T02:03:23.058 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.057+0000 7ff6cfe00640 1 --2- 172.21.15.151:0/2407150717 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5629f43df9b0 0x5629f43e7b00 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff6c0004630 tx=0x7ff6c0012f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7250bc415badacf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:23.058 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.057+0000 7ff6c5600640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T02:03:23.058 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.058+0000 7ff6c5600640 1 -- 172.21.15.151:0/2407150717 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c000fbd0 con 0x5629f43df9b0 2024-04-26T02:03:23.058 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.058+0000 7ff6e143b740 1 -- 172.21.15.151:0/2407150717 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6c8041030 con 0x5629f43df9b0 2024-04-26T02:03:23.058 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.058+0000 7ff6c5600640 1 -- 172.21.15.151:0/2407150717 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8455+0+0 (secure 0 0 0) 0x7ff6c001a040 con 0x5629f43df9b0 2024-04-26T02:03:23.058 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.058+0000 7ff6c5600640 1 -- 172.21.15.151:0/2407150717 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff6c000fd70 con 0x5629f43df9b0 2024-04-26T02:03:23.058 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.058+0000 7ff6e143b740 10 client.26294 fetch_fsmap learned FSMap version 35 2024-04-26T02:03:23.058 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.058+0000 7ff6e143b740 1 -- 172.21.15.151:0/2407150717 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5629f43f9a90 con 0x5629f43df9b0 2024-04-26T02:03:23.060 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.058+0000 7ff6c5600640 1 -- 172.21.15.151:0/2407150717 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6c8041030 con 0x5629f43df9b0 2024-04-26T02:03:23.060 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.059+0000 7ff6c5600640 1 -- 172.21.15.151:0/2407150717 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 35) ==== 8338+0+0 (secure 0 0 0) 0x7ff6c001d070 con 0x5629f43df9b0 2024-04-26T02:03:23.060 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.059+0000 7ff6e143b740 10 client.26294 fetch_fsmap finished waiting for FSMap version 35 2024-04-26T02:03:23.060 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.059+0000 7ff6e143b740 10 client.26294 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:03:23.060 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.059+0000 7ff6e143b740 20 client.26294 populate_metadata read hostname 'smithi151' 2024-04-26T02:03:23.060 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.059+0000 7ff6e143b740 1 --2- 172.21.15.151:0/2407150717 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5629f43faf00 0x5629f43fd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.059+0000 7ff6e143b740 4 client.26294 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:03:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.059+0000 7ff6e143b740 1 -- 172.21.15.151:0/2407150717 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629f43598b0 con 0x5629f43faf00 2024-04-26T02:03:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.059+0000 7ff6d7400640 1 --2- 172.21.15.151:0/2407150717 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5629f43faf00 0x5629f43fd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.060+0000 7ff6d7400640 1 --2- 172.21.15.151:0/2407150717 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5629f43faf00 0x5629f43fd390 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52006c926bafb48f server_cookie=b1d8d3e15d1c674d in_seq=0 out_seq=0 2024-04-26T02:03:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.060+0000 7ff6c5600640 10 client.26294 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:03:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.061+0000 7ff6c5600640 1 -- 172.21.15.151:0/2407150717 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5629f43598b0 con 0x5629f43faf00 2024-04-26T02:03:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.061+0000 7ff6c5600640 10 client.26294 handle_command_reply: tid=0 2024-04-26T02:03:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.061+0000 7ff6e143b740 1 client.26294 shutdown 2024-04-26T02:03:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.061+0000 7ff6e143b740 1 -- 172.21.15.151:0/2407150717 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5629f43df9b0 msgr2=0x5629f43e7b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.061+0000 7ff6e143b740 1 --2- 172.21.15.151:0/2407150717 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5629f43df9b0 0x5629f43e7b00 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff6c0004630 tx=0x7ff6c0012f70 comp rx=0 tx=0).stop 2024-04-26T02:03:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.061+0000 7ff6e143b740 1 -- 172.21.15.151:0/2407150717 shutdown_connections 2024-04-26T02:03:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.061+0000 7ff6e143b740 1 --2- 172.21.15.151:0/2407150717 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5629f43faf00 0x5629f43fd390 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.061+0000 7ff6e143b740 1 --2- 172.21.15.151:0/2407150717 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5629f43eab90 0x5629f43e8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.061+0000 7ff6e143b740 1 --2- 172.21.15.151:0/2407150717 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629f43e02d0 0x5629f43e8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.061+0000 7ff6e143b740 1 --2- 172.21.15.151:0/2407150717 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5629f43df9b0 0x5629f43e7b00 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.061+0000 7ff6c5600640 0 client.26294 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:03:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.061+0000 7ff6e143b740 1 -- 172.21.15.151:0/2407150717 >> 172.21.15.151:0/2407150717 conn(0x5629f43dbec0 msgr2=0x5629f43e1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.061+0000 7ff6e143b740 1 -- 172.21.15.151:0/2407150717 shutdown_connections 2024-04-26T02:03:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.061+0000 7ff6e143b740 1 -- 172.21.15.151:0/2407150717 wait complete. 2024-04-26T02:03:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.061+0000 7ff6e143b740 20 client.26294 trim_cache size 0 max 16384 2024-04-26T02:03:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.078+0000 7ff6e143b740 1 Processor -- start 2024-04-26T02:03:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.078+0000 7ff6e143b740 1 -- start start 2024-04-26T02:03:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.079+0000 7ff6e143b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5629f43df9b0 0x5629f43f6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.079+0000 7ff6e143b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5629f43e02d0 0x5629f43e2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.079+0000 7ff6e143b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629f43eab90 0x5629f43daea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.079+0000 7ff6e143b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d81b24a0 con 0x5629f43e02d0 2024-04-26T02:03:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.079+0000 7ff6e143b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d81afd70 con 0x5629f43df9b0 2024-04-26T02:03:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.079+0000 7ff6e143b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d81adbe0 con 0x5629f43eab90 2024-04-26T02:03:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.079+0000 7ff6cfe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629f43eab90 0x5629f43daea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.079+0000 7ff6d7400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5629f43e02d0 0x5629f43e2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.079+0000 7ff6cfe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629f43eab90 0x5629f43daea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:48878/0 (socket says 172.21.15.151:48878) 2024-04-26T02:03:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.079+0000 7ff6cfe00640 1 -- 172.21.15.151:0/2373172 learned_addr learned my addr 172.21.15.151:0/2373172 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:03:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.079+0000 7ff6cfe00640 1 --2- 172.21.15.151:0/2373172 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5629f43df9b0 0x5629f43f6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.079+0000 7ff6cfe00640 1 -- 172.21.15.151:0/2373172 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5629f43df9b0 msgr2=0x5629f43f6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.079+0000 7ff6cfe00640 1 --2- 172.21.15.151:0/2373172 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5629f43df9b0 0x5629f43f6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.079+0000 7ff6cfe00640 1 -- 172.21.15.151:0/2373172 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5629f43e02d0 msgr2=0x5629f43e2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.079+0000 7ff6cfe00640 1 --2- 172.21.15.151:0/2373172 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5629f43e02d0 0x5629f43e2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.079+0000 7ff6cfe00640 1 -- 172.21.15.151:0/2373172 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629f43e28b0 con 0x5629f43eab90 2024-04-26T02:03:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.080+0000 7ff6cfe00640 1 --2- 172.21.15.151:0/2373172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629f43eab90 0x5629f43daea0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff6c0009fd0 tx=0x7ff6c0000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57e52fc22fb1b67b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.080+0000 7ff6c7e00640 1 -- 172.21.15.151:0/2373172 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c002c040 con 0x5629f43eab90 2024-04-26T02:03:23.080 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.080+0000 7ff6c7e00640 1 -- 172.21.15.151:0/2373172 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6c0012350 con 0x5629f43eab90 2024-04-26T02:03:23.080 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.080+0000 7ff6c7e00640 1 -- 172.21.15.151:0/2373172 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c0028a10 con 0x5629f43eab90 2024-04-26T02:03:23.080 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.081+0000 7ff6e143b740 1 -- 172.21.15.151:0/2373172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629f43eab90 msgr2=0x5629f43daea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:23.080 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.081+0000 7ff6e143b740 1 --2- 172.21.15.151:0/2373172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629f43eab90 0x5629f43daea0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff6c0009fd0 tx=0x7ff6c0000bc0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.080 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.081+0000 7ff6e143b740 1 -- 172.21.15.151:0/2373172 shutdown_connections 2024-04-26T02:03:23.080 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.081+0000 7ff6e143b740 1 --2- 172.21.15.151:0/2373172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629f43eab90 0x5629f43daea0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.080 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.081+0000 7ff6e143b740 1 --2- 172.21.15.151:0/2373172 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5629f43e02d0 0x5629f43e2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.080 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.081+0000 7ff6e143b740 1 --2- 172.21.15.151:0/2373172 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5629f43df9b0 0x5629f43f6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.080 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.081+0000 7ff6e143b740 1 -- 172.21.15.151:0/2373172 >> 172.21.15.151:0/2373172 conn(0x5629f43dbec0 msgr2=0x5629f43da790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:23.080 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.081+0000 7ff6e143b740 1 -- 172.21.15.151:0/2373172 shutdown_connections 2024-04-26T02:03:23.080 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.081+0000 7ff6e143b740 1 -- 172.21.15.151:0/2373172 wait complete. 2024-04-26T02:03:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.081+0000 7ff6e143b740 1 -- start start 2024-04-26T02:03:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.081+0000 7ff6e143b740 1 Processor -- start 2024-04-26T02:03:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.082+0000 7ff6e143b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5629f43df9b0 0x5629f43f2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.082+0000 7ff6e143b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629f43e02d0 0x5629f43f0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.082+0000 7ff6e143b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5629f43eab90 0x5629f43f07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.082+0000 7ff6e143b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d81ae230 con 0x5629f43eab90 2024-04-26T02:03:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.082+0000 7ff6e143b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d81af6a0 con 0x5629f43df9b0 2024-04-26T02:03:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.082+0000 7ff6e143b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d81ae900 con 0x5629f43e02d0 2024-04-26T02:03:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.082+0000 7ff6e143b740 10 client.0 _ll_register_callbacks cb 0x7ff6c00129f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:03:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.082+0000 7ff6d7400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629f43e02d0 0x5629f43f0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.082+0000 7ff6cfe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5629f43eab90 0x5629f43f07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.082+0000 7ff6d7400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629f43e02d0 0x5629f43f0260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:48886/0 (socket says 172.21.15.151:48886) 2024-04-26T02:03:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.082+0000 7ff6cfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5629f43df9b0 0x5629f43f2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.082+0000 7ff6d7400640 1 -- 172.21.15.151:0/1846948253 learned_addr learned my addr 172.21.15.151:0/1846948253 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:03:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.083+0000 7ff6cfe00640 1 -- 172.21.15.151:0/1846948253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629f43e02d0 msgr2=0x5629f43f0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.083+0000 7ff6cfe00640 1 --2- 172.21.15.151:0/1846948253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629f43e02d0 0x5629f43f0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.083+0000 7ff6cfe00640 1 -- 172.21.15.151:0/1846948253 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5629f43eab90 msgr2=0x5629f43f07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.083+0000 7ff6cfe00640 1 --2- 172.21.15.151:0/1846948253 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5629f43eab90 0x5629f43f07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.083+0000 7ff6cfe00640 1 -- 172.21.15.151:0/1846948253 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629f43f0fa0 con 0x5629f43df9b0 2024-04-26T02:03:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.083+0000 7ff6d7400640 1 --2- 172.21.15.151:0/1846948253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629f43e02d0 0x5629f43f0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:03:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.083+0000 7ff6cfe00640 1 --2- 172.21.15.151:0/1846948253 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5629f43df9b0 0x5629f43f2e80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff6c00058c0 tx=0x7ff6c0005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d8949214a1ba822 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.083+0000 7ff6c5600640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T02:03:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.083+0000 7ff6c5600640 1 -- 172.21.15.151:0/1846948253 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c002c030 con 0x5629f43df9b0 2024-04-26T02:03:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.083+0000 7ff6c5600640 1 -- 172.21.15.151:0/1846948253 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8455+0+0 (secure 0 0 0) 0x7ff6c0011ea0 con 0x5629f43df9b0 2024-04-26T02:03:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.083+0000 7ff6e143b740 1 -- 172.21.15.151:0/1846948253 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6c0012350 con 0x5629f43df9b0 2024-04-26T02:03:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.084+0000 7ff6c5600640 1 -- 172.21.15.151:0/1846948253 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff6c00120a0 con 0x5629f43df9b0 2024-04-26T02:03:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.084+0000 7ff6e143b740 10 client.26300 fetch_fsmap learned FSMap version 35 2024-04-26T02:03:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.084+0000 7ff6e143b740 1 -- 172.21.15.151:0/1846948253 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5629f44614c0 con 0x5629f43df9b0 2024-04-26T02:03:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.084+0000 7ff6c5600640 1 -- 172.21.15.151:0/1846948253 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6c0012350 con 0x5629f43df9b0 2024-04-26T02:03:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.084+0000 7ff6c5600640 1 -- 172.21.15.151:0/1846948253 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 35) ==== 8338+0+0 (secure 0 0 0) 0x7ff6c001d040 con 0x5629f43df9b0 2024-04-26T02:03:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.085+0000 7ff6e143b740 10 client.26300 fetch_fsmap finished waiting for FSMap version 35 2024-04-26T02:03:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.085+0000 7ff6e143b740 10 client.26300 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:03:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.085+0000 7ff6e143b740 20 client.26300 populate_metadata read hostname 'smithi151' 2024-04-26T02:03:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.085+0000 7ff6e143b740 1 --2- 172.21.15.151:0/1846948253 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5629f4461ac0 0x5629f4463eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.085+0000 7ff6e143b740 4 client.26300 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T02:03:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.085+0000 7ff6e143b740 1 -- 172.21.15.151:0/1846948253 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5629f4353320 con 0x5629f4461ac0 2024-04-26T02:03:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.085+0000 7ff6d7400640 1 --2- 172.21.15.151:0/1846948253 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5629f4461ac0 0x5629f4463eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.086+0000 7ff6c5600640 10 client.26300 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:03:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.086+0000 7ff6d7400640 1 --2- 172.21.15.151:0/1846948253 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5629f4461ac0 0x5629f4463eb0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5b54261475fdc43 server_cookie=be9a2298f8451bed in_seq=0 out_seq=0 2024-04-26T02:03:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.095+0000 7ff6c5600640 1 -- 172.21.15.151:0/1846948253 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5629f4353320 con 0x5629f4461ac0 2024-04-26T02:03:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.095+0000 7ff6c5600640 10 client.26300 handle_command_reply: tid=0 2024-04-26T02:03:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.095+0000 7ff6e143b740 1 client.26300 shutdown 2024-04-26T02:03:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.095+0000 7ff6e143b740 1 -- 172.21.15.151:0/1846948253 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5629f43df9b0 msgr2=0x5629f43f2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.095+0000 7ff6e143b740 1 --2- 172.21.15.151:0/1846948253 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5629f43df9b0 0x5629f43f2e80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff6c00058c0 tx=0x7ff6c0005f90 comp rx=0 tx=0).stop 2024-04-26T02:03:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.095+0000 7ff6e143b740 1 -- 172.21.15.151:0/1846948253 shutdown_connections 2024-04-26T02:03:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.095+0000 7ff6e143b740 1 --2- 172.21.15.151:0/1846948253 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5629f4461ac0 0x5629f4463eb0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.095+0000 7ff6e143b740 1 --2- 172.21.15.151:0/1846948253 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5629f43eab90 0x5629f43f07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.095+0000 7ff6e143b740 1 --2- 172.21.15.151:0/1846948253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629f43e02d0 0x5629f43f0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.095+0000 7ff6c5600640 0 client.26300 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:03:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.095+0000 7ff6e143b740 1 --2- 172.21.15.151:0/1846948253 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5629f43df9b0 0x5629f43f2e80 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.095+0000 7ff6e143b740 1 -- 172.21.15.151:0/1846948253 >> 172.21.15.151:0/1846948253 conn(0x5629f43dbec0 msgr2=0x5629f43e8350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.095+0000 7ff6e143b740 1 -- 172.21.15.151:0/1846948253 shutdown_connections 2024-04-26T02:03:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.095+0000 7ff6e143b740 1 -- 172.21.15.151:0/1846948253 wait complete. 2024-04-26T02:03:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.095+0000 7ff6e143b740 20 client.26300 trim_cache size 0 max 16384 2024-04-26T02:03:23.095 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:{ 2024-04-26T02:03:23.095 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "message": "", 2024-04-26T02:03:23.095 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "return_code": 0 2024-04-26T02:03:23.095 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:} 2024-04-26T02:03:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.096+0000 7ff6dd000640 1 -- 172.21.15.151:0/2698284320 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7ff6bc0a3d10 msgr2=0x7ff6bc0a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.096+0000 7ff6dd000640 1 --2- 172.21.15.151:0/2698284320 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7ff6bc0a3d10 0x7ff6bc0a61d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff6d000ffd0 tx=0x7ff6d0004a40 comp rx=0 tx=0).stop 2024-04-26T02:03:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.096+0000 7ff6dd000640 1 -- 172.21.15.151:0/2698284320 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff6d810e590 msgr2=0x7ff6d819a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.096+0000 7ff6dd000640 1 --2- 172.21.15.151:0/2698284320 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff6d810e590 0x7ff6d819a5e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0009fd0 tx=0x7ff6d000c800 comp rx=0 tx=0).stop 2024-04-26T02:03:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.096+0000 7ff6dd000640 1 -- 172.21.15.151:0/2698284320 shutdown_connections 2024-04-26T02:03:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.096+0000 7ff6dd000640 1 --2- 172.21.15.151:0/2698284320 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7ff6bc0a3d10 0x7ff6bc0a61d0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.096+0000 7ff6dd000640 1 --2- 172.21.15.151:0/2698284320 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff6d810e590 0x7ff6d819a5e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.096+0000 7ff6dd000640 1 --2- 172.21.15.151:0/2698284320 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6d8074ca0 0x7ff6d81a0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.096+0000 7ff6dd000640 1 --2- 172.21.15.151:0/2698284320 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6d8074380 0x7ff6d81a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.096+0000 7ff6dd000640 1 -- 172.21.15.151:0/2698284320 >> 172.21.15.151:0/2698284320 conn(0x7ff6d80fd8a0 msgr2=0x7ff6d80fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.097+0000 7ff6dd000640 1 -- 172.21.15.151:0/2698284320 shutdown_connections 2024-04-26T02:03:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:23.097+0000 7ff6dd000640 1 -- 172.21.15.151:0/2698284320 wait complete. 2024-04-26T02:03:23.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.385+0000 7f60e2200700 1 -- 172.21.15.151:0/2188786999 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60dc10e3b0 msgr2=0x7f60dc115f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:23.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.385+0000 7f60e2200700 1 --2- 172.21.15.151:0/2188786999 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60dc10e3b0 0x7f60dc115f80 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f60d4009fc0 tx=0x7f60d400b330 comp rx=0 tx=0).stop 2024-04-26T02:03:23.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.385+0000 7f60e2200700 1 -- 172.21.15.151:0/2188786999 shutdown_connections 2024-04-26T02:03:23.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.385+0000 7f60e2200700 1 --2- 172.21.15.151:0/2188786999 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60dc10e3b0 0x7f60dc115f80 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.385+0000 7f60e2200700 1 --2- 172.21.15.151:0/2188786999 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60dc1092d0 0x7f60dc1096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.385+0000 7f60e2200700 1 --2- 172.21.15.151:0/2188786999 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60dc109c80 0x7f60dc10dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.385+0000 7f60e2200700 1 -- 172.21.15.151:0/2188786999 >> 172.21.15.151:0/2188786999 conn(0x7f60dc076210 msgr2=0x7f60dc107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:23.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.385+0000 7f60e2200700 1 -- 172.21.15.151:0/2188786999 shutdown_connections 2024-04-26T02:03:23.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.385+0000 7f60e2200700 1 -- 172.21.15.151:0/2188786999 wait complete. 2024-04-26T02:03:23.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.386+0000 7f60e2200700 1 Processor -- start 2024-04-26T02:03:23.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.386+0000 7f60e2200700 1 -- start start 2024-04-26T02:03:23.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.387+0000 7f60e2200700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60dc1092d0 0x7f60dc19f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:23.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.387+0000 7f60e2200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60dc109c80 0x7f60dc19fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:23.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.387+0000 7f60e2200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60dc10e3b0 0x7f60dc1a3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:23.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.387+0000 7f60e2200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f60dc1192c0 con 0x7f60dc10e3b0 2024-04-26T02:03:23.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.387+0000 7f60e2200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f60dc119140 con 0x7f60dc109c80 2024-04-26T02:03:23.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.387+0000 7f60e2200700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f60dc119440 con 0x7f60dc1092d0 2024-04-26T02:03:23.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.387+0000 7f60dbe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60dc10e3b0 0x7f60dc1a3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:23.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.387+0000 7f60daa00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60dc109c80 0x7f60dc19fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:23.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.387+0000 7f60db400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60dc1092d0 0x7f60dc19f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:23.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.387+0000 7f60daa00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60dc109c80 0x7f60dc19fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:43242/0 (socket says 172.21.15.151:43242) 2024-04-26T02:03:23.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.387+0000 7f60dbe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60dc10e3b0 0x7f60dc1a3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37378/0 (socket says 172.21.15.151:37378) 2024-04-26T02:03:23.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.387+0000 7f60daa00700 1 -- 172.21.15.151:0/706262076 learned_addr learned my addr 172.21.15.151:0/706262076 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:03:23.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.388+0000 7f60daa00700 1 -- 172.21.15.151:0/706262076 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60dc1092d0 msgr2=0x7f60dc19f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:23.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.388+0000 7f60daa00700 1 --2- 172.21.15.151:0/706262076 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60dc1092d0 0x7f60dc19f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.388+0000 7f60daa00700 1 -- 172.21.15.151:0/706262076 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60dc10e3b0 msgr2=0x7f60dc1a3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:23.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.388+0000 7f60daa00700 1 --2- 172.21.15.151:0/706262076 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60dc10e3b0 0x7f60dc1a3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.388+0000 7f60daa00700 1 -- 172.21.15.151:0/706262076 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60dc1a4720 con 0x7f60dc109c80 2024-04-26T02:03:23.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.388+0000 7f60db400700 1 --2- 172.21.15.151:0/706262076 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60dc1092d0 0x7f60dc19f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:03:23.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.388+0000 7f60daa00700 1 --2- 172.21.15.151:0/706262076 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60dc109c80 0x7f60dc19fc50 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f60c800e9f0 tx=0x7f60c800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:23.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.388+0000 7f60d3e00700 1 -- 172.21.15.151:0/706262076 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c8004a30 con 0x7f60dc109c80 2024-04-26T02:03:23.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.388+0000 7f60d3e00700 1 -- 172.21.15.151:0/706262076 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60c800cc70 con 0x7f60dc109c80 2024-04-26T02:03:23.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.388+0000 7f60d3e00700 1 -- 172.21.15.151:0/706262076 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c8005530 con 0x7f60dc109c80 2024-04-26T02:03:23.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.388+0000 7f60e2200700 1 -- 172.21.15.151:0/706262076 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60dc1a4a10 con 0x7f60dc109c80 2024-04-26T02:03:23.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.389+0000 7f60e2200700 1 -- 172.21.15.151:0/706262076 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60dc1ac290 con 0x7f60dc109c80 2024-04-26T02:03:23.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.392+0000 7f60d3e00700 1 -- 172.21.15.151:0/706262076 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f60c8004bd0 con 0x7f60dc109c80 2024-04-26T02:03:23.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.393+0000 7f60ecc82b80 1 Processor -- start 2024-04-26T02:03:23.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.393+0000 7f60ecc82b80 1 -- start start 2024-04-26T02:03:23.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.394+0000 7f60d3e00700 1 --2- 172.21.15.151:0/706262076 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f60c00a3db0 0x7f60c00a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:23.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.394+0000 7f60ecc82b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cbef58d40 0x560cbef59120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:23.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.394+0000 7f60ecc82b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560cbef59660 0x560cbef63f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:23.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.394+0000 7f60ecc82b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560cbef644d0 0x560cbef668c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:23.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.394+0000 7f60ecc82b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560cbeeabf90 con 0x560cbef59660 2024-04-26T02:03:23.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.394+0000 7f60ecc82b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560cbee52830 con 0x560cbef58d40 2024-04-26T02:03:23.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.394+0000 7f60ecc82b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560cbeea7f50 con 0x560cbef644d0 2024-04-26T02:03:23.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.394+0000 7f60d3e00700 1 -- 172.21.15.151:0/706262076 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8455+0+0 (secure 0 0 0) 0x7f60c80d0980 con 0x7f60dc109c80 2024-04-26T02:03:23.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.394+0000 7f60dbe00700 1 --2- 172.21.15.151:0/706262076 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f60c00a3db0 0x7f60c00a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:23.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.394+0000 7f60db400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cbef58d40 0x560cbef59120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:23.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.394+0000 7f60db400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cbef58d40 0x560cbef59120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:43252/0 (socket says 172.21.15.151:43252) 2024-04-26T02:03:23.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.394+0000 7f60dbe00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560cbef644d0 0x560cbef668c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:23.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.394+0000 7f60daa00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560cbef59660 0x560cbef63f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:23.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.394+0000 7f60db400700 1 -- 172.21.15.151:0/4110286503 learned_addr learned my addr 172.21.15.151:0/4110286503 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:03:23.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.395+0000 7f60db400700 1 -- 172.21.15.151:0/4110286503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560cbef644d0 msgr2=0x560cbef668c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:23.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.395+0000 7f60db400700 1 --2- 172.21.15.151:0/4110286503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560cbef644d0 0x560cbef668c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.395+0000 7f60db400700 1 -- 172.21.15.151:0/4110286503 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560cbef59660 msgr2=0x560cbef63f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:23.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.395+0000 7f60db400700 1 --2- 172.21.15.151:0/4110286503 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560cbef59660 0x560cbef63f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.395+0000 7f60db400700 1 -- 172.21.15.151:0/4110286503 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cbee48780 con 0x560cbef58d40 2024-04-26T02:03:23.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.395+0000 7f60daa00700 1 --2- 172.21.15.151:0/4110286503 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560cbef59660 0x560cbef63f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:03:23.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.395+0000 7f60dbe00700 1 --2- 172.21.15.151:0/706262076 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f60c00a3db0 0x7f60c00a6270 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f60d4009da0 tx=0x7f60d4003b80 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:23.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.395+0000 7f60db400700 1 --2- 172.21.15.151:0/4110286503 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cbef58d40 0x560cbef59120 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f60cc011650 tx=0x7f60cc011960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=749e294d24dc171f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:23.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.396+0000 7f60d0c00700 1 -- 172.21.15.151:0/4110286503 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60cc004020 con 0x560cbef58d40 2024-04-26T02:03:23.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.396+0000 7f60d0c00700 1 -- 172.21.15.151:0/4110286503 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60cc004830 con 0x560cbef58d40 2024-04-26T02:03:23.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.396+0000 7f60d0c00700 1 -- 172.21.15.151:0/4110286503 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60cc005430 con 0x560cbef58d40 2024-04-26T02:03:23.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.396+0000 7f60ecc82b80 1 -- 172.21.15.151:0/4110286503 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cbef58d40 msgr2=0x560cbef59120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:23.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.396+0000 7f60ecc82b80 1 --2- 172.21.15.151:0/4110286503 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cbef58d40 0x560cbef59120 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f60cc011650 tx=0x7f60cc011960 comp rx=0 tx=0).stop 2024-04-26T02:03:23.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.396+0000 7f60ecc82b80 1 -- 172.21.15.151:0/4110286503 shutdown_connections 2024-04-26T02:03:23.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.396+0000 7f60ecc82b80 1 --2- 172.21.15.151:0/4110286503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560cbef644d0 0x560cbef668c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.396+0000 7f60ecc82b80 1 --2- 172.21.15.151:0/4110286503 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cbef58d40 0x560cbef59120 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.396+0000 7f60ecc82b80 1 --2- 172.21.15.151:0/4110286503 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560cbef59660 0x560cbef63f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.396+0000 7f60ecc82b80 1 -- 172.21.15.151:0/4110286503 >> 172.21.15.151:0/4110286503 conn(0x560cbee88050 msgr2=0x560cbef55250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:23.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.397+0000 7f60ecc82b80 1 -- 172.21.15.151:0/4110286503 shutdown_connections 2024-04-26T02:03:23.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.397+0000 7f60ecc82b80 1 -- 172.21.15.151:0/4110286503 wait complete. 2024-04-26T02:03:23.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.397+0000 7f60ecc82b80 1 -- start start 2024-04-26T02:03:23.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.397+0000 7f60ecc82b80 1 Processor -- start 2024-04-26T02:03:23.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.398+0000 7f60ecc82b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560cbef58d40 0x560cbef63c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:23.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.398+0000 7f60ecc82b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560cbef59660 0x560cbef60f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:23.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.398+0000 7f60ecc82b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cbef644d0 0x560cbef61460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:23.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.398+0000 7f60ecc82b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560cbef66ea0 con 0x560cbef58d40 2024-04-26T02:03:23.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.398+0000 7f60ecc82b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560cbee65960 con 0x560cbef644d0 2024-04-26T02:03:23.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.398+0000 7f60ecc82b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560cbef67020 con 0x560cbef59660 2024-04-26T02:03:23.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.398+0000 7f60ecc82b80 10 client.0 _ll_register_callbacks cb 0x560cbee44820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:03:23.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.398+0000 7f60db400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560cbef58d40 0x560cbef63c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:23.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.398+0000 7f60daa00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560cbef59660 0x560cbef60f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:23.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.398+0000 7f60dbe00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cbef644d0 0x560cbef61460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:23.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.398+0000 7f60db400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560cbef58d40 0x560cbef63c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37404/0 (socket says 172.21.15.151:37404) 2024-04-26T02:03:23.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.398+0000 7f60db400700 1 -- 172.21.15.151:0/2461865476 learned_addr learned my addr 172.21.15.151:0/2461865476 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:03:23.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.399+0000 7f60dbe00700 1 -- 172.21.15.151:0/2461865476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560cbef59660 msgr2=0x560cbef60f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:23.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.399+0000 7f60dbe00700 1 --2- 172.21.15.151:0/2461865476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560cbef59660 0x560cbef60f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.399+0000 7f60dbe00700 1 -- 172.21.15.151:0/2461865476 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560cbef58d40 msgr2=0x560cbef63c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:23.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.399+0000 7f60dbe00700 1 --2- 172.21.15.151:0/2461865476 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560cbef58d40 0x560cbef63c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.399+0000 7f60dbe00700 1 -- 172.21.15.151:0/2461865476 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cbedf99d0 con 0x560cbef644d0 2024-04-26T02:03:23.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.399+0000 7f60dbe00700 1 --2- 172.21.15.151:0/2461865476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cbef644d0 0x560cbef61460 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f60d4009550 tx=0x7f60d400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78aebb22cc46c527 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:23.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.399+0000 7f60c5600700 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T02:03:23.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.399+0000 7f60c5600700 1 -- 172.21.15.151:0/2461865476 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d4031060 con 0x560cbef644d0 2024-04-26T02:03:23.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.399+0000 7f60c5600700 1 -- 172.21.15.151:0/2461865476 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8455+0+0 (secure 0 0 0) 0x7f60d4007840 con 0x560cbef644d0 2024-04-26T02:03:23.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.399+0000 7f60ecc82b80 1 -- 172.21.15.151:0/2461865476 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60cc004830 con 0x560cbef644d0 2024-04-26T02:03:23.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.400+0000 7f60c5600700 1 -- 172.21.15.151:0/2461865476 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f60d4007a40 con 0x560cbef644d0 2024-04-26T02:03:23.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.400+0000 7f60ecc82b80 10 client.26312 fetch_fsmap learned FSMap version 35 2024-04-26T02:03:23.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.400+0000 7f60ecc82b80 1 -- 172.21.15.151:0/2461865476 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x560cbee42bf0 con 0x560cbef644d0 2024-04-26T02:03:23.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.400+0000 7f60c5600700 1 -- 172.21.15.151:0/2461865476 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60cc004830 con 0x560cbef644d0 2024-04-26T02:03:23.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.400+0000 7f60c5600700 1 -- 172.21.15.151:0/2461865476 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 35) ==== 8338+0+0 (secure 0 0 0) 0x7f60d4011070 con 0x560cbef644d0 2024-04-26T02:03:23.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.401+0000 7f60ecc82b80 10 client.26312 fetch_fsmap finished waiting for FSMap version 35 2024-04-26T02:03:23.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.401+0000 7f60ecc82b80 10 client.26312 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-26T02:03:23.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.401+0000 7f60ecc82b80 20 client.26312 populate_metadata read hostname 'smithi151' 2024-04-26T02:03:23.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.401+0000 7f60ecc82b80 1 --2- 172.21.15.151:0/2461865476 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x560cbef73b40 0x560cbef75f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:23.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.401+0000 7f60ecc82b80 4 client.26312 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:03:23.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.401+0000 7f60ecc82b80 1 -- 172.21.15.151:0/2461865476 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560cbeeb9780 con 0x560cbef73b40 2024-04-26T02:03:23.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.401+0000 7f60db400700 1 --2- 172.21.15.151:0/2461865476 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x560cbef73b40 0x560cbef75f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:23.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.402+0000 7f60db400700 1 --2- 172.21.15.151:0/2461865476 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x560cbef73b40 0x560cbef75f30 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e23d0e26a3e9921 server_cookie=b29cb95bd42a6cff in_seq=0 out_seq=0 2024-04-26T02:03:23.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.402+0000 7f60c5600700 10 client.26312 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:03:23.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.402+0000 7f60c5600700 1 -- 172.21.15.151:0/2461865476 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560cbeeb9780 con 0x560cbef73b40 2024-04-26T02:03:23.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.402+0000 7f60c5600700 10 client.26312 handle_command_reply: tid=0 2024-04-26T02:03:23.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.402+0000 7f60ecc82b80 1 client.26312 shutdown 2024-04-26T02:03:23.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.403+0000 7f60ecc82b80 1 -- 172.21.15.151:0/2461865476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cbef644d0 msgr2=0x560cbef61460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:23.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.403+0000 7f60ecc82b80 1 --2- 172.21.15.151:0/2461865476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cbef644d0 0x560cbef61460 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f60d4009550 tx=0x7f60d400b330 comp rx=0 tx=0).stop 2024-04-26T02:03:23.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.403+0000 7f60ecc82b80 1 -- 172.21.15.151:0/2461865476 shutdown_connections 2024-04-26T02:03:23.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.403+0000 7f60ecc82b80 1 --2- 172.21.15.151:0/2461865476 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x560cbef73b40 0x560cbef75f30 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.403+0000 7f60ecc82b80 1 --2- 172.21.15.151:0/2461865476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cbef644d0 0x560cbef61460 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.403+0000 7f60ecc82b80 1 --2- 172.21.15.151:0/2461865476 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560cbef58d40 0x560cbef63c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.403+0000 7f60ecc82b80 1 --2- 172.21.15.151:0/2461865476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560cbef59660 0x560cbef60f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.403+0000 7f60c5600700 0 client.26312 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:03:23.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.403+0000 7f60ecc82b80 1 -- 172.21.15.151:0/2461865476 >> 172.21.15.151:0/2461865476 conn(0x560cbee88050 msgr2=0x560cbef54e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:23.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.403+0000 7f60ecc82b80 1 -- 172.21.15.151:0/2461865476 shutdown_connections 2024-04-26T02:03:23.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.403+0000 7f60ecc82b80 1 -- 172.21.15.151:0/2461865476 wait complete. 2024-04-26T02:03:23.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.403+0000 7f60ecc82b80 20 client.26312 trim_cache size 0 max 16384 2024-04-26T02:03:23.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.424+0000 7f60ecc82b80 1 Processor -- start 2024-04-26T02:03:23.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.424+0000 7f60ecc82b80 1 -- start start 2024-04-26T02:03:23.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.424+0000 7f60ecc82b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cbef58d40 0x560cbef6fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:23.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.425+0000 7f60ecc82b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560cbef59660 0x560cbef62e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:23.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.425+0000 7f60ecc82b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560cbef644d0 0x560cbef6c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:23.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.425+0000 7f60ecc82b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f60dc1b6040 con 0x560cbef644d0 2024-04-26T02:03:23.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.425+0000 7f60ecc82b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f60dc1b38d0 con 0x560cbef58d40 2024-04-26T02:03:23.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.425+0000 7f60ecc82b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f60dc1b1740 con 0x560cbef59660 2024-04-26T02:03:23.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.425+0000 7f60dbe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560cbef644d0 0x560cbef6c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:23.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.425+0000 7f60dbe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560cbef644d0 0x560cbef6c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37416/0 (socket says 172.21.15.151:37416) 2024-04-26T02:03:23.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.425+0000 7f60dbe00700 1 -- 172.21.15.151:0/1321420414 learned_addr learned my addr 172.21.15.151:0/1321420414 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:03:23.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.425+0000 7f60db400700 1 --2- 172.21.15.151:0/1321420414 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cbef58d40 0x560cbef6fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:23.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.425+0000 7f60daa00700 1 --2- 172.21.15.151:0/1321420414 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560cbef59660 0x560cbef62e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:23.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.425+0000 7f60db400700 1 -- 172.21.15.151:0/1321420414 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560cbef59660 msgr2=0x560cbef62e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:23.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.425+0000 7f60db400700 1 --2- 172.21.15.151:0/1321420414 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560cbef59660 0x560cbef62e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.425+0000 7f60db400700 1 -- 172.21.15.151:0/1321420414 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560cbef644d0 msgr2=0x560cbef6c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:23.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.425+0000 7f60db400700 1 --2- 172.21.15.151:0/1321420414 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560cbef644d0 0x560cbef6c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.425+0000 7f60db400700 1 -- 172.21.15.151:0/1321420414 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cbef59ce0 con 0x560cbef58d40 2024-04-26T02:03:23.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.426+0000 7f60db400700 1 --2- 172.21.15.151:0/1321420414 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cbef58d40 0x560cbef6fe00 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f60cc011cb0 tx=0x7f60cc009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f08999b9717424fd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:23.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.426+0000 7f60c7e00700 1 -- 172.21.15.151:0/1321420414 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60cc030070 con 0x560cbef58d40 2024-04-26T02:03:23.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.426+0000 7f60c7e00700 1 -- 172.21.15.151:0/1321420414 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60cc02a700 con 0x560cbef58d40 2024-04-26T02:03:23.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.426+0000 7f60c7e00700 1 -- 172.21.15.151:0/1321420414 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60cc01d040 con 0x560cbef58d40 2024-04-26T02:03:23.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.427+0000 7f60ecc82b80 1 -- 172.21.15.151:0/1321420414 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cbef58d40 msgr2=0x560cbef6fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:23.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.427+0000 7f60ecc82b80 1 --2- 172.21.15.151:0/1321420414 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cbef58d40 0x560cbef6fe00 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f60cc011cb0 tx=0x7f60cc009310 comp rx=0 tx=0).stop 2024-04-26T02:03:23.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.427+0000 7f60ecc82b80 1 -- 172.21.15.151:0/1321420414 shutdown_connections 2024-04-26T02:03:23.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.427+0000 7f60ecc82b80 1 --2- 172.21.15.151:0/1321420414 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560cbef644d0 0x560cbef6c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.427+0000 7f60ecc82b80 1 --2- 172.21.15.151:0/1321420414 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cbef58d40 0x560cbef6fe00 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.427+0000 7f60ecc82b80 1 --2- 172.21.15.151:0/1321420414 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560cbef59660 0x560cbef62e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.427+0000 7f60ecc82b80 1 -- 172.21.15.151:0/1321420414 >> 172.21.15.151:0/1321420414 conn(0x560cbee88050 msgr2=0x560cbef62750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:23.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.427+0000 7f60ecc82b80 1 -- 172.21.15.151:0/1321420414 shutdown_connections 2024-04-26T02:03:23.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.427+0000 7f60ecc82b80 1 -- 172.21.15.151:0/1321420414 wait complete. 2024-04-26T02:03:23.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.427+0000 7f60ecc82b80 1 -- start start 2024-04-26T02:03:23.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.428+0000 7f60ecc82b80 1 Processor -- start 2024-04-26T02:03:23.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.428+0000 7f60ecc82b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560cbef58d40 0x560cbef6b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:23.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.428+0000 7f60ecc82b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cbef59660 0x560cbef6b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:23.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.428+0000 7f60ecc82b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560cbef644d0 0x560cbefda040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:23.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.428+0000 7f60ecc82b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f60dc1b1d90 con 0x560cbef644d0 2024-04-26T02:03:23.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.428+0000 7f60ecc82b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f60dc1b3200 con 0x560cbef59660 2024-04-26T02:03:23.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.428+0000 7f60ecc82b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f60dc1b2460 con 0x560cbef58d40 2024-04-26T02:03:23.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.428+0000 7f60ecc82b80 10 client.0 _ll_register_callbacks cb 0x560cbee44820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:03:23.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.428+0000 7f60dbe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560cbef644d0 0x560cbefda040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:23.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.428+0000 7f60daa00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cbef59660 0x560cbef6b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:23.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.428+0000 7f60db400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560cbef58d40 0x560cbef6b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:23.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.429+0000 7f60dbe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560cbef644d0 0x560cbefda040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37422/0 (socket says 172.21.15.151:37422) 2024-04-26T02:03:23.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.429+0000 7f60dbe00700 1 -- 172.21.15.151:0/3025263799 learned_addr learned my addr 172.21.15.151:0/3025263799 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:03:23.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.429+0000 7f60daa00700 1 -- 172.21.15.151:0/3025263799 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560cbef58d40 msgr2=0x560cbef6b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:23.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.429+0000 7f60daa00700 1 --2- 172.21.15.151:0/3025263799 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560cbef58d40 0x560cbef6b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.429+0000 7f60daa00700 1 -- 172.21.15.151:0/3025263799 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560cbef644d0 msgr2=0x560cbefda040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:23.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.429+0000 7f60daa00700 1 --2- 172.21.15.151:0/3025263799 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560cbef644d0 0x560cbefda040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.429+0000 7f60daa00700 1 -- 172.21.15.151:0/3025263799 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cbefda580 con 0x560cbef59660 2024-04-26T02:03:23.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.429+0000 7f60daa00700 1 --2- 172.21.15.151:0/3025263799 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cbef59660 0x560cbef6b8c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f60c80ce0b0 tx=0x7f60c8087040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c64cf793b4f36708 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:23.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.429+0000 7f60c5600700 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T02:03:23.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.429+0000 7f60c5600700 1 -- 172.21.15.151:0/3025263799 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c80872b0 con 0x560cbef59660 2024-04-26T02:03:23.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.429+0000 7f60ecc82b80 1 -- 172.21.15.151:0/3025263799 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60cc02a700 con 0x560cbef59660 2024-04-26T02:03:23.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.429+0000 7f60c5600700 1 -- 172.21.15.151:0/3025263799 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8455+0+0 (secure 0 0 0) 0x7f60c809cb60 con 0x560cbef59660 2024-04-26T02:03:23.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.430+0000 7f60c5600700 1 -- 172.21.15.151:0/3025263799 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f60c809c900 con 0x560cbef59660 2024-04-26T02:03:23.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.430+0000 7f60ecc82b80 10 client.26318 fetch_fsmap learned FSMap version 35 2024-04-26T02:03:23.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.430+0000 7f60ecc82b80 1 -- 172.21.15.151:0/3025263799 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x560cbefdb910 con 0x560cbef59660 2024-04-26T02:03:23.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.430+0000 7f60c5600700 1 -- 172.21.15.151:0/3025263799 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60cc02a700 con 0x560cbef59660 2024-04-26T02:03:23.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.430+0000 7f60c5600700 1 -- 172.21.15.151:0/3025263799 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 35) ==== 8338+0+0 (secure 0 0 0) 0x7f60c800c760 con 0x560cbef59660 2024-04-26T02:03:23.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.430+0000 7f60ecc82b80 10 client.26318 fetch_fsmap finished waiting for FSMap version 35 2024-04-26T02:03:23.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.430+0000 7f60ecc82b80 10 client.26318 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-26T02:03:23.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.430+0000 7f60ecc82b80 20 client.26318 populate_metadata read hostname 'smithi151' 2024-04-26T02:03:23.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.430+0000 7f60ecc82b80 1 --2- 172.21.15.151:0/3025263799 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x560cbefdbfd0 0x560cbefde3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:23.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.430+0000 7f60ecc82b80 4 client.26318 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "6face347", "release": true, "await": true}] 2024-04-26T02:03:23.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.430+0000 7f60ecc82b80 1 -- 172.21.15.151:0/3025263799 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "quiesce db", "set_id": "6face347", "release": true, "await": true}) -- 0x560cbeeb3820 con 0x560cbefdbfd0 2024-04-26T02:03:23.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.430+0000 7f60db400700 1 --2- 172.21.15.151:0/3025263799 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x560cbefdbfd0 0x560cbefde3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:23.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.431+0000 7f60c5600700 10 client.26318 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:03:23.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.431+0000 7f60db400700 1 --2- 172.21.15.151:0/3025263799 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x560cbefdbfd0 0x560cbefde3f0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54141ec1198f03a1 server_cookie=16a2654ba4c74331 in_seq=0 out_seq=0 2024-04-26T02:03:23.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.621+0000 7f60c5600700 1 -- 172.21.15.151:0/3025263799 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x560cbeeb3820 con 0x560cbefdbfd0 2024-04-26T02:03:23.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.621+0000 7f60c5600700 10 client.26318 handle_command_reply: tid=0 2024-04-26T02:03:23.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.621+0000 7f60ecc82b80 1 client.26318 shutdown 2024-04-26T02:03:23.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.621+0000 7f60ecc82b80 1 -- 172.21.15.151:0/3025263799 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cbef59660 msgr2=0x560cbef6b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:23.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.621+0000 7f60ecc82b80 1 --2- 172.21.15.151:0/3025263799 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cbef59660 0x560cbef6b8c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f60c80ce0b0 tx=0x7f60c8087040 comp rx=0 tx=0).stop 2024-04-26T02:03:23.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.622+0000 7f60ecc82b80 1 -- 172.21.15.151:0/3025263799 shutdown_connections 2024-04-26T02:03:23.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.622+0000 7f60ecc82b80 1 --2- 172.21.15.151:0/3025263799 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560cbef644d0 0x560cbefda040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.622+0000 7f60ecc82b80 1 --2- 172.21.15.151:0/3025263799 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560cbef58d40 0x560cbef6b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.622+0000 7f60ecc82b80 1 --2- 172.21.15.151:0/3025263799 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x560cbefdbfd0 0x560cbefde3f0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.622+0000 7f60ecc82b80 1 --2- 172.21.15.151:0/3025263799 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cbef59660 0x560cbef6b8c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.622+0000 7f60ecc82b80 1 -- 172.21.15.151:0/3025263799 >> 172.21.15.151:0/3025263799 conn(0x560cbee88050 msgr2=0x560cbef62fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:23.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.622+0000 7f60c5600700 0 client.26318 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:03:23.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.622+0000 7f60ecc82b80 1 -- 172.21.15.151:0/3025263799 shutdown_connections 2024-04-26T02:03:23.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.622+0000 7f60ecc82b80 1 -- 172.21.15.151:0/3025263799 wait complete. 2024-04-26T02:03:23.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.623+0000 7f60ecc82b80 20 client.26318 trim_cache size 0 max 16384 2024-04-26T02:03:23.622 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-26T02:03:23.622 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 35, 2024-04-26T02:03:23.622 INFO:teuthology.orchestra.run.smithi151.stdout: "leader": 24509, 2024-04-26T02:03:23.622 INFO:teuthology.orchestra.run.smithi151.stdout: "set_version": 28, 2024-04-26T02:03:23.622 INFO:teuthology.orchestra.run.smithi151.stdout: "sets": { 2024-04-26T02:03:23.622 INFO:teuthology.orchestra.run.smithi151.stdout: "6face347": { 2024-04-26T02:03:23.622 INFO:teuthology.orchestra.run.smithi151.stdout: "version": 28, 2024-04-26T02:03:23.622 INFO:teuthology.orchestra.run.smithi151.stdout: "age_ref": 0.0, 2024-04-26T02:03:23.623 INFO:teuthology.orchestra.run.smithi151.stdout: "state": { 2024-04-26T02:03:23.623 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "RELEASED", 2024-04-26T02:03:23.623 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 0.0 2024-04-26T02:03:23.623 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:03:23.623 INFO:teuthology.orchestra.run.smithi151.stdout: "timeout": 60.0, 2024-04-26T02:03:23.623 INFO:teuthology.orchestra.run.smithi151.stdout: "expiration": 82.1, 2024-04-26T02:03:23.623 INFO:teuthology.orchestra.run.smithi151.stdout: "members": { 2024-04-26T02:03:23.623 INFO:teuthology.orchestra.run.smithi151.stdout: "file:/": { 2024-04-26T02:03:23.623 INFO:teuthology.orchestra.run.smithi151.stdout: "excluded": false, 2024-04-26T02:03:23.623 INFO:teuthology.orchestra.run.smithi151.stdout: "state": { 2024-04-26T02:03:23.623 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "RELEASED", 2024-04-26T02:03:23.623 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 0.0 2024-04-26T02:03:23.623 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:03:23.623 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:03:23.624 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:03:23.624 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:03:23.624 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:03:23.624 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-26T02:03:23.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.624+0000 7f60e2200700 1 -- 172.21.15.151:0/706262076 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f60c00a3db0 msgr2=0x7f60c00a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:23.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.624+0000 7f60e2200700 1 --2- 172.21.15.151:0/706262076 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f60c00a3db0 0x7f60c00a6270 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f60d4009da0 tx=0x7f60d4003b80 comp rx=0 tx=0).stop 2024-04-26T02:03:23.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.624+0000 7f60e2200700 1 -- 172.21.15.151:0/706262076 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60dc109c80 msgr2=0x7f60dc19fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:23.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.624+0000 7f60e2200700 1 --2- 172.21.15.151:0/706262076 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60dc109c80 0x7f60dc19fc50 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f60c800e9f0 tx=0x7f60c800ed00 comp rx=0 tx=0).stop 2024-04-26T02:03:23.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.624+0000 7f60e2200700 1 -- 172.21.15.151:0/706262076 shutdown_connections 2024-04-26T02:03:23.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.624+0000 7f60e2200700 1 --2- 172.21.15.151:0/706262076 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f60c00a3db0 0x7f60c00a6270 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.624+0000 7f60e2200700 1 --2- 172.21.15.151:0/706262076 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60dc10e3b0 0x7f60dc1a3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.624+0000 7f60e2200700 1 --2- 172.21.15.151:0/706262076 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60dc1092d0 0x7f60dc19f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.624+0000 7f60e2200700 1 --2- 172.21.15.151:0/706262076 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60dc109c80 0x7f60dc19fc50 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:23.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.624+0000 7f60e2200700 1 -- 172.21.15.151:0/706262076 >> 172.21.15.151:0/706262076 conn(0x7f60dc076210 msgr2=0x7f60dc076730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:23.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.624+0000 7f60e2200700 1 -- 172.21.15.151:0/706262076 shutdown_connections 2024-04-26T02:03:23.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:23.624+0000 7f60e2200700 1 -- 172.21.15.151:0/706262076 wait complete. 2024-04-26T02:03:24.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:03:24 smithi172 ceph-mon[32785]: pgmap v307: 97 pgs: 97 active+clean; 112 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.3 KiB/s wr, 6 op/s 2024-04-26T02:03:24.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:03:24 smithi165 ceph-mon[32648]: pgmap v307: 97 pgs: 97 active+clean; 112 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.3 KiB/s wr, 6 op/s 2024-04-26T02:03:24.618 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:03:24 smithi151 ceph-mon[28057]: pgmap v307: 97 pgs: 97 active+clean; 112 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.3 KiB/s wr, 6 op/s 2024-04-26T02:03:24.672 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '6face347', seconds elapsed: 38.9 2024-04-26T02:03:24.672 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.1 2024-04-26T02:03:24.672 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T02:03:25.152 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:03:26.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:03:26 smithi172 ceph-mon[32785]: pgmap v308: 97 pgs: 97 active+clean; 112 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-04-26T02:03:26.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:03:26 smithi165 ceph-mon[32648]: pgmap v308: 97 pgs: 97 active+clean; 112 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-04-26T02:03:26.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:03:26 smithi151 ceph-mon[28057]: pgmap v308: 97 pgs: 97 active+clean; 112 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-04-26T02:03:27.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:27.498+0000 7fbefb000700 1 -- 172.21.15.151:0/1835875029 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbef406afe0 msgr2=0x7fbef406b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:27.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:27.498+0000 7fbefb000700 1 --2- 172.21.15.151:0/1835875029 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbef406afe0 0x7fbef406b440 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fbee0009990 tx=0x7fbee0009ca0 comp rx=0 tx=0).stop 2024-04-26T02:03:27.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:27.498+0000 7fbefb000700 1 -- 172.21.15.151:0/1835875029 shutdown_connections 2024-04-26T02:03:27.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:27.498+0000 7fbefb000700 1 --2- 172.21.15.151:0/1835875029 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbef406b980 0x7fbef410f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:27.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:27.498+0000 7fbefb000700 1 --2- 172.21.15.151:0/1835875029 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbef4108a20 0x7fbef4108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:27.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:27.498+0000 7fbefb000700 1 --2- 172.21.15.151:0/1835875029 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbef406afe0 0x7fbef406b440 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:27.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:27.498+0000 7fbefb000700 1 -- 172.21.15.151:0/1835875029 >> 172.21.15.151:0/1835875029 conn(0x7fbef40788b0 msgr2=0x7fbef4078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:27.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:27.498+0000 7fbefb000700 1 -- 172.21.15.151:0/1835875029 shutdown_connections 2024-04-26T02:03:27.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:27.498+0000 7fbefb000700 1 -- 172.21.15.151:0/1835875029 wait complete. 2024-04-26T02:03:27.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:27.499+0000 7fbefb000700 1 Processor -- start 2024-04-26T02:03:27.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:27.500+0000 7fbefb000700 1 -- start start 2024-04-26T02:03:27.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:27.500+0000 7fbefb000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbef406afe0 0x7fbef4101a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:27.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:27.500+0000 7fbefb000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbef406b980 0x7fbef4101f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:27.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:27.500+0000 7fbefb000700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbef4108a20 0x7fbef4102480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:27.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:27.500+0000 7fbefb000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbef4112be0 con 0x7fbef406b980 2024-04-26T02:03:27.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:27.500+0000 7fbefb000700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbef4112a60 con 0x7fbef406afe0 2024-04-26T02:03:27.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:27.500+0000 7fbefb000700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbef4112d60 con 0x7fbef4108a20 2024-04-26T02:03:27.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:27.501+0000 7fbef3e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbef406afe0 0x7fbef4101a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:27.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:27.501+0000 7fbef3e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbef406afe0 0x7fbef4101a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:43318/0 (socket says 172.21.15.151:43318) 2024-04-26T02:03:27.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:27.501+0000 7fbef3400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbef406b980 0x7fbef4101f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:27.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:27.501+0000 7fbef3e00700 1 -- 172.21.15.151:0/2657967632 learned_addr learned my addr 172.21.15.151:0/2657967632 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:03:27.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:27.501+0000 7fbef9000700 1 --2- 172.21.15.151:0/2657967632 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbef4108a20 0x7fbef4102480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:27.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:27.501+0000 7fbef3e00700 1 -- 172.21.15.151:0/2657967632 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbef4108a20 msgr2=0x7fbef4102480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:27.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:27.501+0000 7fbef3e00700 1 --2- 172.21.15.151:0/2657967632 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbef4108a20 0x7fbef4102480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:27.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:27.501+0000 7fbef3e00700 1 -- 172.21.15.151:0/2657967632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbef406b980 msgr2=0x7fbef4101f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:27.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:27.501+0000 7fbef3e00700 1 --2- 172.21.15.151:0/2657967632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbef406b980 0x7fbef4101f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:27.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:27.501+0000 7fbef3e00700 1 -- 172.21.15.151:0/2657967632 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbef41a75d0 con 0x7fbef406afe0 2024-04-26T02:03:27.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:27.502+0000 7fbef3e00700 1 --2- 172.21.15.151:0/2657967632 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbef406afe0 0x7fbef4101a00 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbee400e9a0 tx=0x7fbee400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:27.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:27.502+0000 7fbef0c00700 1 -- 172.21.15.151:0/2657967632 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee400ca10 con 0x7fbef406afe0 2024-04-26T02:03:27.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:27.502+0000 7fbef0c00700 1 -- 172.21.15.151:0/2657967632 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbee400cbb0 con 0x7fbef406afe0 2024-04-26T02:03:27.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:27.502+0000 7fbefb000700 1 -- 172.21.15.151:0/2657967632 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbef41a78c0 con 0x7fbef406afe0 2024-04-26T02:03:27.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:27.502+0000 7fbef0c00700 1 -- 172.21.15.151:0/2657967632 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee4018680 con 0x7fbef406afe0 2024-04-26T02:03:27.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:27.502+0000 7fbefb000700 1 -- 172.21.15.151:0/2657967632 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbef41a7e20 con 0x7fbef406afe0 2024-04-26T02:03:27.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:27.504+0000 7fbefb000700 1 -- 172.21.15.151:0/2657967632 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbef4069730 con 0x7fbef406afe0 2024-04-26T02:03:27.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:27.505+0000 7fbef0c00700 1 -- 172.21.15.151:0/2657967632 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7fbee4010430 con 0x7fbef406afe0 2024-04-26T02:03:27.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:27.506+0000 7fbef0c00700 1 --2- 172.21.15.151:0/2657967632 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fbed80a3e30 0x7fbed80a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:27.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:27.507+0000 7fbef0c00700 1 -- 172.21.15.151:0/2657967632 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8455+0+0 (secure 0 0 0) 0x7fbee40d0900 con 0x7fbef406afe0 2024-04-26T02:03:27.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:27.507+0000 7fbef3400700 1 --2- 172.21.15.151:0/2657967632 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fbed80a3e30 0x7fbed80a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:27.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:27.507+0000 7fbef3400700 1 --2- 172.21.15.151:0/2657967632 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fbed80a3e30 0x7fbed80a62f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbee00098e0 tx=0x7fbee0005960 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:27.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:27.510+0000 7fbef0c00700 1 -- 172.21.15.151:0/2657967632 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbee409cf80 con 0x7fbef406afe0 2024-04-26T02:03:27.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:27.845+0000 7fbefb000700 1 -- 172.21.15.151:0/2657967632 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbef402df80 con 0x7fbef406afe0 2024-04-26T02:03:27.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:27.846+0000 7fbef0c00700 1 -- 172.21.15.151:0/2657967632 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11889 (secure 0 0 0) 0x7fbee409c880 con 0x7fbef406afe0 2024-04-26T02:03:27.845 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T02:03:27.845 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2937864097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2937864097},{"type":"v1","addr":"172.21.15.151:6835","nonce":2937864097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14373,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/2947090802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2947090802},{"type":"v1","addr":"172.21.15.151:6837","nonce":2947090802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/1356563280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1356563280},{"type":"v1","addr":"172.21.15.151:6839","nonce":1356563280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6841/1075240552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1075240552},{"type":"v1","addr":"172.21.15.151:6841","nonce":1075240552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/504675912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":504675912},{"type":"v1","addr":"172.21.15.165:6833","nonce":504675912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1207580015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1207580015},{"type":"v1","addr":"172.21.15.172:6833","nonce":1207580015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:49:40.746876+0000","modified":"2024-04-26T02:00:05.413774+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24505,"mds_2":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/3598138294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3598138294},{"type":"v1","addr":"172.21.15.165:6835","nonce":3598138294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/1847859054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":1847859054},{"type":"v1","addr":"172.21.15.165:6837","nonce":1847859054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.172:6837/55972537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":55972537},{"type":"v1","addr":"172.21.15.172:6837","nonce":55972537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.172:6839/1242302810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1242302810},{"type":"v1","addr":"172.21.15.172:6839","nonce":1242302810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/1969716254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1969716254},{"type":"v1","addr":"172.21.15.165:6839","nonce":1969716254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25954":{"gid":25954,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/3656630171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3656630171},{"type":"v1","addr":"172.21.15.172:6835","nonce":3656630171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24493,24505,24509]},"id":1}]} 2024-04-26T02:03:27.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:27.849+0000 7fbefb000700 1 -- 172.21.15.151:0/2657967632 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fbed80a3e30 msgr2=0x7fbed80a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:27.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:27.849+0000 7fbefb000700 1 --2- 172.21.15.151:0/2657967632 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fbed80a3e30 0x7fbed80a62f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbee00098e0 tx=0x7fbee0005960 comp rx=0 tx=0).stop 2024-04-26T02:03:27.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:27.849+0000 7fbefb000700 1 -- 172.21.15.151:0/2657967632 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbef406afe0 msgr2=0x7fbef4101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:27.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:27.849+0000 7fbefb000700 1 --2- 172.21.15.151:0/2657967632 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbef406afe0 0x7fbef4101a00 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbee400e9a0 tx=0x7fbee400ecb0 comp rx=0 tx=0).stop 2024-04-26T02:03:27.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:27.849+0000 7fbefb000700 1 -- 172.21.15.151:0/2657967632 shutdown_connections 2024-04-26T02:03:27.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:27.849+0000 7fbefb000700 1 --2- 172.21.15.151:0/2657967632 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fbed80a3e30 0x7fbed80a62f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:27.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:27.849+0000 7fbefb000700 1 --2- 172.21.15.151:0/2657967632 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbef4108a20 0x7fbef4102480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:27.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:27.849+0000 7fbefb000700 1 --2- 172.21.15.151:0/2657967632 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbef406afe0 0x7fbef4101a00 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:27.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:27.849+0000 7fbefb000700 1 --2- 172.21.15.151:0/2657967632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbef406b980 0x7fbef4101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:27.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:27.849+0000 7fbefb000700 1 -- 172.21.15.151:0/2657967632 >> 172.21.15.151:0/2657967632 conn(0x7fbef40788b0 msgr2=0x7fbef40ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:27.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:27.849+0000 7fbefb000700 1 -- 172.21.15.151:0/2657967632 shutdown_connections 2024-04-26T02:03:27.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:27.849+0000 7fbefb000700 1 -- 172.21.15.151:0/2657967632 wait complete. 2024-04-26T02:03:27.850 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 35 2024-04-26T02:03:28.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:03:28 smithi172 ceph-mon[32785]: pgmap v309: 97 pgs: 97 active+clean; 112 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-04-26T02:03:28.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:03:28 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/2657967632' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:03:28.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:03:28 smithi165 ceph-mon[32648]: pgmap v309: 97 pgs: 97 active+clean; 112 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-04-26T02:03:28.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:03:28 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/2657967632' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:03:28.612 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:03:28 smithi151 ceph-mon[28057]: pgmap v309: 97 pgs: 97 active+clean; 112 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-04-26T02:03:28.612 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:03:28 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/2657967632' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:03:28.668 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24509 quiesce db / --timeout 60 --expiration 82.1 --await-for 42.0' 2024-04-26T02:03:28.668 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24509 quiesce db / --timeout 60 --expiration 82.1 --await-for 42.0 2024-04-26T02:03:29.004 INFO:tasks.workunit.client.0.smithi151.stdout:copying to largest ever: 0x63ef9ee 2024-04-26T02:03:29.158 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:03:30.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:03:30 smithi172 ceph-mon[32785]: pgmap v310: 97 pgs: 97 active+clean; 124 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 577 KiB/s wr, 204 op/s 2024-04-26T02:03:30.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:03:30 smithi165 ceph-mon[32648]: pgmap v310: 97 pgs: 97 active+clean; 124 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 577 KiB/s wr, 204 op/s 2024-04-26T02:03:30.613 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:03:30 smithi151 ceph-mon[28057]: pgmap v310: 97 pgs: 97 active+clean; 124 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 577 KiB/s wr, 204 op/s 2024-04-26T02:03:31.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.464+0000 7f8f68800700 1 -- 172.21.15.151:0/445060005 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8f600751b0 msgr2=0x7f8f60075630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:31.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.464+0000 7f8f68800700 1 --2- 172.21.15.151:0/445060005 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8f600751b0 0x7f8f60075630 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8f50009990 tx=0x7f8f50009ca0 comp rx=0 tx=0).stop 2024-04-26T02:03:31.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.464+0000 7f8f68800700 1 -- 172.21.15.151:0/445060005 shutdown_connections 2024-04-26T02:03:31.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.464+0000 7f8f68800700 1 --2- 172.21.15.151:0/445060005 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f60113730 0x7f8f60115b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:31.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.464+0000 7f8f68800700 1 --2- 172.21.15.151:0/445060005 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f60076a60 0x7f8f60074c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:31.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.464+0000 7f8f68800700 1 --2- 172.21.15.151:0/445060005 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8f600751b0 0x7f8f60075630 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:31.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.464+0000 7f8f68800700 1 -- 172.21.15.151:0/445060005 >> 172.21.15.151:0/445060005 conn(0x7f8f60079b10 msgr2=0x7f8f60079f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:31.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.464+0000 7f8f68800700 1 -- 172.21.15.151:0/445060005 shutdown_connections 2024-04-26T02:03:31.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.464+0000 7f8f68800700 1 -- 172.21.15.151:0/445060005 wait complete. 2024-04-26T02:03:31.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.465+0000 7f8f68800700 1 Processor -- start 2024-04-26T02:03:31.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.465+0000 7f8f68800700 1 -- start start 2024-04-26T02:03:31.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.466+0000 7f8f68800700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f600751b0 0x7f8f6019f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:31.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.466+0000 7f8f68800700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8f60076a60 0x7f8f6019fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:31.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.466+0000 7f8f68800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f60113730 0x7f8f601a3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:31.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.466+0000 7f8f68800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8f60119230 con 0x7f8f60113730 2024-04-26T02:03:31.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.466+0000 7f8f68800700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8f601190b0 con 0x7f8f60076a60 2024-04-26T02:03:31.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.466+0000 7f8f68800700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8f601193b0 con 0x7f8f600751b0 2024-04-26T02:03:31.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.466+0000 7f8f65e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f600751b0 0x7f8f6019f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:31.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.466+0000 7f8f65400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8f60076a60 0x7f8f6019fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:31.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.466+0000 7f8f65e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f600751b0 0x7f8f6019f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:38008/0 (socket says 172.21.15.151:38008) 2024-04-26T02:03:31.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.466+0000 7f8f65e00700 1 -- 172.21.15.151:0/1466001261 learned_addr learned my addr 172.21.15.151:0/1466001261 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:03:31.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.466+0000 7f8f66800700 1 --2- 172.21.15.151:0/1466001261 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f60113730 0x7f8f601a3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:31.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.466+0000 7f8f65400700 1 -- 172.21.15.151:0/1466001261 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f600751b0 msgr2=0x7f8f6019f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:31.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.466+0000 7f8f65400700 1 --2- 172.21.15.151:0/1466001261 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f600751b0 0x7f8f6019f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:31.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.466+0000 7f8f65400700 1 -- 172.21.15.151:0/1466001261 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f60113730 msgr2=0x7f8f601a3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:31.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.466+0000 7f8f65400700 1 --2- 172.21.15.151:0/1466001261 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f60113730 0x7f8f601a3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:31.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.467+0000 7f8f65400700 1 -- 172.21.15.151:0/1466001261 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f601a4670 con 0x7f8f60076a60 2024-04-26T02:03:31.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.467+0000 7f8f65e00700 1 --2- 172.21.15.151:0/1466001261 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f600751b0 0x7f8f6019f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:03:31.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.467+0000 7f8f65400700 1 --2- 172.21.15.151:0/1466001261 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8f60076a60 0x7f8f6019fba0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8f50006010 tx=0x7f8f500057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:31.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.467+0000 7f8f5aa00700 1 -- 172.21.15.151:0/1466001261 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f50027070 con 0x7f8f60076a60 2024-04-26T02:03:31.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.467+0000 7f8f5aa00700 1 -- 172.21.15.151:0/1466001261 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f50022470 con 0x7f8f60076a60 2024-04-26T02:03:31.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.467+0000 7f8f5aa00700 1 -- 172.21.15.151:0/1466001261 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f50017650 con 0x7f8f60076a60 2024-04-26T02:03:31.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.467+0000 7f8f68800700 1 -- 172.21.15.151:0/1466001261 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f601a4900 con 0x7f8f60076a60 2024-04-26T02:03:31.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.467+0000 7f8f68800700 1 -- 172.21.15.151:0/1466001261 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f601ac1e0 con 0x7f8f60076a60 2024-04-26T02:03:31.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.471+0000 7f8f5aa00700 1 -- 172.21.15.151:0/1466001261 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f8f5002a070 con 0x7f8f60076a60 2024-04-26T02:03:31.471 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.472+0000 7f8f5aa00700 1 --2- 172.21.15.151:0/1466001261 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f8f480a3ed0 0x7f8f480a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:31.471 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.472+0000 7f8f5aa00700 1 -- 172.21.15.151:0/1466001261 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8455+0+0 (secure 0 0 0) 0x7f8f500d1380 con 0x7f8f60076a60 2024-04-26T02:03:31.471 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.472+0000 7f8f65e00700 1 --2- 172.21.15.151:0/1466001261 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f8f480a3ed0 0x7f8f480a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:31.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.473+0000 7f8f730d8b80 1 Processor -- start 2024-04-26T02:03:31.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.473+0000 7f8f730d8b80 1 -- start start 2024-04-26T02:03:31.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.473+0000 7f8f65e00700 1 --2- 172.21.15.151:0/1466001261 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f8f480a3ed0 0x7f8f480a6390 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8f54011630 tx=0x7f8f54009260 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:31.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.473+0000 7f8f730d8b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d55d5e09b0 0x55d55d5e0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:31.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.473+0000 7f8f730d8b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d55d5e12d0 0x55d55d5ebc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:31.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.474+0000 7f8f730d8b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d55d5ec180 0x55d55d5ee570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:31.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.474+0000 7f8f730d8b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d55d534020 con 0x55d55d5e09b0 2024-04-26T02:03:31.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.474+0000 7f8f730d8b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d55d4da8c0 con 0x55d55d5e12d0 2024-04-26T02:03:31.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.474+0000 7f8f730d8b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d55d52ffe0 con 0x55d55d5ec180 2024-04-26T02:03:31.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.474+0000 7f8f65400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d55d5e12d0 0x55d55d5ebc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:31.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.474+0000 7f8f65400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d55d5e12d0 0x55d55d5ebc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:44302/0 (socket says 172.21.15.151:44302) 2024-04-26T02:03:31.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.474+0000 7f8f65400700 1 -- 172.21.15.151:0/3073204047 learned_addr learned my addr 172.21.15.151:0/3073204047 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:03:31.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.474+0000 7f8f65e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d55d5e09b0 0x55d55d5e0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:31.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.474+0000 7f8f66800700 1 --2- 172.21.15.151:0/3073204047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d55d5ec180 0x55d55d5ee570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:31.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.474+0000 7f8f65400700 1 -- 172.21.15.151:0/3073204047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d55d5ec180 msgr2=0x55d55d5ee570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:31.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.474+0000 7f8f65400700 1 --2- 172.21.15.151:0/3073204047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d55d5ec180 0x55d55d5ee570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:31.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.474+0000 7f8f65400700 1 -- 172.21.15.151:0/3073204047 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d55d5e09b0 msgr2=0x55d55d5e0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:31.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.474+0000 7f8f65400700 1 --2- 172.21.15.151:0/3073204047 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d55d5e09b0 0x55d55d5e0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:31.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.474+0000 7f8f65400700 1 -- 172.21.15.151:0/3073204047 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d55d4d0810 con 0x55d55d5e12d0 2024-04-26T02:03:31.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.475+0000 7f8f65400700 1 --2- 172.21.15.151:0/3073204047 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d55d5e12d0 0x55d55d5ebc40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8f500a5a40 tx=0x7f8f5009cfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef710a4e0ef6ac4a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:31.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.475+0000 7f8f4f400700 1 -- 172.21.15.151:0/3073204047 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5009d390 con 0x55d55d5e12d0 2024-04-26T02:03:31.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.475+0000 7f8f4f400700 1 -- 172.21.15.151:0/3073204047 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f500a1740 con 0x55d55d5e12d0 2024-04-26T02:03:31.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.475+0000 7f8f4f400700 1 -- 172.21.15.151:0/3073204047 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5009d5f0 con 0x55d55d5e12d0 2024-04-26T02:03:31.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.476+0000 7f8f730d8b80 1 -- 172.21.15.151:0/3073204047 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d55d5e12d0 msgr2=0x55d55d5ebc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:31.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.476+0000 7f8f730d8b80 1 --2- 172.21.15.151:0/3073204047 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d55d5e12d0 0x55d55d5ebc40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8f500a5a40 tx=0x7f8f5009cfa0 comp rx=0 tx=0).stop 2024-04-26T02:03:31.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.476+0000 7f8f730d8b80 1 -- 172.21.15.151:0/3073204047 shutdown_connections 2024-04-26T02:03:31.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.476+0000 7f8f730d8b80 1 --2- 172.21.15.151:0/3073204047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d55d5ec180 0x55d55d5ee570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:31.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.476+0000 7f8f730d8b80 1 --2- 172.21.15.151:0/3073204047 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d55d5e09b0 0x55d55d5e0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:31.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.476+0000 7f8f730d8b80 1 --2- 172.21.15.151:0/3073204047 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d55d5e12d0 0x55d55d5ebc40 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:31.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.476+0000 7f8f730d8b80 1 -- 172.21.15.151:0/3073204047 >> 172.21.15.151:0/3073204047 conn(0x55d55d50f730 msgr2=0x55d55d5dce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:31.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.476+0000 7f8f730d8b80 1 -- 172.21.15.151:0/3073204047 shutdown_connections 2024-04-26T02:03:31.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.476+0000 7f8f730d8b80 1 -- 172.21.15.151:0/3073204047 wait complete. 2024-04-26T02:03:31.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.477+0000 7f8f730d8b80 1 -- start start 2024-04-26T02:03:31.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.477+0000 7f8f730d8b80 1 Processor -- start 2024-04-26T02:03:31.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.477+0000 7f8f730d8b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d55d5e09b0 0x55d55d5eb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:31.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.477+0000 7f8f730d8b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d55d5e12d0 0x55d55d5e8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:31.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.477+0000 7f8f730d8b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d55d5ec180 0x55d55d5e91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:31.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.477+0000 7f8f730d8b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d55d5eeab0 con 0x55d55d5e12d0 2024-04-26T02:03:31.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.477+0000 7f8f730d8b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d55d4ed9f0 con 0x55d55d5ec180 2024-04-26T02:03:31.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.477+0000 7f8f730d8b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d55d5eec30 con 0x55d55d5e09b0 2024-04-26T02:03:31.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.478+0000 7f8f730d8b80 10 client.0 _ll_register_callbacks cb 0x55d55d4cc8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:03:31.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.478+0000 7f8f65400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d55d5e12d0 0x55d55d5e8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:31.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.478+0000 7f8f66800700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d55d5ec180 0x55d55d5e91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:31.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.478+0000 7f8f65400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d55d5e12d0 0x55d55d5e8c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:48138/0 (socket says 172.21.15.151:48138) 2024-04-26T02:03:31.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.478+0000 7f8f65400700 1 -- 172.21.15.151:0/2275629285 learned_addr learned my addr 172.21.15.151:0/2275629285 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:03:31.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.478+0000 7f8f65e00700 1 --2- 172.21.15.151:0/2275629285 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d55d5e09b0 0x55d55d5eb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:31.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.478+0000 7f8f65400700 1 -- 172.21.15.151:0/2275629285 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d55d5e09b0 msgr2=0x55d55d5eb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:31.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.478+0000 7f8f65400700 1 --2- 172.21.15.151:0/2275629285 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d55d5e09b0 0x55d55d5eb8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:31.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.478+0000 7f8f65400700 1 -- 172.21.15.151:0/2275629285 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d55d5ec180 msgr2=0x55d55d5e91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:31.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.478+0000 7f8f65400700 1 --2- 172.21.15.151:0/2275629285 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d55d5ec180 0x55d55d5e91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:31.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.478+0000 7f8f65400700 1 -- 172.21.15.151:0/2275629285 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d55d481a60 con 0x55d55d5e12d0 2024-04-26T02:03:31.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.478+0000 7f8f66800700 1 --2- 172.21.15.151:0/2275629285 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d55d5ec180 0x55d55d5e91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:03:31.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.479+0000 7f8f65400700 1 --2- 172.21.15.151:0/2275629285 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d55d5e12d0 0x55d55d5e8c60 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f8f50099070 tx=0x7f8f50022b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6f55192abab3a7a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:31.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.479+0000 7f8f43e00700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-04-26T02:03:31.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.479+0000 7f8f43e00700 1 -- 172.21.15.151:0/2275629285 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f500b5070 con 0x55d55d5e12d0 2024-04-26T02:03:31.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.479+0000 7f8f43e00700 1 -- 172.21.15.151:0/2275629285 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8455+0+0 (secure 0 0 0) 0x7f8f5009d800 con 0x55d55d5e12d0 2024-04-26T02:03:31.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.479+0000 7f8f730d8b80 1 -- 172.21.15.151:0/2275629285 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f500a1740 con 0x55d55d5e12d0 2024-04-26T02:03:31.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.479+0000 7f8f43e00700 1 -- 172.21.15.151:0/2275629285 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8f500a5f70 con 0x55d55d5e12d0 2024-04-26T02:03:31.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.479+0000 7f8f730d8b80 10 client.16236 fetch_fsmap learned FSMap version 35 2024-04-26T02:03:31.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.479+0000 7f8f730d8b80 1 -- 172.21.15.151:0/2275629285 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d55d4cac80 con 0x55d55d5e12d0 2024-04-26T02:03:31.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.480+0000 7f8f43e00700 1 -- 172.21.15.151:0/2275629285 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f500a1740 con 0x55d55d5e12d0 2024-04-26T02:03:31.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.480+0000 7f8f43e00700 1 -- 172.21.15.151:0/2275629285 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 35) ==== 8338+0+0 (secure 0 0 0) 0x7f8f5009c7e0 con 0x55d55d5e12d0 2024-04-26T02:03:31.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.480+0000 7f8f730d8b80 10 client.16236 fetch_fsmap finished waiting for FSMap version 35 2024-04-26T02:03:31.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.480+0000 7f8f730d8b80 10 client.16236 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-26T02:03:31.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.480+0000 7f8f730d8b80 20 client.16236 populate_metadata read hostname 'smithi151' 2024-04-26T02:03:31.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.480+0000 7f8f730d8b80 1 --2- 172.21.15.151:0/2275629285 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55d55d5fb5e0 0x55d55d5fd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:31.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.480+0000 7f8f730d8b80 4 client.16236 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:03:31.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.480+0000 7f8f730d8b80 1 -- 172.21.15.151:0/2275629285 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d55d541810 con 0x55d55d5fb5e0 2024-04-26T02:03:31.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.481+0000 7f8f65e00700 1 --2- 172.21.15.151:0/2275629285 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55d55d5fb5e0 0x55d55d5fd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:31.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.481+0000 7f8f43e00700 10 client.16236 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:03:31.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.481+0000 7f8f65e00700 1 --2- 172.21.15.151:0/2275629285 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55d55d5fb5e0 0x55d55d5fd9d0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c519eae73cd95d4 server_cookie=6253422506ebfc8 in_seq=0 out_seq=0 2024-04-26T02:03:31.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.482+0000 7f8f43e00700 1 -- 172.21.15.151:0/2275629285 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d55d541810 con 0x55d55d5fb5e0 2024-04-26T02:03:31.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.482+0000 7f8f43e00700 10 client.16236 handle_command_reply: tid=0 2024-04-26T02:03:31.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.482+0000 7f8f730d8b80 1 client.16236 shutdown 2024-04-26T02:03:31.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.482+0000 7f8f730d8b80 1 -- 172.21.15.151:0/2275629285 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d55d5e12d0 msgr2=0x55d55d5e8c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:31.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.482+0000 7f8f730d8b80 1 --2- 172.21.15.151:0/2275629285 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d55d5e12d0 0x55d55d5e8c60 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f8f50099070 tx=0x7f8f50022b50 comp rx=0 tx=0).stop 2024-04-26T02:03:31.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.482+0000 7f8f730d8b80 1 -- 172.21.15.151:0/2275629285 shutdown_connections 2024-04-26T02:03:31.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.482+0000 7f8f730d8b80 1 --2- 172.21.15.151:0/2275629285 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55d55d5fb5e0 0x55d55d5fd9d0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:31.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.483+0000 7f8f730d8b80 1 --2- 172.21.15.151:0/2275629285 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d55d5ec180 0x55d55d5e91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:31.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.483+0000 7f8f730d8b80 1 --2- 172.21.15.151:0/2275629285 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d55d5e09b0 0x55d55d5eb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:31.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.483+0000 7f8f730d8b80 1 --2- 172.21.15.151:0/2275629285 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d55d5e12d0 0x55d55d5e8c60 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:31.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.483+0000 7f8f43e00700 0 client.16236 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:03:31.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.483+0000 7f8f730d8b80 1 -- 172.21.15.151:0/2275629285 >> 172.21.15.151:0/2275629285 conn(0x55d55d50f730 msgr2=0x55d55d5dca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:31.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.483+0000 7f8f730d8b80 1 -- 172.21.15.151:0/2275629285 shutdown_connections 2024-04-26T02:03:31.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.483+0000 7f8f730d8b80 1 -- 172.21.15.151:0/2275629285 wait complete. 2024-04-26T02:03:31.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.483+0000 7f8f730d8b80 20 client.16236 trim_cache size 0 max 16384 2024-04-26T02:03:31.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.509+0000 7f8f730d8b80 1 Processor -- start 2024-04-26T02:03:31.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.509+0000 7f8f730d8b80 1 -- start start 2024-04-26T02:03:31.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.510+0000 7f8f730d8b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d55d5e09b0 0x55d55d5e90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:31.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.510+0000 7f8f730d8b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d55d5e12d0 0x55d55d603060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:31.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.510+0000 7f8f730d8b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d55d5ec180 0x55d55d6035a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:31.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.510+0000 7f8f730d8b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8f601b6030 con 0x55d55d5ec180 2024-04-26T02:03:31.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.510+0000 7f8f730d8b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8f601b38c0 con 0x55d55d5e09b0 2024-04-26T02:03:31.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.510+0000 7f8f730d8b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8f601b1730 con 0x55d55d5e12d0 2024-04-26T02:03:31.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.510+0000 7f8f65e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d55d5e09b0 0x55d55d5e90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:31.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.510+0000 7f8f65e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d55d5e09b0 0x55d55d5e90b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:44334/0 (socket says 172.21.15.151:44334) 2024-04-26T02:03:31.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.510+0000 7f8f66800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d55d5ec180 0x55d55d6035a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:31.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.510+0000 7f8f65e00700 1 -- 172.21.15.151:0/3864699282 learned_addr learned my addr 172.21.15.151:0/3864699282 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:03:31.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.510+0000 7f8f65400700 1 --2- 172.21.15.151:0/3864699282 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d55d5e12d0 0x55d55d603060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:31.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.510+0000 7f8f65e00700 1 -- 172.21.15.151:0/3864699282 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d55d5e12d0 msgr2=0x55d55d603060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:31.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.510+0000 7f8f65e00700 1 --2- 172.21.15.151:0/3864699282 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d55d5e12d0 0x55d55d603060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:31.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.510+0000 7f8f65e00700 1 -- 172.21.15.151:0/3864699282 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d55d5ec180 msgr2=0x55d55d6035a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:31.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.510+0000 7f8f65e00700 1 --2- 172.21.15.151:0/3864699282 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d55d5ec180 0x55d55d6035a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:31.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.510+0000 7f8f65e00700 1 -- 172.21.15.151:0/3864699282 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d55d5ebc30 con 0x55d55d5e09b0 2024-04-26T02:03:31.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.511+0000 7f8f65e00700 1 --2- 172.21.15.151:0/3864699282 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d55d5e09b0 0x55d55d5e90b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8f54013010 tx=0x7f8f54011f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=534fdd7c736e178b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:31.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.511+0000 7f8f4ea00700 1 -- 172.21.15.151:0/3864699282 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5401dbb0 con 0x55d55d5e09b0 2024-04-26T02:03:31.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.511+0000 7f8f4ea00700 1 -- 172.21.15.151:0/3864699282 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f540044e0 con 0x55d55d5e09b0 2024-04-26T02:03:31.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.511+0000 7f8f4ea00700 1 -- 172.21.15.151:0/3864699282 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5402c080 con 0x55d55d5e09b0 2024-04-26T02:03:31.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.512+0000 7f8f730d8b80 1 -- 172.21.15.151:0/3864699282 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d55d5e09b0 msgr2=0x55d55d5e90b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:31.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.512+0000 7f8f730d8b80 1 --2- 172.21.15.151:0/3864699282 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d55d5e09b0 0x55d55d5e90b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8f54013010 tx=0x7f8f54011f90 comp rx=0 tx=0).stop 2024-04-26T02:03:31.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.512+0000 7f8f730d8b80 1 -- 172.21.15.151:0/3864699282 shutdown_connections 2024-04-26T02:03:31.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.512+0000 7f8f730d8b80 1 --2- 172.21.15.151:0/3864699282 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d55d5ec180 0x55d55d6035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:31.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.512+0000 7f8f730d8b80 1 --2- 172.21.15.151:0/3864699282 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d55d5e09b0 0x55d55d5e90b0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:31.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.512+0000 7f8f730d8b80 1 --2- 172.21.15.151:0/3864699282 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d55d5e12d0 0x55d55d603060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:31.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.512+0000 7f8f730d8b80 1 -- 172.21.15.151:0/3864699282 >> 172.21.15.151:0/3864699282 conn(0x55d55d50f730 msgr2=0x55d55d5e7cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:31.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.512+0000 7f8f730d8b80 1 -- 172.21.15.151:0/3864699282 shutdown_connections 2024-04-26T02:03:31.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.512+0000 7f8f730d8b80 1 -- 172.21.15.151:0/3864699282 wait complete. 2024-04-26T02:03:31.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.513+0000 7f8f730d8b80 1 -- start start 2024-04-26T02:03:31.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.513+0000 7f8f730d8b80 1 Processor -- start 2024-04-26T02:03:31.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.513+0000 7f8f730d8b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d55d5e09b0 0x55d55d5f3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:31.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.513+0000 7f8f730d8b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d55d5e12d0 0x55d55d5f4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:31.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.513+0000 7f8f730d8b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d55d5ec180 0x55d55d5f2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:31.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.513+0000 7f8f730d8b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8f601b1d80 con 0x55d55d5ec180 2024-04-26T02:03:31.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.513+0000 7f8f730d8b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8f601b31f0 con 0x55d55d5e09b0 2024-04-26T02:03:31.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.513+0000 7f8f730d8b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8f601b2450 con 0x55d55d5e12d0 2024-04-26T02:03:31.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.513+0000 7f8f730d8b80 10 client.0 _ll_register_callbacks cb 0x55d55d4cc8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:03:31.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.514+0000 7f8f65e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d55d5e09b0 0x55d55d5f3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:31.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.514+0000 7f8f65400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d55d5e12d0 0x55d55d5f4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:31.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.514+0000 7f8f65e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d55d5e09b0 0x55d55d5f3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:44336/0 (socket says 172.21.15.151:44336) 2024-04-26T02:03:31.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.514+0000 7f8f65400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d55d5e12d0 0x55d55d5f4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:38046/0 (socket says 172.21.15.151:38046) 2024-04-26T02:03:31.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.514+0000 7f8f65e00700 1 -- 172.21.15.151:0/1088188118 learned_addr learned my addr 172.21.15.151:0/1088188118 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:03:31.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.514+0000 7f8f66800700 1 --2- 172.21.15.151:0/1088188118 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d55d5ec180 0x55d55d5f2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:31.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.514+0000 7f8f65e00700 1 -- 172.21.15.151:0/1088188118 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d55d5e12d0 msgr2=0x55d55d5f4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:31.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.514+0000 7f8f65e00700 1 --2- 172.21.15.151:0/1088188118 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d55d5e12d0 0x55d55d5f4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:31.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.514+0000 7f8f65e00700 1 -- 172.21.15.151:0/1088188118 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d55d5ec180 msgr2=0x55d55d5f2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:31.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.514+0000 7f8f65e00700 1 --2- 172.21.15.151:0/1088188118 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d55d5ec180 0x55d55d5f2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:31.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.514+0000 7f8f65e00700 1 -- 172.21.15.151:0/1088188118 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d55d5f27d0 con 0x55d55d5e09b0 2024-04-26T02:03:31.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.515+0000 7f8f65e00700 1 --2- 172.21.15.151:0/1088188118 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d55d5e09b0 0x55d55d5f3bc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8f54003780 tx=0x7f8f540037b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7237b26904670ac1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:31.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.515+0000 7f8f43e00700 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T02:03:31.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.515+0000 7f8f43e00700 1 -- 172.21.15.151:0/1088188118 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5401dc70 con 0x55d55d5e09b0 2024-04-26T02:03:31.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.515+0000 7f8f43e00700 1 -- 172.21.15.151:0/1088188118 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8455+0+0 (secure 0 0 0) 0x7f8f54009040 con 0x55d55d5e09b0 2024-04-26T02:03:31.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.515+0000 7f8f730d8b80 1 -- 172.21.15.151:0/1088188118 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f540044e0 con 0x55d55d5e09b0 2024-04-26T02:03:31.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.515+0000 7f8f43e00700 1 -- 172.21.15.151:0/1088188118 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8f5401de10 con 0x55d55d5e09b0 2024-04-26T02:03:31.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.515+0000 7f8f730d8b80 10 client.26342 fetch_fsmap learned FSMap version 35 2024-04-26T02:03:31.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.515+0000 7f8f730d8b80 1 -- 172.21.15.151:0/1088188118 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d55d6623e0 con 0x55d55d5e09b0 2024-04-26T02:03:31.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.516+0000 7f8f43e00700 1 -- 172.21.15.151:0/1088188118 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f540044e0 con 0x55d55d5e09b0 2024-04-26T02:03:31.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.516+0000 7f8f43e00700 1 -- 172.21.15.151:0/1088188118 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 35) ==== 8338+0+0 (secure 0 0 0) 0x7f8f54035070 con 0x55d55d5e09b0 2024-04-26T02:03:31.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.516+0000 7f8f730d8b80 10 client.26342 fetch_fsmap finished waiting for FSMap version 35 2024-04-26T02:03:31.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.516+0000 7f8f730d8b80 10 client.26342 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-26T02:03:31.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.516+0000 7f8f730d8b80 20 client.26342 populate_metadata read hostname 'smithi151' 2024-04-26T02:03:31.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.516+0000 7f8f730d8b80 1 --2- 172.21.15.151:0/1088188118 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55d55d662a80 0x55d55d664ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:31.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.516+0000 7f8f730d8b80 4 client.26342 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 82.1, "await_for": 42.0}] 2024-04-26T02:03:31.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.516+0000 7f8f730d8b80 1 -- 172.21.15.151:0/1088188118 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 82.1, "await_for": 42.0}) -- 0x55d55d53b8b0 con 0x55d55d662a80 2024-04-26T02:03:31.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.516+0000 7f8f65400700 1 --2- 172.21.15.151:0/1088188118 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55d55d662a80 0x55d55d664ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:31.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.517+0000 7f8f65400700 1 --2- 172.21.15.151:0/1088188118 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55d55d662a80 0x55d55d664ea0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14bcc41596e805a3 server_cookie=df09c9bdfb140031 in_seq=0 out_seq=0 2024-04-26T02:03:31.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:31.517+0000 7f8f43e00700 10 client.26342 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:03:32.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:32.530+0000 7f8f43e00700 1 -- 172.21.15.151:0/1088188118 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55d55d53b8b0 con 0x55d55d662a80 2024-04-26T02:03:32.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:32.530+0000 7f8f43e00700 10 client.26342 handle_command_reply: tid=0 2024-04-26T02:03:32.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:32.531+0000 7f8f730d8b80 1 client.26342 shutdown 2024-04-26T02:03:32.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:32.531+0000 7f8f730d8b80 1 -- 172.21.15.151:0/1088188118 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d55d5e09b0 msgr2=0x55d55d5f3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:32.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:32.531+0000 7f8f730d8b80 1 --2- 172.21.15.151:0/1088188118 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d55d5e09b0 0x55d55d5f3bc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8f54003780 tx=0x7f8f540037b0 comp rx=0 tx=0).stop 2024-04-26T02:03:32.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:32.531+0000 7f8f730d8b80 1 -- 172.21.15.151:0/1088188118 shutdown_connections 2024-04-26T02:03:32.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:32.531+0000 7f8f730d8b80 1 --2- 172.21.15.151:0/1088188118 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d55d5ec180 0x55d55d5f2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:32.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:32.531+0000 7f8f730d8b80 1 --2- 172.21.15.151:0/1088188118 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55d55d662a80 0x55d55d664ea0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:32.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:32.531+0000 7f8f730d8b80 1 --2- 172.21.15.151:0/1088188118 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d55d5e09b0 0x55d55d5f3bc0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:32.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:32.531+0000 7f8f730d8b80 1 --2- 172.21.15.151:0/1088188118 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d55d5e12d0 0x55d55d5f4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:32.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:32.531+0000 7f8f43e00700 0 client.26342 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:03:32.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:32.531+0000 7f8f730d8b80 1 -- 172.21.15.151:0/1088188118 >> 172.21.15.151:0/1088188118 conn(0x55d55d50f730 msgr2=0x55d55d5ea260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:32.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:32.531+0000 7f8f730d8b80 1 -- 172.21.15.151:0/1088188118 shutdown_connections 2024-04-26T02:03:32.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:32.532+0000 7f8f730d8b80 1 -- 172.21.15.151:0/1088188118 wait complete. 2024-04-26T02:03:32.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:32.532+0000 7f8f730d8b80 20 client.26342 trim_cache size 0 max 16384 2024-04-26T02:03:32.532 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-26T02:03:32.532 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 35, 2024-04-26T02:03:32.532 INFO:teuthology.orchestra.run.smithi151.stdout: "leader": 24509, 2024-04-26T02:03:32.532 INFO:teuthology.orchestra.run.smithi151.stdout: "set_version": 30, 2024-04-26T02:03:32.533 INFO:teuthology.orchestra.run.smithi151.stdout: "sets": { 2024-04-26T02:03:32.533 INFO:teuthology.orchestra.run.smithi151.stdout: "58991260": { 2024-04-26T02:03:32.533 INFO:teuthology.orchestra.run.smithi151.stdout: "version": 30, 2024-04-26T02:03:32.533 INFO:teuthology.orchestra.run.smithi151.stdout: "age_ref": 0.0, 2024-04-26T02:03:32.533 INFO:teuthology.orchestra.run.smithi151.stdout: "state": { 2024-04-26T02:03:32.533 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "QUIESCED", 2024-04-26T02:03:32.533 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 0.0 2024-04-26T02:03:32.533 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:03:32.533 INFO:teuthology.orchestra.run.smithi151.stdout: "timeout": 60.0, 2024-04-26T02:03:32.533 INFO:teuthology.orchestra.run.smithi151.stdout: "expiration": 82.1, 2024-04-26T02:03:32.533 INFO:teuthology.orchestra.run.smithi151.stdout: "members": { 2024-04-26T02:03:32.533 INFO:teuthology.orchestra.run.smithi151.stdout: "file:/": { 2024-04-26T02:03:32.533 INFO:teuthology.orchestra.run.smithi151.stdout: "excluded": false, 2024-04-26T02:03:32.534 INFO:teuthology.orchestra.run.smithi151.stdout: "state": { 2024-04-26T02:03:32.534 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "QUIESCED", 2024-04-26T02:03:32.534 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 0.0 2024-04-26T02:03:32.534 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:03:32.534 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:03:32.534 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:03:32.534 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:03:32.534 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:03:32.534 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-26T02:03:32.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:32.533+0000 7f8f68800700 1 -- 172.21.15.151:0/1466001261 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f8f480a3ed0 msgr2=0x7f8f480a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:32.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:32.533+0000 7f8f68800700 1 --2- 172.21.15.151:0/1466001261 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f8f480a3ed0 0x7f8f480a6390 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8f54011630 tx=0x7f8f54009260 comp rx=0 tx=0).stop 2024-04-26T02:03:32.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:32.533+0000 7f8f68800700 1 -- 172.21.15.151:0/1466001261 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8f60076a60 msgr2=0x7f8f6019fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:32.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:32.533+0000 7f8f68800700 1 --2- 172.21.15.151:0/1466001261 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8f60076a60 0x7f8f6019fba0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8f50006010 tx=0x7f8f500057c0 comp rx=0 tx=0).stop 2024-04-26T02:03:32.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:32.534+0000 7f8f68800700 1 -- 172.21.15.151:0/1466001261 shutdown_connections 2024-04-26T02:03:32.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:32.534+0000 7f8f68800700 1 --2- 172.21.15.151:0/1466001261 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f8f480a3ed0 0x7f8f480a6390 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:32.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:32.534+0000 7f8f68800700 1 --2- 172.21.15.151:0/1466001261 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f60113730 0x7f8f601a3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:32.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:32.534+0000 7f8f68800700 1 --2- 172.21.15.151:0/1466001261 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f600751b0 0x7f8f6019f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:32.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:32.534+0000 7f8f68800700 1 --2- 172.21.15.151:0/1466001261 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8f60076a60 0x7f8f6019fba0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:32.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:32.534+0000 7f8f68800700 1 -- 172.21.15.151:0/1466001261 >> 172.21.15.151:0/1466001261 conn(0x7f8f60079b10 msgr2=0x7f8f601123f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:32.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:32.534+0000 7f8f68800700 1 -- 172.21.15.151:0/1466001261 shutdown_connections 2024-04-26T02:03:32.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:32.534+0000 7f8f68800700 1 -- 172.21.15.151:0/1466001261 wait complete. 2024-04-26T02:03:32.557 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:03:32 smithi151 ceph-mon[28057]: pgmap v311: 97 pgs: 97 active+clean; 124 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 494 KiB/s wr, 175 op/s 2024-04-26T02:03:32.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:03:32 smithi165 ceph-mon[32648]: pgmap v311: 97 pgs: 97 active+clean; 124 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 494 KiB/s wr, 175 op/s 2024-04-26T02:03:32.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:03:32 smithi172 ceph-mon[32785]: pgmap v311: 97 pgs: 97 active+clean; 124 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 494 KiB/s wr, 175 op/s 2024-04-26T02:03:33.379 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '58991260', quiesce took 8.7 seconds. Will release after: 22.1 2024-04-26T02:03:34.555 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:03:34 smithi151 ceph-mon[28057]: pgmap v312: 97 pgs: 97 active+clean; 143 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 808 KiB/s wr, 251 op/s 2024-04-26T02:03:34.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:03:34 smithi165 ceph-mon[32648]: pgmap v312: 97 pgs: 97 active+clean; 143 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 808 KiB/s wr, 251 op/s 2024-04-26T02:03:34.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:03:34 smithi172 ceph-mon[32785]: pgmap v312: 97 pgs: 97 active+clean; 143 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 808 KiB/s wr, 251 op/s 2024-04-26T02:03:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.235+0000 7f8a08000640 1 -- 172.21.15.151:0/2082765679 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a001043b0 msgr2=0x7f8a00104790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.235+0000 7f8a08000640 1 --2- 172.21.15.151:0/2082765679 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a001043b0 0x7f8a00104790 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f89f4009860 tx=0x7f89f402a340 comp rx=0 tx=0).stop 2024-04-26T02:03:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.235+0000 7f8a08000640 1 -- 172.21.15.151:0/2082765679 shutdown_connections 2024-04-26T02:03:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.235+0000 7f8a08000640 1 --2- 172.21.15.151:0/2082765679 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a00109480 0x7f8a00111050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.235+0000 7f8a08000640 1 --2- 172.21.15.151:0/2082765679 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a00104d60 0x7f8a00108da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.235+0000 7f8a08000640 1 --2- 172.21.15.151:0/2082765679 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a001043b0 0x7f8a00104790 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.235+0000 7f8a08000640 1 -- 172.21.15.151:0/2082765679 >> 172.21.15.151:0/2082765679 conn(0x7f8a000ffc50 msgr2=0x7f8a00102070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.236+0000 7f8a08000640 1 -- 172.21.15.151:0/2082765679 shutdown_connections 2024-04-26T02:03:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.236+0000 7f8a08000640 1 -- 172.21.15.151:0/2082765679 wait complete. 2024-04-26T02:03:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.236+0000 7f8a08000640 1 Processor -- start 2024-04-26T02:03:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.237+0000 7f8a08000640 1 -- start start 2024-04-26T02:03:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.237+0000 7f8a08000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a001043b0 0x7f8a001a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.237+0000 7f8a08000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a00109480 0x7f8a001a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.237+0000 7f8a08000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a0019a6e0 0x7f8a0019ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.237+0000 7f8a08000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8a001143c0 con 0x7f8a0019a6e0 2024-04-26T02:03:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.237+0000 7f8a08000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8a00114240 con 0x7f8a001043b0 2024-04-26T02:03:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.237+0000 7f8a08000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8a00114540 con 0x7f8a00109480 2024-04-26T02:03:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.237+0000 7f8a05600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a001043b0 0x7f8a001a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.237+0000 7f8a05600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a001043b0 0x7f8a001a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:44366/0 (socket says 172.21.15.151:44366) 2024-04-26T02:03:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.237+0000 7f8a05600640 1 -- 172.21.15.151:0/2966746367 learned_addr learned my addr 172.21.15.151:0/2966746367 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:03:36.238 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.237+0000 7f8a06000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a0019a6e0 0x7f8a0019ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:36.238 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.237+0000 7f8a04c00640 1 --2- 172.21.15.151:0/2966746367 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a00109480 0x7f8a001a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:36.238 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.238+0000 7f8a06000640 1 -- 172.21.15.151:0/2966746367 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a00109480 msgr2=0x7f8a001a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:36.238 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.238+0000 7f8a06000640 1 --2- 172.21.15.151:0/2966746367 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a00109480 0x7f8a001a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:36.238 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.238+0000 7f8a06000640 1 -- 172.21.15.151:0/2966746367 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a001043b0 msgr2=0x7f8a001a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:36.238 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.238+0000 7f8a06000640 1 --2- 172.21.15.151:0/2966746367 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a001043b0 0x7f8a001a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:36.238 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.238+0000 7f8a06000640 1 -- 172.21.15.151:0/2966746367 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a0019b2e0 con 0x7f8a0019a6e0 2024-04-26T02:03:36.238 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.238+0000 7f8a05600640 1 --2- 172.21.15.151:0/2966746367 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a001043b0 0x7f8a001a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:03:36.238 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.238+0000 7f8a06000640 1 --2- 172.21.15.151:0/2966746367 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a0019a6e0 0x7f8a0019ab60 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f89f8009fd0 tx=0x7f89f800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:36.239 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.238+0000 7f89fe000640 1 -- 172.21.15.151:0/2966746367 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f800ce20 con 0x7f8a0019a6e0 2024-04-26T02:03:36.239 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.238+0000 7f89fe000640 1 -- 172.21.15.151:0/2966746367 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89f8005020 con 0x7f8a0019a6e0 2024-04-26T02:03:36.239 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.238+0000 7f89fe000640 1 -- 172.21.15.151:0/2966746367 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f8010690 con 0x7f8a0019a6e0 2024-04-26T02:03:36.239 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.238+0000 7f8a08000640 1 -- 172.21.15.151:0/2966746367 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a001a73c0 con 0x7f8a0019a6e0 2024-04-26T02:03:36.239 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.239+0000 7f8a08000640 1 -- 172.21.15.151:0/2966746367 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a001a7770 con 0x7f8a0019a6e0 2024-04-26T02:03:36.241 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.242+0000 7f89fe000640 1 -- 172.21.15.151:0/2966746367 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f89f80054d0 con 0x7f8a0019a6e0 2024-04-26T02:03:36.241 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.242+0000 7f8a0ae2b740 1 Processor -- start 2024-04-26T02:03:36.241 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.242+0000 7f8a0ae2b740 1 -- start start 2024-04-26T02:03:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.242+0000 7f8a0ae2b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56075b7f4820 0x56075b7f4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.242+0000 7f8a0ae2b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56075b7f5140 0x56075b7ff4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.242+0000 7f8a0ae2b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56075b7ffa00 0x56075b801df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.242+0000 7f8a0ae2b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56075b75a900 con 0x56075b7f4820 2024-04-26T02:03:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.242+0000 7f8a0ae2b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56075b700ef0 con 0x56075b7ffa00 2024-04-26T02:03:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.242+0000 7f8a0ae2b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56075b7568c0 con 0x56075b7f5140 2024-04-26T02:03:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.243+0000 7f8a04c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56075b7f4820 0x56075b7f4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.243+0000 7f8a04c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56075b7f4820 0x56075b7f4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:48176/0 (socket says 172.21.15.151:48176) 2024-04-26T02:03:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.243+0000 7f8a04c00640 1 -- 172.21.15.151:0/2925969393 learned_addr learned my addr 172.21.15.151:0/2925969393 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:03:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.243+0000 7f8a05600640 1 --2- 172.21.15.151:0/2925969393 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56075b7f5140 0x56075b7ff4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.243+0000 7f8a04c00640 1 --2- 172.21.15.151:0/2925969393 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56075b7ffa00 0x56075b801df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.243+0000 7f8a05600640 1 -- 172.21.15.151:0/2925969393 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56075b7ffa00 msgr2=0x56075b801df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.243+0000 7f8a05600640 1 --2- 172.21.15.151:0/2925969393 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56075b7ffa00 0x56075b801df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.243+0000 7f8a05600640 1 -- 172.21.15.151:0/2925969393 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56075b7f4820 msgr2=0x56075b7f4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.243+0000 7f8a05600640 1 --2- 172.21.15.151:0/2925969393 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56075b7f4820 0x56075b7f4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.243+0000 7f8a05600640 1 -- 172.21.15.151:0/2925969393 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56075b46d210 con 0x56075b7f5140 2024-04-26T02:03:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.243+0000 7f89fe000640 1 --2- 172.21.15.151:0/2966746367 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f89e80a3dc0 0x7f89e80a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.243+0000 7f8a05600640 1 --2- 172.21.15.151:0/2925969393 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56075b7f5140 0x56075b7ff4c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f89f4009830 tx=0x7f89f4005b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22a19f5ab746556 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.244+0000 7f89fe000640 1 -- 172.21.15.151:0/2966746367 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8455+0+0 (secure 0 0 0) 0x7f89f80d0e60 con 0x7f8a0019a6e0 2024-04-26T02:03:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.244+0000 7f8a04c00640 1 --2- 172.21.15.151:0/2966746367 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f89e80a3dc0 0x7f89e80a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:36.251 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.244+0000 7f89f2600640 1 -- 172.21.15.151:0/2925969393 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f4037040 con 0x56075b7f5140 2024-04-26T02:03:36.251 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.244+0000 7f89f2600640 1 -- 172.21.15.151:0/2925969393 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89f4030450 con 0x56075b7f5140 2024-04-26T02:03:36.251 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.244+0000 7f89f2600640 1 -- 172.21.15.151:0/2925969393 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f4032a50 con 0x56075b7f5140 2024-04-26T02:03:36.251 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.245+0000 7f8a0ae2b740 1 -- 172.21.15.151:0/2925969393 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56075b7f5140 msgr2=0x56075b7ff4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:36.251 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.245+0000 7f8a04c00640 1 --2- 172.21.15.151:0/2966746367 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f89e80a3dc0 0x7f89e80a6280 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f89ec009fd0 tx=0x7f89ec009640 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:36.251 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.245+0000 7f8a0ae2b740 1 --2- 172.21.15.151:0/2925969393 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56075b7f5140 0x56075b7ff4c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f89f4009830 tx=0x7f89f4005b20 comp rx=0 tx=0).stop 2024-04-26T02:03:36.252 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.245+0000 7f8a0ae2b740 1 -- 172.21.15.151:0/2925969393 shutdown_connections 2024-04-26T02:03:36.252 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.245+0000 7f8a0ae2b740 1 --2- 172.21.15.151:0/2925969393 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56075b7ffa00 0x56075b801df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:36.252 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.245+0000 7f8a0ae2b740 1 --2- 172.21.15.151:0/2925969393 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56075b7f5140 0x56075b7ff4c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:36.252 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.245+0000 7f8a0ae2b740 1 --2- 172.21.15.151:0/2925969393 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56075b7f4820 0x56075b7f4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:36.252 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.245+0000 7f8a0ae2b740 1 -- 172.21.15.151:0/2925969393 >> 172.21.15.151:0/2925969393 conn(0x56075b7f0d30 msgr2=0x56075b7f1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:36.252 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.245+0000 7f8a0ae2b740 1 -- 172.21.15.151:0/2925969393 shutdown_connections 2024-04-26T02:03:36.252 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.245+0000 7f8a0ae2b740 1 -- 172.21.15.151:0/2925969393 wait complete. 2024-04-26T02:03:36.252 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.245+0000 7f8a0ae2b740 1 -- start start 2024-04-26T02:03:36.252 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.245+0000 7f8a0ae2b740 1 Processor -- start 2024-04-26T02:03:36.252 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.246+0000 7f8a0ae2b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56075b7f4820 0x56075b7fc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:36.252 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.246+0000 7f8a0ae2b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56075b7f5140 0x56075b7fce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:36.252 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.246+0000 7f8a0ae2b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56075b7ffa00 0x56075b7fd360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:36.252 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.246+0000 7f8a0ae2b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56075b8025d0 con 0x56075b7f4820 2024-04-26T02:03:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.246+0000 7f8a0ae2b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56075b714310 con 0x56075b7f5140 2024-04-26T02:03:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.246+0000 7f8a0ae2b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56075b802750 con 0x56075b7ffa00 2024-04-26T02:03:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.246+0000 7f8a0ae2b740 10 client.0 _ll_register_callbacks cb 0x56075b6e2cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:03:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.246+0000 7f8a05600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56075b7f4820 0x56075b7fc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.246+0000 7f8a05600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56075b7f4820 0x56075b7fc8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:48192/0 (socket says 172.21.15.151:48192) 2024-04-26T02:03:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.246+0000 7f8a05600640 1 -- 172.21.15.151:0/1425402652 learned_addr learned my addr 172.21.15.151:0/1425402652 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:03:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.246+0000 7f8a04c00640 1 --2- 172.21.15.151:0/1425402652 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56075b7f5140 0x56075b7fce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.246+0000 7f8a06000640 1 --2- 172.21.15.151:0/1425402652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56075b7ffa00 0x56075b7fd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.246+0000 7f8a05600640 1 -- 172.21.15.151:0/1425402652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56075b7ffa00 msgr2=0x56075b7fd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.246+0000 7f8a05600640 1 --2- 172.21.15.151:0/1425402652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56075b7ffa00 0x56075b7fd360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.246+0000 7f8a05600640 1 -- 172.21.15.151:0/1425402652 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56075b7f5140 msgr2=0x56075b7fce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.246+0000 7f8a05600640 1 --2- 172.21.15.151:0/1425402652 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56075b7f5140 0x56075b7fce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.246+0000 7f8a05600640 1 -- 172.21.15.151:0/1425402652 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56075b539620 con 0x56075b7f4820 2024-04-26T02:03:36.254 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.246+0000 7f8a04c00640 1 --2- 172.21.15.151:0/1425402652 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56075b7f5140 0x56075b7fce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:03:36.254 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.247+0000 7f8a05600640 1 --2- 172.21.15.151:0/1425402652 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56075b7f4820 0x56075b7fc8e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f89f40303f0 tx=0x7f89f4005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3d1789aab47b87 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:36.254 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.247+0000 7f89e6a00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-04-26T02:03:36.254 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.247+0000 7f89e6a00640 1 -- 172.21.15.151:0/1425402652 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f4037030 con 0x56075b7f4820 2024-04-26T02:03:36.254 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.247+0000 7f89e6a00640 1 -- 172.21.15.151:0/1425402652 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8455+0+0 (secure 0 0 0) 0x7f89f4032c00 con 0x56075b7f4820 2024-04-26T02:03:36.254 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.247+0000 7f8a0ae2b740 1 -- 172.21.15.151:0/1425402652 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89f4030450 con 0x56075b7f4820 2024-04-26T02:03:36.254 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.247+0000 7f89e6a00640 1 -- 172.21.15.151:0/1425402652 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f89f4057050 con 0x56075b7f4820 2024-04-26T02:03:36.254 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.247+0000 7f8a0ae2b740 10 client.16254 fetch_fsmap learned FSMap version 35 2024-04-26T02:03:36.254 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.247+0000 7f8a0ae2b740 1 -- 172.21.15.151:0/1425402652 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56075b80e770 con 0x56075b7f4820 2024-04-26T02:03:36.254 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.248+0000 7f89e6a00640 1 -- 172.21.15.151:0/1425402652 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89f4030450 con 0x56075b7f4820 2024-04-26T02:03:36.254 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.248+0000 7f89e6a00640 1 -- 172.21.15.151:0/1425402652 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 35) ==== 8338+0+0 (secure 0 0 0) 0x7f89f4046460 con 0x56075b7f4820 2024-04-26T02:03:36.254 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.248+0000 7f8a0ae2b740 10 client.16254 fetch_fsmap finished waiting for FSMap version 35 2024-04-26T02:03:36.254 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.248+0000 7f8a0ae2b740 10 client.16254 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:03:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.248+0000 7f8a0ae2b740 20 client.16254 populate_metadata read hostname 'smithi151' 2024-04-26T02:03:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.248+0000 7f8a0ae2b740 1 --2- 172.21.15.151:0/1425402652 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x56075b80fac0 0x56075b811eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.248+0000 7f8a0ae2b740 4 client.16254 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:03:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.248+0000 7f8a0ae2b740 1 -- 172.21.15.151:0/1425402652 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56075b76e8b0 con 0x56075b80fac0 2024-04-26T02:03:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.248+0000 7f8a04c00640 1 --2- 172.21.15.151:0/1425402652 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x56075b80fac0 0x56075b811eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.249+0000 7f8a04c00640 1 --2- 172.21.15.151:0/1425402652 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x56075b80fac0 0x56075b811eb0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11622263bdf77064 server_cookie=a5c76ec6f95891c3 in_seq=0 out_seq=0 2024-04-26T02:03:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.249+0000 7f89e6a00640 10 client.16254 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:03:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.250+0000 7f89e6a00640 1 -- 172.21.15.151:0/1425402652 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56075b76e8b0 con 0x56075b80fac0 2024-04-26T02:03:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.250+0000 7f89e6a00640 10 client.16254 handle_command_reply: tid=0 2024-04-26T02:03:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.250+0000 7f8a0ae2b740 1 client.16254 shutdown 2024-04-26T02:03:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.250+0000 7f8a0ae2b740 1 -- 172.21.15.151:0/1425402652 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56075b7f4820 msgr2=0x56075b7fc8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.250+0000 7f8a0ae2b740 1 --2- 172.21.15.151:0/1425402652 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56075b7f4820 0x56075b7fc8e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f89f40303f0 tx=0x7f89f4005970 comp rx=0 tx=0).stop 2024-04-26T02:03:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.250+0000 7f8a0ae2b740 1 -- 172.21.15.151:0/1425402652 shutdown_connections 2024-04-26T02:03:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.250+0000 7f8a0ae2b740 1 --2- 172.21.15.151:0/1425402652 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x56075b80fac0 0x56075b811eb0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.250+0000 7f8a0ae2b740 1 --2- 172.21.15.151:0/1425402652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56075b7ffa00 0x56075b7fd360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.250+0000 7f8a0ae2b740 1 --2- 172.21.15.151:0/1425402652 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56075b7f5140 0x56075b7fce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.250+0000 7f89e6a00640 0 client.16254 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:03:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.250+0000 7f8a0ae2b740 1 --2- 172.21.15.151:0/1425402652 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56075b7f4820 0x56075b7fc8e0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.250+0000 7f8a0ae2b740 1 -- 172.21.15.151:0/1425402652 >> 172.21.15.151:0/1425402652 conn(0x56075b7f0d30 msgr2=0x56075b7f6ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.250+0000 7f8a0ae2b740 1 -- 172.21.15.151:0/1425402652 shutdown_connections 2024-04-26T02:03:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.250+0000 7f8a0ae2b740 1 -- 172.21.15.151:0/1425402652 wait complete. 2024-04-26T02:03:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.250+0000 7f8a0ae2b740 20 client.16254 trim_cache size 0 max 16384 2024-04-26T02:03:36.266 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.266+0000 7f8a0ae2b740 1 Processor -- start 2024-04-26T02:03:36.266 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.266+0000 7f8a0ae2b740 1 -- start start 2024-04-26T02:03:36.266 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.267+0000 7f8a0ae2b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56075b7f4820 0x56075b80b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:36.266 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.267+0000 7f8a0ae2b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56075b7f5140 0x56075b7efd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:36.266 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.267+0000 7f8a0ae2b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56075b7ffa00 0x56075b8175d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:36.266 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.267+0000 7f8a0ae2b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8a001b24a0 con 0x56075b7f5140 2024-04-26T02:03:36.266 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.267+0000 7f8a0ae2b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8a001afd70 con 0x56075b7f4820 2024-04-26T02:03:36.266 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.267+0000 7f8a0ae2b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8a001adbe0 con 0x56075b7ffa00 2024-04-26T02:03:36.268 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.267+0000 7f8a04c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56075b7f5140 0x56075b7efd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:36.268 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.267+0000 7f8a05600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56075b7f4820 0x56075b80b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:36.268 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.267+0000 7f8a04c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56075b7f5140 0x56075b7efd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:48204/0 (socket says 172.21.15.151:48204) 2024-04-26T02:03:36.268 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.267+0000 7f8a04c00640 1 -- 172.21.15.151:0/1836740323 learned_addr learned my addr 172.21.15.151:0/1836740323 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:03:36.268 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.267+0000 7f8a06000640 1 --2- 172.21.15.151:0/1836740323 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56075b7ffa00 0x56075b8175d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:36.268 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.268+0000 7f8a05600640 1 -- 172.21.15.151:0/1836740323 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56075b7ffa00 msgr2=0x56075b8175d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:36.268 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.268+0000 7f8a05600640 1 --2- 172.21.15.151:0/1836740323 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56075b7ffa00 0x56075b8175d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:36.269 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.268+0000 7f8a05600640 1 -- 172.21.15.151:0/1836740323 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56075b7f5140 msgr2=0x56075b7efd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:36.269 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.268+0000 7f8a05600640 1 --2- 172.21.15.151:0/1836740323 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56075b7f5140 0x56075b7efd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:36.269 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.268+0000 7f8a05600640 1 -- 172.21.15.151:0/1836740323 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56075b7f0860 con 0x56075b7f4820 2024-04-26T02:03:36.269 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.268+0000 7f8a05600640 1 --2- 172.21.15.151:0/1836740323 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56075b7f4820 0x56075b80b490 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f89f4046900 tx=0x7f89f4030980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f93b85320060e83d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:36.269 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.268+0000 7f89f1c00640 1 -- 172.21.15.151:0/1836740323 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f4037070 con 0x56075b7f4820 2024-04-26T02:03:36.269 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.268+0000 7f89f1c00640 1 -- 172.21.15.151:0/1836740323 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89f4030450 con 0x56075b7f4820 2024-04-26T02:03:36.269 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.268+0000 7f89f1c00640 1 -- 172.21.15.151:0/1836740323 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f4004030 con 0x56075b7f4820 2024-04-26T02:03:36.269 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.269+0000 7f8a0ae2b740 1 -- 172.21.15.151:0/1836740323 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56075b7f4820 msgr2=0x56075b80b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:36.269 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.269+0000 7f8a0ae2b740 1 --2- 172.21.15.151:0/1836740323 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56075b7f4820 0x56075b80b490 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f89f4046900 tx=0x7f89f4030980 comp rx=0 tx=0).stop 2024-04-26T02:03:36.269 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.269+0000 7f8a0ae2b740 1 -- 172.21.15.151:0/1836740323 shutdown_connections 2024-04-26T02:03:36.269 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.269+0000 7f8a0ae2b740 1 --2- 172.21.15.151:0/1836740323 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56075b7ffa00 0x56075b8175d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:36.269 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.269+0000 7f8a0ae2b740 1 --2- 172.21.15.151:0/1836740323 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56075b7f5140 0x56075b7efd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:36.269 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.269+0000 7f8a0ae2b740 1 --2- 172.21.15.151:0/1836740323 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56075b7f4820 0x56075b80b490 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.269+0000 7f8a0ae2b740 1 -- 172.21.15.151:0/1836740323 >> 172.21.15.151:0/1836740323 conn(0x56075b7f0d30 msgr2=0x56075b7ef600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.269+0000 7f8a0ae2b740 1 -- 172.21.15.151:0/1836740323 shutdown_connections 2024-04-26T02:03:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.269+0000 7f8a0ae2b740 1 -- 172.21.15.151:0/1836740323 wait complete. 2024-04-26T02:03:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.269+0000 7f8a0ae2b740 1 -- start start 2024-04-26T02:03:36.272 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.270+0000 7f8a0ae2b740 1 Processor -- start 2024-04-26T02:03:36.272 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.270+0000 7f8a0ae2b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56075b7f4820 0x56075b807c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:36.272 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.270+0000 7f8a0ae2b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56075b7f5140 0x56075b804ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:36.272 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.270+0000 7f8a0ae2b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56075b7ffa00 0x56075b805530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:36.273 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.270+0000 7f8a0ae2b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8a001ae230 con 0x56075b7ffa00 2024-04-26T02:03:36.273 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.270+0000 7f8a0ae2b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8a001af6a0 con 0x56075b7f4820 2024-04-26T02:03:36.273 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.270+0000 7f8a0ae2b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8a001ae900 con 0x56075b7f5140 2024-04-26T02:03:36.273 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.270+0000 7f8a0ae2b740 10 client.0 _ll_register_callbacks cb 0x7f89f40096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:03:36.273 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.270+0000 7f8a05600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56075b7f4820 0x56075b807c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:36.273 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.270+0000 7f8a06000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56075b7ffa00 0x56075b805530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:36.273 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.270+0000 7f8a05600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56075b7f4820 0x56075b807c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:44404/0 (socket says 172.21.15.151:44404) 2024-04-26T02:03:36.273 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.270+0000 7f8a05600640 1 -- 172.21.15.151:0/461589950 learned_addr learned my addr 172.21.15.151:0/461589950 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:03:36.273 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.270+0000 7f8a04c00640 1 --2- 172.21.15.151:0/461589950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56075b7f5140 0x56075b804ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:36.273 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.271+0000 7f8a05600640 1 -- 172.21.15.151:0/461589950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56075b7f5140 msgr2=0x56075b804ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:36.273 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.271+0000 7f8a05600640 1 --2- 172.21.15.151:0/461589950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56075b7f5140 0x56075b804ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:36.273 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.271+0000 7f8a05600640 1 -- 172.21.15.151:0/461589950 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56075b7ffa00 msgr2=0x56075b805530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:36.273 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.271+0000 7f8a05600640 1 --2- 172.21.15.151:0/461589950 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56075b7ffa00 0x56075b805530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:36.274 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.271+0000 7f8a05600640 1 -- 172.21.15.151:0/461589950 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56075b805c20 con 0x56075b7f4820 2024-04-26T02:03:36.274 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.271+0000 7f8a06000640 1 --2- 172.21.15.151:0/461589950 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56075b7ffa00 0x56075b805530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:03:36.274 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.271+0000 7f8a05600640 1 --2- 172.21.15.151:0/461589950 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56075b7f4820 0x56075b807c60 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f89f4009980 tx=0x7f89f4005930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8dd0b444ca95c7f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:36.274 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.271+0000 7f89e6a00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T02:03:36.274 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.271+0000 7f89e6a00640 1 -- 172.21.15.151:0/461589950 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f405f030 con 0x56075b7f4820 2024-04-26T02:03:36.274 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.271+0000 7f8a0ae2b740 1 -- 172.21.15.151:0/461589950 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89f4030450 con 0x56075b7f4820 2024-04-26T02:03:36.274 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.271+0000 7f89e6a00640 1 -- 172.21.15.151:0/461589950 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8455+0+0 (secure 0 0 0) 0x7f89f4030c90 con 0x56075b7f4820 2024-04-26T02:03:36.274 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.272+0000 7f89e6a00640 1 -- 172.21.15.151:0/461589950 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f89f4037040 con 0x56075b7f4820 2024-04-26T02:03:36.274 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.272+0000 7f8a0ae2b740 10 client.26360 fetch_fsmap learned FSMap version 35 2024-04-26T02:03:36.274 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.272+0000 7f8a0ae2b740 1 -- 172.21.15.151:0/461589950 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56075b875d60 con 0x56075b7f4820 2024-04-26T02:03:36.274 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.272+0000 7f89e6a00640 1 -- 172.21.15.151:0/461589950 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89f4030450 con 0x56075b7f4820 2024-04-26T02:03:36.274 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.272+0000 7f89e6a00640 1 -- 172.21.15.151:0/461589950 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 35) ==== 8338+0+0 (secure 0 0 0) 0x7f89f40035f0 con 0x56075b7f4820 2024-04-26T02:03:36.274 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.272+0000 7f8a0ae2b740 10 client.26360 fetch_fsmap finished waiting for FSMap version 35 2024-04-26T02:03:36.274 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.272+0000 7f8a0ae2b740 10 client.26360 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:03:36.275 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.272+0000 7f8a0ae2b740 20 client.26360 populate_metadata read hostname 'smithi151' 2024-04-26T02:03:36.275 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.272+0000 7f8a0ae2b740 1 --2- 172.21.15.151:0/461589950 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x56075b876310 0x56075b878700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:36.275 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.272+0000 7f8a0ae2b740 4 client.26360 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T02:03:36.275 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.272+0000 7f8a0ae2b740 1 -- 172.21.15.151:0/461589950 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56075b768320 con 0x56075b876310 2024-04-26T02:03:36.275 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.272+0000 7f8a04c00640 1 --2- 172.21.15.151:0/461589950 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x56075b876310 0x56075b878700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:36.275 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.273+0000 7f8a04c00640 1 --2- 172.21.15.151:0/461589950 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x56075b876310 0x56075b878700 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=533e9a1118f3287 server_cookie=94ddc425037c05f0 in_seq=0 out_seq=0 2024-04-26T02:03:36.275 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.273+0000 7f89e6a00640 10 client.26360 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:03:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.286+0000 7f89e6a00640 1 -- 172.21.15.151:0/461589950 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56075b768320 con 0x56075b876310 2024-04-26T02:03:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.286+0000 7f89e6a00640 10 client.26360 handle_command_reply: tid=0 2024-04-26T02:03:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.286+0000 7f8a0ae2b740 1 client.26360 shutdown 2024-04-26T02:03:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.286+0000 7f8a0ae2b740 1 -- 172.21.15.151:0/461589950 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56075b7f4820 msgr2=0x56075b807c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.286+0000 7f8a0ae2b740 1 --2- 172.21.15.151:0/461589950 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56075b7f4820 0x56075b807c60 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f89f4009980 tx=0x7f89f4005930 comp rx=0 tx=0).stop 2024-04-26T02:03:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.286+0000 7f8a0ae2b740 1 -- 172.21.15.151:0/461589950 shutdown_connections 2024-04-26T02:03:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.286+0000 7f8a0ae2b740 1 --2- 172.21.15.151:0/461589950 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x56075b876310 0x56075b878700 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.286+0000 7f8a0ae2b740 1 --2- 172.21.15.151:0/461589950 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56075b7ffa00 0x56075b805530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.286+0000 7f8a0ae2b740 1 --2- 172.21.15.151:0/461589950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56075b7f5140 0x56075b804ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.286+0000 7f8a0ae2b740 1 --2- 172.21.15.151:0/461589950 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56075b7f4820 0x56075b807c60 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.286+0000 7f89e6a00640 0 client.26360 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:03:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.286+0000 7f8a0ae2b740 1 -- 172.21.15.151:0/461589950 >> 172.21.15.151:0/461589950 conn(0x56075b7f0d30 msgr2=0x56075b7fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.287+0000 7f8a0ae2b740 1 -- 172.21.15.151:0/461589950 shutdown_connections 2024-04-26T02:03:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.287+0000 7f8a0ae2b740 1 -- 172.21.15.151:0/461589950 wait complete. 2024-04-26T02:03:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.287+0000 7f8a0ae2b740 20 client.26360 trim_cache size 0 max 16384 2024-04-26T02:03:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:{ 2024-04-26T02:03:36.288 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "message": "", 2024-04-26T02:03:36.288 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "return_code": 0 2024-04-26T02:03:36.288 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:} 2024-04-26T02:03:36.288 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.287+0000 7f8a08000640 1 -- 172.21.15.151:0/2966746367 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f89e80a3dc0 msgr2=0x7f89e80a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:36.288 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.287+0000 7f8a08000640 1 --2- 172.21.15.151:0/2966746367 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f89e80a3dc0 0x7f89e80a6280 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f89ec009fd0 tx=0x7f89ec009640 comp rx=0 tx=0).stop 2024-04-26T02:03:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.287+0000 7f8a08000640 1 -- 172.21.15.151:0/2966746367 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a0019a6e0 msgr2=0x7f8a0019ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.287+0000 7f8a08000640 1 --2- 172.21.15.151:0/2966746367 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a0019a6e0 0x7f8a0019ab60 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f89f8009fd0 tx=0x7f89f800ef10 comp rx=0 tx=0).stop 2024-04-26T02:03:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.287+0000 7f8a08000640 1 -- 172.21.15.151:0/2966746367 shutdown_connections 2024-04-26T02:03:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.287+0000 7f8a08000640 1 --2- 172.21.15.151:0/2966746367 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f89e80a3dc0 0x7f89e80a6280 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.287+0000 7f8a08000640 1 --2- 172.21.15.151:0/2966746367 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a0019a6e0 0x7f8a0019ab60 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.287+0000 7f8a08000640 1 --2- 172.21.15.151:0/2966746367 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a00109480 0x7f8a001a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.287+0000 7f8a08000640 1 --2- 172.21.15.151:0/2966746367 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a001043b0 0x7f8a001a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.288+0000 7f8a08000640 1 -- 172.21.15.151:0/2966746367 >> 172.21.15.151:0/2966746367 conn(0x7f8a000ffc50 msgr2=0x7f8a00101f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.288+0000 7f8a08000640 1 -- 172.21.15.151:0/2966746367 shutdown_connections 2024-04-26T02:03:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:36.288+0000 7f8a08000640 1 -- 172.21.15.151:0/2966746367 wait complete. 2024-04-26T02:03:36.589 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:03:36 smithi172 ceph-mon[32785]: pgmap v313: 97 pgs: 97 active+clean; 144 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 953 KiB/s wr, 312 op/s 2024-04-26T02:03:36.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:03:36 smithi165 ceph-mon[32648]: pgmap v313: 97 pgs: 97 active+clean; 144 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 953 KiB/s wr, 312 op/s 2024-04-26T02:03:36.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:03:36 smithi151 ceph-mon[28057]: pgmap v313: 97 pgs: 97 active+clean; 144 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 953 KiB/s wr, 312 op/s 2024-04-26T02:03:38.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:03:38 smithi165 ceph-mon[32648]: pgmap v314: 97 pgs: 97 active+clean; 144 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 953 KiB/s wr, 313 op/s 2024-04-26T02:03:38.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:03:38 smithi151 ceph-mon[28057]: pgmap v314: 97 pgs: 97 active+clean; 144 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 953 KiB/s wr, 313 op/s 2024-04-26T02:03:38.788 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:03:38 smithi172 ceph-mon[32785]: pgmap v314: 97 pgs: 97 active+clean; 144 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 953 KiB/s wr, 313 op/s 2024-04-26T02:03:40.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:03:40 smithi165 ceph-mon[32648]: pgmap v315: 97 pgs: 97 active+clean; 140 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 961 KiB/s wr, 318 op/s 2024-04-26T02:03:40.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:03:40 smithi151 ceph-mon[28057]: pgmap v315: 97 pgs: 97 active+clean; 140 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 961 KiB/s wr, 318 op/s 2024-04-26T02:03:40.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:03:40 smithi172 ceph-mon[32785]: pgmap v315: 97 pgs: 97 active+clean; 140 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 961 KiB/s wr, 318 op/s 2024-04-26T02:03:42.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:03:42 smithi165 ceph-mon[32648]: pgmap v316: 97 pgs: 97 active+clean; 140 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 482 KiB/s wr, 150 op/s 2024-04-26T02:03:42.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:03:42 smithi151 ceph-mon[28057]: pgmap v316: 97 pgs: 97 active+clean; 140 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 482 KiB/s wr, 150 op/s 2024-04-26T02:03:42.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:03:42 smithi172 ceph-mon[32785]: pgmap v316: 97 pgs: 97 active+clean; 140 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 482 KiB/s wr, 150 op/s 2024-04-26T02:03:44.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:03:44 smithi165 ceph-mon[32648]: pgmap v317: 97 pgs: 97 active+clean; 140 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 482 KiB/s wr, 153 op/s 2024-04-26T02:03:44.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:03:44 smithi151 ceph-mon[28057]: pgmap v317: 97 pgs: 97 active+clean; 140 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 482 KiB/s wr, 153 op/s 2024-04-26T02:03:44.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:03:44 smithi172 ceph-mon[32785]: pgmap v317: 97 pgs: 97 active+clean; 140 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 482 KiB/s wr, 153 op/s 2024-04-26T02:03:46.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:03:46 smithi165 ceph-mon[32648]: pgmap v318: 97 pgs: 97 active+clean; 140 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 155 KiB/s wr, 71 op/s 2024-04-26T02:03:46.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:03:46 smithi151 ceph-mon[28057]: pgmap v318: 97 pgs: 97 active+clean; 140 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 155 KiB/s wr, 71 op/s 2024-04-26T02:03:46.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:03:46 smithi172 ceph-mon[32785]: pgmap v318: 97 pgs: 97 active+clean; 140 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 155 KiB/s wr, 71 op/s 2024-04-26T02:03:48.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:03:48 smithi165 ceph-mon[32648]: pgmap v319: 97 pgs: 97 active+clean; 140 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 7.6 KiB/s wr, 7 op/s 2024-04-26T02:03:48.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:03:48 smithi151 ceph-mon[28057]: pgmap v319: 97 pgs: 97 active+clean; 140 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 7.6 KiB/s wr, 7 op/s 2024-04-26T02:03:48.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:03:48 smithi172 ceph-mon[32785]: pgmap v319: 97 pgs: 97 active+clean; 140 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 7.6 KiB/s wr, 7 op/s 2024-04-26T02:03:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.427+0000 7fd81d200640 1 -- 172.21.15.151:0/2340001969 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd818104bb0 msgr2=0x7fd818108bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.427+0000 7fd81d200640 1 --2- 172.21.15.151:0/2340001969 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd818104bb0 0x7fd818108bf0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd804009900 tx=0x7fd80402a3f0 comp rx=0 tx=0).stop 2024-04-26T02:03:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.428+0000 7fd81d200640 1 -- 172.21.15.151:0/2340001969 shutdown_connections 2024-04-26T02:03:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.428+0000 7fd81d200640 1 --2- 172.21.15.151:0/2340001969 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8181092d0 0x7fd818110ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.428+0000 7fd81d200640 1 --2- 172.21.15.151:0/2340001969 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd818104bb0 0x7fd818108bf0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.428+0000 7fd81d200640 1 --2- 172.21.15.151:0/2340001969 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd818104200 0x7fd8181045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.428+0000 7fd81d200640 1 -- 172.21.15.151:0/2340001969 >> 172.21.15.151:0/2340001969 conn(0x7fd8180ffa60 msgr2=0x7fd818101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.428+0000 7fd81d200640 1 -- 172.21.15.151:0/2340001969 shutdown_connections 2024-04-26T02:03:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.428+0000 7fd81d200640 1 -- 172.21.15.151:0/2340001969 wait complete. 2024-04-26T02:03:49.430 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.429+0000 7fd81d200640 1 Processor -- start 2024-04-26T02:03:49.430 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.429+0000 7fd81d200640 1 -- start start 2024-04-26T02:03:49.430 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.429+0000 7fd81d200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd818104200 0x7fd8181a0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:49.430 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.429+0000 7fd81d200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8181092d0 0x7fd8181a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:49.430 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.429+0000 7fd81d200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd81819a640 0x7fd81819aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:49.430 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.429+0000 7fd81d200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd818114260 con 0x7fd81819a640 2024-04-26T02:03:49.430 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.429+0000 7fd81d200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd8181140e0 con 0x7fd8181092d0 2024-04-26T02:03:49.430 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.429+0000 7fd81d200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd8181143e0 con 0x7fd818104200 2024-04-26T02:03:49.431 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.429+0000 7fd817e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd81819a640 0x7fd81819aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:49.431 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.430+0000 7fd817400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd818104200 0x7fd8181a0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:49.431 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.430+0000 7fd817e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd81819a640 0x7fd81819aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:50374/0 (socket says 172.21.15.151:50374) 2024-04-26T02:03:49.431 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.430+0000 7fd817e00640 1 -- 172.21.15.151:0/932050640 learned_addr learned my addr 172.21.15.151:0/932050640 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:03:49.431 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.430+0000 7fd816a00640 1 --2- 172.21.15.151:0/932050640 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8181092d0 0x7fd8181a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:49.431 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.430+0000 7fd817e00640 1 -- 172.21.15.151:0/932050640 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd818104200 msgr2=0x7fd8181a0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:49.431 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.430+0000 7fd817e00640 1 --2- 172.21.15.151:0/932050640 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd818104200 0x7fd8181a0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:49.431 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.430+0000 7fd817e00640 1 -- 172.21.15.151:0/932050640 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8181092d0 msgr2=0x7fd8181a0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:49.431 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.430+0000 7fd817e00640 1 --2- 172.21.15.151:0/932050640 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8181092d0 0x7fd8181a0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:49.431 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.430+0000 7fd817e00640 1 -- 172.21.15.151:0/932050640 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd81819b160 con 0x7fd81819a640 2024-04-26T02:03:49.431 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.430+0000 7fd817e00640 1 --2- 172.21.15.151:0/932050640 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd81819a640 0x7fd81819aaf0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd81000eb10 tx=0x7fd81000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:49.431 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.431+0000 7fd80fe00640 1 -- 172.21.15.151:0/932050640 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd810004b20 con 0x7fd81819a640 2024-04-26T02:03:49.431 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.431+0000 7fd80fe00640 1 -- 172.21.15.151:0/932050640 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd810004cc0 con 0x7fd81819a640 2024-04-26T02:03:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.431+0000 7fd81d200640 1 -- 172.21.15.151:0/932050640 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd81819b450 con 0x7fd81819a640 2024-04-26T02:03:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.431+0000 7fd80fe00640 1 -- 172.21.15.151:0/932050640 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd810010670 con 0x7fd81819a640 2024-04-26T02:03:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.431+0000 7fd81d200640 1 -- 172.21.15.151:0/932050640 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8181a7650 con 0x7fd81819a640 2024-04-26T02:03:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.434+0000 7fd8215bf740 1 Processor -- start 2024-04-26T02:03:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.434+0000 7fd80fe00640 1 -- 172.21.15.151:0/932050640 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7fd810005020 con 0x7fd81819a640 2024-04-26T02:03:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.435+0000 7fd8215bf740 1 -- start start 2024-04-26T02:03:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.435+0000 7fd8215bf740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606ad6a89b0 0x5606ad6a8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.435+0000 7fd8215bf740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606ad6a92d0 0x5606ad6b3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.435+0000 7fd8215bf740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5606ad6b3b90 0x5606ad6b5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.435+0000 7fd8215bf740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5606ad60e900 con 0x5606ad6a92d0 2024-04-26T02:03:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.435+0000 7fd8215bf740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5606ad5b4ef0 con 0x5606ad6b3b90 2024-04-26T02:03:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.435+0000 7fd8215bf740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5606ad60a8c0 con 0x5606ad6a89b0 2024-04-26T02:03:49.436 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.435+0000 7fd817400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606ad6a92d0 0x5606ad6b3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:49.436 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.435+0000 7fd817400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606ad6a92d0 0x5606ad6b3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:50386/0 (socket says 172.21.15.151:50386) 2024-04-26T02:03:49.436 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.435+0000 7fd817400640 1 -- 172.21.15.151:0/1479484554 learned_addr learned my addr 172.21.15.151:0/1479484554 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:03:49.436 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.435+0000 7fd816a00640 1 --2- 172.21.15.151:0/1479484554 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5606ad6b3b90 0x5606ad6b5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:49.436 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.436+0000 7fd816a00640 1 --2- 172.21.15.151:0/1479484554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606ad6a89b0 0x5606ad6a8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:49.436 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.436+0000 7fd817400640 1 -- 172.21.15.151:0/1479484554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606ad6a89b0 msgr2=0x5606ad6a8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:49.436 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.436+0000 7fd817400640 1 --2- 172.21.15.151:0/1479484554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606ad6a89b0 0x5606ad6a8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:49.436 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.436+0000 7fd817400640 1 -- 172.21.15.151:0/1479484554 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5606ad6b3b90 msgr2=0x5606ad6b5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:49.436 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.436+0000 7fd817400640 1 --2- 172.21.15.151:0/1479484554 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5606ad6b3b90 0x5606ad6b5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:49.436 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.436+0000 7fd817400640 1 -- 172.21.15.151:0/1479484554 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606ad321210 con 0x5606ad6a92d0 2024-04-26T02:03:49.436 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.436+0000 7fd817400640 1 --2- 172.21.15.151:0/1479484554 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606ad6a92d0 0x5606ad6b3650 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd80800b760 tx=0x7fd80800bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0f6ff134bf01cf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:49.437 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.436+0000 7fd80fe00640 1 --2- 172.21.15.151:0/932050640 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fd7fc0a3d90 0x7fd7fc0a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:49.437 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.436+0000 7fd80cc00640 1 -- 172.21.15.151:0/1479484554 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd808003dd0 con 0x5606ad6a92d0 2024-04-26T02:03:49.437 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.436+0000 7fd80fe00640 1 -- 172.21.15.151:0/932050640 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8455+0+0 (secure 0 0 0) 0x7fd8100d0cc0 con 0x7fd81819a640 2024-04-26T02:03:49.437 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.436+0000 7fd80cc00640 1 -- 172.21.15.151:0/1479484554 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd808015070 con 0x5606ad6a92d0 2024-04-26T02:03:49.437 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.436+0000 7fd80cc00640 1 -- 172.21.15.151:0/1479484554 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd808004d20 con 0x5606ad6a92d0 2024-04-26T02:03:49.437 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.436+0000 7fd816a00640 1 --2- 172.21.15.151:0/932050640 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fd7fc0a3d90 0x7fd7fc0a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:49.441 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.437+0000 7fd8215bf740 1 -- 172.21.15.151:0/1479484554 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606ad6a92d0 msgr2=0x5606ad6b3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:49.441 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.437+0000 7fd8215bf740 1 --2- 172.21.15.151:0/1479484554 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606ad6a92d0 0x5606ad6b3650 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd80800b760 tx=0x7fd80800bc20 comp rx=0 tx=0).stop 2024-04-26T02:03:49.441 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.437+0000 7fd816a00640 1 --2- 172.21.15.151:0/932050640 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fd7fc0a3d90 0x7fd7fc0a6250 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd804031440 tx=0x7fd804041040 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:49.441 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.437+0000 7fd8215bf740 1 -- 172.21.15.151:0/1479484554 shutdown_connections 2024-04-26T02:03:49.441 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.437+0000 7fd8215bf740 1 --2- 172.21.15.151:0/1479484554 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5606ad6b3b90 0x5606ad6b5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:49.441 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.437+0000 7fd8215bf740 1 --2- 172.21.15.151:0/1479484554 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606ad6a92d0 0x5606ad6b3650 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:49.441 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.437+0000 7fd8215bf740 1 --2- 172.21.15.151:0/1479484554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606ad6a89b0 0x5606ad6a8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:49.441 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.437+0000 7fd8215bf740 1 -- 172.21.15.151:0/1479484554 >> 172.21.15.151:0/1479484554 conn(0x5606ad6a4ec0 msgr2=0x5606ad6a52a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:49.442 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.437+0000 7fd8215bf740 1 -- 172.21.15.151:0/1479484554 shutdown_connections 2024-04-26T02:03:49.442 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.438+0000 7fd8215bf740 1 -- 172.21.15.151:0/1479484554 wait complete. 2024-04-26T02:03:49.442 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.438+0000 7fd8215bf740 1 -- start start 2024-04-26T02:03:49.442 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.438+0000 7fd8215bf740 1 Processor -- start 2024-04-26T02:03:49.442 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.439+0000 7fd8215bf740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606ad6a89b0 0x5606ad6b0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:49.442 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.439+0000 7fd8215bf740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606ad6a92d0 0x5606ad6b1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:49.442 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.439+0000 7fd8215bf740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5606ad6b3b90 0x5606ad6b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:49.442 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.439+0000 7fd8215bf740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5606ad6b67c0 con 0x5606ad6a89b0 2024-04-26T02:03:49.442 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.439+0000 7fd8215bf740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5606ad5c8310 con 0x5606ad6b3b90 2024-04-26T02:03:49.442 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.439+0000 7fd8215bf740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5606ad6b6940 con 0x5606ad6a92d0 2024-04-26T02:03:49.442 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.439+0000 7fd8215bf740 10 client.0 _ll_register_callbacks cb 0x5606ad596cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:03:49.442 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.439+0000 7fd817400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606ad6a89b0 0x5606ad6b0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:49.443 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.439+0000 7fd817400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606ad6a89b0 0x5606ad6b0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:50394/0 (socket says 172.21.15.151:50394) 2024-04-26T02:03:49.443 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.439+0000 7fd817400640 1 -- 172.21.15.151:0/2903697933 learned_addr learned my addr 172.21.15.151:0/2903697933 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:03:49.443 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.439+0000 7fd816a00640 1 --2- 172.21.15.151:0/2903697933 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606ad6a92d0 0x5606ad6b1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:49.443 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.439+0000 7fd817e00640 1 --2- 172.21.15.151:0/2903697933 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5606ad6b3b90 0x5606ad6b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:49.443 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.439+0000 7fd817400640 1 -- 172.21.15.151:0/2903697933 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606ad6a92d0 msgr2=0x5606ad6b1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:49.443 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.439+0000 7fd817400640 1 --2- 172.21.15.151:0/2903697933 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606ad6a92d0 0x5606ad6b1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:49.443 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.439+0000 7fd817400640 1 -- 172.21.15.151:0/2903697933 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5606ad6b3b90 msgr2=0x5606ad6b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:49.443 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.439+0000 7fd817400640 1 --2- 172.21.15.151:0/2903697933 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5606ad6b3b90 0x5606ad6b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:49.443 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.439+0000 7fd817400640 1 -- 172.21.15.151:0/2903697933 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606ad3ed620 con 0x5606ad6a89b0 2024-04-26T02:03:49.443 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.440+0000 7fd817400640 1 --2- 172.21.15.151:0/2903697933 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606ad6a89b0 0x5606ad6b0b40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd808006010 tx=0x7fd80800bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec8660a9a252723f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:49.443 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.440+0000 7fd801600640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-04-26T02:03:49.443 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.440+0000 7fd801600640 1 -- 172.21.15.151:0/2903697933 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd808028060 con 0x5606ad6a89b0 2024-04-26T02:03:49.443 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.440+0000 7fd801600640 1 -- 172.21.15.151:0/2903697933 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8455+0+0 (secure 0 0 0) 0x7fd808011440 con 0x5606ad6a89b0 2024-04-26T02:03:49.444 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.440+0000 7fd8215bf740 1 -- 172.21.15.151:0/2903697933 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd808015070 con 0x5606ad6a89b0 2024-04-26T02:03:49.444 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.440+0000 7fd801600640 1 -- 172.21.15.151:0/2903697933 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fd808011640 con 0x5606ad6a89b0 2024-04-26T02:03:49.444 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.440+0000 7fd8215bf740 10 client.16272 fetch_fsmap learned FSMap version 35 2024-04-26T02:03:49.444 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.440+0000 7fd8215bf740 1 -- 172.21.15.151:0/2903697933 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5606ad6c2a40 con 0x5606ad6a89b0 2024-04-26T02:03:49.444 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.440+0000 7fd801600640 1 -- 172.21.15.151:0/2903697933 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd808015070 con 0x5606ad6a89b0 2024-04-26T02:03:49.444 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.441+0000 7fd801600640 1 -- 172.21.15.151:0/2903697933 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 35) ==== 8338+0+0 (secure 0 0 0) 0x7fd808022070 con 0x5606ad6a89b0 2024-04-26T02:03:49.444 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.441+0000 7fd8215bf740 10 client.16272 fetch_fsmap finished waiting for FSMap version 35 2024-04-26T02:03:49.444 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.441+0000 7fd8215bf740 10 client.16272 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:03:49.444 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.441+0000 7fd8215bf740 20 client.16272 populate_metadata read hostname 'smithi151' 2024-04-26T02:03:49.444 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.441+0000 7fd8215bf740 1 --2- 172.21.15.151:0/2903697933 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5606ad6c3e40 0x5606ad6c6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:49.444 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.441+0000 7fd8215bf740 4 client.16272 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:03:49.444 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.441+0000 7fd8215bf740 1 -- 172.21.15.151:0/2903697933 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606ad6228b0 con 0x5606ad6c3e40 2024-04-26T02:03:49.444 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.441+0000 7fd816a00640 1 --2- 172.21.15.151:0/2903697933 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5606ad6c3e40 0x5606ad6c6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:49.445 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.442+0000 7fd816a00640 1 --2- 172.21.15.151:0/2903697933 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5606ad6c3e40 0x5606ad6c6230 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb06afc27be32959 server_cookie=6cb75525bd928519 in_seq=0 out_seq=0 2024-04-26T02:03:49.445 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.442+0000 7fd801600640 10 client.16272 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:03:49.446 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.443+0000 7fd801600640 1 -- 172.21.15.151:0/2903697933 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5606ad6228b0 con 0x5606ad6c3e40 2024-04-26T02:03:49.446 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.443+0000 7fd801600640 10 client.16272 handle_command_reply: tid=0 2024-04-26T02:03:49.446 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.443+0000 7fd8215bf740 1 client.16272 shutdown 2024-04-26T02:03:49.446 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.443+0000 7fd8215bf740 1 -- 172.21.15.151:0/2903697933 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606ad6a89b0 msgr2=0x5606ad6b0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:49.446 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.443+0000 7fd8215bf740 1 --2- 172.21.15.151:0/2903697933 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606ad6a89b0 0x5606ad6b0b40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd808006010 tx=0x7fd80800bfb0 comp rx=0 tx=0).stop 2024-04-26T02:03:49.446 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.443+0000 7fd8215bf740 1 -- 172.21.15.151:0/2903697933 shutdown_connections 2024-04-26T02:03:49.446 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.443+0000 7fd8215bf740 1 --2- 172.21.15.151:0/2903697933 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5606ad6c3e40 0x5606ad6c6230 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:49.446 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.443+0000 7fd8215bf740 1 --2- 172.21.15.151:0/2903697933 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5606ad6b3b90 0x5606ad6b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:49.446 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.443+0000 7fd8215bf740 1 --2- 172.21.15.151:0/2903697933 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606ad6a92d0 0x5606ad6b1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:49.446 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.443+0000 7fd801600640 0 client.16272 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:03:49.446 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.443+0000 7fd8215bf740 1 --2- 172.21.15.151:0/2903697933 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606ad6a89b0 0x5606ad6b0b40 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:49.446 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.443+0000 7fd8215bf740 1 -- 172.21.15.151:0/2903697933 >> 172.21.15.151:0/2903697933 conn(0x5606ad6a4ec0 msgr2=0x5606ad6aabf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:49.446 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.444+0000 7fd8215bf740 1 -- 172.21.15.151:0/2903697933 shutdown_connections 2024-04-26T02:03:49.447 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.444+0000 7fd8215bf740 1 -- 172.21.15.151:0/2903697933 wait complete. 2024-04-26T02:03:49.447 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.444+0000 7fd8215bf740 20 client.16272 trim_cache size 0 max 16384 2024-04-26T02:03:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.460+0000 7fd8215bf740 1 Processor -- start 2024-04-26T02:03:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.460+0000 7fd8215bf740 1 -- start start 2024-04-26T02:03:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.460+0000 7fd8215bf740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5606ad6a89b0 0x5606ad6bf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.460+0000 7fd8215bf740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606ad6a92d0 0x5606ad6a3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.461+0000 7fd8215bf740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606ad6b3b90 0x5606ad6cb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.461+0000 7fd8215bf740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd8181b2380 con 0x5606ad6a92d0 2024-04-26T02:03:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.461+0000 7fd8215bf740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd8181afc50 con 0x5606ad6a89b0 2024-04-26T02:03:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.461+0000 7fd8215bf740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd8181adac0 con 0x5606ad6b3b90 2024-04-26T02:03:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.461+0000 7fd817400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5606ad6a89b0 0x5606ad6bf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.461+0000 7fd816a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606ad6a92d0 0x5606ad6a3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.461+0000 7fd817400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5606ad6a89b0 0x5606ad6bf680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:52774/0 (socket says 172.21.15.151:52774) 2024-04-26T02:03:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.461+0000 7fd817400640 1 -- 172.21.15.151:0/1953906180 learned_addr learned my addr 172.21.15.151:0/1953906180 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:03:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.461+0000 7fd817e00640 1 --2- 172.21.15.151:0/1953906180 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606ad6b3b90 0x5606ad6cb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.461+0000 7fd817400640 1 -- 172.21.15.151:0/1953906180 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606ad6b3b90 msgr2=0x5606ad6cb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.461+0000 7fd817400640 1 --2- 172.21.15.151:0/1953906180 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606ad6b3b90 0x5606ad6cb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.461+0000 7fd817400640 1 -- 172.21.15.151:0/1953906180 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606ad6a92d0 msgr2=0x5606ad6a3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.461+0000 7fd817400640 1 --2- 172.21.15.151:0/1953906180 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606ad6a92d0 0x5606ad6a3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.461+0000 7fd817400640 1 -- 172.21.15.151:0/1953906180 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606ad6b5f80 con 0x5606ad6a89b0 2024-04-26T02:03:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.461+0000 7fd817400640 1 --2- 172.21.15.151:0/1953906180 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5606ad6a89b0 0x5606ad6bf680 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd80802db10 tx=0x7fd8080115d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8dd69873d00dca91 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:49.466 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.462+0000 7fd803e00640 1 -- 172.21.15.151:0/1953906180 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd808028070 con 0x5606ad6a89b0 2024-04-26T02:03:49.466 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.462+0000 7fd803e00640 1 -- 172.21.15.151:0/1953906180 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd808015070 con 0x5606ad6a89b0 2024-04-26T02:03:49.466 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.462+0000 7fd803e00640 1 -- 172.21.15.151:0/1953906180 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd808011a40 con 0x5606ad6a89b0 2024-04-26T02:03:49.466 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.462+0000 7fd8215bf740 1 -- 172.21.15.151:0/1953906180 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5606ad6a89b0 msgr2=0x5606ad6bf680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.462+0000 7fd8215bf740 1 --2- 172.21.15.151:0/1953906180 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5606ad6a89b0 0x5606ad6bf680 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd80802db10 tx=0x7fd8080115d0 comp rx=0 tx=0).stop 2024-04-26T02:03:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.463+0000 7fd8215bf740 1 -- 172.21.15.151:0/1953906180 shutdown_connections 2024-04-26T02:03:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.463+0000 7fd8215bf740 1 --2- 172.21.15.151:0/1953906180 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606ad6b3b90 0x5606ad6cb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.463+0000 7fd8215bf740 1 --2- 172.21.15.151:0/1953906180 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606ad6a92d0 0x5606ad6a3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.463+0000 7fd8215bf740 1 --2- 172.21.15.151:0/1953906180 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5606ad6a89b0 0x5606ad6bf680 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.463+0000 7fd8215bf740 1 -- 172.21.15.151:0/1953906180 >> 172.21.15.151:0/1953906180 conn(0x5606ad6a4ec0 msgr2=0x5606ad6a3790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.463+0000 7fd8215bf740 1 -- 172.21.15.151:0/1953906180 shutdown_connections 2024-04-26T02:03:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.463+0000 7fd8215bf740 1 -- 172.21.15.151:0/1953906180 wait complete. 2024-04-26T02:03:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.463+0000 7fd8215bf740 1 -- start start 2024-04-26T02:03:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.463+0000 7fd8215bf740 1 Processor -- start 2024-04-26T02:03:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.464+0000 7fd8215bf740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606ad6a89b0 0x5606ad6bbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.464+0000 7fd8215bf740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606ad6a92d0 0x5606ad6b9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:49.468 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.464+0000 7fd8215bf740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5606ad6b3b90 0x5606ad6b96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:49.468 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.464+0000 7fd8215bf740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd8181ae110 con 0x5606ad6a92d0 2024-04-26T02:03:49.468 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.464+0000 7fd8215bf740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd8181af580 con 0x5606ad6b3b90 2024-04-26T02:03:49.468 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.464+0000 7fd8215bf740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd8181ae7e0 con 0x5606ad6a89b0 2024-04-26T02:03:49.468 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.464+0000 7fd8215bf740 10 client.0 _ll_register_callbacks cb 0x7fd80800be70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:03:49.468 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.464+0000 7fd816a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606ad6a92d0 0x5606ad6b9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:49.468 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.464+0000 7fd817400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606ad6a89b0 0x5606ad6bbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:49.468 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.464+0000 7fd816a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606ad6a92d0 0x5606ad6b9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:50422/0 (socket says 172.21.15.151:50422) 2024-04-26T02:03:49.468 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.464+0000 7fd816a00640 1 -- 172.21.15.151:0/738061692 learned_addr learned my addr 172.21.15.151:0/738061692 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:03:49.468 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.464+0000 7fd817e00640 1 --2- 172.21.15.151:0/738061692 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5606ad6b3b90 0x5606ad6b96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:49.468 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.464+0000 7fd817400640 1 -- 172.21.15.151:0/738061692 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5606ad6b3b90 msgr2=0x5606ad6b96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:49.468 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.464+0000 7fd817400640 1 --2- 172.21.15.151:0/738061692 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5606ad6b3b90 0x5606ad6b96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:49.468 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.464+0000 7fd817400640 1 -- 172.21.15.151:0/738061692 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606ad6a92d0 msgr2=0x5606ad6b9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:49.469 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.464+0000 7fd817400640 1 --2- 172.21.15.151:0/738061692 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606ad6a92d0 0x5606ad6b9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:49.469 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.464+0000 7fd817400640 1 -- 172.21.15.151:0/738061692 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606ad6b9cb0 con 0x5606ad6a89b0 2024-04-26T02:03:49.469 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.464+0000 7fd817e00640 1 --2- 172.21.15.151:0/738061692 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5606ad6b3b90 0x5606ad6b96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:03:49.469 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.464+0000 7fd816a00640 1 --2- 172.21.15.151:0/738061692 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606ad6a92d0 0x5606ad6b9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:03:49.469 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.465+0000 7fd817400640 1 --2- 172.21.15.151:0/738061692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606ad6a89b0 0x5606ad6bbd30 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd808005b60 tx=0x7fd8080058f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1586a2dc1e7a286 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:49.469 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.465+0000 7fd801600640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-26T02:03:49.469 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.465+0000 7fd801600640 1 -- 172.21.15.151:0/738061692 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd808028040 con 0x5606ad6a89b0 2024-04-26T02:03:49.469 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.465+0000 7fd8215bf740 1 -- 172.21.15.151:0/738061692 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd808015070 con 0x5606ad6a89b0 2024-04-26T02:03:49.469 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.465+0000 7fd801600640 1 -- 172.21.15.151:0/738061692 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8455+0+0 (secure 0 0 0) 0x7fd80802d670 con 0x5606ad6a89b0 2024-04-26T02:03:49.469 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.465+0000 7fd801600640 1 -- 172.21.15.151:0/738061692 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fd8080117e0 con 0x5606ad6a89b0 2024-04-26T02:03:49.469 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.465+0000 7fd8215bf740 10 client.26398 fetch_fsmap learned FSMap version 35 2024-04-26T02:03:49.469 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.465+0000 7fd8215bf740 1 -- 172.21.15.151:0/738061692 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5606ad729ec0 con 0x5606ad6a89b0 2024-04-26T02:03:49.469 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.466+0000 7fd801600640 1 -- 172.21.15.151:0/738061692 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd808015070 con 0x5606ad6a89b0 2024-04-26T02:03:49.470 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.466+0000 7fd801600640 1 -- 172.21.15.151:0/738061692 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 35) ==== 8338+0+0 (secure 0 0 0) 0x7fd808022040 con 0x5606ad6a89b0 2024-04-26T02:03:49.470 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.466+0000 7fd8215bf740 10 client.26398 fetch_fsmap finished waiting for FSMap version 35 2024-04-26T02:03:49.470 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.466+0000 7fd8215bf740 10 client.26398 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:03:49.470 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.466+0000 7fd8215bf740 20 client.26398 populate_metadata read hostname 'smithi151' 2024-04-26T02:03:49.470 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.466+0000 7fd8215bf740 1 --2- 172.21.15.151:0/738061692 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5606ad72a640 0x5606ad72ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:49.470 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.466+0000 7fd8215bf740 4 client.26398 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T02:03:49.470 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.466+0000 7fd8215bf740 1 -- 172.21.15.151:0/738061692 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5606ad61c320 con 0x5606ad72a640 2024-04-26T02:03:49.470 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.466+0000 7fd816a00640 1 --2- 172.21.15.151:0/738061692 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5606ad72a640 0x5606ad72ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:49.470 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.467+0000 7fd816a00640 1 --2- 172.21.15.151:0/738061692 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5606ad72a640 0x5606ad72ca60 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a26322d02b7b33e2 server_cookie=865b8facb0ff22b9 in_seq=0 out_seq=0 2024-04-26T02:03:49.470 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.467+0000 7fd801600640 10 client.26398 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:03:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.473+0000 7fd801600640 1 -- 172.21.15.151:0/738061692 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5606ad61c320 con 0x5606ad72a640 2024-04-26T02:03:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.473+0000 7fd801600640 10 client.26398 handle_command_reply: tid=0 2024-04-26T02:03:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.473+0000 7fd8215bf740 1 client.26398 shutdown 2024-04-26T02:03:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.473+0000 7fd8215bf740 1 -- 172.21.15.151:0/738061692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606ad6a89b0 msgr2=0x5606ad6bbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.473+0000 7fd8215bf740 1 --2- 172.21.15.151:0/738061692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606ad6a89b0 0x5606ad6bbd30 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd808005b60 tx=0x7fd8080058f0 comp rx=0 tx=0).stop 2024-04-26T02:03:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.474+0000 7fd8215bf740 1 -- 172.21.15.151:0/738061692 shutdown_connections 2024-04-26T02:03:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.474+0000 7fd8215bf740 1 --2- 172.21.15.151:0/738061692 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5606ad72a640 0x5606ad72ca60 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.474+0000 7fd8215bf740 1 --2- 172.21.15.151:0/738061692 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5606ad6b3b90 0x5606ad6b96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.474+0000 7fd801600640 0 client.26398 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:03:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.474+0000 7fd8215bf740 1 --2- 172.21.15.151:0/738061692 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5606ad6a92d0 0x5606ad6b9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.474+0000 7fd8215bf740 1 --2- 172.21.15.151:0/738061692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606ad6a89b0 0x5606ad6bbd30 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.474+0000 7fd8215bf740 1 -- 172.21.15.151:0/738061692 >> 172.21.15.151:0/738061692 conn(0x5606ad6a4ec0 msgr2=0x5606ad6a4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.474+0000 7fd8215bf740 1 -- 172.21.15.151:0/738061692 shutdown_connections 2024-04-26T02:03:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.474+0000 7fd8215bf740 1 -- 172.21.15.151:0/738061692 wait complete. 2024-04-26T02:03:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.474+0000 7fd8215bf740 20 client.26398 trim_cache size 0 max 16384 2024-04-26T02:03:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:{ 2024-04-26T02:03:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "message": "", 2024-04-26T02:03:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "return_code": 0 2024-04-26T02:03:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:} 2024-04-26T02:03:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.475+0000 7fd81d200640 1 -- 172.21.15.151:0/932050640 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fd7fc0a3d90 msgr2=0x7fd7fc0a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.475+0000 7fd81d200640 1 --2- 172.21.15.151:0/932050640 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fd7fc0a3d90 0x7fd7fc0a6250 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd804031440 tx=0x7fd804041040 comp rx=0 tx=0).stop 2024-04-26T02:03:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.475+0000 7fd81d200640 1 -- 172.21.15.151:0/932050640 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd81819a640 msgr2=0x7fd81819aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.475+0000 7fd81d200640 1 --2- 172.21.15.151:0/932050640 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd81819a640 0x7fd81819aaf0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd81000eb10 tx=0x7fd81000c2d0 comp rx=0 tx=0).stop 2024-04-26T02:03:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.475+0000 7fd81d200640 1 -- 172.21.15.151:0/932050640 shutdown_connections 2024-04-26T02:03:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.475+0000 7fd81d200640 1 --2- 172.21.15.151:0/932050640 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fd7fc0a3d90 0x7fd7fc0a6250 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.475+0000 7fd81d200640 1 --2- 172.21.15.151:0/932050640 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd81819a640 0x7fd81819aaf0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:49.476 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.475+0000 7fd81d200640 1 --2- 172.21.15.151:0/932050640 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8181092d0 0x7fd8181a0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:49.476 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.475+0000 7fd81d200640 1 --2- 172.21.15.151:0/932050640 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd818104200 0x7fd8181a0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:49.476 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.475+0000 7fd81d200640 1 -- 172.21.15.151:0/932050640 >> 172.21.15.151:0/932050640 conn(0x7fd8180ffa60 msgr2=0x7fd818100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:49.476 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.475+0000 7fd81d200640 1 -- 172.21.15.151:0/932050640 shutdown_connections 2024-04-26T02:03:49.476 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:03:49.475+0000 7fd81d200640 1 -- 172.21.15.151:0/932050640 wait complete. 2024-04-26T02:03:50.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:03:50 smithi165 ceph-mon[32648]: pgmap v320: 97 pgs: 97 active+clean; 150 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 40 KiB/s wr, 11 op/s 2024-04-26T02:03:50.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:03:50 smithi151 ceph-mon[28057]: pgmap v320: 97 pgs: 97 active+clean; 150 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 40 KiB/s wr, 11 op/s 2024-04-26T02:03:50.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:03:50 smithi172 ceph-mon[32785]: pgmap v320: 97 pgs: 97 active+clean; 150 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 40 KiB/s wr, 11 op/s 2024-04-26T02:03:52.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:03:52 smithi165 ceph-mon[32648]: pgmap v321: 97 pgs: 97 active+clean; 150 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 32 KiB/s wr, 5 op/s 2024-04-26T02:03:52.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:03:52 smithi151 ceph-mon[28057]: pgmap v321: 97 pgs: 97 active+clean; 150 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 32 KiB/s wr, 5 op/s 2024-04-26T02:03:52.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:03:52 smithi172 ceph-mon[32785]: pgmap v321: 97 pgs: 97 active+clean; 150 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 32 KiB/s wr, 5 op/s 2024-04-26T02:03:54.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:03:54 smithi165 ceph-mon[32648]: pgmap v322: 97 pgs: 97 active+clean; 150 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 35 KiB/s wr, 8 op/s 2024-04-26T02:03:54.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:03:54 smithi151 ceph-mon[28057]: pgmap v322: 97 pgs: 97 active+clean; 150 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 35 KiB/s wr, 8 op/s 2024-04-26T02:03:54.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:03:54 smithi172 ceph-mon[32785]: pgmap v322: 97 pgs: 97 active+clean; 150 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 35 KiB/s wr, 8 op/s 2024-04-26T02:03:55.480 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T02:03:55.933 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:03:56.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:03:56 smithi151 ceph-mon[28057]: pgmap v323: 97 pgs: 97 active+clean; 150 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 35 KiB/s wr, 6 op/s 2024-04-26T02:03:56.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:03:56 smithi172 ceph-mon[32785]: pgmap v323: 97 pgs: 97 active+clean; 150 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 35 KiB/s wr, 6 op/s 2024-04-26T02:03:56.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:03:56 smithi165 ceph-mon[32648]: pgmap v323: 97 pgs: 97 active+clean; 150 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 35 KiB/s wr, 6 op/s 2024-04-26T02:03:58.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:58.227+0000 7f5ced600700 1 -- 172.21.15.151:0/3790889986 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ce810f440 msgr2=0x7f5ce8111830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:58.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:58.227+0000 7f5ced600700 1 --2- 172.21.15.151:0/3790889986 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ce810f440 0x7f5ce8111830 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0009f90 tx=0x7f5ce000b330 comp rx=0 tx=0).stop 2024-04-26T02:03:58.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:58.227+0000 7f5ced600700 1 -- 172.21.15.151:0/3790889986 shutdown_connections 2024-04-26T02:03:58.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:58.227+0000 7f5ced600700 1 --2- 172.21.15.151:0/3790889986 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ce810f440 0x7f5ce8111830 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:58.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:58.227+0000 7f5ced600700 1 --2- 172.21.15.151:0/3790889986 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ce8074c70 0x7f5ce8075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:58.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:58.227+0000 7f5ced600700 1 --2- 172.21.15.151:0/3790889986 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ce8075590 0x7f5ce810ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:58.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:58.227+0000 7f5ced600700 1 -- 172.21.15.151:0/3790889986 >> 172.21.15.151:0/3790889986 conn(0x7f5ce8079bf0 msgr2=0x7f5ce807a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:58.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:58.227+0000 7f5ced600700 1 -- 172.21.15.151:0/3790889986 shutdown_connections 2024-04-26T02:03:58.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:58.227+0000 7f5ced600700 1 -- 172.21.15.151:0/3790889986 wait complete. 2024-04-26T02:03:58.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:58.229+0000 7f5ced600700 1 Processor -- start 2024-04-26T02:03:58.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:58.229+0000 7f5ced600700 1 -- start start 2024-04-26T02:03:58.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:58.229+0000 7f5ced600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ce8074c70 0x7f5ce819b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:58.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:58.229+0000 7f5ced600700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ce8075590 0x7f5ce819b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:58.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:58.229+0000 7f5ced600700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ce810f440 0x7f5ce819fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:58.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:58.229+0000 7f5ced600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce8114ed0 con 0x7f5ce8074c70 2024-04-26T02:03:58.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:58.229+0000 7f5ced600700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce8114d50 con 0x7f5ce8075590 2024-04-26T02:03:58.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:58.229+0000 7f5ced600700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce8115050 con 0x7f5ce810f440 2024-04-26T02:03:58.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:58.230+0000 7f5ce6a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ce8074c70 0x7f5ce819b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:58.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:58.230+0000 7f5ce6a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ce8074c70 0x7f5ce819b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:48374/0 (socket says 172.21.15.151:48374) 2024-04-26T02:03:58.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:58.230+0000 7f5ce6a00700 1 -- 172.21.15.151:0/4173366394 learned_addr learned my addr 172.21.15.151:0/4173366394 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:03:58.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:58.230+0000 7f5ce6000700 1 --2- 172.21.15.151:0/4173366394 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ce8075590 0x7f5ce819b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:58.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:58.230+0000 7f5ce7400700 1 --2- 172.21.15.151:0/4173366394 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ce810f440 0x7f5ce819fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:58.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:58.230+0000 7f5ce6a00700 1 -- 172.21.15.151:0/4173366394 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ce810f440 msgr2=0x7f5ce819fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:58.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:58.230+0000 7f5ce6a00700 1 --2- 172.21.15.151:0/4173366394 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ce810f440 0x7f5ce819fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:58.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:58.230+0000 7f5ce6a00700 1 -- 172.21.15.151:0/4173366394 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ce8075590 msgr2=0x7f5ce819b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:58.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:58.230+0000 7f5ce6a00700 1 --2- 172.21.15.151:0/4173366394 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ce8075590 0x7f5ce819b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:58.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:58.230+0000 7f5ce6a00700 1 -- 172.21.15.151:0/4173366394 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ce81a0300 con 0x7f5ce8074c70 2024-04-26T02:03:58.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:58.231+0000 7f5ce6a00700 1 --2- 172.21.15.151:0/4173366394 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ce8074c70 0x7f5ce819b340 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5cd800e9a0 tx=0x7f5cd800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:58.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:58.231+0000 7f5cdf400700 1 -- 172.21.15.151:0/4173366394 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd800c9c0 con 0x7f5ce8074c70 2024-04-26T02:03:58.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:58.231+0000 7f5ced600700 1 -- 172.21.15.151:0/4173366394 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ce81a05f0 con 0x7f5ce8074c70 2024-04-26T02:03:58.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:58.231+0000 7f5ced600700 1 -- 172.21.15.151:0/4173366394 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ce81a7e70 con 0x7f5ce8074c70 2024-04-26T02:03:58.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:58.231+0000 7f5cdf400700 1 -- 172.21.15.151:0/4173366394 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cd800cb60 con 0x7f5ce8074c70 2024-04-26T02:03:58.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:58.231+0000 7f5cdf400700 1 -- 172.21.15.151:0/4173366394 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd8018680 con 0x7f5ce8074c70 2024-04-26T02:03:58.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:58.232+0000 7f5ced600700 1 -- 172.21.15.151:0/4173366394 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cc8005330 con 0x7f5ce8074c70 2024-04-26T02:03:58.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:58.235+0000 7f5cdf400700 1 -- 172.21.15.151:0/4173366394 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f5cd8010430 con 0x7f5ce8074c70 2024-04-26T02:03:58.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:58.236+0000 7f5cdf400700 1 --2- 172.21.15.151:0/4173366394 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f5ccc0a3e30 0x7f5ccc0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:58.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:58.236+0000 7f5cdf400700 1 -- 172.21.15.151:0/4173366394 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8455+0+0 (secure 0 0 0) 0x7f5cd80d0910 con 0x7f5ce8074c70 2024-04-26T02:03:58.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:58.236+0000 7f5ce6000700 1 --2- 172.21.15.151:0/4173366394 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f5ccc0a3e30 0x7f5ccc0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:58.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:58.237+0000 7f5ce6000700 1 --2- 172.21.15.151:0/4173366394 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f5ccc0a3e30 0x7f5ccc0a62f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8111170 tx=0x7f5cd4009310 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:58.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:58.241+0000 7f5cdf400700 1 -- 172.21.15.151:0/4173366394 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5cd809cf90 con 0x7f5ce8074c70 2024-04-26T02:03:58.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:58.518+0000 7f5ced600700 1 -- 172.21.15.151:0/4173366394 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5cc80058f0 con 0x7f5ce8074c70 2024-04-26T02:03:58.519 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:03:58 smithi151 ceph-mon[28057]: pgmap v324: 97 pgs: 97 active+clean; 150 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 35 KiB/s wr, 6 op/s 2024-04-26T02:03:58.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:58.520+0000 7f5cdf400700 1 -- 172.21.15.151:0/4173366394 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11889 (secure 0 0 0) 0x7f5cd809c890 con 0x7f5ce8074c70 2024-04-26T02:03:58.520 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T02:03:58.520 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2937864097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2937864097},{"type":"v1","addr":"172.21.15.151:6835","nonce":2937864097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14373,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/2947090802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2947090802},{"type":"v1","addr":"172.21.15.151:6837","nonce":2947090802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/1356563280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1356563280},{"type":"v1","addr":"172.21.15.151:6839","nonce":1356563280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6841/1075240552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1075240552},{"type":"v1","addr":"172.21.15.151:6841","nonce":1075240552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/504675912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":504675912},{"type":"v1","addr":"172.21.15.165:6833","nonce":504675912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1207580015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1207580015},{"type":"v1","addr":"172.21.15.172:6833","nonce":1207580015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:49:40.746876+0000","modified":"2024-04-26T02:00:05.413774+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24505,"mds_2":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/3598138294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3598138294},{"type":"v1","addr":"172.21.15.165:6835","nonce":3598138294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/1847859054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":1847859054},{"type":"v1","addr":"172.21.15.165:6837","nonce":1847859054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.172:6837/55972537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":55972537},{"type":"v1","addr":"172.21.15.172:6837","nonce":55972537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.172:6839/1242302810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1242302810},{"type":"v1","addr":"172.21.15.172:6839","nonce":1242302810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/1969716254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1969716254},{"type":"v1","addr":"172.21.15.165:6839","nonce":1969716254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25954":{"gid":25954,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/3656630171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3656630171},{"type":"v1","addr":"172.21.15.172:6835","nonce":3656630171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24493,24505,24509]},"id":1}]} 2024-04-26T02:03:58.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:58.523+0000 7f5ced600700 1 -- 172.21.15.151:0/4173366394 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f5ccc0a3e30 msgr2=0x7f5ccc0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:58.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:58.523+0000 7f5ced600700 1 --2- 172.21.15.151:0/4173366394 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f5ccc0a3e30 0x7f5ccc0a62f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8111170 tx=0x7f5cd4009310 comp rx=0 tx=0).stop 2024-04-26T02:03:58.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:58.523+0000 7f5ced600700 1 -- 172.21.15.151:0/4173366394 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ce8074c70 msgr2=0x7f5ce819b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:58.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:58.523+0000 7f5ced600700 1 --2- 172.21.15.151:0/4173366394 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ce8074c70 0x7f5ce819b340 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5cd800e9a0 tx=0x7f5cd800ecb0 comp rx=0 tx=0).stop 2024-04-26T02:03:58.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:58.524+0000 7f5ced600700 1 -- 172.21.15.151:0/4173366394 shutdown_connections 2024-04-26T02:03:58.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:58.524+0000 7f5ced600700 1 --2- 172.21.15.151:0/4173366394 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f5ccc0a3e30 0x7f5ccc0a62f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:58.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:58.524+0000 7f5ced600700 1 --2- 172.21.15.151:0/4173366394 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ce810f440 0x7f5ce819fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:58.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:58.524+0000 7f5ced600700 1 --2- 172.21.15.151:0/4173366394 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ce8074c70 0x7f5ce819b340 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:58.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:58.524+0000 7f5ced600700 1 --2- 172.21.15.151:0/4173366394 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ce8075590 0x7f5ce819b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:58.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:58.524+0000 7f5ced600700 1 -- 172.21.15.151:0/4173366394 >> 172.21.15.151:0/4173366394 conn(0x7f5ce8079bf0 msgr2=0x7f5ce80ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:58.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:58.524+0000 7f5ced600700 1 -- 172.21.15.151:0/4173366394 shutdown_connections 2024-04-26T02:03:58.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:03:58.524+0000 7f5ced600700 1 -- 172.21.15.151:0/4173366394 wait complete. 2024-04-26T02:03:58.524 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 35 2024-04-26T02:03:58.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:03:58 smithi172 ceph-mon[32785]: pgmap v324: 97 pgs: 97 active+clean; 150 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 35 KiB/s wr, 6 op/s 2024-04-26T02:03:58.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:03:58 smithi165 ceph-mon[32648]: pgmap v324: 97 pgs: 97 active+clean; 150 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 35 KiB/s wr, 6 op/s 2024-04-26T02:03:59.384 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24509 quiesce db --set-id 58991260 --release --await' 2024-04-26T02:03:59.385 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24509 quiesce db --set-id 58991260 --release --await 2024-04-26T02:03:59.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:03:59 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/4173366394' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:03:59.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:03:59 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/4173366394' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:03:59.805 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:03:59.847 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:03:59 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/4173366394' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:04:00.499 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:00 smithi151 ceph-mon[28057]: pgmap v325: 97 pgs: 97 active+clean; 150 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 35 KiB/s wr, 8 op/s 2024-04-26T02:04:00.500 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:00 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T02:04:00.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:00 smithi172 ceph-mon[32785]: pgmap v325: 97 pgs: 97 active+clean; 150 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 35 KiB/s wr, 8 op/s 2024-04-26T02:04:00.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:00 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T02:04:00.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:00 smithi165 ceph-mon[32648]: pgmap v325: 97 pgs: 97 active+clean; 150 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 35 KiB/s wr, 8 op/s 2024-04-26T02:04:00.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:00 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T02:04:01.811 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:01 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T02:04:01.811 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:01 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T02:04:01.811 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:01 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T02:04:01.811 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:01 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T02:04:01.811 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:01 smithi151 ceph-mon[28057]: pgmap v326: 97 pgs: 97 active+clean; 150 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.0 KiB/s wr, 5 op/s 2024-04-26T02:04:01.862 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:01 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T02:04:01.863 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:01 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T02:04:01.863 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:01 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T02:04:01.863 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:01 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T02:04:01.863 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:01 smithi165 ceph-mon[32648]: pgmap v326: 97 pgs: 97 active+clean; 150 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.0 KiB/s wr, 5 op/s 2024-04-26T02:04:02.053 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:01 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T02:04:02.053 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:01 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T02:04:02.054 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:01 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T02:04:02.054 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:01 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T02:04:02.054 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:01 smithi172 ceph-mon[32785]: pgmap v326: 97 pgs: 97 active+clean; 150 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.0 KiB/s wr, 5 op/s 2024-04-26T02:04:02.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.096+0000 7f0382e00700 1 -- 172.21.15.151:0/2271017047 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f037c06b0f0 msgr2=0x7f037c10f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:02.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.096+0000 7f0382e00700 1 --2- 172.21.15.151:0/2271017047 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f037c06b0f0 0x7f037c10f900 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f037400b0a0 tx=0x7f037400b3b0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.096+0000 7f0382e00700 1 -- 172.21.15.151:0/2271017047 shutdown_connections 2024-04-26T02:04:02.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.096+0000 7f0382e00700 1 --2- 172.21.15.151:0/2271017047 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f037c06b0f0 0x7f037c10f900 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.096+0000 7f0382e00700 1 --2- 172.21.15.151:0/2271017047 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f037c108a30 0x7f037c108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.096+0000 7f0382e00700 1 --2- 172.21.15.151:0/2271017047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f037c06a750 0x7f037c06abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.096+0000 7f0382e00700 1 -- 172.21.15.151:0/2271017047 >> 172.21.15.151:0/2271017047 conn(0x7f037c0788f0 msgr2=0x7f037c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:02.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.097+0000 7f0382e00700 1 -- 172.21.15.151:0/2271017047 shutdown_connections 2024-04-26T02:04:02.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.097+0000 7f0382e00700 1 -- 172.21.15.151:0/2271017047 wait complete. 2024-04-26T02:04:02.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.098+0000 7f0382e00700 1 Processor -- start 2024-04-26T02:04:02.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.098+0000 7f0382e00700 1 -- start start 2024-04-26T02:04:02.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.098+0000 7f0382e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f037c06a750 0x7f037c1990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:02.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.098+0000 7f0382e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f037c06b0f0 0x7f037c199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:02.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.098+0000 7f0382e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f037c108a30 0x7f037c19d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:02.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.098+0000 7f0382e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f037c112c10 con 0x7f037c06a750 2024-04-26T02:04:02.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.098+0000 7f0382e00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f037c112a90 con 0x7f037c06b0f0 2024-04-26T02:04:02.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.098+0000 7f0382e00700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f037c112d90 con 0x7f037c108a30 2024-04-26T02:04:02.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.098+0000 7f0373e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f037c06b0f0 0x7f037c199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:02.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.098+0000 7f037be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f037c06a750 0x7f037c1990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:02.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.098+0000 7f0373e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f037c06b0f0 0x7f037c199620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:41354/0 (socket says 172.21.15.151:41354) 2024-04-26T02:04:02.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.098+0000 7f0373e00700 1 -- 172.21.15.151:0/2522795261 learned_addr learned my addr 172.21.15.151:0/2522795261 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:04:02.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.099+0000 7f0380e00700 1 --2- 172.21.15.151:0/2522795261 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f037c108a30 0x7f037c19d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:02.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.099+0000 7f0373e00700 1 -- 172.21.15.151:0/2522795261 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f037c108a30 msgr2=0x7f037c19d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:02.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.099+0000 7f0373e00700 1 --2- 172.21.15.151:0/2522795261 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f037c108a30 0x7f037c19d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.099+0000 7f0373e00700 1 -- 172.21.15.151:0/2522795261 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f037c06a750 msgr2=0x7f037c1990e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:02.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.099+0000 7f0373e00700 1 --2- 172.21.15.151:0/2522795261 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f037c06a750 0x7f037c1990e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.099+0000 7f0373e00700 1 -- 172.21.15.151:0/2522795261 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f037c19e0f0 con 0x7f037c06b0f0 2024-04-26T02:04:02.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.099+0000 7f0373e00700 1 --2- 172.21.15.151:0/2522795261 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f037c06b0f0 0x7f037c199620 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f036400e9a0 tx=0x7f036400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:02.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.100+0000 7f0379600700 1 -- 172.21.15.151:0/2522795261 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036400ca10 con 0x7f037c06b0f0 2024-04-26T02:04:02.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.100+0000 7f0379600700 1 -- 172.21.15.151:0/2522795261 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f036400cbb0 con 0x7f037c06b0f0 2024-04-26T02:04:02.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.100+0000 7f0382e00700 1 -- 172.21.15.151:0/2522795261 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f037c19e3e0 con 0x7f037c06b0f0 2024-04-26T02:04:02.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.100+0000 7f0379600700 1 -- 172.21.15.151:0/2522795261 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0364018680 con 0x7f037c06b0f0 2024-04-26T02:04:02.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.100+0000 7f0382e00700 1 -- 172.21.15.151:0/2522795261 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f037c1a5c60 con 0x7f037c06b0f0 2024-04-26T02:04:02.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.103+0000 7f0379600700 1 -- 172.21.15.151:0/2522795261 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f0364010430 con 0x7f037c06b0f0 2024-04-26T02:04:02.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.104+0000 7f0379600700 1 --2- 172.21.15.151:0/2522795261 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f03600a3db0 0x7f03600a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:02.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.104+0000 7f0379600700 1 -- 172.21.15.151:0/2522795261 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8455+0+0 (secure 0 0 0) 0x7f03640d0990 con 0x7f037c06b0f0 2024-04-26T02:04:02.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.104+0000 7f038d87ab80 1 Processor -- start 2024-04-26T02:04:02.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.104+0000 7f037be00700 1 --2- 172.21.15.151:0/2522795261 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f03600a3db0 0x7f03600a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:02.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.105+0000 7f038d87ab80 1 -- start start 2024-04-26T02:04:02.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.105+0000 7f038d87ab80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af45c4b920 0x55af45c4bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:02.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.105+0000 7f038d87ab80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af45c4c240 0x55af45c56bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:02.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.105+0000 7f038d87ab80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af45c570f0 0x55af45c594e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:02.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.105+0000 7f038d87ab80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55af45b9ef90 con 0x55af45c4b920 2024-04-26T02:04:02.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.105+0000 7f038d87ab80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55af45b45830 con 0x55af45c570f0 2024-04-26T02:04:02.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.105+0000 7f038d87ab80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55af45b9af50 con 0x55af45c4c240 2024-04-26T02:04:02.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.105+0000 7f037be00700 1 --2- 172.21.15.151:0/2522795261 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f03600a3db0 0x7f03600a6270 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f036c009680 tx=0x7f036c009260 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:02.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.105+0000 7f037be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af45c4b920 0x55af45c4bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:02.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.105+0000 7f0373e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af45c4c240 0x55af45c56bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:02.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.106+0000 7f037be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af45c4b920 0x55af45c4bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49290/0 (socket says 172.21.15.151:49290) 2024-04-26T02:04:02.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.106+0000 7f037be00700 1 -- 172.21.15.151:0/861557264 learned_addr learned my addr 172.21.15.151:0/861557264 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:04:02.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.106+0000 7f0380e00700 1 --2- 172.21.15.151:0/861557264 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af45c570f0 0x55af45c594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:02.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.106+0000 7f0373e00700 1 -- 172.21.15.151:0/861557264 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af45c570f0 msgr2=0x55af45c594e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:02.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.106+0000 7f0373e00700 1 --2- 172.21.15.151:0/861557264 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af45c570f0 0x55af45c594e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.106+0000 7f0373e00700 1 -- 172.21.15.151:0/861557264 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af45c4b920 msgr2=0x55af45c4bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:02.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.106+0000 7f0373e00700 1 --2- 172.21.15.151:0/861557264 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af45c4b920 0x55af45c4bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.106+0000 7f0373e00700 1 -- 172.21.15.151:0/861557264 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af45b3b780 con 0x55af45c4c240 2024-04-26T02:04:02.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.106+0000 7f0373e00700 1 --2- 172.21.15.151:0/861557264 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af45c4c240 0x55af45c56bb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f03640ce0c0 tx=0x7f036409c580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b76518ed3efd8676 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:02.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.106+0000 7f0371600700 1 -- 172.21.15.151:0/861557264 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03640a5580 con 0x55af45c4c240 2024-04-26T02:04:02.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.107+0000 7f0371600700 1 -- 172.21.15.151:0/861557264 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0364085e50 con 0x55af45c4c240 2024-04-26T02:04:02.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.107+0000 7f0371600700 1 -- 172.21.15.151:0/861557264 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03640a50f0 con 0x55af45c4c240 2024-04-26T02:04:02.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.107+0000 7f038d87ab80 1 -- 172.21.15.151:0/861557264 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af45c4c240 msgr2=0x55af45c56bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:02.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.107+0000 7f038d87ab80 1 --2- 172.21.15.151:0/861557264 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af45c4c240 0x55af45c56bb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f03640ce0c0 tx=0x7f036409c580 comp rx=0 tx=0).stop 2024-04-26T02:04:02.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.107+0000 7f038d87ab80 1 -- 172.21.15.151:0/861557264 shutdown_connections 2024-04-26T02:04:02.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.107+0000 7f038d87ab80 1 --2- 172.21.15.151:0/861557264 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af45c570f0 0x55af45c594e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.107+0000 7f038d87ab80 1 --2- 172.21.15.151:0/861557264 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af45c4b920 0x55af45c4bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.107+0000 7f038d87ab80 1 --2- 172.21.15.151:0/861557264 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af45c4c240 0x55af45c56bb0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.107+0000 7f038d87ab80 1 -- 172.21.15.151:0/861557264 >> 172.21.15.151:0/861557264 conn(0x55af45b7a6a0 msgr2=0x55af45c47dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:02.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.108+0000 7f038d87ab80 1 -- 172.21.15.151:0/861557264 shutdown_connections 2024-04-26T02:04:02.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.108+0000 7f038d87ab80 1 -- 172.21.15.151:0/861557264 wait complete. 2024-04-26T02:04:02.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.108+0000 7f038d87ab80 1 -- start start 2024-04-26T02:04:02.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.109+0000 7f038d87ab80 1 Processor -- start 2024-04-26T02:04:02.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.109+0000 7f038d87ab80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af45c4b920 0x55af45c568c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:02.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.109+0000 7f038d87ab80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af45c4c240 0x55af45c53bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:02.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.109+0000 7f038d87ab80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af45c570f0 0x55af45c54100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:02.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.109+0000 7f038d87ab80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55af45c59c10 con 0x55af45c570f0 2024-04-26T02:04:02.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.109+0000 7f038d87ab80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55af45b58960 con 0x55af45c4b920 2024-04-26T02:04:02.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.109+0000 7f038d87ab80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55af45c59d90 con 0x55af45c4c240 2024-04-26T02:04:02.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.109+0000 7f038d87ab80 10 client.0 _ll_register_callbacks cb 0x55af45b37820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:04:02.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.109+0000 7f037be00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af45c4b920 0x55af45c568c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:02.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.109+0000 7f037be00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af45c4b920 0x55af45c568c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:41366/0 (socket says 172.21.15.151:41366) 2024-04-26T02:04:02.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.109+0000 7f0380e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af45c570f0 0x55af45c54100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:02.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.109+0000 7f037be00700 1 -- 172.21.15.151:0/2740542814 learned_addr learned my addr 172.21.15.151:0/2740542814 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:04:02.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.109+0000 7f0373e00700 1 --2- 172.21.15.151:0/2740542814 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af45c4c240 0x55af45c53bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:02.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.110+0000 7f037be00700 1 -- 172.21.15.151:0/2740542814 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af45c4c240 msgr2=0x55af45c53bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:02.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.110+0000 7f037be00700 1 --2- 172.21.15.151:0/2740542814 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af45c4c240 0x55af45c53bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.110+0000 7f037be00700 1 -- 172.21.15.151:0/2740542814 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af45c570f0 msgr2=0x55af45c54100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:02.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.110+0000 7f037be00700 1 --2- 172.21.15.151:0/2740542814 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af45c570f0 0x55af45c54100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.110+0000 7f037be00700 1 -- 172.21.15.151:0/2740542814 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af45aec9d0 con 0x55af45c4b920 2024-04-26T02:04:02.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.110+0000 7f0380e00700 1 --2- 172.21.15.151:0/2740542814 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af45c570f0 0x55af45c54100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:04:02.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.110+0000 7f037be00700 1 --2- 172.21.15.151:0/2740542814 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af45c4b920 0x55af45c568c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f036c009870 tx=0x7f036c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a642e034c8126468 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:02.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.110+0000 7f036a000700 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T02:04:02.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.110+0000 7f036a000700 1 -- 172.21.15.151:0/2740542814 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036c01c6e0 con 0x55af45c4b920 2024-04-26T02:04:02.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.111+0000 7f038d87ab80 1 -- 172.21.15.151:0/2740542814 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0364085e50 con 0x55af45c4b920 2024-04-26T02:04:02.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.111+0000 7f036a000700 1 -- 172.21.15.151:0/2740542814 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8455+0+0 (secure 0 0 0) 0x7f036c009040 con 0x55af45c4b920 2024-04-26T02:04:02.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.111+0000 7f036a000700 1 -- 172.21.15.151:0/2740542814 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f036c01c880 con 0x55af45c4b920 2024-04-26T02:04:02.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.111+0000 7f038d87ab80 10 client.26396 fetch_fsmap learned FSMap version 35 2024-04-26T02:04:02.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.111+0000 7f038d87ab80 1 -- 172.21.15.151:0/2740542814 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55af45b35bf0 con 0x55af45c4b920 2024-04-26T02:04:02.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.111+0000 7f036a000700 1 -- 172.21.15.151:0/2740542814 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0364085e50 con 0x55af45c4b920 2024-04-26T02:04:02.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.111+0000 7f036a000700 1 -- 172.21.15.151:0/2740542814 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 35) ==== 8338+0+0 (secure 0 0 0) 0x7f036c027b20 con 0x55af45c4b920 2024-04-26T02:04:02.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.111+0000 7f038d87ab80 10 client.26396 fetch_fsmap finished waiting for FSMap version 35 2024-04-26T02:04:02.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.111+0000 7f038d87ab80 10 client.26396 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-26T02:04:02.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.111+0000 7f038d87ab80 20 client.26396 populate_metadata read hostname 'smithi151' 2024-04-26T02:04:02.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.111+0000 7f038d87ab80 1 --2- 172.21.15.151:0/2740542814 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55af45c66980 0x55af45c68d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:02.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.111+0000 7f038d87ab80 4 client.26396 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:04:02.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.111+0000 7f038d87ab80 1 -- 172.21.15.151:0/2740542814 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af45bac780 con 0x55af45c66980 2024-04-26T02:04:02.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.112+0000 7f0373e00700 1 --2- 172.21.15.151:0/2740542814 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55af45c66980 0x55af45c68d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:02.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.112+0000 7f036a000700 10 client.26396 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:04:02.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.112+0000 7f0373e00700 1 --2- 172.21.15.151:0/2740542814 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55af45c66980 0x55af45c68d70 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a66d438185237fd6 server_cookie=c47c844483b0a356 in_seq=0 out_seq=0 2024-04-26T02:04:02.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.113+0000 7f036a000700 1 -- 172.21.15.151:0/2740542814 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55af45bac780 con 0x55af45c66980 2024-04-26T02:04:02.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.113+0000 7f036a000700 10 client.26396 handle_command_reply: tid=0 2024-04-26T02:04:02.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.113+0000 7f038d87ab80 1 client.26396 shutdown 2024-04-26T02:04:02.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.114+0000 7f038d87ab80 1 -- 172.21.15.151:0/2740542814 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af45c4b920 msgr2=0x55af45c568c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:02.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.114+0000 7f038d87ab80 1 --2- 172.21.15.151:0/2740542814 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af45c4b920 0x55af45c568c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f036c009870 tx=0x7f036c0047c0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.114+0000 7f038d87ab80 1 -- 172.21.15.151:0/2740542814 shutdown_connections 2024-04-26T02:04:02.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.114+0000 7f038d87ab80 1 --2- 172.21.15.151:0/2740542814 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af45c570f0 0x55af45c54100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.114+0000 7f038d87ab80 1 --2- 172.21.15.151:0/2740542814 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55af45c66980 0x55af45c68d70 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.114+0000 7f038d87ab80 1 --2- 172.21.15.151:0/2740542814 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af45c4b920 0x55af45c568c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.114+0000 7f038d87ab80 1 --2- 172.21.15.151:0/2740542814 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af45c4c240 0x55af45c53bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.114+0000 7f036a000700 0 client.26396 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:04:02.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.114+0000 7f038d87ab80 1 -- 172.21.15.151:0/2740542814 >> 172.21.15.151:0/2740542814 conn(0x55af45b7a6a0 msgr2=0x55af45c479c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:02.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.114+0000 7f038d87ab80 1 -- 172.21.15.151:0/2740542814 shutdown_connections 2024-04-26T02:04:02.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.114+0000 7f038d87ab80 1 -- 172.21.15.151:0/2740542814 wait complete. 2024-04-26T02:04:02.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.114+0000 7f038d87ab80 20 client.26396 trim_cache size 0 max 16384 2024-04-26T02:04:02.135 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.136+0000 7f038d87ab80 1 Processor -- start 2024-04-26T02:04:02.135 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.136+0000 7f038d87ab80 1 -- start start 2024-04-26T02:04:02.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.136+0000 7f038d87ab80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af45c4b920 0x55af45c6e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:02.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.136+0000 7f038d87ab80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af45c4c240 0x55af45c6eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:02.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.136+0000 7f038d87ab80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af45c570f0 0x55af45c50e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:02.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.136+0000 7f038d87ab80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f037c1ad9c0 con 0x55af45c4c240 2024-04-26T02:04:02.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.136+0000 7f038d87ab80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f037c1ab250 con 0x55af45c570f0 2024-04-26T02:04:02.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.136+0000 7f038d87ab80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f037c1a90e0 con 0x55af45c4b920 2024-04-26T02:04:02.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.136+0000 7f0380e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af45c570f0 0x55af45c50e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:02.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.136+0000 7f037be00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af45c4b920 0x55af45c6e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:02.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.136+0000 7f037be00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af45c4b920 0x55af45c6e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:44076/0 (socket says 172.21.15.151:44076) 2024-04-26T02:04:02.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.136+0000 7f0380e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af45c570f0 0x55af45c50e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:41372/0 (socket says 172.21.15.151:41372) 2024-04-26T02:04:02.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.136+0000 7f037be00700 1 -- 172.21.15.151:0/871233678 learned_addr learned my addr 172.21.15.151:0/871233678 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:04:02.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.136+0000 7f0373e00700 1 --2- 172.21.15.151:0/871233678 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af45c4c240 0x55af45c6eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:02.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.137+0000 7f037be00700 1 -- 172.21.15.151:0/871233678 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af45c570f0 msgr2=0x55af45c50e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:02.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.137+0000 7f037be00700 1 --2- 172.21.15.151:0/871233678 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af45c570f0 0x55af45c50e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.137+0000 7f037be00700 1 -- 172.21.15.151:0/871233678 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af45c4c240 msgr2=0x55af45c6eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:02.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.137+0000 7f037be00700 1 --2- 172.21.15.151:0/871233678 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af45c4c240 0x55af45c6eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.137+0000 7f037be00700 1 -- 172.21.15.151:0/871233678 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af45c693d0 con 0x55af45c4b920 2024-04-26T02:04:02.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.137+0000 7f037be00700 1 --2- 172.21.15.151:0/871233678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af45c4b920 0x55af45c6e620 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f036c028ff0 tx=0x7f036c009980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ea563d0dde52b47 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:02.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.137+0000 7f0370c00700 1 -- 172.21.15.151:0/871233678 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036c034030 con 0x55af45c4b920 2024-04-26T02:04:02.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.137+0000 7f0370c00700 1 -- 172.21.15.151:0/871233678 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f036c036070 con 0x55af45c4b920 2024-04-26T02:04:02.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.137+0000 7f0370c00700 1 -- 172.21.15.151:0/871233678 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036c027aa0 con 0x55af45c4b920 2024-04-26T02:04:02.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.138+0000 7f038d87ab80 1 -- 172.21.15.151:0/871233678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af45c4b920 msgr2=0x55af45c6e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:02.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.138+0000 7f038d87ab80 1 --2- 172.21.15.151:0/871233678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af45c4b920 0x55af45c6e620 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f036c028ff0 tx=0x7f036c009980 comp rx=0 tx=0).stop 2024-04-26T02:04:02.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.138+0000 7f038d87ab80 1 -- 172.21.15.151:0/871233678 shutdown_connections 2024-04-26T02:04:02.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.138+0000 7f038d87ab80 1 --2- 172.21.15.151:0/871233678 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af45c570f0 0x55af45c50e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.138+0000 7f038d87ab80 1 --2- 172.21.15.151:0/871233678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af45c4b920 0x55af45c6e620 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.138+0000 7f038d87ab80 1 --2- 172.21.15.151:0/871233678 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af45c4c240 0x55af45c6eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.138+0000 7f038d87ab80 1 -- 172.21.15.151:0/871233678 >> 172.21.15.151:0/871233678 conn(0x55af45b7a6a0 msgr2=0x55af45c55c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:02.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.138+0000 7f038d87ab80 1 -- 172.21.15.151:0/871233678 shutdown_connections 2024-04-26T02:04:02.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.138+0000 7f038d87ab80 1 -- 172.21.15.151:0/871233678 wait complete. 2024-04-26T02:04:02.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.139+0000 7f038d87ab80 1 -- start start 2024-04-26T02:04:02.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.139+0000 7f038d87ab80 1 Processor -- start 2024-04-26T02:04:02.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.139+0000 7f038d87ab80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af45c4b920 0x55af45c5f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:02.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.139+0000 7f038d87ab80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af45c4c240 0x55af45c5d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:02.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.140+0000 7f038d87ab80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af45c570f0 0x55af45c5da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:02.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.140+0000 7f038d87ab80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f037c1a9730 con 0x55af45c4b920 2024-04-26T02:04:02.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.140+0000 7f038d87ab80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f037c1aab80 con 0x55af45c570f0 2024-04-26T02:04:02.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.140+0000 7f038d87ab80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f037c1a9de0 con 0x55af45c4c240 2024-04-26T02:04:02.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.140+0000 7f038d87ab80 10 client.0 _ll_register_callbacks cb 0x55af45b37820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:04:02.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.140+0000 7f0373e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af45c4c240 0x55af45c5d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:02.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.140+0000 7f0373e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af45c4c240 0x55af45c5d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:44086/0 (socket says 172.21.15.151:44086) 2024-04-26T02:04:02.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.140+0000 7f0373e00700 1 -- 172.21.15.151:0/990297465 learned_addr learned my addr 172.21.15.151:0/990297465 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:04:02.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.140+0000 7f0380e00700 1 --2- 172.21.15.151:0/990297465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af45c570f0 0x55af45c5da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:02.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.140+0000 7f037be00700 1 --2- 172.21.15.151:0/990297465 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af45c4b920 0x55af45c5f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:02.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.140+0000 7f0373e00700 1 -- 172.21.15.151:0/990297465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af45c570f0 msgr2=0x55af45c5da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:02.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.140+0000 7f0373e00700 1 --2- 172.21.15.151:0/990297465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af45c570f0 0x55af45c5da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.140+0000 7f0373e00700 1 -- 172.21.15.151:0/990297465 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af45c4b920 msgr2=0x55af45c5f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:02.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.140+0000 7f0373e00700 1 --2- 172.21.15.151:0/990297465 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af45c4b920 0x55af45c5f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.140+0000 7f0373e00700 1 -- 172.21.15.151:0/990297465 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af45c5df60 con 0x55af45c4c240 2024-04-26T02:04:02.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.140+0000 7f037be00700 1 --2- 172.21.15.151:0/990297465 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af45c4b920 0x55af45c5f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:04:02.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.140+0000 7f0373e00700 1 --2- 172.21.15.151:0/990297465 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af45c4c240 0x55af45c5d4e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f036409c610 tx=0x7f0364010c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a46e3fa17270bf0a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:02.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.140+0000 7f036a000700 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-26T02:04:02.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.141+0000 7f036a000700 1 -- 172.21.15.151:0/990297465 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0364098bb0 con 0x55af45c4c240 2024-04-26T02:04:02.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.141+0000 7f036a000700 1 -- 172.21.15.151:0/990297465 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8455+0+0 (secure 0 0 0) 0x7f03640872b0 con 0x55af45c4c240 2024-04-26T02:04:02.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.141+0000 7f038d87ab80 1 -- 172.21.15.151:0/990297465 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f036c036070 con 0x55af45c4c240 2024-04-26T02:04:02.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.141+0000 7f036a000700 1 -- 172.21.15.151:0/990297465 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f03640a5580 con 0x55af45c4c240 2024-04-26T02:04:02.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.141+0000 7f038d87ab80 10 client.26422 fetch_fsmap learned FSMap version 35 2024-04-26T02:04:02.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.141+0000 7f038d87ab80 1 -- 172.21.15.151:0/990297465 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55af45ccdc10 con 0x55af45c4c240 2024-04-26T02:04:02.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.141+0000 7f036a000700 1 -- 172.21.15.151:0/990297465 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f036c036070 con 0x55af45c4c240 2024-04-26T02:04:02.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.141+0000 7f036a000700 1 -- 172.21.15.151:0/990297465 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 35) ==== 8338+0+0 (secure 0 0 0) 0x7f03640bb020 con 0x55af45c4c240 2024-04-26T02:04:02.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.141+0000 7f038d87ab80 10 client.26422 fetch_fsmap finished waiting for FSMap version 35 2024-04-26T02:04:02.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.141+0000 7f038d87ab80 10 client.26422 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-26T02:04:02.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.141+0000 7f038d87ab80 20 client.26422 populate_metadata read hostname 'smithi151' 2024-04-26T02:04:02.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.141+0000 7f038d87ab80 1 --2- 172.21.15.151:0/990297465 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55af45ccdea0 0x55af45cd0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:02.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.141+0000 7f038d87ab80 4 client.26422 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "58991260", "release": true, "await": true}] 2024-04-26T02:04:02.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.141+0000 7f038d87ab80 1 -- 172.21.15.151:0/990297465 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "quiesce db", "set_id": "58991260", "release": true, "await": true}) -- 0x55af45ba6820 con 0x55af45ccdea0 2024-04-26T02:04:02.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.142+0000 7f037be00700 1 --2- 172.21.15.151:0/990297465 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55af45ccdea0 0x55af45cd0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:02.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.142+0000 7f037be00700 1 --2- 172.21.15.151:0/990297465 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55af45ccdea0 0x55af45cd0290 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41f4a1133f2492be server_cookie=8a8baf72872aa841 in_seq=0 out_seq=0 2024-04-26T02:04:02.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.142+0000 7f036a000700 10 client.26422 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:04:02.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.374+0000 7f036a000700 1 -- 172.21.15.151:0/990297465 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55af45ba6820 con 0x55af45ccdea0 2024-04-26T02:04:02.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.374+0000 7f036a000700 10 client.26422 handle_command_reply: tid=0 2024-04-26T02:04:02.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.374+0000 7f038d87ab80 1 client.26422 shutdown 2024-04-26T02:04:02.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.374+0000 7f038d87ab80 1 -- 172.21.15.151:0/990297465 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af45c4c240 msgr2=0x55af45c5d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:02.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.374+0000 7f038d87ab80 1 --2- 172.21.15.151:0/990297465 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af45c4c240 0x55af45c5d4e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f036409c610 tx=0x7f0364010c10 comp rx=0 tx=0).stop 2024-04-26T02:04:02.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.375+0000 7f038d87ab80 1 -- 172.21.15.151:0/990297465 shutdown_connections 2024-04-26T02:04:02.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.375+0000 7f038d87ab80 1 --2- 172.21.15.151:0/990297465 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55af45ccdea0 0x55af45cd0290 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.375+0000 7f038d87ab80 1 --2- 172.21.15.151:0/990297465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af45c570f0 0x55af45c5da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.375+0000 7f038d87ab80 1 --2- 172.21.15.151:0/990297465 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af45c4b920 0x55af45c5f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.375+0000 7f038d87ab80 1 --2- 172.21.15.151:0/990297465 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af45c4c240 0x55af45c5d4e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.375+0000 7f038d87ab80 1 -- 172.21.15.151:0/990297465 >> 172.21.15.151:0/990297465 conn(0x55af45b7a6a0 msgr2=0x55af45c524d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:02.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.375+0000 7f036a000700 0 client.26422 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:04:02.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.375+0000 7f038d87ab80 1 -- 172.21.15.151:0/990297465 shutdown_connections 2024-04-26T02:04:02.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.375+0000 7f038d87ab80 1 -- 172.21.15.151:0/990297465 wait complete. 2024-04-26T02:04:02.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.375+0000 7f038d87ab80 20 client.26422 trim_cache size 0 max 16384 2024-04-26T02:04:02.376 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-26T02:04:02.376 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 35, 2024-04-26T02:04:02.376 INFO:teuthology.orchestra.run.smithi151.stdout: "leader": 24509, 2024-04-26T02:04:02.376 INFO:teuthology.orchestra.run.smithi151.stdout: "set_version": 32, 2024-04-26T02:04:02.376 INFO:teuthology.orchestra.run.smithi151.stdout: "sets": { 2024-04-26T02:04:02.376 INFO:teuthology.orchestra.run.smithi151.stdout: "58991260": { 2024-04-26T02:04:02.376 INFO:teuthology.orchestra.run.smithi151.stdout: "version": 32, 2024-04-26T02:04:02.376 INFO:teuthology.orchestra.run.smithi151.stdout: "age_ref": 0.0, 2024-04-26T02:04:02.376 INFO:teuthology.orchestra.run.smithi151.stdout: "state": { 2024-04-26T02:04:02.376 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "RELEASED", 2024-04-26T02:04:02.376 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 0.0 2024-04-26T02:04:02.377 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:04:02.377 INFO:teuthology.orchestra.run.smithi151.stdout: "timeout": 60.0, 2024-04-26T02:04:02.377 INFO:teuthology.orchestra.run.smithi151.stdout: "expiration": 82.1, 2024-04-26T02:04:02.377 INFO:teuthology.orchestra.run.smithi151.stdout: "members": { 2024-04-26T02:04:02.377 INFO:teuthology.orchestra.run.smithi151.stdout: "file:/": { 2024-04-26T02:04:02.377 INFO:teuthology.orchestra.run.smithi151.stdout: "excluded": false, 2024-04-26T02:04:02.377 INFO:teuthology.orchestra.run.smithi151.stdout: "state": { 2024-04-26T02:04:02.377 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "RELEASED", 2024-04-26T02:04:02.377 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 0.0 2024-04-26T02:04:02.377 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:04:02.377 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:04:02.377 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:04:02.377 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:04:02.377 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:04:02.377 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-26T02:04:02.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.376+0000 7f0382e00700 1 -- 172.21.15.151:0/2522795261 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f03600a3db0 msgr2=0x7f03600a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:02.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.376+0000 7f0382e00700 1 --2- 172.21.15.151:0/2522795261 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f03600a3db0 0x7f03600a6270 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f036c009680 tx=0x7f036c009260 comp rx=0 tx=0).stop 2024-04-26T02:04:02.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.376+0000 7f0382e00700 1 -- 172.21.15.151:0/2522795261 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f037c06b0f0 msgr2=0x7f037c199620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:02.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.376+0000 7f0382e00700 1 --2- 172.21.15.151:0/2522795261 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f037c06b0f0 0x7f037c199620 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f036400e9a0 tx=0x7f036400ecb0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.376+0000 7f0382e00700 1 -- 172.21.15.151:0/2522795261 shutdown_connections 2024-04-26T02:04:02.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.376+0000 7f0382e00700 1 --2- 172.21.15.151:0/2522795261 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f03600a3db0 0x7f03600a6270 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.376+0000 7f0382e00700 1 --2- 172.21.15.151:0/2522795261 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f037c108a30 0x7f037c19d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.376+0000 7f0382e00700 1 --2- 172.21.15.151:0/2522795261 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f037c06a750 0x7f037c1990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.376+0000 7f0382e00700 1 --2- 172.21.15.151:0/2522795261 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f037c06b0f0 0x7f037c199620 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.376+0000 7f0382e00700 1 -- 172.21.15.151:0/2522795261 >> 172.21.15.151:0/2522795261 conn(0x7f037c0788f0 msgr2=0x7f037c0ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:02.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.377+0000 7f0382e00700 1 -- 172.21.15.151:0/2522795261 shutdown_connections 2024-04-26T02:04:02.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:04:02.377+0000 7f0382e00700 1 -- 172.21.15.151:0/2522795261 wait complete. 2024-04-26T02:04:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.645+0000 7f0d4aa00640 1 -- 172.21.15.151:0/65585290 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d441031e0 msgr2=0x7f0d441035c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.645+0000 7f0d4aa00640 1 --2- 172.21.15.151:0/65585290 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d441031e0 0x7f0d441035c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0d34009900 tx=0x7f0d3402a510 comp rx=0 tx=0).stop 2024-04-26T02:04:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.645+0000 7f0d4aa00640 1 -- 172.21.15.151:0/65585290 shutdown_connections 2024-04-26T02:04:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.645+0000 7f0d4aa00640 1 --2- 172.21.15.151:0/65585290 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d4410c520 0x7f0d4410e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.645+0000 7f0d4aa00640 1 --2- 172.21.15.151:0/65585290 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d44103b00 0x7f0d4410bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.645+0000 7f0d4aa00640 1 --2- 172.21.15.151:0/65585290 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d441031e0 0x7f0d441035c0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.645+0000 7f0d4aa00640 1 -- 172.21.15.151:0/65585290 >> 172.21.15.151:0/65585290 conn(0x7f0d44077fd0 msgr2=0x7f0d440783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.645+0000 7f0d4aa00640 1 -- 172.21.15.151:0/65585290 shutdown_connections 2024-04-26T02:04:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.645+0000 7f0d4aa00640 1 -- 172.21.15.151:0/65585290 wait complete. 2024-04-26T02:04:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.646+0000 7f0d4aa00640 1 Processor -- start 2024-04-26T02:04:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.646+0000 7f0d4aa00640 1 -- start start 2024-04-26T02:04:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.646+0000 7f0d4aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d441031e0 0x7f0d441a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.646+0000 7f0d4aa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d44103b00 0x7f0d441a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.646+0000 7f0d4aa00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d4410c520 0x7f0d4419a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.647+0000 7f0d4aa00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0d44112200 con 0x7f0d44103b00 2024-04-26T02:04:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.647+0000 7f0d4aa00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0d44112080 con 0x7f0d441031e0 2024-04-26T02:04:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.647+0000 7f0d4aa00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0d44112380 con 0x7f0d4410c520 2024-04-26T02:04:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.647+0000 7f0d43e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d441031e0 0x7f0d441a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.647+0000 7f0d43e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d441031e0 0x7f0d441a0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:41388/0 (socket says 172.21.15.151:41388) 2024-04-26T02:04:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.647+0000 7f0d43e00640 1 -- 172.21.15.151:0/1295426949 learned_addr learned my addr 172.21.15.151:0/1295426949 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:04:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.647+0000 7f0d43400640 1 --2- 172.21.15.151:0/1295426949 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d44103b00 0x7f0d441a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.647+0000 7f0d48a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d4410c520 0x7f0d4419a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.647+0000 7f0d43e00640 1 -- 172.21.15.151:0/1295426949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d4410c520 msgr2=0x7f0d4419a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.647+0000 7f0d43e00640 1 --2- 172.21.15.151:0/1295426949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d4410c520 0x7f0d4419a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.647+0000 7f0d43e00640 1 -- 172.21.15.151:0/1295426949 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d44103b00 msgr2=0x7f0d441a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.647+0000 7f0d43e00640 1 --2- 172.21.15.151:0/1295426949 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d44103b00 0x7f0d441a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.647+0000 7f0d43e00640 1 -- 172.21.15.151:0/1295426949 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d4419b020 con 0x7f0d441031e0 2024-04-26T02:04:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.647+0000 7f0d48a00640 1 --2- 172.21.15.151:0/1295426949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d4410c520 0x7f0d4419a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:04:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.648+0000 7f0d43e00640 1 --2- 172.21.15.151:0/1295426949 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d441031e0 0x7f0d441a0650 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0d34005ab0 tx=0x7f0d34003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.648+0000 7f0d40c00640 1 -- 172.21.15.151:0/1295426949 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3403f070 con 0x7f0d441031e0 2024-04-26T02:04:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.648+0000 7f0d40c00640 1 -- 172.21.15.151:0/1295426949 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d34031b60 con 0x7f0d441031e0 2024-04-26T02:04:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.648+0000 7f0d40c00640 1 -- 172.21.15.151:0/1295426949 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d34035030 con 0x7f0d441031e0 2024-04-26T02:04:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.648+0000 7f0d4aa00640 1 -- 172.21.15.151:0/1295426949 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d4419b2b0 con 0x7f0d441031e0 2024-04-26T02:04:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.648+0000 7f0d4aa00640 1 -- 172.21.15.151:0/1295426949 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d441a73f0 con 0x7f0d441031e0 2024-04-26T02:04:02.650 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.651+0000 7f0d40c00640 1 -- 172.21.15.151:0/1295426949 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f0d34042070 con 0x7f0d441031e0 2024-04-26T02:04:02.651 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.652+0000 7f0d4d775740 1 Processor -- start 2024-04-26T02:04:02.651 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.652+0000 7f0d4d775740 1 -- start start 2024-04-26T02:04:02.651 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.652+0000 7f0d4d775740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c87121f760 0x55c87121fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.652+0000 7f0d4d775740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c871220080 0x55c871232cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.652+0000 7f0d4d775740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c871233210 0x55c871235600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.652+0000 7f0d4d775740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c871185900 con 0x55c87121f760 2024-04-26T02:04:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.652+0000 7f0d4d775740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55c87112bef0 con 0x55c871220080 2024-04-26T02:04:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.652+0000 7f0d4d775740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c8711818c0 con 0x55c871233210 2024-04-26T02:04:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.653+0000 7f0d43e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c871220080 0x55c871232cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.653+0000 7f0d43e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c871220080 0x55c871232cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:41400/0 (socket says 172.21.15.151:41400) 2024-04-26T02:04:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.653+0000 7f0d43e00640 1 -- 172.21.15.151:0/1020243343 learned_addr learned my addr 172.21.15.151:0/1020243343 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:04:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.653+0000 7f0d43400640 1 --2- 172.21.15.151:0/1020243343 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c871233210 0x55c871235600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.653+0000 7f0d48a00640 1 --2- 172.21.15.151:0/1020243343 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c87121f760 0x55c87121fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.653+0000 7f0d43e00640 1 -- 172.21.15.151:0/1020243343 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c871233210 msgr2=0x55c871235600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.653+0000 7f0d43e00640 1 --2- 172.21.15.151:0/1020243343 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c871233210 0x55c871235600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.653+0000 7f0d43e00640 1 -- 172.21.15.151:0/1020243343 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c87121f760 msgr2=0x55c87121fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.653+0000 7f0d43e00640 1 --2- 172.21.15.151:0/1020243343 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c87121f760 0x55c87121fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.653+0000 7f0d43e00640 1 -- 172.21.15.151:0/1020243343 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c870e98210 con 0x55c871220080 2024-04-26T02:04:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.653+0000 7f0d48a00640 1 --2- 172.21.15.151:0/1020243343 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c87121f760 0x55c87121fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:04:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.653+0000 7f0d40c00640 1 --2- 172.21.15.151:0/1295426949 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f0d280a3e40 0x7f0d280a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.654+0000 7f0d40c00640 1 -- 172.21.15.151:0/1295426949 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8455+0+0 (secure 0 0 0) 0x7f0d340f0e60 con 0x7f0d441031e0 2024-04-26T02:04:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.654+0000 7f0d43e00640 1 --2- 172.21.15.151:0/1020243343 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c871220080 0x55c871232cd0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0d340ee4d0 tx=0x7f0d340f90e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd78e0ee400e320c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.654+0000 7f0d48a00640 1 --2- 172.21.15.151:0/1295426949 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f0d280a3e40 0x7f0d280a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.654+0000 7f0d39200640 1 -- 172.21.15.151:0/1020243343 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d34049070 con 0x55c871220080 2024-04-26T02:04:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.654+0000 7f0d39200640 1 -- 172.21.15.151:0/1020243343 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d340bce90 con 0x55c871220080 2024-04-26T02:04:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.654+0000 7f0d39200640 1 -- 172.21.15.151:0/1020243343 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d340bc7f0 con 0x55c871220080 2024-04-26T02:04:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.655+0000 7f0d4d775740 1 -- 172.21.15.151:0/1020243343 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c871220080 msgr2=0x55c871232cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.655+0000 7f0d4d775740 1 --2- 172.21.15.151:0/1020243343 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c871220080 0x55c871232cd0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0d340ee4d0 tx=0x7f0d340f90e0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.655+0000 7f0d48a00640 1 --2- 172.21.15.151:0/1295426949 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f0d280a3e40 0x7f0d280a6300 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c005fd0 tx=0x7f0d3c009500 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.655+0000 7f0d4d775740 1 -- 172.21.15.151:0/1020243343 shutdown_connections 2024-04-26T02:04:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.655+0000 7f0d4d775740 1 --2- 172.21.15.151:0/1020243343 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c871233210 0x55c871235600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.655+0000 7f0d4d775740 1 --2- 172.21.15.151:0/1020243343 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c871220080 0x55c871232cd0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.655+0000 7f0d4d775740 1 --2- 172.21.15.151:0/1020243343 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c87121f760 0x55c87121fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.655+0000 7f0d4d775740 1 -- 172.21.15.151:0/1020243343 >> 172.21.15.151:0/1020243343 conn(0x55c87121bd30 msgr2=0x55c87121c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.656+0000 7f0d4d775740 1 -- 172.21.15.151:0/1020243343 shutdown_connections 2024-04-26T02:04:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.656+0000 7f0d4d775740 1 -- 172.21.15.151:0/1020243343 wait complete. 2024-04-26T02:04:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.658+0000 7f0d4d775740 1 -- start start 2024-04-26T02:04:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.659+0000 7f0d4d775740 1 Processor -- start 2024-04-26T02:04:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.661+0000 7f0d4d775740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c87121f760 0x55c871230190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.661+0000 7f0d4d775740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c871220080 0x55c8712306d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.661+0000 7f0d4d775740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c871233210 0x55c871230c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.661+0000 7f0d4d775740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c871235ba0 con 0x55c871220080 2024-04-26T02:04:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.661+0000 7f0d4d775740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55c87113f310 con 0x55c871233210 2024-04-26T02:04:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.661+0000 7f0d4d775740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c871235d20 con 0x55c87121f760 2024-04-26T02:04:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.661+0000 7f0d4d775740 10 client.0 _ll_register_callbacks cb 0x55c87110dcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:04:02.661 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.662+0000 7f0d43e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c87121f760 0x55c871230190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:02.661 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.662+0000 7f0d43400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c871220080 0x55c8712306d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:02.661 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.662+0000 7f0d43e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c87121f760 0x55c871230190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:44110/0 (socket says 172.21.15.151:44110) 2024-04-26T02:04:02.661 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.662+0000 7f0d43e00640 1 -- 172.21.15.151:0/3736545283 learned_addr learned my addr 172.21.15.151:0/3736545283 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:04:02.661 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.662+0000 7f0d48a00640 1 --2- 172.21.15.151:0/3736545283 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c871233210 0x55c871230c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.662+0000 7f0d43400640 1 -- 172.21.15.151:0/3736545283 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c87121f760 msgr2=0x55c871230190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.662+0000 7f0d43400640 1 --2- 172.21.15.151:0/3736545283 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c87121f760 0x55c871230190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.662+0000 7f0d43400640 1 -- 172.21.15.151:0/3736545283 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c871233210 msgr2=0x55c871230c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.662+0000 7f0d43400640 1 --2- 172.21.15.151:0/3736545283 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c871233210 0x55c871230c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.662+0000 7f0d43400640 1 -- 172.21.15.151:0/3736545283 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c870f64620 con 0x55c871220080 2024-04-26T02:04:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.663+0000 7f0d43400640 1 --2- 172.21.15.151:0/3736545283 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c871220080 0x55c8712306d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x55c871222340 tx=0x7f0d30012b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5354d2887e421108 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.663+0000 7f0d2d600640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-04-26T02:04:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.664+0000 7f0d2d600640 1 -- 172.21.15.151:0/3736545283 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d30004020 con 0x55c871220080 2024-04-26T02:04:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.664+0000 7f0d4d775740 1 -- 172.21.15.151:0/3736545283 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d340bce90 con 0x55c871220080 2024-04-26T02:04:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.664+0000 7f0d2d600640 1 -- 172.21.15.151:0/3736545283 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8455+0+0 (secure 0 0 0) 0x7f0d300041c0 con 0x55c871220080 2024-04-26T02:04:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.664+0000 7f0d2d600640 1 -- 172.21.15.151:0/3736545283 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f0d300043c0 con 0x55c871220080 2024-04-26T02:04:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.664+0000 7f0d4d775740 10 client.16314 fetch_fsmap learned FSMap version 35 2024-04-26T02:04:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.664+0000 7f0d4d775740 1 -- 172.21.15.151:0/3736545283 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55c871241e40 con 0x55c871220080 2024-04-26T02:04:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.664+0000 7f0d2d600640 1 -- 172.21.15.151:0/3736545283 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d340bce90 con 0x55c871220080 2024-04-26T02:04:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.665+0000 7f0d2d600640 1 -- 172.21.15.151:0/3736545283 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 35) ==== 8338+0+0 (secure 0 0 0) 0x7f0d30003a30 con 0x55c871220080 2024-04-26T02:04:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.665+0000 7f0d4d775740 10 client.16314 fetch_fsmap finished waiting for FSMap version 35 2024-04-26T02:04:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.665+0000 7f0d4d775740 10 client.16314 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:04:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.665+0000 7f0d4d775740 20 client.16314 populate_metadata read hostname 'smithi151' 2024-04-26T02:04:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.665+0000 7f0d4d775740 1 --2- 172.21.15.151:0/3736545283 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55c871243300 0x55c871245710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.665+0000 7f0d4d775740 4 client.16314 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:04:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.665+0000 7f0d4d775740 1 -- 172.21.15.151:0/3736545283 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c8711998b0 con 0x55c871243300 2024-04-26T02:04:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.665+0000 7f0d43e00640 1 --2- 172.21.15.151:0/3736545283 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55c871243300 0x55c871245710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.666+0000 7f0d43e00640 1 --2- 172.21.15.151:0/3736545283 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55c871243300 0x55c871245710 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cae1de923a3e38a server_cookie=e0549563a0e8d181 in_seq=0 out_seq=0 2024-04-26T02:04:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.666+0000 7f0d2d600640 10 client.16314 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:04:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.666+0000 7f0d2d600640 1 -- 172.21.15.151:0/3736545283 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c8711998b0 con 0x55c871243300 2024-04-26T02:04:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.666+0000 7f0d2d600640 10 client.16314 handle_command_reply: tid=0 2024-04-26T02:04:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.667+0000 7f0d4d775740 1 client.16314 shutdown 2024-04-26T02:04:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.667+0000 7f0d4d775740 1 -- 172.21.15.151:0/3736545283 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c871220080 msgr2=0x55c8712306d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.667+0000 7f0d4d775740 1 --2- 172.21.15.151:0/3736545283 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c871220080 0x55c8712306d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x55c871222340 tx=0x7f0d30012b60 comp rx=0 tx=0).stop 2024-04-26T02:04:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.667+0000 7f0d4d775740 1 -- 172.21.15.151:0/3736545283 shutdown_connections 2024-04-26T02:04:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.667+0000 7f0d4d775740 1 --2- 172.21.15.151:0/3736545283 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55c871243300 0x55c871245710 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.667+0000 7f0d4d775740 1 --2- 172.21.15.151:0/3736545283 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c871233210 0x55c871230c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.667+0000 7f0d4d775740 1 --2- 172.21.15.151:0/3736545283 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c871220080 0x55c8712306d0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.667+0000 7f0d4d775740 1 --2- 172.21.15.151:0/3736545283 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c87121f760 0x55c871230190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.667+0000 7f0d4d775740 1 -- 172.21.15.151:0/3736545283 >> 172.21.15.151:0/3736545283 conn(0x55c87121bd30 msgr2=0x55c871221a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.667+0000 7f0d2d600640 0 client.16314 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:04:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.667+0000 7f0d4d775740 1 -- 172.21.15.151:0/3736545283 shutdown_connections 2024-04-26T02:04:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.667+0000 7f0d4d775740 1 -- 172.21.15.151:0/3736545283 wait complete. 2024-04-26T02:04:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.667+0000 7f0d4d775740 20 client.16314 trim_cache size 0 max 16384 2024-04-26T02:04:02.684 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.685+0000 7f0d4d775740 1 Processor -- start 2024-04-26T02:04:02.685 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.686+0000 7f0d4d775740 1 -- start start 2024-04-26T02:04:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.687+0000 7f0d4d775740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c87121f760 0x55c87123ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.687+0000 7f0d4d775740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c871220080 0x55c871220860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.687+0000 7f0d4d775740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c871233210 0x55c87124b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.687+0000 7f0d4d775740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0d441ae0d0 con 0x55c871233210 2024-04-26T02:04:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.687+0000 7f0d4d775740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0d441ab9a0 con 0x55c871220080 2024-04-26T02:04:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.687+0000 7f0d4d775740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0d441a9830 con 0x55c87121f760 2024-04-26T02:04:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.687+0000 7f0d43e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c87121f760 0x55c87123ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.687+0000 7f0d43e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c87121f760 0x55c87123ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:44114/0 (socket says 172.21.15.151:44114) 2024-04-26T02:04:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.687+0000 7f0d43e00640 1 -- 172.21.15.151:0/313292240 learned_addr learned my addr 172.21.15.151:0/313292240 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:04:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.688+0000 7f0d43e00640 1 -- 172.21.15.151:0/313292240 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c871220080 msgr2=0x55c871220860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.688+0000 7f0d43e00640 1 --2- 172.21.15.151:0/313292240 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c871220080 0x55c871220860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.688+0000 7f0d43e00640 1 -- 172.21.15.151:0/313292240 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c871233210 msgr2=0x55c87124b020 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T02:04:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.689+0000 7f0d48a00640 1 --2- 172.21.15.151:0/313292240 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c871233210 0x55c87124b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.689+0000 7f0d43e00640 1 --2- 172.21.15.151:0/313292240 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c871233210 0x55c87124b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.689+0000 7f0d43e00640 1 -- 172.21.15.151:0/313292240 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c87122f160 con 0x55c87121f760 2024-04-26T02:04:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.689+0000 7f0d48a00640 1 --2- 172.21.15.151:0/313292240 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c871233210 0x55c87124b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T02:04:02.689 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.689+0000 7f0d43e00640 1 --2- 172.21.15.151:0/313292240 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c87121f760 0x55c87123ea60 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0d34000c00 tx=0x7f0d34005c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d44966996cbb9f1d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:02.689 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.690+0000 7f0d2fe00640 1 -- 172.21.15.151:0/313292240 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d34049040 con 0x55c87121f760 2024-04-26T02:04:02.689 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.690+0000 7f0d2fe00640 1 -- 172.21.15.151:0/313292240 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d340c1190 con 0x55c87121f760 2024-04-26T02:04:02.689 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.690+0000 7f0d2fe00640 1 -- 172.21.15.151:0/313292240 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d340f46e0 con 0x55c87121f760 2024-04-26T02:04:02.690 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.691+0000 7f0d4d775740 1 -- 172.21.15.151:0/313292240 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c87121f760 msgr2=0x55c87123ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:02.690 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.691+0000 7f0d4d775740 1 --2- 172.21.15.151:0/313292240 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c87121f760 0x55c87123ea60 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0d34000c00 tx=0x7f0d34005c90 comp rx=0 tx=0).stop 2024-04-26T02:04:02.690 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.692+0000 7f0d4d775740 1 -- 172.21.15.151:0/313292240 shutdown_connections 2024-04-26T02:04:02.691 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.692+0000 7f0d4d775740 1 --2- 172.21.15.151:0/313292240 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c871233210 0x55c87124b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.691 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.692+0000 7f0d4d775740 1 --2- 172.21.15.151:0/313292240 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c871220080 0x55c871220860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.691 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.692+0000 7f0d4d775740 1 --2- 172.21.15.151:0/313292240 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c87121f760 0x55c87123ea60 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.691 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.692+0000 7f0d4d775740 1 -- 172.21.15.151:0/313292240 >> 172.21.15.151:0/313292240 conn(0x55c87121bd30 msgr2=0x55c87121ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:02.691 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.692+0000 7f0d4d775740 1 -- 172.21.15.151:0/313292240 shutdown_connections 2024-04-26T02:04:02.691 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.692+0000 7f0d4d775740 1 -- 172.21.15.151:0/313292240 wait complete. 2024-04-26T02:04:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.692+0000 7f0d4d775740 1 -- start start 2024-04-26T02:04:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.693+0000 7f0d4d775740 1 Processor -- start 2024-04-26T02:04:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.693+0000 7f0d4d775740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c87121f760 0x55c87123ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.693+0000 7f0d4d775740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c871220080 0x55c87123b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.693+0000 7f0d4d775740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c871233210 0x55c871239120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.693+0000 7f0d4d775740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0d441a9e80 con 0x55c87121f760 2024-04-26T02:04:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.693+0000 7f0d4d775740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0d441ab2d0 con 0x55c871220080 2024-04-26T02:04:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.693+0000 7f0d4d775740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0d441aa530 con 0x55c871233210 2024-04-26T02:04:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.694+0000 7f0d4d775740 10 client.0 _ll_register_callbacks cb 0x7f0d30009dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:04:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.694+0000 7f0d43400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c871220080 0x55c87123b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.694+0000 7f0d43400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c871220080 0x55c87123b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:41434/0 (socket says 172.21.15.151:41434) 2024-04-26T02:04:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.694+0000 7f0d43e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c87121f760 0x55c87123ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.694+0000 7f0d43400640 1 -- 172.21.15.151:0/3082053582 learned_addr learned my addr 172.21.15.151:0/3082053582 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:04:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.694+0000 7f0d48a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c871233210 0x55c871239120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.694+0000 7f0d43400640 1 -- 172.21.15.151:0/3082053582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c871233210 msgr2=0x55c871239120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.694+0000 7f0d43400640 1 --2- 172.21.15.151:0/3082053582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c871233210 0x55c871239120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.694+0000 7f0d43400640 1 -- 172.21.15.151:0/3082053582 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c87121f760 msgr2=0x55c87123ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.694+0000 7f0d43400640 1 --2- 172.21.15.151:0/3082053582 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c87121f760 0x55c87123ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.694+0000 7f0d43400640 1 -- 172.21.15.151:0/3082053582 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8712398b0 con 0x55c871220080 2024-04-26T02:04:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.695+0000 7f0d43400640 1 --2- 172.21.15.151:0/3082053582 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c871220080 0x55c87123b1c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0d30009ed0 tx=0x7f0d3001c7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67e0fbfc27fc7da4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.695+0000 7f0d2d600640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T02:04:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.696+0000 7f0d2d600640 1 -- 172.21.15.151:0/3082053582 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d30029070 con 0x55c871220080 2024-04-26T02:04:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.696+0000 7f0d2d600640 1 -- 172.21.15.151:0/3082053582 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8455+0+0 (secure 0 0 0) 0x7f0d30021020 con 0x55c871220080 2024-04-26T02:04:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.696+0000 7f0d4d775740 1 -- 172.21.15.151:0/3082053582 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d340c1190 con 0x55c871220080 2024-04-26T02:04:02.696 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.697+0000 7f0d2d600640 1 -- 172.21.15.151:0/3082053582 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f0d30004200 con 0x55c871220080 2024-04-26T02:04:02.696 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.697+0000 7f0d4d775740 10 client.26417 fetch_fsmap learned FSMap version 35 2024-04-26T02:04:02.696 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.697+0000 7f0d4d775740 1 -- 172.21.15.151:0/3082053582 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55c8712aa270 con 0x55c871220080 2024-04-26T02:04:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.697+0000 7f0d2d600640 1 -- 172.21.15.151:0/3082053582 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d340c1190 con 0x55c871220080 2024-04-26T02:04:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.698+0000 7f0d2d600640 1 -- 172.21.15.151:0/3082053582 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 35) ==== 8338+0+0 (secure 0 0 0) 0x7f0d300035f0 con 0x55c871220080 2024-04-26T02:04:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.698+0000 7f0d4d775740 10 client.26417 fetch_fsmap finished waiting for FSMap version 35 2024-04-26T02:04:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.698+0000 7f0d4d775740 10 client.26417 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:04:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.698+0000 7f0d4d775740 20 client.26417 populate_metadata read hostname 'smithi151' 2024-04-26T02:04:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.698+0000 7f0d4d775740 1 --2- 172.21.15.151:0/3082053582 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55c8712aaa10 0x55c8712ace30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.698+0000 7f0d4d775740 4 client.26417 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T02:04:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.698+0000 7f0d4d775740 1 -- 172.21.15.151:0/3082053582 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c871193320 con 0x55c8712aaa10 2024-04-26T02:04:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.698+0000 7f0d43e00640 1 --2- 172.21.15.151:0/3082053582 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55c8712aaa10 0x55c8712ace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.698+0000 7f0d43e00640 1 --2- 172.21.15.151:0/3082053582 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55c8712aaa10 0x55c8712ace30 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50551097c015ecf0 server_cookie=d3a589e5e36df812 in_seq=0 out_seq=0 2024-04-26T02:04:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.698+0000 7f0d2d600640 10 client.26417 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:04:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.711+0000 7f0d2d600640 1 -- 172.21.15.151:0/3082053582 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c871193320 con 0x55c8712aaa10 2024-04-26T02:04:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.711+0000 7f0d2d600640 10 client.26417 handle_command_reply: tid=0 2024-04-26T02:04:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.711+0000 7f0d4d775740 1 client.26417 shutdown 2024-04-26T02:04:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.712+0000 7f0d4d775740 1 -- 172.21.15.151:0/3082053582 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c871220080 msgr2=0x55c87123b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.712+0000 7f0d4d775740 1 --2- 172.21.15.151:0/3082053582 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c871220080 0x55c87123b1c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0d30009ed0 tx=0x7f0d3001c7c0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.712+0000 7f0d4d775740 1 -- 172.21.15.151:0/3082053582 shutdown_connections 2024-04-26T02:04:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.712+0000 7f0d4d775740 1 --2- 172.21.15.151:0/3082053582 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55c8712aaa10 0x55c8712ace30 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.712+0000 7f0d4d775740 1 --2- 172.21.15.151:0/3082053582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c871233210 0x55c871239120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.712+0000 7f0d4d775740 1 --2- 172.21.15.151:0/3082053582 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c871220080 0x55c87123b1c0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.712+0000 7f0d2d600640 0 client.26417 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:04:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.712+0000 7f0d4d775740 1 --2- 172.21.15.151:0/3082053582 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c87121f760 0x55c87123ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.712+0000 7f0d4d775740 1 -- 172.21.15.151:0/3082053582 >> 172.21.15.151:0/3082053582 conn(0x55c87121bd30 msgr2=0x55c871230130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.712+0000 7f0d4d775740 1 -- 172.21.15.151:0/3082053582 shutdown_connections 2024-04-26T02:04:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.712+0000 7f0d4d775740 1 -- 172.21.15.151:0/3082053582 wait complete. 2024-04-26T02:04:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.712+0000 7f0d4d775740 20 client.26417 trim_cache size 0 max 16384 2024-04-26T02:04:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:{ 2024-04-26T02:04:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "message": "", 2024-04-26T02:04:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "return_code": 0 2024-04-26T02:04:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:} 2024-04-26T02:04:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.713+0000 7f0d4aa00640 1 -- 172.21.15.151:0/1295426949 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f0d280a3e40 msgr2=0x7f0d280a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.713+0000 7f0d4aa00640 1 --2- 172.21.15.151:0/1295426949 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f0d280a3e40 0x7f0d280a6300 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c005fd0 tx=0x7f0d3c009500 comp rx=0 tx=0).stop 2024-04-26T02:04:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.713+0000 7f0d4aa00640 1 -- 172.21.15.151:0/1295426949 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d441031e0 msgr2=0x7f0d441a0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.713+0000 7f0d4aa00640 1 --2- 172.21.15.151:0/1295426949 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d441031e0 0x7f0d441a0650 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0d34005ab0 tx=0x7f0d34003c10 comp rx=0 tx=0).stop 2024-04-26T02:04:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.713+0000 7f0d4aa00640 1 -- 172.21.15.151:0/1295426949 shutdown_connections 2024-04-26T02:04:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.713+0000 7f0d4aa00640 1 --2- 172.21.15.151:0/1295426949 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f0d280a3e40 0x7f0d280a6300 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.713+0000 7f0d4aa00640 1 --2- 172.21.15.151:0/1295426949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d4410c520 0x7f0d4419a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.713+0000 7f0d4aa00640 1 --2- 172.21.15.151:0/1295426949 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d44103b00 0x7f0d441a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.713+0000 7f0d4aa00640 1 --2- 172.21.15.151:0/1295426949 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d441031e0 0x7f0d441a0650 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.713+0000 7f0d4aa00640 1 -- 172.21.15.151:0/1295426949 >> 172.21.15.151:0/1295426949 conn(0x7f0d44077fd0 msgr2=0x7f0d44108430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.713+0000 7f0d4aa00640 1 -- 172.21.15.151:0/1295426949 shutdown_connections 2024-04-26T02:04:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:02.713+0000 7f0d4aa00640 1 -- 172.21.15.151:0/1295426949 wait complete. 2024-04-26T02:04:03.686 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '58991260', seconds elapsed: 39.0 2024-04-26T02:04:03.847 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:03 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T02:04:03.847 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:03 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T02:04:03.847 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:03 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T02:04:03.847 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:03 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T02:04:03.847 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:03 smithi165 ceph-mon[32648]: pgmap v327: 97 pgs: 97 active+clean; 154 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 100 KiB/s wr, 39 op/s 2024-04-26T02:04:03.847 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:03 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:03.847 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:03 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:03.847 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:03 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:03.847 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:03 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:03.848 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:03 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:03.848 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:03 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:03.848 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:03 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:03.848 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:03 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:03.848 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:03 smithi165 ceph-mon[32648]: Adjusting osd_memory_target on smithi165 to 176.2M 2024-04-26T02:04:03.848 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:03 smithi165 ceph-mon[32648]: Unable to set osd_memory_target on smithi165 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-26T02:04:03.848 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:03 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:03.848 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:03 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:03.848 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:03 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:03.848 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:03 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:03.848 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:03 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:03.848 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:03 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:03.848 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:03 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:03.848 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:03 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:03.849 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:03 smithi165 ceph-mon[32648]: Adjusting osd_memory_target on smithi172 to 176.2M 2024-04-26T02:04:03.849 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:03 smithi165 ceph-mon[32648]: Unable to set osd_memory_target on smithi172 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-26T02:04:03.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:03 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T02:04:03.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:03 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T02:04:03.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:03 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T02:04:03.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:03 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T02:04:03.873 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:03 smithi151 ceph-mon[28057]: pgmap v327: 97 pgs: 97 active+clean; 154 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 100 KiB/s wr, 39 op/s 2024-04-26T02:04:03.873 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:03 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:03.873 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:03 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:03.873 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:03 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:03.873 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:03 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:03.873 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:03 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:03.873 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:03 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:03.873 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:03 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:03.873 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:03 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:03.873 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:03 smithi151 ceph-mon[28057]: Adjusting osd_memory_target on smithi165 to 176.2M 2024-04-26T02:04:03.873 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:03 smithi151 ceph-mon[28057]: Unable to set osd_memory_target on smithi165 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-26T02:04:03.873 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:03 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:03.873 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:03 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:03.874 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:03 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:03.874 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:03 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:03.874 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:03 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:03.874 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:03 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:03.874 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:03 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:03.874 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:03 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:03.874 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:03 smithi151 ceph-mon[28057]: Adjusting osd_memory_target on smithi172 to 176.2M 2024-04-26T02:04:03.874 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:03 smithi151 ceph-mon[28057]: Unable to set osd_memory_target on smithi172 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-26T02:04:04.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:03 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T02:04:04.038 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:03 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T02:04:04.038 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:03 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T02:04:04.038 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:03 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T02:04:04.038 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:03 smithi172 ceph-mon[32785]: pgmap v327: 97 pgs: 97 active+clean; 154 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 100 KiB/s wr, 39 op/s 2024-04-26T02:04:04.038 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:03 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:04.038 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:03 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:04.038 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:03 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:04.038 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:03 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:04.038 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:03 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:04.039 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:03 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:04.039 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:03 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:04.039 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:03 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:04.039 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:03 smithi172 ceph-mon[32785]: Adjusting osd_memory_target on smithi165 to 176.2M 2024-04-26T02:04:04.039 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:03 smithi172 ceph-mon[32785]: Unable to set osd_memory_target on smithi165 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-26T02:04:04.039 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:03 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:04.039 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:03 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:04.039 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:03 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:04.039 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:03 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:04.039 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:03 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:04.039 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:03 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:04.039 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:03 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:04.039 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:03 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:04.039 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:03 smithi172 ceph-mon[32785]: Adjusting osd_memory_target on smithi172 to 176.2M 2024-04-26T02:04:04.040 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:03 smithi172 ceph-mon[32785]: Unable to set osd_memory_target on smithi172 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-26T02:04:05.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:05 smithi165 ceph-mon[32648]: osdmap e105: 12 total, 12 up, 12 in 2024-04-26T02:04:05.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:05 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T02:04:05.847 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:05 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T02:04:05.847 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:05 smithi165 ceph-mon[32648]: pgmap v329: 97 pgs: 97 active+clean; 154 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 147 KiB/s wr, 53 op/s 2024-04-26T02:04:05.860 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:05 smithi151 ceph-mon[28057]: osdmap e105: 12 total, 12 up, 12 in 2024-04-26T02:04:05.860 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:05 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T02:04:05.860 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:05 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T02:04:05.860 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:05 smithi151 ceph-mon[28057]: pgmap v329: 97 pgs: 97 active+clean; 154 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 147 KiB/s wr, 53 op/s 2024-04-26T02:04:06.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:05 smithi172 ceph-mon[32785]: osdmap e105: 12 total, 12 up, 12 in 2024-04-26T02:04:06.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:05 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T02:04:06.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:05 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T02:04:06.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:05 smithi172 ceph-mon[32785]: pgmap v329: 97 pgs: 97 active+clean; 154 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 147 KiB/s wr, 53 op/s 2024-04-26T02:04:06.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:06 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:06.873 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:06 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:06.873 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:06 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:06.873 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:06 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:06.873 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:06 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:06.873 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:06 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:06.873 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:06 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:06.874 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:06 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:06.874 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:06 smithi151 ceph-mon[28057]: Adjusting osd_memory_target on smithi151 to 176.4M 2024-04-26T02:04:06.874 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:06 smithi151 ceph-mon[28057]: Unable to set osd_memory_target on smithi151 to 185064857: error parsing value: Value '185064857' is below minimum 939524096 2024-04-26T02:04:06.874 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:06 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:04:06.874 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:06 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:04:06.874 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:06 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T02:04:06.938 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:06 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:06.938 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:06 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:06.938 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:06 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:06.939 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:06 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:06.939 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:06 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:06.939 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:06 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:06.939 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:06 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:06.940 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:06 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:06.940 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:06 smithi172 ceph-mon[32785]: Adjusting osd_memory_target on smithi151 to 176.4M 2024-04-26T02:04:06.940 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:06 smithi172 ceph-mon[32785]: Unable to set osd_memory_target on smithi151 to 185064857: error parsing value: Value '185064857' is below minimum 939524096 2024-04-26T02:04:06.940 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:06 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:04:06.940 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:06 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:04:06.940 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:06 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T02:04:07.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:06 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:07.097 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:06 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:07.097 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:06 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:07.097 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:06 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:07.097 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:06 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:07.097 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:06 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:07.097 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:06 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:07.097 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:06 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:04:07.097 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:06 smithi165 ceph-mon[32648]: Adjusting osd_memory_target on smithi151 to 176.4M 2024-04-26T02:04:07.097 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:06 smithi165 ceph-mon[32648]: Unable to set osd_memory_target on smithi151 to 185064857: error parsing value: Value '185064857' is below minimum 939524096 2024-04-26T02:04:07.097 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:06 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:04:07.097 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:06 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:04:07.097 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:06 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T02:04:07.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:07 smithi151 ceph-mon[28057]: pgmap v330: 97 pgs: 97 active+clean; 154 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 147 KiB/s wr, 53 op/s 2024-04-26T02:04:07.948 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:07 smithi172 ceph-mon[32785]: pgmap v330: 97 pgs: 97 active+clean; 154 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 147 KiB/s wr, 53 op/s 2024-04-26T02:04:08.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:07 smithi165 ceph-mon[32648]: pgmap v330: 97 pgs: 97 active+clean; 154 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 147 KiB/s wr, 53 op/s 2024-04-26T02:04:10.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:10 smithi172 ceph-mon[32785]: pgmap v331: 97 pgs: 97 active+clean; 176 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 837 KiB/s wr, 284 op/s 2024-04-26T02:04:10.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:10 smithi172 ceph-mon[32785]: osdmap e106: 12 total, 12 up, 12 in 2024-04-26T02:04:10.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:10 smithi165 ceph-mon[32648]: pgmap v331: 97 pgs: 97 active+clean; 176 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 837 KiB/s wr, 284 op/s 2024-04-26T02:04:10.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:10 smithi165 ceph-mon[32648]: osdmap e106: 12 total, 12 up, 12 in 2024-04-26T02:04:10.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:10 smithi151 ceph-mon[28057]: pgmap v331: 97 pgs: 97 active+clean; 176 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 837 KiB/s wr, 284 op/s 2024-04-26T02:04:10.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:10 smithi151 ceph-mon[28057]: osdmap e106: 12 total, 12 up, 12 in 2024-04-26T02:04:12.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:12 smithi172 ceph-mon[32785]: pgmap v333: 97 pgs: 97 active+clean; 176 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 900 KiB/s wr, 304 op/s 2024-04-26T02:04:12.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:12 smithi165 ceph-mon[32648]: pgmap v333: 97 pgs: 97 active+clean; 176 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 900 KiB/s wr, 304 op/s 2024-04-26T02:04:12.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:12 smithi151 ceph-mon[28057]: pgmap v333: 97 pgs: 97 active+clean; 176 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 900 KiB/s wr, 304 op/s 2024-04-26T02:04:14.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:14 smithi172 ceph-mon[32785]: pgmap v334: 97 pgs: 97 active+clean; 174 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 1.3 MiB/s wr, 438 op/s 2024-04-26T02:04:14.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:14 smithi165 ceph-mon[32648]: pgmap v334: 97 pgs: 97 active+clean; 174 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 1.3 MiB/s wr, 438 op/s 2024-04-26T02:04:14.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:14 smithi151 ceph-mon[28057]: pgmap v334: 97 pgs: 97 active+clean; 174 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 1.3 MiB/s wr, 438 op/s 2024-04-26T02:04:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.883+0000 7f23ba600640 1 -- 172.21.15.151:0/2721201580 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23b410e840 msgr2=0x7f23b4110c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.883+0000 7f23ba600640 1 --2- 172.21.15.151:0/2721201580 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23b410e840 0x7f23b4110c30 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f23ac009ca0 tx=0x7f23ac02a950 comp rx=0 tx=0).stop 2024-04-26T02:04:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.884+0000 7f23ba600640 1 -- 172.21.15.151:0/2721201580 shutdown_connections 2024-04-26T02:04:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.884+0000 7f23ba600640 1 --2- 172.21.15.151:0/2721201580 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23b410e840 0x7f23b4110c30 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.884+0000 7f23ba600640 1 --2- 172.21.15.151:0/2721201580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23b4074b70 0x7f23b410e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.884+0000 7f23ba600640 1 --2- 172.21.15.151:0/2721201580 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23b4074250 0x7f23b4074630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.884+0000 7f23ba600640 1 -- 172.21.15.151:0/2721201580 >> 172.21.15.151:0/2721201580 conn(0x7f23b40793e0 msgr2=0x7f23b40797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.884+0000 7f23ba600640 1 -- 172.21.15.151:0/2721201580 shutdown_connections 2024-04-26T02:04:15.884 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.884+0000 7f23ba600640 1 -- 172.21.15.151:0/2721201580 wait complete. 2024-04-26T02:04:15.884 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.885+0000 7f23ba600640 1 Processor -- start 2024-04-26T02:04:15.884 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.886+0000 7f23ba600640 1 -- start start 2024-04-26T02:04:15.885 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.886+0000 7f23ba600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23b4074250 0x7f23b41a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:15.885 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.886+0000 7f23ba600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23b4074b70 0x7f23b41a0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:15.885 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.886+0000 7f23ba600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23b410e840 0x7f23b419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:15.885 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.886+0000 7f23ba600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f23b4114430 con 0x7f23b4074b70 2024-04-26T02:04:15.885 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.886+0000 7f23ba600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f23b41142b0 con 0x7f23b410e840 2024-04-26T02:04:15.885 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.886+0000 7f23ba600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f23b41145b0 con 0x7f23b4074250 2024-04-26T02:04:15.886 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.886+0000 7f23b3400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23b4074250 0x7f23b41a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:15.887 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.886+0000 7f23b3400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23b4074250 0x7f23b41a0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:43458/0 (socket says 172.21.15.151:43458) 2024-04-26T02:04:15.887 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.886+0000 7f23b3400640 1 -- 172.21.15.151:0/75288590 learned_addr learned my addr 172.21.15.151:0/75288590 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:04:15.887 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.886+0000 7f23b3e00640 1 --2- 172.21.15.151:0/75288590 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23b410e840 0x7f23b419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:15.887 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.887+0000 7f23b3400640 1 -- 172.21.15.151:0/75288590 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23b410e840 msgr2=0x7f23b419a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:15.887 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.887+0000 7f23b3400640 1 --2- 172.21.15.151:0/75288590 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23b410e840 0x7f23b419a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:15.887 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.887+0000 7f23b3400640 1 -- 172.21.15.151:0/75288590 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23b4074b70 msgr2=0x7f23b41a0970 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T02:04:15.887 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.887+0000 7f23b3400640 1 --2- 172.21.15.151:0/75288590 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23b4074b70 0x7f23b41a0970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:15.887 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.887+0000 7f23b3400640 1 -- 172.21.15.151:0/75288590 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23b419ae00 con 0x7f23b4074250 2024-04-26T02:04:15.887 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.887+0000 7f23b3400640 1 --2- 172.21.15.151:0/75288590 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23b4074250 0x7f23b41a0430 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f23a400e9f0 tx=0x7f23a400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:15.888 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.887+0000 7f23b0c00640 1 -- 172.21.15.151:0/75288590 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a400cd80 con 0x7f23b4074250 2024-04-26T02:04:15.888 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.888+0000 7f23b0c00640 1 -- 172.21.15.151:0/75288590 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23a40108e0 con 0x7f23b4074250 2024-04-26T02:04:15.888 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.888+0000 7f23ba600640 1 -- 172.21.15.151:0/75288590 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23b419b0f0 con 0x7f23b4074250 2024-04-26T02:04:15.888 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.888+0000 7f23b0c00640 1 -- 172.21.15.151:0/75288590 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a4018860 con 0x7f23b4074250 2024-04-26T02:04:15.888 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.888+0000 7f23ba600640 1 -- 172.21.15.151:0/75288590 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23b41a74d0 con 0x7f23b4074250 2024-04-26T02:04:15.901 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.900+0000 7f23b0c00640 1 -- 172.21.15.151:0/75288590 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f23a4018a00 con 0x7f23b4074250 2024-04-26T02:04:15.903 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.904+0000 7f23b0c00640 1 --2- 172.21.15.151:0/75288590 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f23980a3ec0 0x7f23980a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:15.903 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.904+0000 7f23bd5e2740 1 Processor -- start 2024-04-26T02:04:15.904 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.904+0000 7f23abe00640 1 --2- 172.21.15.151:0/75288590 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f23980a3ec0 0x7f23980a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:15.904 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.904+0000 7f23b0c00640 1 -- 172.21.15.151:0/75288590 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8455+0+0 (secure 0 0 0) 0x7f23a40d0b50 con 0x7f23b4074250 2024-04-26T02:04:15.904 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.905+0000 7f23bd5e2740 1 -- start start 2024-04-26T02:04:15.904 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.905+0000 7f23bd5e2740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560f30b38030 0x560f30b38410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:15.904 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.905+0000 7f23bd5e2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f30b38950 0x560f30b42cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:15.904 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.905+0000 7f23bd5e2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560f30b43210 0x560f30b45600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:15.904 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.905+0000 7f23bd5e2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560f30a95900 con 0x560f30b38950 2024-04-26T02:04:15.904 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.905+0000 7f23bd5e2740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560f30a3bef0 con 0x560f30b38030 2024-04-26T02:04:15.904 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.905+0000 7f23bd5e2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560f30a918c0 con 0x560f30b43210 2024-04-26T02:04:15.905 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.906+0000 7f23b3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560f30b38030 0x560f30b38410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:15.905 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.906+0000 7f23b3e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560f30b43210 0x560f30b45600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:15.906 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.906+0000 7f23b3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560f30b38030 0x560f30b38410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:49672/0 (socket says 172.21.15.151:49672) 2024-04-26T02:04:15.906 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.906+0000 7f23b3e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560f30b43210 0x560f30b45600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:43466/0 (socket says 172.21.15.151:43466) 2024-04-26T02:04:15.906 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.906+0000 7f23b3400640 1 -- 172.21.15.151:0/4022697795 learned_addr learned my addr 172.21.15.151:0/4022697795 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:04:15.906 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.906+0000 7f23abe00640 1 --2- 172.21.15.151:0/75288590 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f23980a3ec0 0x7f23980a6380 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f23b41016d0 tx=0x7f239c005f90 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:15.906 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.906+0000 7f23b3400640 1 -- 172.21.15.151:0/4022697795 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560f30b43210 msgr2=0x560f30b45600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:15.906 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.906+0000 7f23b3400640 1 --2- 172.21.15.151:0/4022697795 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560f30b43210 0x560f30b45600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:15.906 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.906+0000 7f23b3400640 1 -- 172.21.15.151:0/4022697795 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f30b38950 msgr2=0x560f30b42cd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T02:04:15.906 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.906+0000 7f23abe00640 1 --2- 172.21.15.151:0/4022697795 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f30b38950 0x560f30b42cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:15.906 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.907+0000 7f23b3400640 1 --2- 172.21.15.151:0/4022697795 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f30b38950 0x560f30b42cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:15.906 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.907+0000 7f23b3400640 1 -- 172.21.15.151:0/4022697795 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f307a8210 con 0x560f30b38030 2024-04-26T02:04:15.906 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.907+0000 7f23abe00640 1 --2- 172.21.15.151:0/4022697795 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f30b38950 0x560f30b42cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T02:04:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.907+0000 7f23b3400640 1 --2- 172.21.15.151:0/4022697795 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560f30b38030 0x560f30b38410 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f23a4098f30 tx=0x7f23a409cad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6a6d9960d955f7e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.907+0000 7f23a8c00640 1 -- 172.21.15.151:0/4022697795 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a40bd0b0 con 0x560f30b38030 2024-04-26T02:04:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.907+0000 7f23a8c00640 1 -- 172.21.15.151:0/4022697795 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23a409add0 con 0x560f30b38030 2024-04-26T02:04:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.907+0000 7f23a8c00640 1 -- 172.21.15.151:0/4022697795 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a409bf20 con 0x560f30b38030 2024-04-26T02:04:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.909+0000 7f23bd5e2740 1 -- 172.21.15.151:0/4022697795 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560f30b38030 msgr2=0x560f30b38410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:15.909 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.909+0000 7f23bd5e2740 1 --2- 172.21.15.151:0/4022697795 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560f30b38030 0x560f30b38410 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f23a4098f30 tx=0x7f23a409cad0 comp rx=0 tx=0).stop 2024-04-26T02:04:15.909 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.909+0000 7f23bd5e2740 1 -- 172.21.15.151:0/4022697795 shutdown_connections 2024-04-26T02:04:15.909 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.909+0000 7f23bd5e2740 1 --2- 172.21.15.151:0/4022697795 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560f30b43210 0x560f30b45600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:15.909 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.909+0000 7f23bd5e2740 1 --2- 172.21.15.151:0/4022697795 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f30b38950 0x560f30b42cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:15.909 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.909+0000 7f23bd5e2740 1 --2- 172.21.15.151:0/4022697795 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560f30b38030 0x560f30b38410 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:15.909 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.909+0000 7f23bd5e2740 1 -- 172.21.15.151:0/4022697795 >> 172.21.15.151:0/4022697795 conn(0x560f30b34540 msgr2=0x560f30b34920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:15.909 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.909+0000 7f23bd5e2740 1 -- 172.21.15.151:0/4022697795 shutdown_connections 2024-04-26T02:04:15.909 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.909+0000 7f23bd5e2740 1 -- 172.21.15.151:0/4022697795 wait complete. 2024-04-26T02:04:15.912 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.910+0000 7f23bd5e2740 1 -- start start 2024-04-26T02:04:15.912 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.910+0000 7f23bd5e2740 1 Processor -- start 2024-04-26T02:04:15.912 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.911+0000 7f23bd5e2740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560f30b38030 0x560f30b40260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:15.912 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.911+0000 7f23bd5e2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560f30b38950 0x560f30b407a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:15.912 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.911+0000 7f23bd5e2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f30b43210 0x560f30b40ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:15.912 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.911+0000 7f23bd5e2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560f30b45e40 con 0x560f30b43210 2024-04-26T02:04:15.912 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.911+0000 7f23bd5e2740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560f30a4f310 con 0x560f30b38030 2024-04-26T02:04:15.912 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.911+0000 7f23bd5e2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560f30b45fc0 con 0x560f30b38950 2024-04-26T02:04:15.913 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.911+0000 7f23bd5e2740 10 client.0 _ll_register_callbacks cb 0x560f30a1dcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:04:15.913 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.911+0000 7f23b3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560f30b38030 0x560f30b40260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:15.913 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.911+0000 7f23b3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560f30b38030 0x560f30b40260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:49682/0 (socket says 172.21.15.151:49682) 2024-04-26T02:04:15.913 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.912+0000 7f23b3400640 1 -- 172.21.15.151:0/3139198977 learned_addr learned my addr 172.21.15.151:0/3139198977 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:04:15.913 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.912+0000 7f23abe00640 1 --2- 172.21.15.151:0/3139198977 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560f30b38950 0x560f30b407a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:15.913 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.912+0000 7f23b3e00640 1 --2- 172.21.15.151:0/3139198977 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f30b43210 0x560f30b40ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:15.913 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.912+0000 7f23b3400640 1 -- 172.21.15.151:0/3139198977 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560f30b38950 msgr2=0x560f30b407a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:15.913 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.912+0000 7f23b3400640 1 --2- 172.21.15.151:0/3139198977 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560f30b38950 0x560f30b407a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:15.913 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.912+0000 7f23b3400640 1 -- 172.21.15.151:0/3139198977 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f30b43210 msgr2=0x560f30b40ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:15.913 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.912+0000 7f23b3400640 1 --2- 172.21.15.151:0/3139198977 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f30b43210 0x560f30b40ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:15.913 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.912+0000 7f23b3400640 1 -- 172.21.15.151:0/3139198977 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f30874620 con 0x560f30b38030 2024-04-26T02:04:15.913 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.912+0000 7f23b3e00640 1 --2- 172.21.15.151:0/3139198977 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f30b43210 0x560f30b40ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T02:04:15.913 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.912+0000 7f23b3400640 1 --2- 172.21.15.151:0/3139198977 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560f30b38030 0x560f30b40260 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f23a4098ee0 tx=0x7f23a4009590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5dc09091a3a5e6f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:15.913 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.912+0000 7f23a1600640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T02:04:15.914 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.913+0000 7f23a1600640 1 -- 172.21.15.151:0/3139198977 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a40ce500 con 0x560f30b38030 2024-04-26T02:04:15.914 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.913+0000 7f23a1600640 1 -- 172.21.15.151:0/3139198977 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8455+0+0 (secure 0 0 0) 0x7f23a40ce6a0 con 0x560f30b38030 2024-04-26T02:04:15.914 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.913+0000 7f23bd5e2740 1 -- 172.21.15.151:0/3139198977 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23a409add0 con 0x560f30b38030 2024-04-26T02:04:15.914 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.913+0000 7f23a1600640 1 -- 172.21.15.151:0/3139198977 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f23a4080050 con 0x560f30b38030 2024-04-26T02:04:15.914 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.913+0000 7f23bd5e2740 10 client.26429 fetch_fsmap learned FSMap version 35 2024-04-26T02:04:15.914 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.913+0000 7f23bd5e2740 1 -- 172.21.15.151:0/3139198977 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x560f30b52110 con 0x560f30b38030 2024-04-26T02:04:15.914 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.913+0000 7f23a1600640 1 -- 172.21.15.151:0/3139198977 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23a409add0 con 0x560f30b38030 2024-04-26T02:04:15.915 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.913+0000 7f23a1600640 1 -- 172.21.15.151:0/3139198977 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 35) ==== 8338+0+0 (secure 0 0 0) 0x7f23a40b8020 con 0x560f30b38030 2024-04-26T02:04:15.915 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.913+0000 7f23bd5e2740 10 client.26429 fetch_fsmap finished waiting for FSMap version 35 2024-04-26T02:04:15.915 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.913+0000 7f23bd5e2740 10 client.26429 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:04:15.915 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.913+0000 7f23bd5e2740 20 client.26429 populate_metadata read hostname 'smithi151' 2024-04-26T02:04:15.915 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.914+0000 7f23bd5e2740 1 --2- 172.21.15.151:0/3139198977 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x560f30b53460 0x560f30b55850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:15.915 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.914+0000 7f23bd5e2740 4 client.26429 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:04:15.915 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.914+0000 7f23bd5e2740 1 -- 172.21.15.151:0/3139198977 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f30aa98b0 con 0x560f30b53460 2024-04-26T02:04:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.914+0000 7f23abe00640 1 --2- 172.21.15.151:0/3139198977 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x560f30b53460 0x560f30b55850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.916+0000 7f23abe00640 1 --2- 172.21.15.151:0/3139198977 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x560f30b53460 0x560f30b55850 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c59ab38b48b1f37 server_cookie=76e4a819601f6612 in_seq=0 out_seq=0 2024-04-26T02:04:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.916+0000 7f23a1600640 10 client.26429 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:04:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.917+0000 7f23a1600640 1 -- 172.21.15.151:0/3139198977 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560f30aa98b0 con 0x560f30b53460 2024-04-26T02:04:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.917+0000 7f23a1600640 10 client.26429 handle_command_reply: tid=0 2024-04-26T02:04:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.917+0000 7f23bd5e2740 1 client.26429 shutdown 2024-04-26T02:04:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.917+0000 7f23bd5e2740 1 -- 172.21.15.151:0/3139198977 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560f30b38030 msgr2=0x560f30b40260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.917+0000 7f23bd5e2740 1 --2- 172.21.15.151:0/3139198977 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560f30b38030 0x560f30b40260 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f23a4098ee0 tx=0x7f23a4009590 comp rx=0 tx=0).stop 2024-04-26T02:04:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.918+0000 7f23bd5e2740 1 -- 172.21.15.151:0/3139198977 shutdown_connections 2024-04-26T02:04:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.918+0000 7f23bd5e2740 1 --2- 172.21.15.151:0/3139198977 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x560f30b53460 0x560f30b55850 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.918+0000 7f23bd5e2740 1 --2- 172.21.15.151:0/3139198977 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f30b43210 0x560f30b40ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.918+0000 7f23bd5e2740 1 --2- 172.21.15.151:0/3139198977 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560f30b38950 0x560f30b407a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.918+0000 7f23bd5e2740 1 --2- 172.21.15.151:0/3139198977 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560f30b38030 0x560f30b40260 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.918+0000 7f23bd5e2740 1 -- 172.21.15.151:0/3139198977 >> 172.21.15.151:0/3139198977 conn(0x560f30b34540 msgr2=0x560f30b3a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.918+0000 7f23a1600640 0 client.26429 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:04:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.918+0000 7f23bd5e2740 1 -- 172.21.15.151:0/3139198977 shutdown_connections 2024-04-26T02:04:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.918+0000 7f23bd5e2740 1 -- 172.21.15.151:0/3139198977 wait complete. 2024-04-26T02:04:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.918+0000 7f23bd5e2740 20 client.26429 trim_cache size 0 max 16384 2024-04-26T02:04:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.946+0000 7f23bd5e2740 1 Processor -- start 2024-04-26T02:04:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.947+0000 7f23bd5e2740 1 -- start start 2024-04-26T02:04:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.947+0000 7f23bd5e2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f30b38030 0x560f30b3f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.947+0000 7f23bd5e2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560f30b38950 0x560f30b40b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.947+0000 7f23bd5e2740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560f30b43210 0x560f30b2ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.947+0000 7f23bd5e2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f23b41ae230 con 0x560f30b38030 2024-04-26T02:04:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.947+0000 7f23bd5e2740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f23b41abb00 con 0x560f30b43210 2024-04-26T02:04:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.947+0000 7f23bd5e2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f23b41a99b0 con 0x560f30b38950 2024-04-26T02:04:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.947+0000 7f23b3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560f30b43210 0x560f30b2ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.947+0000 7f23abe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560f30b38950 0x560f30b40b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.947+0000 7f23b3400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f30b38030 0x560f30b3f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.947+0000 7f23b3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560f30b43210 0x560f30b2ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:49684/0 (socket says 172.21.15.151:49684) 2024-04-26T02:04:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.947+0000 7f23b3e00640 1 -- 172.21.15.151:0/2493585964 learned_addr learned my addr 172.21.15.151:0/2493585964 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:04:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.948+0000 7f23abe00640 1 -- 172.21.15.151:0/2493585964 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560f30b43210 msgr2=0x560f30b2ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.948+0000 7f23abe00640 1 --2- 172.21.15.151:0/2493585964 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560f30b43210 0x560f30b2ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.948+0000 7f23abe00640 1 -- 172.21.15.151:0/2493585964 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f30b38030 msgr2=0x560f30b3f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.948+0000 7f23abe00640 1 --2- 172.21.15.151:0/2493585964 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f30b38030 0x560f30b3f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.948+0000 7f23abe00640 1 -- 172.21.15.151:0/2493585964 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f30b45f00 con 0x560f30b38950 2024-04-26T02:04:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.948+0000 7f23abe00640 1 --2- 172.21.15.151:0/2493585964 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560f30b38950 0x560f30b40b00 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x560f30b41830 tx=0x7f239c016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10f96ef28be07ca9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.948+0000 7f23a3e00640 1 -- 172.21.15.151:0/2493585964 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f239c025340 con 0x560f30b38950 2024-04-26T02:04:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.948+0000 7f23a3e00640 1 -- 172.21.15.151:0/2493585964 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f239c029030 con 0x560f30b38950 2024-04-26T02:04:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.948+0000 7f23a3e00640 1 -- 172.21.15.151:0/2493585964 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f239c010c60 con 0x560f30b38950 2024-04-26T02:04:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.949+0000 7f23bd5e2740 1 -- 172.21.15.151:0/2493585964 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560f30b38950 msgr2=0x560f30b40b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.949+0000 7f23bd5e2740 1 --2- 172.21.15.151:0/2493585964 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560f30b38950 0x560f30b40b00 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x560f30b41830 tx=0x7f239c016000 comp rx=0 tx=0).stop 2024-04-26T02:04:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.949+0000 7f23bd5e2740 1 -- 172.21.15.151:0/2493585964 shutdown_connections 2024-04-26T02:04:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.949+0000 7f23bd5e2740 1 --2- 172.21.15.151:0/2493585964 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560f30b43210 0x560f30b2ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.949+0000 7f23bd5e2740 1 --2- 172.21.15.151:0/2493585964 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560f30b38950 0x560f30b40b00 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.949+0000 7f23bd5e2740 1 --2- 172.21.15.151:0/2493585964 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f30b38030 0x560f30b3f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.949+0000 7f23bd5e2740 1 -- 172.21.15.151:0/2493585964 >> 172.21.15.151:0/2493585964 conn(0x560f30b34540 msgr2=0x560f30b2a600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.949+0000 7f23bd5e2740 1 -- 172.21.15.151:0/2493585964 shutdown_connections 2024-04-26T02:04:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.949+0000 7f23bd5e2740 1 -- 172.21.15.151:0/2493585964 wait complete. 2024-04-26T02:04:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.950+0000 7f23bd5e2740 1 -- start start 2024-04-26T02:04:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.950+0000 7f23bd5e2740 1 Processor -- start 2024-04-26T02:04:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.951+0000 7f23bd5e2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560f30b38030 0x560f30b572f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.951+0000 7f23bd5e2740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560f30b38950 0x560f30b57830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.951+0000 7f23bd5e2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f30b43210 0x560f30bb7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.951+0000 7f23bd5e2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f23b41aa000 con 0x560f30b43210 2024-04-26T02:04:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.951+0000 7f23bd5e2740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f23b41ab430 con 0x560f30b38950 2024-04-26T02:04:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.951+0000 7f23bd5e2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f23b41aa690 con 0x560f30b38030 2024-04-26T02:04:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.951+0000 7f23bd5e2740 10 client.0 _ll_register_callbacks cb 0x7f23a409c4e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:04:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.951+0000 7f23b3400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560f30b38030 0x560f30b572f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.951+0000 7f23b3400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560f30b38030 0x560f30b572f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:43500/0 (socket says 172.21.15.151:43500) 2024-04-26T02:04:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.951+0000 7f23b3400640 1 -- 172.21.15.151:0/2562333053 learned_addr learned my addr 172.21.15.151:0/2562333053 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:04:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.951+0000 7f23abe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560f30b38950 0x560f30b57830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.951+0000 7f23b3e00640 1 --2- 172.21.15.151:0/2562333053 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f30b43210 0x560f30bb7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.951+0000 7f23b3400640 1 -- 172.21.15.151:0/2562333053 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560f30b38950 msgr2=0x560f30b57830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.951+0000 7f23b3400640 1 --2- 172.21.15.151:0/2562333053 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560f30b38950 0x560f30b57830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.951+0000 7f23b3400640 1 -- 172.21.15.151:0/2562333053 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f30b43210 msgr2=0x560f30bb7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.951+0000 7f23b3400640 1 --2- 172.21.15.151:0/2562333053 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f30b43210 0x560f30bb7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.951+0000 7f23b3400640 1 -- 172.21.15.151:0/2562333053 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f30bb7b80 con 0x560f30b38030 2024-04-26T02:04:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.951+0000 7f23b3e00640 1 --2- 172.21.15.151:0/2562333053 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f30b43210 0x560f30bb7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:04:15.955 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.952+0000 7f23b3400640 1 --2- 172.21.15.151:0/2562333053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560f30b38030 0x560f30b572f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f23a400c6a0 tx=0x7f23a409d1d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=581c820d8ecbe161 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:15.955 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.952+0000 7f23a1600640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-26T02:04:15.955 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.952+0000 7f23a1600640 1 -- 172.21.15.151:0/2562333053 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a409bd30 con 0x560f30b38030 2024-04-26T02:04:15.955 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.952+0000 7f23a1600640 1 -- 172.21.15.151:0/2562333053 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8455+0+0 (secure 0 0 0) 0x7f23a40ce500 con 0x560f30b38030 2024-04-26T02:04:15.955 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.952+0000 7f23bd5e2740 1 -- 172.21.15.151:0/2562333053 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f239c029030 con 0x560f30b38030 2024-04-26T02:04:15.955 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.952+0000 7f23a1600640 1 -- 172.21.15.151:0/2562333053 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f23a40bd130 con 0x560f30b38030 2024-04-26T02:04:15.955 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.952+0000 7f23bd5e2740 10 client.26458 fetch_fsmap learned FSMap version 35 2024-04-26T02:04:15.955 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.952+0000 7f23bd5e2740 1 -- 172.21.15.151:0/2562333053 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x560f30bb8da0 con 0x560f30b38030 2024-04-26T02:04:15.955 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.952+0000 7f23a1600640 1 -- 172.21.15.151:0/2562333053 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f239c029030 con 0x560f30b38030 2024-04-26T02:04:15.955 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.953+0000 7f23a1600640 1 -- 172.21.15.151:0/2562333053 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 35) ==== 8338+0+0 (secure 0 0 0) 0x7f23a40b8070 con 0x560f30b38030 2024-04-26T02:04:15.955 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.953+0000 7f23bd5e2740 10 client.26458 fetch_fsmap finished waiting for FSMap version 35 2024-04-26T02:04:15.955 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.953+0000 7f23bd5e2740 10 client.26458 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:04:15.956 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.953+0000 7f23bd5e2740 20 client.26458 populate_metadata read hostname 'smithi151' 2024-04-26T02:04:15.956 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.953+0000 7f23bd5e2740 1 --2- 172.21.15.151:0/2562333053 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x560f30bb92d0 0x560f30bbb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:15.956 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.953+0000 7f23bd5e2740 4 client.26458 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T02:04:15.956 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.953+0000 7f23bd5e2740 1 -- 172.21.15.151:0/2562333053 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560f30aa3320 con 0x560f30bb92d0 2024-04-26T02:04:15.956 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.953+0000 7f23abe00640 1 --2- 172.21.15.151:0/2562333053 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x560f30bb92d0 0x560f30bbb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:15.956 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.954+0000 7f23a1600640 10 client.26458 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:04:15.956 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.954+0000 7f23abe00640 1 --2- 172.21.15.151:0/2562333053 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x560f30bb92d0 0x560f30bbb6c0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9479638d090a100 server_cookie=6e9f69438cf3b1ba in_seq=0 out_seq=0 2024-04-26T02:04:15.965 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.966+0000 7f23a1600640 1 -- 172.21.15.151:0/2562333053 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560f30aa3320 con 0x560f30bb92d0 2024-04-26T02:04:15.966 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.966+0000 7f23a1600640 10 client.26458 handle_command_reply: tid=0 2024-04-26T02:04:15.966 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.966+0000 7f23bd5e2740 1 client.26458 shutdown 2024-04-26T02:04:15.966 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.966+0000 7f23bd5e2740 1 -- 172.21.15.151:0/2562333053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560f30b38030 msgr2=0x560f30b572f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:15.966 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.966+0000 7f23bd5e2740 1 --2- 172.21.15.151:0/2562333053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560f30b38030 0x560f30b572f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f23a400c6a0 tx=0x7f23a409d1d0 comp rx=0 tx=0).stop 2024-04-26T02:04:15.966 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.966+0000 7f23bd5e2740 1 -- 172.21.15.151:0/2562333053 shutdown_connections 2024-04-26T02:04:15.966 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.966+0000 7f23bd5e2740 1 --2- 172.21.15.151:0/2562333053 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x560f30bb92d0 0x560f30bbb6c0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:15.966 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.966+0000 7f23bd5e2740 1 --2- 172.21.15.151:0/2562333053 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f30b43210 0x560f30bb7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:15.966 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.966+0000 7f23bd5e2740 1 --2- 172.21.15.151:0/2562333053 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560f30b38950 0x560f30b57830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:15.966 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.966+0000 7f23bd5e2740 1 --2- 172.21.15.151:0/2562333053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560f30b38030 0x560f30b572f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:15.966 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.966+0000 7f23bd5e2740 1 -- 172.21.15.151:0/2562333053 >> 172.21.15.151:0/2562333053 conn(0x560f30b34540 msgr2=0x560f30b40b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:15.966 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.966+0000 7f23a1600640 0 client.26458 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:04:15.966 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.967+0000 7f23bd5e2740 1 -- 172.21.15.151:0/2562333053 shutdown_connections 2024-04-26T02:04:15.967 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.967+0000 7f23bd5e2740 1 -- 172.21.15.151:0/2562333053 wait complete. 2024-04-26T02:04:15.967 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.967+0000 7f23bd5e2740 20 client.26458 trim_cache size 0 max 16384 2024-04-26T02:04:15.967 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:{ 2024-04-26T02:04:15.967 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "message": "", 2024-04-26T02:04:15.967 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "return_code": 0 2024-04-26T02:04:15.967 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:} 2024-04-26T02:04:15.968 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.968+0000 7f23ba600640 1 -- 172.21.15.151:0/75288590 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f23980a3ec0 msgr2=0x7f23980a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:15.968 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.968+0000 7f23ba600640 1 --2- 172.21.15.151:0/75288590 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f23980a3ec0 0x7f23980a6380 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f23b41016d0 tx=0x7f239c005f90 comp rx=0 tx=0).stop 2024-04-26T02:04:15.968 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.968+0000 7f23ba600640 1 -- 172.21.15.151:0/75288590 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23b4074250 msgr2=0x7f23b41a0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:15.968 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.968+0000 7f23ba600640 1 --2- 172.21.15.151:0/75288590 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23b4074250 0x7f23b41a0430 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f23a400e9f0 tx=0x7f23a400eeb0 comp rx=0 tx=0).stop 2024-04-26T02:04:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.968+0000 7f23ba600640 1 -- 172.21.15.151:0/75288590 shutdown_connections 2024-04-26T02:04:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.968+0000 7f23ba600640 1 --2- 172.21.15.151:0/75288590 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f23980a3ec0 0x7f23980a6380 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.968+0000 7f23ba600640 1 --2- 172.21.15.151:0/75288590 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23b410e840 0x7f23b419a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.968+0000 7f23ba600640 1 --2- 172.21.15.151:0/75288590 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23b4074b70 0x7f23b41a0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.968+0000 7f23ba600640 1 --2- 172.21.15.151:0/75288590 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23b4074250 0x7f23b41a0430 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.968+0000 7f23ba600640 1 -- 172.21.15.151:0/75288590 >> 172.21.15.151:0/75288590 conn(0x7f23b40793e0 msgr2=0x7f23b40fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.968+0000 7f23ba600640 1 -- 172.21.15.151:0/75288590 shutdown_connections 2024-04-26T02:04:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:15.968+0000 7f23ba600640 1 -- 172.21.15.151:0/75288590 wait complete. 2024-04-26T02:04:16.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:16 smithi165 ceph-mon[32648]: pgmap v335: 97 pgs: 97 active+clean; 169 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 1.2 MiB/s wr, 423 op/s 2024-04-26T02:04:16.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:16 smithi151 ceph-mon[28057]: pgmap v335: 97 pgs: 97 active+clean; 169 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 1.2 MiB/s wr, 423 op/s 2024-04-26T02:04:16.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:16 smithi172 ceph-mon[32785]: pgmap v335: 97 pgs: 97 active+clean; 169 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 1.2 MiB/s wr, 423 op/s 2024-04-26T02:04:18.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:18 smithi165 ceph-mon[32648]: pgmap v336: 97 pgs: 97 active+clean; 169 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 1.2 MiB/s wr, 423 op/s 2024-04-26T02:04:18.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:18 smithi151 ceph-mon[28057]: pgmap v336: 97 pgs: 97 active+clean; 169 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 1.2 MiB/s wr, 423 op/s 2024-04-26T02:04:18.658 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:18 smithi172 ceph-mon[32785]: pgmap v336: 97 pgs: 97 active+clean; 169 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 1.2 MiB/s wr, 423 op/s 2024-04-26T02:04:20.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:20 smithi165 ceph-mon[32648]: pgmap v337: 97 pgs: 97 active+clean; 162 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 1.1 MiB/s wr, 384 op/s 2024-04-26T02:04:20.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:20 smithi151 ceph-mon[28057]: pgmap v337: 97 pgs: 97 active+clean; 162 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 1.1 MiB/s wr, 384 op/s 2024-04-26T02:04:20.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:20 smithi172 ceph-mon[32785]: pgmap v337: 97 pgs: 97 active+clean; 162 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 1.1 MiB/s wr, 384 op/s 2024-04-26T02:04:22.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:22 smithi165 ceph-mon[32648]: pgmap v338: 97 pgs: 97 active+clean; 162 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 988 KiB/s wr, 329 op/s 2024-04-26T02:04:22.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:22 smithi151 ceph-mon[28057]: pgmap v338: 97 pgs: 97 active+clean; 162 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 988 KiB/s wr, 329 op/s 2024-04-26T02:04:22.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:22 smithi172 ceph-mon[32785]: pgmap v338: 97 pgs: 97 active+clean; 162 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 988 KiB/s wr, 329 op/s 2024-04-26T02:04:24.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:24 smithi165 ceph-mon[32648]: pgmap v339: 97 pgs: 97 active+clean; 166 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.3 MiB/s wr, 435 op/s 2024-04-26T02:04:24.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:24 smithi151 ceph-mon[28057]: pgmap v339: 97 pgs: 97 active+clean; 166 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.3 MiB/s wr, 435 op/s 2024-04-26T02:04:24.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:24 smithi172 ceph-mon[32785]: pgmap v339: 97 pgs: 97 active+clean; 166 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.3 MiB/s wr, 435 op/s 2024-04-26T02:04:26.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:26 smithi165 ceph-mon[32648]: pgmap v340: 97 pgs: 97 active+clean; 160 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 1.1 MiB/s wr, 366 op/s 2024-04-26T02:04:26.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:26 smithi151 ceph-mon[28057]: pgmap v340: 97 pgs: 97 active+clean; 160 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 1.1 MiB/s wr, 366 op/s 2024-04-26T02:04:26.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:26 smithi172 ceph-mon[32785]: pgmap v340: 97 pgs: 97 active+clean; 160 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 1.1 MiB/s wr, 366 op/s 2024-04-26T02:04:28.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:28 smithi165 ceph-mon[32648]: pgmap v341: 97 pgs: 97 active+clean; 160 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 950 KiB/s wr, 320 op/s 2024-04-26T02:04:28.624 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:28 smithi151 ceph-mon[28057]: pgmap v341: 97 pgs: 97 active+clean; 160 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 950 KiB/s wr, 320 op/s 2024-04-26T02:04:28.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:28 smithi172 ceph-mon[32785]: pgmap v341: 97 pgs: 97 active+clean; 160 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 950 KiB/s wr, 320 op/s 2024-04-26T02:04:29.127 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.127+0000 7f2796a00640 1 -- 172.21.15.151:0/20674130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2790100e10 msgr2=0x7f279010ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:29.127 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.127+0000 7f2796a00640 1 --2- 172.21.15.151:0/20674130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2790100e10 0x7f279010ede0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f2788009cd0 tx=0x7f278802a8e0 comp rx=0 tx=0).stop 2024-04-26T02:04:29.127 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.127+0000 7f2796a00640 1 -- 172.21.15.151:0/20674130 shutdown_connections 2024-04-26T02:04:29.127 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.127+0000 7f2796a00640 1 --2- 172.21.15.151:0/20674130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2790100e10 0x7f279010ede0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:29.128 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.127+0000 7f2796a00640 1 --2- 172.21.15.151:0/20674130 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2790100450 0x7f27901008d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:29.128 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.127+0000 7f2796a00640 1 --2- 172.21.15.151:0/20674130 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2790106f30 0x7f2790107310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:29.128 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.127+0000 7f2796a00640 1 -- 172.21.15.151:0/20674130 >> 172.21.15.151:0/20674130 conn(0x7f27900773d0 msgr2=0x7f27900fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:29.128 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.127+0000 7f2796a00640 1 -- 172.21.15.151:0/20674130 shutdown_connections 2024-04-26T02:04:29.128 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.128+0000 7f2796a00640 1 -- 172.21.15.151:0/20674130 wait complete. 2024-04-26T02:04:29.131 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.129+0000 7f2796a00640 1 Processor -- start 2024-04-26T02:04:29.131 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.129+0000 7f2796a00640 1 -- start start 2024-04-26T02:04:29.131 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.130+0000 7f2796a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2790100450 0x7f279019e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:29.131 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.130+0000 7f2796a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2790100e10 0x7f279019e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:29.132 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.130+0000 7f2796a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2790106f30 0x7f2790198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:29.132 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.130+0000 7f2796a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f27901122f0 con 0x7f2790100e10 2024-04-26T02:04:29.132 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.130+0000 7f2796a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2790112170 con 0x7f2790100450 2024-04-26T02:04:29.132 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.130+0000 7f2796a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2790112470 con 0x7f2790106f30 2024-04-26T02:04:29.132 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.130+0000 7f278fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2790100450 0x7f279019e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:29.132 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.130+0000 7f278fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2790100450 0x7f279019e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:60290/0 (socket says 172.21.15.151:60290) 2024-04-26T02:04:29.132 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.130+0000 7f278fe00640 1 -- 172.21.15.151:0/1831360675 learned_addr learned my addr 172.21.15.151:0/1831360675 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:04:29.132 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.130+0000 7f2794a00640 1 --2- 172.21.15.151:0/1831360675 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2790106f30 0x7f2790198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:29.133 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.130+0000 7f2787e00640 1 --2- 172.21.15.151:0/1831360675 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2790100e10 0x7f279019e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:29.133 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.130+0000 7f278fe00640 1 -- 172.21.15.151:0/1831360675 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2790106f30 msgr2=0x7f2790198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:29.133 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.130+0000 7f278fe00640 1 --2- 172.21.15.151:0/1831360675 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2790106f30 0x7f2790198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:29.133 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.130+0000 7f278fe00640 1 -- 172.21.15.151:0/1831360675 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2790100e10 msgr2=0x7f279019e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:29.133 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.130+0000 7f278fe00640 1 --2- 172.21.15.151:0/1831360675 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2790100e10 0x7f279019e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:29.133 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.130+0000 7f278fe00640 1 -- 172.21.15.151:0/1831360675 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2790198d10 con 0x7f2790100450 2024-04-26T02:04:29.133 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.131+0000 7f278fe00640 1 --2- 172.21.15.151:0/1831360675 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2790100450 0x7f279019e3e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f278000e9f0 tx=0x7f278000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:29.133 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.131+0000 7f278d600640 1 -- 172.21.15.151:0/1831360675 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278000cd80 con 0x7f2790100450 2024-04-26T02:04:29.134 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.131+0000 7f278d600640 1 -- 172.21.15.151:0/1831360675 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27800108e0 con 0x7f2790100450 2024-04-26T02:04:29.134 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.131+0000 7f2796a00640 1 -- 172.21.15.151:0/1831360675 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2790199000 con 0x7f2790100450 2024-04-26T02:04:29.134 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.131+0000 7f278d600640 1 -- 172.21.15.151:0/1831360675 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2780018980 con 0x7f2790100450 2024-04-26T02:04:29.134 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.131+0000 7f2796a00640 1 -- 172.21.15.151:0/1831360675 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27901a52a0 con 0x7f2790100450 2024-04-26T02:04:29.136 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.135+0000 7f27997e8740 1 Processor -- start 2024-04-26T02:04:29.136 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.136+0000 7f27997e8740 1 -- start start 2024-04-26T02:04:29.137 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.136+0000 7f278d600640 1 -- 172.21.15.151:0/1831360675 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f2780010430 con 0x7f2790100450 2024-04-26T02:04:29.137 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.136+0000 7f27997e8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d50088820 0x562d50088c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:29.137 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.136+0000 7f27997e8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d50089140 0x562d500934c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:29.137 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.136+0000 7f27997e8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d50093a00 0x562d50095df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:29.137 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.136+0000 7f27997e8740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562d4ffee900 con 0x562d50093a00 2024-04-26T02:04:29.137 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.136+0000 7f27997e8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562d4ff94ef0 con 0x562d50089140 2024-04-26T02:04:29.137 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.136+0000 7f27997e8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562d4ffea8c0 con 0x562d50088820 2024-04-26T02:04:29.137 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.136+0000 7f2794a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d50088820 0x562d50088c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:29.137 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.136+0000 7f2787e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d50093a00 0x562d50095df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:29.137 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.136+0000 7f2794a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d50088820 0x562d50088c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:35564/0 (socket says 172.21.15.151:35564) 2024-04-26T02:04:29.137 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.136+0000 7f2794a00640 1 -- 172.21.15.151:0/3692986808 learned_addr learned my addr 172.21.15.151:0/3692986808 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:04:29.137 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.137+0000 7f278fe00640 1 --2- 172.21.15.151:0/3692986808 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d50089140 0x562d500934c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:29.137 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.137+0000 7f2794a00640 1 -- 172.21.15.151:0/3692986808 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d50089140 msgr2=0x562d500934c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:29.138 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.137+0000 7f2794a00640 1 --2- 172.21.15.151:0/3692986808 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d50089140 0x562d500934c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:29.138 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.137+0000 7f2794a00640 1 -- 172.21.15.151:0/3692986808 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d50093a00 msgr2=0x562d50095df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:29.138 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.137+0000 7f2794a00640 1 --2- 172.21.15.151:0/3692986808 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d50093a00 0x562d50095df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:29.138 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.137+0000 7f2794a00640 1 -- 172.21.15.151:0/3692986808 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d4fd01210 con 0x562d50088820 2024-04-26T02:04:29.138 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.137+0000 7f2794a00640 1 --2- 172.21.15.151:0/3692986808 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d50088820 0x562d50088c00 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2788009cd0 tx=0x7f2788005f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64f6d9e780de233a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:29.138 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.138+0000 7f2785200640 1 -- 172.21.15.151:0/3692986808 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2788048070 con 0x562d50088820 2024-04-26T02:04:29.138 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.138+0000 7f2785200640 1 -- 172.21.15.151:0/3692986808 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2788036070 con 0x562d50088820 2024-04-26T02:04:29.138 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.138+0000 7f2785200640 1 -- 172.21.15.151:0/3692986808 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2788004460 con 0x562d50088820 2024-04-26T02:04:29.141 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.138+0000 7f278d600640 1 --2- 172.21.15.151:0/1831360675 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f27740a3dc0 0x7f27740a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:29.141 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.138+0000 7f27997e8740 1 -- 172.21.15.151:0/3692986808 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d50088820 msgr2=0x562d50088c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:29.142 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.138+0000 7f27997e8740 1 --2- 172.21.15.151:0/3692986808 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d50088820 0x562d50088c00 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2788009cd0 tx=0x7f2788005f80 comp rx=0 tx=0).stop 2024-04-26T02:04:29.142 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.139+0000 7f27997e8740 1 -- 172.21.15.151:0/3692986808 shutdown_connections 2024-04-26T02:04:29.142 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.139+0000 7f27997e8740 1 --2- 172.21.15.151:0/3692986808 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d50093a00 0x562d50095df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:29.142 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.139+0000 7f27997e8740 1 --2- 172.21.15.151:0/3692986808 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d50089140 0x562d500934c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:29.142 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.139+0000 7f27997e8740 1 --2- 172.21.15.151:0/3692986808 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d50088820 0x562d50088c00 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:29.142 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.139+0000 7f27997e8740 1 -- 172.21.15.151:0/3692986808 >> 172.21.15.151:0/3692986808 conn(0x562d50084d30 msgr2=0x562d50085110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:29.142 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.139+0000 7f278fe00640 1 --2- 172.21.15.151:0/1831360675 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f27740a3dc0 0x7f27740a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:29.142 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.139+0000 7f278d600640 1 -- 172.21.15.151:0/1831360675 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8455+0+0 (secure 0 0 0) 0x7f27800d0ad0 con 0x7f2790100450 2024-04-26T02:04:29.142 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.139+0000 7f27997e8740 1 -- 172.21.15.151:0/3692986808 shutdown_connections 2024-04-26T02:04:29.142 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.139+0000 7f27997e8740 1 -- 172.21.15.151:0/3692986808 wait complete. 2024-04-26T02:04:29.142 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.139+0000 7f278fe00640 1 --2- 172.21.15.151:0/1831360675 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f27740a3dc0 0x7f27740a6280 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f27800ce310 tx=0x7f27800d4610 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.139+0000 7f27997e8740 1 -- start start 2024-04-26T02:04:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.140+0000 7f27997e8740 1 Processor -- start 2024-04-26T02:04:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.140+0000 7f27997e8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d50088820 0x562d50090a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.140+0000 7f27997e8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d50089140 0x562d50090f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.140+0000 7f27997e8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d50093a00 0x562d500914d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.140+0000 7f27997e8740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562d500965d0 con 0x562d50093a00 2024-04-26T02:04:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.140+0000 7f27997e8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562d4ffa8310 con 0x562d50088820 2024-04-26T02:04:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.140+0000 7f27997e8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562d50096750 con 0x562d50089140 2024-04-26T02:04:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.140+0000 7f27997e8740 10 client.0 _ll_register_callbacks cb 0x562d4ff76cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:04:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.140+0000 7f2787e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d50089140 0x562d50090f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.140+0000 7f2787e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d50089140 0x562d50090f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:35568/0 (socket says 172.21.15.151:35568) 2024-04-26T02:04:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.140+0000 7f2794a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d50093a00 0x562d500914d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.140+0000 7f2787e00640 1 -- 172.21.15.151:0/3797554955 learned_addr learned my addr 172.21.15.151:0/3797554955 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:04:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.140+0000 7f2794a00640 1 --2- 172.21.15.151:0/3797554955 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d50088820 0x562d50090a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:29.144 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.141+0000 7f2787e00640 1 -- 172.21.15.151:0/3797554955 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d50088820 msgr2=0x562d50090a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:29.144 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.141+0000 7f2787e00640 1 --2- 172.21.15.151:0/3797554955 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d50088820 0x562d50090a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:29.144 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.141+0000 7f2787e00640 1 -- 172.21.15.151:0/3797554955 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d50093a00 msgr2=0x562d500914d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:29.144 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.141+0000 7f2787e00640 1 --2- 172.21.15.151:0/3797554955 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d50093a00 0x562d500914d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:29.144 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.141+0000 7f2787e00640 1 -- 172.21.15.151:0/3797554955 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d4fdcd620 con 0x562d50089140 2024-04-26T02:04:29.144 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.141+0000 7f2794a00640 1 --2- 172.21.15.151:0/3797554955 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d50088820 0x562d50090a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:04:29.144 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.141+0000 7f2787e00640 1 --2- 172.21.15.151:0/3797554955 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d50089140 0x562d50090f90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f27780124d0 tx=0x7f2778012990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e58f37bca5143d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:29.144 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.141+0000 7f277d600640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-26T02:04:29.144 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.141+0000 7f277d600640 1 -- 172.21.15.151:0/3797554955 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2778004320 con 0x562d50089140 2024-04-26T02:04:29.144 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.141+0000 7f277d600640 1 -- 172.21.15.151:0/3797554955 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8455+0+0 (secure 0 0 0) 0x7f27780035f0 con 0x562d50089140 2024-04-26T02:04:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.141+0000 7f27997e8740 1 -- 172.21.15.151:0/3797554955 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2788036070 con 0x562d50089140 2024-04-26T02:04:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.142+0000 7f277d600640 1 -- 172.21.15.151:0/3797554955 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f27780044c0 con 0x562d50089140 2024-04-26T02:04:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.142+0000 7f27997e8740 10 client.26470 fetch_fsmap learned FSMap version 35 2024-04-26T02:04:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.142+0000 7f27997e8740 1 -- 172.21.15.151:0/3797554955 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562d500a2950 con 0x562d50089140 2024-04-26T02:04:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.142+0000 7f277d600640 1 -- 172.21.15.151:0/3797554955 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2788036070 con 0x562d50089140 2024-04-26T02:04:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.142+0000 7f277d600640 1 -- 172.21.15.151:0/3797554955 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 35) ==== 8338+0+0 (secure 0 0 0) 0x7f2778011af0 con 0x562d50089140 2024-04-26T02:04:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.142+0000 7f27997e8740 10 client.26470 fetch_fsmap finished waiting for FSMap version 35 2024-04-26T02:04:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.142+0000 7f27997e8740 10 client.26470 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:04:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.142+0000 7f27997e8740 20 client.26470 populate_metadata read hostname 'smithi151' 2024-04-26T02:04:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.142+0000 7f27997e8740 1 --2- 172.21.15.151:0/3797554955 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x562d500a3e10 0x562d500a6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.142+0000 7f27997e8740 4 client.26470 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:04:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.142+0000 7f27997e8740 1 -- 172.21.15.151:0/3797554955 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d500028b0 con 0x562d500a3e10 2024-04-26T02:04:29.147 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.143+0000 7f2794a00640 1 --2- 172.21.15.151:0/3797554955 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x562d500a3e10 0x562d500a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:29.147 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.143+0000 7f277d600640 10 client.26470 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:04:29.147 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.143+0000 7f2794a00640 1 --2- 172.21.15.151:0/3797554955 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x562d500a3e10 0x562d500a6240 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4237f40a460ef369 server_cookie=f47b908de6b43ff6 in_seq=0 out_seq=0 2024-04-26T02:04:29.147 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.144+0000 7f277d600640 1 -- 172.21.15.151:0/3797554955 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562d500028b0 con 0x562d500a3e10 2024-04-26T02:04:29.147 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.144+0000 7f277d600640 10 client.26470 handle_command_reply: tid=0 2024-04-26T02:04:29.147 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.144+0000 7f27997e8740 1 client.26470 shutdown 2024-04-26T02:04:29.147 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.144+0000 7f27997e8740 1 -- 172.21.15.151:0/3797554955 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d50089140 msgr2=0x562d50090f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:29.147 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.144+0000 7f27997e8740 1 --2- 172.21.15.151:0/3797554955 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d50089140 0x562d50090f90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f27780124d0 tx=0x7f2778012990 comp rx=0 tx=0).stop 2024-04-26T02:04:29.147 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.145+0000 7f27997e8740 1 -- 172.21.15.151:0/3797554955 shutdown_connections 2024-04-26T02:04:29.147 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.145+0000 7f27997e8740 1 --2- 172.21.15.151:0/3797554955 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x562d500a3e10 0x562d500a6240 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:29.148 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.145+0000 7f27997e8740 1 --2- 172.21.15.151:0/3797554955 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d50093a00 0x562d500914d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:29.148 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.145+0000 7f277d600640 0 client.26470 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:04:29.148 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.145+0000 7f27997e8740 1 --2- 172.21.15.151:0/3797554955 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d50089140 0x562d50090f90 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:29.148 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.145+0000 7f27997e8740 1 --2- 172.21.15.151:0/3797554955 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d50088820 0x562d50090a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:29.148 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.145+0000 7f27997e8740 1 -- 172.21.15.151:0/3797554955 >> 172.21.15.151:0/3797554955 conn(0x562d50084d30 msgr2=0x562d5008aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:29.148 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.145+0000 7f27997e8740 1 -- 172.21.15.151:0/3797554955 shutdown_connections 2024-04-26T02:04:29.148 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.145+0000 7f27997e8740 1 -- 172.21.15.151:0/3797554955 wait complete. 2024-04-26T02:04:29.148 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.145+0000 7f27997e8740 20 client.26470 trim_cache size 0 max 16384 2024-04-26T02:04:29.170 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.171+0000 7f27997e8740 1 Processor -- start 2024-04-26T02:04:29.170 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.171+0000 7f27997e8740 1 -- start start 2024-04-26T02:04:29.171 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.171+0000 7f27997e8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d50088820 0x562d5008b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:29.171 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.171+0000 7f27997e8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d50089140 0x562d50091120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:29.171 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.171+0000 7f27997e8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d50093a00 0x562d500abb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:29.171 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.171+0000 7f27997e8740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f27901b02b0 con 0x562d50088820 2024-04-26T02:04:29.171 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.171+0000 7f27997e8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f27901adb80 con 0x562d50089140 2024-04-26T02:04:29.171 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.171+0000 7f27997e8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f27901ab9f0 con 0x562d50093a00 2024-04-26T02:04:29.172 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.172+0000 7f2787e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d50089140 0x562d50091120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:29.172 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.172+0000 7f2787e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d50089140 0x562d50091120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:60324/0 (socket says 172.21.15.151:60324) 2024-04-26T02:04:29.172 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.172+0000 7f2787e00640 1 -- 172.21.15.151:0/3646304589 learned_addr learned my addr 172.21.15.151:0/3646304589 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:04:29.172 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.172+0000 7f2794a00640 1 --2- 172.21.15.151:0/3646304589 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d50088820 0x562d5008b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:29.172 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.172+0000 7f2794a00640 1 --2- 172.21.15.151:0/3646304589 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d50093a00 0x562d500abb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:29.172 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.172+0000 7f2787e00640 1 -- 172.21.15.151:0/3646304589 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d50093a00 msgr2=0x562d500abb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:29.172 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.172+0000 7f2787e00640 1 --2- 172.21.15.151:0/3646304589 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d50093a00 0x562d500abb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:29.173 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.172+0000 7f2787e00640 1 -- 172.21.15.151:0/3646304589 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d50088820 msgr2=0x562d5008b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:29.173 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.172+0000 7f2787e00640 1 --2- 172.21.15.151:0/3646304589 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d50088820 0x562d5008b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:29.173 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.172+0000 7f2787e00640 1 -- 172.21.15.151:0/3646304589 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d50088f70 con 0x562d50089140 2024-04-26T02:04:29.173 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.173+0000 7f2787e00640 1 --2- 172.21.15.151:0/3646304589 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d50089140 0x562d50091120 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2778000c00 tx=0x7f2778005900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e81beaef6167598 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:29.173 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.173+0000 7f277fe00640 1 -- 172.21.15.151:0/3646304589 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2778029040 con 0x562d50089140 2024-04-26T02:04:29.173 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.173+0000 7f277fe00640 1 -- 172.21.15.151:0/3646304589 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2778011b30 con 0x562d50089140 2024-04-26T02:04:29.173 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.173+0000 7f277fe00640 1 -- 172.21.15.151:0/3646304589 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277800fc40 con 0x562d50089140 2024-04-26T02:04:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.174+0000 7f27997e8740 1 -- 172.21.15.151:0/3646304589 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d50089140 msgr2=0x562d50091120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.174+0000 7f27997e8740 1 --2- 172.21.15.151:0/3646304589 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d50089140 0x562d50091120 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2778000c00 tx=0x7f2778005900 comp rx=0 tx=0).stop 2024-04-26T02:04:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.174+0000 7f27997e8740 1 -- 172.21.15.151:0/3646304589 shutdown_connections 2024-04-26T02:04:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.174+0000 7f27997e8740 1 --2- 172.21.15.151:0/3646304589 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d50093a00 0x562d500abb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.174+0000 7f27997e8740 1 --2- 172.21.15.151:0/3646304589 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d50089140 0x562d50091120 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.174+0000 7f27997e8740 1 --2- 172.21.15.151:0/3646304589 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d50088820 0x562d5008b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.174+0000 7f27997e8740 1 -- 172.21.15.151:0/3646304589 >> 172.21.15.151:0/3646304589 conn(0x562d50084d30 msgr2=0x562d50090a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.174+0000 7f27997e8740 1 -- 172.21.15.151:0/3646304589 shutdown_connections 2024-04-26T02:04:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.174+0000 7f27997e8740 1 -- 172.21.15.151:0/3646304589 wait complete. 2024-04-26T02:04:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.175+0000 7f27997e8740 1 -- start start 2024-04-26T02:04:29.177 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.175+0000 7f27997e8740 1 Processor -- start 2024-04-26T02:04:29.177 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.175+0000 7f27997e8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d50088820 0x562d50099740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:29.177 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.175+0000 7f27997e8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d50089140 0x562d50099c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:29.177 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.175+0000 7f27997e8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d50093a00 0x562d5010d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:29.177 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.175+0000 7f27997e8740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f27901ac040 con 0x562d50093a00 2024-04-26T02:04:29.177 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.175+0000 7f27997e8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f27901ad4b0 con 0x562d50088820 2024-04-26T02:04:29.177 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.175+0000 7f27997e8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f27901ac710 con 0x562d50089140 2024-04-26T02:04:29.177 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.175+0000 7f27997e8740 10 client.0 _ll_register_callbacks cb 0x7f2778009c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:04:29.177 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.176+0000 7f2794a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d50088820 0x562d50099740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:29.177 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.176+0000 7f2787e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d50089140 0x562d50099c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:29.177 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.176+0000 7f2794a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d50088820 0x562d50099740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:60336/0 (socket says 172.21.15.151:60336) 2024-04-26T02:04:29.177 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.176+0000 7f2794a00640 1 -- 172.21.15.151:0/604221521 learned_addr learned my addr 172.21.15.151:0/604221521 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:04:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.176+0000 7f2787e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d50089140 0x562d50099c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:35592/0 (socket says 172.21.15.151:35592) 2024-04-26T02:04:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.176+0000 7f2794a00640 1 --2- 172.21.15.151:0/604221521 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d50093a00 0x562d5010d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.176+0000 7f2787e00640 1 -- 172.21.15.151:0/604221521 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d50088820 msgr2=0x562d50099740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.176+0000 7f2787e00640 1 --2- 172.21.15.151:0/604221521 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d50088820 0x562d50099740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.176+0000 7f2787e00640 1 -- 172.21.15.151:0/604221521 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d50093a00 msgr2=0x562d5010d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.176+0000 7f2787e00640 1 --2- 172.21.15.151:0/604221521 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d50093a00 0x562d5010d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.176+0000 7f2787e00640 1 -- 172.21.15.151:0/604221521 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d5009a550 con 0x562d50089140 2024-04-26T02:04:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.176+0000 7f2787e00640 1 --2- 172.21.15.151:0/604221521 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d50089140 0x562d50099c80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2778011f90 tx=0x7f2778009fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf9c49abfce3b52a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.176+0000 7f277d600640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-26T02:04:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.177+0000 7f277d600640 1 -- 172.21.15.151:0/604221521 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277803b060 con 0x562d50089140 2024-04-26T02:04:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.177+0000 7f27997e8740 1 -- 172.21.15.151:0/604221521 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2778011b30 con 0x562d50089140 2024-04-26T02:04:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.177+0000 7f277d600640 1 -- 172.21.15.151:0/604221521 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8455+0+0 (secure 0 0 0) 0x7f277800fa80 con 0x562d50089140 2024-04-26T02:04:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.177+0000 7f277d600640 1 -- 172.21.15.151:0/604221521 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2778029070 con 0x562d50089140 2024-04-26T02:04:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.177+0000 7f27997e8740 10 client.26476 fetch_fsmap learned FSMap version 35 2024-04-26T02:04:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.177+0000 7f27997e8740 1 -- 172.21.15.151:0/604221521 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562d5010ee30 con 0x562d50089140 2024-04-26T02:04:29.180 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.177+0000 7f277d600640 1 -- 172.21.15.151:0/604221521 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2778011b30 con 0x562d50089140 2024-04-26T02:04:29.180 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.177+0000 7f277d600640 1 -- 172.21.15.151:0/604221521 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 35) ==== 8338+0+0 (secure 0 0 0) 0x7f2778040470 con 0x562d50089140 2024-04-26T02:04:29.180 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.177+0000 7f27997e8740 10 client.26476 fetch_fsmap finished waiting for FSMap version 35 2024-04-26T02:04:29.180 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.177+0000 7f27997e8740 10 client.26476 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:04:29.180 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.177+0000 7f27997e8740 20 client.26476 populate_metadata read hostname 'smithi151' 2024-04-26T02:04:29.180 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.178+0000 7f27997e8740 1 --2- 172.21.15.151:0/604221521 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x562d5010f550 0x562d50111970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:29.180 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.178+0000 7f27997e8740 4 client.26476 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T02:04:29.180 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.178+0000 7f27997e8740 1 -- 172.21.15.151:0/604221521 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562d4fffc320 con 0x562d5010f550 2024-04-26T02:04:29.180 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.178+0000 7f2794a00640 1 --2- 172.21.15.151:0/604221521 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x562d5010f550 0x562d50111970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:29.180 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.178+0000 7f2794a00640 1 --2- 172.21.15.151:0/604221521 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x562d5010f550 0x562d50111970 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9fbf736d259b8ed server_cookie=a40ca63fe4bd4251 in_seq=0 out_seq=0 2024-04-26T02:04:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.178+0000 7f277d600640 10 client.26476 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:04:29.190 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.190+0000 7f277d600640 1 -- 172.21.15.151:0/604221521 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562d4fffc320 con 0x562d5010f550 2024-04-26T02:04:29.190 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.190+0000 7f277d600640 10 client.26476 handle_command_reply: tid=0 2024-04-26T02:04:29.190 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.190+0000 7f27997e8740 1 client.26476 shutdown 2024-04-26T02:04:29.190 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.190+0000 7f27997e8740 1 -- 172.21.15.151:0/604221521 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d50089140 msgr2=0x562d50099c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:29.190 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.190+0000 7f27997e8740 1 --2- 172.21.15.151:0/604221521 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d50089140 0x562d50099c80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2778011f90 tx=0x7f2778009fa0 comp rx=0 tx=0).stop 2024-04-26T02:04:29.190 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.191+0000 7f27997e8740 1 -- 172.21.15.151:0/604221521 shutdown_connections 2024-04-26T02:04:29.191 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.191+0000 7f27997e8740 1 --2- 172.21.15.151:0/604221521 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x562d5010f550 0x562d50111970 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:29.191 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.191+0000 7f27997e8740 1 --2- 172.21.15.151:0/604221521 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d50093a00 0x562d5010d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:29.191 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.191+0000 7f27997e8740 1 --2- 172.21.15.151:0/604221521 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d50089140 0x562d50099c80 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:29.191 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.191+0000 7f27997e8740 1 --2- 172.21.15.151:0/604221521 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d50088820 0x562d50099740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:29.191 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.191+0000 7f27997e8740 1 -- 172.21.15.151:0/604221521 >> 172.21.15.151:0/604221521 conn(0x562d50084d30 msgr2=0x562d50090a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:29.191 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.191+0000 7f277d600640 0 client.26476 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:04:29.191 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.191+0000 7f27997e8740 1 -- 172.21.15.151:0/604221521 shutdown_connections 2024-04-26T02:04:29.191 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.191+0000 7f27997e8740 1 -- 172.21.15.151:0/604221521 wait complete. 2024-04-26T02:04:29.191 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.191+0000 7f27997e8740 20 client.26476 trim_cache size 0 max 16384 2024-04-26T02:04:29.192 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:{ 2024-04-26T02:04:29.192 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "message": "", 2024-04-26T02:04:29.192 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "return_code": 0 2024-04-26T02:04:29.192 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:} 2024-04-26T02:04:29.193 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.192+0000 7f2796a00640 1 -- 172.21.15.151:0/1831360675 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f27740a3dc0 msgr2=0x7f27740a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:29.193 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.192+0000 7f2796a00640 1 --2- 172.21.15.151:0/1831360675 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f27740a3dc0 0x7f27740a6280 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f27800ce310 tx=0x7f27800d4610 comp rx=0 tx=0).stop 2024-04-26T02:04:29.193 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.192+0000 7f2796a00640 1 -- 172.21.15.151:0/1831360675 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2790100450 msgr2=0x7f279019e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:29.193 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.192+0000 7f2796a00640 1 --2- 172.21.15.151:0/1831360675 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2790100450 0x7f279019e3e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f278000e9f0 tx=0x7f278000eeb0 comp rx=0 tx=0).stop 2024-04-26T02:04:29.193 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.193+0000 7f2796a00640 1 -- 172.21.15.151:0/1831360675 shutdown_connections 2024-04-26T02:04:29.193 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.193+0000 7f2796a00640 1 --2- 172.21.15.151:0/1831360675 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f27740a3dc0 0x7f27740a6280 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:29.193 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.193+0000 7f2796a00640 1 --2- 172.21.15.151:0/1831360675 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2790106f30 0x7f2790198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:29.193 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.193+0000 7f2796a00640 1 --2- 172.21.15.151:0/1831360675 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2790100e10 0x7f279019e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:29.193 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.193+0000 7f2796a00640 1 --2- 172.21.15.151:0/1831360675 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2790100450 0x7f279019e3e0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:29.193 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.193+0000 7f2796a00640 1 -- 172.21.15.151:0/1831360675 >> 172.21.15.151:0/1831360675 conn(0x7f27900773d0 msgr2=0x7f2790104eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:29.193 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.193+0000 7f2796a00640 1 -- 172.21.15.151:0/1831360675 shutdown_connections 2024-04-26T02:04:29.194 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:29.193+0000 7f2796a00640 1 -- 172.21.15.151:0/1831360675 wait complete. 2024-04-26T02:04:30.287 INFO:journalctl@ceph.mds.f.smithi172.stdout:Apr 26 02:04:30 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-f[63189]: ignoring --setuser ceph since I am not root 2024-04-26T02:04:30.287 INFO:journalctl@ceph.mds.f.smithi172.stdout:Apr 26 02:04:30 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-f[63189]: ignoring --setgroup ceph since I am not root 2024-04-26T02:04:30.288 INFO:journalctl@ceph.mds.f.smithi172.stdout:Apr 26 02:04:30 smithi172 ceph-mds[63193]: -- 172.21.15.172:0/3218355209 <== mon.1 v2:172.21.15.172:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555d56135a40 con 0x555d56109000 2024-04-26T02:04:30.288 INFO:journalctl@ceph.mds.f.smithi172.stdout:Apr 26 02:04:30 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-f[63189]: starting mds.f at 2024-04-26T02:04:30.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:30 smithi165 ceph-mon[32648]: pgmap v342: 97 pgs: 97 active+clean; 166 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.4 MiB/s wr, 467 op/s 2024-04-26T02:04:30.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:30 smithi165 ceph-mon[32648]: daemon mds.f restarted 2024-04-26T02:04:30.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:30 smithi151 ceph-mon[28057]: pgmap v342: 97 pgs: 97 active+clean; 166 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.4 MiB/s wr, 467 op/s 2024-04-26T02:04:30.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:30 smithi151 ceph-mon[28057]: daemon mds.f restarted 2024-04-26T02:04:30.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:30 smithi172 ceph-mon[32785]: pgmap v342: 97 pgs: 97 active+clean; 166 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.4 MiB/s wr, 467 op/s 2024-04-26T02:04:30.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:30 smithi172 ceph-mon[32785]: daemon mds.f restarted 2024-04-26T02:04:31.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:31 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T02:04:31.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:31 smithi165 ceph-mon[32648]: osdmap e107: 12 total, 12 up, 12 in 2024-04-26T02:04:31.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:31 smithi165 ceph-mon[32648]: mds.? [v2:172.21.15.172:6834/1218148590,v1:172.21.15.172:6835/1218148590] up:boot 2024-04-26T02:04:31.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:31 smithi165 ceph-mon[32648]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T02:04:31.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:31 smithi165 ceph-mon[32648]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T02:04:31.623 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:31 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T02:04:31.623 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:31 smithi151 ceph-mon[28057]: osdmap e107: 12 total, 12 up, 12 in 2024-04-26T02:04:31.623 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:31 smithi151 ceph-mon[28057]: mds.? [v2:172.21.15.172:6834/1218148590,v1:172.21.15.172:6835/1218148590] up:boot 2024-04-26T02:04:31.623 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:31 smithi151 ceph-mon[28057]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T02:04:31.623 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:31 smithi151 ceph-mon[28057]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T02:04:31.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:31 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T02:04:31.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:31 smithi172 ceph-mon[32785]: osdmap e107: 12 total, 12 up, 12 in 2024-04-26T02:04:31.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:31 smithi172 ceph-mon[32785]: mds.? [v2:172.21.15.172:6834/1218148590,v1:172.21.15.172:6835/1218148590] up:boot 2024-04-26T02:04:31.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:31 smithi172 ceph-mon[32785]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T02:04:31.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:31 smithi172 ceph-mon[32785]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T02:04:32.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:32 smithi151 ceph-mon[28057]: pgmap v344: 97 pgs: 97 active+clean; 166 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 1.1 MiB/s wr, 365 op/s 2024-04-26T02:04:32.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:32 smithi172 ceph-mon[32785]: pgmap v344: 97 pgs: 97 active+clean; 166 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 1.1 MiB/s wr, 365 op/s 2024-04-26T02:04:32.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:32 smithi165 ceph-mon[32648]: pgmap v344: 97 pgs: 97 active+clean; 166 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 1.1 MiB/s wr, 365 op/s 2024-04-26T02:04:34.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:34 smithi151 ceph-mon[28057]: pgmap v345: 97 pgs: 97 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 1.1 MiB/s wr, 381 op/s 2024-04-26T02:04:34.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:34 smithi172 ceph-mon[32785]: pgmap v345: 97 pgs: 97 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 1.1 MiB/s wr, 381 op/s 2024-04-26T02:04:34.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:34 smithi165 ceph-mon[32648]: pgmap v345: 97 pgs: 97 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 1.1 MiB/s wr, 381 op/s 2024-04-26T02:04:36.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:36 smithi151 ceph-mon[28057]: pgmap v346: 97 pgs: 97 active+clean; 154 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 1.1 MiB/s wr, 385 op/s 2024-04-26T02:04:36.662 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:36 smithi165 ceph-mon[32648]: pgmap v346: 97 pgs: 97 active+clean; 154 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 1.1 MiB/s wr, 385 op/s 2024-04-26T02:04:36.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:36 smithi172 ceph-mon[32785]: pgmap v346: 97 pgs: 97 active+clean; 154 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 1.1 MiB/s wr, 385 op/s 2024-04-26T02:04:38.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:38 smithi151 ceph-mon[28057]: pgmap v347: 97 pgs: 97 active+clean; 154 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 1.1 MiB/s wr, 385 op/s 2024-04-26T02:04:38.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:38 smithi172 ceph-mon[32785]: pgmap v347: 97 pgs: 97 active+clean; 154 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 1.1 MiB/s wr, 385 op/s 2024-04-26T02:04:38.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:38 smithi165 ceph-mon[32648]: pgmap v347: 97 pgs: 97 active+clean; 154 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 1.1 MiB/s wr, 385 op/s 2024-04-26T02:04:40.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:40 smithi151 ceph-mon[28057]: pgmap v348: 97 pgs: 97 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 1.2 MiB/s wr, 398 op/s 2024-04-26T02:04:40.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:40 smithi172 ceph-mon[32785]: pgmap v348: 97 pgs: 97 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 1.2 MiB/s wr, 398 op/s 2024-04-26T02:04:40.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:40 smithi165 ceph-mon[32648]: pgmap v348: 97 pgs: 97 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 1.2 MiB/s wr, 398 op/s 2024-04-26T02:04:42.352 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.354+0000 7f1ed9c00640 1 -- 172.21.15.151:0/3062267170 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ed4101fc0 msgr2=0x7f1ed41023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:42.352 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.354+0000 7f1ed9c00640 1 --2- 172.21.15.151:0/3062267170 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ed4101fc0 0x7f1ed41023a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1ec4009900 tx=0x7f1ec402a410 comp rx=0 tx=0).stop 2024-04-26T02:04:42.353 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.354+0000 7f1ed9c00640 1 -- 172.21.15.151:0/3062267170 shutdown_connections 2024-04-26T02:04:42.353 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.354+0000 7f1ed9c00640 1 --2- 172.21.15.151:0/3062267170 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1ed4107090 0x7f1ed410ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:42.353 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.354+0000 7f1ed9c00640 1 --2- 172.21.15.151:0/3062267170 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ed4102970 0x7f1ed41069b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:42.353 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.354+0000 7f1ed9c00640 1 --2- 172.21.15.151:0/3062267170 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ed4101fc0 0x7f1ed41023a0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:42.353 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.354+0000 7f1ed9c00640 1 -- 172.21.15.151:0/3062267170 >> 172.21.15.151:0/3062267170 conn(0x7f1ed40fd8a0 msgr2=0x7f1ed40ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:42.353 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.354+0000 7f1ed9c00640 1 -- 172.21.15.151:0/3062267170 shutdown_connections 2024-04-26T02:04:42.353 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.354+0000 7f1ed9c00640 1 -- 172.21.15.151:0/3062267170 wait complete. 2024-04-26T02:04:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.355+0000 7f1ed9c00640 1 Processor -- start 2024-04-26T02:04:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.355+0000 7f1ed9c00640 1 -- start start 2024-04-26T02:04:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.356+0000 7f1ed9c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ed4101fc0 0x7f1ed41a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.356+0000 7f1ed9c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1ed4107090 0x7f1ed41a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.356+0000 7f1ed9c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ed419a680 0x7f1ed419ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.356+0000 7f1ed9c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed4112020 con 0x7f1ed4107090 2024-04-26T02:04:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.356+0000 7f1ed9c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed4111ea0 con 0x7f1ed4101fc0 2024-04-26T02:04:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.356+0000 7f1ed9c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed41121a0 con 0x7f1ed419a680 2024-04-26T02:04:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.356+0000 7f1ed3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ed4101fc0 0x7f1ed41a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.356+0000 7f1ed3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ed4101fc0 0x7f1ed41a05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:49164/0 (socket says 172.21.15.151:49164) 2024-04-26T02:04:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.356+0000 7f1ed3e00640 1 -- 172.21.15.151:0/3463249684 learned_addr learned my addr 172.21.15.151:0/3463249684 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:04:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.356+0000 7f1ed3400640 1 --2- 172.21.15.151:0/3463249684 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1ed4107090 0x7f1ed41a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.356+0000 7f1ed9200640 1 --2- 172.21.15.151:0/3463249684 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ed419a680 0x7f1ed419ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.356+0000 7f1ed3e00640 1 -- 172.21.15.151:0/3463249684 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ed419a680 msgr2=0x7f1ed419ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.356+0000 7f1ed3e00640 1 --2- 172.21.15.151:0/3463249684 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ed419a680 0x7f1ed419ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.356+0000 7f1ed3e00640 1 -- 172.21.15.151:0/3463249684 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1ed4107090 msgr2=0x7f1ed41a0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.356+0000 7f1ed3e00640 1 --2- 172.21.15.151:0/3463249684 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1ed4107090 0x7f1ed41a0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.356+0000 7f1ed3e00640 1 -- 172.21.15.151:0/3463249684 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ed419b280 con 0x7f1ed4101fc0 2024-04-26T02:04:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.356+0000 7f1ed3400640 1 --2- 172.21.15.151:0/3463249684 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1ed4107090 0x7f1ed41a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:04:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.356+0000 7f1ed9200640 1 --2- 172.21.15.151:0/3463249684 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ed419a680 0x7f1ed419ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:04:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.357+0000 7f1ed3e00640 1 --2- 172.21.15.151:0/3463249684 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ed4101fc0 0x7f1ed41a05c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1ec4005ab0 tx=0x7f1ec4003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.357+0000 7f1ed0c00640 1 -- 172.21.15.151:0/3463249684 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec403f070 con 0x7f1ed4101fc0 2024-04-26T02:04:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.357+0000 7f1ed0c00640 1 -- 172.21.15.151:0/3463249684 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ec4031b60 con 0x7f1ed4101fc0 2024-04-26T02:04:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.357+0000 7f1ed0c00640 1 -- 172.21.15.151:0/3463249684 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec4035030 con 0x7f1ed4101fc0 2024-04-26T02:04:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.357+0000 7f1ed9c00640 1 -- 172.21.15.151:0/3463249684 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ed419b510 con 0x7f1ed4101fc0 2024-04-26T02:04:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.358+0000 7f1ed9c00640 1 -- 172.21.15.151:0/3463249684 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ed41a7600 con 0x7f1ed4101fc0 2024-04-26T02:04:42.360 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.362+0000 7f1ed0c00640 1 -- 172.21.15.151:0/3463249684 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f1ec4042070 con 0x7f1ed4101fc0 2024-04-26T02:04:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.362+0000 7f1ede00a740 1 Processor -- start 2024-04-26T02:04:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.362+0000 7f1ede00a740 1 -- start start 2024-04-26T02:04:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.363+0000 7f1ede00a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bdf83829b0 0x55bdf8382d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.363+0000 7f1ede00a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdf83832d0 0x55bdf838d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.363+0000 7f1ede00a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdf838db90 0x55bdf838ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.363+0000 7f1ede00a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55bdf82e8900 con 0x55bdf83832d0 2024-04-26T02:04:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.363+0000 7f1ede00a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55bdf828eef0 con 0x55bdf83829b0 2024-04-26T02:04:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.363+0000 7f1ede00a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55bdf82e48c0 con 0x55bdf838db90 2024-04-26T02:04:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.363+0000 7f1ed9200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bdf83829b0 0x55bdf8382d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.363+0000 7f1ed9200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bdf83829b0 0x55bdf8382d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:49180/0 (socket says 172.21.15.151:49180) 2024-04-26T02:04:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.363+0000 7f1ed9200640 1 -- 172.21.15.151:0/1311252868 learned_addr learned my addr 172.21.15.151:0/1311252868 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:04:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.363+0000 7f1ed3400640 1 --2- 172.21.15.151:0/1311252868 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdf838db90 0x55bdf838ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.363+0000 7f1ed3e00640 1 --2- 172.21.15.151:0/1311252868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdf83832d0 0x55bdf838d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.364+0000 7f1ed9200640 1 -- 172.21.15.151:0/1311252868 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdf838db90 msgr2=0x55bdf838ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.364+0000 7f1ed9200640 1 --2- 172.21.15.151:0/1311252868 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdf838db90 0x55bdf838ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.364+0000 7f1ed9200640 1 -- 172.21.15.151:0/1311252868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdf83832d0 msgr2=0x55bdf838d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.364+0000 7f1ed9200640 1 --2- 172.21.15.151:0/1311252868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdf83832d0 0x55bdf838d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.364+0000 7f1ed9200640 1 -- 172.21.15.151:0/1311252868 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdf7ffb210 con 0x55bdf83829b0 2024-04-26T02:04:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.364+0000 7f1ed3400640 1 --2- 172.21.15.151:0/1311252868 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdf838db90 0x55bdf838ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:04:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.364+0000 7f1ed0c00640 1 --2- 172.21.15.151:0/3463249684 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f1eb80a3d90 0x7f1eb80a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.364+0000 7f1ed9200640 1 --2- 172.21.15.151:0/1311252868 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bdf83829b0 0x55bdf8382d90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc009fd0 tx=0x7f1ecc010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a557fdb4d2456a77 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.364+0000 7f1ed0c00640 1 -- 172.21.15.151:0/3463249684 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8411+0+0 (secure 0 0 0) 0x7f1ec40f0f30 con 0x7f1ed4101fc0 2024-04-26T02:04:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.364+0000 7f1ed3e00640 1 --2- 172.21.15.151:0/3463249684 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f1eb80a3d90 0x7f1eb80a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.364+0000 7f1ec9600640 1 -- 172.21.15.151:0/1311252868 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc011700 con 0x55bdf83829b0 2024-04-26T02:04:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.364+0000 7f1ec9600640 1 -- 172.21.15.151:0/1311252868 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ecc011d50 con 0x55bdf83829b0 2024-04-26T02:04:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.364+0000 7f1ec9600640 1 -- 172.21.15.151:0/1311252868 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc00e3c0 con 0x55bdf83829b0 2024-04-26T02:04:42.366 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.365+0000 7f1ede00a740 1 -- 172.21.15.151:0/1311252868 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bdf83829b0 msgr2=0x55bdf8382d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:42.367 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.365+0000 7f1ed3e00640 1 --2- 172.21.15.151:0/3463249684 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f1eb80a3d90 0x7f1eb80a6250 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1ec40ee7a0 tx=0x7f1ec4031400 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:42.367 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.365+0000 7f1ede00a740 1 --2- 172.21.15.151:0/1311252868 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bdf83829b0 0x55bdf8382d90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc009fd0 tx=0x7f1ecc010860 comp rx=0 tx=0).stop 2024-04-26T02:04:42.367 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.365+0000 7f1ede00a740 1 -- 172.21.15.151:0/1311252868 shutdown_connections 2024-04-26T02:04:42.367 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.365+0000 7f1ede00a740 1 --2- 172.21.15.151:0/1311252868 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdf838db90 0x55bdf838ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:42.367 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.365+0000 7f1ede00a740 1 --2- 172.21.15.151:0/1311252868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdf83832d0 0x55bdf838d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:42.367 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.365+0000 7f1ede00a740 1 --2- 172.21.15.151:0/1311252868 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bdf83829b0 0x55bdf8382d90 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:42.367 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.365+0000 7f1ede00a740 1 -- 172.21.15.151:0/1311252868 >> 172.21.15.151:0/1311252868 conn(0x55bdf837eec0 msgr2=0x55bdf837f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:42.367 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.366+0000 7f1ede00a740 1 -- 172.21.15.151:0/1311252868 shutdown_connections 2024-04-26T02:04:42.367 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.366+0000 7f1ede00a740 1 -- 172.21.15.151:0/1311252868 wait complete. 2024-04-26T02:04:42.367 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.366+0000 7f1ede00a740 1 -- start start 2024-04-26T02:04:42.367 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.366+0000 7f1ede00a740 1 Processor -- start 2024-04-26T02:04:42.367 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.367+0000 7f1ede00a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdf83829b0 0x55bdf838abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:42.367 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.367+0000 7f1ede00a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdf83832d0 0x55bdf838b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.367+0000 7f1ede00a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bdf838db90 0x55bdf838b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.367+0000 7f1ede00a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55bdf83907c0 con 0x55bdf83832d0 2024-04-26T02:04:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.367+0000 7f1ede00a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55bdf82a2310 con 0x55bdf838db90 2024-04-26T02:04:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.367+0000 7f1ede00a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55bdf8390940 con 0x55bdf83829b0 2024-04-26T02:04:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.367+0000 7f1ede00a740 10 client.0 _ll_register_callbacks cb 0x55bdf8270cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:04:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.367+0000 7f1ed9200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdf83829b0 0x55bdf838abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.367+0000 7f1ed9200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdf83829b0 0x55bdf838abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:50650/0 (socket says 172.21.15.151:50650) 2024-04-26T02:04:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.367+0000 7f1ed9200640 1 -- 172.21.15.151:0/533108853 learned_addr learned my addr 172.21.15.151:0/533108853 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:04:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.367+0000 7f1ed9200640 1 --2- 172.21.15.151:0/533108853 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bdf838db90 0x55bdf838b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.367+0000 7f1ed3400640 1 --2- 172.21.15.151:0/533108853 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdf83832d0 0x55bdf838b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.368+0000 7f1ed9200640 1 -- 172.21.15.151:0/533108853 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bdf838db90 msgr2=0x55bdf838b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.368+0000 7f1ed9200640 1 --2- 172.21.15.151:0/533108853 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bdf838db90 0x55bdf838b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.368+0000 7f1ed9200640 1 -- 172.21.15.151:0/533108853 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdf83832d0 msgr2=0x55bdf838b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.368+0000 7f1ed9200640 1 --2- 172.21.15.151:0/533108853 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdf83832d0 0x55bdf838b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.368+0000 7f1ed9200640 1 -- 172.21.15.151:0/533108853 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdf80c7620 con 0x55bdf83829b0 2024-04-26T02:04:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.368+0000 7f1ed9200640 1 --2- 172.21.15.151:0/533108853 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdf83829b0 0x55bdf838abb0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x55bdf8379460 tx=0x7f1ecc01ba60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f0f07594b1b9ff1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.368+0000 7f1ec2000640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-26T02:04:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.371+0000 7f1ec2000640 1 -- 172.21.15.151:0/533108853 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc024040 con 0x55bdf83829b0 2024-04-26T02:04:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.371+0000 7f1ec2000640 1 -- 172.21.15.151:0/533108853 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8411+0+0 (secure 0 0 0) 0x7f1ecc011a60 con 0x55bdf83829b0 2024-04-26T02:04:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.371+0000 7f1ede00a740 1 -- 172.21.15.151:0/533108853 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ecc011d50 con 0x55bdf83829b0 2024-04-26T02:04:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.372+0000 7f1ec2000640 1 -- 172.21.15.151:0/533108853 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f1ecc004a40 con 0x55bdf83829b0 2024-04-26T02:04:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.372+0000 7f1ede00a740 10 client.26494 fetch_fsmap learned FSMap version 37 2024-04-26T02:04:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.372+0000 7f1ede00a740 1 -- 172.21.15.151:0/533108853 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55bdf839cab0 con 0x55bdf83829b0 2024-04-26T02:04:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.372+0000 7f1ec2000640 1 -- 172.21.15.151:0/533108853 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ecc011d50 con 0x55bdf83829b0 2024-04-26T02:04:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.373+0000 7f1ec2000640 1 -- 172.21.15.151:0/533108853 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8338+0+0 (secure 0 0 0) 0x7f1ecc005770 con 0x55bdf83829b0 2024-04-26T02:04:42.372 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.373+0000 7f1ede00a740 10 client.26494 fetch_fsmap finished waiting for FSMap version 37 2024-04-26T02:04:42.372 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.373+0000 7f1ede00a740 10 client.26494 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:04:42.372 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.373+0000 7f1ede00a740 20 client.26494 populate_metadata read hostname 'smithi151' 2024-04-26T02:04:42.372 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.373+0000 7f1ede00a740 1 --2- 172.21.15.151:0/533108853 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55bdf839dec0 0x55bdf83a02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:42.372 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.373+0000 7f1ede00a740 4 client.26494 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:04:42.372 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.373+0000 7f1ede00a740 1 -- 172.21.15.151:0/533108853 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bdf82fc8b0 con 0x55bdf839dec0 2024-04-26T02:04:42.372 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.373+0000 7f1ed3400640 1 --2- 172.21.15.151:0/533108853 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55bdf839dec0 0x55bdf83a02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:42.373 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.374+0000 7f1ed3400640 1 --2- 172.21.15.151:0/533108853 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55bdf839dec0 0x55bdf83a02b0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a133ba71915eabda server_cookie=74a3ad84791cb5b0 in_seq=0 out_seq=0 2024-04-26T02:04:42.374 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.374+0000 7f1ec2000640 10 client.26494 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:04:42.374 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.375+0000 7f1ec2000640 1 -- 172.21.15.151:0/533108853 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bdf82fc8b0 con 0x55bdf839dec0 2024-04-26T02:04:42.374 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.375+0000 7f1ec2000640 10 client.26494 handle_command_reply: tid=0 2024-04-26T02:04:42.374 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.375+0000 7f1ede00a740 1 client.26494 shutdown 2024-04-26T02:04:42.374 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.375+0000 7f1ede00a740 1 -- 172.21.15.151:0/533108853 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdf83829b0 msgr2=0x55bdf838abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:42.374 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.375+0000 7f1ede00a740 1 --2- 172.21.15.151:0/533108853 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdf83829b0 0x55bdf838abb0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x55bdf8379460 tx=0x7f1ecc01ba60 comp rx=0 tx=0).stop 2024-04-26T02:04:42.374 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.375+0000 7f1ede00a740 1 -- 172.21.15.151:0/533108853 shutdown_connections 2024-04-26T02:04:42.374 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.375+0000 7f1ede00a740 1 --2- 172.21.15.151:0/533108853 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55bdf839dec0 0x55bdf83a02b0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:42.374 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.375+0000 7f1ede00a740 1 --2- 172.21.15.151:0/533108853 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bdf838db90 0x55bdf838b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:42.374 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.375+0000 7f1ede00a740 1 --2- 172.21.15.151:0/533108853 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdf83832d0 0x55bdf838b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:42.374 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.375+0000 7f1ec2000640 0 client.26494 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:04:42.374 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.375+0000 7f1ede00a740 1 --2- 172.21.15.151:0/533108853 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdf83829b0 0x55bdf838abb0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.375+0000 7f1ede00a740 1 -- 172.21.15.151:0/533108853 >> 172.21.15.151:0/533108853 conn(0x55bdf837eec0 msgr2=0x55bdf8384c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.375+0000 7f1ede00a740 1 -- 172.21.15.151:0/533108853 shutdown_connections 2024-04-26T02:04:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.375+0000 7f1ede00a740 1 -- 172.21.15.151:0/533108853 wait complete. 2024-04-26T02:04:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.376+0000 7f1ede00a740 20 client.26494 trim_cache size 0 max 16384 2024-04-26T02:04:42.404 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.404+0000 7f1ede00a740 1 Processor -- start 2024-04-26T02:04:42.405 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.404+0000 7f1ede00a740 1 -- start start 2024-04-26T02:04:42.405 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.404+0000 7f1ede00a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdf83829b0 0x55bdf8399680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:42.405 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.404+0000 7f1ede00a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bdf83832d0 0x55bdf8385370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:42.405 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.404+0000 7f1ede00a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdf838db90 0x55bdf838b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:42.405 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.404+0000 7f1ede00a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed41b23d0 con 0x55bdf838db90 2024-04-26T02:04:42.405 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.404+0000 7f1ede00a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed41afca0 con 0x55bdf83832d0 2024-04-26T02:04:42.405 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.404+0000 7f1ede00a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed41adaf0 con 0x55bdf83829b0 2024-04-26T02:04:42.405 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.405+0000 7f1ed3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bdf83832d0 0x55bdf8385370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:42.405 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.405+0000 7f1ed3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bdf83832d0 0x55bdf8385370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:49192/0 (socket says 172.21.15.151:49192) 2024-04-26T02:04:42.405 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.405+0000 7f1ed9200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdf83829b0 0x55bdf8399680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:42.405 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.405+0000 7f1ed3400640 1 -- 172.21.15.151:0/1842676261 learned_addr learned my addr 172.21.15.151:0/1842676261 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:04:42.405 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.405+0000 7f1ed9200640 1 --2- 172.21.15.151:0/1842676261 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdf838db90 0x55bdf838b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:42.406 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.405+0000 7f1ed3400640 1 -- 172.21.15.151:0/1842676261 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdf83829b0 msgr2=0x55bdf8399680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:42.406 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.405+0000 7f1ed3400640 1 --2- 172.21.15.151:0/1842676261 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdf83829b0 0x55bdf8399680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:42.406 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.405+0000 7f1ed3400640 1 -- 172.21.15.151:0/1842676261 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdf838db90 msgr2=0x55bdf838b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:42.406 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.405+0000 7f1ed3400640 1 --2- 172.21.15.151:0/1842676261 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdf838db90 0x55bdf838b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:42.406 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.405+0000 7f1ed3400640 1 -- 172.21.15.151:0/1842676261 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdf8379490 con 0x55bdf83832d0 2024-04-26T02:04:42.406 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.405+0000 7f1ed9200640 1 --2- 172.21.15.151:0/1842676261 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdf838db90 0x55bdf838b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:04:42.406 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.406+0000 7f1ed3400640 1 --2- 172.21.15.151:0/1842676261 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bdf83832d0 0x55bdf8385370 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1ed419bed0 tx=0x7f1ebc005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=593ead87deec2374 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:42.406 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.406+0000 7f1ec8c00640 1 -- 172.21.15.151:0/1842676261 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc0202f0 con 0x55bdf83832d0 2024-04-26T02:04:42.406 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.406+0000 7f1ec8c00640 1 -- 172.21.15.151:0/1842676261 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ebc011440 con 0x55bdf83832d0 2024-04-26T02:04:42.406 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.406+0000 7f1ec8c00640 1 -- 172.21.15.151:0/1842676261 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc020940 con 0x55bdf83832d0 2024-04-26T02:04:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.407+0000 7f1ede00a740 1 -- 172.21.15.151:0/1842676261 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bdf83832d0 msgr2=0x55bdf8385370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.407+0000 7f1ede00a740 1 --2- 172.21.15.151:0/1842676261 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bdf83832d0 0x55bdf8385370 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1ed419bed0 tx=0x7f1ebc005f90 comp rx=0 tx=0).stop 2024-04-26T02:04:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.407+0000 7f1ede00a740 1 -- 172.21.15.151:0/1842676261 shutdown_connections 2024-04-26T02:04:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.407+0000 7f1ede00a740 1 --2- 172.21.15.151:0/1842676261 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdf838db90 0x55bdf838b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.407+0000 7f1ede00a740 1 --2- 172.21.15.151:0/1842676261 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bdf83832d0 0x55bdf8385370 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.407+0000 7f1ede00a740 1 --2- 172.21.15.151:0/1842676261 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdf83829b0 0x55bdf8399680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.407+0000 7f1ede00a740 1 -- 172.21.15.151:0/1842676261 >> 172.21.15.151:0/1842676261 conn(0x55bdf837eec0 msgr2=0x55bdf838ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.407+0000 7f1ede00a740 1 -- 172.21.15.151:0/1842676261 shutdown_connections 2024-04-26T02:04:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.407+0000 7f1ede00a740 1 -- 172.21.15.151:0/1842676261 wait complete. 2024-04-26T02:04:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.408+0000 7f1ede00a740 1 -- start start 2024-04-26T02:04:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.408+0000 7f1ede00a740 1 Processor -- start 2024-04-26T02:04:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.408+0000 7f1ede00a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bdf83829b0 0x55bdf8396090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.408+0000 7f1ede00a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdf83832d0 0x55bdf8393420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.408+0000 7f1ede00a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdf838db90 0x55bdf8393b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.408+0000 7f1ede00a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed41ae160 con 0x55bdf83832d0 2024-04-26T02:04:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.408+0000 7f1ede00a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed41af5d0 con 0x55bdf83829b0 2024-04-26T02:04:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.408+0000 7f1ede00a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed41ae830 con 0x55bdf838db90 2024-04-26T02:04:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.408+0000 7f1ede00a740 10 client.0 _ll_register_callbacks cb 0x7f1ecc009ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:04:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.409+0000 7f1ed3400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdf83832d0 0x55bdf8393420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.409+0000 7f1ed9200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bdf83829b0 0x55bdf8396090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.409+0000 7f1ed3400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdf83832d0 0x55bdf8393420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:40646/0 (socket says 172.21.15.151:40646) 2024-04-26T02:04:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.409+0000 7f1ed3400640 1 -- 172.21.15.151:0/1250663545 learned_addr learned my addr 172.21.15.151:0/1250663545 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:04:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.409+0000 7f1ed9200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bdf83829b0 0x55bdf8396090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:49194/0 (socket says 172.21.15.151:49194) 2024-04-26T02:04:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.409+0000 7f1ed9200640 1 --2- 172.21.15.151:0/1250663545 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdf838db90 0x55bdf8393b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.409+0000 7f1ed3400640 1 -- 172.21.15.151:0/1250663545 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdf838db90 msgr2=0x55bdf8393b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.409+0000 7f1ed3400640 1 --2- 172.21.15.151:0/1250663545 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdf838db90 0x55bdf8393b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.409+0000 7f1ed3400640 1 -- 172.21.15.151:0/1250663545 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bdf83829b0 msgr2=0x55bdf8396090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.409+0000 7f1ed3400640 1 --2- 172.21.15.151:0/1250663545 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bdf83829b0 0x55bdf8396090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.409+0000 7f1ed3400640 1 -- 172.21.15.151:0/1250663545 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdf8403250 con 0x55bdf83832d0 2024-04-26T02:04:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.410+0000 7f1ed3400640 1 --2- 172.21.15.151:0/1250663545 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdf83832d0 0x55bdf8393420 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc000c00 tx=0x7f1ebc011eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f862fb1f73218686 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.410+0000 7f1ec2000640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-04-26T02:04:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.410+0000 7f1ec2000640 1 -- 172.21.15.151:0/1250663545 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc0052c0 con 0x55bdf83832d0 2024-04-26T02:04:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.410+0000 7f1ec2000640 1 -- 172.21.15.151:0/1250663545 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8411+0+0 (secure 0 0 0) 0x7f1ebc005460 con 0x55bdf83832d0 2024-04-26T02:04:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.410+0000 7f1ede00a740 1 -- 172.21.15.151:0/1250663545 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ebc011440 con 0x55bdf83832d0 2024-04-26T02:04:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.410+0000 7f1ec2000640 1 -- 172.21.15.151:0/1250663545 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f1ebc0115f0 con 0x55bdf83832d0 2024-04-26T02:04:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.410+0000 7f1ede00a740 10 client.16362 fetch_fsmap learned FSMap version 37 2024-04-26T02:04:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.410+0000 7f1ede00a740 1 -- 172.21.15.151:0/1250663545 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55bdf8404390 con 0x55bdf83832d0 2024-04-26T02:04:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.410+0000 7f1ec2000640 1 -- 172.21.15.151:0/1250663545 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ebc011440 con 0x55bdf83832d0 2024-04-26T02:04:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.411+0000 7f1ec2000640 1 -- 172.21.15.151:0/1250663545 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 37) ==== 8338+0+0 (secure 0 0 0) 0x7f1ebc039020 con 0x55bdf83832d0 2024-04-26T02:04:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.411+0000 7f1ede00a740 10 client.16362 fetch_fsmap finished waiting for FSMap version 37 2024-04-26T02:04:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.411+0000 7f1ede00a740 10 client.16362 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:04:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.411+0000 7f1ede00a740 20 client.16362 populate_metadata read hostname 'smithi151' 2024-04-26T02:04:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.411+0000 7f1ede00a740 1 --2- 172.21.15.151:0/1250663545 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55bdf84049e0 0x55bdf8406e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.411+0000 7f1ede00a740 4 client.16362 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T02:04:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.411+0000 7f1ede00a740 1 -- 172.21.15.151:0/1250663545 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bdf82f6320 con 0x55bdf84049e0 2024-04-26T02:04:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.411+0000 7f1ed9200640 1 --2- 172.21.15.151:0/1250663545 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55bdf84049e0 0x55bdf8406e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:42.414 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.412+0000 7f1ed9200640 1 --2- 172.21.15.151:0/1250663545 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55bdf84049e0 0x55bdf8406e00 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2fe138fa176ca5c server_cookie=daf7ed11a9fe2e4d in_seq=0 out_seq=0 2024-04-26T02:04:42.414 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.412+0000 7f1ec2000640 10 client.16362 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:04:42.427 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.429+0000 7f1ec2000640 1 -- 172.21.15.151:0/1250663545 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bdf82f6320 con 0x55bdf84049e0 2024-04-26T02:04:42.427 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.429+0000 7f1ec2000640 10 client.16362 handle_command_reply: tid=0 2024-04-26T02:04:42.427 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.429+0000 7f1ede00a740 1 client.16362 shutdown 2024-04-26T02:04:42.428 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.430+0000 7f1ede00a740 1 -- 172.21.15.151:0/1250663545 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdf83832d0 msgr2=0x55bdf8393420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:42.428 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.430+0000 7f1ede00a740 1 --2- 172.21.15.151:0/1250663545 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdf83832d0 0x55bdf8393420 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc000c00 tx=0x7f1ebc011eb0 comp rx=0 tx=0).stop 2024-04-26T02:04:42.429 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.430+0000 7f1ede00a740 1 -- 172.21.15.151:0/1250663545 shutdown_connections 2024-04-26T02:04:42.429 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.430+0000 7f1ede00a740 1 --2- 172.21.15.151:0/1250663545 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x55bdf84049e0 0x55bdf8406e00 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:42.429 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.430+0000 7f1ede00a740 1 --2- 172.21.15.151:0/1250663545 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdf838db90 0x55bdf8393b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:42.429 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.430+0000 7f1ede00a740 1 --2- 172.21.15.151:0/1250663545 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdf83832d0 0x55bdf8393420 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:42.429 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.430+0000 7f1ede00a740 1 --2- 172.21.15.151:0/1250663545 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bdf83829b0 0x55bdf8396090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:42.429 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.430+0000 7f1ec2000640 0 client.16362 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:04:42.429 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.430+0000 7f1ede00a740 1 -- 172.21.15.151:0/1250663545 >> 172.21.15.151:0/1250663545 conn(0x55bdf837eec0 msgr2=0x55bdf837d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:42.429 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.430+0000 7f1ede00a740 1 -- 172.21.15.151:0/1250663545 shutdown_connections 2024-04-26T02:04:42.429 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.430+0000 7f1ede00a740 1 -- 172.21.15.151:0/1250663545 wait complete. 2024-04-26T02:04:42.429 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.430+0000 7f1ede00a740 20 client.16362 trim_cache size 0 max 16384 2024-04-26T02:04:42.430 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:{ 2024-04-26T02:04:42.430 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "message": "", 2024-04-26T02:04:42.430 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "return_code": 0 2024-04-26T02:04:42.430 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:} 2024-04-26T02:04:42.430 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.431+0000 7f1ed9c00640 1 -- 172.21.15.151:0/3463249684 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f1eb80a3d90 msgr2=0x7f1eb80a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:42.431 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.431+0000 7f1ed9c00640 1 --2- 172.21.15.151:0/3463249684 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f1eb80a3d90 0x7f1eb80a6250 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1ec40ee7a0 tx=0x7f1ec4031400 comp rx=0 tx=0).stop 2024-04-26T02:04:42.431 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.431+0000 7f1ed9c00640 1 -- 172.21.15.151:0/3463249684 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ed4101fc0 msgr2=0x7f1ed41a05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:42.431 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.431+0000 7f1ed9c00640 1 --2- 172.21.15.151:0/3463249684 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ed4101fc0 0x7f1ed41a05c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1ec4005ab0 tx=0x7f1ec4003c10 comp rx=0 tx=0).stop 2024-04-26T02:04:42.431 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.431+0000 7f1ed9c00640 1 -- 172.21.15.151:0/3463249684 shutdown_connections 2024-04-26T02:04:42.431 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.431+0000 7f1ed9c00640 1 --2- 172.21.15.151:0/3463249684 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f1eb80a3d90 0x7f1eb80a6250 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:42.431 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.431+0000 7f1ed9c00640 1 --2- 172.21.15.151:0/3463249684 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ed419a680 0x7f1ed419ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:42.431 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.431+0000 7f1ed9c00640 1 --2- 172.21.15.151:0/3463249684 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1ed4107090 0x7f1ed41a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:42.431 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.431+0000 7f1ed9c00640 1 --2- 172.21.15.151:0/3463249684 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ed4101fc0 0x7f1ed41a05c0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:42.431 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.431+0000 7f1ed9c00640 1 -- 172.21.15.151:0/3463249684 >> 172.21.15.151:0/3463249684 conn(0x7f1ed40fd8a0 msgr2=0x7f1ed40ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:42.431 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.431+0000 7f1ed9c00640 1 -- 172.21.15.151:0/3463249684 shutdown_connections 2024-04-26T02:04:42.431 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:42.431+0000 7f1ed9c00640 1 -- 172.21.15.151:0/3463249684 wait complete. 2024-04-26T02:04:42.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:42 smithi151 ceph-mon[28057]: pgmap v349: 97 pgs: 97 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.1 MiB/s wr, 374 op/s 2024-04-26T02:04:42.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:42 smithi172 ceph-mon[32785]: pgmap v349: 97 pgs: 97 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.1 MiB/s wr, 374 op/s 2024-04-26T02:04:42.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:42 smithi165 ceph-mon[32648]: pgmap v349: 97 pgs: 97 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.1 MiB/s wr, 374 op/s 2024-04-26T02:04:44.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:44 smithi172 ceph-mon[32785]: pgmap v350: 97 pgs: 97 active+clean; 168 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 1.4 MiB/s wr, 436 op/s 2024-04-26T02:04:44.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:44 smithi165 ceph-mon[32648]: pgmap v350: 97 pgs: 97 active+clean; 168 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 1.4 MiB/s wr, 436 op/s 2024-04-26T02:04:44.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:44 smithi151 ceph-mon[28057]: pgmap v350: 97 pgs: 97 active+clean; 168 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 1.4 MiB/s wr, 436 op/s 2024-04-26T02:04:46.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:46 smithi172 ceph-mon[32785]: pgmap v351: 97 pgs: 97 active+clean; 160 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 1.1 MiB/s wr, 351 op/s 2024-04-26T02:04:46.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:46 smithi165 ceph-mon[32648]: pgmap v351: 97 pgs: 97 active+clean; 160 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 1.1 MiB/s wr, 351 op/s 2024-04-26T02:04:46.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:46 smithi151 ceph-mon[28057]: pgmap v351: 97 pgs: 97 active+clean; 160 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 1.1 MiB/s wr, 351 op/s 2024-04-26T02:04:48.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:48 smithi172 ceph-mon[32785]: pgmap v352: 97 pgs: 97 active+clean; 160 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.0 MiB/s wr, 304 op/s 2024-04-26T02:04:48.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:48 smithi165 ceph-mon[32648]: pgmap v352: 97 pgs: 97 active+clean; 160 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.0 MiB/s wr, 304 op/s 2024-04-26T02:04:48.873 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:48 smithi151 ceph-mon[28057]: pgmap v352: 97 pgs: 97 active+clean; 160 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.0 MiB/s wr, 304 op/s 2024-04-26T02:04:50.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:50 smithi172 ceph-mon[32785]: pgmap v353: 97 pgs: 97 active+clean; 172 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.5 MiB/s wr, 473 op/s 2024-04-26T02:04:50.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:50 smithi165 ceph-mon[32648]: pgmap v353: 97 pgs: 97 active+clean; 172 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.5 MiB/s wr, 473 op/s 2024-04-26T02:04:50.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:50 smithi151 ceph-mon[28057]: pgmap v353: 97 pgs: 97 active+clean; 172 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.5 MiB/s wr, 473 op/s 2024-04-26T02:04:52.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:52 smithi172 ceph-mon[32785]: pgmap v354: 97 pgs: 97 active+clean; 172 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 1013 KiB/s wr, 315 op/s 2024-04-26T02:04:52.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:52 smithi165 ceph-mon[32648]: pgmap v354: 97 pgs: 97 active+clean; 172 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 1013 KiB/s wr, 315 op/s 2024-04-26T02:04:52.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:52 smithi151 ceph-mon[28057]: pgmap v354: 97 pgs: 97 active+clean; 172 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 1013 KiB/s wr, 315 op/s 2024-04-26T02:04:54.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:54 smithi172 ceph-mon[32785]: pgmap v355: 97 pgs: 97 active+clean; 176 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.2 MiB/s wr, 416 op/s 2024-04-26T02:04:54.847 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:54 smithi165 ceph-mon[32648]: pgmap v355: 97 pgs: 97 active+clean; 176 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.2 MiB/s wr, 416 op/s 2024-04-26T02:04:54.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:54 smithi151 ceph-mon[28057]: pgmap v355: 97 pgs: 97 active+clean; 176 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.2 MiB/s wr, 416 op/s 2024-04-26T02:04:55.579 INFO:tasks.workunit.client.0.smithi151.stdout:All 50000 operations completed A-OK! 2024-04-26T02:04:55.585 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.586+0000 7f87e1a00640 1 -- 172.21.15.151:0/861487891 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f87dc1010c0 msgr2=0x7f87dc1014a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:55.585 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.586+0000 7f87e1a00640 1 --2- 172.21.15.151:0/861487891 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f87dc1010c0 0x7f87dc1014a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f87cc009980 tx=0x7f87cc02a420 comp rx=0 tx=0).stop 2024-04-26T02:04:55.585 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.586+0000 7f87e1a00640 1 -- 172.21.15.151:0/861487891 shutdown_connections 2024-04-26T02:04:55.586 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.586+0000 7f87e1a00640 1 --2- 172.21.15.151:0/861487891 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87dc106190 0x7f87dc10dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:55.586 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.586+0000 7f87e1a00640 1 --2- 172.21.15.151:0/861487891 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87dc101a70 0x7f87dc105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:55.586 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.586+0000 7f87e1a00640 1 --2- 172.21.15.151:0/861487891 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f87dc1010c0 0x7f87dc1014a0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:55.586 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.586+0000 7f87e1a00640 1 -- 172.21.15.151:0/861487891 >> 172.21.15.151:0/861487891 conn(0x7f87dc077fd0 msgr2=0x7f87dc0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:55.586 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.587+0000 7f87e1a00640 1 -- 172.21.15.151:0/861487891 shutdown_connections 2024-04-26T02:04:55.587 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.587+0000 7f87e1a00640 1 -- 172.21.15.151:0/861487891 wait complete. 2024-04-26T02:04:55.587 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.588+0000 7f87e1a00640 1 Processor -- start 2024-04-26T02:04:55.587 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.588+0000 7f87e1a00640 1 -- start start 2024-04-26T02:04:55.588 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.588+0000 7f87e1a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87dc106190 0x7f87dc0738e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:55.588 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.588+0000 7f87e1a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f87dc06d950 0x7f87dc06ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:55.588 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.588+0000 7f87e1a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87dc06e2f0 0x7f87dc1ab4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:55.588 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.588+0000 7f87e1a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f87dc06c230 con 0x7f87dc06e2f0 2024-04-26T02:04:55.588 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.588+0000 7f87e1a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f87dc06c0b0 con 0x7f87dc06d950 2024-04-26T02:04:55.588 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.588+0000 7f87e1a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f87dc06c3b0 con 0x7f87dc106190 2024-04-26T02:04:55.588 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.588+0000 7f87da000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f87dc06d950 0x7f87dc06ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:55.588 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.588+0000 7f87daa00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87dc106190 0x7f87dc0738e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:55.589 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.588+0000 7f87daa00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87dc106190 0x7f87dc0738e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:38898/0 (socket says 172.21.15.151:38898) 2024-04-26T02:04:55.589 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.588+0000 7f87da000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f87dc06d950 0x7f87dc06ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:42834/0 (socket says 172.21.15.151:42834) 2024-04-26T02:04:55.589 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.588+0000 7f87daa00640 1 -- 172.21.15.151:0/4059004907 learned_addr learned my addr 172.21.15.151:0/4059004907 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:04:55.589 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.589+0000 7f87db400640 1 --2- 172.21.15.151:0/4059004907 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87dc06e2f0 0x7f87dc1ab4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:55.589 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.589+0000 7f87daa00640 1 -- 172.21.15.151:0/4059004907 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f87dc06d950 msgr2=0x7f87dc06ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:55.589 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.589+0000 7f87daa00640 1 --2- 172.21.15.151:0/4059004907 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f87dc06d950 0x7f87dc06ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:55.589 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.589+0000 7f87daa00640 1 -- 172.21.15.151:0/4059004907 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87dc06e2f0 msgr2=0x7f87dc1ab4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:55.589 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.589+0000 7f87daa00640 1 --2- 172.21.15.151:0/4059004907 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87dc06e2f0 0x7f87dc1ab4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:55.589 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.589+0000 7f87daa00640 1 -- 172.21.15.151:0/4059004907 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87dc1aba00 con 0x7f87dc106190 2024-04-26T02:04:55.589 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.589+0000 7f87db400640 1 --2- 172.21.15.151:0/4059004907 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87dc06e2f0 0x7f87dc1ab4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:04:55.590 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.589+0000 7f87daa00640 1 --2- 172.21.15.151:0/4059004907 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87dc106190 0x7f87dc0738e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f87cc02aa10 tx=0x7f87cc003cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:55.590 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.590+0000 7f87d3400640 1 -- 172.21.15.151:0/4059004907 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87cc03f020 con 0x7f87dc106190 2024-04-26T02:04:55.590 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.590+0000 7f87d3400640 1 -- 172.21.15.151:0/4059004907 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87cc031d60 con 0x7f87dc106190 2024-04-26T02:04:55.590 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.590+0000 7f87d3400640 1 -- 172.21.15.151:0/4059004907 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87cc035070 con 0x7f87dc106190 2024-04-26T02:04:55.590 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.590+0000 7f87e1a00640 1 -- 172.21.15.151:0/4059004907 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87dc1abbd0 con 0x7f87dc106190 2024-04-26T02:04:55.591 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.590+0000 7f87e1a00640 1 -- 172.21.15.151:0/4059004907 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87dc1ac070 con 0x7f87dc106190 2024-04-26T02:04:55.593 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.594+0000 7f87e49d3740 1 Processor -- start 2024-04-26T02:04:55.593 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.594+0000 7f87e49d3740 1 -- start start 2024-04-26T02:04:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.594+0000 7f87e49d3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d11194820 0x562d11194c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.594+0000 7f87e49d3740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d11195140 0x562d1119f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.594+0000 7f87e49d3740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d1119fa00 0x562d111a1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.594+0000 7f87e49d3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562d110fa900 con 0x562d11194820 2024-04-26T02:04:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.594+0000 7f87e49d3740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562d110a0ef0 con 0x562d11195140 2024-04-26T02:04:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.594+0000 7f87e49d3740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562d110f68c0 con 0x562d1119fa00 2024-04-26T02:04:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.595+0000 7f87db400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d11194820 0x562d11194c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.595+0000 7f87db400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d11194820 0x562d11194c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:39552/0 (socket says 172.21.15.151:39552) 2024-04-26T02:04:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.595+0000 7f87db400640 1 -- 172.21.15.151:0/1558548127 learned_addr learned my addr 172.21.15.151:0/1558548127 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:04:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.595+0000 7f87da000640 1 --2- 172.21.15.151:0/1558548127 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d1119fa00 0x562d111a1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.595+0000 7f87d3400640 1 -- 172.21.15.151:0/4059004907 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f87cc042020 con 0x7f87dc106190 2024-04-26T02:04:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.595+0000 7f87da000640 1 -- 172.21.15.151:0/1558548127 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d11195140 msgr2=0x562d1119f4c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T02:04:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.595+0000 7f87da000640 1 --2- 172.21.15.151:0/1558548127 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d11195140 0x562d1119f4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.595+0000 7f87da000640 1 -- 172.21.15.151:0/1558548127 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d11194820 msgr2=0x562d11194c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.595+0000 7f87da000640 1 --2- 172.21.15.151:0/1558548127 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d11194820 0x562d11194c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.595+0000 7f87da000640 1 -- 172.21.15.151:0/1558548127 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d10e0d210 con 0x562d1119fa00 2024-04-26T02:04:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.595+0000 7f87db400640 1 --2- 172.21.15.151:0/1558548127 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d11194820 0x562d11194c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:04:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.595+0000 7f87da000640 1 --2- 172.21.15.151:0/1558548127 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d1119fa00 0x562d111a1df0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f87c400bbe0 tx=0x7f87c4012530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3345490bc668f95c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.596+0000 7f87cbe00640 1 -- 172.21.15.151:0/1558548127 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c4012c40 con 0x562d1119fa00 2024-04-26T02:04:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.596+0000 7f87cbe00640 1 -- 172.21.15.151:0/1558548127 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87c4012de0 con 0x562d1119fa00 2024-04-26T02:04:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.596+0000 7f87cbe00640 1 -- 172.21.15.151:0/1558548127 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c4003dd0 con 0x562d1119fa00 2024-04-26T02:04:55.600 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.597+0000 7f87e49d3740 1 -- 172.21.15.151:0/1558548127 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d1119fa00 msgr2=0x562d111a1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:55.600 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.597+0000 7f87e49d3740 1 --2- 172.21.15.151:0/1558548127 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d1119fa00 0x562d111a1df0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f87c400bbe0 tx=0x7f87c4012530 comp rx=0 tx=0).stop 2024-04-26T02:04:55.600 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.597+0000 7f87e49d3740 1 -- 172.21.15.151:0/1558548127 shutdown_connections 2024-04-26T02:04:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.597+0000 7f87e49d3740 1 --2- 172.21.15.151:0/1558548127 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d1119fa00 0x562d111a1df0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.597+0000 7f87e49d3740 1 --2- 172.21.15.151:0/1558548127 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d11195140 0x562d1119f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.597+0000 7f87e49d3740 1 --2- 172.21.15.151:0/1558548127 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d11194820 0x562d11194c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.597+0000 7f87e49d3740 1 -- 172.21.15.151:0/1558548127 >> 172.21.15.151:0/1558548127 conn(0x562d11190d30 msgr2=0x562d11191110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.597+0000 7f87d3400640 1 --2- 172.21.15.151:0/4059004907 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f87c00a3e40 0x7f87c00a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.597+0000 7f87e49d3740 1 -- 172.21.15.151:0/1558548127 shutdown_connections 2024-04-26T02:04:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.597+0000 7f87e49d3740 1 -- 172.21.15.151:0/1558548127 wait complete. 2024-04-26T02:04:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.597+0000 7f87d3400640 1 -- 172.21.15.151:0/4059004907 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8411+0+0 (secure 0 0 0) 0x7f87cc0f0df0 con 0x7f87dc106190 2024-04-26T02:04:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.597+0000 7f87db400640 1 --2- 172.21.15.151:0/4059004907 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f87c00a3e40 0x7f87c00a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.598+0000 7f87e49d3740 1 -- start start 2024-04-26T02:04:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.598+0000 7f87e49d3740 1 Processor -- start 2024-04-26T02:04:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.598+0000 7f87e49d3740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d11194820 0x562d1119c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.598+0000 7f87e49d3740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d11195140 0x562d1119ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.599+0000 7f87e49d3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d1119fa00 0x562d1119d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.599+0000 7f87e49d3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562d111a2630 con 0x562d1119fa00 2024-04-26T02:04:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.599+0000 7f87e49d3740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562d110b4310 con 0x562d11195140 2024-04-26T02:04:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.599+0000 7f87e49d3740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562d111a27b0 con 0x562d11194820 2024-04-26T02:04:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.599+0000 7f87e49d3740 10 client.0 _ll_register_callbacks cb 0x562d11082cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:04:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.599+0000 7f87daa00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d11194820 0x562d1119c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.599+0000 7f87db400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d1119fa00 0x562d1119d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.599+0000 7f87daa00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d11194820 0x562d1119c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:38918/0 (socket says 172.21.15.151:38918) 2024-04-26T02:04:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.599+0000 7f87daa00640 1 -- 172.21.15.151:0/1938523286 learned_addr learned my addr 172.21.15.151:0/1938523286 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:04:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.599+0000 7f87da000640 1 --2- 172.21.15.151:0/1938523286 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d11195140 0x562d1119ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.599+0000 7f87da000640 1 -- 172.21.15.151:0/1938523286 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d11194820 msgr2=0x562d1119c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.599+0000 7f87da000640 1 --2- 172.21.15.151:0/1938523286 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d11194820 0x562d1119c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.599+0000 7f87da000640 1 -- 172.21.15.151:0/1938523286 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d1119fa00 msgr2=0x562d1119d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.599+0000 7f87da000640 1 --2- 172.21.15.151:0/1938523286 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d1119fa00 0x562d1119d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.599+0000 7f87da000640 1 -- 172.21.15.151:0/1938523286 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d10ed9620 con 0x562d11195140 2024-04-26T02:04:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.599+0000 7f87db400640 1 --2- 172.21.15.151:0/4059004907 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f87c00a3e40 0x7f87c00a6300 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f87d4009f70 tx=0x7f87d4009820 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.600+0000 7f87da000640 1 --2- 172.21.15.151:0/1938523286 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d11195140 0x562d1119ced0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f87c4003b10 tx=0x7f87c4003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbde0afeedd2789b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.600+0000 7f87c8c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T02:04:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.600+0000 7f87c8c00640 1 -- 172.21.15.151:0/1938523286 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c4012c40 con 0x562d11195140 2024-04-26T02:04:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.600+0000 7f87c8c00640 1 -- 172.21.15.151:0/1938523286 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8411+0+0 (secure 0 0 0) 0x7f87c4003f20 con 0x562d11195140 2024-04-26T02:04:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.600+0000 7f87e49d3740 1 -- 172.21.15.151:0/1938523286 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87c4012de0 con 0x562d11195140 2024-04-26T02:04:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.600+0000 7f87c8c00640 1 -- 172.21.15.151:0/1938523286 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f87c4031050 con 0x562d11195140 2024-04-26T02:04:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.600+0000 7f87e49d3740 10 client.26486 fetch_fsmap learned FSMap version 37 2024-04-26T02:04:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.600+0000 7f87e49d3740 1 -- 172.21.15.151:0/1938523286 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x562d111ae8d0 con 0x562d11195140 2024-04-26T02:04:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.600+0000 7f87c8c00640 1 -- 172.21.15.151:0/1938523286 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87c4012de0 con 0x562d11195140 2024-04-26T02:04:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.601+0000 7f87c8c00640 1 -- 172.21.15.151:0/1938523286 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 37) ==== 8338+0+0 (secure 0 0 0) 0x7f87c4023b60 con 0x562d11195140 2024-04-26T02:04:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.601+0000 7f87e49d3740 10 client.26486 fetch_fsmap finished waiting for FSMap version 37 2024-04-26T02:04:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.601+0000 7f87e49d3740 10 client.26486 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:04:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.601+0000 7f87e49d3740 20 client.26486 populate_metadata read hostname 'smithi151' 2024-04-26T02:04:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.601+0000 7f87e49d3740 1 --2- 172.21.15.151:0/1938523286 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x562d111afcd0 0x562d111b20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.601+0000 7f87e49d3740 4 client.26486 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:04:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.601+0000 7f87e49d3740 1 -- 172.21.15.151:0/1938523286 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d1110e8b0 con 0x562d111afcd0 2024-04-26T02:04:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.601+0000 7f87daa00640 1 --2- 172.21.15.151:0/1938523286 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x562d111afcd0 0x562d111b20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.602+0000 7f87c8c00640 10 client.26486 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:04:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.602+0000 7f87daa00640 1 --2- 172.21.15.151:0/1938523286 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x562d111afcd0 0x562d111b20c0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ec9b9e2a071530a server_cookie=705894734d93bf5f in_seq=0 out_seq=0 2024-04-26T02:04:55.605 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.602+0000 7f87c8c00640 1 -- 172.21.15.151:0/1938523286 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562d1110e8b0 con 0x562d111afcd0 2024-04-26T02:04:55.605 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.602+0000 7f87c8c00640 10 client.26486 handle_command_reply: tid=0 2024-04-26T02:04:55.606 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.602+0000 7f87e49d3740 1 client.26486 shutdown 2024-04-26T02:04:55.606 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.603+0000 7f87e49d3740 1 -- 172.21.15.151:0/1938523286 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d11195140 msgr2=0x562d1119ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:55.606 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.603+0000 7f87e49d3740 1 --2- 172.21.15.151:0/1938523286 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d11195140 0x562d1119ced0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f87c4003b10 tx=0x7f87c4003820 comp rx=0 tx=0).stop 2024-04-26T02:04:55.606 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.603+0000 7f87e49d3740 1 -- 172.21.15.151:0/1938523286 shutdown_connections 2024-04-26T02:04:55.606 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.603+0000 7f87e49d3740 1 --2- 172.21.15.151:0/1938523286 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x562d111afcd0 0x562d111b20c0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:55.606 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.603+0000 7f87e49d3740 1 --2- 172.21.15.151:0/1938523286 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d1119fa00 0x562d1119d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:55.606 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.603+0000 7f87e49d3740 1 --2- 172.21.15.151:0/1938523286 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d11195140 0x562d1119ced0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:55.606 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.603+0000 7f87e49d3740 1 --2- 172.21.15.151:0/1938523286 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d11194820 0x562d1119c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:55.606 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.603+0000 7f87c8c00640 0 client.26486 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:04:55.606 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.603+0000 7f87e49d3740 1 -- 172.21.15.151:0/1938523286 >> 172.21.15.151:0/1938523286 conn(0x562d11190d30 msgr2=0x562d11196ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:55.606 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.603+0000 7f87e49d3740 1 -- 172.21.15.151:0/1938523286 shutdown_connections 2024-04-26T02:04:55.606 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.603+0000 7f87e49d3740 1 -- 172.21.15.151:0/1938523286 wait complete. 2024-04-26T02:04:55.606 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.603+0000 7f87e49d3740 20 client.26486 trim_cache size 0 max 16384 2024-04-26T02:04:55.616 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-26T02:04:55.616 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-26T02:04:55.621 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.619+0000 7f87e49d3740 1 Processor -- start 2024-04-26T02:04:55.621 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.620+0000 7f87e49d3740 1 -- start start 2024-04-26T02:04:55.621 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.620+0000 7f87e49d3740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d11194820 0x562d111ab4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:55.621 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.620+0000 7f87e49d3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d11195140 0x562d1119baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:55.621 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.620+0000 7f87e49d3740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d1119fa00 0x562d1119d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:55.621 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.620+0000 7f87e49d3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f87dc1b6de0 con 0x562d11195140 2024-04-26T02:04:55.621 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.620+0000 7f87e49d3740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f87dc1b46b0 con 0x562d1119fa00 2024-04-26T02:04:55.621 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.620+0000 7f87e49d3740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f87dc1b2520 con 0x562d11194820 2024-04-26T02:04:55.622 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.620+0000 7f87daa00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d11194820 0x562d111ab4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:55.622 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.620+0000 7f87db400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d1119fa00 0x562d1119d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:55.622 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.620+0000 7f87daa00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d11194820 0x562d111ab4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:38932/0 (socket says 172.21.15.151:38932) 2024-04-26T02:04:55.622 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.620+0000 7f87daa00640 1 -- 172.21.15.151:0/1121265712 learned_addr learned my addr 172.21.15.151:0/1121265712 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:04:55.622 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.620+0000 7f87da000640 1 --2- 172.21.15.151:0/1121265712 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d11195140 0x562d1119baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:55.622 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.620+0000 7f87daa00640 1 -- 172.21.15.151:0/1121265712 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d1119fa00 msgr2=0x562d1119d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:55.622 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.621+0000 7f87daa00640 1 --2- 172.21.15.151:0/1121265712 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d1119fa00 0x562d1119d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:55.622 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.621+0000 7f87daa00640 1 -- 172.21.15.151:0/1121265712 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d11195140 msgr2=0x562d1119baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:55.622 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.621+0000 7f87daa00640 1 --2- 172.21.15.151:0/1121265712 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d11195140 0x562d1119baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:55.622 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.621+0000 7f87daa00640 1 -- 172.21.15.151:0/1121265712 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d1119aee0 con 0x562d11194820 2024-04-26T02:04:55.622 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.621+0000 7f87db400640 1 --2- 172.21.15.151:0/1121265712 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d1119fa00 0x562d1119d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:04:55.622 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.621+0000 7f87daa00640 1 --2- 172.21.15.151:0/1121265712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d11194820 0x562d111ab4f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f87cc031440 tx=0x7f87cc000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9abb7602a7c811a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:55.622 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.621+0000 7f87cb400640 1 -- 172.21.15.151:0/1121265712 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87cc0bce00 con 0x562d11194820 2024-04-26T02:04:55.623 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.621+0000 7f87cb400640 1 -- 172.21.15.151:0/1121265712 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87cc0c11b0 con 0x562d11194820 2024-04-26T02:04:55.623 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.621+0000 7f87cb400640 1 -- 172.21.15.151:0/1121265712 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87cc03b4f0 con 0x562d11194820 2024-04-26T02:04:55.626 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.622+0000 7f87e49d3740 1 -- 172.21.15.151:0/1121265712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d11194820 msgr2=0x562d111ab4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:55.626 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.622+0000 7f87e49d3740 1 --2- 172.21.15.151:0/1121265712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d11194820 0x562d111ab4f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f87cc031440 tx=0x7f87cc000bc0 comp rx=0 tx=0).stop 2024-04-26T02:04:55.626 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.622+0000 7f87e49d3740 1 -- 172.21.15.151:0/1121265712 shutdown_connections 2024-04-26T02:04:55.626 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.622+0000 7f87e49d3740 1 --2- 172.21.15.151:0/1121265712 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d1119fa00 0x562d1119d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:55.626 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.622+0000 7f87e49d3740 1 --2- 172.21.15.151:0/1121265712 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d11195140 0x562d1119baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:55.626 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.622+0000 7f87e49d3740 1 --2- 172.21.15.151:0/1121265712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d11194820 0x562d111ab4f0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:55.626 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.622+0000 7f87e49d3740 1 -- 172.21.15.151:0/1121265712 >> 172.21.15.151:0/1121265712 conn(0x562d11190d30 msgr2=0x562d1118f600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:55.626 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.622+0000 7f87e49d3740 1 -- 172.21.15.151:0/1121265712 shutdown_connections 2024-04-26T02:04:55.626 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.622+0000 7f87e49d3740 1 -- 172.21.15.151:0/1121265712 wait complete. 2024-04-26T02:04:55.626 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.623+0000 7f87e49d3740 1 -- start start 2024-04-26T02:04:55.627 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.623+0000 7f87e49d3740 1 Processor -- start 2024-04-26T02:04:55.627 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.623+0000 7f87e49d3740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d11194820 0x562d111a4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:55.627 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.623+0000 7f87e49d3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d11195140 0x562d111a5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:55.627 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.623+0000 7f87e49d3740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d1119fa00 0x562d11213f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:55.627 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.623+0000 7f87e49d3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f87dc1b2b70 con 0x562d11195140 2024-04-26T02:04:55.627 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.623+0000 7f87e49d3740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f87dc1b3fe0 con 0x562d11194820 2024-04-26T02:04:55.627 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.623+0000 7f87e49d3740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f87dc1b3240 con 0x562d1119fa00 2024-04-26T02:04:55.627 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.624+0000 7f87e49d3740 10 client.0 _ll_register_callbacks cb 0x7f87c4005d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:04:55.627 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.624+0000 7f87daa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d11194820 0x562d111a4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:55.627 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.624+0000 7f87daa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d11194820 0x562d111a4bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:42882/0 (socket says 172.21.15.151:42882) 2024-04-26T02:04:55.627 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.624+0000 7f87daa00640 1 -- 172.21.15.151:0/2553201829 learned_addr learned my addr 172.21.15.151:0/2553201829 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:04:55.627 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.624+0000 7f87da000640 1 --2- 172.21.15.151:0/2553201829 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d11195140 0x562d111a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:55.628 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.624+0000 7f87db400640 1 --2- 172.21.15.151:0/2553201829 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d1119fa00 0x562d11213f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:55.628 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.624+0000 7f87daa00640 1 -- 172.21.15.151:0/2553201829 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d1119fa00 msgr2=0x562d11213f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:55.628 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.624+0000 7f87daa00640 1 --2- 172.21.15.151:0/2553201829 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d1119fa00 0x562d11213f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:55.628 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.624+0000 7f87daa00640 1 -- 172.21.15.151:0/2553201829 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d11195140 msgr2=0x562d111a5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:55.628 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.624+0000 7f87daa00640 1 --2- 172.21.15.151:0/2553201829 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d11195140 0x562d111a5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:55.628 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.624+0000 7f87daa00640 1 -- 172.21.15.151:0/2553201829 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d112144d0 con 0x562d11194820 2024-04-26T02:04:55.628 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.624+0000 7f87daa00640 1 --2- 172.21.15.151:0/2553201829 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d11194820 0x562d111a4bc0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f87cc003f90 tx=0x7f87cc009680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a613bf5e1ca522b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:55.628 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.624+0000 7f87c8c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T02:04:55.628 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.624+0000 7f87c8c00640 1 -- 172.21.15.151:0/2553201829 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87cc0d5020 con 0x562d11194820 2024-04-26T02:04:55.628 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.625+0000 7f87c8c00640 1 -- 172.21.15.151:0/2553201829 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8411+0+0 (secure 0 0 0) 0x7f87cc0d6220 con 0x562d11194820 2024-04-26T02:04:55.628 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.625+0000 7f87e49d3740 1 -- 172.21.15.151:0/2553201829 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87cc0c11b0 con 0x562d11194820 2024-04-26T02:04:55.628 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.625+0000 7f87c8c00640 1 -- 172.21.15.151:0/2553201829 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f87cc0ee830 con 0x562d11194820 2024-04-26T02:04:55.628 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.625+0000 7f87e49d3740 10 client.26492 fetch_fsmap learned FSMap version 37 2024-04-26T02:04:55.628 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.625+0000 7f87e49d3740 1 -- 172.21.15.151:0/2553201829 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x562d11215750 con 0x562d11194820 2024-04-26T02:04:55.628 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.625+0000 7f87c8c00640 1 -- 172.21.15.151:0/2553201829 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87cc0c11b0 con 0x562d11194820 2024-04-26T02:04:55.629 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.625+0000 7f87c8c00640 1 -- 172.21.15.151:0/2553201829 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 37) ==== 8338+0+0 (secure 0 0 0) 0x7f87cc0c56e0 con 0x562d11194820 2024-04-26T02:04:55.629 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.625+0000 7f87e49d3740 10 client.26492 fetch_fsmap finished waiting for FSMap version 37 2024-04-26T02:04:55.629 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.625+0000 7f87e49d3740 10 client.26492 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:04:55.629 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.625+0000 7f87e49d3740 20 client.26492 populate_metadata read hostname 'smithi151' 2024-04-26T02:04:55.629 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.625+0000 7f87e49d3740 1 --2- 172.21.15.151:0/2553201829 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x562d11215c60 0x562d11218080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:55.629 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.625+0000 7f87e49d3740 4 client.26492 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T02:04:55.629 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.625+0000 7f87e49d3740 1 -- 172.21.15.151:0/2553201829 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562d11108320 con 0x562d11215c60 2024-04-26T02:04:55.629 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.626+0000 7f87da000640 1 --2- 172.21.15.151:0/2553201829 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x562d11215c60 0x562d11218080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:55.629 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.626+0000 7f87da000640 1 --2- 172.21.15.151:0/2553201829 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x562d11215c60 0x562d11218080 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64998b13a68edd92 server_cookie=b412c5b3a89ac649 in_seq=0 out_seq=0 2024-04-26T02:04:55.629 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.626+0000 7f87c8c00640 10 client.26492 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:04:55.642 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.642+0000 7f87c8c00640 1 -- 172.21.15.151:0/2553201829 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562d11108320 con 0x562d11215c60 2024-04-26T02:04:55.642 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.642+0000 7f87c8c00640 10 client.26492 handle_command_reply: tid=0 2024-04-26T02:04:55.642 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.642+0000 7f87e49d3740 1 client.26492 shutdown 2024-04-26T02:04:55.642 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.643+0000 7f87e49d3740 1 -- 172.21.15.151:0/2553201829 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d11194820 msgr2=0x562d111a4bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:55.642 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.643+0000 7f87e49d3740 1 --2- 172.21.15.151:0/2553201829 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d11194820 0x562d111a4bc0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f87cc003f90 tx=0x7f87cc009680 comp rx=0 tx=0).stop 2024-04-26T02:04:55.643 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.643+0000 7f87e49d3740 1 -- 172.21.15.151:0/2553201829 shutdown_connections 2024-04-26T02:04:55.643 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.643+0000 7f87e49d3740 1 --2- 172.21.15.151:0/2553201829 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x562d11215c60 0x562d11218080 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:55.643 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.643+0000 7f87e49d3740 1 --2- 172.21.15.151:0/2553201829 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d1119fa00 0x562d11213f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:55.643 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.643+0000 7f87e49d3740 1 --2- 172.21.15.151:0/2553201829 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562d11195140 0x562d111a5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:55.643 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.643+0000 7f87e49d3740 1 --2- 172.21.15.151:0/2553201829 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562d11194820 0x562d111a4bc0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:55.643 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.643+0000 7f87c8c00640 0 client.26492 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:04:55.643 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.643+0000 7f87e49d3740 1 -- 172.21.15.151:0/2553201829 >> 172.21.15.151:0/2553201829 conn(0x562d11190d30 msgr2=0x562d1119b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:55.643 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.643+0000 7f87e49d3740 1 -- 172.21.15.151:0/2553201829 shutdown_connections 2024-04-26T02:04:55.643 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.643+0000 7f87e49d3740 1 -- 172.21.15.151:0/2553201829 wait complete. 2024-04-26T02:04:55.643 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.643+0000 7f87e49d3740 20 client.26492 trim_cache size 0 max 16384 2024-04-26T02:04:55.644 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:{ 2024-04-26T02:04:55.644 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "message": "", 2024-04-26T02:04:55.644 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "return_code": 0 2024-04-26T02:04:55.644 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:} 2024-04-26T02:04:55.644 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.644+0000 7f87e1a00640 1 -- 172.21.15.151:0/4059004907 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f87c00a3e40 msgr2=0x7f87c00a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.644+0000 7f87e1a00640 1 --2- 172.21.15.151:0/4059004907 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f87c00a3e40 0x7f87c00a6300 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f87d4009f70 tx=0x7f87d4009820 comp rx=0 tx=0).stop 2024-04-26T02:04:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.644+0000 7f87e1a00640 1 -- 172.21.15.151:0/4059004907 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87dc106190 msgr2=0x7f87dc0738e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.644+0000 7f87e1a00640 1 --2- 172.21.15.151:0/4059004907 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87dc106190 0x7f87dc0738e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f87cc02aa10 tx=0x7f87cc003cd0 comp rx=0 tx=0).stop 2024-04-26T02:04:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.644+0000 7f87e1a00640 1 -- 172.21.15.151:0/4059004907 shutdown_connections 2024-04-26T02:04:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.644+0000 7f87e1a00640 1 --2- 172.21.15.151:0/4059004907 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f87c00a3e40 0x7f87c00a6300 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.644+0000 7f87e1a00640 1 --2- 172.21.15.151:0/4059004907 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87dc06e2f0 0x7f87dc1ab4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.644+0000 7f87e1a00640 1 --2- 172.21.15.151:0/4059004907 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f87dc06d950 0x7f87dc06ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.644+0000 7f87e1a00640 1 --2- 172.21.15.151:0/4059004907 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87dc106190 0x7f87dc0738e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.644+0000 7f87e1a00640 1 -- 172.21.15.151:0/4059004907 >> 172.21.15.151:0/4059004907 conn(0x7f87dc077fd0 msgr2=0x7f87dc0feee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.644+0000 7f87e1a00640 1 -- 172.21.15.151:0/4059004907 shutdown_connections 2024-04-26T02:04:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:04:55.644+0000 7f87e1a00640 1 -- 172.21.15.151:0/4059004907 wait complete. 2024-04-26T02:04:56.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:56 smithi172 ceph-mon[32785]: pgmap v356: 97 pgs: 97 active+clean; 181 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 1.1 MiB/s wr, 381 op/s 2024-04-26T02:04:56.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:56 smithi165 ceph-mon[32648]: pgmap v356: 97 pgs: 97 active+clean; 181 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 1.1 MiB/s wr, 381 op/s 2024-04-26T02:04:56.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:56 smithi151 ceph-mon[28057]: pgmap v356: 97 pgs: 97 active+clean; 181 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 1.1 MiB/s wr, 381 op/s 2024-04-26T02:04:58.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:04:58 smithi172 ceph-mon[32785]: pgmap v357: 97 pgs: 97 active+clean; 181 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 1011 KiB/s wr, 339 op/s 2024-04-26T02:04:58.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:04:58 smithi165 ceph-mon[32648]: pgmap v357: 97 pgs: 97 active+clean; 181 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 1011 KiB/s wr, 339 op/s 2024-04-26T02:04:58.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:04:58 smithi151 ceph-mon[28057]: pgmap v357: 97 pgs: 97 active+clean; 181 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 1011 KiB/s wr, 339 op/s 2024-04-26T02:05:00.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:05:00 smithi172 ceph-mon[32785]: pgmap v358: 97 pgs: 97 active+clean; 161 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 1.5 MiB/s wr, 460 op/s 2024-04-26T02:05:00.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:05:00 smithi165 ceph-mon[32648]: pgmap v358: 97 pgs: 97 active+clean; 161 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 1.5 MiB/s wr, 460 op/s 2024-04-26T02:05:00.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:05:00 smithi151 ceph-mon[28057]: pgmap v358: 97 pgs: 97 active+clean; 161 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 1.5 MiB/s wr, 460 op/s 2024-04-26T02:05:02.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:05:02 smithi172 ceph-mon[32785]: pgmap v359: 97 pgs: 97 active+clean; 161 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 986 KiB/s wr, 291 op/s 2024-04-26T02:05:02.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:05:02 smithi165 ceph-mon[32648]: pgmap v359: 97 pgs: 97 active+clean; 161 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 986 KiB/s wr, 291 op/s 2024-04-26T02:05:02.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:05:02 smithi151 ceph-mon[28057]: pgmap v359: 97 pgs: 97 active+clean; 161 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 986 KiB/s wr, 291 op/s 2024-04-26T02:05:04.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:05:04 smithi172 ceph-mon[32785]: pgmap v360: 97 pgs: 97 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 1.8 MiB/s wr, 386 op/s 2024-04-26T02:05:04.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:05:04 smithi165 ceph-mon[32648]: pgmap v360: 97 pgs: 97 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 1.8 MiB/s wr, 386 op/s 2024-04-26T02:05:04.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:05:04 smithi151 ceph-mon[28057]: pgmap v360: 97 pgs: 97 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 1.8 MiB/s wr, 386 op/s 2024-04-26T02:05:06.677 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:05:06 smithi151 ceph-mon[28057]: pgmap v361: 97 pgs: 97 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.5 MiB/s wr, 317 op/s 2024-04-26T02:05:06.677 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:05:06 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T02:05:06.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:05:06 smithi172 ceph-mon[32785]: pgmap v361: 97 pgs: 97 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.5 MiB/s wr, 317 op/s 2024-04-26T02:05:06.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:05:06 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T02:05:06.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:05:06 smithi165 ceph-mon[32648]: pgmap v361: 97 pgs: 97 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.5 MiB/s wr, 317 op/s 2024-04-26T02:05:06.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:05:06 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T02:05:08.598 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:05:08 smithi165 ceph-mon[32648]: pgmap v362: 97 pgs: 97 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.3 MiB/s wr, 247 op/s 2024-04-26T02:05:08.598 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:05:08 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:05:08.599 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:05:08 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:05:08.599 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:05:08 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T02:05:08.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:05:08 smithi172 ceph-mon[32785]: pgmap v362: 97 pgs: 97 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.3 MiB/s wr, 247 op/s 2024-04-26T02:05:08.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:05:08 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:05:08.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:05:08 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:05:08.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:05:08 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T02:05:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.790+0000 7f7ed2400640 1 -- 172.21.15.151:0/167064097 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ecc104bb0 msgr2=0x7f7ecc108bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.790+0000 7f7ed2400640 1 --2- 172.21.15.151:0/167064097 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ecc104bb0 0x7f7ecc108bf0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8009900 tx=0x7f7eb802a3f0 comp rx=0 tx=0).stop 2024-04-26T02:05:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.790+0000 7f7ed2400640 1 -- 172.21.15.151:0/167064097 shutdown_connections 2024-04-26T02:05:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.790+0000 7f7ed2400640 1 --2- 172.21.15.151:0/167064097 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7ecc1092d0 0x7f7ecc110ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.790+0000 7f7ed2400640 1 --2- 172.21.15.151:0/167064097 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ecc104bb0 0x7f7ecc108bf0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.790+0000 7f7ed2400640 1 --2- 172.21.15.151:0/167064097 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ecc104200 0x7f7ecc1045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.790+0000 7f7ed2400640 1 -- 172.21.15.151:0/167064097 >> 172.21.15.151:0/167064097 conn(0x7f7ecc0ffa60 msgr2=0x7f7ecc101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:05:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.790+0000 7f7ed2400640 1 -- 172.21.15.151:0/167064097 shutdown_connections 2024-04-26T02:05:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.790+0000 7f7ed2400640 1 -- 172.21.15.151:0/167064097 wait complete. 2024-04-26T02:05:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.791+0000 7f7ed2400640 1 Processor -- start 2024-04-26T02:05:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.792+0000 7f7ed2400640 1 -- start start 2024-04-26T02:05:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.792+0000 7f7ed2400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ecc104200 0x7f7ecc1a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.792+0000 7f7ed2400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ecc1092d0 0x7f7ecc1a0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.792+0000 7f7ed2400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7ecc19a5e0 0x7f7ecc19aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.792+0000 7f7ed2400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc114240 con 0x7f7ecc1092d0 2024-04-26T02:05:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.792+0000 7f7ed2400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc1140c0 con 0x7f7ecc19a5e0 2024-04-26T02:05:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.792+0000 7f7ed2400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc1143c0 con 0x7f7ecc104200 2024-04-26T02:05:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.792+0000 7f7ed1a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7ecc19a5e0 0x7f7ecc19aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.792+0000 7f7ed1a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7ecc19a5e0 0x7f7ecc19aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:32922/0 (socket says 172.21.15.151:32922) 2024-04-26T02:05:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.792+0000 7f7ed1a00640 1 -- 172.21.15.151:0/500031366 learned_addr learned my addr 172.21.15.151:0/500031366 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:05:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.792+0000 7f7ed1000640 1 --2- 172.21.15.151:0/500031366 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ecc104200 0x7f7ecc1a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.792+0000 7f7ecbe00640 1 --2- 172.21.15.151:0/500031366 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ecc1092d0 0x7f7ecc1a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.793+0000 7f7ed1a00640 1 -- 172.21.15.151:0/500031366 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ecc104200 msgr2=0x7f7ecc1a0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.793+0000 7f7ed1a00640 1 --2- 172.21.15.151:0/500031366 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ecc104200 0x7f7ecc1a0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.793+0000 7f7ed1a00640 1 -- 172.21.15.151:0/500031366 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ecc1092d0 msgr2=0x7f7ecc1a0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:08.794 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.793+0000 7f7ed1a00640 1 --2- 172.21.15.151:0/500031366 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ecc1092d0 0x7f7ecc1a0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:08.794 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.793+0000 7f7ed1a00640 1 -- 172.21.15.151:0/500031366 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ecc19b100 con 0x7f7ecc19a5e0 2024-04-26T02:05:08.794 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.793+0000 7f7ed1a00640 1 --2- 172.21.15.151:0/500031366 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7ecc19a5e0 0x7f7ecc19aa90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7ec400eb10 tx=0x7f7ec400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:05:08.794 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.793+0000 7f7ec9600640 1 -- 172.21.15.151:0/500031366 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec4004b20 con 0x7f7ecc19a5e0 2024-04-26T02:05:08.794 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.793+0000 7f7ec9600640 1 -- 172.21.15.151:0/500031366 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ec4004cc0 con 0x7f7ecc19a5e0 2024-04-26T02:05:08.794 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.793+0000 7f7ed2400640 1 -- 172.21.15.151:0/500031366 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ecc19b3f0 con 0x7f7ecc19a5e0 2024-04-26T02:05:08.794 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.793+0000 7f7ec9600640 1 -- 172.21.15.151:0/500031366 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec4010670 con 0x7f7ecc19a5e0 2024-04-26T02:05:08.794 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.793+0000 7f7ed2400640 1 -- 172.21.15.151:0/500031366 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ecc1a7640 con 0x7f7ecc19a5e0 2024-04-26T02:05:08.796 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.797+0000 7f7ed6770740 1 Processor -- start 2024-04-26T02:05:08.797 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.798+0000 7f7ec9600640 1 -- 172.21.15.151:0/500031366 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f7ec4005020 con 0x7f7ecc19a5e0 2024-04-26T02:05:08.798 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.798+0000 7f7ed6770740 1 -- start start 2024-04-26T02:05:08.798 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.798+0000 7f7ed6770740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637a4d509b0 0x5637a4d50d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:08.798 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.798+0000 7f7ed6770740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637a4d512d0 0x5637a4d5b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:08.798 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.798+0000 7f7ed6770740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5637a4d5bb90 0x5637a4d5df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:08.798 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.798+0000 7f7ed6770740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5637a4cb6900 con 0x5637a4d5bb90 2024-04-26T02:05:08.798 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.798+0000 7f7ed6770740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5637a4c5cef0 con 0x5637a4d509b0 2024-04-26T02:05:08.798 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.798+0000 7f7ed6770740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5637a4cb28c0 con 0x5637a4d512d0 2024-04-26T02:05:08.798 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.799+0000 7f7ecbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637a4d509b0 0x5637a4d50d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:08.798 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.799+0000 7f7ecbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637a4d509b0 0x5637a4d50d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:32934/0 (socket says 172.21.15.151:32934) 2024-04-26T02:05:08.799 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.799+0000 7f7ecbe00640 1 -- 172.21.15.151:0/137488573 learned_addr learned my addr 172.21.15.151:0/137488573 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:05:08.799 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.799+0000 7f7ed1000640 1 --2- 172.21.15.151:0/137488573 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637a4d512d0 0x5637a4d5b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:08.799 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.799+0000 7f7ecbe00640 1 --2- 172.21.15.151:0/137488573 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5637a4d5bb90 0x5637a4d5df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:08.800 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.799+0000 7f7ed1000640 1 -- 172.21.15.151:0/137488573 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637a4d509b0 msgr2=0x5637a4d50d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:05:08.801 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.799+0000 7f7ed1000640 1 --2- 172.21.15.151:0/137488573 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637a4d509b0 0x5637a4d50d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:08.801 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.799+0000 7f7ed1000640 1 -- 172.21.15.151:0/137488573 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5637a4d5bb90 msgr2=0x5637a4d5df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:05:08.801 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.799+0000 7f7ed1000640 1 --2- 172.21.15.151:0/137488573 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5637a4d5bb90 0x5637a4d5df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:08.801 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.799+0000 7f7ed1000640 1 -- 172.21.15.151:0/137488573 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637a49c9210 con 0x5637a4d512d0 2024-04-26T02:05:08.801 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.800+0000 7f7ed1000640 1 --2- 172.21.15.151:0/137488573 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637a4d512d0 0x5637a4d5b650 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0005fd0 tx=0x7f7ec00107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc1fdc05aecf79e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:05:08.801 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.800+0000 7f7ec9600640 1 --2- 172.21.15.151:0/500031366 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f7eb40a3d90 0x7f7eb40a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:08.801 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.800+0000 7f7ebe000640 1 -- 172.21.15.151:0/137488573 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec001c480 con 0x5637a4d512d0 2024-04-26T02:05:08.801 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.800+0000 7f7ebe000640 1 -- 172.21.15.151:0/137488573 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ec001cad0 con 0x5637a4d512d0 2024-04-26T02:05:08.801 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.800+0000 7f7ebe000640 1 -- 172.21.15.151:0/137488573 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec0011af0 con 0x5637a4d512d0 2024-04-26T02:05:08.801 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.800+0000 7f7ec9600640 1 -- 172.21.15.151:0/500031366 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8411+0+0 (secure 0 0 0) 0x7f7ec40d0c90 con 0x7f7ecc19a5e0 2024-04-26T02:05:08.801 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.800+0000 7f7ecbe00640 1 --2- 172.21.15.151:0/500031366 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f7eb40a3d90 0x7f7eb40a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:08.801 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.801+0000 7f7ed6770740 1 -- 172.21.15.151:0/137488573 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637a4d512d0 msgr2=0x5637a4d5b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:08.801 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.801+0000 7f7ecbe00640 1 --2- 172.21.15.151:0/500031366 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f7eb40a3d90 0x7f7eb40a6250 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8000c00 tx=0x7f7eb8005a20 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:05:08.802 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.801+0000 7f7ed6770740 1 --2- 172.21.15.151:0/137488573 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637a4d512d0 0x5637a4d5b650 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0005fd0 tx=0x7f7ec00107a0 comp rx=0 tx=0).stop 2024-04-26T02:05:08.802 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.801+0000 7f7ed6770740 1 -- 172.21.15.151:0/137488573 shutdown_connections 2024-04-26T02:05:08.802 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.801+0000 7f7ed6770740 1 --2- 172.21.15.151:0/137488573 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5637a4d5bb90 0x5637a4d5df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:08.802 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.801+0000 7f7ed6770740 1 --2- 172.21.15.151:0/137488573 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637a4d512d0 0x5637a4d5b650 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:08.802 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.801+0000 7f7ed6770740 1 --2- 172.21.15.151:0/137488573 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637a4d509b0 0x5637a4d50d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:08.802 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.801+0000 7f7ed6770740 1 -- 172.21.15.151:0/137488573 >> 172.21.15.151:0/137488573 conn(0x5637a4d4cec0 msgr2=0x5637a4d4d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:05:08.802 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.801+0000 7f7ed6770740 1 -- 172.21.15.151:0/137488573 shutdown_connections 2024-04-26T02:05:08.802 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.801+0000 7f7ed6770740 1 -- 172.21.15.151:0/137488573 wait complete. 2024-04-26T02:05:08.802 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.801+0000 7f7ed6770740 1 -- start start 2024-04-26T02:05:08.802 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.802+0000 7f7ed6770740 1 Processor -- start 2024-04-26T02:05:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.802+0000 7f7ed6770740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5637a4d509b0 0x5637a4d58a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.802+0000 7f7ed6770740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637a4d512d0 0x5637a4d58fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.802+0000 7f7ed6770740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637a4d5bb90 0x5637a4d59500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.802+0000 7f7ed6770740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5637a4d5e7c0 con 0x5637a4d509b0 2024-04-26T02:05:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.802+0000 7f7ed6770740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5637a4c70310 con 0x5637a4d5bb90 2024-04-26T02:05:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.802+0000 7f7ed6770740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5637a4d5e940 con 0x5637a4d512d0 2024-04-26T02:05:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.802+0000 7f7ed6770740 10 client.0 _ll_register_callbacks cb 0x5637a4c3ecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:05:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.802+0000 7f7ed1a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637a4d5bb90 0x5637a4d59500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.803+0000 7f7ed1000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5637a4d509b0 0x5637a4d58a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.803+0000 7f7ed1000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5637a4d509b0 0x5637a4d58a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42770/0 (socket says 172.21.15.151:42770) 2024-04-26T02:05:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.803+0000 7f7ed1000640 1 -- 172.21.15.151:0/2122980228 learned_addr learned my addr 172.21.15.151:0/2122980228 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:05:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.803+0000 7f7ecbe00640 1 --2- 172.21.15.151:0/2122980228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637a4d512d0 0x5637a4d58fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.803+0000 7f7ed1a00640 1 -- 172.21.15.151:0/2122980228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637a4d512d0 msgr2=0x5637a4d58fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:05:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.803+0000 7f7ed1a00640 1 --2- 172.21.15.151:0/2122980228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637a4d512d0 0x5637a4d58fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.803+0000 7f7ed1a00640 1 -- 172.21.15.151:0/2122980228 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5637a4d509b0 msgr2=0x5637a4d58a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:05:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.803+0000 7f7ed1a00640 1 --2- 172.21.15.151:0/2122980228 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5637a4d509b0 0x5637a4d58a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.803+0000 7f7ed1a00640 1 -- 172.21.15.151:0/2122980228 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637a4a95620 con 0x5637a4d5bb90 2024-04-26T02:05:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.803+0000 7f7ed1000640 1 --2- 172.21.15.151:0/2122980228 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5637a4d509b0 0x5637a4d58a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:05:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.803+0000 7f7ed1a00640 1 --2- 172.21.15.151:0/2122980228 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637a4d5bb90 0x5637a4d59500 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7ec40ce3f0 tx=0x7f7ec409c5a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd8a3c3ba6cb3f05 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:05:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.803+0000 7f7eb2a00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T02:05:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.803+0000 7f7eb2a00640 1 -- 172.21.15.151:0/2122980228 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec40b6060 con 0x5637a4d5bb90 2024-04-26T02:05:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.803+0000 7f7eb2a00640 1 -- 172.21.15.151:0/2122980228 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8411+0+0 (secure 0 0 0) 0x7f7ec40bd070 con 0x5637a4d5bb90 2024-04-26T02:05:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.803+0000 7f7ed6770740 1 -- 172.21.15.151:0/2122980228 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ec001cad0 con 0x5637a4d5bb90 2024-04-26T02:05:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.804+0000 7f7eb2a00640 1 -- 172.21.15.151:0/2122980228 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f7ec40a0ff0 con 0x5637a4d5bb90 2024-04-26T02:05:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.804+0000 7f7ed6770740 10 client.26504 fetch_fsmap learned FSMap version 37 2024-04-26T02:05:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.804+0000 7f7ed6770740 1 -- 172.21.15.151:0/2122980228 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5637a4d6aa50 con 0x5637a4d5bb90 2024-04-26T02:05:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.804+0000 7f7eb2a00640 1 -- 172.21.15.151:0/2122980228 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ec001cad0 con 0x5637a4d5bb90 2024-04-26T02:05:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.804+0000 7f7eb2a00640 1 -- 172.21.15.151:0/2122980228 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 37) ==== 8338+0+0 (secure 0 0 0) 0x7f7ec40a5420 con 0x5637a4d5bb90 2024-04-26T02:05:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.804+0000 7f7ed6770740 10 client.26504 fetch_fsmap finished waiting for FSMap version 37 2024-04-26T02:05:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.804+0000 7f7ed6770740 10 client.26504 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:05:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.804+0000 7f7ed6770740 20 client.26504 populate_metadata read hostname 'smithi151' 2024-04-26T02:05:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.804+0000 7f7ed6770740 1 --2- 172.21.15.151:0/2122980228 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5637a4d6bf10 0x5637a4d6e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.804+0000 7f7ed6770740 4 client.26504 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:05:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.804+0000 7f7ed6770740 1 -- 172.21.15.151:0/2122980228 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637a4cca8b0 con 0x5637a4d6bf10 2024-04-26T02:05:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.805+0000 7f7ed1000640 1 --2- 172.21.15.151:0/2122980228 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5637a4d6bf10 0x5637a4d6e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.805+0000 7f7ed1000640 1 --2- 172.21.15.151:0/2122980228 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5637a4d6bf10 0x5637a4d6e300 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=614404b5ec13e8fa server_cookie=aefddcd1fd640b64 in_seq=0 out_seq=0 2024-04-26T02:05:08.808 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.805+0000 7f7eb2a00640 10 client.26504 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:05:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.807+0000 7f7eb2a00640 1 -- 172.21.15.151:0/2122980228 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5637a4cca8b0 con 0x5637a4d6bf10 2024-04-26T02:05:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.807+0000 7f7eb2a00640 10 client.26504 handle_command_reply: tid=0 2024-04-26T02:05:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.807+0000 7f7ed6770740 1 client.26504 shutdown 2024-04-26T02:05:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.807+0000 7f7ed6770740 1 -- 172.21.15.151:0/2122980228 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637a4d5bb90 msgr2=0x5637a4d59500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.807+0000 7f7ed6770740 1 --2- 172.21.15.151:0/2122980228 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637a4d5bb90 0x5637a4d59500 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7ec40ce3f0 tx=0x7f7ec409c5a0 comp rx=0 tx=0).stop 2024-04-26T02:05:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.807+0000 7f7ed6770740 1 -- 172.21.15.151:0/2122980228 shutdown_connections 2024-04-26T02:05:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.807+0000 7f7ed6770740 1 --2- 172.21.15.151:0/2122980228 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5637a4d6bf10 0x5637a4d6e300 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.807+0000 7f7ed6770740 1 --2- 172.21.15.151:0/2122980228 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637a4d5bb90 0x5637a4d59500 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.807+0000 7f7eb2a00640 0 client.26504 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:05:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.807+0000 7f7ed6770740 1 --2- 172.21.15.151:0/2122980228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637a4d512d0 0x5637a4d58fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.807+0000 7f7ed6770740 1 --2- 172.21.15.151:0/2122980228 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5637a4d509b0 0x5637a4d58a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.807+0000 7f7ed6770740 1 -- 172.21.15.151:0/2122980228 >> 172.21.15.151:0/2122980228 conn(0x5637a4d4cec0 msgr2=0x5637a4d52bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:05:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.807+0000 7f7ed6770740 1 -- 172.21.15.151:0/2122980228 shutdown_connections 2024-04-26T02:05:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.807+0000 7f7ed6770740 1 -- 172.21.15.151:0/2122980228 wait complete. 2024-04-26T02:05:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.808+0000 7f7ed6770740 20 client.26504 trim_cache size 0 max 16384 2024-04-26T02:05:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.824+0000 7f7ed6770740 1 Processor -- start 2024-04-26T02:05:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.824+0000 7f7ed6770740 1 -- start start 2024-04-26T02:05:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.825+0000 7f7ed6770740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637a4d509b0 0x5637a4d676b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.825+0000 7f7ed6770740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637a4d512d0 0x5637a4d53370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.825+0000 7f7ed6770740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5637a4d5bb90 0x5637a4d4bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.825+0000 7f7ed6770740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc1b23b0 con 0x5637a4d5bb90 2024-04-26T02:05:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.825+0000 7f7ed6770740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc1afc80 con 0x5637a4d509b0 2024-04-26T02:05:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.825+0000 7f7ed6770740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc1adb10 con 0x5637a4d512d0 2024-04-26T02:05:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.825+0000 7f7ecbe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637a4d512d0 0x5637a4d53370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.825+0000 7f7ed1000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637a4d509b0 0x5637a4d676b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.825+0000 7f7ecbe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637a4d512d0 0x5637a4d53370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:50922/0 (socket says 172.21.15.151:50922) 2024-04-26T02:05:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.825+0000 7f7ecbe00640 1 -- 172.21.15.151:0/1364880219 learned_addr learned my addr 172.21.15.151:0/1364880219 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:05:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.825+0000 7f7ecbe00640 1 -- 172.21.15.151:0/1364880219 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637a4d509b0 msgr2=0x5637a4d676b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:05:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.825+0000 7f7ecbe00640 1 --2- 172.21.15.151:0/1364880219 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637a4d509b0 0x5637a4d676b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:08.826 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.825+0000 7f7ed1a00640 1 --2- 172.21.15.151:0/1364880219 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5637a4d5bb90 0x5637a4d4bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:08.826 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.825+0000 7f7ecbe00640 1 -- 172.21.15.151:0/1364880219 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5637a4d5bb90 msgr2=0x5637a4d4bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:05:08.826 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.826+0000 7f7ecbe00640 1 --2- 172.21.15.151:0/1364880219 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5637a4d5bb90 0x5637a4d4bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:08.826 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.826+0000 7f7ecbe00640 1 -- 172.21.15.151:0/1364880219 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637a4d56fd0 con 0x5637a4d512d0 2024-04-26T02:05:08.826 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.826+0000 7f7ecbe00640 1 --2- 172.21.15.151:0/1364880219 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637a4d512d0 0x5637a4d53370 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f7eb802a9e0 tx=0x7f7eb8009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ef81cb32a68ea47 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:05:08.826 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.826+0000 7f7ebd600640 1 -- 172.21.15.151:0/1364880219 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb8031820 con 0x5637a4d512d0 2024-04-26T02:05:08.826 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.826+0000 7f7ebd600640 1 -- 172.21.15.151:0/1364880219 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7eb803d450 con 0x5637a4d512d0 2024-04-26T02:05:08.826 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.826+0000 7f7ebd600640 1 -- 172.21.15.151:0/1364880219 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb803c6b0 con 0x5637a4d512d0 2024-04-26T02:05:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.827+0000 7f7ed6770740 1 -- 172.21.15.151:0/1364880219 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637a4d512d0 msgr2=0x5637a4d53370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.827+0000 7f7ed6770740 1 --2- 172.21.15.151:0/1364880219 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637a4d512d0 0x5637a4d53370 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f7eb802a9e0 tx=0x7f7eb8009260 comp rx=0 tx=0).stop 2024-04-26T02:05:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.827+0000 7f7ed6770740 1 -- 172.21.15.151:0/1364880219 shutdown_connections 2024-04-26T02:05:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.827+0000 7f7ed6770740 1 --2- 172.21.15.151:0/1364880219 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5637a4d5bb90 0x5637a4d4bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.827+0000 7f7ed6770740 1 --2- 172.21.15.151:0/1364880219 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637a4d512d0 0x5637a4d53370 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.827+0000 7f7ed6770740 1 --2- 172.21.15.151:0/1364880219 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637a4d509b0 0x5637a4d676b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.827+0000 7f7ed6770740 1 -- 172.21.15.151:0/1364880219 >> 172.21.15.151:0/1364880219 conn(0x5637a4d4cec0 msgr2=0x5637a4d4b790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:05:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.827+0000 7f7ed6770740 1 -- 172.21.15.151:0/1364880219 shutdown_connections 2024-04-26T02:05:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.827+0000 7f7ed6770740 1 -- 172.21.15.151:0/1364880219 wait complete. 2024-04-26T02:05:08.830 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.827+0000 7f7ed6770740 1 -- start start 2024-04-26T02:05:08.830 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.827+0000 7f7ed6770740 1 Processor -- start 2024-04-26T02:05:08.830 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.828+0000 7f7ed6770740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637a4d509b0 0x5637a4d63e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:08.830 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.828+0000 7f7ed6770740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637a4d512d0 0x5637a4d61190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:08.830 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.828+0000 7f7ed6770740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5637a4d5bb90 0x5637a4d616d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:08.830 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.828+0000 7f7ed6770740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc1ae160 con 0x5637a4d5bb90 2024-04-26T02:05:08.830 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.828+0000 7f7ed6770740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc1af5b0 con 0x5637a4d509b0 2024-04-26T02:05:08.830 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.828+0000 7f7ed6770740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc1ae810 con 0x5637a4d512d0 2024-04-26T02:05:08.830 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.828+0000 7f7ed6770740 10 client.0 _ll_register_callbacks cb 0x7f7ec4098f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:05:08.830 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.828+0000 7f7ed1000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637a4d509b0 0x5637a4d63e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:08.830 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.828+0000 7f7ed1a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5637a4d5bb90 0x5637a4d616d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:08.830 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.828+0000 7f7ed1a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5637a4d5bb90 0x5637a4d616d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42782/0 (socket says 172.21.15.151:42782) 2024-04-26T02:05:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.828+0000 7f7ecbe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637a4d512d0 0x5637a4d61190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.828+0000 7f7ed1a00640 1 -- 172.21.15.151:0/2413582359 learned_addr learned my addr 172.21.15.151:0/2413582359 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:05:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.829+0000 7f7ecbe00640 1 -- 172.21.15.151:0/2413582359 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637a4d509b0 msgr2=0x5637a4d63e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:05:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.829+0000 7f7ecbe00640 1 --2- 172.21.15.151:0/2413582359 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637a4d509b0 0x5637a4d63e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.829+0000 7f7ecbe00640 1 -- 172.21.15.151:0/2413582359 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5637a4d5bb90 msgr2=0x5637a4d616d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:05:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.829+0000 7f7ecbe00640 1 --2- 172.21.15.151:0/2413582359 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5637a4d5bb90 0x5637a4d616d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.829+0000 7f7ecbe00640 1 -- 172.21.15.151:0/2413582359 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637a4d61d40 con 0x5637a4d512d0 2024-04-26T02:05:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.829+0000 7f7ed1000640 1 --2- 172.21.15.151:0/2413582359 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637a4d509b0 0x5637a4d63e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:05:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.829+0000 7f7ecbe00640 1 --2- 172.21.15.151:0/2413582359 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637a4d512d0 0x5637a4d61190 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x5637a4d5e460 tx=0x7f7eb802a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f24cc786544f4b0e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:05:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.829+0000 7f7eb2a00640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-26T02:05:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.829+0000 7f7eb2a00640 1 -- 172.21.15.151:0/2413582359 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb803d800 con 0x5637a4d512d0 2024-04-26T02:05:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.829+0000 7f7eb2a00640 1 -- 172.21.15.151:0/2413582359 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8411+0+0 (secure 0 0 0) 0x7f7eb803d9a0 con 0x5637a4d512d0 2024-04-26T02:05:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.829+0000 7f7ed6770740 1 -- 172.21.15.151:0/2413582359 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7eb803d450 con 0x5637a4d512d0 2024-04-26T02:05:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.829+0000 7f7eb2a00640 1 -- 172.21.15.151:0/2413582359 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f7eb803dba0 con 0x5637a4d512d0 2024-04-26T02:05:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.829+0000 7f7ed6770740 10 client.26536 fetch_fsmap learned FSMap version 37 2024-04-26T02:05:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.829+0000 7f7ed6770740 1 -- 172.21.15.151:0/2413582359 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5637a4dd22f0 con 0x5637a4d512d0 2024-04-26T02:05:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.830+0000 7f7eb2a00640 1 -- 172.21.15.151:0/2413582359 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7eb803d450 con 0x5637a4d512d0 2024-04-26T02:05:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.830+0000 7f7eb2a00640 1 -- 172.21.15.151:0/2413582359 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8338+0+0 (secure 0 0 0) 0x7f7eb80556a0 con 0x5637a4d512d0 2024-04-26T02:05:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.830+0000 7f7ed6770740 10 client.26536 fetch_fsmap finished waiting for FSMap version 37 2024-04-26T02:05:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.830+0000 7f7ed6770740 10 client.26536 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:05:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.830+0000 7f7ed6770740 20 client.26536 populate_metadata read hostname 'smithi151' 2024-04-26T02:05:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.830+0000 7f7ed6770740 1 --2- 172.21.15.151:0/2413582359 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5637a4d6bf10 0x5637a4dd4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.830+0000 7f7ed6770740 4 client.26536 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T02:05:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.830+0000 7f7ed6770740 1 -- 172.21.15.151:0/2413582359 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5637a4cc4320 con 0x5637a4d6bf10 2024-04-26T02:05:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.830+0000 7f7ed1000640 1 --2- 172.21.15.151:0/2413582359 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5637a4d6bf10 0x5637a4dd4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.831+0000 7f7ed1000640 1 --2- 172.21.15.151:0/2413582359 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5637a4d6bf10 0x5637a4dd4a20 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e232deba6ab149ec server_cookie=46c37a447c13b47c in_seq=0 out_seq=0 2024-04-26T02:05:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:08.831+0000 7f7eb2a00640 10 client.26536 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:05:08.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:05:08 smithi151 ceph-mon[28057]: pgmap v362: 97 pgs: 97 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.3 MiB/s wr, 247 op/s 2024-04-26T02:05:08.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:05:08 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:05:08.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:05:08 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:05:08.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:05:08 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T02:05:09.608 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:09.609+0000 7f7eb2a00640 1 -- 172.21.15.151:0/2413582359 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5637a4cc4320 con 0x5637a4d6bf10 2024-04-26T02:05:09.608 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:09.609+0000 7f7eb2a00640 10 client.26536 handle_command_reply: tid=0 2024-04-26T02:05:09.609 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:09.609+0000 7f7ed6770740 1 client.26536 shutdown 2024-04-26T02:05:09.610 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:09.611+0000 7f7ed6770740 1 -- 172.21.15.151:0/2413582359 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637a4d512d0 msgr2=0x5637a4d61190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:09.610 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:09.611+0000 7f7ed6770740 1 --2- 172.21.15.151:0/2413582359 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637a4d512d0 0x5637a4d61190 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x5637a4d5e460 tx=0x7f7eb802a3f0 comp rx=0 tx=0).stop 2024-04-26T02:05:09.610 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:09.611+0000 7f7ed6770740 1 -- 172.21.15.151:0/2413582359 shutdown_connections 2024-04-26T02:05:09.610 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:09.611+0000 7f7ed6770740 1 --2- 172.21.15.151:0/2413582359 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5637a4d6bf10 0x5637a4dd4a20 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:09.611 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:09.611+0000 7f7ed6770740 1 --2- 172.21.15.151:0/2413582359 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5637a4d5bb90 0x5637a4d616d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:09.611 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:09.611+0000 7f7ed6770740 1 --2- 172.21.15.151:0/2413582359 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637a4d512d0 0x5637a4d61190 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:09.611 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:09.611+0000 7f7ed6770740 1 --2- 172.21.15.151:0/2413582359 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637a4d509b0 0x5637a4d63e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:09.611 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:09.611+0000 7f7eb2a00640 0 client.26536 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:05:09.611 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:09.611+0000 7f7ed6770740 1 -- 172.21.15.151:0/2413582359 >> 172.21.15.151:0/2413582359 conn(0x5637a4d4cec0 msgr2=0x5637a4d4c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:05:09.611 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:09.612+0000 7f7ed6770740 1 -- 172.21.15.151:0/2413582359 shutdown_connections 2024-04-26T02:05:09.611 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:09.612+0000 7f7ed6770740 1 -- 172.21.15.151:0/2413582359 wait complete. 2024-04-26T02:05:09.611 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:09.612+0000 7f7ed6770740 20 client.26536 trim_cache size 0 max 16384 2024-04-26T02:05:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:{ 2024-04-26T02:05:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "message": "", 2024-04-26T02:05:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout: "return_code": 0 2024-04-26T02:05:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stdout:} 2024-04-26T02:05:09.613 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:09.614+0000 7f7ed2400640 1 -- 172.21.15.151:0/500031366 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f7eb40a3d90 msgr2=0x7f7eb40a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:09.613 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:09.614+0000 7f7ed2400640 1 --2- 172.21.15.151:0/500031366 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f7eb40a3d90 0x7f7eb40a6250 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8000c00 tx=0x7f7eb8005a20 comp rx=0 tx=0).stop 2024-04-26T02:05:09.613 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:09.614+0000 7f7ed2400640 1 -- 172.21.15.151:0/500031366 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7ecc19a5e0 msgr2=0x7f7ecc19aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:09.613 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:09.614+0000 7f7ed2400640 1 --2- 172.21.15.151:0/500031366 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7ecc19a5e0 0x7f7ecc19aa90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7ec400eb10 tx=0x7f7ec400c2d0 comp rx=0 tx=0).stop 2024-04-26T02:05:09.613 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:09.614+0000 7f7ed2400640 1 -- 172.21.15.151:0/500031366 shutdown_connections 2024-04-26T02:05:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:09.614+0000 7f7ed2400640 1 --2- 172.21.15.151:0/500031366 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f7eb40a3d90 0x7f7eb40a6250 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:09.614+0000 7f7ed2400640 1 --2- 172.21.15.151:0/500031366 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7ecc19a5e0 0x7f7ecc19aa90 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:09.614+0000 7f7ed2400640 1 --2- 172.21.15.151:0/500031366 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ecc1092d0 0x7f7ecc1a0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:09.614+0000 7f7ed2400640 1 --2- 172.21.15.151:0/500031366 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ecc104200 0x7f7ecc1a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:09.614+0000 7f7ed2400640 1 -- 172.21.15.151:0/500031366 >> 172.21.15.151:0/500031366 conn(0x7f7ecc0ffa60 msgr2=0x7f7ecc100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:05:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:09.615+0000 7f7ed2400640 1 -- 172.21.15.151:0/500031366 shutdown_connections 2024-04-26T02:05:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:09.615+0000 7f7ed2400640 1 -- 172.21.15.151:0/500031366 wait complete. 2024-04-26T02:05:10.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:05:10 smithi172 ceph-mon[32785]: pgmap v363: 97 pgs: 97 active+clean; 190 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.1 MiB/s wr, 346 op/s 2024-04-26T02:05:10.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:05:10 smithi165 ceph-mon[32648]: pgmap v363: 97 pgs: 97 active+clean; 190 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.1 MiB/s wr, 346 op/s 2024-04-26T02:05:10.873 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:05:10 smithi151 ceph-mon[28057]: pgmap v363: 97 pgs: 97 active+clean; 190 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.1 MiB/s wr, 346 op/s 2024-04-26T02:05:12.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:05:12 smithi172 ceph-mon[32785]: pgmap v364: 97 pgs: 97 active+clean; 190 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 225 op/s 2024-04-26T02:05:12.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:05:12 smithi172 ceph-mon[32785]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T02:05:12.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:05:12 smithi165 ceph-mon[32648]: pgmap v364: 97 pgs: 97 active+clean; 190 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 225 op/s 2024-04-26T02:05:12.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:05:12 smithi165 ceph-mon[32648]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T02:05:12.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:05:12 smithi151 ceph-mon[28057]: pgmap v364: 97 pgs: 97 active+clean; 190 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 225 op/s 2024-04-26T02:05:12.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:05:12 smithi151 ceph-mon[28057]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T02:05:13.092 INFO:tasks.workunit:Stopping ['suites/fsx.sh'] on client.0... 2024-04-26T02:05:13.093 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-26T02:05:14.241 DEBUG:teuthology.parallel:result is None 2024-04-26T02:05:14.241 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-26T02:05:14.321 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-26T02:05:14.321 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-04-26T02:05:14.332 INFO:tasks.quiescer:joining Quiescers 2024-04-26T02:05:14.332 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-04-26T02:05:14.332 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T02:05:14.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:05:14 smithi151 ceph-mon[28057]: pgmap v365: 97 pgs: 97 active+clean; 177 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 427 op/s 2024-04-26T02:05:14.762 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:05:14.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:05:14 smithi172 ceph-mon[32785]: pgmap v365: 97 pgs: 97 active+clean; 177 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 427 op/s 2024-04-26T02:05:14.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:05:14 smithi165 ceph-mon[32648]: pgmap v365: 97 pgs: 97 active+clean; 177 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 427 op/s 2024-04-26T02:05:16.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:05:16 smithi172 ceph-mon[32785]: pgmap v366: 97 pgs: 97 active+clean; 173 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 379 op/s 2024-04-26T02:05:16.788 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:05:16 smithi172 ceph-mon[32785]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T02:05:16.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:05:16 smithi165 ceph-mon[32648]: pgmap v366: 97 pgs: 97 active+clean; 173 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 379 op/s 2024-04-26T02:05:16.847 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:05:16 smithi165 ceph-mon[32648]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T02:05:16.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:05:16 smithi151 ceph-mon[28057]: pgmap v366: 97 pgs: 97 active+clean; 173 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 379 op/s 2024-04-26T02:05:16.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:05:16 smithi151 ceph-mon[28057]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T02:05:17.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:17.522+0000 7fc893000700 1 -- 172.21.15.151:0/3126622823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc88c108a50 msgr2=0x7fc88c108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:17.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:17.522+0000 7fc893000700 1 --2- 172.21.15.151:0/3126622823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc88c108a50 0x7fc88c108e30 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc87c0098f0 tx=0x7fc87c009c00 comp rx=0 tx=0).stop 2024-04-26T02:05:17.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:17.522+0000 7fc893000700 1 -- 172.21.15.151:0/3126622823 shutdown_connections 2024-04-26T02:05:17.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:17.522+0000 7fc893000700 1 --2- 172.21.15.151:0/3126622823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc88c103450 0x7fc88c10f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:17.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:17.522+0000 7fc893000700 1 --2- 172.21.15.151:0/3126622823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc88c108a50 0x7fc88c108e30 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:17.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:17.522+0000 7fc893000700 1 --2- 172.21.15.151:0/3126622823 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc88c102a90 0x7fc88c102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:17.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:17.522+0000 7fc893000700 1 -- 172.21.15.151:0/3126622823 >> 172.21.15.151:0/3126622823 conn(0x7fc88c079ba0 msgr2=0x7fc88c079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:05:17.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:17.522+0000 7fc893000700 1 -- 172.21.15.151:0/3126622823 shutdown_connections 2024-04-26T02:05:17.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:17.522+0000 7fc893000700 1 -- 172.21.15.151:0/3126622823 wait complete. 2024-04-26T02:05:17.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:17.523+0000 7fc893000700 1 Processor -- start 2024-04-26T02:05:17.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:17.524+0000 7fc893000700 1 -- start start 2024-04-26T02:05:17.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:17.524+0000 7fc893000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc88c102a90 0x7fc88c19b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:17.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:17.524+0000 7fc893000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc88c103450 0x7fc88c19b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:17.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:17.524+0000 7fc893000700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc88c108a50 0x7fc88c19fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:17.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:17.524+0000 7fc893000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc88c112c40 con 0x7fc88c103450 2024-04-26T02:05:17.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:17.524+0000 7fc893000700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc88c112ac0 con 0x7fc88c102a90 2024-04-26T02:05:17.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:17.524+0000 7fc893000700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc88c112dc0 con 0x7fc88c108a50 2024-04-26T02:05:17.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:17.524+0000 7fc88b400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc88c103450 0x7fc88c19b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:17.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:17.524+0000 7fc88b400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc88c103450 0x7fc88c19b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:54690/0 (socket says 172.21.15.151:54690) 2024-04-26T02:05:17.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:17.524+0000 7fc88b400700 1 -- 172.21.15.151:0/1047962317 learned_addr learned my addr 172.21.15.151:0/1047962317 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:05:17.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:17.524+0000 7fc88be00700 1 --2- 172.21.15.151:0/1047962317 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc88c102a90 0x7fc88c19b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:17.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:17.524+0000 7fc891000700 1 --2- 172.21.15.151:0/1047962317 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc88c108a50 0x7fc88c19fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:17.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:17.525+0000 7fc88b400700 1 -- 172.21.15.151:0/1047962317 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc88c108a50 msgr2=0x7fc88c19fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:17.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:17.525+0000 7fc88b400700 1 --2- 172.21.15.151:0/1047962317 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc88c108a50 0x7fc88c19fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:17.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:17.525+0000 7fc88b400700 1 -- 172.21.15.151:0/1047962317 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc88c102a90 msgr2=0x7fc88c19b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:17.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:17.525+0000 7fc88b400700 1 --2- 172.21.15.151:0/1047962317 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc88c102a90 0x7fc88c19b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:17.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:17.525+0000 7fc88b400700 1 -- 172.21.15.151:0/1047962317 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc88c1a03b0 con 0x7fc88c103450 2024-04-26T02:05:17.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:17.525+0000 7fc891000700 1 --2- 172.21.15.151:0/1047962317 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc88c108a50 0x7fc88c19fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:05:17.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:17.525+0000 7fc88b400700 1 --2- 172.21.15.151:0/1047962317 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc88c103450 0x7fc88c19b8e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc87800eb30 tx=0x7fc87800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:05:17.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:17.525+0000 7fc888c00700 1 -- 172.21.15.151:0/1047962317 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc87800cbd0 con 0x7fc88c103450 2024-04-26T02:05:17.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:17.525+0000 7fc893000700 1 -- 172.21.15.151:0/1047962317 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc88c1a06a0 con 0x7fc88c103450 2024-04-26T02:05:17.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:17.525+0000 7fc888c00700 1 -- 172.21.15.151:0/1047962317 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8780044d0 con 0x7fc88c103450 2024-04-26T02:05:17.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:17.525+0000 7fc888c00700 1 -- 172.21.15.151:0/1047962317 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc878003e30 con 0x7fc88c103450 2024-04-26T02:05:17.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:17.526+0000 7fc893000700 1 -- 172.21.15.151:0/1047962317 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc88c1a7f20 con 0x7fc88c103450 2024-04-26T02:05:17.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:17.527+0000 7fc893000700 1 -- 172.21.15.151:0/1047962317 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc88c069730 con 0x7fc88c103450 2024-04-26T02:05:17.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:17.529+0000 7fc888c00700 1 -- 172.21.15.151:0/1047962317 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7fc878004000 con 0x7fc88c103450 2024-04-26T02:05:17.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:17.530+0000 7fc888c00700 1 --2- 172.21.15.151:0/1047962317 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fc8700a3e30 0x7fc8700a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:17.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:17.530+0000 7fc888c00700 1 -- 172.21.15.151:0/1047962317 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8411+0+0 (secure 0 0 0) 0x7fc8780d0af0 con 0x7fc88c103450 2024-04-26T02:05:17.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:17.531+0000 7fc88be00700 1 --2- 172.21.15.151:0/1047962317 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fc8700a3e30 0x7fc8700a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:17.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:17.531+0000 7fc88be00700 1 --2- 172.21.15.151:0/1047962317 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fc8700a3e30 0x7fc8700a62f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc87c0098c0 tx=0x7fc87c019040 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:05:17.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:17.535+0000 7fc888c00700 1 -- 172.21.15.151:0/1047962317 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc8780042f0 con 0x7fc88c103450 2024-04-26T02:05:17.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:17.810+0000 7fc893000700 1 -- 172.21.15.151:0/1047962317 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc88c02df80 con 0x7fc88c103450 2024-04-26T02:05:17.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:17.812+0000 7fc888c00700 1 -- 172.21.15.151:0/1047962317 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11891 (secure 0 0 0) 0x7fc87809d1a0 con 0x7fc88c103450 2024-04-26T02:05:17.811 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T02:05:17.812 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2937864097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2937864097},{"type":"v1","addr":"172.21.15.151:6835","nonce":2937864097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14373,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/2947090802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2947090802},{"type":"v1","addr":"172.21.15.151:6837","nonce":2947090802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/1356563280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1356563280},{"type":"v1","addr":"172.21.15.151:6839","nonce":1356563280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6841/1075240552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1075240552},{"type":"v1","addr":"172.21.15.151:6841","nonce":1075240552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/504675912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":504675912},{"type":"v1","addr":"172.21.15.165:6833","nonce":504675912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1207580015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1207580015},{"type":"v1","addr":"172.21.15.172:6833","nonce":1207580015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:49:40.746876+0000","modified":"2024-04-26T02:05:15.281789+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24505,"mds_2":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/3598138294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3598138294},{"type":"v1","addr":"172.21.15.165:6835","nonce":3598138294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/1847859054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":1847859054},{"type":"v1","addr":"172.21.15.165:6837","nonce":1847859054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.172:6837/55972537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":55972537},{"type":"v1","addr":"172.21.15.172:6837","nonce":55972537}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.172:6839/1242302810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1242302810},{"type":"v1","addr":"172.21.15.172:6839","nonce":1242302810}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/1969716254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1969716254},{"type":"v1","addr":"172.21.15.165:6839","nonce":1969716254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26482":{"gid":26482,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/1218148590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":1218148590},{"type":"v1","addr":"172.21.15.172:6835","nonce":1218148590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24509,24493,24505]},"id":1}]} 2024-04-26T02:05:17.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:17.815+0000 7fc893000700 1 -- 172.21.15.151:0/1047962317 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fc8700a3e30 msgr2=0x7fc8700a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:17.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:17.815+0000 7fc893000700 1 --2- 172.21.15.151:0/1047962317 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fc8700a3e30 0x7fc8700a62f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc87c0098c0 tx=0x7fc87c019040 comp rx=0 tx=0).stop 2024-04-26T02:05:17.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:17.815+0000 7fc893000700 1 -- 172.21.15.151:0/1047962317 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc88c103450 msgr2=0x7fc88c19b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:17.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:17.815+0000 7fc893000700 1 --2- 172.21.15.151:0/1047962317 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc88c103450 0x7fc88c19b8e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc87800eb30 tx=0x7fc87800ee40 comp rx=0 tx=0).stop 2024-04-26T02:05:17.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:17.815+0000 7fc893000700 1 -- 172.21.15.151:0/1047962317 shutdown_connections 2024-04-26T02:05:17.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:17.815+0000 7fc893000700 1 --2- 172.21.15.151:0/1047962317 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fc8700a3e30 0x7fc8700a62f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:17.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:17.815+0000 7fc893000700 1 --2- 172.21.15.151:0/1047962317 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc88c108a50 0x7fc88c19fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:17.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:17.815+0000 7fc893000700 1 --2- 172.21.15.151:0/1047962317 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc88c102a90 0x7fc88c19b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:17.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:17.815+0000 7fc893000700 1 --2- 172.21.15.151:0/1047962317 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc88c103450 0x7fc88c19b8e0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:17.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:17.815+0000 7fc893000700 1 -- 172.21.15.151:0/1047962317 >> 172.21.15.151:0/1047962317 conn(0x7fc88c079ba0 msgr2=0x7fc88c10dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:05:17.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:17.816+0000 7fc893000700 1 -- 172.21.15.151:0/1047962317 shutdown_connections 2024-04-26T02:05:17.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:17.816+0000 7fc893000700 1 -- 172.21.15.151:0/1047962317 wait complete. 2024-04-26T02:05:17.816 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 39 2024-04-26T02:05:18.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:05:18 smithi172 ceph-mon[32785]: pgmap v367: 97 pgs: 97 active+clean; 173 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 347 op/s 2024-04-26T02:05:18.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:05:18 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/1047962317' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:05:18.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:05:18 smithi165 ceph-mon[32648]: pgmap v367: 97 pgs: 97 active+clean; 173 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 347 op/s 2024-04-26T02:05:18.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:05:18 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/1047962317' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:05:18.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:05:18 smithi151 ceph-mon[28057]: pgmap v367: 97 pgs: 97 active+clean; 173 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 347 op/s 2024-04-26T02:05:18.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:05:18 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/1047962317' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:05:18.970 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24509 quiesce db --cancel --all' 2024-04-26T02:05:18.970 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24509 quiesce db --cancel --all 2024-04-26T02:05:19.418 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:05:20.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:05:20 smithi172 ceph-mon[32785]: pgmap v368: 97 pgs: 97 active+clean; 174 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 378 op/s 2024-04-26T02:05:20.835 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:05:20 smithi151 ceph-mon[28057]: pgmap v368: 97 pgs: 97 active+clean; 174 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 378 op/s 2024-04-26T02:05:20.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:05:20 smithi165 ceph-mon[32648]: pgmap v368: 97 pgs: 97 active+clean; 174 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 378 op/s 2024-04-26T02:05:21.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.761+0000 7fdbf0c00700 1 -- 172.21.15.151:0/3223351765 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdbec105870 msgr2=0x7fdbec1098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:21.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.761+0000 7fdbf0c00700 1 --2- 172.21.15.151:0/3223351765 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdbec105870 0x7fdbec1098c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fdbd80099c0 tx=0x7fdbd8009cd0 comp rx=0 tx=0).stop 2024-04-26T02:05:21.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.761+0000 7fdbf0c00700 1 -- 172.21.15.151:0/3223351765 shutdown_connections 2024-04-26T02:05:21.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.761+0000 7fdbf0c00700 1 --2- 172.21.15.151:0/3223351765 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdbec109fa0 0x7fdbec111b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:21.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.761+0000 7fdbf0c00700 1 --2- 172.21.15.151:0/3223351765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbec104ec0 0x7fdbec1052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:21.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.761+0000 7fdbf0c00700 1 --2- 172.21.15.151:0/3223351765 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdbec105870 0x7fdbec1098c0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:21.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.761+0000 7fdbf0c00700 1 -- 172.21.15.151:0/3223351765 >> 172.21.15.151:0/3223351765 conn(0x7fdbec100750 msgr2=0x7fdbec102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:05:21.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.762+0000 7fdbf0c00700 1 -- 172.21.15.151:0/3223351765 shutdown_connections 2024-04-26T02:05:21.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.762+0000 7fdbf0c00700 1 -- 172.21.15.151:0/3223351765 wait complete. 2024-04-26T02:05:21.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.763+0000 7fdbf0c00700 1 Processor -- start 2024-04-26T02:05:21.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.763+0000 7fdbf0c00700 1 -- start start 2024-04-26T02:05:21.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.763+0000 7fdbf0c00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdbec104ec0 0x7fdbec19b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:21.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.763+0000 7fdbf0c00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbec105870 0x7fdbec19b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:21.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.763+0000 7fdbf0c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdbec109fa0 0x7fdbec19fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:21.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.763+0000 7fdbf0c00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdbec114e60 con 0x7fdbec109fa0 2024-04-26T02:05:21.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.763+0000 7fdbf0c00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fdbec114ce0 con 0x7fdbec104ec0 2024-04-26T02:05:21.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.763+0000 7fdbf0c00700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdbec114fe0 con 0x7fdbec105870 2024-04-26T02:05:21.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.764+0000 7fdbea800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdbec109fa0 0x7fdbec19fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:21.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.764+0000 7fdbea800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdbec109fa0 0x7fdbec19fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:48302/0 (socket says 172.21.15.151:48302) 2024-04-26T02:05:21.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.764+0000 7fdbea800700 1 -- 172.21.15.151:0/826138695 learned_addr learned my addr 172.21.15.151:0/826138695 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:05:21.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.764+0000 7fdbe9e00700 1 --2- 172.21.15.151:0/826138695 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdbec104ec0 0x7fdbec19b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:21.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.764+0000 7fdbe9400700 1 --2- 172.21.15.151:0/826138695 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbec105870 0x7fdbec19b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:21.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.764+0000 7fdbe9e00700 1 -- 172.21.15.151:0/826138695 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbec105870 msgr2=0x7fdbec19b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:21.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.764+0000 7fdbe9e00700 1 --2- 172.21.15.151:0/826138695 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbec105870 0x7fdbec19b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:21.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.764+0000 7fdbe9e00700 1 -- 172.21.15.151:0/826138695 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdbec109fa0 msgr2=0x7fdbec19fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:21.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.764+0000 7fdbe9e00700 1 --2- 172.21.15.151:0/826138695 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdbec109fa0 0x7fdbec19fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:21.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.764+0000 7fdbe9e00700 1 -- 172.21.15.151:0/826138695 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbec1a0330 con 0x7fdbec104ec0 2024-04-26T02:05:21.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.764+0000 7fdbea800700 1 --2- 172.21.15.151:0/826138695 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdbec109fa0 0x7fdbec19fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:05:21.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.764+0000 7fdbe9e00700 1 --2- 172.21.15.151:0/826138695 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdbec104ec0 0x7fdbec19b370 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc00d950 tx=0x7fdbdc00dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:05:21.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.765+0000 7fdbe2a00700 1 -- 172.21.15.151:0/826138695 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc016070 con 0x7fdbec104ec0 2024-04-26T02:05:21.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.765+0000 7fdbe2a00700 1 -- 172.21.15.151:0/826138695 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbdc00fd40 con 0x7fdbec104ec0 2024-04-26T02:05:21.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.765+0000 7fdbf0c00700 1 -- 172.21.15.151:0/826138695 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbec1a0620 con 0x7fdbec104ec0 2024-04-26T02:05:21.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.765+0000 7fdbe2a00700 1 -- 172.21.15.151:0/826138695 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc00e3b0 con 0x7fdbec104ec0 2024-04-26T02:05:21.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.765+0000 7fdbf0c00700 1 -- 172.21.15.151:0/826138695 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbec1a7ea0 con 0x7fdbec104ec0 2024-04-26T02:05:21.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.768+0000 7fdbe2a00700 1 -- 172.21.15.151:0/826138695 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7fdbdc019070 con 0x7fdbec104ec0 2024-04-26T02:05:21.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.770+0000 7fdbfb5a0b80 1 Processor -- start 2024-04-26T02:05:21.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.770+0000 7fdbfb5a0b80 1 -- start start 2024-04-26T02:05:21.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.770+0000 7fdbe2a00700 1 --2- 172.21.15.151:0/826138695 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fdbd00a3e30 0x7fdbd00a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:21.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.770+0000 7fdbfb5a0b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591e2b36cb0 0x5591e2b37090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:21.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.770+0000 7fdbfb5a0b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591e2b375d0 0x5591e2b41f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:21.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.770+0000 7fdbfb5a0b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5591e2b42440 0x5591e2b44830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:21.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.770+0000 7fdbfb5a0b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5591e2a89f00 con 0x5591e2b36cb0 2024-04-26T02:05:21.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.770+0000 7fdbfb5a0b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5591e2a307a0 con 0x5591e2b42440 2024-04-26T02:05:21.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.770+0000 7fdbfb5a0b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5591e2a85ec0 con 0x5591e2b375d0 2024-04-26T02:05:21.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.770+0000 7fdbe2a00700 1 -- 172.21.15.151:0/826138695 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8411+0+0 (secure 0 0 0) 0x7fdbdc0d0150 con 0x7fdbec104ec0 2024-04-26T02:05:21.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.771+0000 7fdbe9e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591e2b36cb0 0x5591e2b37090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:21.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.771+0000 7fdbea800700 1 --2- 172.21.15.151:0/826138695 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fdbd00a3e30 0x7fdbd00a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:21.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.771+0000 7fdbe9e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591e2b36cb0 0x5591e2b37090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:48312/0 (socket says 172.21.15.151:48312) 2024-04-26T02:05:21.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.771+0000 7fdbe9e00700 1 -- 172.21.15.151:0/2605947406 learned_addr learned my addr 172.21.15.151:0/2605947406 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:05:21.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.771+0000 7fdbe9400700 1 --2- 172.21.15.151:0/2605947406 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591e2b375d0 0x5591e2b41f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:21.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.771+0000 7fdbea800700 1 --2- 172.21.15.151:0/2605947406 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5591e2b42440 0x5591e2b44830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:21.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.771+0000 7fdbe9400700 1 -- 172.21.15.151:0/2605947406 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5591e2b42440 msgr2=0x5591e2b44830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:05:21.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.771+0000 7fdbe9400700 1 --2- 172.21.15.151:0/2605947406 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5591e2b42440 0x5591e2b44830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:21.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.771+0000 7fdbe9400700 1 -- 172.21.15.151:0/2605947406 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591e2b36cb0 msgr2=0x5591e2b37090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:05:21.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.771+0000 7fdbe9400700 1 --2- 172.21.15.151:0/2605947406 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591e2b36cb0 0x5591e2b37090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:21.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.771+0000 7fdbe9400700 1 -- 172.21.15.151:0/2605947406 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591e2a266f0 con 0x5591e2b375d0 2024-04-26T02:05:21.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.771+0000 7fdbea800700 1 --2- 172.21.15.151:0/826138695 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fdbd00a3e30 0x7fdbd00a62f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4009fd0 tx=0x7fdbe4009470 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:05:21.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.772+0000 7fdbe9400700 1 --2- 172.21.15.151:0/2605947406 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591e2b375d0 0x5591e2b41f00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fdbd800b580 tx=0x7fdbd8005bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd6c940ed12948d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:05:21.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.772+0000 7fdbd7400700 1 -- 172.21.15.151:0/2605947406 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd8028070 con 0x5591e2b375d0 2024-04-26T02:05:21.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.772+0000 7fdbd7400700 1 -- 172.21.15.151:0/2605947406 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbd80037c0 con 0x5591e2b375d0 2024-04-26T02:05:21.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.772+0000 7fdbd7400700 1 -- 172.21.15.151:0/2605947406 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd800b9d0 con 0x5591e2b375d0 2024-04-26T02:05:21.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.773+0000 7fdbfb5a0b80 1 -- 172.21.15.151:0/2605947406 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591e2b375d0 msgr2=0x5591e2b41f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:21.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.773+0000 7fdbfb5a0b80 1 --2- 172.21.15.151:0/2605947406 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591e2b375d0 0x5591e2b41f00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fdbd800b580 tx=0x7fdbd8005bc0 comp rx=0 tx=0).stop 2024-04-26T02:05:21.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.773+0000 7fdbfb5a0b80 1 -- 172.21.15.151:0/2605947406 shutdown_connections 2024-04-26T02:05:21.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.773+0000 7fdbfb5a0b80 1 --2- 172.21.15.151:0/2605947406 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5591e2b42440 0x5591e2b44830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:21.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.773+0000 7fdbfb5a0b80 1 --2- 172.21.15.151:0/2605947406 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591e2b36cb0 0x5591e2b37090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:21.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.773+0000 7fdbfb5a0b80 1 --2- 172.21.15.151:0/2605947406 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591e2b375d0 0x5591e2b41f00 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:21.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.773+0000 7fdbfb5a0b80 1 -- 172.21.15.151:0/2605947406 >> 172.21.15.151:0/2605947406 conn(0x5591e2a65fc0 msgr2=0x5591e2b331c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:05:21.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.773+0000 7fdbfb5a0b80 1 -- 172.21.15.151:0/2605947406 shutdown_connections 2024-04-26T02:05:21.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.773+0000 7fdbfb5a0b80 1 -- 172.21.15.151:0/2605947406 wait complete. 2024-04-26T02:05:21.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.774+0000 7fdbfb5a0b80 1 -- start start 2024-04-26T02:05:21.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.774+0000 7fdbfb5a0b80 1 Processor -- start 2024-04-26T02:05:21.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.774+0000 7fdbfb5a0b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591e2b36cb0 0x5591e2b41b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:21.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.774+0000 7fdbfb5a0b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5591e2b375d0 0x5591e2b3ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:21.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.774+0000 7fdbfb5a0b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591e2b42440 0x5591e2b3f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:21.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.774+0000 7fdbfb5a0b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5591e2b44ed0 con 0x5591e2b42440 2024-04-26T02:05:21.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.774+0000 7fdbfb5a0b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5591e2a438d0 con 0x5591e2b375d0 2024-04-26T02:05:21.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.774+0000 7fdbfb5a0b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5591e2b45050 con 0x5591e2b36cb0 2024-04-26T02:05:21.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.774+0000 7fdbfb5a0b80 10 client.0 _ll_register_callbacks cb 0x5591e2a22790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:05:21.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.775+0000 7fdbea800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591e2b42440 0x5591e2b3f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:21.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.775+0000 7fdbe9400700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5591e2b375d0 0x5591e2b3ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:21.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.775+0000 7fdbea800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591e2b42440 0x5591e2b3f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:48314/0 (socket says 172.21.15.151:48314) 2024-04-26T02:05:21.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.775+0000 7fdbea800700 1 -- 172.21.15.151:0/144150052 learned_addr learned my addr 172.21.15.151:0/144150052 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:05:21.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.775+0000 7fdbe9e00700 1 --2- 172.21.15.151:0/144150052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591e2b36cb0 0x5591e2b41b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:21.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.775+0000 7fdbe9400700 1 -- 172.21.15.151:0/144150052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591e2b36cb0 msgr2=0x5591e2b41b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:05:21.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.775+0000 7fdbe9400700 1 --2- 172.21.15.151:0/144150052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591e2b36cb0 0x5591e2b41b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:21.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.775+0000 7fdbe9400700 1 -- 172.21.15.151:0/144150052 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591e2b42440 msgr2=0x5591e2b3f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:05:21.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.775+0000 7fdbe9400700 1 --2- 172.21.15.151:0/144150052 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591e2b42440 0x5591e2b3f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:21.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.775+0000 7fdbe9400700 1 -- 172.21.15.151:0/144150052 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591e29d7940 con 0x5591e2b375d0 2024-04-26T02:05:21.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.776+0000 7fdbe9400700 1 --2- 172.21.15.151:0/144150052 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5591e2b375d0 0x5591e2b3ee40 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8009350 tx=0x7fdbd80058e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da867c5b7c863597 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:05:21.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.776+0000 7fdbcbe00700 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T02:05:21.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.776+0000 7fdbcbe00700 1 -- 172.21.15.151:0/144150052 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd8030020 con 0x5591e2b375d0 2024-04-26T02:05:21.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.776+0000 7fdbfb5a0b80 1 -- 172.21.15.151:0/144150052 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbd80037c0 con 0x5591e2b375d0 2024-04-26T02:05:21.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.776+0000 7fdbcbe00700 1 -- 172.21.15.151:0/144150052 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8411+0+0 (secure 0 0 0) 0x7fdbd8005310 con 0x5591e2b375d0 2024-04-26T02:05:21.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.776+0000 7fdbcbe00700 1 -- 172.21.15.151:0/144150052 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fdbd803b050 con 0x5591e2b375d0 2024-04-26T02:05:21.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.776+0000 7fdbfb5a0b80 10 client.26528 fetch_fsmap learned FSMap version 39 2024-04-26T02:05:21.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.776+0000 7fdbfb5a0b80 1 -- 172.21.15.151:0/144150052 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5591e2a20b60 con 0x5591e2b375d0 2024-04-26T02:05:21.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.776+0000 7fdbcbe00700 1 -- 172.21.15.151:0/144150052 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbd80037c0 con 0x5591e2b375d0 2024-04-26T02:05:21.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.777+0000 7fdbcbe00700 1 -- 172.21.15.151:0/144150052 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fdbd802b070 con 0x5591e2b375d0 2024-04-26T02:05:21.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.777+0000 7fdbfb5a0b80 10 client.26528 fetch_fsmap finished waiting for FSMap version 39 2024-04-26T02:05:21.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.777+0000 7fdbfb5a0b80 10 client.26528 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-26T02:05:21.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.777+0000 7fdbfb5a0b80 20 client.26528 populate_metadata read hostname 'smithi151' 2024-04-26T02:05:21.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.777+0000 7fdbfb5a0b80 1 --2- 172.21.15.151:0/144150052 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5591e2b51a10 0x5591e2b53e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:21.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.777+0000 7fdbfb5a0b80 4 client.26528 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:05:21.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.777+0000 7fdbfb5a0b80 1 -- 172.21.15.151:0/144150052 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591e2a976f0 con 0x5591e2b51a10 2024-04-26T02:05:21.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.777+0000 7fdbe9e00700 1 --2- 172.21.15.151:0/144150052 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5591e2b51a10 0x5591e2b53e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:21.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.778+0000 7fdbe9e00700 1 --2- 172.21.15.151:0/144150052 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5591e2b51a10 0x5591e2b53e00 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd56f32ab9941ad6 server_cookie=5fb7aa7c27566645 in_seq=0 out_seq=0 2024-04-26T02:05:21.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.778+0000 7fdbcbe00700 10 client.26528 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:05:21.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.778+0000 7fdbcbe00700 1 -- 172.21.15.151:0/144150052 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5591e2a976f0 con 0x5591e2b51a10 2024-04-26T02:05:21.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.778+0000 7fdbcbe00700 10 client.26528 handle_command_reply: tid=0 2024-04-26T02:05:21.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.778+0000 7fdbfb5a0b80 1 client.26528 shutdown 2024-04-26T02:05:21.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.779+0000 7fdbfb5a0b80 1 -- 172.21.15.151:0/144150052 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5591e2b375d0 msgr2=0x5591e2b3ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:21.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.779+0000 7fdbfb5a0b80 1 --2- 172.21.15.151:0/144150052 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5591e2b375d0 0x5591e2b3ee40 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8009350 tx=0x7fdbd80058e0 comp rx=0 tx=0).stop 2024-04-26T02:05:21.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.779+0000 7fdbfb5a0b80 1 -- 172.21.15.151:0/144150052 shutdown_connections 2024-04-26T02:05:21.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.779+0000 7fdbfb5a0b80 1 --2- 172.21.15.151:0/144150052 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591e2b42440 0x5591e2b3f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:21.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.779+0000 7fdbfb5a0b80 1 --2- 172.21.15.151:0/144150052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591e2b36cb0 0x5591e2b41b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:21.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.779+0000 7fdbfb5a0b80 1 --2- 172.21.15.151:0/144150052 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5591e2b51a10 0x5591e2b53e00 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:21.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.779+0000 7fdbfb5a0b80 1 --2- 172.21.15.151:0/144150052 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5591e2b375d0 0x5591e2b3ee40 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:21.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.779+0000 7fdbfb5a0b80 1 -- 172.21.15.151:0/144150052 >> 172.21.15.151:0/144150052 conn(0x5591e2a65fc0 msgr2=0x5591e2b32db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:05:21.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.779+0000 7fdbcbe00700 0 client.26528 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:05:21.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.779+0000 7fdbfb5a0b80 1 -- 172.21.15.151:0/144150052 shutdown_connections 2024-04-26T02:05:21.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.779+0000 7fdbfb5a0b80 1 -- 172.21.15.151:0/144150052 wait complete. 2024-04-26T02:05:21.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.779+0000 7fdbfb5a0b80 20 client.26528 trim_cache size 0 max 16384 2024-04-26T02:05:21.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.800+0000 7fdbfb5a0b80 1 Processor -- start 2024-04-26T02:05:21.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.800+0000 7fdbfb5a0b80 1 -- start start 2024-04-26T02:05:21.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.801+0000 7fdbfb5a0b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591e2b36cb0 0x5591e2b39460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:21.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.801+0000 7fdbfb5a0b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5591e2b375d0 0x5591e2b3c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:21.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.801+0000 7fdbfb5a0b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591e2b42440 0x5591e2b3c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:21.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.801+0000 7fdbfb5a0b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdbec1b1ba0 con 0x5591e2b36cb0 2024-04-26T02:05:21.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.801+0000 7fdbfb5a0b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fdbec1af400 con 0x5591e2b375d0 2024-04-26T02:05:21.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.801+0000 7fdbfb5a0b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdbec1ad270 con 0x5591e2b42440 2024-04-26T02:05:21.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.801+0000 7fdbe9e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591e2b36cb0 0x5591e2b39460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:21.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.801+0000 7fdbe9e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591e2b36cb0 0x5591e2b39460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:48322/0 (socket says 172.21.15.151:48322) 2024-04-26T02:05:21.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.801+0000 7fdbe9e00700 1 -- 172.21.15.151:0/4240593336 learned_addr learned my addr 172.21.15.151:0/4240593336 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:05:21.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.801+0000 7fdbe9400700 1 --2- 172.21.15.151:0/4240593336 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5591e2b375d0 0x5591e2b3c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:21.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.801+0000 7fdbea800700 1 --2- 172.21.15.151:0/4240593336 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591e2b42440 0x5591e2b3c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:21.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.802+0000 7fdbe9400700 1 -- 172.21.15.151:0/4240593336 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591e2b42440 msgr2=0x5591e2b3c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:05:21.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.802+0000 7fdbe9400700 1 --2- 172.21.15.151:0/4240593336 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591e2b42440 0x5591e2b3c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:21.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.802+0000 7fdbe9400700 1 -- 172.21.15.151:0/4240593336 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591e2b36cb0 msgr2=0x5591e2b39460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:05:21.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.802+0000 7fdbe9400700 1 --2- 172.21.15.151:0/4240593336 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591e2b36cb0 0x5591e2b39460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:21.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.802+0000 7fdbe9400700 1 -- 172.21.15.151:0/4240593336 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591e2b38b20 con 0x5591e2b375d0 2024-04-26T02:05:21.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.802+0000 7fdbe9e00700 1 --2- 172.21.15.151:0/4240593336 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591e2b36cb0 0x5591e2b39460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:05:21.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.802+0000 7fdbe9400700 1 --2- 172.21.15.151:0/4240593336 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5591e2b375d0 0x5591e2b3c100 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fdbd80098e0 tx=0x7fdbd8004fe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27177768cee3a91a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:05:21.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.802+0000 7fdbd6a00700 1 -- 172.21.15.151:0/4240593336 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd8030040 con 0x5591e2b375d0 2024-04-26T02:05:21.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.802+0000 7fdbd6a00700 1 -- 172.21.15.151:0/4240593336 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbd800fac0 con 0x5591e2b375d0 2024-04-26T02:05:21.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.802+0000 7fdbd6a00700 1 -- 172.21.15.151:0/4240593336 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd8028040 con 0x5591e2b375d0 2024-04-26T02:05:21.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.803+0000 7fdbfb5a0b80 1 -- 172.21.15.151:0/4240593336 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5591e2b375d0 msgr2=0x5591e2b3c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:21.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.803+0000 7fdbfb5a0b80 1 --2- 172.21.15.151:0/4240593336 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5591e2b375d0 0x5591e2b3c100 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fdbd80098e0 tx=0x7fdbd8004fe0 comp rx=0 tx=0).stop 2024-04-26T02:05:21.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.803+0000 7fdbfb5a0b80 1 -- 172.21.15.151:0/4240593336 shutdown_connections 2024-04-26T02:05:21.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.803+0000 7fdbfb5a0b80 1 --2- 172.21.15.151:0/4240593336 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591e2b42440 0x5591e2b3c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:21.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.803+0000 7fdbfb5a0b80 1 --2- 172.21.15.151:0/4240593336 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591e2b36cb0 0x5591e2b39460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:21.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.803+0000 7fdbfb5a0b80 1 --2- 172.21.15.151:0/4240593336 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5591e2b375d0 0x5591e2b3c100 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:21.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.803+0000 7fdbfb5a0b80 1 -- 172.21.15.151:0/4240593336 >> 172.21.15.151:0/4240593336 conn(0x5591e2a65fc0 msgr2=0x5591e2b38d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:05:21.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.803+0000 7fdbfb5a0b80 1 -- 172.21.15.151:0/4240593336 shutdown_connections 2024-04-26T02:05:21.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.803+0000 7fdbfb5a0b80 1 -- 172.21.15.151:0/4240593336 wait complete. 2024-04-26T02:05:21.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.804+0000 7fdbfb5a0b80 1 -- start start 2024-04-26T02:05:21.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.804+0000 7fdbfb5a0b80 1 Processor -- start 2024-04-26T02:05:21.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.804+0000 7fdbfb5a0b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5591e2b36cb0 0x5591e2b560d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:21.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.804+0000 7fdbfb5a0b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591e2b375d0 0x5591e2b56610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:21.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.804+0000 7fdbfb5a0b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591e2b42440 0x5591e2bb1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:21.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.804+0000 7fdbfb5a0b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdbec1ad8c0 con 0x5591e2b42440 2024-04-26T02:05:21.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.804+0000 7fdbfb5a0b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fdbec1aed30 con 0x5591e2b36cb0 2024-04-26T02:05:21.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.804+0000 7fdbfb5a0b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdbec1adf90 con 0x5591e2b375d0 2024-04-26T02:05:21.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.804+0000 7fdbfb5a0b80 10 client.0 _ll_register_callbacks cb 0x5591e2a22790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:05:21.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.805+0000 7fdbe9e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5591e2b36cb0 0x5591e2b560d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:21.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.805+0000 7fdbe9e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5591e2b36cb0 0x5591e2b560d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:41386/0 (socket says 172.21.15.151:41386) 2024-04-26T02:05:21.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.805+0000 7fdbea800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591e2b42440 0x5591e2bb1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:21.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.805+0000 7fdbe9e00700 1 -- 172.21.15.151:0/3558710130 learned_addr learned my addr 172.21.15.151:0/3558710130 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:05:21.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.805+0000 7fdbe9400700 1 --2- 172.21.15.151:0/3558710130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591e2b375d0 0x5591e2b56610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:21.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.805+0000 7fdbe9e00700 1 -- 172.21.15.151:0/3558710130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591e2b375d0 msgr2=0x5591e2b56610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:05:21.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.805+0000 7fdbe9e00700 1 --2- 172.21.15.151:0/3558710130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591e2b375d0 0x5591e2b56610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:21.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.805+0000 7fdbe9e00700 1 -- 172.21.15.151:0/3558710130 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591e2b42440 msgr2=0x5591e2bb1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:05:21.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.805+0000 7fdbe9e00700 1 --2- 172.21.15.151:0/3558710130 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591e2b42440 0x5591e2bb1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:21.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.805+0000 7fdbe9e00700 1 -- 172.21.15.151:0/3558710130 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591e2b37c50 con 0x5591e2b36cb0 2024-04-26T02:05:21.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.805+0000 7fdbea800700 1 --2- 172.21.15.151:0/3558710130 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591e2b42440 0x5591e2bb1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:05:21.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.805+0000 7fdbe9e00700 1 --2- 172.21.15.151:0/3558710130 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5591e2b36cb0 0x5591e2b560d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc098ad0 tx=0x7fdbdc09c100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9ab01a4ad48b456 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:05:21.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.805+0000 7fdbcbe00700 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T02:05:21.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.806+0000 7fdbcbe00700 1 -- 172.21.15.151:0/3558710130 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc020040 con 0x5591e2b36cb0 2024-04-26T02:05:21.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.806+0000 7fdbcbe00700 1 -- 172.21.15.151:0/3558710130 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8411+0+0 (secure 0 0 0) 0x7fdbdc09a400 con 0x5591e2b36cb0 2024-04-26T02:05:21.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.806+0000 7fdbfb5a0b80 1 -- 172.21.15.151:0/3558710130 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbd800fac0 con 0x5591e2b36cb0 2024-04-26T02:05:21.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.806+0000 7fdbcbe00700 1 -- 172.21.15.151:0/3558710130 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fdbdc0a4d70 con 0x5591e2b36cb0 2024-04-26T02:05:21.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.806+0000 7fdbfb5a0b80 10 client.26534 fetch_fsmap learned FSMap version 39 2024-04-26T02:05:21.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.806+0000 7fdbfb5a0b80 1 -- 172.21.15.151:0/3558710130 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5591e2bb9440 con 0x5591e2b36cb0 2024-04-26T02:05:21.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.806+0000 7fdbcbe00700 1 -- 172.21.15.151:0/3558710130 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbd800fac0 con 0x5591e2b36cb0 2024-04-26T02:05:21.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.806+0000 7fdbcbe00700 1 -- 172.21.15.151:0/3558710130 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fdbdc09c360 con 0x5591e2b36cb0 2024-04-26T02:05:21.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.806+0000 7fdbfb5a0b80 10 client.26534 fetch_fsmap finished waiting for FSMap version 39 2024-04-26T02:05:21.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.806+0000 7fdbfb5a0b80 10 client.26534 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-26T02:05:21.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.806+0000 7fdbfb5a0b80 20 client.26534 populate_metadata read hostname 'smithi151' 2024-04-26T02:05:21.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.806+0000 7fdbfb5a0b80 1 --2- 172.21.15.151:0/3558710130 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5591e2bb9ad0 0x5591e2bbbef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:21.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.806+0000 7fdbfb5a0b80 4 client.26534 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-04-26T02:05:21.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.806+0000 7fdbfb5a0b80 1 -- 172.21.15.151:0/3558710130 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x5591e2a91790 con 0x5591e2bb9ad0 2024-04-26T02:05:21.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.807+0000 7fdbe9400700 1 --2- 172.21.15.151:0/3558710130 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5591e2bb9ad0 0x5591e2bbbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:21.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.807+0000 7fdbe9400700 1 --2- 172.21.15.151:0/3558710130 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5591e2bb9ad0 0x5591e2bbbef0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=650d5b728ba9b67f server_cookie=e1fd1ce9d8f991f0 in_seq=0 out_seq=0 2024-04-26T02:05:21.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.807+0000 7fdbcbe00700 10 client.26534 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:05:21.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.807+0000 7fdbcbe00700 1 -- 172.21.15.151:0/3558710130 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x5591e2a91790 con 0x5591e2bb9ad0 2024-04-26T02:05:21.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.807+0000 7fdbcbe00700 10 client.26534 handle_command_reply: tid=0 2024-04-26T02:05:21.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.807+0000 7fdbfb5a0b80 1 client.26534 shutdown 2024-04-26T02:05:21.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.808+0000 7fdbfb5a0b80 1 -- 172.21.15.151:0/3558710130 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5591e2b36cb0 msgr2=0x5591e2b560d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:21.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.808+0000 7fdbfb5a0b80 1 --2- 172.21.15.151:0/3558710130 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5591e2b36cb0 0x5591e2b560d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc098ad0 tx=0x7fdbdc09c100 comp rx=0 tx=0).stop 2024-04-26T02:05:21.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.808+0000 7fdbfb5a0b80 1 -- 172.21.15.151:0/3558710130 shutdown_connections 2024-04-26T02:05:21.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.808+0000 7fdbfb5a0b80 1 --2- 172.21.15.151:0/3558710130 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591e2b42440 0x5591e2bb1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:21.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.808+0000 7fdbfb5a0b80 1 --2- 172.21.15.151:0/3558710130 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x5591e2bb9ad0 0x5591e2bbbef0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:21.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.808+0000 7fdbfb5a0b80 1 --2- 172.21.15.151:0/3558710130 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5591e2b36cb0 0x5591e2b560d0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:21.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.808+0000 7fdbfb5a0b80 1 --2- 172.21.15.151:0/3558710130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591e2b375d0 0x5591e2b56610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:21.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.808+0000 7fdbfb5a0b80 1 -- 172.21.15.151:0/3558710130 >> 172.21.15.151:0/3558710130 conn(0x5591e2a65fc0 msgr2=0x5591e2b395e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:05:21.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.808+0000 7fdbcbe00700 0 client.26534 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:05:21.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.808+0000 7fdbfb5a0b80 1 -- 172.21.15.151:0/3558710130 shutdown_connections 2024-04-26T02:05:21.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.808+0000 7fdbfb5a0b80 1 -- 172.21.15.151:0/3558710130 wait complete. 2024-04-26T02:05:21.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.808+0000 7fdbfb5a0b80 20 client.26534 trim_cache size 0 max 16384 2024-04-26T02:05:21.812 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-26T02:05:21.812 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 39, 2024-04-26T02:05:21.813 INFO:teuthology.orchestra.run.smithi151.stdout: "leader": 24509, 2024-04-26T02:05:21.813 INFO:teuthology.orchestra.run.smithi151.stdout: "set_version": 32, 2024-04-26T02:05:21.813 INFO:teuthology.orchestra.run.smithi151.stdout: "sets": {} 2024-04-26T02:05:21.813 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-26T02:05:21.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.809+0000 7fdbf0c00700 1 -- 172.21.15.151:0/826138695 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fdbd00a3e30 msgr2=0x7fdbd00a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:21.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.809+0000 7fdbf0c00700 1 --2- 172.21.15.151:0/826138695 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fdbd00a3e30 0x7fdbd00a62f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4009fd0 tx=0x7fdbe4009470 comp rx=0 tx=0).stop 2024-04-26T02:05:21.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.809+0000 7fdbf0c00700 1 -- 172.21.15.151:0/826138695 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdbec104ec0 msgr2=0x7fdbec19b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:21.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.809+0000 7fdbf0c00700 1 --2- 172.21.15.151:0/826138695 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdbec104ec0 0x7fdbec19b370 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc00d950 tx=0x7fdbdc00dc60 comp rx=0 tx=0).stop 2024-04-26T02:05:21.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.809+0000 7fdbf0c00700 1 -- 172.21.15.151:0/826138695 shutdown_connections 2024-04-26T02:05:21.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.809+0000 7fdbf0c00700 1 --2- 172.21.15.151:0/826138695 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fdbd00a3e30 0x7fdbd00a62f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:21.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.809+0000 7fdbf0c00700 1 --2- 172.21.15.151:0/826138695 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdbec109fa0 0x7fdbec19fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:21.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.809+0000 7fdbf0c00700 1 --2- 172.21.15.151:0/826138695 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdbec104ec0 0x7fdbec19b370 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:21.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.809+0000 7fdbf0c00700 1 --2- 172.21.15.151:0/826138695 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbec105870 0x7fdbec19b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:21.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.809+0000 7fdbf0c00700 1 -- 172.21.15.151:0/826138695 >> 172.21.15.151:0/826138695 conn(0x7fdbec100750 msgr2=0x7fdbec102a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:05:21.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.809+0000 7fdbf0c00700 1 -- 172.21.15.151:0/826138695 shutdown_connections 2024-04-26T02:05:21.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:21.809+0000 7fdbf0c00700 1 -- 172.21.15.151:0/826138695 wait complete. 2024-04-26T02:05:22.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:05:22 smithi151 ceph-mon[28057]: pgmap v369: 97 pgs: 97 active+clean; 174 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 905 KiB/s wr, 280 op/s 2024-04-26T02:05:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.763+0000 7fa3f7200640 1 -- 172.21.15.151:0/3400501304 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3f8102fb0 msgr2=0x7fa3f8103390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.763+0000 7fa3f7200640 1 --2- 172.21.15.151:0/3400501304 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3f8102fb0 0x7fa3f8103390 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8009be0 tx=0x7fa3e802a750 comp rx=0 tx=0).stop 2024-04-26T02:05:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.764+0000 7fa3f7200640 1 -- 172.21.15.151:0/3400501304 shutdown_connections 2024-04-26T02:05:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.764+0000 7fa3f7200640 1 --2- 172.21.15.151:0/3400501304 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa3f810c380 0x7fa3f810e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.764+0000 7fa3f7200640 1 --2- 172.21.15.151:0/3400501304 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3f81038d0 0x7fa3f810bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.764+0000 7fa3f7200640 1 --2- 172.21.15.151:0/3400501304 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3f8102fb0 0x7fa3f8103390 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.764+0000 7fa3f7200640 1 -- 172.21.15.151:0/3400501304 >> 172.21.15.151:0/3400501304 conn(0x7fa3f80fc770 msgr2=0x7fa3f80feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:05:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.764+0000 7fa3f7200640 1 -- 172.21.15.151:0/3400501304 shutdown_connections 2024-04-26T02:05:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.764+0000 7fa3f7200640 1 -- 172.21.15.151:0/3400501304 wait complete. 2024-04-26T02:05:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.765+0000 7fa3f7200640 1 Processor -- start 2024-04-26T02:05:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.765+0000 7fa3f7200640 1 -- start start 2024-04-26T02:05:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.765+0000 7fa3f7200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3f8102fb0 0x7fa3f81a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.765+0000 7fa3f7200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3f81038d0 0x7fa3f81a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.765+0000 7fa3f7200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa3f810c380 0x7fa3f819a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.765+0000 7fa3f7200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f8111fc0 con 0x7fa3f810c380 2024-04-26T02:05:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.765+0000 7fa3f7200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f8111e40 con 0x7fa3f8102fb0 2024-04-26T02:05:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.765+0000 7fa3f7200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f8112140 con 0x7fa3f81038d0 2024-04-26T02:05:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.766+0000 7fa3f6800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa3f810c380 0x7fa3f819a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.766+0000 7fa3f5e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3f8102fb0 0x7fa3f81a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.766+0000 7fa3f5e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3f8102fb0 0x7fa3f81a0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:41402/0 (socket says 172.21.15.151:41402) 2024-04-26T02:05:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.766+0000 7fa3f5e00640 1 -- 172.21.15.151:0/2934078500 learned_addr learned my addr 172.21.15.151:0/2934078500 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:05:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.766+0000 7fa3ed400640 1 --2- 172.21.15.151:0/2934078500 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3f81038d0 0x7fa3f81a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.766+0000 7fa3f5e00640 1 -- 172.21.15.151:0/2934078500 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3f81038d0 msgr2=0x7fa3f81a0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.766+0000 7fa3f5e00640 1 --2- 172.21.15.151:0/2934078500 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3f81038d0 0x7fa3f81a0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.766+0000 7fa3f5e00640 1 -- 172.21.15.151:0/2934078500 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa3f810c380 msgr2=0x7fa3f819a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.766+0000 7fa3f5e00640 1 --2- 172.21.15.151:0/2934078500 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa3f810c380 0x7fa3f819a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.766+0000 7fa3f5e00640 1 -- 172.21.15.151:0/2934078500 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3f819ad80 con 0x7fa3f8102fb0 2024-04-26T02:05:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.766+0000 7fa3f5e00640 1 --2- 172.21.15.151:0/2934078500 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3f8102fb0 0x7fa3f81a0400 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8034040 tx=0x7fa3e8003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:05:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.767+0000 7fa3ef400640 1 -- 172.21.15.151:0/2934078500 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e8041020 con 0x7fa3f8102fb0 2024-04-26T02:05:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.767+0000 7fa3ef400640 1 -- 172.21.15.151:0/2934078500 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3e80318f0 con 0x7fa3f8102fb0 2024-04-26T02:05:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.767+0000 7fa3f7200640 1 -- 172.21.15.151:0/2934078500 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3f819b010 con 0x7fa3f8102fb0 2024-04-26T02:05:22.768 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.767+0000 7fa3ef400640 1 -- 172.21.15.151:0/2934078500 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e8036040 con 0x7fa3f8102fb0 2024-04-26T02:05:22.768 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.767+0000 7fa3f7200640 1 -- 172.21.15.151:0/2934078500 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3f81a72c0 con 0x7fa3f8102fb0 2024-04-26T02:05:22.769 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.770+0000 7fa3ef400640 1 -- 172.21.15.151:0/2934078500 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7fa3e8044020 con 0x7fa3f8102fb0 2024-04-26T02:05:22.769 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.770+0000 7fa3ffa0d740 1 Processor -- start 2024-04-26T02:05:22.769 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.771+0000 7fa3ffa0d740 1 -- start start 2024-04-26T02:05:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.771+0000 7fa3ffa0d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558526f3a9b0 0x558526f3ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.771+0000 7fa3ffa0d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558526f3b2d0 0x558526f45650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.771+0000 7fa3ffa0d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558526f45b90 0x558526f47f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.771+0000 7fa3ffa0d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558526ea0900 con 0x558526f3a9b0 2024-04-26T02:05:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.771+0000 7fa3ffa0d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558526e46ef0 con 0x558526f45b90 2024-04-26T02:05:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.771+0000 7fa3ffa0d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558526e9c8c0 con 0x558526f3b2d0 2024-04-26T02:05:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.771+0000 7fa3f6800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558526f3a9b0 0x558526f3ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.771+0000 7fa3ed400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558526f45b90 0x558526f47f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.771+0000 7fa3f5e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558526f3b2d0 0x558526f45650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.771+0000 7fa3f6800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558526f3a9b0 0x558526f3ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:48348/0 (socket says 172.21.15.151:48348) 2024-04-26T02:05:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.771+0000 7fa3f6800640 1 -- 172.21.15.151:0/1848785273 learned_addr learned my addr 172.21.15.151:0/1848785273 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:05:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.772+0000 7fa3ed400640 1 -- 172.21.15.151:0/1848785273 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558526f3b2d0 msgr2=0x558526f45650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:05:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.772+0000 7fa3ed400640 1 --2- 172.21.15.151:0/1848785273 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558526f3b2d0 0x558526f45650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.772+0000 7fa3ed400640 1 -- 172.21.15.151:0/1848785273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558526f3a9b0 msgr2=0x558526f3ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:05:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.772+0000 7fa3ed400640 1 --2- 172.21.15.151:0/1848785273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558526f3a9b0 0x558526f3ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.772+0000 7fa3ed400640 1 -- 172.21.15.151:0/1848785273 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558526bb3210 con 0x558526f45b90 2024-04-26T02:05:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.772+0000 7fa3f6800640 1 --2- 172.21.15.151:0/1848785273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558526f3a9b0 0x558526f3ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:05:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.772+0000 7fa3ed400640 1 --2- 172.21.15.151:0/1848785273 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558526f45b90 0x558526f47f80 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa3f810e900 tx=0x7fa3e0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f170f08a15bfefaf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:05:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.772+0000 7fa3ef400640 1 --2- 172.21.15.151:0/2934078500 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fa3dc0a3d90 0x7fa3dc0a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.772+0000 7fa3e7000640 1 -- 172.21.15.151:0/1848785273 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e0004230 con 0x558526f45b90 2024-04-26T02:05:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.772+0000 7fa3e7000640 1 -- 172.21.15.151:0/1848785273 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3e0009e10 con 0x558526f45b90 2024-04-26T02:05:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.772+0000 7fa3e7000640 1 -- 172.21.15.151:0/1848785273 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e0004c60 con 0x558526f45b90 2024-04-26T02:05:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.772+0000 7fa3ef400640 1 -- 172.21.15.151:0/2934078500 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8411+0+0 (secure 0 0 0) 0x7fa3e80f1200 con 0x7fa3f8102fb0 2024-04-26T02:05:22.774 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.772+0000 7fa3f6800640 1 --2- 172.21.15.151:0/2934078500 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fa3dc0a3d90 0x7fa3dc0a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.773+0000 7fa3ffa0d740 1 -- 172.21.15.151:0/1848785273 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558526f45b90 msgr2=0x558526f47f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.773+0000 7fa3ffa0d740 1 --2- 172.21.15.151:0/1848785273 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558526f45b90 0x558526f47f80 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa3f810e900 tx=0x7fa3e0011810 comp rx=0 tx=0).stop 2024-04-26T02:05:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.773+0000 7fa3f6800640 1 --2- 172.21.15.151:0/2934078500 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fa3dc0a3d90 0x7fa3dc0a6250 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0009fd0 tx=0x7fa3f0009260 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:05:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.773+0000 7fa3ffa0d740 1 -- 172.21.15.151:0/1848785273 shutdown_connections 2024-04-26T02:05:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.773+0000 7fa3ffa0d740 1 --2- 172.21.15.151:0/1848785273 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558526f45b90 0x558526f47f80 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.773+0000 7fa3ffa0d740 1 --2- 172.21.15.151:0/1848785273 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558526f3b2d0 0x558526f45650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.773+0000 7fa3ffa0d740 1 --2- 172.21.15.151:0/1848785273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558526f3a9b0 0x558526f3ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.773+0000 7fa3ffa0d740 1 -- 172.21.15.151:0/1848785273 >> 172.21.15.151:0/1848785273 conn(0x558526f36ec0 msgr2=0x558526f372a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:05:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.773+0000 7fa3ffa0d740 1 -- 172.21.15.151:0/1848785273 shutdown_connections 2024-04-26T02:05:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.773+0000 7fa3ffa0d740 1 -- 172.21.15.151:0/1848785273 wait complete. 2024-04-26T02:05:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.774+0000 7fa3ffa0d740 1 -- start start 2024-04-26T02:05:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.774+0000 7fa3ffa0d740 1 Processor -- start 2024-04-26T02:05:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.774+0000 7fa3ffa0d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558526f3a9b0 0x558526f42a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.775+0000 7fa3ffa0d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558526f3b2d0 0x558526f42fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.775+0000 7fa3ffa0d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558526f45b90 0x558526f434f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.775+0000 7fa3ffa0d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558526f487c0 con 0x558526f45b90 2024-04-26T02:05:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.775+0000 7fa3ffa0d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558526e5a310 con 0x558526f3b2d0 2024-04-26T02:05:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.775+0000 7fa3ffa0d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558526f48940 con 0x558526f3a9b0 2024-04-26T02:05:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.775+0000 7fa3ffa0d740 10 client.0 _ll_register_callbacks cb 0x558526e28cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:05:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.775+0000 7fa3f6800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558526f45b90 0x558526f434f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:22.782 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.775+0000 7fa3ed400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558526f3b2d0 0x558526f42fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:22.782 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.775+0000 7fa3f5e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558526f3a9b0 0x558526f42a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:22.782 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.775+0000 7fa3ed400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558526f3b2d0 0x558526f42fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:41428/0 (socket says 172.21.15.151:41428) 2024-04-26T02:05:22.782 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.775+0000 7fa3ed400640 1 -- 172.21.15.151:0/2831552205 learned_addr learned my addr 172.21.15.151:0/2831552205 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:05:22.782 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.775+0000 7fa3f6800640 1 --2- 172.21.15.151:0/2831552205 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558526f45b90 0x558526f434f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:48356/0 (socket says 172.21.15.151:48356) 2024-04-26T02:05:22.782 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.775+0000 7fa3f5e00640 1 -- 172.21.15.151:0/2831552205 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558526f3b2d0 msgr2=0x558526f42fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:05:22.782 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.775+0000 7fa3f5e00640 1 --2- 172.21.15.151:0/2831552205 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558526f3b2d0 0x558526f42fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:22.783 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.775+0000 7fa3f5e00640 1 -- 172.21.15.151:0/2831552205 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558526f45b90 msgr2=0x558526f434f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:05:22.783 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.775+0000 7fa3f5e00640 1 --2- 172.21.15.151:0/2831552205 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558526f45b90 0x558526f434f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:22.783 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.775+0000 7fa3f5e00640 1 -- 172.21.15.151:0/2831552205 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558526c7f620 con 0x558526f3a9b0 2024-04-26T02:05:22.783 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.775+0000 7fa3ed400640 1 --2- 172.21.15.151:0/2831552205 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558526f3b2d0 0x558526f42fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:05:22.783 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.776+0000 7fa3f5e00640 1 --2- 172.21.15.151:0/2831552205 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558526f3a9b0 0x558526f42a70 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa3e80ee960 tx=0x7fa3e80f47e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d100d235d5ebf4a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:05:22.783 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.776+0000 7fa3db400640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-26T02:05:22.783 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.776+0000 7fa3db400640 1 -- 172.21.15.151:0/2831552205 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e80bcf80 con 0x558526f3a9b0 2024-04-26T02:05:22.784 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.776+0000 7fa3db400640 1 -- 172.21.15.151:0/2831552205 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8411+0+0 (secure 0 0 0) 0x7fa3e8030c00 con 0x558526f3a9b0 2024-04-26T02:05:22.784 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.776+0000 7fa3ffa0d740 1 -- 172.21.15.151:0/2831552205 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3e0009e10 con 0x558526f3a9b0 2024-04-26T02:05:22.784 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.776+0000 7fa3db400640 1 -- 172.21.15.151:0/2831552205 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa3e80bcb10 con 0x558526f3a9b0 2024-04-26T02:05:22.784 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.776+0000 7fa3ffa0d740 10 client.26572 fetch_fsmap learned FSMap version 39 2024-04-26T02:05:22.784 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.776+0000 7fa3ffa0d740 1 -- 172.21.15.151:0/2831552205 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x558526f48820 con 0x558526f3a9b0 2024-04-26T02:05:22.784 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.776+0000 7fa3db400640 1 -- 172.21.15.151:0/2831552205 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3e0009e10 con 0x558526f3a9b0 2024-04-26T02:05:22.784 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.777+0000 7fa3db400640 1 -- 172.21.15.151:0/2831552205 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fa3e80eebe0 con 0x558526f3a9b0 2024-04-26T02:05:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.777+0000 7fa3ffa0d740 10 client.26572 fetch_fsmap finished waiting for FSMap version 39 2024-04-26T02:05:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.777+0000 7fa3ffa0d740 10 client.26572 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:05:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.777+0000 7fa3ffa0d740 20 client.26572 populate_metadata read hostname 'smithi151' 2024-04-26T02:05:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.777+0000 7fa3ffa0d740 1 --2- 172.21.15.151:0/2831552205 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x558526f55e20 0x558526f58290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.777+0000 7fa3ffa0d740 4 client.26572 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:05:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.777+0000 7fa3ffa0d740 1 -- 172.21.15.151:0/2831552205 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558526eb48b0 con 0x558526f55e20 2024-04-26T02:05:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.777+0000 7fa3ed400640 1 --2- 172.21.15.151:0/2831552205 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x558526f55e20 0x558526f58290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:22.786 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.778+0000 7fa3ed400640 1 --2- 172.21.15.151:0/2831552205 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x558526f55e20 0x558526f58290 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1750c5b412849b99 server_cookie=5899395384a1ae1d in_seq=0 out_seq=0 2024-04-26T02:05:22.786 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.778+0000 7fa3db400640 10 client.26572 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:05:22.786 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.778+0000 7fa3db400640 1 -- 172.21.15.151:0/2831552205 <== mds.0 v2:172.21.15.165:6838/1969716254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558526eb48b0 con 0x558526f55e20 2024-04-26T02:05:22.786 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.778+0000 7fa3db400640 10 client.26572 handle_command_reply: tid=0 2024-04-26T02:05:22.786 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.779+0000 7fa3ffa0d740 1 client.26572 shutdown 2024-04-26T02:05:22.786 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.779+0000 7fa3ffa0d740 1 -- 172.21.15.151:0/2831552205 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558526f3a9b0 msgr2=0x558526f42a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:22.786 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.779+0000 7fa3ffa0d740 1 --2- 172.21.15.151:0/2831552205 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558526f3a9b0 0x558526f42a70 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa3e80ee960 tx=0x7fa3e80f47e0 comp rx=0 tx=0).stop 2024-04-26T02:05:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.779+0000 7fa3ffa0d740 1 -- 172.21.15.151:0/2831552205 shutdown_connections 2024-04-26T02:05:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.779+0000 7fa3ffa0d740 1 --2- 172.21.15.151:0/2831552205 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x558526f55e20 0x558526f58290 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.779+0000 7fa3ffa0d740 1 --2- 172.21.15.151:0/2831552205 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558526f45b90 0x558526f434f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.779+0000 7fa3ffa0d740 1 --2- 172.21.15.151:0/2831552205 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558526f3b2d0 0x558526f42fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.779+0000 7fa3db400640 0 client.26572 ms_handle_reset on v2:172.21.15.165:6838/1969716254 2024-04-26T02:05:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.779+0000 7fa3ffa0d740 1 --2- 172.21.15.151:0/2831552205 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558526f3a9b0 0x558526f42a70 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.779+0000 7fa3ffa0d740 1 -- 172.21.15.151:0/2831552205 >> 172.21.15.151:0/2831552205 conn(0x558526f36ec0 msgr2=0x558526f3cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:05:22.788 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.779+0000 7fa3ffa0d740 1 -- 172.21.15.151:0/2831552205 shutdown_connections 2024-04-26T02:05:22.788 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.779+0000 7fa3ffa0d740 1 -- 172.21.15.151:0/2831552205 wait complete. 2024-04-26T02:05:22.788 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.779+0000 7fa3ffa0d740 20 client.26572 trim_cache size 0 max 16384 2024-04-26T02:05:22.789 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:05:22 smithi172 ceph-mon[32785]: pgmap v369: 97 pgs: 97 active+clean; 174 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 905 KiB/s wr, 280 op/s 2024-04-26T02:05:22.790 INFO:tasks.quiescer:done joining Quiescers 2024-04-26T02:05:22.790 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-04-26T02:05:22.806 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi151.front.sepia.ceph.com 2024-04-26T02:05:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.798+0000 7fa3ffa0d740 1 Processor -- start 2024-04-26T02:05:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.798+0000 7fa3ffa0d740 1 -- start start 2024-04-26T02:05:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.798+0000 7fa3ffa0d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558526f3a9b0 0x558526f51680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.798+0000 7fa3ffa0d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558526f3b2d0 0x558526f5db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.798+0000 7fa3ffa0d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558526f45b90 0x558526f5e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.798+0000 7fa3ffa0d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f81b2300 con 0x558526f3b2d0 2024-04-26T02:05:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.798+0000 7fa3ffa0d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f81afba0 con 0x558526f45b90 2024-04-26T02:05:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.798+0000 7fa3ffa0d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f81ad9f0 con 0x558526f3a9b0 2024-04-26T02:05:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.799+0000 7fa3ed400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558526f3b2d0 0x558526f5db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.799+0000 7fa3ed400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558526f3b2d0 0x558526f5db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:48370/0 (socket says 172.21.15.151:48370) 2024-04-26T02:05:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.799+0000 7fa3ed400640 1 -- 172.21.15.151:0/4041230347 learned_addr learned my addr 172.21.15.151:0/4041230347 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:05:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.799+0000 7fa3f5e00640 1 --2- 172.21.15.151:0/4041230347 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558526f3a9b0 0x558526f51680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.799+0000 7fa3f6800640 1 --2- 172.21.15.151:0/4041230347 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558526f45b90 0x558526f5e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.799+0000 7fa3ed400640 1 -- 172.21.15.151:0/4041230347 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558526f3a9b0 msgr2=0x558526f51680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:05:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.799+0000 7fa3ed400640 1 --2- 172.21.15.151:0/4041230347 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558526f3a9b0 0x558526f51680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.799+0000 7fa3ed400640 1 -- 172.21.15.151:0/4041230347 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558526f45b90 msgr2=0x558526f5e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:05:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.799+0000 7fa3ed400640 1 --2- 172.21.15.151:0/4041230347 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558526f45b90 0x558526f5e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.799+0000 7fa3ed400640 1 -- 172.21.15.151:0/4041230347 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558526f3c040 con 0x558526f3b2d0 2024-04-26T02:05:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.799+0000 7fa3ed400640 1 --2- 172.21.15.151:0/4041230347 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558526f3b2d0 0x558526f5db70 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0004880 tx=0x7fa3e0005d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4cc14df91b023ee server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:05:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.800+0000 7fa3e6600640 1 -- 172.21.15.151:0/4041230347 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e0030070 con 0x558526f3b2d0 2024-04-26T02:05:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.800+0000 7fa3e6600640 1 -- 172.21.15.151:0/4041230347 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3e0009e10 con 0x558526f3b2d0 2024-04-26T02:05:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.800+0000 7fa3e6600640 1 -- 172.21.15.151:0/4041230347 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e001e040 con 0x558526f3b2d0 2024-04-26T02:05:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.801+0000 7fa3ffa0d740 1 -- 172.21.15.151:0/4041230347 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558526f3b2d0 msgr2=0x558526f5db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.801+0000 7fa3ffa0d740 1 --2- 172.21.15.151:0/4041230347 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558526f3b2d0 0x558526f5db70 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0004880 tx=0x7fa3e0005d70 comp rx=0 tx=0).stop 2024-04-26T02:05:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.801+0000 7fa3ffa0d740 1 -- 172.21.15.151:0/4041230347 shutdown_connections 2024-04-26T02:05:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.801+0000 7fa3ffa0d740 1 --2- 172.21.15.151:0/4041230347 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558526f45b90 0x558526f5e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.801+0000 7fa3ffa0d740 1 --2- 172.21.15.151:0/4041230347 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558526f3b2d0 0x558526f5db70 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.801+0000 7fa3ffa0d740 1 --2- 172.21.15.151:0/4041230347 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558526f3a9b0 0x558526f51680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.801+0000 7fa3ffa0d740 1 -- 172.21.15.151:0/4041230347 >> 172.21.15.151:0/4041230347 conn(0x558526f36ec0 msgr2=0x558526f3ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:05:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.801+0000 7fa3ffa0d740 1 -- 172.21.15.151:0/4041230347 shutdown_connections 2024-04-26T02:05:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.801+0000 7fa3ffa0d740 1 -- 172.21.15.151:0/4041230347 wait complete. 2024-04-26T02:05:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.801+0000 7fa3ffa0d740 1 -- start start 2024-04-26T02:05:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.801+0000 7fa3ffa0d740 1 Processor -- start 2024-04-26T02:05:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.802+0000 7fa3ffa0d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558526f3a9b0 0x558526f4ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.802+0000 7fa3ffa0d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558526f3b2d0 0x558526f4b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.802+0000 7fa3ffa0d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558526f45b90 0x558526fba3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.802+0000 7fa3ffa0d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f81ae060 con 0x558526f45b90 2024-04-26T02:05:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.802+0000 7fa3ffa0d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f81af4d0 con 0x558526f3a9b0 2024-04-26T02:05:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.802+0000 7fa3ffa0d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f81ae730 con 0x558526f3b2d0 2024-04-26T02:05:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.802+0000 7fa3ffa0d740 10 client.0 _ll_register_callbacks cb 0x7fa3e8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:05:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.802+0000 7fa3f5e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558526f3a9b0 0x558526f4ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.802+0000 7fa3f5e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558526f3a9b0 0x558526f4ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:41460/0 (socket says 172.21.15.151:41460) 2024-04-26T02:05:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.802+0000 7fa3f5e00640 1 -- 172.21.15.151:0/686326707 learned_addr learned my addr 172.21.15.151:0/686326707 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:05:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.802+0000 7fa3f6800640 1 --2- 172.21.15.151:0/686326707 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558526f45b90 0x558526fba3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.802+0000 7fa3ed400640 1 --2- 172.21.15.151:0/686326707 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558526f3b2d0 0x558526f4b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.802+0000 7fa3f5e00640 1 -- 172.21.15.151:0/686326707 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558526f3b2d0 msgr2=0x558526f4b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:05:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.802+0000 7fa3f5e00640 1 --2- 172.21.15.151:0/686326707 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558526f3b2d0 0x558526f4b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.802+0000 7fa3f5e00640 1 -- 172.21.15.151:0/686326707 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558526f45b90 msgr2=0x558526fba3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:05:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.802+0000 7fa3f5e00640 1 --2- 172.21.15.151:0/686326707 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558526f45b90 0x558526fba3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.802+0000 7fa3f5e00640 1 -- 172.21.15.151:0/686326707 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558526fba9d0 con 0x558526f3a9b0 2024-04-26T02:05:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.803+0000 7fa3f5e00640 1 --2- 172.21.15.151:0/686326707 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558526f3a9b0 0x558526f4ad90 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa3e80b9b80 tx=0x7fa3e8031670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a3ce6e4bfc3921 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:05:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.803+0000 7fa3db400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T02:05:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.803+0000 7fa3db400640 1 -- 172.21.15.151:0/686326707 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e80bccf0 con 0x558526f3a9b0 2024-04-26T02:05:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.803+0000 7fa3db400640 1 -- 172.21.15.151:0/686326707 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8411+0+0 (secure 0 0 0) 0x7fa3e80c12f0 con 0x558526f3a9b0 2024-04-26T02:05:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.803+0000 7fa3ffa0d740 1 -- 172.21.15.151:0/686326707 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3e0009e10 con 0x558526f3a9b0 2024-04-26T02:05:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.803+0000 7fa3db400640 1 -- 172.21.15.151:0/686326707 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa3e80bcf80 con 0x558526f3a9b0 2024-04-26T02:05:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.803+0000 7fa3ffa0d740 10 client.26552 fetch_fsmap learned FSMap version 39 2024-04-26T02:05:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.803+0000 7fa3ffa0d740 1 -- 172.21.15.151:0/686326707 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x558526fba9d0 con 0x558526f3a9b0 2024-04-26T02:05:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.803+0000 7fa3db400640 1 -- 172.21.15.151:0/686326707 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3e0009e10 con 0x558526f3a9b0 2024-04-26T02:05:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.803+0000 7fa3db400640 1 -- 172.21.15.151:0/686326707 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fa3e80c5990 con 0x558526f3a9b0 2024-04-26T02:05:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.804+0000 7fa3ffa0d740 10 client.26552 fetch_fsmap finished waiting for FSMap version 39 2024-04-26T02:05:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.804+0000 7fa3ffa0d740 10 client.26552 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:05:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.804+0000 7fa3ffa0d740 20 client.26552 populate_metadata read hostname 'smithi151' 2024-04-26T02:05:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.804+0000 7fa3ffa0d740 1 --2- 172.21.15.151:0/686326707 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x558526fbc020 0x558526fbe410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.804+0000 7fa3ffa0d740 4 client.26552 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T02:05:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.804+0000 7fa3ffa0d740 1 -- 172.21.15.151:0/686326707 --> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558526eae320 con 0x558526fbc020 2024-04-26T02:05:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.804+0000 7fa3ed400640 1 --2- 172.21.15.151:0/686326707 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x558526fbc020 0x558526fbe410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.804+0000 7fa3ed400640 1 --2- 172.21.15.151:0/686326707 >> [v2:172.21.15.165:6838/1969716254,v1:172.21.15.165:6839/1969716254] conn(0x558526fbc020 0x558526fbe410 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84b23a180f416698 server_cookie=db77992723f4ed87 in_seq=0 out_seq=0 2024-04-26T02:05:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi151.front.sepia.ceph.com.smithi151.stderr:2024-04-26T02:05:22.804+0000 7fa3db400640 10 client.26552 ms_handle_connect on v2:172.21.15.165:6838/1969716254 2024-04-26T02:05:22.820 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-26T02:05:22.830 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-26T02:05:22.830 INFO:teuthology.task.exec:Executing custom commands... 2024-04-26T02:05:22.830 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi151.front.sepia.ceph.com 2024-04-26T02:05:22.830 DEBUG:teuthology.orchestra.run.smithi151:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-26T02:05:22.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:05:22 smithi165 ceph-mon[32648]: pgmap v369: 97 pgs: 97 active+clean; 174 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 905 KiB/s wr, 280 op/s 2024-04-26T02:05:22.926 DEBUG:teuthology.orchestra.run.smithi151:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-26T02:05:23.002 DEBUG:teuthology.orchestra.run.smithi151:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-26T02:05:23.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.194+0000 7f9bee400640 1 -- 172.21.15.151:0/3025852752 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9be8100d60 msgr2=0x7f9be8101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:23.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.194+0000 7f9bee400640 1 --2- 172.21.15.151:0/3025852752 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9be8100d60 0x7f9be8101140 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9bd8009900 tx=0x7f9bd802a3f0 comp rx=0 tx=0).stop 2024-04-26T02:05:23.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.194+0000 7f9bee400640 1 -- 172.21.15.151:0/3025852752 shutdown_connections 2024-04-26T02:05:23.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.194+0000 7f9bee400640 1 --2- 172.21.15.151:0/3025852752 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9be810e910 0x7f9be8110d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:23.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.194+0000 7f9bee400640 1 --2- 172.21.15.151:0/3025852752 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9be8101680 0x7f9be810e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:23.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.194+0000 7f9bee400640 1 --2- 172.21.15.151:0/3025852752 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9be8100d60 0x7f9be8101140 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:23.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.194+0000 7f9bee400640 1 -- 172.21.15.151:0/3025852752 >> 172.21.15.151:0/3025852752 conn(0x7f9be8077fd0 msgr2=0x7f9be80783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:05:23.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.194+0000 7f9bee400640 1 -- 172.21.15.151:0/3025852752 shutdown_connections 2024-04-26T02:05:23.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.194+0000 7f9bee400640 1 -- 172.21.15.151:0/3025852752 wait complete. 2024-04-26T02:05:23.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.195+0000 7f9bee400640 1 Processor -- start 2024-04-26T02:05:23.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.195+0000 7f9bee400640 1 -- start start 2024-04-26T02:05:23.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.196+0000 7f9bee400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9be8100d60 0x7f9be81a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:23.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.196+0000 7f9bee400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9be8101680 0x7f9be81a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:23.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.196+0000 7f9bee400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9be810e910 0x7f9be819a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:23.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.196+0000 7f9bee400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9be81143d0 con 0x7f9be810e910 2024-04-26T02:05:23.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.196+0000 7f9bee400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9be8114250 con 0x7f9be8101680 2024-04-26T02:05:23.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.196+0000 7f9bee400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9be8114550 con 0x7f9be8100d60 2024-04-26T02:05:23.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.196+0000 7f9be7e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9be810e910 0x7f9be819a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:23.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.196+0000 7f9be7400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9be8100d60 0x7f9be81a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:23.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.196+0000 7f9be7e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9be810e910 0x7f9be819a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:48388/0 (socket says 172.21.15.151:48388) 2024-04-26T02:05:23.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.196+0000 7f9be7e00640 1 -- 172.21.15.151:0/2267863075 learned_addr learned my addr 172.21.15.151:0/2267863075 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:05:23.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.196+0000 7f9bdfe00640 1 --2- 172.21.15.151:0/2267863075 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9be8101680 0x7f9be81a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:23.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.197+0000 7f9be7e00640 1 -- 172.21.15.151:0/2267863075 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9be8100d60 msgr2=0x7f9be81a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:23.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.197+0000 7f9be7e00640 1 --2- 172.21.15.151:0/2267863075 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9be8100d60 0x7f9be81a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:23.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.197+0000 7f9be7e00640 1 -- 172.21.15.151:0/2267863075 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9be8101680 msgr2=0x7f9be81a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:23.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.197+0000 7f9be7e00640 1 --2- 172.21.15.151:0/2267863075 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9be8101680 0x7f9be81a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:23.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.197+0000 7f9be7e00640 1 -- 172.21.15.151:0/2267863075 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9be819af90 con 0x7f9be810e910 2024-04-26T02:05:23.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.197+0000 7f9be7e00640 1 --2- 172.21.15.151:0/2267863075 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9be810e910 0x7f9be819a6d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9be000e970 tx=0x7f9be000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:05:23.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.197+0000 7f9be4c00640 1 -- 172.21.15.151:0/2267863075 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be000ccc0 con 0x7f9be810e910 2024-04-26T02:05:23.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.197+0000 7f9bee400640 1 -- 172.21.15.151:0/2267863075 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9be819b280 con 0x7f9be810e910 2024-04-26T02:05:23.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.197+0000 7f9be4c00640 1 -- 172.21.15.151:0/2267863075 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9be00054d0 con 0x7f9be810e910 2024-04-26T02:05:23.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.197+0000 7f9be4c00640 1 -- 172.21.15.151:0/2267863075 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be0010430 con 0x7f9be810e910 2024-04-26T02:05:23.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.198+0000 7f9bee400640 1 -- 172.21.15.151:0/2267863075 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9be81a7590 con 0x7f9be810e910 2024-04-26T02:05:23.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.199+0000 7f9bee400640 1 -- 172.21.15.151:0/2267863075 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9be8069e00 con 0x7f9be810e910 2024-04-26T02:05:23.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.201+0000 7f9be4c00640 1 -- 172.21.15.151:0/2267863075 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f9be0005020 con 0x7f9be810e910 2024-04-26T02:05:23.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.202+0000 7f9be4c00640 1 --2- 172.21.15.151:0/2267863075 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f9bcc0a3dc0 0x7f9bcc0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:23.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.202+0000 7f9be4c00640 1 -- 172.21.15.151:0/2267863075 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8411+0+0 (secure 0 0 0) 0x7f9be00d0aa0 con 0x7f9be810e910 2024-04-26T02:05:23.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.202+0000 7f9be7400640 1 --2- 172.21.15.151:0/2267863075 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f9bcc0a3dc0 0x7f9bcc0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:23.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.203+0000 7f9be7400640 1 --2- 172.21.15.151:0/2267863075 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f9bcc0a3dc0 0x7f9bcc0a6280 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9bd8031440 tx=0x7f9bd8005b50 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:05:23.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.207+0000 7f9be4c00640 1 -- 172.21.15.151:0/2267863075 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9be009d150 con 0x7f9be810e910 2024-04-26T02:05:23.433 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.433+0000 7f9bee400640 1 -- 172.21.15.151:0/2267863075 --> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f9be819b4b0 con 0x7f9bcc0a3dc0 2024-04-26T02:05:23.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.446+0000 7f9be4c00640 1 -- 172.21.15.151:0/2267863075 <== mgr.25025 v2:172.21.15.151:6800/2427564083 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f9be819b4b0 con 0x7f9bcc0a3dc0 2024-04-26T02:05:23.445 INFO:teuthology.orchestra.run.smithi151.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-26T00:00:00", "created": "2024-04-26T01:53:38", "first": "2024-04-26T01:54:00", "last": "2024-04-26T02:05:00", "last_pruned": "2024-04-26T02:04:00", "created_count": 12, "pruned_count": 5, "active": true} 2024-04-26T02:05:23.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.448+0000 7f9bee400640 1 -- 172.21.15.151:0/2267863075 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f9bcc0a3dc0 msgr2=0x7f9bcc0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:23.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.448+0000 7f9bee400640 1 --2- 172.21.15.151:0/2267863075 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f9bcc0a3dc0 0x7f9bcc0a6280 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9bd8031440 tx=0x7f9bd8005b50 comp rx=0 tx=0).stop 2024-04-26T02:05:23.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.448+0000 7f9bee400640 1 -- 172.21.15.151:0/2267863075 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9be810e910 msgr2=0x7f9be819a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:23.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.448+0000 7f9bee400640 1 --2- 172.21.15.151:0/2267863075 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9be810e910 0x7f9be819a6d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9be000e970 tx=0x7f9be000ee30 comp rx=0 tx=0).stop 2024-04-26T02:05:23.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.449+0000 7f9bee400640 1 -- 172.21.15.151:0/2267863075 shutdown_connections 2024-04-26T02:05:23.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.449+0000 7f9bee400640 1 --2- 172.21.15.151:0/2267863075 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f9bcc0a3dc0 0x7f9bcc0a6280 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:23.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.449+0000 7f9bee400640 1 --2- 172.21.15.151:0/2267863075 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9be810e910 0x7f9be819a6d0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:23.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.449+0000 7f9bee400640 1 --2- 172.21.15.151:0/2267863075 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9be8101680 0x7f9be81a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:23.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.449+0000 7f9bee400640 1 --2- 172.21.15.151:0/2267863075 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9be8100d60 0x7f9be81a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:23.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.449+0000 7f9bee400640 1 -- 172.21.15.151:0/2267863075 >> 172.21.15.151:0/2267863075 conn(0x7f9be8077fd0 msgr2=0x7f9be810fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:05:23.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.449+0000 7f9bee400640 1 -- 172.21.15.151:0/2267863075 shutdown_connections 2024-04-26T02:05:23.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.449+0000 7f9bee400640 1 -- 172.21.15.151:0/2267863075 wait complete. 2024-04-26T02:05:23.467 DEBUG:teuthology.orchestra.run.smithi151:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-26T02:05:23.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.666+0000 7f587e800640 1 -- 172.21.15.151:0/127402393 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5878103b00 msgr2=0x7f587810bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:23.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.666+0000 7f587e800640 1 --2- 172.21.15.151:0/127402393 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5878103b00 0x7f587810bfe0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f5860009900 tx=0x7f586002a510 comp rx=0 tx=0).stop 2024-04-26T02:05:23.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.666+0000 7f587e800640 1 -- 172.21.15.151:0/127402393 shutdown_connections 2024-04-26T02:05:23.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.666+0000 7f587e800640 1 --2- 172.21.15.151:0/127402393 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f587810c520 0x7f587810e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:23.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.666+0000 7f587e800640 1 --2- 172.21.15.151:0/127402393 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5878103b00 0x7f587810bfe0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:23.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.666+0000 7f587e800640 1 --2- 172.21.15.151:0/127402393 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58781031e0 0x7f58781035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:23.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.666+0000 7f587e800640 1 -- 172.21.15.151:0/127402393 >> 172.21.15.151:0/127402393 conn(0x7f5878077fd0 msgr2=0x7f58780783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:05:23.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.666+0000 7f587e800640 1 -- 172.21.15.151:0/127402393 shutdown_connections 2024-04-26T02:05:23.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.666+0000 7f587e800640 1 -- 172.21.15.151:0/127402393 wait complete. 2024-04-26T02:05:23.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.667+0000 7f587e800640 1 Processor -- start 2024-04-26T02:05:23.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.668+0000 7f587e800640 1 -- start start 2024-04-26T02:05:23.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.668+0000 7f587e800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58781031e0 0x7f58781a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:23.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.668+0000 7f587e800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5878103b00 0x7f58781a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:23.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.668+0000 7f587e800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f587810c520 0x7f587819a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:23.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.668+0000 7f587e800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5878112190 con 0x7f5878103b00 2024-04-26T02:05:23.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.668+0000 7f587e800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5878112010 con 0x7f58781031e0 2024-04-26T02:05:23.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.668+0000 7f587e800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5878112310 con 0x7f587810c520 2024-04-26T02:05:23.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.668+0000 7f5876a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5878103b00 0x7f58781a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:23.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.668+0000 7f5876a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5878103b00 0x7f58781a0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:48404/0 (socket says 172.21.15.151:48404) 2024-04-26T02:05:23.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.668+0000 7f5876a00640 1 -- 172.21.15.151:0/1503179718 learned_addr learned my addr 172.21.15.151:0/1503179718 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:05:23.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.668+0000 7f5877400640 1 --2- 172.21.15.151:0/1503179718 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58781031e0 0x7f58781a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:23.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.668+0000 7f5877e00640 1 --2- 172.21.15.151:0/1503179718 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f587810c520 0x7f587819a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:23.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.669+0000 7f5876a00640 1 -- 172.21.15.151:0/1503179718 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f587810c520 msgr2=0x7f587819a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:23.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.669+0000 7f5876a00640 1 --2- 172.21.15.151:0/1503179718 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f587810c520 0x7f587819a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:23.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.669+0000 7f5876a00640 1 -- 172.21.15.151:0/1503179718 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58781031e0 msgr2=0x7f58781a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:23.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.669+0000 7f5876a00640 1 --2- 172.21.15.151:0/1503179718 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58781031e0 0x7f58781a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:23.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.669+0000 7f5876a00640 1 -- 172.21.15.151:0/1503179718 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f587819b010 con 0x7f5878103b00 2024-04-26T02:05:23.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.669+0000 7f5877400640 1 --2- 172.21.15.151:0/1503179718 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58781031e0 0x7f58781a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:05:23.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.669+0000 7f5876a00640 1 --2- 172.21.15.151:0/1503179718 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5878103b00 0x7f58781a0b80 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5860005940 tx=0x7f58600040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:05:23.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.669+0000 7f586fe00640 1 -- 172.21.15.151:0/1503179718 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586003f030 con 0x7f5878103b00 2024-04-26T02:05:23.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.669+0000 7f586fe00640 1 -- 172.21.15.151:0/1503179718 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58600044c0 con 0x7f5878103b00 2024-04-26T02:05:23.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.669+0000 7f587e800640 1 -- 172.21.15.151:0/1503179718 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f587819b2a0 con 0x7f5878103b00 2024-04-26T02:05:23.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.670+0000 7f586fe00640 1 -- 172.21.15.151:0/1503179718 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5860035070 con 0x7f5878103b00 2024-04-26T02:05:23.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.670+0000 7f587e800640 1 -- 172.21.15.151:0/1503179718 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58781a73e0 con 0x7f5878103b00 2024-04-26T02:05:23.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.671+0000 7f587e800640 1 -- 172.21.15.151:0/1503179718 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5878069e00 con 0x7f5878103b00 2024-04-26T02:05:23.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.673+0000 7f586fe00640 1 -- 172.21.15.151:0/1503179718 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f5860042030 con 0x7f5878103b00 2024-04-26T02:05:23.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.675+0000 7f586fe00640 1 --2- 172.21.15.151:0/1503179718 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f585c0a3dc0 0x7f585c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:23.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.675+0000 7f586fe00640 1 -- 172.21.15.151:0/1503179718 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8411+0+0 (secure 0 0 0) 0x7f58600f0af0 con 0x7f5878103b00 2024-04-26T02:05:23.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.675+0000 7f5877400640 1 --2- 172.21.15.151:0/1503179718 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f585c0a3dc0 0x7f585c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:23.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.676+0000 7f5877400640 1 --2- 172.21.15.151:0/1503179718 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f585c0a3dc0 0x7f585c0a6280 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5868005fd0 tx=0x7f5868009500 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:05:23.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.680+0000 7f586fe00640 1 -- 172.21.15.151:0/1503179718 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f58600bcb50 con 0x7f5878103b00 2024-04-26T02:05:23.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.906+0000 7f587e800640 1 -- 172.21.15.151:0/1503179718 --> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f5878108be0 con 0x7f585c0a3dc0 2024-04-26T02:05:23.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.919+0000 7f586fe00640 1 -- 172.21.15.151:0/1503179718 <== mgr.25025 v2:172.21.15.151:6800/2427564083 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f5878108be0 con 0x7f585c0a3dc0 2024-04-26T02:05:23.919 INFO:teuthology.orchestra.run.smithi151.stdout:/ 1m 6m3h 2024-04-26T02:05:23.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.922+0000 7f587e800640 1 -- 172.21.15.151:0/1503179718 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f585c0a3dc0 msgr2=0x7f585c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:23.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.922+0000 7f587e800640 1 --2- 172.21.15.151:0/1503179718 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f585c0a3dc0 0x7f585c0a6280 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5868005fd0 tx=0x7f5868009500 comp rx=0 tx=0).stop 2024-04-26T02:05:23.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.922+0000 7f587e800640 1 -- 172.21.15.151:0/1503179718 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5878103b00 msgr2=0x7f58781a0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:23.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.922+0000 7f587e800640 1 --2- 172.21.15.151:0/1503179718 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5878103b00 0x7f58781a0b80 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5860005940 tx=0x7f58600040c0 comp rx=0 tx=0).stop 2024-04-26T02:05:23.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.922+0000 7f587e800640 1 -- 172.21.15.151:0/1503179718 shutdown_connections 2024-04-26T02:05:23.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.922+0000 7f587e800640 1 --2- 172.21.15.151:0/1503179718 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f585c0a3dc0 0x7f585c0a6280 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:23.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.922+0000 7f587e800640 1 --2- 172.21.15.151:0/1503179718 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f587810c520 0x7f587819a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:23.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.922+0000 7f587e800640 1 --2- 172.21.15.151:0/1503179718 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5878103b00 0x7f58781a0b80 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:23.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.922+0000 7f587e800640 1 --2- 172.21.15.151:0/1503179718 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58781031e0 0x7f58781a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:23.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.922+0000 7f587e800640 1 -- 172.21.15.151:0/1503179718 >> 172.21.15.151:0/1503179718 conn(0x7f5878077fd0 msgr2=0x7f58781084d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:05:23.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.922+0000 7f587e800640 1 -- 172.21.15.151:0/1503179718 shutdown_connections 2024-04-26T02:05:23.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:23.923+0000 7f587e800640 1 -- 172.21.15.151:0/1503179718 wait complete. 2024-04-26T02:05:23.942 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-26T02:05:23.953 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T02:05:24.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.150+0000 7f0048e00640 1 -- 172.21.15.151:0/422352380 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00441041b0 msgr2=0x7f0044104590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:24.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.150+0000 7f0048e00640 1 --2- 172.21.15.151:0/422352380 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00441041b0 0x7f0044104590 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0034009ca0 tx=0x7f003402a8c0 comp rx=0 tx=0).stop 2024-04-26T02:05:24.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.150+0000 7f0048e00640 1 -- 172.21.15.151:0/422352380 shutdown_connections 2024-04-26T02:05:24.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.150+0000 7f0048e00640 1 --2- 172.21.15.151:0/422352380 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0044109280 0x7f0044110e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:24.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.150+0000 7f0048e00640 1 --2- 172.21.15.151:0/422352380 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0044104b60 0x7f0044108ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:24.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.150+0000 7f0048e00640 1 --2- 172.21.15.151:0/422352380 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00441041b0 0x7f0044104590 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:24.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.150+0000 7f0048e00640 1 -- 172.21.15.151:0/422352380 >> 172.21.15.151:0/422352380 conn(0x7f00440ff9f0 msgr2=0x7f0044101e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:05:24.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.150+0000 7f0048e00640 1 -- 172.21.15.151:0/422352380 shutdown_connections 2024-04-26T02:05:24.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.150+0000 7f0048e00640 1 -- 172.21.15.151:0/422352380 wait complete. 2024-04-26T02:05:24.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.151+0000 7f0048e00640 1 Processor -- start 2024-04-26T02:05:24.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.151+0000 7f0048e00640 1 -- start start 2024-04-26T02:05:24.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.152+0000 7f0048e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00441041b0 0x7f00441a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:24.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.152+0000 7f0048e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0044109280 0x7f00441a0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:24.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.152+0000 7f0048e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f004419a610 0x7f004419aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:24.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.152+0000 7f0048e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f00441141c0 con 0x7f004419a610 2024-04-26T02:05:24.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.152+0000 7f0048e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0044114040 con 0x7f0044109280 2024-04-26T02:05:24.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.152+0000 7f0048e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0044114340 con 0x7f00441041b0 2024-04-26T02:05:24.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.152+0000 7f0043400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00441041b0 0x7f00441a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:24.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.152+0000 7f0043400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00441041b0 0x7f00441a0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:51438/0 (socket says 172.21.15.151:51438) 2024-04-26T02:05:24.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.152+0000 7f0043400640 1 -- 172.21.15.151:0/1366029762 learned_addr learned my addr 172.21.15.151:0/1366029762 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:05:24.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.152+0000 7f0043e00640 1 --2- 172.21.15.151:0/1366029762 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f004419a610 0x7f004419aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:24.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.152+0000 7f003aa00640 1 --2- 172.21.15.151:0/1366029762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0044109280 0x7f00441a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:24.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.152+0000 7f0043400640 1 -- 172.21.15.151:0/1366029762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0044109280 msgr2=0x7f00441a0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:24.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.152+0000 7f0043400640 1 --2- 172.21.15.151:0/1366029762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0044109280 0x7f00441a0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:24.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.152+0000 7f0043400640 1 -- 172.21.15.151:0/1366029762 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f004419a610 msgr2=0x7f004419aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:24.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.152+0000 7f0043400640 1 --2- 172.21.15.151:0/1366029762 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f004419a610 0x7f004419aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:24.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.152+0000 7f0043400640 1 -- 172.21.15.151:0/1366029762 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f004419b130 con 0x7f00441041b0 2024-04-26T02:05:24.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.153+0000 7f0043400640 1 --2- 172.21.15.151:0/1366029762 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00441041b0 0x7f00441a0410 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f003402ade0 tx=0x7f0034031770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:05:24.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.153+0000 7f0040c00640 1 -- 172.21.15.151:0/1366029762 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0034003b70 con 0x7f00441041b0 2024-04-26T02:05:24.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.153+0000 7f0040c00640 1 -- 172.21.15.151:0/1366029762 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0034003d10 con 0x7f00441041b0 2024-04-26T02:05:24.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.153+0000 7f0048e00640 1 -- 172.21.15.151:0/1366029762 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f004419b3c0 con 0x7f00441041b0 2024-04-26T02:05:24.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.153+0000 7f0040c00640 1 -- 172.21.15.151:0/1366029762 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f003402fa50 con 0x7f00441041b0 2024-04-26T02:05:24.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.153+0000 7f0048e00640 1 -- 172.21.15.151:0/1366029762 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00441a7610 con 0x7f00441041b0 2024-04-26T02:05:24.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.154+0000 7f0048e00640 1 -- 172.21.15.151:0/1366029762 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00440690e0 con 0x7f00441041b0 2024-04-26T02:05:24.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.157+0000 7f0040c00640 1 -- 172.21.15.151:0/1366029762 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f0034004580 con 0x7f00441041b0 2024-04-26T02:05:24.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.158+0000 7f0040c00640 1 --2- 172.21.15.151:0/1366029762 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f00280a3da0 0x7f00280a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:24.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.158+0000 7f0040c00640 1 -- 172.21.15.151:0/1366029762 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8411+0+0 (secure 0 0 0) 0x7f00340f1c90 con 0x7f00441041b0 2024-04-26T02:05:24.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.158+0000 7f003aa00640 1 --2- 172.21.15.151:0/1366029762 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f00280a3da0 0x7f00280a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:24.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.159+0000 7f003aa00640 1 --2- 172.21.15.151:0/1366029762 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f00280a3da0 0x7f00280a6260 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f004410a860 tx=0x7f002c005f90 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:05:24.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.160+0000 7f0040c00640 1 -- 172.21.15.151:0/1366029762 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f00340be450 con 0x7f00441041b0 2024-04-26T02:05:24.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.364+0000 7f0048e00640 1 -- 172.21.15.151:0/1366029762 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f004419baf0 con 0x7f00441041b0 2024-04-26T02:05:24.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.366+0000 7f0040c00640 1 -- 172.21.15.151:0/1366029762 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) ==== 74+0+23008 (secure 0 0 0) 0x7f00340bdd50 con 0x7f00441041b0 2024-04-26T02:05:24.366 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T02:05:24.366 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":107,"fsid":"c0e49e64-036d-11ef-bc93-c7b262605968","created":"2024-04-26T01:40:07.483197+0000","modified":"2024-04-26T02:04:30.019077+0000","last_up_change":"2024-04-26T01:48:21.167691+0000","last_in_change":"2024-04-26T01:48:00.556990+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T01:44:11.411722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T01:49:31.756032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T01:49:36.529819+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":105,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ca27e814-a982-482c-ae1a-e596aac5c59a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":669937998},{"type":"v1","addr":"172.21.15.151:6803","nonce":669937998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":669937998},{"type":"v1","addr":"172.21.15.151:6805","nonce":669937998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":669937998},{"type":"v1","addr":"172.21.15.151:6809","nonce":669937998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":669937998},{"type":"v1","addr":"172.21.15.151:6807","nonce":669937998}]},"public_addr":"172.21.15.151:6803/669937998","cluster_addr":"172.21.15.151:6805/669937998","heartbeat_back_addr":"172.21.15.151:6809/669937998","heartbeat_front_addr":"172.21.15.151:6807/669937998","state":["exists","up"]},{"osd":1,"uuid":"e7d192c3-9624-4625-9ccb-a76c114d8aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":1356168171},{"type":"v1","addr":"172.21.15.165:6801","nonce":1356168171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":1356168171},{"type":"v1","addr":"172.21.15.165:6803","nonce":1356168171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":1356168171},{"type":"v1","addr":"172.21.15.165:6807","nonce":1356168171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":1356168171},{"type":"v1","addr":"172.21.15.165:6805","nonce":1356168171}]},"public_addr":"172.21.15.165:6801/1356168171","cluster_addr":"172.21.15.165:6803/1356168171","heartbeat_back_addr":"172.21.15.165:6807/1356168171","heartbeat_front_addr":"172.21.15.165:6805/1356168171","state":["exists","up"]},{"osd":2,"uuid":"0631df59-1ecc-48b4-b00a-48b0118dfc24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":237502635},{"type":"v1","addr":"172.21.15.172:6801","nonce":237502635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":237502635},{"type":"v1","addr":"172.21.15.172:6803","nonce":237502635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":237502635},{"type":"v1","addr":"172.21.15.172:6807","nonce":237502635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":237502635},{"type":"v1","addr":"172.21.15.172:6805","nonce":237502635}]},"public_addr":"172.21.15.172:6801/237502635","cluster_addr":"172.21.15.172:6803/237502635","heartbeat_back_addr":"172.21.15.172:6807/237502635","heartbeat_front_addr":"172.21.15.172:6805/237502635","state":["exists","up"]},{"osd":3,"uuid":"73076e49-2f8a-4a3d-99eb-7b50aff448aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3442279476},{"type":"v1","addr":"172.21.15.151:6811","nonce":3442279476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3442279476},{"type":"v1","addr":"172.21.15.151:6813","nonce":3442279476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":3442279476},{"type":"v1","addr":"172.21.15.151:6817","nonce":3442279476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3442279476},{"type":"v1","addr":"172.21.15.151:6815","nonce":3442279476}]},"public_addr":"172.21.15.151:6811/3442279476","cluster_addr":"172.21.15.151:6813/3442279476","heartbeat_back_addr":"172.21.15.151:6817/3442279476","heartbeat_front_addr":"172.21.15.151:6815/3442279476","state":["exists","up"]},{"osd":4,"uuid":"0230d615-548c-43f0-b8aa-7130f4cbbc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4029980723},{"type":"v1","addr":"172.21.15.165:6809","nonce":4029980723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4029980723},{"type":"v1","addr":"172.21.15.165:6811","nonce":4029980723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4029980723},{"type":"v1","addr":"172.21.15.165:6815","nonce":4029980723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4029980723},{"type":"v1","addr":"172.21.15.165:6813","nonce":4029980723}]},"public_addr":"172.21.15.165:6809/4029980723","cluster_addr":"172.21.15.165:6811/4029980723","heartbeat_back_addr":"172.21.15.165:6815/4029980723","heartbeat_front_addr":"172.21.15.165:6813/4029980723","state":["exists","up"]},{"osd":5,"uuid":"9739f44b-2605-47dd-82fb-d30034742bf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2216578687},{"type":"v1","addr":"172.21.15.172:6809","nonce":2216578687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2216578687},{"type":"v1","addr":"172.21.15.172:6811","nonce":2216578687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2216578687},{"type":"v1","addr":"172.21.15.172:6815","nonce":2216578687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2216578687},{"type":"v1","addr":"172.21.15.172:6813","nonce":2216578687}]},"public_addr":"172.21.15.172:6809/2216578687","cluster_addr":"172.21.15.172:6811/2216578687","heartbeat_back_addr":"172.21.15.172:6815/2216578687","heartbeat_front_addr":"172.21.15.172:6813/2216578687","state":["exists","up"]},{"osd":6,"uuid":"4290baaa-6160-49cb-a4e0-c96f070f2ef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1723252315},{"type":"v1","addr":"172.21.15.151:6819","nonce":1723252315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1723252315},{"type":"v1","addr":"172.21.15.151:6821","nonce":1723252315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":1723252315},{"type":"v1","addr":"172.21.15.151:6825","nonce":1723252315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1723252315},{"type":"v1","addr":"172.21.15.151:6823","nonce":1723252315}]},"public_addr":"172.21.15.151:6819/1723252315","cluster_addr":"172.21.15.151:6821/1723252315","heartbeat_back_addr":"172.21.15.151:6825/1723252315","heartbeat_front_addr":"172.21.15.151:6823/1723252315","state":["exists","up"]},{"osd":7,"uuid":"9f5315f2-036c-4e2f-92ed-3321a9e3c860","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":1977803781},{"type":"v1","addr":"172.21.15.165:6817","nonce":1977803781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":1977803781},{"type":"v1","addr":"172.21.15.165:6819","nonce":1977803781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":1977803781},{"type":"v1","addr":"172.21.15.165:6823","nonce":1977803781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":1977803781},{"type":"v1","addr":"172.21.15.165:6821","nonce":1977803781}]},"public_addr":"172.21.15.165:6817/1977803781","cluster_addr":"172.21.15.165:6819/1977803781","heartbeat_back_addr":"172.21.15.165:6823/1977803781","heartbeat_front_addr":"172.21.15.165:6821/1977803781","state":["exists","up"]},{"osd":8,"uuid":"8ddbe36d-cea7-45ef-a3ca-044a0e64ff85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":491199512},{"type":"v1","addr":"172.21.15.172:6817","nonce":491199512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":491199512},{"type":"v1","addr":"172.21.15.172:6819","nonce":491199512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":491199512},{"type":"v1","addr":"172.21.15.172:6823","nonce":491199512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":491199512},{"type":"v1","addr":"172.21.15.172:6821","nonce":491199512}]},"public_addr":"172.21.15.172:6817/491199512","cluster_addr":"172.21.15.172:6819/491199512","heartbeat_back_addr":"172.21.15.172:6823/491199512","heartbeat_front_addr":"172.21.15.172:6821/491199512","state":["exists","up"]},{"osd":9,"uuid":"40654ba0-1bd9-4b54-97ae-f1ed6c4785d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":1818923828},{"type":"v1","addr":"172.21.15.151:6827","nonce":1818923828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":1818923828},{"type":"v1","addr":"172.21.15.151:6829","nonce":1818923828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1818923828},{"type":"v1","addr":"172.21.15.151:6833","nonce":1818923828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":1818923828},{"type":"v1","addr":"172.21.15.151:6831","nonce":1818923828}]},"public_addr":"172.21.15.151:6827/1818923828","cluster_addr":"172.21.15.151:6829/1818923828","heartbeat_back_addr":"172.21.15.151:6833/1818923828","heartbeat_front_addr":"172.21.15.151:6831/1818923828","state":["exists","up"]},{"osd":10,"uuid":"19e75a1a-7bd0-41d9-b0cc-0fa4d1475dda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":2632621809},{"type":"v1","addr":"172.21.15.165:6825","nonce":2632621809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":2632621809},{"type":"v1","addr":"172.21.15.165:6827","nonce":2632621809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":2632621809},{"type":"v1","addr":"172.21.15.165:6831","nonce":2632621809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":2632621809},{"type":"v1","addr":"172.21.15.165:6829","nonce":2632621809}]},"public_addr":"172.21.15.165:6825/2632621809","cluster_addr":"172.21.15.165:6827/2632621809","heartbeat_back_addr":"172.21.15.165:6831/2632621809","heartbeat_front_addr":"172.21.15.165:6829/2632621809","state":["exists","up"]},{"osd":11,"uuid":"aa84e5f5-93a3-4375-8544-a1175f0eb06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":886604365},{"type":"v1","addr":"172.21.15.172:6825","nonce":886604365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":886604365},{"type":"v1","addr":"172.21.15.172:6827","nonce":886604365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":886604365},{"type":"v1","addr":"172.21.15.172:6831","nonce":886604365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":886604365},{"type":"v1","addr":"172.21.15.172:6829","nonce":886604365}]},"public_addr":"172.21.15.172:6825/886604365","cluster_addr":"172.21.15.172:6827/886604365","heartbeat_back_addr":"172.21.15.172:6831/886604365","heartbeat_front_addr":"172.21.15.172:6829/886604365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:43:18.296278+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:43:42.860620+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:44:07.073601+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:44:39.631760+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:45:04.771214+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:45:29.606674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:46:03.101851+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:46:28.351995+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:46:53.445435+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:47:27.391179+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:47:53.543638+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:48:18.687811+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.151:6800/2251256505":"2024-04-27T01:53:36.653535+0000","172.21.15.151:0/2357384001":"2024-04-27T01:53:36.653535+0000","172.21.15.151:6801/796314699":"2024-04-27T01:41:25.234322+0000","172.21.15.151:6800/796314699":"2024-04-27T01:41:25.234322+0000","172.21.15.151:0/4066781930":"2024-04-27T01:53:36.653535+0000","172.21.15.151:0/1173525554":"2024-04-27T01:41:25.234322+0000","172.21.15.151:0/1342496801":"2024-04-27T01:40:50.492554+0000","172.21.15.151:0/755039608":"2024-04-27T01:53:36.653535+0000","172.21.15.151:0/11665926":"2024-04-27T01:40:50.492554+0000","172.21.15.151:0/1765448952":"2024-04-27T01:40:50.492554+0000","172.21.15.151:0/3463483606":"2024-04-27T01:53:36.653535+0000","172.21.15.151:6801/2251256505":"2024-04-27T01:53:36.653535+0000","172.21.15.151:0/46007430":"2024-04-27T01:41:25.234322+0000","172.21.15.151:6800/244725060":"2024-04-27T01:40:50.492554+0000","172.21.15.151:0/3703862543":"2024-04-27T01:41:25.234322+0000","172.21.15.151:6801/244725060":"2024-04-27T01:40:50.492554+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T02:05:24.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.369+0000 7f0048e00640 1 -- 172.21.15.151:0/1366029762 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f00280a3da0 msgr2=0x7f00280a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:24.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.369+0000 7f0048e00640 1 --2- 172.21.15.151:0/1366029762 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f00280a3da0 0x7f00280a6260 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f004410a860 tx=0x7f002c005f90 comp rx=0 tx=0).stop 2024-04-26T02:05:24.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.369+0000 7f0048e00640 1 -- 172.21.15.151:0/1366029762 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00441041b0 msgr2=0x7f00441a0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:24.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.369+0000 7f0048e00640 1 --2- 172.21.15.151:0/1366029762 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00441041b0 0x7f00441a0410 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f003402ade0 tx=0x7f0034031770 comp rx=0 tx=0).stop 2024-04-26T02:05:24.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.369+0000 7f0048e00640 1 -- 172.21.15.151:0/1366029762 shutdown_connections 2024-04-26T02:05:24.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.369+0000 7f0048e00640 1 --2- 172.21.15.151:0/1366029762 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f00280a3da0 0x7f00280a6260 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:24.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.369+0000 7f0048e00640 1 --2- 172.21.15.151:0/1366029762 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f004419a610 0x7f004419aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:24.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.369+0000 7f0048e00640 1 --2- 172.21.15.151:0/1366029762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0044109280 0x7f00441a0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:24.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.369+0000 7f0048e00640 1 --2- 172.21.15.151:0/1366029762 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00441041b0 0x7f00441a0410 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:24.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.369+0000 7f0048e00640 1 -- 172.21.15.151:0/1366029762 >> 172.21.15.151:0/1366029762 conn(0x7f00440ff9f0 msgr2=0x7f0044100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:05:24.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.369+0000 7f0048e00640 1 -- 172.21.15.151:0/1366029762 shutdown_connections 2024-04-26T02:05:24.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.369+0000 7f0048e00640 1 -- 172.21.15.151:0/1366029762 wait complete. 2024-04-26T02:05:24.388 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T01:44:11.411722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T01:49:31.756032+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T01:49:36.529819+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 105, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T02:05:24.388 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T02:05:24.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.581+0000 7f982d800640 1 -- 172.21.15.151:0/2757164002 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9828108e90 msgr2=0x7f98281152b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:24.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.581+0000 7f982d800640 1 --2- 172.21.15.151:0/2757164002 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9828108e90 0x7f98281152b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f9820009ca0 tx=0x7f982002a950 comp rx=0 tx=0).stop 2024-04-26T02:05:24.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.582+0000 7f982d800640 1 -- 172.21.15.151:0/2757164002 shutdown_connections 2024-04-26T02:05:24.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.582+0000 7f982d800640 1 --2- 172.21.15.151:0/2757164002 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9828108e90 0x7f98281152b0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:24.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.582+0000 7f982d800640 1 --2- 172.21.15.151:0/2757164002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98281084f0 0x7f9828108950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:24.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.582+0000 7f982d800640 1 --2- 172.21.15.151:0/2757164002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f982810d3c0 0x7f982810d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:24.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.582+0000 7f982d800640 1 -- 172.21.15.151:0/2757164002 >> 172.21.15.151:0/2757164002 conn(0x7f9828077d70 msgr2=0x7f9828106d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:05:24.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.582+0000 7f982d800640 1 -- 172.21.15.151:0/2757164002 shutdown_connections 2024-04-26T02:05:24.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.582+0000 7f982d800640 1 -- 172.21.15.151:0/2757164002 wait complete. 2024-04-26T02:05:24.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.583+0000 7f982d800640 1 Processor -- start 2024-04-26T02:05:24.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.583+0000 7f982d800640 1 -- start start 2024-04-26T02:05:24.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.583+0000 7f982d800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f98281084f0 0x7f98281a4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:24.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.583+0000 7f982d800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9828108e90 0x7f98281a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:24.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.583+0000 7f982d800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f982810d3c0 0x7f982819ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:24.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.583+0000 7f982d800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9828118700 con 0x7f98281084f0 2024-04-26T02:05:24.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.583+0000 7f982d800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9828118580 con 0x7f9828108e90 2024-04-26T02:05:24.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.583+0000 7f982d800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9828118880 con 0x7f982810d3c0 2024-04-26T02:05:24.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.583+0000 7f9827e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f98281084f0 0x7f98281a4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:24.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.583+0000 7f982ce00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f982810d3c0 0x7f982819ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:24.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.583+0000 7f982ce00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f982810d3c0 0x7f982819ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:51458/0 (socket says 172.21.15.151:51458) 2024-04-26T02:05:24.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.583+0000 7f9827e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f98281084f0 0x7f98281a4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:48442/0 (socket says 172.21.15.151:48442) 2024-04-26T02:05:24.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.583+0000 7f982ce00640 1 -- 172.21.15.151:0/3215390181 learned_addr learned my addr 172.21.15.151:0/3215390181 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:05:24.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.583+0000 7f981f400640 1 --2- 172.21.15.151:0/3215390181 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9828108e90 0x7f98281a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:24.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.584+0000 7f982ce00640 1 -- 172.21.15.151:0/3215390181 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9828108e90 msgr2=0x7f98281a4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:24.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.584+0000 7f982ce00640 1 --2- 172.21.15.151:0/3215390181 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9828108e90 0x7f98281a4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:24.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.584+0000 7f982ce00640 1 -- 172.21.15.151:0/3215390181 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f98281084f0 msgr2=0x7f98281a4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:24.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.584+0000 7f982ce00640 1 --2- 172.21.15.151:0/3215390181 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f98281084f0 0x7f98281a4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:24.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.584+0000 7f982ce00640 1 -- 172.21.15.151:0/3215390181 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f982819f1a0 con 0x7f982810d3c0 2024-04-26T02:05:24.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.584+0000 7f9827e00640 1 --2- 172.21.15.151:0/3215390181 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f98281084f0 0x7f98281a4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:05:24.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.584+0000 7f982ce00640 1 --2- 172.21.15.151:0/3215390181 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f982810d3c0 0x7f982819ea20 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f982002af40 tx=0x7f9820003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:05:24.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.584+0000 7f9825600640 1 -- 172.21.15.151:0/3215390181 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9820034040 con 0x7f982810d3c0 2024-04-26T02:05:24.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.584+0000 7f9825600640 1 -- 172.21.15.151:0/3215390181 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f982002f870 con 0x7f982810d3c0 2024-04-26T02:05:24.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.584+0000 7f982d800640 1 -- 172.21.15.151:0/3215390181 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f982819f430 con 0x7f982810d3c0 2024-04-26T02:05:24.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.584+0000 7f9825600640 1 -- 172.21.15.151:0/3215390181 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9820004500 con 0x7f982810d3c0 2024-04-26T02:05:24.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.585+0000 7f982d800640 1 -- 172.21.15.151:0/3215390181 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98281ab6e0 con 0x7f982810d3c0 2024-04-26T02:05:24.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.585+0000 7f982d800640 1 -- 172.21.15.151:0/3215390181 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98280690e0 con 0x7f982810d3c0 2024-04-26T02:05:24.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.587+0000 7f9825600640 1 -- 172.21.15.151:0/3215390181 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f9820037020 con 0x7f982810d3c0 2024-04-26T02:05:24.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.589+0000 7f9825600640 1 --2- 172.21.15.151:0/3215390181 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f98080a3d20 0x7f98080a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:24.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.589+0000 7f9825600640 1 -- 172.21.15.151:0/3215390181 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8411+0+0 (secure 0 0 0) 0x7f98200f1420 con 0x7f982810d3c0 2024-04-26T02:05:24.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.589+0000 7f9827e00640 1 --2- 172.21.15.151:0/3215390181 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f98080a3d20 0x7f98080a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:24.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.590+0000 7f9827e00640 1 --2- 172.21.15.151:0/3215390181 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f98080a3d20 0x7f98080a61e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f98180095f0 tx=0x7f9818005f90 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:05:24.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.591+0000 7f9825600640 1 -- 172.21.15.151:0/3215390181 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f982002fa20 con 0x7f982810d3c0 2024-04-26T02:05:24.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.780+0000 7f982d800640 1 -- 172.21.15.151:0/3215390181 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9828110900 con 0x7f982810d3c0 2024-04-26T02:05:24.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.781+0000 7f9825600640 1 -- 172.21.15.151:0/3215390181 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v107) ==== 93+0+10 (secure 0 0 0) 0x7f98200bdbe0 con 0x7f982810d3c0 2024-04-26T02:05:24.781 INFO:teuthology.orchestra.run.smithi151.stdout:pg_num: 1 2024-04-26T02:05:24.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.784+0000 7f982d800640 1 -- 172.21.15.151:0/3215390181 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f98080a3d20 msgr2=0x7f98080a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:24.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.784+0000 7f982d800640 1 --2- 172.21.15.151:0/3215390181 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f98080a3d20 0x7f98080a61e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f98180095f0 tx=0x7f9818005f90 comp rx=0 tx=0).stop 2024-04-26T02:05:24.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.784+0000 7f982d800640 1 -- 172.21.15.151:0/3215390181 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f982810d3c0 msgr2=0x7f982819ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:24.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.784+0000 7f982d800640 1 --2- 172.21.15.151:0/3215390181 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f982810d3c0 0x7f982819ea20 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f982002af40 tx=0x7f9820003c10 comp rx=0 tx=0).stop 2024-04-26T02:05:24.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.784+0000 7f982d800640 1 -- 172.21.15.151:0/3215390181 shutdown_connections 2024-04-26T02:05:24.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.784+0000 7f982d800640 1 --2- 172.21.15.151:0/3215390181 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f98080a3d20 0x7f98080a61e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:24.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.784+0000 7f982d800640 1 --2- 172.21.15.151:0/3215390181 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f982810d3c0 0x7f982819ea20 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:24.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.784+0000 7f982d800640 1 --2- 172.21.15.151:0/3215390181 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9828108e90 0x7f98281a4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:24.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.784+0000 7f982d800640 1 --2- 172.21.15.151:0/3215390181 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f98281084f0 0x7f98281a4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:24.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.784+0000 7f982d800640 1 -- 172.21.15.151:0/3215390181 >> 172.21.15.151:0/3215390181 conn(0x7f9828077d70 msgr2=0x7f98280783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:05:24.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.785+0000 7f982d800640 1 -- 172.21.15.151:0/3215390181 shutdown_connections 2024-04-26T02:05:24.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:24.785+0000 7f982d800640 1 -- 172.21.15.151:0/3215390181 wait complete. 2024-04-26T02:05:24.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:05:24 smithi172 ceph-mon[32785]: pgmap v370: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 958 KiB/s wr, 302 op/s 2024-04-26T02:05:24.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:05:24 smithi172 ceph-mon[32785]: from='client.16440 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T02:05:24.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:05:24 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/1366029762' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T02:05:24.800 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:05:24 smithi151 ceph-mon[28057]: pgmap v370: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 958 KiB/s wr, 302 op/s 2024-04-26T02:05:24.800 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:05:24 smithi151 ceph-mon[28057]: from='client.16440 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T02:05:24.800 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:05:24 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/1366029762' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T02:05:24.802 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T02:05:24.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:05:24 smithi165 ceph-mon[32648]: pgmap v370: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 958 KiB/s wr, 302 op/s 2024-04-26T02:05:24.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:05:24 smithi165 ceph-mon[32648]: from='client.16440 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T02:05:24.847 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:05:24 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/1366029762' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T02:05:25.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.004+0000 7f1da6000640 1 -- 172.21.15.151:0/828134684 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1da010c440 msgr2=0x7f1da010e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:25.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.004+0000 7f1da6000640 1 --2- 172.21.15.151:0/828134684 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1da010c440 0x7f1da010e870 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1d98009be0 tx=0x7f1d9802a6c0 comp rx=0 tx=0).stop 2024-04-26T02:05:25.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.004+0000 7f1da6000640 1 -- 172.21.15.151:0/828134684 shutdown_connections 2024-04-26T02:05:25.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.004+0000 7f1da6000640 1 --2- 172.21.15.151:0/828134684 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1da010c440 0x7f1da010e870 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:25.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.004+0000 7f1da6000640 1 --2- 172.21.15.151:0/828134684 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1da0103990 0x7f1da010be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:25.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.004+0000 7f1da6000640 1 --2- 172.21.15.151:0/828134684 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1da0103070 0x7f1da0103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:25.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.004+0000 7f1da6000640 1 -- 172.21.15.151:0/828134684 >> 172.21.15.151:0/828134684 conn(0x7f1da00fc7b0 msgr2=0x7f1da00febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:05:25.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.005+0000 7f1da6000640 1 -- 172.21.15.151:0/828134684 shutdown_connections 2024-04-26T02:05:25.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.005+0000 7f1da6000640 1 -- 172.21.15.151:0/828134684 wait complete. 2024-04-26T02:05:25.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.005+0000 7f1da6000640 1 Processor -- start 2024-04-26T02:05:25.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.006+0000 7f1da6000640 1 -- start start 2024-04-26T02:05:25.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.006+0000 7f1da6000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1da0103070 0x7f1da01a0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:25.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.006+0000 7f1da6000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1da0103990 0x7f1da01a0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:25.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.006+0000 7f1da6000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1da010c440 0x7f1da019a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:25.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.006+0000 7f1da6000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1da0112090 con 0x7f1da010c440 2024-04-26T02:05:25.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.006+0000 7f1da6000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1da0111f10 con 0x7f1da0103070 2024-04-26T02:05:25.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.006+0000 7f1da6000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1da0112210 con 0x7f1da0103990 2024-04-26T02:05:25.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.006+0000 7f1da5600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1da010c440 0x7f1da019a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:25.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.006+0000 7f1da5600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1da010c440 0x7f1da019a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:48464/0 (socket says 172.21.15.151:48464) 2024-04-26T02:05:25.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.006+0000 7f1da5600640 1 -- 172.21.15.151:0/2298900740 learned_addr learned my addr 172.21.15.151:0/2298900740 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:05:25.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.006+0000 7f1da4c00640 1 --2- 172.21.15.151:0/2298900740 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1da0103070 0x7f1da01a0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:25.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.006+0000 7f1d9fe00640 1 --2- 172.21.15.151:0/2298900740 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1da0103990 0x7f1da01a0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:25.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.007+0000 7f1da5600640 1 -- 172.21.15.151:0/2298900740 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1da0103990 msgr2=0x7f1da01a0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:25.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.007+0000 7f1da5600640 1 --2- 172.21.15.151:0/2298900740 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1da0103990 0x7f1da01a0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:25.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.007+0000 7f1da5600640 1 -- 172.21.15.151:0/2298900740 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1da0103070 msgr2=0x7f1da01a0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:25.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.007+0000 7f1da5600640 1 --2- 172.21.15.151:0/2298900740 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1da0103070 0x7f1da01a0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:25.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.007+0000 7f1da5600640 1 -- 172.21.15.151:0/2298900740 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1da019add0 con 0x7f1da010c440 2024-04-26T02:05:25.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.007+0000 7f1da5600640 1 --2- 172.21.15.151:0/2298900740 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1da010c440 0x7f1da019a650 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1d98009b30 tx=0x7f1d98003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:05:25.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.007+0000 7f1d9d600640 1 -- 172.21.15.151:0/2298900740 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d98040020 con 0x7f1da010c440 2024-04-26T02:05:25.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.007+0000 7f1da6000640 1 -- 172.21.15.151:0/2298900740 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1da019b060 con 0x7f1da010c440 2024-04-26T02:05:25.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.007+0000 7f1d9d600640 1 -- 172.21.15.151:0/2298900740 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d980318f0 con 0x7f1da010c440 2024-04-26T02:05:25.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.008+0000 7f1d9d600640 1 -- 172.21.15.151:0/2298900740 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d98036040 con 0x7f1da010c440 2024-04-26T02:05:25.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.008+0000 7f1da6000640 1 -- 172.21.15.151:0/2298900740 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1da01a7310 con 0x7f1da010c440 2024-04-26T02:05:25.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.009+0000 7f1da6000640 1 -- 172.21.15.151:0/2298900740 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1da00690e0 con 0x7f1da010c440 2024-04-26T02:05:25.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.011+0000 7f1d9d600640 1 -- 172.21.15.151:0/2298900740 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f1d98043020 con 0x7f1da010c440 2024-04-26T02:05:25.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.012+0000 7f1d9d600640 1 --2- 172.21.15.151:0/2298900740 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f1d880a3d20 0x7f1d880a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:25.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.012+0000 7f1d9d600640 1 -- 172.21.15.151:0/2298900740 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8411+0+0 (secure 0 0 0) 0x7f1d980f16d0 con 0x7f1da010c440 2024-04-26T02:05:25.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.012+0000 7f1da4c00640 1 --2- 172.21.15.151:0/2298900740 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f1d880a3d20 0x7f1d880a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:25.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.013+0000 7f1da4c00640 1 --2- 172.21.15.151:0/2298900740 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f1d880a3d20 0x7f1d880a61e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1d94009bd0 tx=0x7f1d94005f90 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:05:25.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.015+0000 7f1d9d600640 1 -- 172.21.15.151:0/2298900740 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1d9804a020 con 0x7f1da010c440 2024-04-26T02:05:25.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.220+0000 7f1da6000640 1 -- 172.21.15.151:0/2298900740 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1da01a7540 con 0x7f1da010c440 2024-04-26T02:05:25.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.221+0000 7f1d9d600640 1 -- 172.21.15.151:0/2298900740 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v107) ==== 104+0+11 (secure 0 0 0) 0x7f1d980bdd80 con 0x7f1da010c440 2024-04-26T02:05:25.220 INFO:teuthology.orchestra.run.smithi151.stdout:pg_num: 32 2024-04-26T02:05:25.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.223+0000 7f1da6000640 1 -- 172.21.15.151:0/2298900740 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f1d880a3d20 msgr2=0x7f1d880a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:25.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.223+0000 7f1da6000640 1 --2- 172.21.15.151:0/2298900740 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f1d880a3d20 0x7f1d880a61e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1d94009bd0 tx=0x7f1d94005f90 comp rx=0 tx=0).stop 2024-04-26T02:05:25.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.223+0000 7f1da6000640 1 -- 172.21.15.151:0/2298900740 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1da010c440 msgr2=0x7f1da019a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:25.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.223+0000 7f1da6000640 1 --2- 172.21.15.151:0/2298900740 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1da010c440 0x7f1da019a650 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1d98009b30 tx=0x7f1d98003c10 comp rx=0 tx=0).stop 2024-04-26T02:05:25.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.223+0000 7f1da6000640 1 -- 172.21.15.151:0/2298900740 shutdown_connections 2024-04-26T02:05:25.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.224+0000 7f1da6000640 1 --2- 172.21.15.151:0/2298900740 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f1d880a3d20 0x7f1d880a61e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:25.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.224+0000 7f1da6000640 1 --2- 172.21.15.151:0/2298900740 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1da010c440 0x7f1da019a650 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:25.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.224+0000 7f1da6000640 1 --2- 172.21.15.151:0/2298900740 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1da0103990 0x7f1da01a0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:25.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.224+0000 7f1da6000640 1 --2- 172.21.15.151:0/2298900740 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1da0103070 0x7f1da01a0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:25.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.224+0000 7f1da6000640 1 -- 172.21.15.151:0/2298900740 >> 172.21.15.151:0/2298900740 conn(0x7f1da00fc7b0 msgr2=0x7f1da0100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:05:25.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.224+0000 7f1da6000640 1 -- 172.21.15.151:0/2298900740 shutdown_connections 2024-04-26T02:05:25.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.224+0000 7f1da6000640 1 -- 172.21.15.151:0/2298900740 wait complete. 2024-04-26T02:05:25.243 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T02:05:25.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.450+0000 7f8ce1600640 1 -- 172.21.15.151:0/2584980050 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8cdc108580 msgr2=0x7f8cdc108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:25.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.450+0000 7f8ce1600640 1 --2- 172.21.15.151:0/2584980050 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8cdc108580 0x7f8cdc108a00 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4009900 tx=0x7f8cc402a3f0 comp rx=0 tx=0).stop 2024-04-26T02:05:25.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.450+0000 7f8ce1600640 1 -- 172.21.15.151:0/2584980050 shutdown_connections 2024-04-26T02:05:25.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.450+0000 7f8ce1600640 1 --2- 172.21.15.151:0/2584980050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8cdc108f40 0x7f8cdc1153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:25.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.450+0000 7f8ce1600640 1 --2- 172.21.15.151:0/2584980050 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8cdc108580 0x7f8cdc108a00 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:25.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.450+0000 7f8ce1600640 1 --2- 172.21.15.151:0/2584980050 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8cdc10e5f0 0x7f8cdc10e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:25.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.450+0000 7f8ce1600640 1 -- 172.21.15.151:0/2584980050 >> 172.21.15.151:0/2584980050 conn(0x7f8cdc076490 msgr2=0x7f8cdc0788b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:05:25.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.450+0000 7f8ce1600640 1 -- 172.21.15.151:0/2584980050 shutdown_connections 2024-04-26T02:05:25.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.450+0000 7f8ce1600640 1 -- 172.21.15.151:0/2584980050 wait complete. 2024-04-26T02:05:25.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.451+0000 7f8ce1600640 1 Processor -- start 2024-04-26T02:05:25.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.452+0000 7f8ce1600640 1 -- start start 2024-04-26T02:05:25.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.452+0000 7f8ce1600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8cdc108580 0x7f8cdc19edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:25.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.452+0000 7f8ce1600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8cdc108f40 0x7f8cdc19f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:25.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.452+0000 7f8ce1600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8cdc10e5f0 0x7f8cdc1a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:25.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.452+0000 7f8ce1600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8cdc118760 con 0x7f8cdc10e5f0 2024-04-26T02:05:25.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.452+0000 7f8ce1600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8cdc1185e0 con 0x7f8cdc108580 2024-04-26T02:05:25.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.452+0000 7f8ce1600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8cdc1188e0 con 0x7f8cdc108f40 2024-04-26T02:05:25.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.453+0000 7f8cdb400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8cdc10e5f0 0x7f8cdc1a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:25.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.453+0000 7f8cdaa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8cdc108580 0x7f8cdc19edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:25.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.453+0000 7f8cd3e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8cdc108f40 0x7f8cdc19f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:25.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.453+0000 7f8cd3e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8cdc108f40 0x7f8cdc19f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:51478/0 (socket says 172.21.15.151:51478) 2024-04-26T02:05:25.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.453+0000 7f8cdb400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8cdc10e5f0 0x7f8cdc1a3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:48476/0 (socket says 172.21.15.151:48476) 2024-04-26T02:05:25.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.453+0000 7f8cdb400640 1 -- 172.21.15.151:0/2130153209 learned_addr learned my addr 172.21.15.151:0/2130153209 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:05:25.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.453+0000 7f8cdaa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8cdc108580 0x7f8cdc19edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:41572/0 (socket says 172.21.15.151:41572) 2024-04-26T02:05:25.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.453+0000 7f8cdb400640 1 -- 172.21.15.151:0/2130153209 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8cdc108f40 msgr2=0x7f8cdc19f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:25.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.453+0000 7f8cdb400640 1 --2- 172.21.15.151:0/2130153209 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8cdc108f40 0x7f8cdc19f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:25.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.453+0000 7f8cdb400640 1 -- 172.21.15.151:0/2130153209 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8cdc108580 msgr2=0x7f8cdc19edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:25.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.453+0000 7f8cdb400640 1 --2- 172.21.15.151:0/2130153209 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8cdc108580 0x7f8cdc19edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:25.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.453+0000 7f8cdb400640 1 -- 172.21.15.151:0/2130153209 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cdc1a3d60 con 0x7f8cdc10e5f0 2024-04-26T02:05:25.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.453+0000 7f8cdb400640 1 --2- 172.21.15.151:0/2130153209 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8cdc10e5f0 0x7f8cdc1a3630 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8cd400efd0 tx=0x7f8cd400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:05:25.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.454+0000 7f8cd3400640 1 -- 172.21.15.151:0/2130153209 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd4003b70 con 0x7f8cdc10e5f0 2024-04-26T02:05:25.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.454+0000 7f8cd3400640 1 -- 172.21.15.151:0/2130153209 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cd4003d10 con 0x7f8cdc10e5f0 2024-04-26T02:05:25.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.454+0000 7f8ce1600640 1 -- 172.21.15.151:0/2130153209 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cdc1a4050 con 0x7f8cdc10e5f0 2024-04-26T02:05:25.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.454+0000 7f8cd3400640 1 -- 172.21.15.151:0/2130153209 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd40108a0 con 0x7f8cdc10e5f0 2024-04-26T02:05:25.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.454+0000 7f8ce1600640 1 -- 172.21.15.151:0/2130153209 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cdc1ab8e0 con 0x7f8cdc10e5f0 2024-04-26T02:05:25.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.455+0000 7f8ce1600640 1 -- 172.21.15.151:0/2130153209 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cdc0691c0 con 0x7f8cdc10e5f0 2024-04-26T02:05:25.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.457+0000 7f8cd3400640 1 -- 172.21.15.151:0/2130153209 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f8cd4004020 con 0x7f8cdc10e5f0 2024-04-26T02:05:25.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.458+0000 7f8cd3400640 1 --2- 172.21.15.151:0/2130153209 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f8cc00a3da0 0x7f8cc00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:25.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.458+0000 7f8cd3400640 1 -- 172.21.15.151:0/2130153209 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8411+0+0 (secure 0 0 0) 0x7f8cd40d0b50 con 0x7f8cdc10e5f0 2024-04-26T02:05:25.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.458+0000 7f8cdaa00640 1 --2- 172.21.15.151:0/2130153209 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f8cc00a3da0 0x7f8cc00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:25.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.459+0000 7f8cdaa00640 1 --2- 172.21.15.151:0/2130153209 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f8cc00a3da0 0x7f8cc00a6260 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc009b50 tx=0x7f8ccc005f90 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:05:25.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.467+0000 7f8cd3400640 1 -- 172.21.15.151:0/2130153209 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8cd409d200 con 0x7f8cdc10e5f0 2024-04-26T02:05:25.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.660+0000 7f8ce1600640 1 -- 172.21.15.151:0/2130153209 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8cdc02d5c0 con 0x7f8cdc10e5f0 2024-04-26T02:05:25.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.661+0000 7f8cd3400640 1 -- 172.21.15.151:0/2130153209 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v107) ==== 100+0+11 (secure 0 0 0) 0x7f8cd409cb00 con 0x7f8cdc10e5f0 2024-04-26T02:05:25.661 INFO:teuthology.orchestra.run.smithi151.stdout:pg_num: 64 2024-04-26T02:05:25.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.663+0000 7f8ce1600640 1 -- 172.21.15.151:0/2130153209 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f8cc00a3da0 msgr2=0x7f8cc00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:25.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.663+0000 7f8ce1600640 1 --2- 172.21.15.151:0/2130153209 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f8cc00a3da0 0x7f8cc00a6260 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc009b50 tx=0x7f8ccc005f90 comp rx=0 tx=0).stop 2024-04-26T02:05:25.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.664+0000 7f8ce1600640 1 -- 172.21.15.151:0/2130153209 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8cdc10e5f0 msgr2=0x7f8cdc1a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:25.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.664+0000 7f8ce1600640 1 --2- 172.21.15.151:0/2130153209 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8cdc10e5f0 0x7f8cdc1a3630 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8cd400efd0 tx=0x7f8cd400c6a0 comp rx=0 tx=0).stop 2024-04-26T02:05:25.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.664+0000 7f8ce1600640 1 -- 172.21.15.151:0/2130153209 shutdown_connections 2024-04-26T02:05:25.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.664+0000 7f8ce1600640 1 --2- 172.21.15.151:0/2130153209 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f8cc00a3da0 0x7f8cc00a6260 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:25.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.664+0000 7f8ce1600640 1 --2- 172.21.15.151:0/2130153209 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8cdc10e5f0 0x7f8cdc1a3630 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:25.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.664+0000 7f8ce1600640 1 --2- 172.21.15.151:0/2130153209 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8cdc108f40 0x7f8cdc19f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:25.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.664+0000 7f8ce1600640 1 --2- 172.21.15.151:0/2130153209 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8cdc108580 0x7f8cdc19edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:25.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.664+0000 7f8ce1600640 1 -- 172.21.15.151:0/2130153209 >> 172.21.15.151:0/2130153209 conn(0x7f8cdc076490 msgr2=0x7f8cdc077f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:05:25.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.664+0000 7f8ce1600640 1 -- 172.21.15.151:0/2130153209 shutdown_connections 2024-04-26T02:05:25.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.664+0000 7f8ce1600640 1 -- 172.21.15.151:0/2130153209 wait complete. 2024-04-26T02:05:25.681 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-26T02:05:25.749 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:05:25 smithi151 ceph-mon[28057]: from='client.16446 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T02:05:25.749 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:05:25 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/3215390181' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T02:05:25.749 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:05:25 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/2298900740' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T02:05:25.789 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:05:25 smithi172 ceph-mon[32785]: from='client.16446 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T02:05:25.789 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:05:25 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/3215390181' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T02:05:25.789 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:05:25 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/2298900740' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T02:05:25.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:05:25 smithi165 ceph-mon[32648]: from='client.16446 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T02:05:25.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:05:25 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/3215390181' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T02:05:25.847 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:05:25 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/2298900740' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T02:05:25.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.880+0000 7ffade800640 1 -- 172.21.15.151:0/2779920713 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffad8109460 msgr2=0x7ffad8111030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:25.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.880+0000 7ffade800640 1 --2- 172.21.15.151:0/2779920713 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffad8109460 0x7ffad8111030 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ffad0009ca0 tx=0x7ffad002a8c0 comp rx=0 tx=0).stop 2024-04-26T02:05:25.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.880+0000 7ffade800640 1 -- 172.21.15.151:0/2779920713 shutdown_connections 2024-04-26T02:05:25.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.880+0000 7ffade800640 1 --2- 172.21.15.151:0/2779920713 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffad8109460 0x7ffad8111030 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:25.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.880+0000 7ffade800640 1 --2- 172.21.15.151:0/2779920713 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffad8104d40 0x7ffad8108d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:25.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.880+0000 7ffade800640 1 --2- 172.21.15.151:0/2779920713 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffad8104390 0x7ffad8104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:25.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.880+0000 7ffade800640 1 -- 172.21.15.151:0/2779920713 >> 172.21.15.151:0/2779920713 conn(0x7ffad80ffc50 msgr2=0x7ffad8102070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:05:25.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.880+0000 7ffade800640 1 -- 172.21.15.151:0/2779920713 shutdown_connections 2024-04-26T02:05:25.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.880+0000 7ffade800640 1 -- 172.21.15.151:0/2779920713 wait complete. 2024-04-26T02:05:25.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.881+0000 7ffade800640 1 Processor -- start 2024-04-26T02:05:25.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.881+0000 7ffade800640 1 -- start start 2024-04-26T02:05:25.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.882+0000 7ffade800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffad8104390 0x7ffad81a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:25.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.882+0000 7ffade800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffad8109460 0x7ffad81a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:25.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.882+0000 7ffade800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffad819a7a0 0x7ffad819ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:25.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.882+0000 7ffade800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffad8114400 con 0x7ffad8104390 2024-04-26T02:05:25.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.882+0000 7ffade800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ffad8114280 con 0x7ffad819a7a0 2024-04-26T02:05:25.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.882+0000 7ffade800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ffad8114580 con 0x7ffad8109460 2024-04-26T02:05:25.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.882+0000 7ffad7400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffad8104390 0x7ffad81a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:25.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.882+0000 7ffad6a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffad8109460 0x7ffad81a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:25.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.882+0000 7ffad6a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffad8109460 0x7ffad81a0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:51498/0 (socket says 172.21.15.151:51498) 2024-04-26T02:05:25.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.882+0000 7ffad7400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffad8104390 0x7ffad81a0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:48482/0 (socket says 172.21.15.151:48482) 2024-04-26T02:05:25.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.882+0000 7ffad6a00640 1 -- 172.21.15.151:0/2777285439 learned_addr learned my addr 172.21.15.151:0/2777285439 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:05:25.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.882+0000 7ffad7e00640 1 --2- 172.21.15.151:0/2777285439 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffad819a7a0 0x7ffad819ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:25.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.883+0000 7ffad6a00640 1 -- 172.21.15.151:0/2777285439 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffad819a7a0 msgr2=0x7ffad819ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:25.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.883+0000 7ffad6a00640 1 --2- 172.21.15.151:0/2777285439 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffad819a7a0 0x7ffad819ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:25.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.883+0000 7ffad6a00640 1 -- 172.21.15.151:0/2777285439 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffad8104390 msgr2=0x7ffad81a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:25.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.883+0000 7ffad6a00640 1 --2- 172.21.15.151:0/2777285439 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffad8104390 0x7ffad81a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:25.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.883+0000 7ffad6a00640 1 -- 172.21.15.151:0/2777285439 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffad819b3a0 con 0x7ffad8109460 2024-04-26T02:05:25.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.883+0000 7ffad7400640 1 --2- 172.21.15.151:0/2777285439 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffad8104390 0x7ffad81a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:05:25.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.883+0000 7ffad6a00640 1 --2- 172.21.15.151:0/2777285439 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffad8109460 0x7ffad81a0bd0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ffac400e9c0 tx=0x7ffac400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:05:25.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.883+0000 7ffacfe00640 1 -- 172.21.15.151:0/2777285439 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac400cd80 con 0x7ffad8109460 2024-04-26T02:05:25.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.883+0000 7ffade800640 1 -- 172.21.15.151:0/2777285439 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffad81a7430 con 0x7ffad8109460 2024-04-26T02:05:25.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.883+0000 7ffacfe00640 1 -- 172.21.15.151:0/2777285439 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffac40054d0 con 0x7ffad8109460 2024-04-26T02:05:25.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.883+0000 7ffacfe00640 1 -- 172.21.15.151:0/2777285439 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac4010430 con 0x7ffad8109460 2024-04-26T02:05:25.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.883+0000 7ffade800640 1 -- 172.21.15.151:0/2777285439 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffad81a7890 con 0x7ffad8109460 2024-04-26T02:05:25.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.884+0000 7ffade800640 1 -- 172.21.15.151:0/2777285439 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffad8105860 con 0x7ffad8109460 2024-04-26T02:05:25.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.887+0000 7ffacfe00640 1 -- 172.21.15.151:0/2777285439 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7ffac4004fd0 con 0x7ffad8109460 2024-04-26T02:05:25.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.889+0000 7ffacfe00640 1 --2- 172.21.15.151:0/2777285439 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7ffabc0a3da0 0x7ffabc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:25.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.889+0000 7ffacfe00640 1 -- 172.21.15.151:0/2777285439 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8411+0+0 (secure 0 0 0) 0x7ffac40d0a80 con 0x7ffad8109460 2024-04-26T02:05:25.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.889+0000 7ffad7400640 1 --2- 172.21.15.151:0/2777285439 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7ffabc0a3da0 0x7ffabc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:25.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.890+0000 7ffad7400640 1 --2- 172.21.15.151:0/2777285439 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7ffabc0a3da0 0x7ffabc0a6260 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ffac8009fc0 tx=0x7ffac8005f90 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:05:25.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:25.890+0000 7ffacfe00640 1 -- 172.21.15.151:0/2777285439 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffac409d170 con 0x7ffad8109460 2024-04-26T02:05:26.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:26.146+0000 7ffade800640 1 -- 172.21.15.151:0/2777285439 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7ffad8104d40 con 0x7ffad8109460 2024-04-26T02:05:26.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:26.158+0000 7ffacfe00640 1 -- 172.21.15.151:0/2777285439 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v27) ==== 81+0+746072 (secure 0 0 0) 0x7ffac409ca70 con 0x7ffad8109460 2024-04-26T02:05:26.170 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T02:05:26.171 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-26T02:05:26.171 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 27, 2024-04-26T02:05:26.171 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.171 INFO:teuthology.orchestra.run.smithi151.stdout: "active_gid": 25025, 2024-04-26T02:05:26.171 INFO:teuthology.orchestra.run.smithi151.stdout: "active_name": "x", 2024-04-26T02:05:26.171 INFO:teuthology.orchestra.run.smithi151.stdout: "active_addrs": { 2024-04-26T02:05:26.171 INFO:teuthology.orchestra.run.smithi151.stdout: "addrvec": [ 2024-04-26T02:05:26.171 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.171 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "v2", 2024-04-26T02:05:26.171 INFO:teuthology.orchestra.run.smithi151.stdout: "addr": "172.21.15.151:6800", 2024-04-26T02:05:26.171 INFO:teuthology.orchestra.run.smithi151.stdout: "nonce": 2427564083 2024-04-26T02:05:26.171 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.172 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.172 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "v1", 2024-04-26T02:05:26.172 INFO:teuthology.orchestra.run.smithi151.stdout: "addr": "172.21.15.151:6801", 2024-04-26T02:05:26.172 INFO:teuthology.orchestra.run.smithi151.stdout: "nonce": 2427564083 2024-04-26T02:05:26.172 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.172 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-26T02:05:26.172 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.172 INFO:teuthology.orchestra.run.smithi151.stdout: "active_addr": "172.21.15.151:6801/2427564083", 2024-04-26T02:05:26.172 INFO:teuthology.orchestra.run.smithi151.stdout: "active_change": "2024-04-26T01:53:36.654078+0000", 2024-04-26T02:05:26.172 INFO:teuthology.orchestra.run.smithi151.stdout: "active_mgr_features": 4540701547738038271, 2024-04-26T02:05:26.172 INFO:teuthology.orchestra.run.smithi151.stdout: "available": true, 2024-04-26T02:05:26.172 INFO:teuthology.orchestra.run.smithi151.stdout: "standbys": [ 2024-04-26T02:05:26.172 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.172 INFO:teuthology.orchestra.run.smithi151.stdout: "gid": 25022, 2024-04-26T02:05:26.173 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "y", 2024-04-26T02:05:26.173 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr_features": 4540701547738038271, 2024-04-26T02:05:26.173 INFO:teuthology.orchestra.run.smithi151.stdout: "available_modules": [ 2024-04-26T02:05:26.173 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.173 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "alerts", 2024-04-26T02:05:26.173 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.173 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.173 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.173 INFO:teuthology.orchestra.run.smithi151.stdout: "interval": { 2024-04-26T02:05:26.173 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "interval", 2024-04-26T02:05:26.173 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-04-26T02:05:26.173 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.173 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.173 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "60", 2024-04-26T02:05:26.174 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.174 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.174 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.174 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "How frequently to reexamine health status", 2024-04-26T02:05:26.174 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.174 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.174 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.174 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.174 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.174 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.174 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.174 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.174 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.175 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.175 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.175 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.175 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.175 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.175 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.175 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.175 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.175 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.175 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.175 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.175 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.175 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.175 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.175 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.176 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.176 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.176 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.176 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.176 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.176 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.176 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.176 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.176 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.176 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.176 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.176 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.176 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.177 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.177 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.177 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.177 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.177 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.177 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.177 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.177 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.177 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.177 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.177 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.177 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.177 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.177 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.178 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.178 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.178 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.178 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.178 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.178 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.178 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.178 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.178 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.178 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.178 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.178 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.178 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.178 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.178 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.178 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.179 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.179 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.179 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.179 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.179 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.179 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.179 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.179 INFO:teuthology.orchestra.run.smithi151.stdout: "smtp_destination": { 2024-04-26T02:05:26.179 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "smtp_destination", 2024-04-26T02:05:26.179 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.179 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.179 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.179 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.179 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.179 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.179 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.180 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Email address to send alerts to", 2024-04-26T02:05:26.180 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.180 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.180 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.180 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.180 INFO:teuthology.orchestra.run.smithi151.stdout: "smtp_from_name": { 2024-04-26T02:05:26.180 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "smtp_from_name", 2024-04-26T02:05:26.180 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.180 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.180 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.180 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "Ceph", 2024-04-26T02:05:26.180 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.180 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.180 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.180 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Email From: name", 2024-04-26T02:05:26.180 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.181 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.181 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.181 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.181 INFO:teuthology.orchestra.run.smithi151.stdout: "smtp_host": { 2024-04-26T02:05:26.181 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "smtp_host", 2024-04-26T02:05:26.181 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.181 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.181 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.181 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.181 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.181 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.181 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.181 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "SMTP server", 2024-04-26T02:05:26.181 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.182 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.182 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.182 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.182 INFO:teuthology.orchestra.run.smithi151.stdout: "smtp_password": { 2024-04-26T02:05:26.182 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "smtp_password", 2024-04-26T02:05:26.182 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.182 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.182 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.182 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.182 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.182 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.182 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.182 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Password to authenticate with", 2024-04-26T02:05:26.182 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.183 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.183 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.183 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.183 INFO:teuthology.orchestra.run.smithi151.stdout: "smtp_port": { 2024-04-26T02:05:26.183 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "smtp_port", 2024-04-26T02:05:26.183 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.183 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.183 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.183 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "465", 2024-04-26T02:05:26.183 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.183 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.183 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.183 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "SMTP port", 2024-04-26T02:05:26.183 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.183 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.183 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.184 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.184 INFO:teuthology.orchestra.run.smithi151.stdout: "smtp_sender": { 2024-04-26T02:05:26.184 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "smtp_sender", 2024-04-26T02:05:26.184 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.184 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.184 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.184 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.184 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.184 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.184 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.184 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "SMTP envelope sender", 2024-04-26T02:05:26.184 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.184 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.184 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.184 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.185 INFO:teuthology.orchestra.run.smithi151.stdout: "smtp_ssl": { 2024-04-26T02:05:26.185 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "smtp_ssl", 2024-04-26T02:05:26.185 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.185 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.185 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.185 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.185 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.185 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.185 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.185 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-26T02:05:26.185 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.185 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.185 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.185 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.185 INFO:teuthology.orchestra.run.smithi151.stdout: "smtp_user": { 2024-04-26T02:05:26.185 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "smtp_user", 2024-04-26T02:05:26.186 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.186 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.186 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.186 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.186 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.186 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.186 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.186 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "User to authenticate as", 2024-04-26T02:05:26.186 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.186 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.186 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.186 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.186 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.186 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.186 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.187 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "balancer", 2024-04-26T02:05:26.187 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.187 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.187 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.187 INFO:teuthology.orchestra.run.smithi151.stdout: "active": { 2024-04-26T02:05:26.187 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "active", 2024-04-26T02:05:26.187 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.187 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.187 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.187 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.187 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.187 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.187 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.187 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "automatically balance PGs across cluster", 2024-04-26T02:05:26.187 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.188 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.188 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.188 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.188 INFO:teuthology.orchestra.run.smithi151.stdout: "begin_time": { 2024-04-26T02:05:26.188 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "begin_time", 2024-04-26T02:05:26.188 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.188 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.188 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.188 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0000", 2024-04-26T02:05:26.188 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.188 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.188 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.188 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "beginning time of day to automatically balance", 2024-04-26T02:05:26.188 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T02:05:26.188 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.189 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.189 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.189 INFO:teuthology.orchestra.run.smithi151.stdout: "begin_weekday": { 2024-04-26T02:05:26.189 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "begin_weekday", 2024-04-26T02:05:26.189 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "uint", 2024-04-26T02:05:26.189 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.189 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.189 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-04-26T02:05:26.189 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "0", 2024-04-26T02:05:26.189 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "6", 2024-04-26T02:05:26.189 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.189 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-26T02:05:26.189 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T02:05:26.189 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.190 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.190 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.190 INFO:teuthology.orchestra.run.smithi151.stdout: "crush_compat_max_iterations": { 2024-04-26T02:05:26.190 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "crush_compat_max_iterations", 2024-04-26T02:05:26.190 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "uint", 2024-04-26T02:05:26.190 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.190 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.190 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "25", 2024-04-26T02:05:26.190 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "1", 2024-04-26T02:05:26.190 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "250", 2024-04-26T02:05:26.190 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.190 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-26T02:05:26.190 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.190 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.191 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.191 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.191 INFO:teuthology.orchestra.run.smithi151.stdout: "crush_compat_metrics": { 2024-04-26T02:05:26.191 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "crush_compat_metrics", 2024-04-26T02:05:26.191 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.191 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.191 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.191 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "pgs,objects,bytes", 2024-04-26T02:05:26.191 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.191 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.191 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.191 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-26T02:05:26.191 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-26T02:05:26.191 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.192 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.192 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.192 INFO:teuthology.orchestra.run.smithi151.stdout: "crush_compat_step": { 2024-04-26T02:05:26.192 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "crush_compat_step", 2024-04-26T02:05:26.192 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "float", 2024-04-26T02:05:26.192 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.192 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.192 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0.5", 2024-04-26T02:05:26.192 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "0.001", 2024-04-26T02:05:26.192 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "0.999", 2024-04-26T02:05:26.192 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.192 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "aggressiveness of optimization", 2024-04-26T02:05:26.192 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-26T02:05:26.192 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.192 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.193 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.193 INFO:teuthology.orchestra.run.smithi151.stdout: "end_time": { 2024-04-26T02:05:26.193 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "end_time", 2024-04-26T02:05:26.193 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.193 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.193 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.193 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "2359", 2024-04-26T02:05:26.193 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.193 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.193 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.193 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "ending time of day to automatically balance", 2024-04-26T02:05:26.193 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T02:05:26.193 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.193 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.193 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.193 INFO:teuthology.orchestra.run.smithi151.stdout: "end_weekday": { 2024-04-26T02:05:26.194 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "end_weekday", 2024-04-26T02:05:26.194 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "uint", 2024-04-26T02:05:26.194 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.194 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.194 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-04-26T02:05:26.194 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "0", 2024-04-26T02:05:26.194 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "6", 2024-04-26T02:05:26.194 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.194 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-26T02:05:26.194 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T02:05:26.194 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.194 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.194 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.194 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.194 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.195 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.195 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.195 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.195 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.195 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.195 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.195 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.195 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.195 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.195 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.195 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.195 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.195 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.195 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.195 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.195 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.197 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.197 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.197 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.197 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.197 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.197 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.197 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.197 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.197 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.197 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.197 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.198 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.198 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.198 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.198 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.198 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.198 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.198 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.198 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.198 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.198 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.198 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.198 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.198 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.198 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.199 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.199 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.199 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.199 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.199 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.199 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.199 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.199 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.199 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.199 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.199 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.199 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.199 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.199 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.199 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.199 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.200 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.200 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.200 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.200 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.200 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.200 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.200 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.200 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.200 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.200 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.200 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.200 INFO:teuthology.orchestra.run.smithi151.stdout: "min_score": { 2024-04-26T02:05:26.200 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "min_score", 2024-04-26T02:05:26.200 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "float", 2024-04-26T02:05:26.200 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.201 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.201 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-04-26T02:05:26.201 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.201 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.201 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.201 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-26T02:05:26.201 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.201 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.201 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.201 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.201 INFO:teuthology.orchestra.run.smithi151.stdout: "mode": { 2024-04-26T02:05:26.201 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "mode", 2024-04-26T02:05:26.201 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.201 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.202 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.202 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "upmap", 2024-04-26T02:05:26.202 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.202 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.202 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.202 INFO:teuthology.orchestra.run.smithi151.stdout: "crush-compat", 2024-04-26T02:05:26.202 INFO:teuthology.orchestra.run.smithi151.stdout: "none", 2024-04-26T02:05:26.202 INFO:teuthology.orchestra.run.smithi151.stdout: "read", 2024-04-26T02:05:26.202 INFO:teuthology.orchestra.run.smithi151.stdout: "upmap", 2024-04-26T02:05:26.202 INFO:teuthology.orchestra.run.smithi151.stdout: "upmap-read" 2024-04-26T02:05:26.202 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.202 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Balancer mode", 2024-04-26T02:05:26.202 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.202 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.203 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.203 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.203 INFO:teuthology.orchestra.run.smithi151.stdout: "pool_ids": { 2024-04-26T02:05:26.203 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "pool_ids", 2024-04-26T02:05:26.203 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.203 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.203 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.203 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.203 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.203 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.203 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.203 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-26T02:05:26.203 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.203 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.203 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.204 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.204 INFO:teuthology.orchestra.run.smithi151.stdout: "sleep_interval": { 2024-04-26T02:05:26.204 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sleep_interval", 2024-04-26T02:05:26.204 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-04-26T02:05:26.204 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.204 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.204 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "60", 2024-04-26T02:05:26.204 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.204 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.204 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.204 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-26T02:05:26.204 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.204 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.204 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.204 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.205 INFO:teuthology.orchestra.run.smithi151.stdout: "upmap_max_deviation": { 2024-04-26T02:05:26.205 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "upmap_max_deviation", 2024-04-26T02:05:26.205 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.205 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.205 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.205 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "5", 2024-04-26T02:05:26.205 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "1", 2024-04-26T02:05:26.205 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.205 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.205 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-26T02:05:26.205 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-26T02:05:26.205 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.205 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.205 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.206 INFO:teuthology.orchestra.run.smithi151.stdout: "upmap_max_optimizations": { 2024-04-26T02:05:26.206 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "upmap_max_optimizations", 2024-04-26T02:05:26.206 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "uint", 2024-04-26T02:05:26.206 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.206 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.206 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "10", 2024-04-26T02:05:26.206 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.206 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.206 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.206 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-26T02:05:26.206 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.206 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.207 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.207 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.207 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.207 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.207 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.207 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "cephadm", 2024-04-26T02:05:26.207 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.207 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.207 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.207 INFO:teuthology.orchestra.run.smithi151.stdout: "agent_down_multiplier": { 2024-04-26T02:05:26.207 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "agent_down_multiplier", 2024-04-26T02:05:26.207 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "float", 2024-04-26T02:05:26.207 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.207 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.208 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "3.0", 2024-04-26T02:05:26.208 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.208 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.208 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.208 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-26T02:05:26.208 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.208 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.208 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.208 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.208 INFO:teuthology.orchestra.run.smithi151.stdout: "agent_refresh_rate": { 2024-04-26T02:05:26.208 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "agent_refresh_rate", 2024-04-26T02:05:26.208 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-04-26T02:05:26.208 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.208 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.208 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "20", 2024-04-26T02:05:26.209 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.209 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.209 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.209 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-26T02:05:26.209 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.209 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.209 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.209 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.209 INFO:teuthology.orchestra.run.smithi151.stdout: "agent_starting_port": { 2024-04-26T02:05:26.209 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "agent_starting_port", 2024-04-26T02:05:26.209 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.209 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.209 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.210 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "4721", 2024-04-26T02:05:26.210 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.210 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.210 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.210 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-26T02:05:26.210 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.210 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.210 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.210 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.210 INFO:teuthology.orchestra.run.smithi151.stdout: "allow_ptrace": { 2024-04-26T02:05:26.210 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "allow_ptrace", 2024-04-26T02:05:26.210 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.211 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.211 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.211 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.211 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.211 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.211 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.211 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-26T02:05:26.211 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-26T02:05:26.211 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.211 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.211 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.211 INFO:teuthology.orchestra.run.smithi151.stdout: "autotune_interval": { 2024-04-26T02:05:26.211 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "autotune_interval", 2024-04-26T02:05:26.212 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-04-26T02:05:26.212 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.212 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.212 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "600", 2024-04-26T02:05:26.212 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.212 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.212 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.212 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-26T02:05:26.212 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.212 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.212 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.212 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.212 INFO:teuthology.orchestra.run.smithi151.stdout: "autotune_memory_target_ratio": { 2024-04-26T02:05:26.212 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "autotune_memory_target_ratio", 2024-04-26T02:05:26.213 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "float", 2024-04-26T02:05:26.213 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.213 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.213 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0.7", 2024-04-26T02:05:26.213 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.213 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.213 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.213 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-26T02:05:26.213 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.213 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.213 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.213 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.214 INFO:teuthology.orchestra.run.smithi151.stdout: "cephadm_log_destination": { 2024-04-26T02:05:26.214 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "cephadm_log_destination", 2024-04-26T02:05:26.214 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.214 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.214 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.214 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.214 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.214 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.214 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.214 INFO:teuthology.orchestra.run.smithi151.stdout: "file", 2024-04-26T02:05:26.215 INFO:teuthology.orchestra.run.smithi151.stdout: "file,syslog", 2024-04-26T02:05:26.215 INFO:teuthology.orchestra.run.smithi151.stdout: "syslog" 2024-04-26T02:05:26.215 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.215 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-26T02:05:26.215 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.215 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.215 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.215 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.215 INFO:teuthology.orchestra.run.smithi151.stdout: "cgroups_split": { 2024-04-26T02:05:26.215 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "cgroups_split", 2024-04-26T02:05:26.215 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.215 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.215 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.215 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.216 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.216 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.216 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.216 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-26T02:05:26.216 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.216 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.216 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.216 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.216 INFO:teuthology.orchestra.run.smithi151.stdout: "config_checks_enabled": { 2024-04-26T02:05:26.216 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "config_checks_enabled", 2024-04-26T02:05:26.216 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.216 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.216 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.216 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.216 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.217 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.217 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.217 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-26T02:05:26.217 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.217 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.217 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.217 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.217 INFO:teuthology.orchestra.run.smithi151.stdout: "config_dashboard": { 2024-04-26T02:05:26.217 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "config_dashboard", 2024-04-26T02:05:26.217 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.217 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.217 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.217 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.217 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.217 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.218 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.218 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-26T02:05:26.218 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.218 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.218 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.218 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.218 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_alertmanager": { 2024-04-26T02:05:26.218 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_alertmanager", 2024-04-26T02:05:26.218 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.218 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.218 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.218 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-26T02:05:26.218 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.219 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.219 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.219 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Prometheus container image", 2024-04-26T02:05:26.219 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.219 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.219 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.219 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.219 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_base": { 2024-04-26T02:05:26.219 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_base", 2024-04-26T02:05:26.219 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.219 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.219 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.219 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-26T02:05:26.219 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.219 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.220 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.220 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Container image name, without the tag", 2024-04-26T02:05:26.220 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.220 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.220 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.220 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.220 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_elasticsearch": { 2024-04-26T02:05:26.220 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_elasticsearch", 2024-04-26T02:05:26.220 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.220 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.220 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.220 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-26T02:05:26.220 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.220 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.220 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.221 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "elasticsearch container image", 2024-04-26T02:05:26.221 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.221 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.221 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.221 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.221 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_grafana": { 2024-04-26T02:05:26.221 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_grafana", 2024-04-26T02:05:26.221 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.221 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.221 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.221 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-26T02:05:26.221 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.221 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.221 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.221 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Prometheus container image", 2024-04-26T02:05:26.222 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.222 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.222 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.222 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.222 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_haproxy": { 2024-04-26T02:05:26.222 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_haproxy", 2024-04-26T02:05:26.222 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.222 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.222 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.222 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-26T02:05:26.222 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.222 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.222 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.222 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "HAproxy container image", 2024-04-26T02:05:26.223 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.223 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.223 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.223 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.223 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_jaeger_agent": { 2024-04-26T02:05:26.223 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_jaeger_agent", 2024-04-26T02:05:26.223 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.223 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.223 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.223 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-26T02:05:26.223 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.223 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.223 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.223 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Jaeger agent container image", 2024-04-26T02:05:26.223 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.224 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.224 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.224 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.224 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_jaeger_collector": { 2024-04-26T02:05:26.224 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_jaeger_collector", 2024-04-26T02:05:26.224 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.224 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.224 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.224 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-26T02:05:26.224 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.224 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.224 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.224 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Jaeger collector container image", 2024-04-26T02:05:26.224 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.225 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.225 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.225 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.225 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_jaeger_query": { 2024-04-26T02:05:26.225 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_jaeger_query", 2024-04-26T02:05:26.225 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.225 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.225 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.225 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-26T02:05:26.225 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.225 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.225 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.225 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Jaeger query container image", 2024-04-26T02:05:26.225 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.225 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.226 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.226 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.226 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_keepalived": { 2024-04-26T02:05:26.226 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_keepalived", 2024-04-26T02:05:26.226 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.226 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.226 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.226 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-26T02:05:26.226 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.226 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.226 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.226 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Keepalived container image", 2024-04-26T02:05:26.226 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.226 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.226 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.227 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.227 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_loki": { 2024-04-26T02:05:26.227 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_loki", 2024-04-26T02:05:26.227 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.227 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.227 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.227 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-26T02:05:26.227 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.227 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.227 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.227 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Loki container image", 2024-04-26T02:05:26.227 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.227 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.227 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.227 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.228 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_node_exporter": { 2024-04-26T02:05:26.228 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_node_exporter", 2024-04-26T02:05:26.228 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.228 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.228 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.228 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-26T02:05:26.228 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.228 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.228 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.228 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Prometheus container image", 2024-04-26T02:05:26.228 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.228 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.228 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.228 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.229 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_nvmeof": { 2024-04-26T02:05:26.229 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_nvmeof", 2024-04-26T02:05:26.229 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.229 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.229 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.229 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-26T02:05:26.229 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.229 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.229 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.229 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Nvme-of container image", 2024-04-26T02:05:26.229 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.229 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.229 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.229 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.229 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_prometheus": { 2024-04-26T02:05:26.230 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_prometheus", 2024-04-26T02:05:26.230 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.230 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.230 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.230 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-26T02:05:26.230 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.230 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.230 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.230 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Prometheus container image", 2024-04-26T02:05:26.230 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.230 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.230 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.230 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.231 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_promtail": { 2024-04-26T02:05:26.231 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_promtail", 2024-04-26T02:05:26.231 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.231 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.231 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.231 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-26T02:05:26.231 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.231 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.231 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.231 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Promtail container image", 2024-04-26T02:05:26.231 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.231 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.231 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.231 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.231 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_samba": { 2024-04-26T02:05:26.232 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_samba", 2024-04-26T02:05:26.232 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.232 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.232 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.232 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-26T02:05:26.232 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.232 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.232 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.232 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Samba/SMB container image", 2024-04-26T02:05:26.232 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.232 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.232 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.232 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.232 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_snmp_gateway": { 2024-04-26T02:05:26.232 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_snmp_gateway", 2024-04-26T02:05:26.232 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.233 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.233 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.233 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-26T02:05:26.233 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.233 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.233 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.233 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "SNMP Gateway container image", 2024-04-26T02:05:26.233 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.233 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.233 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.233 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.233 INFO:teuthology.orchestra.run.smithi151.stdout: "container_init": { 2024-04-26T02:05:26.233 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_init", 2024-04-26T02:05:26.233 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.233 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.234 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.234 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.234 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.234 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.234 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.234 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Run podman/docker with `--init`", 2024-04-26T02:05:26.234 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.234 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.234 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.234 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.234 INFO:teuthology.orchestra.run.smithi151.stdout: "daemon_cache_timeout": { 2024-04-26T02:05:26.234 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "daemon_cache_timeout", 2024-04-26T02:05:26.234 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-04-26T02:05:26.235 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.235 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.235 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "600", 2024-04-26T02:05:26.235 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.235 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.235 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.235 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-26T02:05:26.235 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.235 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.235 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.235 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.235 INFO:teuthology.orchestra.run.smithi151.stdout: "default_cephadm_command_timeout": { 2024-04-26T02:05:26.235 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "default_cephadm_command_timeout", 2024-04-26T02:05:26.235 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.235 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.235 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.236 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "900", 2024-04-26T02:05:26.236 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.236 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.236 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.236 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-26T02:05:26.236 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.236 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.236 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.236 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.236 INFO:teuthology.orchestra.run.smithi151.stdout: "default_registry": { 2024-04-26T02:05:26.236 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "default_registry", 2024-04-26T02:05:26.236 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.236 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.236 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.236 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "docker.io", 2024-04-26T02:05:26.237 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.237 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.237 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.237 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-26T02:05:26.237 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.237 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.237 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.237 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.237 INFO:teuthology.orchestra.run.smithi151.stdout: "device_cache_timeout": { 2024-04-26T02:05:26.237 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "device_cache_timeout", 2024-04-26T02:05:26.237 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-04-26T02:05:26.237 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.237 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.237 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "1800", 2024-04-26T02:05:26.238 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.238 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.238 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.238 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "seconds to cache device inventory", 2024-04-26T02:05:26.238 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.238 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.238 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.238 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.238 INFO:teuthology.orchestra.run.smithi151.stdout: "device_enhanced_scan": { 2024-04-26T02:05:26.238 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "device_enhanced_scan", 2024-04-26T02:05:26.238 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.238 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.238 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.239 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.239 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.239 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.239 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.239 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-26T02:05:26.239 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.239 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.239 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.239 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.239 INFO:teuthology.orchestra.run.smithi151.stdout: "facts_cache_timeout": { 2024-04-26T02:05:26.239 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "facts_cache_timeout", 2024-04-26T02:05:26.239 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-04-26T02:05:26.239 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.239 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.240 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "60", 2024-04-26T02:05:26.240 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.240 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.240 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.240 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "seconds to cache host facts data", 2024-04-26T02:05:26.240 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.240 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.240 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.240 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.240 INFO:teuthology.orchestra.run.smithi151.stdout: "grafana_dashboards_path": { 2024-04-26T02:05:26.240 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "grafana_dashboards_path", 2024-04-26T02:05:26.240 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.240 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.241 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.241 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-26T02:05:26.241 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.241 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.241 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.241 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-26T02:05:26.241 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.241 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.241 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.241 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.241 INFO:teuthology.orchestra.run.smithi151.stdout: "host_check_interval": { 2024-04-26T02:05:26.241 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "host_check_interval", 2024-04-26T02:05:26.241 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-04-26T02:05:26.242 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.242 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.242 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "600", 2024-04-26T02:05:26.242 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.242 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.242 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.242 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "how frequently to perform a host check", 2024-04-26T02:05:26.242 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.242 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.242 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.242 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.242 INFO:teuthology.orchestra.run.smithi151.stdout: "hw_monitoring": { 2024-04-26T02:05:26.242 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "hw_monitoring", 2024-04-26T02:05:26.243 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.243 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.243 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.243 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.243 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.243 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.243 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.243 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-26T02:05:26.243 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.243 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.243 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.243 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.243 INFO:teuthology.orchestra.run.smithi151.stdout: "inventory_list_all": { 2024-04-26T02:05:26.243 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "inventory_list_all", 2024-04-26T02:05:26.243 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.244 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.244 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.244 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.244 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.244 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.244 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.244 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-26T02:05:26.244 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.244 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.244 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.244 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.244 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.244 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.244 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.245 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.245 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.245 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.245 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.245 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.245 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.245 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.245 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.245 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.245 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.245 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.245 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.245 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.245 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.246 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.246 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.246 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.246 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.246 INFO:teuthology.orchestra.run.smithi151.stdout: "log_refresh_metadata": { 2024-04-26T02:05:26.246 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_refresh_metadata", 2024-04-26T02:05:26.246 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.246 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.246 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.246 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.246 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.246 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.246 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.247 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-26T02:05:26.247 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.247 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.247 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.247 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.247 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.247 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.247 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.247 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.247 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.247 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.247 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.247 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.247 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.248 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-26T02:05:26.248 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.248 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.248 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.248 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.248 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.248 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.248 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.248 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.248 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.248 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.248 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.248 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.248 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.249 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.249 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.249 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.249 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.249 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.249 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.249 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.249 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.249 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.249 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.249 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.249 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.249 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.249 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.250 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.250 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.250 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.250 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.250 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.250 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.250 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.250 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.250 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.250 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.250 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.250 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.250 INFO:teuthology.orchestra.run.smithi151.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-26T02:05:26.250 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-26T02:05:26.251 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.251 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.251 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.251 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.251 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.251 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.251 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.251 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-26T02:05:26.251 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.251 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.251 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.251 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.251 INFO:teuthology.orchestra.run.smithi151.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-26T02:05:26.251 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-26T02:05:26.251 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.252 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.252 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.252 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "*", 2024-04-26T02:05:26.252 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.252 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.252 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.252 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-26T02:05:26.252 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.252 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.252 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.252 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.252 INFO:teuthology.orchestra.run.smithi151.stdout: "max_count_per_host": { 2024-04-26T02:05:26.252 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "max_count_per_host", 2024-04-26T02:05:26.252 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.253 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.253 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.253 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "10", 2024-04-26T02:05:26.253 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.253 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.253 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.253 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "max number of daemons per service per host", 2024-04-26T02:05:26.253 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.253 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.253 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.253 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.253 INFO:teuthology.orchestra.run.smithi151.stdout: "max_osd_draining_count": { 2024-04-26T02:05:26.253 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "max_osd_draining_count", 2024-04-26T02:05:26.253 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.253 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.253 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.254 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "10", 2024-04-26T02:05:26.254 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.254 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.254 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.254 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-26T02:05:26.254 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.254 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.254 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.254 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.254 INFO:teuthology.orchestra.run.smithi151.stdout: "migration_current": { 2024-04-26T02:05:26.254 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "migration_current", 2024-04-26T02:05:26.254 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.254 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.254 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.254 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.255 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.255 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.255 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.255 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "internal - do not modify", 2024-04-26T02:05:26.255 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.255 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.255 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.255 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.255 INFO:teuthology.orchestra.run.smithi151.stdout: "mode": { 2024-04-26T02:05:26.255 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "mode", 2024-04-26T02:05:26.255 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.255 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.255 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.255 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "root", 2024-04-26T02:05:26.256 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.256 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.256 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.256 INFO:teuthology.orchestra.run.smithi151.stdout: "cephadm-package", 2024-04-26T02:05:26.256 INFO:teuthology.orchestra.run.smithi151.stdout: "root" 2024-04-26T02:05:26.256 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.256 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "mode for remote execution of cephadm", 2024-04-26T02:05:26.256 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.256 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.256 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.256 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.256 INFO:teuthology.orchestra.run.smithi151.stdout: "oob_default_addr": { 2024-04-26T02:05:26.256 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "oob_default_addr", 2024-04-26T02:05:26.256 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.257 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.257 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.257 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "169.254.1.1", 2024-04-26T02:05:26.257 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.257 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.257 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.257 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-26T02:05:26.257 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.257 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.257 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.257 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.257 INFO:teuthology.orchestra.run.smithi151.stdout: "prometheus_alerts_path": { 2024-04-26T02:05:26.257 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "prometheus_alerts_path", 2024-04-26T02:05:26.258 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.258 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.258 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.258 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-26T02:05:26.258 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.258 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.258 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.258 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-26T02:05:26.258 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.258 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.258 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.258 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.258 INFO:teuthology.orchestra.run.smithi151.stdout: "registry_insecure": { 2024-04-26T02:05:26.259 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "registry_insecure", 2024-04-26T02:05:26.259 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.259 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.259 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.259 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.259 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.259 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.259 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.259 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-26T02:05:26.259 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.259 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.259 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.259 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.260 INFO:teuthology.orchestra.run.smithi151.stdout: "registry_password": { 2024-04-26T02:05:26.260 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "registry_password", 2024-04-26T02:05:26.260 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.260 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.260 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.260 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.260 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.260 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.260 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.260 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-26T02:05:26.260 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.260 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.260 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.260 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.261 INFO:teuthology.orchestra.run.smithi151.stdout: "registry_url": { 2024-04-26T02:05:26.261 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "registry_url", 2024-04-26T02:05:26.261 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.261 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.261 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.261 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.261 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.261 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.261 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.261 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-26T02:05:26.261 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.261 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.261 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.261 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.262 INFO:teuthology.orchestra.run.smithi151.stdout: "registry_username": { 2024-04-26T02:05:26.262 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "registry_username", 2024-04-26T02:05:26.262 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.262 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.262 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.262 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.262 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.262 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.262 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.262 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-26T02:05:26.262 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.262 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.262 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.262 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.263 INFO:teuthology.orchestra.run.smithi151.stdout: "secure_monitoring_stack": { 2024-04-26T02:05:26.263 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "secure_monitoring_stack", 2024-04-26T02:05:26.263 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.263 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.263 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.263 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.263 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.263 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.263 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.263 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-26T02:05:26.263 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.263 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.263 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.263 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.263 INFO:teuthology.orchestra.run.smithi151.stdout: "service_discovery_port": { 2024-04-26T02:05:26.264 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "service_discovery_port", 2024-04-26T02:05:26.264 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.264 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.264 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.264 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "8765", 2024-04-26T02:05:26.264 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.264 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.264 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.264 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "cephadm service discovery port", 2024-04-26T02:05:26.264 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.264 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.264 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.264 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.264 INFO:teuthology.orchestra.run.smithi151.stdout: "ssh_config_file": { 2024-04-26T02:05:26.265 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ssh_config_file", 2024-04-26T02:05:26.265 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.265 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.265 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.265 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.265 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.265 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.265 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.265 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-26T02:05:26.265 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.265 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.265 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.265 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.265 INFO:teuthology.orchestra.run.smithi151.stdout: "use_agent": { 2024-04-26T02:05:26.265 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "use_agent", 2024-04-26T02:05:26.266 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.266 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.266 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.266 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.266 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.266 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.266 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.266 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-26T02:05:26.266 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.266 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.266 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.266 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.266 INFO:teuthology.orchestra.run.smithi151.stdout: "use_repo_digest": { 2024-04-26T02:05:26.266 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "use_repo_digest", 2024-04-26T02:05:26.267 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.267 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.267 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.267 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.267 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.267 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.267 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.267 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-26T02:05:26.267 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.267 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.267 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.267 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.267 INFO:teuthology.orchestra.run.smithi151.stdout: "warn_on_failed_host_check": { 2024-04-26T02:05:26.267 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "warn_on_failed_host_check", 2024-04-26T02:05:26.267 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.268 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.268 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.268 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.268 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.268 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.268 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.268 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "raise a health warning if the host check fails", 2024-04-26T02:05:26.268 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.268 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.268 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.268 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.268 INFO:teuthology.orchestra.run.smithi151.stdout: "warn_on_stray_daemons": { 2024-04-26T02:05:26.268 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "warn_on_stray_daemons", 2024-04-26T02:05:26.268 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.269 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.269 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.269 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.269 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.269 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.269 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.269 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-26T02:05:26.269 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.269 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.269 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.269 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.269 INFO:teuthology.orchestra.run.smithi151.stdout: "warn_on_stray_hosts": { 2024-04-26T02:05:26.269 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "warn_on_stray_hosts", 2024-04-26T02:05:26.269 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.269 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.270 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.270 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.270 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.270 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.270 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.270 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-26T02:05:26.270 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.270 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.270 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.270 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.270 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.270 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.270 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.271 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "crash", 2024-04-26T02:05:26.271 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.271 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.271 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.271 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.271 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.271 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.271 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.271 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.271 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.271 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.271 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.271 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.271 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.272 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.272 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.272 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.272 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.272 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.272 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.272 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.272 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.272 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.272 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.272 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.272 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.272 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.272 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.272 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.273 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.273 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.273 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.273 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.273 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.273 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.273 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.273 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.273 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.273 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.273 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.273 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.273 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.273 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.273 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.274 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.274 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.274 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.274 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.274 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.274 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.274 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.274 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.274 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.274 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.274 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.274 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.274 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.274 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.274 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.275 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.275 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.275 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.275 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.275 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.275 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.275 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.275 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.275 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.275 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.275 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.275 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.275 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.275 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.276 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.276 INFO:teuthology.orchestra.run.smithi151.stdout: "retain_interval": { 2024-04-26T02:05:26.276 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "retain_interval", 2024-04-26T02:05:26.276 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-04-26T02:05:26.276 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.276 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.276 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "31536000", 2024-04-26T02:05:26.276 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.276 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.276 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.276 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-26T02:05:26.276 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.276 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.276 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.276 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.277 INFO:teuthology.orchestra.run.smithi151.stdout: "warn_recent_interval": { 2024-04-26T02:05:26.277 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "warn_recent_interval", 2024-04-26T02:05:26.277 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-04-26T02:05:26.277 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.277 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.277 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "1209600", 2024-04-26T02:05:26.277 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.277 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.277 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.277 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-26T02:05:26.277 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.277 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.277 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.277 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.278 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.278 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.278 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.278 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "dashboard", 2024-04-26T02:05:26.278 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.278 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.278 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.278 INFO:teuthology.orchestra.run.smithi151.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-26T02:05:26.278 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-26T02:05:26.278 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.278 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.278 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.278 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "10", 2024-04-26T02:05:26.278 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.279 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.279 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.279 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.279 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.279 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.279 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.279 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.279 INFO:teuthology.orchestra.run.smithi151.stdout: "ALERTMANAGER_API_HOST": { 2024-04-26T02:05:26.279 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-26T02:05:26.279 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.279 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.279 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.279 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.279 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.279 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.280 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.280 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.280 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.280 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.280 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.280 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.280 INFO:teuthology.orchestra.run.smithi151.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-26T02:05:26.280 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-26T02:05:26.280 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.280 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.280 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.280 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.280 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.280 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.280 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.281 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.281 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.281 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.281 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.281 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.281 INFO:teuthology.orchestra.run.smithi151.stdout: "AUDIT_API_ENABLED": { 2024-04-26T02:05:26.281 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "AUDIT_API_ENABLED", 2024-04-26T02:05:26.281 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.281 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.281 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.281 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.281 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.281 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.281 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.282 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.282 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.282 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.282 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.282 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.282 INFO:teuthology.orchestra.run.smithi151.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-26T02:05:26.282 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-26T02:05:26.282 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.282 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.282 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.282 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.282 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.282 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.282 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.283 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.283 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.283 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.283 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.283 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.283 INFO:teuthology.orchestra.run.smithi151.stdout: "ENABLE_BROWSABLE_API": { 2024-04-26T02:05:26.283 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-26T02:05:26.283 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.283 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.283 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.283 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.283 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.283 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.283 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.283 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.284 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.284 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.284 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.284 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.284 INFO:teuthology.orchestra.run.smithi151.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-26T02:05:26.284 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-26T02:05:26.284 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.284 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.284 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.284 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.284 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.284 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.284 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.285 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.285 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.285 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.285 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.285 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.285 INFO:teuthology.orchestra.run.smithi151.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-26T02:05:26.285 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-26T02:05:26.285 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.285 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.285 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.285 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.285 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.286 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.286 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.286 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.286 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.286 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.286 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.286 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.286 INFO:teuthology.orchestra.run.smithi151.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-26T02:05:26.286 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-26T02:05:26.286 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.286 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.286 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.286 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.286 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.287 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.287 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.287 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.287 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.287 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.287 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.287 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.287 INFO:teuthology.orchestra.run.smithi151.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-26T02:05:26.287 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-26T02:05:26.287 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.287 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.287 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.287 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.287 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.288 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.288 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.288 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.288 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.288 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.288 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.288 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.288 INFO:teuthology.orchestra.run.smithi151.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-26T02:05:26.288 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-26T02:05:26.288 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.288 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.288 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.288 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.288 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.289 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.289 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.289 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.289 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.289 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.289 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.289 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.289 INFO:teuthology.orchestra.run.smithi151.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-26T02:05:26.289 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-26T02:05:26.289 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.289 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.289 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.289 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.289 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.290 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.290 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.290 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.290 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.290 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.290 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.290 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.290 INFO:teuthology.orchestra.run.smithi151.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-26T02:05:26.290 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-26T02:05:26.290 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.290 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.290 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.290 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.291 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.291 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.291 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.291 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.291 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.291 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.291 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.291 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.291 INFO:teuthology.orchestra.run.smithi151.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-26T02:05:26.291 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-26T02:05:26.291 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.291 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.291 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.291 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.291 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.292 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.292 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.292 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.292 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.292 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.292 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.292 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.292 INFO:teuthology.orchestra.run.smithi151.stdout: "GRAFANA_API_PASSWORD": { 2024-04-26T02:05:26.292 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-26T02:05:26.292 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.292 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.292 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.292 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "admin", 2024-04-26T02:05:26.292 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.293 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.293 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.293 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.293 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.293 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.293 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.293 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.293 INFO:teuthology.orchestra.run.smithi151.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-26T02:05:26.293 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-26T02:05:26.293 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.293 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.293 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.293 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.293 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.294 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.294 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.294 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.294 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.294 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.294 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.294 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.294 INFO:teuthology.orchestra.run.smithi151.stdout: "GRAFANA_API_URL": { 2024-04-26T02:05:26.294 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "GRAFANA_API_URL", 2024-04-26T02:05:26.294 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.294 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.294 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.294 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.294 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.295 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.295 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.295 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.295 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.295 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.295 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.295 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.295 INFO:teuthology.orchestra.run.smithi151.stdout: "GRAFANA_API_USERNAME": { 2024-04-26T02:05:26.295 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-26T02:05:26.295 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.295 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.295 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.295 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "admin", 2024-04-26T02:05:26.295 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.296 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.296 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.296 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.296 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.296 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.296 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.296 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.296 INFO:teuthology.orchestra.run.smithi151.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-26T02:05:26.296 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-26T02:05:26.296 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.296 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.296 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.296 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.296 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.296 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.297 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.297 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.297 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.297 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.297 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.297 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.297 INFO:teuthology.orchestra.run.smithi151.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-26T02:05:26.297 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-26T02:05:26.297 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.297 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.297 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.297 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.297 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.298 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.298 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.298 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.298 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.298 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.298 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.298 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.298 INFO:teuthology.orchestra.run.smithi151.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-26T02:05:26.298 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-26T02:05:26.298 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.298 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.298 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.298 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.299 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.299 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.299 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.299 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.299 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.299 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.299 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.299 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.299 INFO:teuthology.orchestra.run.smithi151.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-26T02:05:26.299 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-26T02:05:26.299 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.299 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.299 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.299 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.299 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.300 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.300 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.300 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.300 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.300 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.300 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.300 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.300 INFO:teuthology.orchestra.run.smithi151.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-26T02:05:26.300 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-26T02:05:26.300 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.300 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.300 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.300 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "[]", 2024-04-26T02:05:26.300 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.301 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.301 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.301 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.301 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.301 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.301 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.301 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.301 INFO:teuthology.orchestra.run.smithi151.stdout: "MULTICLUSTER_CONFIG": { 2024-04-26T02:05:26.301 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-26T02:05:26.301 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.301 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.301 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.302 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "{}", 2024-04-26T02:05:26.302 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.302 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.302 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.302 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.302 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.302 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.302 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.302 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.302 INFO:teuthology.orchestra.run.smithi151.stdout: "PROMETHEUS_API_HOST": { 2024-04-26T02:05:26.302 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-26T02:05:26.302 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.302 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.302 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.303 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.303 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.303 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.303 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.303 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.303 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.303 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.303 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.303 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.303 INFO:teuthology.orchestra.run.smithi151.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-26T02:05:26.303 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-26T02:05:26.303 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.303 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.304 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.304 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.304 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.304 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.304 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.304 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.304 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.304 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.304 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.304 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.304 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-26T02:05:26.304 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-26T02:05:26.304 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.304 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.305 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.305 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.305 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.305 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.305 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.305 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.305 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.305 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.305 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.305 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.305 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-26T02:05:26.305 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-26T02:05:26.305 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.306 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.306 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.306 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.306 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.306 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.306 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.306 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.306 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.306 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.306 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.306 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.306 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-26T02:05:26.306 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-26T02:05:26.307 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.307 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.307 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.307 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.307 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.307 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.307 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.307 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.307 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.307 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.307 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.307 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.307 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-26T02:05:26.308 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-26T02:05:26.308 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.308 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.308 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.308 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.308 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.308 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.308 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.308 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.308 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.308 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.308 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.308 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.308 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-26T02:05:26.309 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-26T02:05:26.309 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.309 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.309 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.309 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.309 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.309 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.309 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.309 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.309 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.309 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.309 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.309 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.309 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-26T02:05:26.310 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-26T02:05:26.310 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.310 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.310 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.310 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.310 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.310 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.310 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.310 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.310 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.310 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.310 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.310 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.311 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-26T02:05:26.311 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-26T02:05:26.311 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.311 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.311 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.311 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.311 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.311 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.311 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.311 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.311 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.311 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.311 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.311 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.311 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_ENABLED": { 2024-04-26T02:05:26.312 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-26T02:05:26.312 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.312 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.312 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.312 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.312 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.312 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.312 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.312 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.312 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.312 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.312 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.312 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.312 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-26T02:05:26.312 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-26T02:05:26.313 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.313 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.313 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.313 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-26T02:05:26.313 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.313 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.313 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.313 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.313 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.313 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.313 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.313 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.313 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-26T02:05:26.313 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-26T02:05:26.313 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.314 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.314 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.314 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "10", 2024-04-26T02:05:26.314 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.314 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.314 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.314 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.314 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.314 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.314 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.314 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.314 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-26T02:05:26.314 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-26T02:05:26.314 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.315 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.315 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.315 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "8", 2024-04-26T02:05:26.315 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.315 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.315 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.315 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.315 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.315 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.315 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.315 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.315 INFO:teuthology.orchestra.run.smithi151.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-26T02:05:26.315 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-26T02:05:26.315 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.316 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.316 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.316 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "45", 2024-04-26T02:05:26.316 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.316 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.316 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.316 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.316 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.316 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.316 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.316 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.316 INFO:teuthology.orchestra.run.smithi151.stdout: "RGW_API_ACCESS_KEY": { 2024-04-26T02:05:26.316 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-26T02:05:26.316 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.317 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.317 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.317 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.317 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.317 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.317 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.317 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.317 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.317 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.317 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.317 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.317 INFO:teuthology.orchestra.run.smithi151.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-26T02:05:26.317 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-26T02:05:26.318 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.318 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.318 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.318 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "admin", 2024-04-26T02:05:26.318 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.318 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.318 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.318 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.318 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.318 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.318 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.318 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.318 INFO:teuthology.orchestra.run.smithi151.stdout: "RGW_API_SECRET_KEY": { 2024-04-26T02:05:26.318 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-26T02:05:26.319 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.319 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.319 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.319 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.319 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.319 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.319 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.319 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.319 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.319 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.319 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.319 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.319 INFO:teuthology.orchestra.run.smithi151.stdout: "RGW_API_SSL_VERIFY": { 2024-04-26T02:05:26.319 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-26T02:05:26.320 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.320 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.320 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.320 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.320 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.320 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.320 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.320 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.320 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.320 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.320 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.320 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.320 INFO:teuthology.orchestra.run.smithi151.stdout: "UNSAFE_TLS_v1_2": { 2024-04-26T02:05:26.320 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-26T02:05:26.321 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.321 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.321 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.321 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.321 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.321 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.321 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.321 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.321 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.321 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.321 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.321 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.321 INFO:teuthology.orchestra.run.smithi151.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-26T02:05:26.322 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-26T02:05:26.322 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.322 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.322 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.322 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-04-26T02:05:26.322 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.322 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.322 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.322 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.322 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.322 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.322 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.322 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.323 INFO:teuthology.orchestra.run.smithi151.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-26T02:05:26.323 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-26T02:05:26.323 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.323 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.323 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.323 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "10", 2024-04-26T02:05:26.323 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.323 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.323 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.323 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.323 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.323 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.323 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.323 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.323 INFO:teuthology.orchestra.run.smithi151.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-26T02:05:26.324 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-26T02:05:26.324 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.324 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.324 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.324 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "5", 2024-04-26T02:05:26.324 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.324 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.324 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.324 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.324 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.324 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.324 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.324 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.324 INFO:teuthology.orchestra.run.smithi151.stdout: "cross_origin_url": { 2024-04-26T02:05:26.325 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "cross_origin_url", 2024-04-26T02:05:26.325 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.325 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.325 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.325 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.325 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.325 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.325 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.325 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.325 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.325 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.325 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.325 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.326 INFO:teuthology.orchestra.run.smithi151.stdout: "crt_file": { 2024-04-26T02:05:26.326 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "crt_file", 2024-04-26T02:05:26.326 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.326 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.326 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.326 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.326 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.326 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.326 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.326 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.326 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.326 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.327 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.327 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.327 INFO:teuthology.orchestra.run.smithi151.stdout: "debug": { 2024-04-26T02:05:26.327 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "debug", 2024-04-26T02:05:26.327 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.327 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.327 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.327 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.327 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.327 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.327 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.327 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Enable/disable debug options", 2024-04-26T02:05:26.327 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.327 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.328 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.328 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.328 INFO:teuthology.orchestra.run.smithi151.stdout: "jwt_token_ttl": { 2024-04-26T02:05:26.328 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "jwt_token_ttl", 2024-04-26T02:05:26.328 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.328 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.328 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.328 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "28800", 2024-04-26T02:05:26.328 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.328 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.328 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.328 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.328 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.329 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.329 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.329 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.329 INFO:teuthology.orchestra.run.smithi151.stdout: "key_file": { 2024-04-26T02:05:26.329 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "key_file", 2024-04-26T02:05:26.329 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.329 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.329 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.329 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.329 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.329 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.329 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.329 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.330 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.330 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.330 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.330 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.330 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.330 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.330 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.330 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.330 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.330 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.330 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.330 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.330 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.330 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.331 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.331 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.331 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.331 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.331 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.331 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.331 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.331 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.331 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.331 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.331 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.331 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.331 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.332 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.332 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.332 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.332 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.332 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.332 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.332 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.332 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.332 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.332 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.332 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.332 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.332 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.333 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.333 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.333 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.333 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.333 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.333 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.333 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.333 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.333 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.333 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.333 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.333 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.333 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.333 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.333 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.334 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.334 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.334 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.334 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.334 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.334 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.334 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.334 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.334 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.334 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.334 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.334 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.334 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.334 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.335 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.335 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.335 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.335 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.335 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.335 INFO:teuthology.orchestra.run.smithi151.stdout: "motd": { 2024-04-26T02:05:26.335 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "motd", 2024-04-26T02:05:26.335 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.335 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.335 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.335 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.335 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.335 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.335 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.336 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "The message of the day", 2024-04-26T02:05:26.336 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.336 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.336 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.336 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.336 INFO:teuthology.orchestra.run.smithi151.stdout: "redirect_resolve_ip_addr": { 2024-04-26T02:05:26.336 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "redirect_resolve_ip_addr", 2024-04-26T02:05:26.336 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.336 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.336 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.336 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.336 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.336 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.336 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.336 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.337 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.337 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.337 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.337 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.337 INFO:teuthology.orchestra.run.smithi151.stdout: "server_addr": { 2024-04-26T02:05:26.337 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "server_addr", 2024-04-26T02:05:26.337 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.337 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.337 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.337 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "::", 2024-04-26T02:05:26.337 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.337 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.337 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.337 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.338 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.338 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.338 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.338 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.338 INFO:teuthology.orchestra.run.smithi151.stdout: "server_port": { 2024-04-26T02:05:26.338 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "server_port", 2024-04-26T02:05:26.338 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.338 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.338 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.338 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "8080", 2024-04-26T02:05:26.338 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.338 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.338 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.339 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.339 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.339 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.339 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.339 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.339 INFO:teuthology.orchestra.run.smithi151.stdout: "ssl": { 2024-04-26T02:05:26.339 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ssl", 2024-04-26T02:05:26.339 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.339 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.339 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.339 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.339 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.339 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.339 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.340 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.340 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.340 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.340 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.340 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.340 INFO:teuthology.orchestra.run.smithi151.stdout: "ssl_server_port": { 2024-04-26T02:05:26.340 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ssl_server_port", 2024-04-26T02:05:26.340 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.340 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.340 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.340 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "8443", 2024-04-26T02:05:26.340 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.340 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.340 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.341 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.341 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.341 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.341 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.341 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.341 INFO:teuthology.orchestra.run.smithi151.stdout: "standby_behaviour": { 2024-04-26T02:05:26.341 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "standby_behaviour", 2024-04-26T02:05:26.341 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.341 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.341 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.341 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "redirect", 2024-04-26T02:05:26.341 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.341 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.341 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.342 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.342 INFO:teuthology.orchestra.run.smithi151.stdout: "redirect" 2024-04-26T02:05:26.342 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.342 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.342 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.342 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.342 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.342 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.342 INFO:teuthology.orchestra.run.smithi151.stdout: "standby_error_status_code": { 2024-04-26T02:05:26.342 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "standby_error_status_code", 2024-04-26T02:05:26.342 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.342 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.342 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.342 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "500", 2024-04-26T02:05:26.343 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "400", 2024-04-26T02:05:26.343 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "599", 2024-04-26T02:05:26.343 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.343 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.343 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.343 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.343 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.343 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.343 INFO:teuthology.orchestra.run.smithi151.stdout: "url_prefix": { 2024-04-26T02:05:26.343 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "url_prefix", 2024-04-26T02:05:26.343 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.343 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.343 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.343 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.343 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.343 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.344 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.344 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.344 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.344 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.344 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.344 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.344 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.344 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.344 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.344 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "devicehealth", 2024-04-26T02:05:26.344 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.344 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.344 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.344 INFO:teuthology.orchestra.run.smithi151.stdout: "enable_monitoring": { 2024-04-26T02:05:26.344 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "enable_monitoring", 2024-04-26T02:05:26.344 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.345 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.345 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.345 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.345 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.345 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.345 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.345 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "monitor device health metrics", 2024-04-26T02:05:26.345 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.345 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.345 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.345 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.345 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.345 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.345 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.345 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.345 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.346 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.346 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.346 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.346 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.346 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.346 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.346 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.346 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.346 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.346 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.346 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.346 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.346 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.346 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.347 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.347 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.347 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.347 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.347 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.347 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.347 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.347 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.347 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.347 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.347 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.347 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.347 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.347 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.347 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.348 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.348 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.348 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.348 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.348 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.348 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.348 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.348 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.348 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.348 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.348 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.348 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.348 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.348 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.348 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.348 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.349 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.349 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.349 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.349 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.349 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.349 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.349 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.349 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.349 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.349 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.349 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.349 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.349 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.349 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.349 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.349 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.350 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.350 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.350 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.350 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.350 INFO:teuthology.orchestra.run.smithi151.stdout: "mark_out_threshold": { 2024-04-26T02:05:26.350 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "mark_out_threshold", 2024-04-26T02:05:26.350 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-04-26T02:05:26.350 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.350 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.350 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "2419200", 2024-04-26T02:05:26.350 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.350 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.351 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.351 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-26T02:05:26.351 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.351 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.351 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.351 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.351 INFO:teuthology.orchestra.run.smithi151.stdout: "pool_name": { 2024-04-26T02:05:26.351 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "pool_name", 2024-04-26T02:05:26.351 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.351 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.351 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.351 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "device_health_metrics", 2024-04-26T02:05:26.351 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.351 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.352 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.352 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-26T02:05:26.352 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.352 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.352 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.352 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.352 INFO:teuthology.orchestra.run.smithi151.stdout: "retention_period": { 2024-04-26T02:05:26.352 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "retention_period", 2024-04-26T02:05:26.352 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-04-26T02:05:26.352 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.352 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.352 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "15552000", 2024-04-26T02:05:26.352 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.352 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.352 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.352 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "how long to retain device health metrics", 2024-04-26T02:05:26.353 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.353 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.353 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.353 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.353 INFO:teuthology.orchestra.run.smithi151.stdout: "scrape_frequency": { 2024-04-26T02:05:26.353 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "scrape_frequency", 2024-04-26T02:05:26.353 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-04-26T02:05:26.353 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.353 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.353 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "86400", 2024-04-26T02:05:26.353 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.353 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.353 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.353 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-26T02:05:26.353 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.354 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.354 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.354 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.354 INFO:teuthology.orchestra.run.smithi151.stdout: "self_heal": { 2024-04-26T02:05:26.354 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "self_heal", 2024-04-26T02:05:26.354 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.354 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.354 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.354 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.354 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.354 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.354 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.354 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-26T02:05:26.354 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.355 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.355 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.355 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.355 INFO:teuthology.orchestra.run.smithi151.stdout: "sleep_interval": { 2024-04-26T02:05:26.355 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sleep_interval", 2024-04-26T02:05:26.355 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-04-26T02:05:26.355 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.355 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.355 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "600", 2024-04-26T02:05:26.355 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.355 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.355 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.355 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "how frequently to wake up and check device health", 2024-04-26T02:05:26.355 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.355 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.355 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.356 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.356 INFO:teuthology.orchestra.run.smithi151.stdout: "warn_threshold": { 2024-04-26T02:05:26.356 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "warn_threshold", 2024-04-26T02:05:26.356 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-04-26T02:05:26.356 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.356 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.356 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "7257600", 2024-04-26T02:05:26.356 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.356 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.356 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.356 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-26T02:05:26.356 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.356 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.356 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.356 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.357 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.357 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.357 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.357 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "diskprediction_local", 2024-04-26T02:05:26.357 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.357 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.357 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.357 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.357 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.357 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.357 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.357 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.357 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.357 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.357 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.358 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.358 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.358 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.358 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.358 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.358 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.358 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.358 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.358 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.358 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.358 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.358 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.358 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.359 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.359 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.359 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.359 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.359 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.359 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.359 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.359 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.359 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.359 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.359 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.359 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.359 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.359 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.359 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.360 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.360 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.360 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.360 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.360 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.360 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.360 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.360 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.360 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.360 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.360 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.360 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.360 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.360 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.360 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.360 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.361 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.361 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.361 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.361 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.361 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.361 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.361 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.361 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.361 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.361 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.361 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.361 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.361 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.361 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.361 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.361 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.362 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.362 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.362 INFO:teuthology.orchestra.run.smithi151.stdout: "predict_interval": { 2024-04-26T02:05:26.362 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "predict_interval", 2024-04-26T02:05:26.362 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.362 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.362 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.362 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "86400", 2024-04-26T02:05:26.362 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.362 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.362 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.362 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.362 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.362 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.363 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.363 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.363 INFO:teuthology.orchestra.run.smithi151.stdout: "predictor_model": { 2024-04-26T02:05:26.363 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "predictor_model", 2024-04-26T02:05:26.363 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.363 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.363 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.363 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "prophetstor", 2024-04-26T02:05:26.363 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.363 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.363 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.363 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.363 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.363 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.364 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.364 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.364 INFO:teuthology.orchestra.run.smithi151.stdout: "sleep_interval": { 2024-04-26T02:05:26.364 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sleep_interval", 2024-04-26T02:05:26.364 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.364 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.364 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.364 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "600", 2024-04-26T02:05:26.364 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.364 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.364 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.364 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.364 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.364 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.364 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.365 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.365 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.365 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.365 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.365 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "influx", 2024-04-26T02:05:26.365 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": false, 2024-04-26T02:05:26.365 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "influxdb python module not found", 2024-04-26T02:05:26.365 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.365 INFO:teuthology.orchestra.run.smithi151.stdout: "batch_size": { 2024-04-26T02:05:26.365 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "batch_size", 2024-04-26T02:05:26.365 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.365 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.365 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.365 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "5000", 2024-04-26T02:05:26.365 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.366 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.366 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.366 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-26T02:05:26.366 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.366 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.366 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.366 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.366 INFO:teuthology.orchestra.run.smithi151.stdout: "database": { 2024-04-26T02:05:26.366 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "database", 2024-04-26T02:05:26.366 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.366 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.366 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.366 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "ceph", 2024-04-26T02:05:26.366 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.367 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.367 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.367 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-26T02:05:26.367 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.367 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.367 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.367 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.367 INFO:teuthology.orchestra.run.smithi151.stdout: "hostname": { 2024-04-26T02:05:26.367 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "hostname", 2024-04-26T02:05:26.367 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.367 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.367 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.367 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.368 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.368 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.368 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.368 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "InfluxDB server hostname", 2024-04-26T02:05:26.368 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.368 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.368 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.368 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.368 INFO:teuthology.orchestra.run.smithi151.stdout: "interval": { 2024-04-26T02:05:26.368 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "interval", 2024-04-26T02:05:26.368 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-04-26T02:05:26.368 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.368 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.368 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "30", 2024-04-26T02:05:26.369 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "5", 2024-04-26T02:05:26.369 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.369 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.369 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-26T02:05:26.369 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.369 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.369 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.369 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.369 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.369 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.369 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.369 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.369 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.370 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.370 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.370 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.370 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.370 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.370 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.370 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.370 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.370 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.370 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.370 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.370 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.370 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.370 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.370 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.371 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.371 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.371 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.371 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.371 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.371 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.371 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.371 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.371 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.371 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.371 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.371 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.371 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.372 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.372 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.372 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.372 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.372 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.372 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.372 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.372 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.372 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.372 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.372 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.372 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.372 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.373 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.373 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.373 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.373 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.373 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.373 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.373 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.373 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.373 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.373 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.373 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.373 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.373 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.373 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.374 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.374 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.374 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.374 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.374 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.374 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.374 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.374 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.374 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.374 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.374 INFO:teuthology.orchestra.run.smithi151.stdout: "password": { 2024-04-26T02:05:26.374 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "password", 2024-04-26T02:05:26.374 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.375 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.375 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.375 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.375 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.375 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.375 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.375 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "password of InfluxDB server user", 2024-04-26T02:05:26.375 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.375 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.375 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.375 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.375 INFO:teuthology.orchestra.run.smithi151.stdout: "port": { 2024-04-26T02:05:26.375 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "port", 2024-04-26T02:05:26.375 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.375 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.376 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.376 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "8086", 2024-04-26T02:05:26.376 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.376 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.376 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.376 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "InfluxDB server port", 2024-04-26T02:05:26.376 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.376 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.376 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.376 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.376 INFO:teuthology.orchestra.run.smithi151.stdout: "ssl": { 2024-04-26T02:05:26.376 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ssl", 2024-04-26T02:05:26.376 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.377 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.377 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.377 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "false", 2024-04-26T02:05:26.377 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.377 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.377 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.377 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T02:05:26.377 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.377 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.377 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.377 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.377 INFO:teuthology.orchestra.run.smithi151.stdout: "threads": { 2024-04-26T02:05:26.377 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "threads", 2024-04-26T02:05:26.378 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.378 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.378 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.378 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "5", 2024-04-26T02:05:26.378 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "1", 2024-04-26T02:05:26.378 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "32", 2024-04-26T02:05:26.378 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.378 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-26T02:05:26.378 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.378 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.378 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.378 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.378 INFO:teuthology.orchestra.run.smithi151.stdout: "username": { 2024-04-26T02:05:26.378 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "username", 2024-04-26T02:05:26.379 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.379 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.379 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.379 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.379 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.379 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.379 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.379 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "username of InfluxDB server user", 2024-04-26T02:05:26.379 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.379 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.379 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.379 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.379 INFO:teuthology.orchestra.run.smithi151.stdout: "verify_ssl": { 2024-04-26T02:05:26.379 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "verify_ssl", 2024-04-26T02:05:26.380 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.380 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.380 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.380 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "true", 2024-04-26T02:05:26.380 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.380 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.380 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.380 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T02:05:26.380 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.380 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.380 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.380 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.380 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.380 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.380 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.381 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "insights", 2024-04-26T02:05:26.381 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.381 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.381 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.381 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.381 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.381 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.381 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.381 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.381 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.381 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.381 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.381 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.381 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.381 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.381 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.382 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.382 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.382 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.382 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.382 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.382 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.382 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.382 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.382 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.382 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.382 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.382 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.382 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.382 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.382 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.383 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.383 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.383 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.383 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.383 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.383 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.383 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.383 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.383 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.383 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.383 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.383 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.383 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.383 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.383 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.383 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.383 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.384 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.384 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.384 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.384 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.384 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.384 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.384 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.384 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.384 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.384 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.384 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.384 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.384 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.384 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.385 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.385 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.385 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.385 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.385 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.385 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.385 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.385 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.385 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.385 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.385 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.385 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.385 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.385 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.385 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.385 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "iostat", 2024-04-26T02:05:26.386 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.386 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.386 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.386 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.386 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.386 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.386 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.386 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.386 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.386 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.386 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.386 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.386 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.386 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.387 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.387 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.387 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.387 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.387 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.387 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.387 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.387 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.387 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.387 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.387 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.387 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.387 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.387 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.387 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.388 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.388 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.388 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.388 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.388 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.388 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.388 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.388 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.388 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.388 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.388 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.388 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.388 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.388 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.388 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.388 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.389 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.389 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.389 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.389 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.389 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.389 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.389 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.389 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.389 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.389 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.389 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.389 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.389 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.389 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.390 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.390 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.390 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.390 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.390 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.390 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.390 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.390 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.390 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.390 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.390 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.390 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.391 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.391 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.391 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.391 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.391 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.391 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "k8sevents", 2024-04-26T02:05:26.391 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.391 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.391 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.391 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_event_retention_days": { 2024-04-26T02:05:26.391 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ceph_event_retention_days", 2024-04-26T02:05:26.391 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.391 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.391 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.392 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "7", 2024-04-26T02:05:26.392 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.392 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.392 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.392 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-26T02:05:26.392 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.392 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.392 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.392 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.392 INFO:teuthology.orchestra.run.smithi151.stdout: "config_check_secs": { 2024-04-26T02:05:26.392 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "config_check_secs", 2024-04-26T02:05:26.392 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.392 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.392 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.393 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "10", 2024-04-26T02:05:26.393 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "10", 2024-04-26T02:05:26.393 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.393 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.393 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-26T02:05:26.393 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.393 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.393 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.393 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.393 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.393 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.393 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.393 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.393 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.394 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.394 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.394 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.394 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.394 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.394 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.394 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.394 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.394 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.394 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.394 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.394 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.394 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.395 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.395 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.395 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.395 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.395 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.395 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.395 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.395 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.395 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.395 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.395 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.395 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.395 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.395 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.396 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.396 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.396 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.396 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.396 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.396 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.396 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.396 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.396 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.396 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.396 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.396 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.396 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.397 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.397 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.397 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.397 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.397 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.397 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.397 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.397 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.397 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.397 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.397 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.397 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.397 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.397 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.398 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.398 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.398 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.398 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.398 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.398 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.398 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.398 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.398 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.398 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.398 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.398 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.398 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.399 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.399 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "localpool", 2024-04-26T02:05:26.399 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.399 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.399 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.399 INFO:teuthology.orchestra.run.smithi151.stdout: "failure_domain": { 2024-04-26T02:05:26.399 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "failure_domain", 2024-04-26T02:05:26.399 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.399 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.399 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.399 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "host", 2024-04-26T02:05:26.399 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.399 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.400 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.400 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "failure domain for any created local pool", 2024-04-26T02:05:26.400 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-26T02:05:26.400 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.400 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.400 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.400 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.400 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.400 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.400 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.400 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.400 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.400 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.400 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.400 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.401 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.401 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.401 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.401 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.401 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.401 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.401 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.401 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.401 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.401 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.401 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.401 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.401 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.401 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.402 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.402 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.402 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.402 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.402 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.402 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.402 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.402 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.402 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.402 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.402 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.402 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.402 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.403 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.403 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.403 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.403 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.403 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.403 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.403 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.403 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.403 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.403 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.403 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.403 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.403 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.403 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.403 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.404 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.404 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.404 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.404 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.404 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.404 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.404 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.404 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.404 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.404 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.404 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.405 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.405 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.405 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.405 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.405 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.405 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.405 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.405 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.405 INFO:teuthology.orchestra.run.smithi151.stdout: "min_size": { 2024-04-26T02:05:26.405 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "min_size", 2024-04-26T02:05:26.405 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.405 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.405 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.405 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.406 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.406 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.406 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.406 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "default min_size for any created local pool", 2024-04-26T02:05:26.406 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-26T02:05:26.406 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.406 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.406 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.406 INFO:teuthology.orchestra.run.smithi151.stdout: "num_rep": { 2024-04-26T02:05:26.406 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "num_rep", 2024-04-26T02:05:26.406 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.406 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.406 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.407 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "3", 2024-04-26T02:05:26.407 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.407 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.407 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.407 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "default replica count for any created local pool", 2024-04-26T02:05:26.407 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.407 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.407 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.407 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.407 INFO:teuthology.orchestra.run.smithi151.stdout: "pg_num": { 2024-04-26T02:05:26.407 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "pg_num", 2024-04-26T02:05:26.407 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.407 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.407 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.407 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "128", 2024-04-26T02:05:26.408 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.408 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.408 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.408 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "default pg_num for any created local pool", 2024-04-26T02:05:26.408 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.408 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.408 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.408 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.408 INFO:teuthology.orchestra.run.smithi151.stdout: "prefix": { 2024-04-26T02:05:26.408 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "prefix", 2024-04-26T02:05:26.408 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.408 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.408 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.409 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.409 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.409 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.409 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.409 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "name prefix for any created local pool", 2024-04-26T02:05:26.409 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.409 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.409 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.409 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.409 INFO:teuthology.orchestra.run.smithi151.stdout: "subtree": { 2024-04-26T02:05:26.409 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "subtree", 2024-04-26T02:05:26.409 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.409 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.409 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.410 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "rack", 2024-04-26T02:05:26.410 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.410 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.410 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.410 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-26T02:05:26.410 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-26T02:05:26.410 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.410 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.410 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.410 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.410 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.410 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.410 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "mds_autoscaler", 2024-04-26T02:05:26.410 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.411 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.411 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.411 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.411 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.411 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.411 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.411 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.411 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.411 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.411 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.411 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.411 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.411 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.411 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.412 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.412 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.412 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.412 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.412 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.412 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.412 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.412 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.412 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.412 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.412 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.412 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.412 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.412 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.412 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.412 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.413 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.413 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.413 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.413 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.413 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.413 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.413 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.413 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.413 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.413 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.413 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.413 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.413 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.413 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.413 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.414 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.414 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.414 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.414 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.414 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.414 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.414 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.414 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.414 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.414 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.414 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.414 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.414 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.415 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.415 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.415 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.415 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.415 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.415 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.415 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.415 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.415 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.415 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.415 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.415 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.415 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.416 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.416 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.416 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.416 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.416 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "mirroring", 2024-04-26T02:05:26.416 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.416 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.416 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.416 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.416 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.416 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.416 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.416 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.416 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.416 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.416 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.417 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.417 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.417 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.417 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.417 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.417 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.417 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.417 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.417 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.417 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.417 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.417 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.417 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.417 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.417 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.418 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.418 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.418 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.418 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.418 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.418 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.418 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.418 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.418 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.418 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.418 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.418 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.419 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.419 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.419 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.419 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.419 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.419 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.419 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.419 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.419 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.419 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.419 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.419 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.419 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.419 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.419 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.420 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.420 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.420 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.420 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.420 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.420 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.420 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.420 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.420 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.420 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.420 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.420 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.420 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.420 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.420 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.421 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.421 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.421 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.421 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.421 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.421 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.421 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.421 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.421 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "nfs", 2024-04-26T02:05:26.421 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.421 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.421 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.421 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.421 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.421 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.422 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.422 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.422 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.422 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.422 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.422 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.422 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.422 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.422 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.422 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.422 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.422 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.423 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.423 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.423 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.423 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.423 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.423 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.423 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.423 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.423 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.423 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.423 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.423 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.423 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.423 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.423 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.424 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.424 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.424 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.424 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.424 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.424 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.424 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.424 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.424 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.424 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.424 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.424 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.424 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.424 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.425 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.425 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.425 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.425 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.425 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.425 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.425 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.425 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.425 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.425 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.425 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.425 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.425 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.425 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.426 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.426 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.426 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.426 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.426 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.426 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.426 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.426 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.426 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.426 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.426 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.426 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.426 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.426 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.427 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.427 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "orchestrator", 2024-04-26T02:05:26.427 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.427 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.427 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.427 INFO:teuthology.orchestra.run.smithi151.stdout: "fail_fs": { 2024-04-26T02:05:26.427 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "fail_fs", 2024-04-26T02:05:26.427 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.427 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.427 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.427 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.427 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.427 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.427 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.428 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-26T02:05:26.428 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.428 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.428 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.428 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.428 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.428 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.428 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.428 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.428 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.428 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.428 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.428 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.429 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.429 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.429 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.429 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.429 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.429 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.429 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.429 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.429 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.429 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.429 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.429 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.429 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.430 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.430 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.430 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.430 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.430 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.430 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.430 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.430 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.430 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.430 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.430 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.430 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.430 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.430 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.431 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.431 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.431 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.431 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.431 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.431 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.431 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.431 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.431 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.431 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.431 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.431 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.432 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.432 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.432 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.432 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.432 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.432 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.432 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.432 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.432 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.432 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.432 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.432 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.432 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.433 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.433 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.433 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.433 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.433 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.433 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.433 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.433 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.433 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.433 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.433 INFO:teuthology.orchestra.run.smithi151.stdout: "orchestrator": { 2024-04-26T02:05:26.433 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "orchestrator", 2024-04-26T02:05:26.433 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.434 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.434 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.434 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.434 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.434 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.434 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.434 INFO:teuthology.orchestra.run.smithi151.stdout: "cephadm", 2024-04-26T02:05:26.434 INFO:teuthology.orchestra.run.smithi151.stdout: "rook", 2024-04-26T02:05:26.434 INFO:teuthology.orchestra.run.smithi151.stdout: "test_orchestrator" 2024-04-26T02:05:26.434 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.434 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Orchestrator backend", 2024-04-26T02:05:26.434 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.434 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.434 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.435 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.435 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.435 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.435 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.435 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "osd_perf_query", 2024-04-26T02:05:26.435 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.435 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.435 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.435 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.435 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.435 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.435 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.435 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.436 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.436 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.436 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.436 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.436 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.436 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.436 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.436 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.436 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.436 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.436 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.436 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.436 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.437 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.437 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.437 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.437 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.437 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.437 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.437 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.437 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.437 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.437 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.437 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.437 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.437 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.437 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.438 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.438 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.438 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.438 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.438 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.438 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.438 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.438 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.438 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.438 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.438 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.438 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.438 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.438 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.439 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.439 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.439 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.439 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.439 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.439 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.439 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.439 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.439 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.439 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.439 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.439 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.439 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.439 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.440 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.440 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.440 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.440 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.440 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.440 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.440 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.440 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.440 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.440 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.440 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.440 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.440 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.440 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "osd_support", 2024-04-26T02:05:26.441 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.441 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.441 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.441 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.441 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.441 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.441 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.441 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.441 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.441 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.441 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.441 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.441 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.441 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.441 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.442 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.442 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.442 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.442 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.442 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.442 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.442 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.442 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.442 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.442 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.442 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.442 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.442 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.442 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.442 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.443 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.443 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.443 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.443 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.443 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.443 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.443 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.443 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.443 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.443 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.443 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.443 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.443 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.443 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.443 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.444 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.444 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.444 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.444 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.444 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.444 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.444 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.444 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.444 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.444 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.444 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.444 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.444 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.444 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.444 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.445 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.445 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.445 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.445 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.445 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.445 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.445 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.445 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.445 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.445 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.445 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.445 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.445 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.445 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.446 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.446 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.446 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "pg_autoscaler", 2024-04-26T02:05:26.446 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.446 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.446 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.446 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.446 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.446 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.446 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.446 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.446 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.447 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.447 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.447 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.447 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.447 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.447 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.447 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.447 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.447 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.447 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.447 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.447 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.447 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.447 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.448 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.448 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.448 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.448 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.448 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.448 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.448 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.448 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.448 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.448 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.448 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.448 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.448 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.448 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.448 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.448 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.449 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.449 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.449 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.449 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.449 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.449 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.449 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.449 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.449 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.449 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.449 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.449 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.449 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.449 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.449 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.449 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.450 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.450 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.450 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.450 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.450 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.450 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.450 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.450 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.450 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.450 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.450 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.450 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.450 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.451 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.451 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.451 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.451 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.451 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.451 INFO:teuthology.orchestra.run.smithi151.stdout: "sleep_interval": { 2024-04-26T02:05:26.451 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sleep_interval", 2024-04-26T02:05:26.451 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-04-26T02:05:26.451 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.451 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.451 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "60", 2024-04-26T02:05:26.451 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.451 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.451 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.451 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.452 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.452 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.452 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.452 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.452 INFO:teuthology.orchestra.run.smithi151.stdout: "threshold": { 2024-04-26T02:05:26.452 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "threshold", 2024-04-26T02:05:26.452 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "float", 2024-04-26T02:05:26.452 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.452 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.452 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "3.0", 2024-04-26T02:05:26.452 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "1.0", 2024-04-26T02:05:26.452 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.452 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.452 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "scaling threshold", 2024-04-26T02:05:26.452 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-26T02:05:26.452 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.453 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.453 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.453 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.453 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.453 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.453 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "progress", 2024-04-26T02:05:26.453 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.453 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.453 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.453 INFO:teuthology.orchestra.run.smithi151.stdout: "allow_pg_recovery_event": { 2024-04-26T02:05:26.453 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "allow_pg_recovery_event", 2024-04-26T02:05:26.453 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.453 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.453 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.453 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.453 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.454 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.454 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.454 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-26T02:05:26.454 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.454 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.454 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.454 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.454 INFO:teuthology.orchestra.run.smithi151.stdout: "enabled": { 2024-04-26T02:05:26.454 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "enabled", 2024-04-26T02:05:26.454 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.454 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.454 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.454 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.455 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.455 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.455 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.455 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.455 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.455 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.455 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.455 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.455 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.455 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.455 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.455 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.455 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.455 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.455 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.455 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.455 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.456 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.456 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.456 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.456 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.456 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.456 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.456 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.456 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.456 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.456 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.456 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.456 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.456 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.456 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.456 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.456 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.457 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.457 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.457 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.457 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.457 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.457 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.457 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.457 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.457 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.457 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.457 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.457 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.457 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.457 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.457 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.457 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.458 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.458 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.458 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.458 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.458 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.458 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.458 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.458 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.458 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.458 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.458 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.458 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.458 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.459 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.459 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.459 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.459 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.459 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.459 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.459 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.459 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.459 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.459 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.459 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.459 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.459 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.459 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.460 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.460 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.460 INFO:teuthology.orchestra.run.smithi151.stdout: "max_completed_events": { 2024-04-26T02:05:26.460 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "max_completed_events", 2024-04-26T02:05:26.460 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.460 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.460 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.460 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "50", 2024-04-26T02:05:26.460 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.460 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.460 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.460 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "number of past completed events to remember", 2024-04-26T02:05:26.460 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.461 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.461 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.461 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.461 INFO:teuthology.orchestra.run.smithi151.stdout: "sleep_interval": { 2024-04-26T02:05:26.461 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sleep_interval", 2024-04-26T02:05:26.461 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-04-26T02:05:26.461 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.461 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.461 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "5", 2024-04-26T02:05:26.461 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.461 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.461 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.461 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "how long the module is going to sleep", 2024-04-26T02:05:26.461 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.462 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.462 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.462 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.462 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.462 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.462 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.462 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "prometheus", 2024-04-26T02:05:26.462 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.462 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.462 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.462 INFO:teuthology.orchestra.run.smithi151.stdout: "cache": { 2024-04-26T02:05:26.462 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "cache", 2024-04-26T02:05:26.462 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.463 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.463 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.463 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.463 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.463 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.463 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.463 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.463 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.463 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.463 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.463 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.463 INFO:teuthology.orchestra.run.smithi151.stdout: "exclude_perf_counters": { 2024-04-26T02:05:26.463 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "exclude_perf_counters", 2024-04-26T02:05:26.463 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.463 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.464 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.464 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.464 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.464 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.464 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.464 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-26T02:05:26.464 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-26T02:05:26.464 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.464 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.464 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.464 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.464 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.464 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.464 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.464 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.465 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.465 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.465 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.465 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.465 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.465 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.465 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.465 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.465 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.465 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.465 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.465 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.465 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.465 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.465 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.465 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.466 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.466 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.466 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.466 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.466 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.466 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.466 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.466 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.466 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.466 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.466 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.466 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.466 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.466 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.467 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.467 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.467 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.467 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.467 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.467 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.467 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.467 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.467 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.467 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.467 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.467 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.467 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.467 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.468 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.468 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.468 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.468 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.468 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.468 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.468 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.468 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.468 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.468 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.468 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.468 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.468 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.468 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.469 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.469 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.469 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.469 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.469 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.469 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.469 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.469 INFO:teuthology.orchestra.run.smithi151.stdout: "rbd_stats_pools": { 2024-04-26T02:05:26.469 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rbd_stats_pools", 2024-04-26T02:05:26.469 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.469 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.469 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.469 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.470 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.470 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.470 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.470 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.470 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.470 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.470 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.470 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.470 INFO:teuthology.orchestra.run.smithi151.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-26T02:05:26.470 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-26T02:05:26.470 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.470 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.470 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.470 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "300", 2024-04-26T02:05:26.471 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.471 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.471 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.471 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.471 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.471 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.471 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.471 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.471 INFO:teuthology.orchestra.run.smithi151.stdout: "scrape_interval": { 2024-04-26T02:05:26.471 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "scrape_interval", 2024-04-26T02:05:26.471 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "float", 2024-04-26T02:05:26.471 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.471 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.471 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "15.0", 2024-04-26T02:05:26.471 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.472 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.472 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.472 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.472 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.472 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.472 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.472 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.472 INFO:teuthology.orchestra.run.smithi151.stdout: "server_addr": { 2024-04-26T02:05:26.472 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "server_addr", 2024-04-26T02:05:26.472 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.472 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.472 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.472 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "::", 2024-04-26T02:05:26.472 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.472 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.472 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.473 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-26T02:05:26.473 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.473 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.473 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.473 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.473 INFO:teuthology.orchestra.run.smithi151.stdout: "server_port": { 2024-04-26T02:05:26.473 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "server_port", 2024-04-26T02:05:26.473 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.473 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.473 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.473 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "9283", 2024-04-26T02:05:26.473 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.473 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.473 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.474 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-26T02:05:26.474 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.474 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.474 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.474 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.474 INFO:teuthology.orchestra.run.smithi151.stdout: "stale_cache_strategy": { 2024-04-26T02:05:26.474 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "stale_cache_strategy", 2024-04-26T02:05:26.474 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.474 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.474 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.474 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "log", 2024-04-26T02:05:26.474 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.474 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.475 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.475 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.475 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.475 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.475 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.475 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.475 INFO:teuthology.orchestra.run.smithi151.stdout: "standby_behaviour": { 2024-04-26T02:05:26.475 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "standby_behaviour", 2024-04-26T02:05:26.475 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.475 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.475 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.475 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "default", 2024-04-26T02:05:26.475 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.475 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.475 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.476 INFO:teuthology.orchestra.run.smithi151.stdout: "default", 2024-04-26T02:05:26.476 INFO:teuthology.orchestra.run.smithi151.stdout: "error" 2024-04-26T02:05:26.476 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.476 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.476 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.476 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.476 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.476 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.476 INFO:teuthology.orchestra.run.smithi151.stdout: "standby_error_status_code": { 2024-04-26T02:05:26.476 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "standby_error_status_code", 2024-04-26T02:05:26.476 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.476 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.476 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.476 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "500", 2024-04-26T02:05:26.476 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "400", 2024-04-26T02:05:26.476 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "599", 2024-04-26T02:05:26.477 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.477 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.477 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.477 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.477 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.477 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.477 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.477 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.477 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.477 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rbd_support", 2024-04-26T02:05:26.477 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.477 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.477 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.477 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.477 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.478 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.478 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.478 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.478 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.478 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.478 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.478 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.478 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.478 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.478 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.478 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.478 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.478 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.478 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.478 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.479 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.479 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.479 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.479 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.479 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.479 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.479 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.479 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.479 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.479 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.479 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.479 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.479 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.479 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.479 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.480 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.480 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.480 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.480 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.480 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.480 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.480 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.480 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.480 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.480 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.480 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.480 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.480 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.480 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.480 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.480 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.481 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.481 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.481 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.481 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.481 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.481 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.481 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.481 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.481 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.481 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.481 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.481 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.481 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.481 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.481 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.482 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.482 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.482 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.482 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.482 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.482 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.482 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.482 INFO:teuthology.orchestra.run.smithi151.stdout: "max_concurrent_snap_create": { 2024-04-26T02:05:26.482 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "max_concurrent_snap_create", 2024-04-26T02:05:26.482 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.482 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.482 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.482 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "10", 2024-04-26T02:05:26.482 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.482 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.483 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.483 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.483 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.483 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.483 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.483 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.483 INFO:teuthology.orchestra.run.smithi151.stdout: "mirror_snapshot_schedule": { 2024-04-26T02:05:26.483 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "mirror_snapshot_schedule", 2024-04-26T02:05:26.483 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.483 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.483 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.483 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.483 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.483 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.483 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.484 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.484 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.484 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.484 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.484 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.484 INFO:teuthology.orchestra.run.smithi151.stdout: "trash_purge_schedule": { 2024-04-26T02:05:26.484 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "trash_purge_schedule", 2024-04-26T02:05:26.484 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.484 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.484 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.484 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.485 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.485 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.485 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.485 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.485 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.485 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.485 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.485 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.485 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.485 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.485 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.485 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "restful", 2024-04-26T02:05:26.485 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.485 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.486 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.486 INFO:teuthology.orchestra.run.smithi151.stdout: "enable_auth": { 2024-04-26T02:05:26.486 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "enable_auth", 2024-04-26T02:05:26.486 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.486 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.486 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.486 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.486 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.486 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.486 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.486 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.486 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.486 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.487 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.487 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.487 INFO:teuthology.orchestra.run.smithi151.stdout: "key_file": { 2024-04-26T02:05:26.487 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "key_file", 2024-04-26T02:05:26.487 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.487 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.487 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.487 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.487 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.487 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.487 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.487 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.487 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.488 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.488 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.488 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.488 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.488 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.488 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.488 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.488 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.488 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.488 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.488 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.488 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.488 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.488 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.488 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.489 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.489 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.489 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.489 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.489 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.489 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.489 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.489 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.489 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.489 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.489 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.489 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.489 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.489 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.490 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.490 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.490 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.490 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.490 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.490 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.490 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.490 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.490 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.490 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.490 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.490 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.490 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.491 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.491 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.491 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.491 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.491 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.491 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.491 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.491 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.491 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.491 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.491 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.491 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.491 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.491 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.492 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.492 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.492 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.492 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.492 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.492 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.492 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.492 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.492 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.492 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.492 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.492 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.492 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.492 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.493 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.493 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.493 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.493 INFO:teuthology.orchestra.run.smithi151.stdout: "server_addr": { 2024-04-26T02:05:26.493 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "server_addr", 2024-04-26T02:05:26.493 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.493 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.493 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.493 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.493 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.493 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.493 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.493 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.494 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.494 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.494 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.494 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.494 INFO:teuthology.orchestra.run.smithi151.stdout: "server_port": { 2024-04-26T02:05:26.494 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "server_port", 2024-04-26T02:05:26.494 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.494 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.494 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.494 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.494 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.494 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.494 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.495 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.495 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.495 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.495 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.495 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.495 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.495 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.495 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.495 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rgw", 2024-04-26T02:05:26.495 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.495 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.495 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.495 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.496 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.496 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.496 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.496 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.496 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.496 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.496 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.496 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.496 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.496 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.496 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.496 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.496 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.496 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.496 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.497 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.497 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.497 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.497 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.497 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.497 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.497 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.497 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.497 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.497 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.497 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.497 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.497 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.497 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.498 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.498 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.498 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.498 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.498 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.498 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.498 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.498 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.498 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.498 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.498 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.498 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.498 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.498 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.499 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.499 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.499 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.499 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.499 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.499 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.499 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.499 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.499 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.499 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.499 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.499 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.499 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.500 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.500 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.500 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.500 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.500 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.500 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.500 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.500 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.500 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.500 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.500 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.500 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.500 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.501 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.501 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.501 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.501 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rook", 2024-04-26T02:05:26.501 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.501 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.501 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.501 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.501 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.501 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.501 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.501 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.501 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.502 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.502 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.502 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.502 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.502 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.502 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.502 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.502 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.502 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.502 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.502 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.502 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.502 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.503 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.503 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.503 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.503 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.503 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.503 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.503 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.503 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.503 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.503 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.503 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.503 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.503 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.504 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.504 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.504 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.504 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.504 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.504 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.504 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.504 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.504 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.504 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.504 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.504 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.504 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.504 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.505 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.505 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.505 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.505 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.505 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.505 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.505 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.505 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.505 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.505 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.505 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.505 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.505 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.505 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.505 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.505 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.506 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.506 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.506 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.506 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.506 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.506 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.506 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.506 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.506 INFO:teuthology.orchestra.run.smithi151.stdout: "storage_class": { 2024-04-26T02:05:26.506 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "storage_class", 2024-04-26T02:05:26.506 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.506 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.506 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.507 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "local", 2024-04-26T02:05:26.507 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.507 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.507 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.507 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-26T02:05:26.507 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.507 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.507 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.507 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.507 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.507 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.507 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.507 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "selftest", 2024-04-26T02:05:26.507 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.507 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.507 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.508 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.508 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.508 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.508 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.508 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.508 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.508 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.508 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.508 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.508 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.508 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.508 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.508 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.508 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.508 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.508 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.509 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.509 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.509 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.509 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.509 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.509 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.509 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.509 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.509 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.509 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.509 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.509 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.509 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.509 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.510 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.510 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.510 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.510 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.510 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.510 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.510 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.510 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.510 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.510 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.510 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.510 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.510 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.511 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.511 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.511 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.511 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.511 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.511 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.511 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.511 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.511 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.511 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.511 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.511 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.511 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.511 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.511 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.511 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.511 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.512 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.512 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.512 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.512 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.512 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.512 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.512 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.512 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.512 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.512 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.512 INFO:teuthology.orchestra.run.smithi151.stdout: "roption1": { 2024-04-26T02:05:26.512 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "roption1", 2024-04-26T02:05:26.512 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.512 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.513 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.513 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.513 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.513 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.513 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.513 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.513 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.513 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.513 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.513 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.513 INFO:teuthology.orchestra.run.smithi151.stdout: "roption2": { 2024-04-26T02:05:26.513 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "roption2", 2024-04-26T02:05:26.513 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.513 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.513 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.514 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "xyz", 2024-04-26T02:05:26.514 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.514 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.514 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.514 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.514 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.514 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.514 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.514 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.514 INFO:teuthology.orchestra.run.smithi151.stdout: "rwoption1": { 2024-04-26T02:05:26.514 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rwoption1", 2024-04-26T02:05:26.514 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.514 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.514 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.515 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.515 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.515 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.515 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.515 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.515 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.515 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.515 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.515 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.515 INFO:teuthology.orchestra.run.smithi151.stdout: "rwoption2": { 2024-04-26T02:05:26.515 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rwoption2", 2024-04-26T02:05:26.515 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.515 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.515 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.515 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.515 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.516 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.516 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.516 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.516 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.516 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.516 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.516 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.516 INFO:teuthology.orchestra.run.smithi151.stdout: "rwoption3": { 2024-04-26T02:05:26.516 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rwoption3", 2024-04-26T02:05:26.516 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "float", 2024-04-26T02:05:26.516 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.516 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.516 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.516 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.516 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.516 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.517 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.517 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.517 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.517 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.517 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.517 INFO:teuthology.orchestra.run.smithi151.stdout: "rwoption4": { 2024-04-26T02:05:26.517 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rwoption4", 2024-04-26T02:05:26.517 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.517 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.517 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.517 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.517 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.517 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.517 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.517 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.517 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.518 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.518 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.518 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.518 INFO:teuthology.orchestra.run.smithi151.stdout: "rwoption5": { 2024-04-26T02:05:26.518 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rwoption5", 2024-04-26T02:05:26.518 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.518 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.518 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.518 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.518 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.518 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.518 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.518 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.519 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.519 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.519 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.519 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.519 INFO:teuthology.orchestra.run.smithi151.stdout: "rwoption6": { 2024-04-26T02:05:26.519 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rwoption6", 2024-04-26T02:05:26.519 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.519 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.519 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.519 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.519 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.519 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.519 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.519 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.519 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.520 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.520 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.520 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.520 INFO:teuthology.orchestra.run.smithi151.stdout: "rwoption7": { 2024-04-26T02:05:26.520 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rwoption7", 2024-04-26T02:05:26.520 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.520 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.520 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.520 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.520 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "1", 2024-04-26T02:05:26.520 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "42", 2024-04-26T02:05:26.520 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.520 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.520 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.521 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.521 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.521 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.521 INFO:teuthology.orchestra.run.smithi151.stdout: "testkey": { 2024-04-26T02:05:26.521 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "testkey", 2024-04-26T02:05:26.521 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.521 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.521 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.521 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.521 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.521 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.521 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.521 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.521 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.521 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.522 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.522 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.522 INFO:teuthology.orchestra.run.smithi151.stdout: "testlkey": { 2024-04-26T02:05:26.522 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "testlkey", 2024-04-26T02:05:26.522 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.522 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.522 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.522 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.522 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.522 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.522 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.522 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.522 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.522 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.522 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.522 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.523 INFO:teuthology.orchestra.run.smithi151.stdout: "testnewline": { 2024-04-26T02:05:26.523 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "testnewline", 2024-04-26T02:05:26.523 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.523 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.523 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.523 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.523 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.523 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.523 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.523 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.523 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.523 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.523 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.523 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.523 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.523 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.524 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.524 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "snap_schedule", 2024-04-26T02:05:26.524 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.524 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.524 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.524 INFO:teuthology.orchestra.run.smithi151.stdout: "allow_m_granularity": { 2024-04-26T02:05:26.524 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "allow_m_granularity", 2024-04-26T02:05:26.524 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.524 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.524 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.524 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.524 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.524 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.524 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.524 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "allow minute scheduled snapshots", 2024-04-26T02:05:26.524 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.525 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.525 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.525 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.525 INFO:teuthology.orchestra.run.smithi151.stdout: "dump_on_update": { 2024-04-26T02:05:26.525 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "dump_on_update", 2024-04-26T02:05:26.525 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.525 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.525 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.525 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.525 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.525 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.525 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.525 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "dump database to debug log on update", 2024-04-26T02:05:26.525 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.526 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.526 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.526 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.526 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.526 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.526 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.526 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.526 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.526 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.526 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.526 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.526 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.526 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.526 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.526 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.527 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.527 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.527 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.527 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.527 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.527 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.527 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.527 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.527 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.527 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.527 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.527 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.527 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.527 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.527 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.528 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.528 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.528 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.528 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.528 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.528 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.528 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.528 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.528 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.528 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.528 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.528 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.528 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.528 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.528 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.528 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.529 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.529 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.529 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.529 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.529 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.529 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.529 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.529 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.529 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.529 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.529 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.529 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.529 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.529 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.529 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.529 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.530 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.530 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.530 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.530 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.530 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.530 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.530 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.530 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.530 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.530 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.530 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.530 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.530 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.530 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.530 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "stats", 2024-04-26T02:05:26.531 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.531 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.531 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.531 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.531 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.531 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.531 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.531 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.531 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.531 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.531 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.531 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.531 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.531 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.531 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.532 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.532 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.532 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.532 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.532 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.532 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.532 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.532 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.532 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.532 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.532 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.532 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.532 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.532 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.532 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.532 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.533 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.533 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.533 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.533 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.533 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.533 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.533 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.533 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.533 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.533 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.533 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.533 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.533 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.533 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.533 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.533 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.534 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.534 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.534 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.534 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.534 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.534 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.534 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.534 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.534 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.534 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.534 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.534 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.534 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.534 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.535 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.535 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.535 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.535 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.535 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.535 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.535 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.535 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.535 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.535 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.535 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.535 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.535 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.535 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.536 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.536 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "status", 2024-04-26T02:05:26.536 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.536 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.536 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.536 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.536 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.536 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.536 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.536 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.536 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.536 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.536 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.536 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.536 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.536 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.537 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.537 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.537 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.537 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.537 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.537 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.537 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.537 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.537 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.537 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.537 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.537 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.537 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.537 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.537 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.538 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.538 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.538 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.538 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.538 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.538 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.538 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.538 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.538 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.538 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.538 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.538 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.539 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.539 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.539 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.539 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.539 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.539 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.539 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.539 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.539 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.539 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.539 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.539 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.539 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.539 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.539 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.540 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.540 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.540 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.540 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.540 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.540 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.540 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.540 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.540 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.540 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.540 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.540 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.540 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.540 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.540 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.541 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.541 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.541 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.541 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.541 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.541 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "telegraf", 2024-04-26T02:05:26.541 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.541 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.541 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.541 INFO:teuthology.orchestra.run.smithi151.stdout: "address": { 2024-04-26T02:05:26.541 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "address", 2024-04-26T02:05:26.541 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.541 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.541 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.542 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-26T02:05:26.542 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.542 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.542 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.542 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.542 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.542 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.542 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.542 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.542 INFO:teuthology.orchestra.run.smithi151.stdout: "interval": { 2024-04-26T02:05:26.542 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "interval", 2024-04-26T02:05:26.542 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-04-26T02:05:26.542 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.542 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.542 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "15", 2024-04-26T02:05:26.543 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.543 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.543 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.543 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.543 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.543 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.543 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.543 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.543 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.543 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.543 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.543 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.543 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.543 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.544 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.544 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.544 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.544 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.544 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.544 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.544 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.544 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.544 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.544 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.544 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.544 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.544 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.544 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.545 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.545 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.545 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.545 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.545 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.545 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.545 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.545 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.545 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.545 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.545 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.545 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.545 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.545 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.545 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.546 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.546 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.546 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.546 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.546 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.546 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.546 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.546 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.546 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.546 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.546 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.546 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.546 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.546 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.546 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.547 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.547 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.547 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.547 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.547 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.547 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.547 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.547 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.547 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.547 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.547 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.547 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.547 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.547 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.547 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.547 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.548 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.548 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.548 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.548 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.548 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.548 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.548 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.548 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "telemetry", 2024-04-26T02:05:26.548 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.548 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.548 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.548 INFO:teuthology.orchestra.run.smithi151.stdout: "channel_basic": { 2024-04-26T02:05:26.548 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "channel_basic", 2024-04-26T02:05:26.548 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.548 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.549 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.549 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.549 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.549 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.549 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.549 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-26T02:05:26.549 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.549 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.549 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.549 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.549 INFO:teuthology.orchestra.run.smithi151.stdout: "channel_crash": { 2024-04-26T02:05:26.549 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "channel_crash", 2024-04-26T02:05:26.549 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.549 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.549 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.550 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.550 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.550 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.550 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.550 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-26T02:05:26.550 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.550 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.550 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.550 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.550 INFO:teuthology.orchestra.run.smithi151.stdout: "channel_device": { 2024-04-26T02:05:26.550 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "channel_device", 2024-04-26T02:05:26.550 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.550 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.550 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.551 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.551 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.551 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.551 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.551 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-26T02:05:26.551 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.551 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.551 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.551 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.551 INFO:teuthology.orchestra.run.smithi151.stdout: "channel_ident": { 2024-04-26T02:05:26.551 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "channel_ident", 2024-04-26T02:05:26.551 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.551 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.552 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.552 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.552 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.552 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.552 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.552 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-26T02:05:26.552 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.552 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.552 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.552 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.552 INFO:teuthology.orchestra.run.smithi151.stdout: "channel_perf": { 2024-04-26T02:05:26.552 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "channel_perf", 2024-04-26T02:05:26.552 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.552 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.553 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.553 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.553 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.553 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.553 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.553 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-26T02:05:26.553 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.553 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.553 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.553 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.553 INFO:teuthology.orchestra.run.smithi151.stdout: "contact": { 2024-04-26T02:05:26.553 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "contact", 2024-04-26T02:05:26.553 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.554 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.554 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.554 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.554 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.554 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.554 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.554 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.554 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.554 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.554 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.554 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.554 INFO:teuthology.orchestra.run.smithi151.stdout: "description": { 2024-04-26T02:05:26.554 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "description", 2024-04-26T02:05:26.555 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.555 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.555 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.555 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.555 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.555 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.555 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.555 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.555 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.555 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.555 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.555 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.555 INFO:teuthology.orchestra.run.smithi151.stdout: "device_url": { 2024-04-26T02:05:26.555 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "device_url", 2024-04-26T02:05:26.556 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.556 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.556 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.556 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-26T02:05:26.556 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.556 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.556 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.556 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.556 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.556 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.556 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.556 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.556 INFO:teuthology.orchestra.run.smithi151.stdout: "enabled": { 2024-04-26T02:05:26.556 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "enabled", 2024-04-26T02:05:26.556 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.556 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.557 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.557 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.557 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.557 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.557 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.557 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.557 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.557 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.557 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.557 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.557 INFO:teuthology.orchestra.run.smithi151.stdout: "interval": { 2024-04-26T02:05:26.557 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "interval", 2024-04-26T02:05:26.557 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.557 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.557 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.557 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "24", 2024-04-26T02:05:26.558 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "8", 2024-04-26T02:05:26.558 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.558 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.558 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.558 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.558 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.558 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.558 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.558 INFO:teuthology.orchestra.run.smithi151.stdout: "last_opt_revision": { 2024-04-26T02:05:26.558 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "last_opt_revision", 2024-04-26T02:05:26.558 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.558 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.558 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.558 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "1", 2024-04-26T02:05:26.558 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.559 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.559 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.559 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.559 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.559 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.559 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.559 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.559 INFO:teuthology.orchestra.run.smithi151.stdout: "leaderboard": { 2024-04-26T02:05:26.559 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "leaderboard", 2024-04-26T02:05:26.559 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.559 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.559 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.559 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.559 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.559 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.559 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.559 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.560 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.560 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.560 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.560 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.560 INFO:teuthology.orchestra.run.smithi151.stdout: "leaderboard_description": { 2024-04-26T02:05:26.560 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "leaderboard_description", 2024-04-26T02:05:26.560 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.560 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.560 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.560 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.560 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.560 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.560 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.560 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.561 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.561 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.561 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.561 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.561 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.561 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.561 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.561 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.561 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.561 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.561 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.561 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.561 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.561 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.562 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.562 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.562 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.562 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.562 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.562 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.562 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.562 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.562 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.562 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.562 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.562 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.562 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.562 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.562 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.563 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.563 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.563 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.563 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.563 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.563 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.563 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.563 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.563 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.563 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.563 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.563 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.563 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.563 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.563 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.564 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.564 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.564 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.564 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.564 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.564 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.564 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.564 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.564 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.564 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.564 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.564 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.564 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.564 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.565 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.565 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.565 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.565 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.565 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.565 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.565 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.565 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.565 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.565 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.565 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.565 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.565 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.565 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.565 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.566 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.566 INFO:teuthology.orchestra.run.smithi151.stdout: "organization": { 2024-04-26T02:05:26.566 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "organization", 2024-04-26T02:05:26.566 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.566 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.566 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.566 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.566 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.566 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.566 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.566 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.566 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.566 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.566 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.566 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.567 INFO:teuthology.orchestra.run.smithi151.stdout: "proxy": { 2024-04-26T02:05:26.567 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "proxy", 2024-04-26T02:05:26.567 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.567 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.567 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.567 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.567 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.567 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.567 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.567 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.567 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.567 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.567 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.567 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.568 INFO:teuthology.orchestra.run.smithi151.stdout: "url": { 2024-04-26T02:05:26.568 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "url", 2024-04-26T02:05:26.568 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.568 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.568 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.568 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-26T02:05:26.568 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.568 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.568 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.568 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.568 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.568 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.568 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.568 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.569 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.569 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.569 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.569 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "test_orchestrator", 2024-04-26T02:05:26.569 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.569 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.569 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.569 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.569 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.569 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.569 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.569 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.569 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.569 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.569 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.570 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.570 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.570 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.570 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.570 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.570 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.570 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.570 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.570 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.570 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.570 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.570 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.570 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.570 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.571 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.571 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.571 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.571 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.571 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.571 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.571 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.571 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.571 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.571 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.571 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.571 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.571 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.571 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.572 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.572 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.572 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.572 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.572 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.572 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.572 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.572 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.572 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.572 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.572 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.572 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.572 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.572 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.572 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.573 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.573 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.573 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.573 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.573 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.573 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.573 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.573 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.573 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.573 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.573 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.573 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.573 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.573 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.574 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.574 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.574 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.574 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.574 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.574 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.574 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.574 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.574 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "volumes", 2024-04-26T02:05:26.574 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.574 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.574 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.574 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.574 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.575 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.575 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.575 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.575 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.575 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.575 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.575 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.575 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.575 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.575 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.575 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.575 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.575 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.575 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.576 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.576 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.576 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.576 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.576 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.576 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.576 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.576 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.576 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.576 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.576 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.576 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.576 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.576 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.577 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.577 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.577 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.577 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.577 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.577 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.577 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.577 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.577 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.577 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.577 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.577 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.577 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.577 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.578 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.578 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.578 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.578 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.578 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.578 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.578 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.578 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.578 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.578 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.578 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.578 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.578 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.579 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.579 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.579 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.579 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.579 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.579 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.579 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.579 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.579 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.579 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.579 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.579 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.579 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.579 INFO:teuthology.orchestra.run.smithi151.stdout: "max_concurrent_clones": { 2024-04-26T02:05:26.580 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "max_concurrent_clones", 2024-04-26T02:05:26.580 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.580 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.580 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.580 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "4", 2024-04-26T02:05:26.580 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.580 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.580 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.580 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-26T02:05:26.580 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.580 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.580 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.580 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.581 INFO:teuthology.orchestra.run.smithi151.stdout: "periodic_async_work": { 2024-04-26T02:05:26.581 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "periodic_async_work", 2024-04-26T02:05:26.581 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.581 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.581 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.581 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.581 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.581 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.581 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.581 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Periodically check for async work", 2024-04-26T02:05:26.581 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.581 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.581 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.581 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.581 INFO:teuthology.orchestra.run.smithi151.stdout: "snapshot_clone_delay": { 2024-04-26T02:05:26.582 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "snapshot_clone_delay", 2024-04-26T02:05:26.582 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.582 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.582 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.582 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-04-26T02:05:26.582 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.582 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.582 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.582 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-26T02:05:26.582 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.582 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.582 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.582 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.583 INFO:teuthology.orchestra.run.smithi151.stdout: "snapshot_clone_no_wait": { 2024-04-26T02:05:26.583 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "snapshot_clone_no_wait", 2024-04-26T02:05:26.583 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.583 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.583 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.583 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.583 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.583 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.583 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.583 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-26T02:05:26.583 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.583 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.583 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.583 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.584 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.584 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.584 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.584 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "zabbix", 2024-04-26T02:05:26.584 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.584 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.584 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.584 INFO:teuthology.orchestra.run.smithi151.stdout: "discovery_interval": { 2024-04-26T02:05:26.584 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "discovery_interval", 2024-04-26T02:05:26.584 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "uint", 2024-04-26T02:05:26.584 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.584 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.584 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "100", 2024-04-26T02:05:26.584 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.585 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.585 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.585 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.585 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.585 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.585 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.585 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.585 INFO:teuthology.orchestra.run.smithi151.stdout: "identifier": { 2024-04-26T02:05:26.585 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "identifier", 2024-04-26T02:05:26.585 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.585 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.585 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.585 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.585 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.586 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.586 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.586 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.586 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.586 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.586 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.586 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.586 INFO:teuthology.orchestra.run.smithi151.stdout: "interval": { 2024-04-26T02:05:26.586 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "interval", 2024-04-26T02:05:26.586 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-04-26T02:05:26.586 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.586 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.586 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "60", 2024-04-26T02:05:26.586 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.587 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.587 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.587 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.587 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.587 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.587 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.587 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.587 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.587 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.587 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.587 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.587 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.587 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.587 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.588 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.588 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.588 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.588 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.588 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.588 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.588 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.588 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.588 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.588 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.588 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.588 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.588 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.588 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.589 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.589 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.589 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.589 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.589 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.589 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.589 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.589 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.589 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.589 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.589 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.589 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.589 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.589 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.590 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.590 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.590 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.590 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.590 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.590 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.590 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.590 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.590 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.590 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.590 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.590 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.590 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.591 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.591 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.591 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.591 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.591 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.591 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.591 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.591 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.591 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.591 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.591 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.591 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.591 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.591 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.591 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.592 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.592 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.592 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.592 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.592 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.592 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.592 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.592 INFO:teuthology.orchestra.run.smithi151.stdout: "zabbix_host": { 2024-04-26T02:05:26.592 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "zabbix_host", 2024-04-26T02:05:26.592 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.592 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.592 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.593 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.593 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.593 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.593 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.593 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.593 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.593 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.593 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.593 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.593 INFO:teuthology.orchestra.run.smithi151.stdout: "zabbix_port": { 2024-04-26T02:05:26.593 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "zabbix_port", 2024-04-26T02:05:26.593 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.593 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.593 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.593 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "10051", 2024-04-26T02:05:26.594 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.594 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.594 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.594 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.594 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.594 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.594 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.594 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.594 INFO:teuthology.orchestra.run.smithi151.stdout: "zabbix_sender": { 2024-04-26T02:05:26.594 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "zabbix_sender", 2024-04-26T02:05:26.594 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.594 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.594 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.594 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-26T02:05:26.595 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.595 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.595 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.595 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.595 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.595 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.595 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.595 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.595 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.595 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.595 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-26T02:05:26.595 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.595 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.595 INFO:teuthology.orchestra.run.smithi151.stdout: "gid": 25042, 2024-04-26T02:05:26.595 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "z", 2024-04-26T02:05:26.595 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr_features": 4540701547738038271, 2024-04-26T02:05:26.596 INFO:teuthology.orchestra.run.smithi151.stdout: "available_modules": [ 2024-04-26T02:05:26.596 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.596 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "alerts", 2024-04-26T02:05:26.596 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.596 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.596 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.596 INFO:teuthology.orchestra.run.smithi151.stdout: "interval": { 2024-04-26T02:05:26.596 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "interval", 2024-04-26T02:05:26.596 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-04-26T02:05:26.596 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.596 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.596 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "60", 2024-04-26T02:05:26.596 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.596 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.596 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.597 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "How frequently to reexamine health status", 2024-04-26T02:05:26.597 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.597 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.597 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.597 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.597 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.597 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.597 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.597 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.597 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.597 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.597 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.597 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.597 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.598 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.598 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.598 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.598 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.598 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.598 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.598 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.598 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.598 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.598 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.598 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.598 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.598 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.598 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.599 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.599 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.599 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.599 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.599 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.599 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.599 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.599 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.599 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.599 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.599 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.599 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.599 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.599 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.599 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.599 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.600 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.600 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.600 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.600 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.600 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.600 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.600 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.600 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.600 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.600 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.600 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.600 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.600 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.600 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.601 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.601 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.601 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.601 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.601 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.601 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.601 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.601 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.601 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.601 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.601 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.601 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.601 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.601 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.602 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.602 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.602 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.602 INFO:teuthology.orchestra.run.smithi151.stdout: "smtp_destination": { 2024-04-26T02:05:26.602 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "smtp_destination", 2024-04-26T02:05:26.602 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.602 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.602 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.602 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.602 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.602 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.602 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.602 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Email address to send alerts to", 2024-04-26T02:05:26.602 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.603 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.603 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.603 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.603 INFO:teuthology.orchestra.run.smithi151.stdout: "smtp_from_name": { 2024-04-26T02:05:26.603 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "smtp_from_name", 2024-04-26T02:05:26.603 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.603 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.603 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.603 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "Ceph", 2024-04-26T02:05:26.603 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.603 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.603 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.603 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Email From: name", 2024-04-26T02:05:26.603 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.603 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.604 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.604 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.604 INFO:teuthology.orchestra.run.smithi151.stdout: "smtp_host": { 2024-04-26T02:05:26.604 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "smtp_host", 2024-04-26T02:05:26.604 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.604 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.604 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.604 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.604 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.604 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.604 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.604 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "SMTP server", 2024-04-26T02:05:26.604 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.604 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.605 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.605 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.605 INFO:teuthology.orchestra.run.smithi151.stdout: "smtp_password": { 2024-04-26T02:05:26.605 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "smtp_password", 2024-04-26T02:05:26.605 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.605 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.605 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.605 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.605 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.605 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.605 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.605 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Password to authenticate with", 2024-04-26T02:05:26.605 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.605 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.606 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.606 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.606 INFO:teuthology.orchestra.run.smithi151.stdout: "smtp_port": { 2024-04-26T02:05:26.606 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "smtp_port", 2024-04-26T02:05:26.606 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.606 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.606 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.606 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "465", 2024-04-26T02:05:26.606 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.606 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.606 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.606 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "SMTP port", 2024-04-26T02:05:26.606 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.606 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.607 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.607 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.607 INFO:teuthology.orchestra.run.smithi151.stdout: "smtp_sender": { 2024-04-26T02:05:26.607 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "smtp_sender", 2024-04-26T02:05:26.607 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.607 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.607 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.607 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.607 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.607 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.607 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.607 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "SMTP envelope sender", 2024-04-26T02:05:26.607 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.607 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.608 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.608 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.608 INFO:teuthology.orchestra.run.smithi151.stdout: "smtp_ssl": { 2024-04-26T02:05:26.608 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "smtp_ssl", 2024-04-26T02:05:26.608 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.608 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.608 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.608 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.608 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.608 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.608 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.608 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-26T02:05:26.608 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.608 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.608 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.609 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.609 INFO:teuthology.orchestra.run.smithi151.stdout: "smtp_user": { 2024-04-26T02:05:26.609 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "smtp_user", 2024-04-26T02:05:26.609 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.609 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.609 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.609 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.609 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.609 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.609 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.609 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "User to authenticate as", 2024-04-26T02:05:26.609 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.609 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.609 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.610 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.610 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.610 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.610 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.610 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "balancer", 2024-04-26T02:05:26.610 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.610 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.610 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.610 INFO:teuthology.orchestra.run.smithi151.stdout: "active": { 2024-04-26T02:05:26.610 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "active", 2024-04-26T02:05:26.610 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.610 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.610 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.610 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.611 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.611 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.611 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.611 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "automatically balance PGs across cluster", 2024-04-26T02:05:26.611 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.611 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.611 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.611 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.611 INFO:teuthology.orchestra.run.smithi151.stdout: "begin_time": { 2024-04-26T02:05:26.611 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "begin_time", 2024-04-26T02:05:26.611 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.611 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.611 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.612 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0000", 2024-04-26T02:05:26.612 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.612 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.612 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.612 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "beginning time of day to automatically balance", 2024-04-26T02:05:26.612 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T02:05:26.612 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.612 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.612 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.612 INFO:teuthology.orchestra.run.smithi151.stdout: "begin_weekday": { 2024-04-26T02:05:26.612 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "begin_weekday", 2024-04-26T02:05:26.612 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "uint", 2024-04-26T02:05:26.612 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.612 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.613 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-04-26T02:05:26.613 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "0", 2024-04-26T02:05:26.613 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "6", 2024-04-26T02:05:26.613 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.613 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-26T02:05:26.613 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T02:05:26.613 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.613 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.613 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.613 INFO:teuthology.orchestra.run.smithi151.stdout: "crush_compat_max_iterations": { 2024-04-26T02:05:26.613 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "crush_compat_max_iterations", 2024-04-26T02:05:26.613 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "uint", 2024-04-26T02:05:26.613 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.613 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.613 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "25", 2024-04-26T02:05:26.614 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "1", 2024-04-26T02:05:26.614 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "250", 2024-04-26T02:05:26.614 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.614 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-26T02:05:26.614 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.614 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.614 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.614 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.614 INFO:teuthology.orchestra.run.smithi151.stdout: "crush_compat_metrics": { 2024-04-26T02:05:26.614 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "crush_compat_metrics", 2024-04-26T02:05:26.614 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.614 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.614 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.615 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "pgs,objects,bytes", 2024-04-26T02:05:26.615 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.615 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.615 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.615 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-26T02:05:26.615 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-26T02:05:26.615 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.615 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.615 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.615 INFO:teuthology.orchestra.run.smithi151.stdout: "crush_compat_step": { 2024-04-26T02:05:26.615 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "crush_compat_step", 2024-04-26T02:05:26.615 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "float", 2024-04-26T02:05:26.615 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.615 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.615 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0.5", 2024-04-26T02:05:26.616 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "0.001", 2024-04-26T02:05:26.616 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "0.999", 2024-04-26T02:05:26.616 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.616 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "aggressiveness of optimization", 2024-04-26T02:05:26.616 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-26T02:05:26.616 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.616 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.616 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.616 INFO:teuthology.orchestra.run.smithi151.stdout: "end_time": { 2024-04-26T02:05:26.616 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "end_time", 2024-04-26T02:05:26.616 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.616 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.616 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.616 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "2359", 2024-04-26T02:05:26.616 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.617 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.617 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.617 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "ending time of day to automatically balance", 2024-04-26T02:05:26.617 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T02:05:26.617 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.617 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.617 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.617 INFO:teuthology.orchestra.run.smithi151.stdout: "end_weekday": { 2024-04-26T02:05:26.617 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "end_weekday", 2024-04-26T02:05:26.617 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "uint", 2024-04-26T02:05:26.617 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.617 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.617 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-04-26T02:05:26.617 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "0", 2024-04-26T02:05:26.617 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "6", 2024-04-26T02:05:26.618 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.618 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-26T02:05:26.618 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T02:05:26.618 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.618 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.618 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.618 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.618 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.618 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.618 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.618 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.618 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.618 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.619 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.619 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.619 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.619 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.619 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.619 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.619 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.619 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.619 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.619 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.619 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.619 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.619 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.620 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.620 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.620 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.620 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.620 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.620 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.620 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.620 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.620 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.620 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.620 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.620 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.620 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.620 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.621 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.621 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.621 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.621 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.621 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.621 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.621 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.621 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.621 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.621 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.621 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.621 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.621 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.621 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.621 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.622 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.622 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.622 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.622 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.622 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.622 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.622 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.622 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.622 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.622 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.622 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.622 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.622 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.623 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.623 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.623 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.623 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.623 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.623 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.623 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.623 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.623 INFO:teuthology.orchestra.run.smithi151.stdout: "min_score": { 2024-04-26T02:05:26.623 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "min_score", 2024-04-26T02:05:26.623 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "float", 2024-04-26T02:05:26.623 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.623 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.623 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-04-26T02:05:26.623 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.624 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.624 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.624 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-26T02:05:26.624 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.624 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.624 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.624 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.624 INFO:teuthology.orchestra.run.smithi151.stdout: "mode": { 2024-04-26T02:05:26.624 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "mode", 2024-04-26T02:05:26.624 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.624 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.624 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.624 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "upmap", 2024-04-26T02:05:26.624 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.624 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.625 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.625 INFO:teuthology.orchestra.run.smithi151.stdout: "crush-compat", 2024-04-26T02:05:26.625 INFO:teuthology.orchestra.run.smithi151.stdout: "none", 2024-04-26T02:05:26.625 INFO:teuthology.orchestra.run.smithi151.stdout: "read", 2024-04-26T02:05:26.625 INFO:teuthology.orchestra.run.smithi151.stdout: "upmap", 2024-04-26T02:05:26.625 INFO:teuthology.orchestra.run.smithi151.stdout: "upmap-read" 2024-04-26T02:05:26.625 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.625 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Balancer mode", 2024-04-26T02:05:26.625 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.625 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.625 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.625 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.625 INFO:teuthology.orchestra.run.smithi151.stdout: "pool_ids": { 2024-04-26T02:05:26.625 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "pool_ids", 2024-04-26T02:05:26.626 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.626 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.626 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.626 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.626 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.626 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.626 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.626 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-26T02:05:26.626 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.626 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.626 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.626 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.626 INFO:teuthology.orchestra.run.smithi151.stdout: "sleep_interval": { 2024-04-26T02:05:26.626 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sleep_interval", 2024-04-26T02:05:26.627 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-04-26T02:05:26.627 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.627 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.627 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "60", 2024-04-26T02:05:26.627 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.627 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.627 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.627 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-26T02:05:26.627 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.627 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.627 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.627 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.627 INFO:teuthology.orchestra.run.smithi151.stdout: "upmap_max_deviation": { 2024-04-26T02:05:26.627 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "upmap_max_deviation", 2024-04-26T02:05:26.628 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.628 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.628 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.628 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "5", 2024-04-26T02:05:26.628 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "1", 2024-04-26T02:05:26.628 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.628 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.628 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-26T02:05:26.628 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-26T02:05:26.628 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.628 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.628 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.628 INFO:teuthology.orchestra.run.smithi151.stdout: "upmap_max_optimizations": { 2024-04-26T02:05:26.628 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "upmap_max_optimizations", 2024-04-26T02:05:26.629 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "uint", 2024-04-26T02:05:26.629 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.629 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.629 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "10", 2024-04-26T02:05:26.629 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.629 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.629 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.629 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-26T02:05:26.629 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.629 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.629 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.629 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.629 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.629 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.630 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.630 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "cephadm", 2024-04-26T02:05:26.630 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.630 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.630 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.630 INFO:teuthology.orchestra.run.smithi151.stdout: "agent_down_multiplier": { 2024-04-26T02:05:26.630 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "agent_down_multiplier", 2024-04-26T02:05:26.630 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "float", 2024-04-26T02:05:26.630 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.630 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.630 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "3.0", 2024-04-26T02:05:26.630 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.630 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.631 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.631 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-26T02:05:26.631 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.631 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.631 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.631 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.631 INFO:teuthology.orchestra.run.smithi151.stdout: "agent_refresh_rate": { 2024-04-26T02:05:26.631 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "agent_refresh_rate", 2024-04-26T02:05:26.631 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-04-26T02:05:26.631 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.631 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.631 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "20", 2024-04-26T02:05:26.631 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.631 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.632 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.632 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-26T02:05:26.632 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.632 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.632 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.632 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.632 INFO:teuthology.orchestra.run.smithi151.stdout: "agent_starting_port": { 2024-04-26T02:05:26.632 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "agent_starting_port", 2024-04-26T02:05:26.632 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.632 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.632 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.632 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "4721", 2024-04-26T02:05:26.632 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.633 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.633 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.633 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-26T02:05:26.633 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.633 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.633 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.633 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.633 INFO:teuthology.orchestra.run.smithi151.stdout: "allow_ptrace": { 2024-04-26T02:05:26.633 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "allow_ptrace", 2024-04-26T02:05:26.633 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.633 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.633 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.633 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.633 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.634 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.634 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.634 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-26T02:05:26.634 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-26T02:05:26.634 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.634 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.634 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.634 INFO:teuthology.orchestra.run.smithi151.stdout: "autotune_interval": { 2024-04-26T02:05:26.634 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "autotune_interval", 2024-04-26T02:05:26.634 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-04-26T02:05:26.634 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.634 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.634 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "600", 2024-04-26T02:05:26.634 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.635 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.635 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.635 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-26T02:05:26.635 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.635 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.635 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.635 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.635 INFO:teuthology.orchestra.run.smithi151.stdout: "autotune_memory_target_ratio": { 2024-04-26T02:05:26.635 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "autotune_memory_target_ratio", 2024-04-26T02:05:26.635 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "float", 2024-04-26T02:05:26.635 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.635 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.635 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0.7", 2024-04-26T02:05:26.635 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.636 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.636 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.636 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-26T02:05:26.636 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.636 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.636 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.636 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.636 INFO:teuthology.orchestra.run.smithi151.stdout: "cephadm_log_destination": { 2024-04-26T02:05:26.636 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "cephadm_log_destination", 2024-04-26T02:05:26.636 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.636 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.636 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.636 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.636 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.637 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.637 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.637 INFO:teuthology.orchestra.run.smithi151.stdout: "file", 2024-04-26T02:05:26.637 INFO:teuthology.orchestra.run.smithi151.stdout: "file,syslog", 2024-04-26T02:05:26.637 INFO:teuthology.orchestra.run.smithi151.stdout: "syslog" 2024-04-26T02:05:26.637 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.637 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-26T02:05:26.637 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.637 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.637 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.637 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.637 INFO:teuthology.orchestra.run.smithi151.stdout: "cgroups_split": { 2024-04-26T02:05:26.637 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "cgroups_split", 2024-04-26T02:05:26.637 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.638 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.638 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.638 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.638 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.638 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.638 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.638 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-26T02:05:26.638 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.638 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.638 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.638 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.638 INFO:teuthology.orchestra.run.smithi151.stdout: "config_checks_enabled": { 2024-04-26T02:05:26.638 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "config_checks_enabled", 2024-04-26T02:05:26.639 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.639 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.639 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.639 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.639 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.639 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.639 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.639 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-26T02:05:26.639 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.639 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.639 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.639 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.639 INFO:teuthology.orchestra.run.smithi151.stdout: "config_dashboard": { 2024-04-26T02:05:26.640 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "config_dashboard", 2024-04-26T02:05:26.640 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.640 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.640 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.640 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.640 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.640 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.640 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.640 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-26T02:05:26.640 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.640 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.640 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.640 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.640 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_alertmanager": { 2024-04-26T02:05:26.641 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_alertmanager", 2024-04-26T02:05:26.641 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.641 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.641 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.641 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-26T02:05:26.641 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.641 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.641 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.641 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Prometheus container image", 2024-04-26T02:05:26.641 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.641 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.641 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.641 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.641 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_base": { 2024-04-26T02:05:26.642 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_base", 2024-04-26T02:05:26.642 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.642 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.642 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.642 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-26T02:05:26.642 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.642 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.642 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.642 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Container image name, without the tag", 2024-04-26T02:05:26.642 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.642 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.642 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.642 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.643 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_elasticsearch": { 2024-04-26T02:05:26.643 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_elasticsearch", 2024-04-26T02:05:26.643 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.643 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.643 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.643 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-26T02:05:26.643 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.643 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.643 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.643 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "elasticsearch container image", 2024-04-26T02:05:26.643 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.643 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.643 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.643 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.644 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_grafana": { 2024-04-26T02:05:26.644 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_grafana", 2024-04-26T02:05:26.644 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.644 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.644 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.644 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-26T02:05:26.644 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.644 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.644 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.644 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Prometheus container image", 2024-04-26T02:05:26.644 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.644 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.644 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.644 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.644 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_haproxy": { 2024-04-26T02:05:26.645 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_haproxy", 2024-04-26T02:05:26.645 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.645 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.645 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.645 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-26T02:05:26.645 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.645 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.645 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.645 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "HAproxy container image", 2024-04-26T02:05:26.645 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.645 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.645 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.645 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.645 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_jaeger_agent": { 2024-04-26T02:05:26.645 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_jaeger_agent", 2024-04-26T02:05:26.646 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.646 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.646 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.646 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-26T02:05:26.646 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.646 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.646 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.646 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Jaeger agent container image", 2024-04-26T02:05:26.646 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.646 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.646 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.646 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.646 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_jaeger_collector": { 2024-04-26T02:05:26.646 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_jaeger_collector", 2024-04-26T02:05:26.647 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.647 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.647 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.647 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-26T02:05:26.647 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.647 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.647 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.647 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Jaeger collector container image", 2024-04-26T02:05:26.647 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.647 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.647 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.647 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.647 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_jaeger_query": { 2024-04-26T02:05:26.647 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_jaeger_query", 2024-04-26T02:05:26.647 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.648 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.648 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.648 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-26T02:05:26.648 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.648 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.648 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.648 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Jaeger query container image", 2024-04-26T02:05:26.648 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.648 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.648 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.648 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.648 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_keepalived": { 2024-04-26T02:05:26.648 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_keepalived", 2024-04-26T02:05:26.648 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.648 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.648 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.649 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-26T02:05:26.649 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.649 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.649 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.649 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Keepalived container image", 2024-04-26T02:05:26.649 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.649 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.649 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.649 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.649 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_loki": { 2024-04-26T02:05:26.649 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_loki", 2024-04-26T02:05:26.649 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.649 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.649 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.649 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-26T02:05:26.650 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.650 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.650 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.650 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Loki container image", 2024-04-26T02:05:26.650 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.650 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.650 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.650 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.650 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_node_exporter": { 2024-04-26T02:05:26.650 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_node_exporter", 2024-04-26T02:05:26.650 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.650 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.650 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.651 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-26T02:05:26.651 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.651 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.651 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.651 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Prometheus container image", 2024-04-26T02:05:26.651 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.651 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.651 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.651 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.651 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_nvmeof": { 2024-04-26T02:05:26.651 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_nvmeof", 2024-04-26T02:05:26.651 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.651 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.651 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.651 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-26T02:05:26.652 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.652 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.652 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.652 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Nvme-of container image", 2024-04-26T02:05:26.652 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.652 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.652 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.652 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.652 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_prometheus": { 2024-04-26T02:05:26.652 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_prometheus", 2024-04-26T02:05:26.652 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.652 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.652 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.652 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-26T02:05:26.653 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.653 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.653 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.653 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Prometheus container image", 2024-04-26T02:05:26.653 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.653 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.653 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.653 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.653 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_promtail": { 2024-04-26T02:05:26.653 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_promtail", 2024-04-26T02:05:26.653 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.653 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.653 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.653 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-26T02:05:26.653 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.654 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.654 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.654 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Promtail container image", 2024-04-26T02:05:26.654 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.654 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.654 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.654 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.654 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_samba": { 2024-04-26T02:05:26.654 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_samba", 2024-04-26T02:05:26.654 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.654 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.654 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.654 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-26T02:05:26.655 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.655 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.655 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.655 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Samba/SMB container image", 2024-04-26T02:05:26.655 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.655 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.655 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.655 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.655 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_snmp_gateway": { 2024-04-26T02:05:26.655 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_snmp_gateway", 2024-04-26T02:05:26.655 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.655 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.655 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.656 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-26T02:05:26.656 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.656 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.656 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.656 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "SNMP Gateway container image", 2024-04-26T02:05:26.656 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.656 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.656 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.656 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.656 INFO:teuthology.orchestra.run.smithi151.stdout: "container_init": { 2024-04-26T02:05:26.656 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_init", 2024-04-26T02:05:26.656 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.656 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.656 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.656 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.657 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.657 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.657 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.657 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Run podman/docker with `--init`", 2024-04-26T02:05:26.657 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.657 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.657 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.657 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.657 INFO:teuthology.orchestra.run.smithi151.stdout: "daemon_cache_timeout": { 2024-04-26T02:05:26.657 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "daemon_cache_timeout", 2024-04-26T02:05:26.657 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-04-26T02:05:26.657 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.657 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.658 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "600", 2024-04-26T02:05:26.658 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.658 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.658 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.658 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-26T02:05:26.658 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.658 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.658 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.658 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.658 INFO:teuthology.orchestra.run.smithi151.stdout: "default_cephadm_command_timeout": { 2024-04-26T02:05:26.658 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "default_cephadm_command_timeout", 2024-04-26T02:05:26.658 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.658 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.658 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.659 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "900", 2024-04-26T02:05:26.659 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.659 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.659 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.659 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-26T02:05:26.659 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.659 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.659 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.659 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.659 INFO:teuthology.orchestra.run.smithi151.stdout: "default_registry": { 2024-04-26T02:05:26.659 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "default_registry", 2024-04-26T02:05:26.659 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.659 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.659 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.659 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "docker.io", 2024-04-26T02:05:26.660 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.660 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.660 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.660 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-26T02:05:26.660 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.660 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.660 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.660 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.660 INFO:teuthology.orchestra.run.smithi151.stdout: "device_cache_timeout": { 2024-04-26T02:05:26.660 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "device_cache_timeout", 2024-04-26T02:05:26.660 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-04-26T02:05:26.660 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.660 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.660 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "1800", 2024-04-26T02:05:26.660 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.661 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.661 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.661 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "seconds to cache device inventory", 2024-04-26T02:05:26.661 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.661 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.661 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.661 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.661 INFO:teuthology.orchestra.run.smithi151.stdout: "device_enhanced_scan": { 2024-04-26T02:05:26.661 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "device_enhanced_scan", 2024-04-26T02:05:26.661 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.661 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.661 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.661 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.661 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.662 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.662 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.662 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-26T02:05:26.662 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.662 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.662 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.662 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.662 INFO:teuthology.orchestra.run.smithi151.stdout: "facts_cache_timeout": { 2024-04-26T02:05:26.662 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "facts_cache_timeout", 2024-04-26T02:05:26.662 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-04-26T02:05:26.662 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.662 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.663 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "60", 2024-04-26T02:05:26.663 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.663 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.663 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.663 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "seconds to cache host facts data", 2024-04-26T02:05:26.663 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.663 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.663 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.663 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.663 INFO:teuthology.orchestra.run.smithi151.stdout: "grafana_dashboards_path": { 2024-04-26T02:05:26.663 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "grafana_dashboards_path", 2024-04-26T02:05:26.663 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.663 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.663 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.663 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-26T02:05:26.664 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.664 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.664 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.664 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-26T02:05:26.664 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.664 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.664 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.664 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.664 INFO:teuthology.orchestra.run.smithi151.stdout: "host_check_interval": { 2024-04-26T02:05:26.664 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "host_check_interval", 2024-04-26T02:05:26.664 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-04-26T02:05:26.664 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.664 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.665 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "600", 2024-04-26T02:05:26.665 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.665 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.665 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.665 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "how frequently to perform a host check", 2024-04-26T02:05:26.665 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.665 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.665 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.665 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.665 INFO:teuthology.orchestra.run.smithi151.stdout: "hw_monitoring": { 2024-04-26T02:05:26.665 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "hw_monitoring", 2024-04-26T02:05:26.665 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.665 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.665 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.666 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.666 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.666 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.666 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.666 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-26T02:05:26.666 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.666 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.666 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.666 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.666 INFO:teuthology.orchestra.run.smithi151.stdout: "inventory_list_all": { 2024-04-26T02:05:26.666 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "inventory_list_all", 2024-04-26T02:05:26.666 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.666 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.666 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.667 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.667 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.667 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.667 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.667 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-26T02:05:26.667 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.667 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.667 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.667 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.667 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.667 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.667 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.667 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.668 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.668 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.668 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.668 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.668 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.668 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.668 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.668 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.668 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.668 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.668 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.668 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.668 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.668 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.669 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.669 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.669 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.669 INFO:teuthology.orchestra.run.smithi151.stdout: "log_refresh_metadata": { 2024-04-26T02:05:26.669 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_refresh_metadata", 2024-04-26T02:05:26.669 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.669 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.669 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.669 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.669 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.669 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.669 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.669 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-26T02:05:26.669 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.670 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.670 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.670 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.670 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.670 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.670 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.670 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.670 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.670 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.670 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.670 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.670 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.671 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-26T02:05:26.671 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.671 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.671 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.671 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.671 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.671 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.671 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.671 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.671 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.671 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.671 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.671 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.671 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.672 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.672 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.672 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.672 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.672 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.672 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.672 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.672 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.672 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.672 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.672 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.672 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.672 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.672 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.672 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.673 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.673 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.673 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.673 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.673 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.673 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.673 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.673 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.673 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.673 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.673 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.673 INFO:teuthology.orchestra.run.smithi151.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-26T02:05:26.673 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-26T02:05:26.673 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.674 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.674 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.674 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.674 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.674 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.674 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.674 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-26T02:05:26.674 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.674 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.674 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.674 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.674 INFO:teuthology.orchestra.run.smithi151.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-26T02:05:26.674 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-26T02:05:26.674 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.675 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.675 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.675 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "*", 2024-04-26T02:05:26.675 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.675 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.675 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.675 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-26T02:05:26.675 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.675 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.675 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.675 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.675 INFO:teuthology.orchestra.run.smithi151.stdout: "max_count_per_host": { 2024-04-26T02:05:26.675 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "max_count_per_host", 2024-04-26T02:05:26.676 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.676 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.676 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.676 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "10", 2024-04-26T02:05:26.676 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.676 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.676 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.676 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "max number of daemons per service per host", 2024-04-26T02:05:26.676 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.676 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.676 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.676 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.676 INFO:teuthology.orchestra.run.smithi151.stdout: "max_osd_draining_count": { 2024-04-26T02:05:26.677 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "max_osd_draining_count", 2024-04-26T02:05:26.677 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.677 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.677 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.677 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "10", 2024-04-26T02:05:26.677 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.677 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.677 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.677 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-26T02:05:26.677 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.677 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.677 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.677 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.678 INFO:teuthology.orchestra.run.smithi151.stdout: "migration_current": { 2024-04-26T02:05:26.678 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "migration_current", 2024-04-26T02:05:26.678 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.678 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.678 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.678 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.678 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.678 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.678 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.678 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "internal - do not modify", 2024-04-26T02:05:26.678 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.678 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.678 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.678 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.679 INFO:teuthology.orchestra.run.smithi151.stdout: "mode": { 2024-04-26T02:05:26.679 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "mode", 2024-04-26T02:05:26.679 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.679 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.679 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.679 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "root", 2024-04-26T02:05:26.679 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.679 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.679 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.679 INFO:teuthology.orchestra.run.smithi151.stdout: "cephadm-package", 2024-04-26T02:05:26.679 INFO:teuthology.orchestra.run.smithi151.stdout: "root" 2024-04-26T02:05:26.679 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.679 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "mode for remote execution of cephadm", 2024-04-26T02:05:26.679 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.679 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.680 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.680 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.680 INFO:teuthology.orchestra.run.smithi151.stdout: "oob_default_addr": { 2024-04-26T02:05:26.680 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "oob_default_addr", 2024-04-26T02:05:26.680 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.680 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.680 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.680 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "169.254.1.1", 2024-04-26T02:05:26.680 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.680 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.680 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.680 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-26T02:05:26.681 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.681 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.681 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.681 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.681 INFO:teuthology.orchestra.run.smithi151.stdout: "prometheus_alerts_path": { 2024-04-26T02:05:26.681 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "prometheus_alerts_path", 2024-04-26T02:05:26.681 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.681 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.681 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.681 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-26T02:05:26.681 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.681 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.681 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.682 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-26T02:05:26.682 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.682 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.682 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.682 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.682 INFO:teuthology.orchestra.run.smithi151.stdout: "registry_insecure": { 2024-04-26T02:05:26.682 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "registry_insecure", 2024-04-26T02:05:26.682 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.682 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.682 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.682 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.682 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.682 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.682 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.683 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-26T02:05:26.683 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.683 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.683 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.683 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.683 INFO:teuthology.orchestra.run.smithi151.stdout: "registry_password": { 2024-04-26T02:05:26.683 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "registry_password", 2024-04-26T02:05:26.683 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.683 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.683 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.683 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.683 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.683 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.683 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.683 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-26T02:05:26.684 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.684 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.684 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.684 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.684 INFO:teuthology.orchestra.run.smithi151.stdout: "registry_url": { 2024-04-26T02:05:26.684 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "registry_url", 2024-04-26T02:05:26.684 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.684 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.684 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.684 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.684 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.684 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.684 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.684 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-26T02:05:26.685 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.685 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.685 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.685 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.685 INFO:teuthology.orchestra.run.smithi151.stdout: "registry_username": { 2024-04-26T02:05:26.685 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "registry_username", 2024-04-26T02:05:26.685 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.685 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.685 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.685 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.685 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.685 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.685 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.685 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-26T02:05:26.685 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.686 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.686 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.686 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.686 INFO:teuthology.orchestra.run.smithi151.stdout: "secure_monitoring_stack": { 2024-04-26T02:05:26.686 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "secure_monitoring_stack", 2024-04-26T02:05:26.686 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.686 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.686 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.686 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.686 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.686 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.686 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.686 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-26T02:05:26.687 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.687 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.687 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.687 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.687 INFO:teuthology.orchestra.run.smithi151.stdout: "service_discovery_port": { 2024-04-26T02:05:26.687 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "service_discovery_port", 2024-04-26T02:05:26.687 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.687 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.687 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.687 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "8765", 2024-04-26T02:05:26.687 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.687 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.687 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.687 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "cephadm service discovery port", 2024-04-26T02:05:26.688 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.688 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.688 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.688 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.688 INFO:teuthology.orchestra.run.smithi151.stdout: "ssh_config_file": { 2024-04-26T02:05:26.688 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ssh_config_file", 2024-04-26T02:05:26.688 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.688 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.688 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.688 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.688 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.688 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.688 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.688 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-26T02:05:26.688 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.689 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.689 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.689 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.689 INFO:teuthology.orchestra.run.smithi151.stdout: "use_agent": { 2024-04-26T02:05:26.689 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "use_agent", 2024-04-26T02:05:26.689 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.689 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.689 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.689 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.689 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.689 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.689 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.689 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-26T02:05:26.689 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.690 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.690 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.690 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.690 INFO:teuthology.orchestra.run.smithi151.stdout: "use_repo_digest": { 2024-04-26T02:05:26.690 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "use_repo_digest", 2024-04-26T02:05:26.690 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.690 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.690 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.690 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.690 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.690 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.690 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.690 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-26T02:05:26.690 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.691 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.691 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.691 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.691 INFO:teuthology.orchestra.run.smithi151.stdout: "warn_on_failed_host_check": { 2024-04-26T02:05:26.691 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "warn_on_failed_host_check", 2024-04-26T02:05:26.691 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.691 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.691 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.691 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.691 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.691 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.691 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.691 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "raise a health warning if the host check fails", 2024-04-26T02:05:26.691 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.692 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.692 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.692 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.692 INFO:teuthology.orchestra.run.smithi151.stdout: "warn_on_stray_daemons": { 2024-04-26T02:05:26.692 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "warn_on_stray_daemons", 2024-04-26T02:05:26.692 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.692 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.692 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.692 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.692 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.692 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.692 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.692 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-26T02:05:26.693 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.693 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.693 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.693 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.693 INFO:teuthology.orchestra.run.smithi151.stdout: "warn_on_stray_hosts": { 2024-04-26T02:05:26.693 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "warn_on_stray_hosts", 2024-04-26T02:05:26.693 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.693 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.693 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.693 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.693 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.693 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.693 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.694 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-26T02:05:26.694 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.694 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.694 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.694 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.694 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.694 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.694 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.694 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "crash", 2024-04-26T02:05:26.694 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.694 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.694 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.694 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.694 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.695 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.695 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.695 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.695 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.695 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.695 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.695 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.695 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.695 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.695 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.695 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.695 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.695 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.695 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.696 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.696 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.696 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.696 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.696 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.696 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.696 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.696 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.696 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.696 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.696 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.696 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.696 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.696 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.697 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.697 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.697 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.697 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.697 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.697 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.697 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.697 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.697 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.697 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.697 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.697 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.697 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.697 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.698 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.698 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.698 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.698 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.698 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.698 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.698 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.698 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.698 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.698 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.698 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.698 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.698 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.699 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.699 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.699 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.699 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.699 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.699 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.699 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.699 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.699 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.699 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.699 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.699 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.699 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.699 INFO:teuthology.orchestra.run.smithi151.stdout: "retain_interval": { 2024-04-26T02:05:26.699 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "retain_interval", 2024-04-26T02:05:26.700 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-04-26T02:05:26.700 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.700 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.700 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "31536000", 2024-04-26T02:05:26.700 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.700 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.700 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.700 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-26T02:05:26.700 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.700 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.700 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.700 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.700 INFO:teuthology.orchestra.run.smithi151.stdout: "warn_recent_interval": { 2024-04-26T02:05:26.700 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "warn_recent_interval", 2024-04-26T02:05:26.700 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-04-26T02:05:26.701 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.701 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.701 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "1209600", 2024-04-26T02:05:26.701 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.701 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.701 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.701 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-26T02:05:26.701 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.701 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.701 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.701 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.701 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.701 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.701 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.702 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "dashboard", 2024-04-26T02:05:26.702 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.702 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.702 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.702 INFO:teuthology.orchestra.run.smithi151.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-26T02:05:26.702 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-26T02:05:26.702 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.702 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.702 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.702 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "10", 2024-04-26T02:05:26.702 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.702 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.702 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.702 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.703 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.703 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.703 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.703 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.703 INFO:teuthology.orchestra.run.smithi151.stdout: "ALERTMANAGER_API_HOST": { 2024-04-26T02:05:26.703 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-26T02:05:26.703 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.703 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.703 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.703 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.703 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.703 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.703 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.703 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.703 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.704 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.704 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.704 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.704 INFO:teuthology.orchestra.run.smithi151.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-26T02:05:26.704 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-26T02:05:26.704 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.704 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.704 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.704 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.704 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.704 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.704 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.704 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.704 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.705 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.705 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.705 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.705 INFO:teuthology.orchestra.run.smithi151.stdout: "AUDIT_API_ENABLED": { 2024-04-26T02:05:26.705 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "AUDIT_API_ENABLED", 2024-04-26T02:05:26.705 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.705 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.705 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.705 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.705 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.705 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.705 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.705 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.705 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.706 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.706 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.706 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.706 INFO:teuthology.orchestra.run.smithi151.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-26T02:05:26.706 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-26T02:05:26.706 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.706 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.706 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.706 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.706 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.706 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.706 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.706 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.707 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.707 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.707 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.707 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.707 INFO:teuthology.orchestra.run.smithi151.stdout: "ENABLE_BROWSABLE_API": { 2024-04-26T02:05:26.707 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-26T02:05:26.707 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.707 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.707 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.707 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.707 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.707 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.707 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.707 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.707 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.708 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.708 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.708 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.708 INFO:teuthology.orchestra.run.smithi151.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-26T02:05:26.708 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-26T02:05:26.708 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.708 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.708 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.708 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.708 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.708 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.708 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.708 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.708 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.709 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.709 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.709 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.709 INFO:teuthology.orchestra.run.smithi151.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-26T02:05:26.709 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-26T02:05:26.709 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.709 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.709 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.709 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.709 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.709 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.709 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.709 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.709 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.710 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.710 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.710 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.710 INFO:teuthology.orchestra.run.smithi151.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-26T02:05:26.710 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-26T02:05:26.710 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.710 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.710 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.710 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.710 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.710 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.710 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.710 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.711 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.711 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.711 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.711 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.711 INFO:teuthology.orchestra.run.smithi151.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-26T02:05:26.711 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-26T02:05:26.711 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.711 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.711 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.711 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.711 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.711 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.711 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.711 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.712 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.712 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.712 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.712 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.712 INFO:teuthology.orchestra.run.smithi151.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-26T02:05:26.712 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-26T02:05:26.712 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.712 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.712 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.712 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.712 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.712 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.712 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.712 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.712 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.713 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.713 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.713 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.713 INFO:teuthology.orchestra.run.smithi151.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-26T02:05:26.713 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-26T02:05:26.713 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.713 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.713 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.713 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.713 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.713 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.713 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.713 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.714 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.714 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.714 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.714 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.714 INFO:teuthology.orchestra.run.smithi151.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-26T02:05:26.714 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-26T02:05:26.714 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.714 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.714 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.714 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.714 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.714 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.714 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.714 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.715 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.715 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.715 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.715 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.715 INFO:teuthology.orchestra.run.smithi151.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-26T02:05:26.715 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-26T02:05:26.715 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.715 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.715 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.715 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.715 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.715 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.715 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.715 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.715 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.716 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.716 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.716 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.716 INFO:teuthology.orchestra.run.smithi151.stdout: "GRAFANA_API_PASSWORD": { 2024-04-26T02:05:26.716 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-26T02:05:26.716 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.716 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.716 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.716 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "admin", 2024-04-26T02:05:26.716 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.716 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.716 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.716 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.716 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.716 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.717 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.717 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.717 INFO:teuthology.orchestra.run.smithi151.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-26T02:05:26.717 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-26T02:05:26.717 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.717 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.717 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.717 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.717 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.717 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.717 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.717 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.717 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.717 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.717 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.718 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.718 INFO:teuthology.orchestra.run.smithi151.stdout: "GRAFANA_API_URL": { 2024-04-26T02:05:26.718 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "GRAFANA_API_URL", 2024-04-26T02:05:26.718 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.718 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.718 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.718 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.718 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.718 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.718 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.718 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.718 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.718 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.718 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.719 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.719 INFO:teuthology.orchestra.run.smithi151.stdout: "GRAFANA_API_USERNAME": { 2024-04-26T02:05:26.719 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-26T02:05:26.719 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.719 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.719 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.719 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "admin", 2024-04-26T02:05:26.719 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.719 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.719 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.719 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.719 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.719 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.720 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.720 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.720 INFO:teuthology.orchestra.run.smithi151.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-26T02:05:26.720 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-26T02:05:26.720 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.720 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.720 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.720 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.720 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.720 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.720 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.720 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.720 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.720 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.720 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.721 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.721 INFO:teuthology.orchestra.run.smithi151.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-26T02:05:26.721 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-26T02:05:26.721 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.721 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.721 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.721 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.721 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.721 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.721 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.721 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.721 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.721 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.722 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.722 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.722 INFO:teuthology.orchestra.run.smithi151.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-26T02:05:26.722 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-26T02:05:26.722 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.722 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.722 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.722 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.722 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.722 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.722 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.722 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.722 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.722 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.723 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.723 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.723 INFO:teuthology.orchestra.run.smithi151.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-26T02:05:26.723 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-26T02:05:26.723 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.723 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.723 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.723 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.723 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.723 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.723 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.723 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.723 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.723 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.724 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.724 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.724 INFO:teuthology.orchestra.run.smithi151.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-26T02:05:26.724 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-26T02:05:26.724 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.724 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.724 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.724 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "[]", 2024-04-26T02:05:26.724 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.724 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.724 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.724 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.724 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.724 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.724 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.725 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.725 INFO:teuthology.orchestra.run.smithi151.stdout: "MULTICLUSTER_CONFIG": { 2024-04-26T02:05:26.725 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-26T02:05:26.725 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.725 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.725 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.725 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "{}", 2024-04-26T02:05:26.725 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.725 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.725 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.725 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.725 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.725 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.725 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.725 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.726 INFO:teuthology.orchestra.run.smithi151.stdout: "PROMETHEUS_API_HOST": { 2024-04-26T02:05:26.726 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-26T02:05:26.726 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.726 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.726 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.726 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.726 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.726 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.726 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.726 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.726 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.726 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.726 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.726 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.727 INFO:teuthology.orchestra.run.smithi151.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-26T02:05:26.727 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-26T02:05:26.727 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.727 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.727 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.727 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.727 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.727 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.727 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.727 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.727 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.727 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.727 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.727 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.727 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-26T02:05:26.728 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-26T02:05:26.728 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.728 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.728 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.728 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.728 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.728 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.728 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.728 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.728 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.728 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.728 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.728 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.728 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-26T02:05:26.729 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-26T02:05:26.729 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.729 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.729 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.729 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.729 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.729 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.729 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.729 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.729 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.729 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.729 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.729 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.729 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-26T02:05:26.730 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-26T02:05:26.730 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.730 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.730 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.730 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.730 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.730 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.730 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.730 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.730 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.730 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.730 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.730 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.731 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-26T02:05:26.731 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-26T02:05:26.731 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.731 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.731 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.731 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.731 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.731 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.731 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.731 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.731 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.731 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.731 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.731 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.732 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-26T02:05:26.732 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-26T02:05:26.732 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.732 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.732 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.732 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.732 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.732 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.732 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.732 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.732 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.732 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.732 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.732 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.733 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-26T02:05:26.733 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-26T02:05:26.733 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.733 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.733 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.733 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.733 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.733 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.733 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.733 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.733 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.733 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.733 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.733 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.734 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-26T02:05:26.734 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-26T02:05:26.734 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.734 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.734 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.734 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.734 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.734 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.734 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.734 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.734 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.734 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.734 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.734 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.735 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_ENABLED": { 2024-04-26T02:05:26.735 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-26T02:05:26.735 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.735 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.735 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.735 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.735 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.735 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.735 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.735 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.735 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.735 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.735 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.735 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.736 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-26T02:05:26.736 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-26T02:05:26.736 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.736 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.736 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.736 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-26T02:05:26.736 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.736 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.736 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.736 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.736 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.736 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.736 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.736 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.737 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-26T02:05:26.737 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-26T02:05:26.737 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.737 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.737 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.737 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "10", 2024-04-26T02:05:26.737 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.737 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.737 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.737 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.737 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.737 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.737 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.737 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.738 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-26T02:05:26.738 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-26T02:05:26.738 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.738 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.738 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.738 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "8", 2024-04-26T02:05:26.738 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.738 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.738 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.738 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.738 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.738 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.738 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.738 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.739 INFO:teuthology.orchestra.run.smithi151.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-26T02:05:26.739 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-26T02:05:26.739 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.739 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.739 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.739 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "45", 2024-04-26T02:05:26.739 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.739 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.739 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.739 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.739 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.739 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.739 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.739 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.740 INFO:teuthology.orchestra.run.smithi151.stdout: "RGW_API_ACCESS_KEY": { 2024-04-26T02:05:26.740 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-26T02:05:26.740 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.740 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.740 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.740 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.740 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.740 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.740 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.740 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.740 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.740 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.740 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.740 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.741 INFO:teuthology.orchestra.run.smithi151.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-26T02:05:26.741 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-26T02:05:26.741 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.741 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.741 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.741 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "admin", 2024-04-26T02:05:26.741 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.741 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.741 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.741 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.741 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.741 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.741 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.741 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.741 INFO:teuthology.orchestra.run.smithi151.stdout: "RGW_API_SECRET_KEY": { 2024-04-26T02:05:26.741 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-26T02:05:26.742 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.742 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.742 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.742 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.742 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.742 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.742 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.742 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.742 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.742 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.742 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.742 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.742 INFO:teuthology.orchestra.run.smithi151.stdout: "RGW_API_SSL_VERIFY": { 2024-04-26T02:05:26.743 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-26T02:05:26.743 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.743 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.743 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.743 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.743 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.743 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.743 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.743 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.743 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.743 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.743 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.743 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.743 INFO:teuthology.orchestra.run.smithi151.stdout: "UNSAFE_TLS_v1_2": { 2024-04-26T02:05:26.743 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-26T02:05:26.744 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.744 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.744 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.744 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.744 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.744 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.744 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.744 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.744 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.744 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.744 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.744 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.744 INFO:teuthology.orchestra.run.smithi151.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-26T02:05:26.744 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-26T02:05:26.744 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.744 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.745 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.745 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-04-26T02:05:26.745 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.745 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.745 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.745 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.745 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.745 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.745 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.745 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.745 INFO:teuthology.orchestra.run.smithi151.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-26T02:05:26.745 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-26T02:05:26.745 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.745 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.745 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.746 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "10", 2024-04-26T02:05:26.746 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.746 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.746 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.746 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.746 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.746 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.746 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.746 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.746 INFO:teuthology.orchestra.run.smithi151.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-26T02:05:26.746 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-26T02:05:26.746 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.746 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.746 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.747 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "5", 2024-04-26T02:05:26.747 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.747 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.747 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.747 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.747 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.747 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.747 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.747 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.747 INFO:teuthology.orchestra.run.smithi151.stdout: "cross_origin_url": { 2024-04-26T02:05:26.747 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "cross_origin_url", 2024-04-26T02:05:26.747 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.747 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.748 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.748 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.748 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.748 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.748 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.748 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.748 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.748 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.748 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.748 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.748 INFO:teuthology.orchestra.run.smithi151.stdout: "crt_file": { 2024-04-26T02:05:26.748 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "crt_file", 2024-04-26T02:05:26.748 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.748 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.748 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.749 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.749 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.749 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.749 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.749 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.749 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.749 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.749 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.749 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.749 INFO:teuthology.orchestra.run.smithi151.stdout: "debug": { 2024-04-26T02:05:26.749 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "debug", 2024-04-26T02:05:26.749 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.749 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.750 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.750 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.750 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.750 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.750 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.750 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Enable/disable debug options", 2024-04-26T02:05:26.750 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.750 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.750 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.750 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.750 INFO:teuthology.orchestra.run.smithi151.stdout: "jwt_token_ttl": { 2024-04-26T02:05:26.750 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "jwt_token_ttl", 2024-04-26T02:05:26.750 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.750 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.751 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.751 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "28800", 2024-04-26T02:05:26.751 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.751 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.751 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.751 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.751 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.751 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.751 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.751 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.751 INFO:teuthology.orchestra.run.smithi151.stdout: "key_file": { 2024-04-26T02:05:26.751 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "key_file", 2024-04-26T02:05:26.751 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.751 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.751 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.752 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.752 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.752 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.752 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.752 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.752 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.752 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.752 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.752 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.752 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.752 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.752 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.752 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.752 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.752 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.753 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.753 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.753 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.753 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.753 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.753 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.753 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.753 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.753 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.753 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.753 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.753 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.753 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.753 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.753 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.754 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.754 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.754 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.754 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.754 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.754 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.754 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.754 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.754 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.754 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.754 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.754 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.754 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.754 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.755 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.755 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.755 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.755 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.755 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.755 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.755 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.755 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.755 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.755 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.755 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.755 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.755 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.755 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.756 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.756 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.756 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.756 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.756 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.756 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.756 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.756 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.756 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.756 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.756 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.756 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.756 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.756 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.756 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.757 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.757 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.757 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.757 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.757 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.757 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.757 INFO:teuthology.orchestra.run.smithi151.stdout: "motd": { 2024-04-26T02:05:26.757 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "motd", 2024-04-26T02:05:26.757 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.757 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.757 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.757 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.757 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.757 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.758 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.758 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "The message of the day", 2024-04-26T02:05:26.758 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.758 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.758 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.758 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.758 INFO:teuthology.orchestra.run.smithi151.stdout: "redirect_resolve_ip_addr": { 2024-04-26T02:05:26.758 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "redirect_resolve_ip_addr", 2024-04-26T02:05:26.758 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.758 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.758 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.758 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.759 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.759 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.759 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.759 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.759 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.759 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.759 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.759 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.759 INFO:teuthology.orchestra.run.smithi151.stdout: "server_addr": { 2024-04-26T02:05:26.759 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "server_addr", 2024-04-26T02:05:26.759 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.759 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.759 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.759 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "::", 2024-04-26T02:05:26.760 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.760 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.760 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.760 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.760 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.760 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.760 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.760 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.760 INFO:teuthology.orchestra.run.smithi151.stdout: "server_port": { 2024-04-26T02:05:26.760 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "server_port", 2024-04-26T02:05:26.760 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.760 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.760 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.761 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "8080", 2024-04-26T02:05:26.761 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.761 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.761 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.761 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.761 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.761 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.761 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.761 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.761 INFO:teuthology.orchestra.run.smithi151.stdout: "ssl": { 2024-04-26T02:05:26.761 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ssl", 2024-04-26T02:05:26.761 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.761 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.761 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.762 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.762 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.762 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.762 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.762 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.762 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.762 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.762 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.762 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.762 INFO:teuthology.orchestra.run.smithi151.stdout: "ssl_server_port": { 2024-04-26T02:05:26.762 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ssl_server_port", 2024-04-26T02:05:26.762 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.762 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.763 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.763 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "8443", 2024-04-26T02:05:26.763 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.763 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.763 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.763 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.763 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.763 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.763 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.763 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.763 INFO:teuthology.orchestra.run.smithi151.stdout: "standby_behaviour": { 2024-04-26T02:05:26.763 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "standby_behaviour", 2024-04-26T02:05:26.763 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.763 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.764 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.764 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "redirect", 2024-04-26T02:05:26.764 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.764 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.764 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.764 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.764 INFO:teuthology.orchestra.run.smithi151.stdout: "redirect" 2024-04-26T02:05:26.764 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.764 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.764 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.764 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.764 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.764 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.764 INFO:teuthology.orchestra.run.smithi151.stdout: "standby_error_status_code": { 2024-04-26T02:05:26.765 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "standby_error_status_code", 2024-04-26T02:05:26.765 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.765 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.765 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.765 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "500", 2024-04-26T02:05:26.765 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "400", 2024-04-26T02:05:26.765 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "599", 2024-04-26T02:05:26.765 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.765 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.765 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.765 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.765 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.765 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.766 INFO:teuthology.orchestra.run.smithi151.stdout: "url_prefix": { 2024-04-26T02:05:26.766 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "url_prefix", 2024-04-26T02:05:26.766 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.766 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.766 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.766 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.766 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.766 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.766 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.766 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.766 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.766 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.766 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.767 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.767 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.767 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.767 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.767 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "devicehealth", 2024-04-26T02:05:26.767 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.767 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.767 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.767 INFO:teuthology.orchestra.run.smithi151.stdout: "enable_monitoring": { 2024-04-26T02:05:26.767 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "enable_monitoring", 2024-04-26T02:05:26.767 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.767 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.767 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.768 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.768 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.768 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.768 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.768 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "monitor device health metrics", 2024-04-26T02:05:26.768 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.768 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.768 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.768 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.768 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.768 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.768 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.768 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.768 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.769 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.769 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.769 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.769 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.769 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.769 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.769 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.769 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.769 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.769 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.769 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.769 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.769 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.769 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.770 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.770 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.770 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.770 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.770 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.770 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.770 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.770 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.770 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.770 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.770 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.770 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.770 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.770 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.771 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.771 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.771 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.771 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.771 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.771 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.771 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.771 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.771 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.771 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.771 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.771 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.771 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.771 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.772 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.772 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.772 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.772 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.772 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.772 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.772 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.772 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.772 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.772 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.772 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.772 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.773 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.773 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.773 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.773 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.773 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.773 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.773 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.773 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.773 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.773 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.773 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.773 INFO:teuthology.orchestra.run.smithi151.stdout: "mark_out_threshold": { 2024-04-26T02:05:26.773 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "mark_out_threshold", 2024-04-26T02:05:26.773 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-04-26T02:05:26.774 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.774 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.774 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "2419200", 2024-04-26T02:05:26.774 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.774 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.774 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.774 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-26T02:05:26.774 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.774 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.774 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.774 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.774 INFO:teuthology.orchestra.run.smithi151.stdout: "pool_name": { 2024-04-26T02:05:26.774 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "pool_name", 2024-04-26T02:05:26.775 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.775 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.775 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.775 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "device_health_metrics", 2024-04-26T02:05:26.775 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.775 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.775 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.775 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-26T02:05:26.775 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.775 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.775 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.775 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.775 INFO:teuthology.orchestra.run.smithi151.stdout: "retention_period": { 2024-04-26T02:05:26.775 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "retention_period", 2024-04-26T02:05:26.775 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-04-26T02:05:26.776 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.776 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.776 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "15552000", 2024-04-26T02:05:26.776 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.776 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.776 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.776 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "how long to retain device health metrics", 2024-04-26T02:05:26.776 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.776 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.776 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.776 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.776 INFO:teuthology.orchestra.run.smithi151.stdout: "scrape_frequency": { 2024-04-26T02:05:26.776 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "scrape_frequency", 2024-04-26T02:05:26.776 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-04-26T02:05:26.776 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.777 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.777 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "86400", 2024-04-26T02:05:26.777 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.777 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.777 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.777 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-26T02:05:26.777 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.777 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.777 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.777 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.777 INFO:teuthology.orchestra.run.smithi151.stdout: "self_heal": { 2024-04-26T02:05:26.777 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "self_heal", 2024-04-26T02:05:26.777 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.777 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.777 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.778 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.778 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.778 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.778 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.778 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-26T02:05:26.778 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.778 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.778 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.778 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.778 INFO:teuthology.orchestra.run.smithi151.stdout: "sleep_interval": { 2024-04-26T02:05:26.778 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sleep_interval", 2024-04-26T02:05:26.778 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-04-26T02:05:26.778 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.778 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.778 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "600", 2024-04-26T02:05:26.779 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.779 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.779 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.779 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "how frequently to wake up and check device health", 2024-04-26T02:05:26.779 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.779 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.779 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.779 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.779 INFO:teuthology.orchestra.run.smithi151.stdout: "warn_threshold": { 2024-04-26T02:05:26.779 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "warn_threshold", 2024-04-26T02:05:26.779 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-04-26T02:05:26.779 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.779 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.779 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "7257600", 2024-04-26T02:05:26.779 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.779 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.780 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.780 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-26T02:05:26.780 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.780 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.780 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.780 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.780 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.780 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.780 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.780 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "diskprediction_local", 2024-04-26T02:05:26.780 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.780 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.780 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.781 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.781 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.781 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.781 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.781 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.781 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.781 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.781 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.781 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.781 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.781 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.781 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.781 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.781 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.781 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.782 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.782 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.782 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.782 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.782 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.782 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.782 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.782 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.782 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.782 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.782 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.782 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.782 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.783 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.783 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.783 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.783 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.783 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.783 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.783 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.783 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.783 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.783 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.783 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.783 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.783 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.783 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.783 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.784 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.784 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.784 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.784 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.784 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.784 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.784 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.784 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.784 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.784 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.784 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.784 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.784 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.784 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.785 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.785 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.785 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.785 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.785 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.785 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.785 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.785 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.785 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.785 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.785 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.785 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.785 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.786 INFO:teuthology.orchestra.run.smithi151.stdout: "predict_interval": { 2024-04-26T02:05:26.786 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "predict_interval", 2024-04-26T02:05:26.786 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.786 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.786 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.786 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "86400", 2024-04-26T02:05:26.786 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.786 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.786 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.786 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.786 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.786 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.787 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.787 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.787 INFO:teuthology.orchestra.run.smithi151.stdout: "predictor_model": { 2024-04-26T02:05:26.787 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "predictor_model", 2024-04-26T02:05:26.787 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.787 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.787 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.787 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "prophetstor", 2024-04-26T02:05:26.787 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.787 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.787 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.787 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.787 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.788 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.788 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.788 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.788 INFO:teuthology.orchestra.run.smithi151.stdout: "sleep_interval": { 2024-04-26T02:05:26.788 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sleep_interval", 2024-04-26T02:05:26.788 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.788 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.788 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.788 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "600", 2024-04-26T02:05:26.788 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.788 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.788 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.788 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.788 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.788 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.789 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.789 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.789 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.789 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.789 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.789 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "influx", 2024-04-26T02:05:26.789 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": false, 2024-04-26T02:05:26.789 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "influxdb python module not found", 2024-04-26T02:05:26.789 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.789 INFO:teuthology.orchestra.run.smithi151.stdout: "batch_size": { 2024-04-26T02:05:26.789 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "batch_size", 2024-04-26T02:05:26.789 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.789 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.790 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.790 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "5000", 2024-04-26T02:05:26.790 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.790 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.790 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.790 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-26T02:05:26.790 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.790 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.790 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.790 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.790 INFO:teuthology.orchestra.run.smithi151.stdout: "database": { 2024-04-26T02:05:26.790 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "database", 2024-04-26T02:05:26.790 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.791 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.791 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.791 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "ceph", 2024-04-26T02:05:26.791 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.791 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.791 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.791 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-26T02:05:26.791 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.791 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.791 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.791 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.791 INFO:teuthology.orchestra.run.smithi151.stdout: "hostname": { 2024-04-26T02:05:26.791 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "hostname", 2024-04-26T02:05:26.791 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.792 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.792 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.792 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.792 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.792 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.792 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.792 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "InfluxDB server hostname", 2024-04-26T02:05:26.792 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.792 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.792 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.792 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.792 INFO:teuthology.orchestra.run.smithi151.stdout: "interval": { 2024-04-26T02:05:26.792 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "interval", 2024-04-26T02:05:26.792 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-04-26T02:05:26.792 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.792 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.793 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "30", 2024-04-26T02:05:26.793 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "5", 2024-04-26T02:05:26.793 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.793 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.793 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-26T02:05:26.793 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.793 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.793 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.793 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.793 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.793 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.793 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.793 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.794 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.794 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.794 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.794 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.794 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.794 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.794 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.794 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.794 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.794 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.794 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.794 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.794 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.794 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.795 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.795 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.795 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.795 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.795 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.795 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.795 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.795 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.795 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.795 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.795 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.795 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.795 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.795 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.796 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.796 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.796 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.796 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.796 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.796 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.796 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.796 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.796 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.796 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.796 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.796 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.796 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.796 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.797 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.797 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.797 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.797 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.797 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.797 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.797 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.797 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.797 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.797 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.797 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.797 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.797 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.797 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.798 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.798 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.798 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.798 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.798 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.798 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.798 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.798 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.798 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.798 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.798 INFO:teuthology.orchestra.run.smithi151.stdout: "password": { 2024-04-26T02:05:26.798 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "password", 2024-04-26T02:05:26.799 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.799 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.799 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.799 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.799 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.799 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.799 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.799 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "password of InfluxDB server user", 2024-04-26T02:05:26.799 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.799 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.799 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.799 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.799 INFO:teuthology.orchestra.run.smithi151.stdout: "port": { 2024-04-26T02:05:26.799 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "port", 2024-04-26T02:05:26.799 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.800 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.800 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.800 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "8086", 2024-04-26T02:05:26.800 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.800 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.800 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.800 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "InfluxDB server port", 2024-04-26T02:05:26.800 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.800 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.800 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.800 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.800 INFO:teuthology.orchestra.run.smithi151.stdout: "ssl": { 2024-04-26T02:05:26.801 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ssl", 2024-04-26T02:05:26.801 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.801 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.801 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.801 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "false", 2024-04-26T02:05:26.801 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.801 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.801 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.801 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T02:05:26.801 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.801 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.801 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.801 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.802 INFO:teuthology.orchestra.run.smithi151.stdout: "threads": { 2024-04-26T02:05:26.802 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "threads", 2024-04-26T02:05:26.802 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.802 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.802 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.802 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "5", 2024-04-26T02:05:26.802 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "1", 2024-04-26T02:05:26.802 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "32", 2024-04-26T02:05:26.802 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.802 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-26T02:05:26.802 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.802 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.802 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.803 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.803 INFO:teuthology.orchestra.run.smithi151.stdout: "username": { 2024-04-26T02:05:26.803 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "username", 2024-04-26T02:05:26.803 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.803 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.803 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.803 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.803 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.803 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.803 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.803 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "username of InfluxDB server user", 2024-04-26T02:05:26.803 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.803 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.803 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.804 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.804 INFO:teuthology.orchestra.run.smithi151.stdout: "verify_ssl": { 2024-04-26T02:05:26.804 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "verify_ssl", 2024-04-26T02:05:26.804 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.804 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.804 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.804 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "true", 2024-04-26T02:05:26.804 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.804 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.804 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.804 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T02:05:26.804 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.804 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.804 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.805 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.805 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.805 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.805 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.805 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "insights", 2024-04-26T02:05:26.805 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.805 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.805 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.805 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.805 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.805 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.805 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.805 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.805 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.805 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.805 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.806 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.806 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.806 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.806 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.806 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.806 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.806 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.806 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.806 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.806 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.806 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.806 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.806 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.807 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.807 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.807 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.807 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.807 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.807 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.807 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.807 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.807 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.807 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.807 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.807 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.807 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.807 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.808 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.808 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.808 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.808 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.808 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.808 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.808 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.808 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.808 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.808 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.808 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.808 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.808 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.808 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.808 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.808 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.809 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.809 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.809 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.809 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.809 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.809 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.809 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.809 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.809 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.809 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.809 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.809 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.809 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.809 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.809 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.810 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.810 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.810 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.810 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.810 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.810 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.810 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.810 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "iostat", 2024-04-26T02:05:26.810 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.810 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.810 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.810 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.810 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.810 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.811 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.811 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.811 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.811 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.811 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.811 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.811 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.811 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.811 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.811 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.811 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.811 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.811 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.811 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.812 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.812 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.812 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.812 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.812 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.812 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.812 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.812 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.812 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.812 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.812 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.812 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.812 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.812 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.813 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.813 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.813 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.813 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.813 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.813 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.813 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.813 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.813 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.813 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.813 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.813 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.813 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.813 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.813 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.814 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.814 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.814 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.814 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.814 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.814 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.814 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.814 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.814 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.814 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.814 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.814 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.814 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.815 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.815 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.815 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.815 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.815 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.815 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.815 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.815 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.815 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.815 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.815 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.815 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.815 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.815 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.815 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "k8sevents", 2024-04-26T02:05:26.815 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.816 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.816 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.816 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_event_retention_days": { 2024-04-26T02:05:26.816 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ceph_event_retention_days", 2024-04-26T02:05:26.816 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.816 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.816 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.816 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "7", 2024-04-26T02:05:26.816 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.816 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.816 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.816 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-26T02:05:26.816 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.816 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.816 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.817 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.817 INFO:teuthology.orchestra.run.smithi151.stdout: "config_check_secs": { 2024-04-26T02:05:26.817 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "config_check_secs", 2024-04-26T02:05:26.817 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.817 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.817 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.817 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "10", 2024-04-26T02:05:26.817 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "10", 2024-04-26T02:05:26.817 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.817 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.817 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-26T02:05:26.817 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.817 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.817 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.817 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.818 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.818 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.818 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.818 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.818 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.818 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.818 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.818 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.818 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.818 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.818 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.818 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.818 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.819 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.819 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.819 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.819 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.819 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.819 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.819 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.819 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.819 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.819 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.819 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.819 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.819 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.819 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.820 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.820 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.820 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.820 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.820 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.820 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.820 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.820 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.820 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.820 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.820 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.820 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.820 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.820 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.820 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.821 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.821 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.821 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.821 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.821 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.821 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.821 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.821 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.821 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.821 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.821 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.821 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.821 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.821 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.821 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.822 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.822 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.822 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.822 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.822 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.822 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.822 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.822 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.822 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.822 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.822 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.822 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.822 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.822 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.823 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.823 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.823 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "localpool", 2024-04-26T02:05:26.823 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.823 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.823 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.823 INFO:teuthology.orchestra.run.smithi151.stdout: "failure_domain": { 2024-04-26T02:05:26.823 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "failure_domain", 2024-04-26T02:05:26.823 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.823 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.823 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.823 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "host", 2024-04-26T02:05:26.823 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.823 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.824 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.824 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "failure domain for any created local pool", 2024-04-26T02:05:26.824 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-26T02:05:26.824 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.824 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.824 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.824 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.824 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.824 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.824 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.824 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.824 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.824 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.824 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.825 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.825 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.825 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.825 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.825 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.825 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.825 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.825 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.825 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.825 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.825 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.825 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.825 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.826 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.826 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.826 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.826 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.826 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.826 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.826 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.826 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.826 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.826 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.827 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.827 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.827 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.827 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.827 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.827 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.827 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.827 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.827 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.827 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.827 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.827 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.827 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.828 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.828 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.828 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.828 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.828 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.828 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.828 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.828 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.828 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.828 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.828 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.828 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.828 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.828 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.829 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.829 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.829 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.829 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.829 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.829 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.829 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.829 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.829 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.829 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.829 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.829 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.829 INFO:teuthology.orchestra.run.smithi151.stdout: "min_size": { 2024-04-26T02:05:26.829 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "min_size", 2024-04-26T02:05:26.830 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.830 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.830 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.830 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.830 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.830 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.830 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.830 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "default min_size for any created local pool", 2024-04-26T02:05:26.830 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-26T02:05:26.830 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.830 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.830 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.830 INFO:teuthology.orchestra.run.smithi151.stdout: "num_rep": { 2024-04-26T02:05:26.831 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "num_rep", 2024-04-26T02:05:26.831 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.831 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.831 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.831 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "3", 2024-04-26T02:05:26.831 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.831 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.831 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.831 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "default replica count for any created local pool", 2024-04-26T02:05:26.831 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.831 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.831 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.831 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.831 INFO:teuthology.orchestra.run.smithi151.stdout: "pg_num": { 2024-04-26T02:05:26.832 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "pg_num", 2024-04-26T02:05:26.832 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.832 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.832 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.832 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "128", 2024-04-26T02:05:26.832 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.832 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.832 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.832 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "default pg_num for any created local pool", 2024-04-26T02:05:26.832 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.832 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.832 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.832 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.832 INFO:teuthology.orchestra.run.smithi151.stdout: "prefix": { 2024-04-26T02:05:26.833 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "prefix", 2024-04-26T02:05:26.833 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.833 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.833 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.833 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.833 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.833 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.833 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.833 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "name prefix for any created local pool", 2024-04-26T02:05:26.833 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.833 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.833 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.833 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.833 INFO:teuthology.orchestra.run.smithi151.stdout: "subtree": { 2024-04-26T02:05:26.834 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "subtree", 2024-04-26T02:05:26.834 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.834 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.834 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.834 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "rack", 2024-04-26T02:05:26.834 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.834 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.834 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.834 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-26T02:05:26.834 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-26T02:05:26.834 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.834 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.835 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.835 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.835 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.835 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.835 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "mds_autoscaler", 2024-04-26T02:05:26.835 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.835 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.835 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.835 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.835 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.835 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.835 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.835 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.835 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.835 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.836 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.836 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.836 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.836 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.836 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.836 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.836 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.836 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.836 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.836 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.836 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.836 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.837 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.837 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.837 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.837 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.837 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.837 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.837 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.837 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.837 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.837 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.837 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.837 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.837 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.837 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.838 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.838 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.838 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.838 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.838 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.838 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.838 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.838 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.838 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.838 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.838 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.838 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.838 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.839 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.839 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.839 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.839 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.839 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.839 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.839 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.839 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.839 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.839 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.839 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.839 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.839 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.839 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.840 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.840 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.840 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.840 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.840 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.840 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.840 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.840 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.840 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.840 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.840 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.840 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.840 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.841 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "mirroring", 2024-04-26T02:05:26.841 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.841 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.841 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.841 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.841 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.841 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.841 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.841 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.841 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.841 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.841 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.841 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.842 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.842 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.842 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.842 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.842 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.842 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.842 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.842 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.842 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.842 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.842 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.842 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.842 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.843 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.843 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.843 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.843 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.843 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.843 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.843 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.843 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.843 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.843 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.843 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.843 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.843 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.843 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.844 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.844 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.844 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.844 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.844 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.844 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.844 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.844 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.844 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.844 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.844 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.844 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.844 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.844 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.844 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.845 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.845 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.845 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.845 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.845 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.845 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.845 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.845 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.845 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.845 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.845 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.845 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.845 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.845 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.846 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.846 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.846 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.846 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.846 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.846 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.846 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.846 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.846 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "nfs", 2024-04-26T02:05:26.846 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.846 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.846 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.846 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.846 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.847 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.847 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.847 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.847 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.847 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.847 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.847 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.847 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.847 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.847 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.847 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.847 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.847 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.847 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.847 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.848 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.848 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.848 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.848 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.848 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.848 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.848 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.848 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.848 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.848 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.848 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.848 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.848 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.848 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.848 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.849 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.849 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.849 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.849 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.849 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.849 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.849 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.849 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.849 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.849 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.849 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.849 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.849 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.850 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.850 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.850 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.850 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.850 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.850 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.850 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.850 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.850 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.850 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.850 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.850 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.850 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.851 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.851 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.851 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.851 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.851 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.851 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.851 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.851 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.851 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.851 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.851 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.851 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.851 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.851 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.852 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.852 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "orchestrator", 2024-04-26T02:05:26.852 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.852 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.852 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.852 INFO:teuthology.orchestra.run.smithi151.stdout: "fail_fs": { 2024-04-26T02:05:26.852 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "fail_fs", 2024-04-26T02:05:26.852 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.852 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.852 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.852 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.852 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.852 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.852 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.853 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-26T02:05:26.853 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.853 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.853 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.853 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.853 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.853 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.853 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.853 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.853 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.853 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.853 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.853 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.853 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.854 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.854 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.854 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.854 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.854 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.854 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.854 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.854 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.854 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.854 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.854 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.854 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.854 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.855 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.855 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.855 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.855 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.855 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.855 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.855 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.855 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.855 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.855 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.855 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.855 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.855 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.855 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.856 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.856 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.856 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.856 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.856 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.856 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.856 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.856 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.856 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.856 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.856 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.856 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.856 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.856 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.857 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.857 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.857 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.857 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.857 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.857 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.857 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.857 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.857 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.857 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.857 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.857 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.857 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.857 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.857 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.858 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.858 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.858 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.858 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.858 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.858 INFO:teuthology.orchestra.run.smithi151.stdout: "orchestrator": { 2024-04-26T02:05:26.858 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "orchestrator", 2024-04-26T02:05:26.858 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.858 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.858 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.858 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.858 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.858 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.859 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.859 INFO:teuthology.orchestra.run.smithi151.stdout: "cephadm", 2024-04-26T02:05:26.859 INFO:teuthology.orchestra.run.smithi151.stdout: "rook", 2024-04-26T02:05:26.859 INFO:teuthology.orchestra.run.smithi151.stdout: "test_orchestrator" 2024-04-26T02:05:26.859 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.859 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Orchestrator backend", 2024-04-26T02:05:26.859 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.859 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.859 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.859 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.859 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.859 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.859 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.860 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "osd_perf_query", 2024-04-26T02:05:26.860 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.860 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.860 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.860 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.860 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.860 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.860 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.860 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.860 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.860 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.860 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.860 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.861 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.861 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.861 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.861 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.861 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.861 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.861 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.861 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.861 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.861 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.861 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.861 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.861 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.861 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.862 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.862 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.862 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.862 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.862 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.862 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.862 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.862 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.862 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.862 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.862 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.862 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.862 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.862 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.863 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.863 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.863 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.863 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.863 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.863 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.863 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.863 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.863 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.863 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.863 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.863 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.863 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.863 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.863 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.864 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.864 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.864 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.864 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.864 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.864 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.864 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.864 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.864 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.864 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.864 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.864 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.864 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.865 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.865 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.865 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.865 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.865 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.865 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.865 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.865 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.865 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "osd_support", 2024-04-26T02:05:26.865 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.865 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.865 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.865 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.865 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.865 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.866 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.866 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.866 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.866 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.866 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.866 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.866 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.866 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.866 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.866 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.866 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.866 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.866 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.866 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.867 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.867 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.867 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.867 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.867 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.867 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.867 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.867 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.867 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.867 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.867 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.867 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.867 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.867 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.867 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.868 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.868 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.868 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.868 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.868 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.868 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.868 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.868 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.868 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.868 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.868 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.868 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.868 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.868 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.869 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.869 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.869 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.869 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.869 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.869 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.869 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.869 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.869 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.869 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.869 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.869 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.869 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.869 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.869 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.870 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.870 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.870 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.870 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.870 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.870 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.870 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.870 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.870 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.870 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.870 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.870 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.870 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "pg_autoscaler", 2024-04-26T02:05:26.870 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.871 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.871 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.871 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.871 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.871 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.871 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.871 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.871 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.871 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.871 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.871 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.871 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.871 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.871 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.871 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.872 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.872 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.872 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.872 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.872 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.872 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.872 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.872 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.872 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.872 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.872 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.872 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.872 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.872 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.873 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.873 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.873 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.873 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.873 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.873 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.873 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.873 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.873 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.873 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.873 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.873 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.873 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.873 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.874 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.874 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.874 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.874 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.874 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.874 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.874 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.874 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.874 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.874 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.874 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.874 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.874 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.875 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.875 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.875 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.875 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.875 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.875 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.875 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.875 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.875 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.875 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.875 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.875 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.875 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.876 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.876 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.876 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.876 INFO:teuthology.orchestra.run.smithi151.stdout: "sleep_interval": { 2024-04-26T02:05:26.876 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sleep_interval", 2024-04-26T02:05:26.876 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-04-26T02:05:26.876 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.876 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.876 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "60", 2024-04-26T02:05:26.876 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.876 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.876 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.876 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.876 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.876 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.877 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.877 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.877 INFO:teuthology.orchestra.run.smithi151.stdout: "threshold": { 2024-04-26T02:05:26.877 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "threshold", 2024-04-26T02:05:26.877 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "float", 2024-04-26T02:05:26.877 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.877 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.877 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "3.0", 2024-04-26T02:05:26.877 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "1.0", 2024-04-26T02:05:26.877 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.877 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.877 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "scaling threshold", 2024-04-26T02:05:26.877 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-26T02:05:26.877 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.877 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.877 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.878 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.878 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.878 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.878 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "progress", 2024-04-26T02:05:26.878 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.878 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.878 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.878 INFO:teuthology.orchestra.run.smithi151.stdout: "allow_pg_recovery_event": { 2024-04-26T02:05:26.878 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "allow_pg_recovery_event", 2024-04-26T02:05:26.878 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.878 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.878 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.878 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.879 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.879 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.879 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.879 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-26T02:05:26.879 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.879 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.879 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.879 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.879 INFO:teuthology.orchestra.run.smithi151.stdout: "enabled": { 2024-04-26T02:05:26.879 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "enabled", 2024-04-26T02:05:26.879 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.879 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.879 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.879 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.879 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.879 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.880 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.880 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.880 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.880 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.880 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.880 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.880 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.880 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.880 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.880 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.880 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.880 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.880 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.880 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.880 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.880 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.881 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.881 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.881 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.881 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.881 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.881 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.881 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.881 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.881 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.881 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.881 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.881 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.881 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.881 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.881 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.881 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.882 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.882 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.882 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.882 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.882 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.882 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.882 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.882 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.882 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.882 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.882 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.882 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.882 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.882 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.883 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.883 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.883 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.883 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.883 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.883 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.883 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.883 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.883 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.883 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.883 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.883 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.883 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.883 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.883 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.884 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.884 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.884 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.884 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.884 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.884 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.884 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.884 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.884 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.884 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.884 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.884 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.884 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.885 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.885 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.885 INFO:teuthology.orchestra.run.smithi151.stdout: "max_completed_events": { 2024-04-26T02:05:26.885 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "max_completed_events", 2024-04-26T02:05:26.885 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.885 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.885 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.885 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "50", 2024-04-26T02:05:26.885 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.885 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.885 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.885 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "number of past completed events to remember", 2024-04-26T02:05:26.885 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.886 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.886 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.886 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.886 INFO:teuthology.orchestra.run.smithi151.stdout: "sleep_interval": { 2024-04-26T02:05:26.886 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sleep_interval", 2024-04-26T02:05:26.886 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-04-26T02:05:26.886 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.886 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.886 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "5", 2024-04-26T02:05:26.886 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.886 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.886 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.886 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "how long the module is going to sleep", 2024-04-26T02:05:26.886 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.887 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.887 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.887 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.887 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.887 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.887 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.887 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "prometheus", 2024-04-26T02:05:26.887 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.887 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.887 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.887 INFO:teuthology.orchestra.run.smithi151.stdout: "cache": { 2024-04-26T02:05:26.887 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "cache", 2024-04-26T02:05:26.887 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.887 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.888 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.888 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.888 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.888 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.888 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.888 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.888 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.888 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.888 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.888 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.888 INFO:teuthology.orchestra.run.smithi151.stdout: "exclude_perf_counters": { 2024-04-26T02:05:26.888 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "exclude_perf_counters", 2024-04-26T02:05:26.888 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.888 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.888 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.888 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.889 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.889 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.889 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.889 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-26T02:05:26.889 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-26T02:05:26.889 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.889 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.889 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.889 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.889 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.889 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.889 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.889 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.889 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.889 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.889 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.890 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.890 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.890 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.890 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.890 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.890 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.890 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.890 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.890 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.890 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.890 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.890 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.890 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.891 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.891 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.891 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.891 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.891 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.891 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.891 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.891 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.891 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.891 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.891 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.891 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.891 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.891 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.892 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.892 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.892 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.892 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.892 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.892 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.892 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.892 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.892 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.892 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.892 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.892 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.892 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.892 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.893 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.893 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.893 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.893 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.893 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.893 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.893 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.893 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.893 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.893 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.893 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.893 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.893 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.893 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.894 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.894 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.894 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.894 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.894 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.894 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.894 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.894 INFO:teuthology.orchestra.run.smithi151.stdout: "rbd_stats_pools": { 2024-04-26T02:05:26.894 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rbd_stats_pools", 2024-04-26T02:05:26.894 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.894 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.894 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.894 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.895 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.895 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.895 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.895 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.895 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.895 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.895 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.895 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.895 INFO:teuthology.orchestra.run.smithi151.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-26T02:05:26.895 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-26T02:05:26.895 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.895 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.895 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.895 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "300", 2024-04-26T02:05:26.895 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.896 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.896 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.896 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.896 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.896 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.896 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.896 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.896 INFO:teuthology.orchestra.run.smithi151.stdout: "scrape_interval": { 2024-04-26T02:05:26.896 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "scrape_interval", 2024-04-26T02:05:26.896 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "float", 2024-04-26T02:05:26.896 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.896 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.896 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "15.0", 2024-04-26T02:05:26.896 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.897 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.897 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.897 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.897 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.897 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.897 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.897 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.897 INFO:teuthology.orchestra.run.smithi151.stdout: "server_addr": { 2024-04-26T02:05:26.897 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "server_addr", 2024-04-26T02:05:26.897 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.897 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.897 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.897 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "::", 2024-04-26T02:05:26.897 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.897 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.898 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.898 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-26T02:05:26.898 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.898 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.898 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.898 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.898 INFO:teuthology.orchestra.run.smithi151.stdout: "server_port": { 2024-04-26T02:05:26.898 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "server_port", 2024-04-26T02:05:26.898 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.898 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.898 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.898 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "9283", 2024-04-26T02:05:26.898 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.898 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.899 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.899 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-26T02:05:26.899 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.899 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.899 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.899 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.899 INFO:teuthology.orchestra.run.smithi151.stdout: "stale_cache_strategy": { 2024-04-26T02:05:26.899 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "stale_cache_strategy", 2024-04-26T02:05:26.899 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.899 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.899 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.899 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "log", 2024-04-26T02:05:26.899 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.899 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.900 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.900 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.900 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.900 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.900 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.900 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.900 INFO:teuthology.orchestra.run.smithi151.stdout: "standby_behaviour": { 2024-04-26T02:05:26.900 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "standby_behaviour", 2024-04-26T02:05:26.900 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.900 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.900 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.900 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "default", 2024-04-26T02:05:26.900 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.900 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.901 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.901 INFO:teuthology.orchestra.run.smithi151.stdout: "default", 2024-04-26T02:05:26.901 INFO:teuthology.orchestra.run.smithi151.stdout: "error" 2024-04-26T02:05:26.901 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.901 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.901 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.901 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.901 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.901 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.901 INFO:teuthology.orchestra.run.smithi151.stdout: "standby_error_status_code": { 2024-04-26T02:05:26.901 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "standby_error_status_code", 2024-04-26T02:05:26.901 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.901 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.902 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.902 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "500", 2024-04-26T02:05:26.902 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "400", 2024-04-26T02:05:26.902 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "599", 2024-04-26T02:05:26.902 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.902 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.902 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.902 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.902 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.902 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.902 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.902 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.902 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.902 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rbd_support", 2024-04-26T02:05:26.903 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.903 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.903 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.903 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.903 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.903 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.903 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.903 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.903 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.903 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.903 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.903 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.903 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.903 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.903 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.904 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.904 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.904 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.904 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.904 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.904 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.904 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.904 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.904 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.904 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.904 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.904 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.904 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.904 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.904 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.905 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.905 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.905 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.905 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.905 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.905 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.905 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.905 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.905 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.905 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.905 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.905 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.905 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.905 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.905 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.906 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.906 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.906 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.906 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.906 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.906 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.906 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.906 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.906 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.906 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.906 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.906 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.906 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.906 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.906 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.907 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.907 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.907 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.907 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.907 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.907 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.907 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.907 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.907 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.907 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.907 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.907 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.907 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.907 INFO:teuthology.orchestra.run.smithi151.stdout: "max_concurrent_snap_create": { 2024-04-26T02:05:26.908 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "max_concurrent_snap_create", 2024-04-26T02:05:26.908 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.908 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.908 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.908 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "10", 2024-04-26T02:05:26.908 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.908 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.908 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.908 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.908 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.908 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.908 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.908 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.908 INFO:teuthology.orchestra.run.smithi151.stdout: "mirror_snapshot_schedule": { 2024-04-26T02:05:26.908 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "mirror_snapshot_schedule", 2024-04-26T02:05:26.909 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.909 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.909 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.909 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.909 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.909 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.909 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.909 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.909 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.909 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.909 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.909 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.909 INFO:teuthology.orchestra.run.smithi151.stdout: "trash_purge_schedule": { 2024-04-26T02:05:26.909 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "trash_purge_schedule", 2024-04-26T02:05:26.910 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.910 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.910 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.910 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.910 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.910 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.910 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.910 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.910 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.910 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.910 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.910 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.910 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.911 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.911 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.911 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "restful", 2024-04-26T02:05:26.911 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.911 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.911 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.911 INFO:teuthology.orchestra.run.smithi151.stdout: "enable_auth": { 2024-04-26T02:05:26.911 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "enable_auth", 2024-04-26T02:05:26.911 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.911 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.911 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.911 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.911 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.912 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.912 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.912 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.912 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.912 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.912 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.912 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.912 INFO:teuthology.orchestra.run.smithi151.stdout: "key_file": { 2024-04-26T02:05:26.912 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "key_file", 2024-04-26T02:05:26.912 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.912 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.912 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.912 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.912 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.913 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.913 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.913 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.913 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.913 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.913 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.913 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.913 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.913 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.913 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.913 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.913 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.913 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.914 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.914 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.914 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.914 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.914 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.914 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.914 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.914 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.914 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.914 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.914 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.914 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.914 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.915 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.915 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.915 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.915 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.915 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.915 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.915 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.915 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.915 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.915 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.915 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.915 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.915 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.915 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.915 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.916 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.916 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.916 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.916 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.916 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.916 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.916 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.916 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.916 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.916 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.916 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.916 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.916 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.916 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.917 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.917 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.917 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.917 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.917 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.917 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.917 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.917 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.917 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.917 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.917 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.917 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.917 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.917 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.918 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.918 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.918 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.918 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.918 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.918 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.918 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.918 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.918 INFO:teuthology.orchestra.run.smithi151.stdout: "server_addr": { 2024-04-26T02:05:26.918 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "server_addr", 2024-04-26T02:05:26.918 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.918 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.918 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.919 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.919 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.919 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.919 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.919 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.919 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.919 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.919 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.919 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.919 INFO:teuthology.orchestra.run.smithi151.stdout: "server_port": { 2024-04-26T02:05:26.919 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "server_port", 2024-04-26T02:05:26.919 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.919 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.919 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.919 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.919 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.920 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.920 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.920 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.920 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.920 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.920 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.920 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.920 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.920 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.920 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.920 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rgw", 2024-04-26T02:05:26.920 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.920 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.920 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.920 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.921 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.921 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.921 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.921 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.921 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.921 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.921 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.921 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.921 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.921 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.921 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.921 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.921 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.921 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.922 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.922 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.922 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.922 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.922 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.922 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.922 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.922 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.922 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.922 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.922 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.922 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.922 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.923 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.923 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.923 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.923 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.923 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.923 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.923 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.923 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.923 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.923 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.923 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.923 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.923 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.923 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.924 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.924 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.924 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.924 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.924 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.924 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.924 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.924 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.924 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.924 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.924 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.924 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.924 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.924 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.924 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.925 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.925 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.925 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.925 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.925 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.925 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.925 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.925 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.925 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.925 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.925 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.925 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.925 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.925 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.925 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.926 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.926 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rook", 2024-04-26T02:05:26.926 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.926 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.926 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.926 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.926 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.926 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.926 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.926 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.926 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.926 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.926 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.927 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.927 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.927 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.927 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.927 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.927 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.927 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.927 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.927 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.927 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.927 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.927 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.927 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.927 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.927 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.928 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.928 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.928 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.928 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.928 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.928 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.928 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.928 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.928 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.928 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.928 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.928 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.928 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.928 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.928 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.928 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.929 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.929 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.929 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.929 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.929 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.929 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.929 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.929 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.929 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.929 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.929 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.929 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.929 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.929 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.929 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.929 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.930 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.930 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.930 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.930 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.930 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.930 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.930 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.930 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.930 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.930 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.930 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.930 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.930 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.930 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.931 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.931 INFO:teuthology.orchestra.run.smithi151.stdout: "storage_class": { 2024-04-26T02:05:26.931 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "storage_class", 2024-04-26T02:05:26.931 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.931 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.931 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.931 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "local", 2024-04-26T02:05:26.931 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.931 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.931 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.931 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-26T02:05:26.931 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.931 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.931 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.931 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.932 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.932 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.932 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.932 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "selftest", 2024-04-26T02:05:26.932 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.932 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.932 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.932 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.932 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.932 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.932 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.932 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.932 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.932 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.932 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.932 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.933 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.933 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.933 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.933 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.933 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.933 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.933 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.933 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.933 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.933 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.933 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.933 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.933 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.933 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.933 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.933 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.934 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.934 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.934 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.934 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.934 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.934 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.934 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.934 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.934 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.934 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.934 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.934 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.934 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.934 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.934 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.935 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.935 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.935 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.935 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.935 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.935 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.935 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.935 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.935 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.935 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.935 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.935 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.935 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.935 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.935 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.936 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.936 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.936 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.936 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.936 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.936 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.936 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.936 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.936 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.936 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.936 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.936 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.936 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.936 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.936 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.936 INFO:teuthology.orchestra.run.smithi151.stdout: "roption1": { 2024-04-26T02:05:26.937 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "roption1", 2024-04-26T02:05:26.937 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.937 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.937 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.937 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.937 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.937 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.937 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.937 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.937 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.937 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.937 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.937 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.937 INFO:teuthology.orchestra.run.smithi151.stdout: "roption2": { 2024-04-26T02:05:26.937 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "roption2", 2024-04-26T02:05:26.937 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.938 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.938 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.938 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "xyz", 2024-04-26T02:05:26.938 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.938 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.938 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.938 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.938 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.938 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.938 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.938 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.938 INFO:teuthology.orchestra.run.smithi151.stdout: "rwoption1": { 2024-04-26T02:05:26.938 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rwoption1", 2024-04-26T02:05:26.938 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.939 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.939 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.939 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.939 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.939 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.939 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.939 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.939 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.939 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.939 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.939 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.939 INFO:teuthology.orchestra.run.smithi151.stdout: "rwoption2": { 2024-04-26T02:05:26.939 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rwoption2", 2024-04-26T02:05:26.939 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.940 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.940 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.940 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.940 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.940 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.940 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.940 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.940 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.940 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.940 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.940 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.940 INFO:teuthology.orchestra.run.smithi151.stdout: "rwoption3": { 2024-04-26T02:05:26.940 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rwoption3", 2024-04-26T02:05:26.940 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "float", 2024-04-26T02:05:26.941 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.941 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.941 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.941 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.941 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.941 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.941 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.941 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.941 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.941 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.941 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.941 INFO:teuthology.orchestra.run.smithi151.stdout: "rwoption4": { 2024-04-26T02:05:26.941 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rwoption4", 2024-04-26T02:05:26.942 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.942 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.942 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.942 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.942 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.942 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.942 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.942 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.942 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.942 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.942 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.942 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.943 INFO:teuthology.orchestra.run.smithi151.stdout: "rwoption5": { 2024-04-26T02:05:26.943 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rwoption5", 2024-04-26T02:05:26.943 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.943 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.943 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.943 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.943 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.943 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.943 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.943 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.943 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.943 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.943 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.943 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.944 INFO:teuthology.orchestra.run.smithi151.stdout: "rwoption6": { 2024-04-26T02:05:26.944 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rwoption6", 2024-04-26T02:05:26.944 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.944 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.944 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.944 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.944 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.944 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.944 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.944 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.944 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.944 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.944 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.944 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.945 INFO:teuthology.orchestra.run.smithi151.stdout: "rwoption7": { 2024-04-26T02:05:26.945 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rwoption7", 2024-04-26T02:05:26.945 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.945 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.945 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.945 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.945 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "1", 2024-04-26T02:05:26.945 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "42", 2024-04-26T02:05:26.945 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.945 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.945 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.945 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.945 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.946 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.946 INFO:teuthology.orchestra.run.smithi151.stdout: "testkey": { 2024-04-26T02:05:26.946 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "testkey", 2024-04-26T02:05:26.946 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.946 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.946 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.946 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.946 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.946 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.946 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.946 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.946 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.946 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.946 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.947 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.947 INFO:teuthology.orchestra.run.smithi151.stdout: "testlkey": { 2024-04-26T02:05:26.947 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "testlkey", 2024-04-26T02:05:26.947 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.947 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.947 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.947 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.947 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.947 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.947 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.947 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.947 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.947 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.948 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.948 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.948 INFO:teuthology.orchestra.run.smithi151.stdout: "testnewline": { 2024-04-26T02:05:26.948 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "testnewline", 2024-04-26T02:05:26.948 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.948 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.948 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.948 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.948 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.948 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.948 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.948 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.948 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.948 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.949 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.949 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.949 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.949 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.949 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.949 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "snap_schedule", 2024-04-26T02:05:26.949 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.949 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.949 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.949 INFO:teuthology.orchestra.run.smithi151.stdout: "allow_m_granularity": { 2024-04-26T02:05:26.949 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "allow_m_granularity", 2024-04-26T02:05:26.949 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.949 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.950 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.950 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.950 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.950 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.950 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.950 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "allow minute scheduled snapshots", 2024-04-26T02:05:26.950 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.950 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.950 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.950 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.950 INFO:teuthology.orchestra.run.smithi151.stdout: "dump_on_update": { 2024-04-26T02:05:26.950 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "dump_on_update", 2024-04-26T02:05:26.951 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.951 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.951 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.951 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.951 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.951 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.951 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.951 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "dump database to debug log on update", 2024-04-26T02:05:26.951 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.951 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.951 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.951 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.951 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.952 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.952 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.952 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.952 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.952 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.952 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.952 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.952 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.952 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.952 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.952 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.952 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.952 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.952 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.953 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.953 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.953 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.953 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.953 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.953 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.953 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.953 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.953 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.953 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.953 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.953 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.953 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.953 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.953 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.954 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.954 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.954 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.954 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.954 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.954 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.954 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.954 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.954 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.954 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.954 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.954 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.954 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.955 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.955 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.955 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.955 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.955 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.955 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.955 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.955 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.955 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.955 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.955 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.955 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.955 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.955 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.955 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.956 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.956 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.956 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.956 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.956 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.956 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.956 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.956 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.956 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.956 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.956 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.956 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.956 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.956 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.956 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.957 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "stats", 2024-04-26T02:05:26.957 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.957 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.957 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.957 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.957 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.957 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.957 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.957 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.957 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.957 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.957 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.957 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.957 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.957 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.958 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.958 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.958 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.958 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.958 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.958 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.958 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.958 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.958 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.958 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.958 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.958 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.958 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.959 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.959 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.959 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.959 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.959 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.959 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.959 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.959 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.959 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.959 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.959 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.959 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.959 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.959 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.959 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.960 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.960 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.960 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.960 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.960 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.960 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.960 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.960 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.960 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.960 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.960 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.960 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.960 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.960 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.960 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.960 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.961 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.961 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.961 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.961 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.961 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.961 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.961 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.961 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.961 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.961 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.961 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.961 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.961 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.961 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.962 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.962 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.962 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.962 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.962 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "status", 2024-04-26T02:05:26.962 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.962 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.962 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.962 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.962 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.962 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.962 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.963 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.963 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.963 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.963 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.963 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.963 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.963 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.963 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.963 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.963 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.963 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.963 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.964 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.964 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.964 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.964 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.964 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.964 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.964 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.964 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.964 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.964 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.964 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.964 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.964 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.964 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.964 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.965 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.965 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.965 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.965 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.965 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.965 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.965 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.965 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.965 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.965 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.965 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.965 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.965 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.965 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.966 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.966 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.966 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.966 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.966 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.966 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.966 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.966 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.966 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.966 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.966 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.966 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.966 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.966 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.967 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.967 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.967 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.967 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.967 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.967 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.967 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.967 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.967 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.967 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.967 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.967 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.967 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.967 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.967 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "telegraf", 2024-04-26T02:05:26.968 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.968 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.968 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.968 INFO:teuthology.orchestra.run.smithi151.stdout: "address": { 2024-04-26T02:05:26.968 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "address", 2024-04-26T02:05:26.968 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.968 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.968 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.968 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-26T02:05:26.968 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.968 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.968 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.968 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.968 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.969 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.969 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.969 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.969 INFO:teuthology.orchestra.run.smithi151.stdout: "interval": { 2024-04-26T02:05:26.969 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "interval", 2024-04-26T02:05:26.969 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-04-26T02:05:26.969 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.969 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.969 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "15", 2024-04-26T02:05:26.969 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.969 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.969 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.969 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.969 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.969 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.970 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.970 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.970 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.970 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.970 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.970 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.970 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.970 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.970 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.970 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.970 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.970 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.970 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.970 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.971 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.971 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.971 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.971 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.971 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.971 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.971 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.971 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.971 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.971 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.971 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.971 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.971 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.971 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.971 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.972 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.972 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.972 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.972 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.972 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.972 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.972 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.972 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.972 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.972 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.972 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.972 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.972 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.972 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.972 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.973 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.973 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.973 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.973 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.973 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.973 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.973 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.973 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.973 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.973 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.973 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.973 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.973 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.973 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.974 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.974 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.974 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.974 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.974 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.974 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.974 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.974 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.974 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.974 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.974 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.974 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.974 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.974 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.974 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.975 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.975 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.975 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "telemetry", 2024-04-26T02:05:26.975 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.975 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.975 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.975 INFO:teuthology.orchestra.run.smithi151.stdout: "channel_basic": { 2024-04-26T02:05:26.975 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "channel_basic", 2024-04-26T02:05:26.975 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.975 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.975 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.975 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.975 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.975 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.976 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.976 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-26T02:05:26.976 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.976 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.976 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.976 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.976 INFO:teuthology.orchestra.run.smithi151.stdout: "channel_crash": { 2024-04-26T02:05:26.976 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "channel_crash", 2024-04-26T02:05:26.976 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.976 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.976 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.976 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.976 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.976 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.977 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.977 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-26T02:05:26.977 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.977 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.977 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.977 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.977 INFO:teuthology.orchestra.run.smithi151.stdout: "channel_device": { 2024-04-26T02:05:26.977 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "channel_device", 2024-04-26T02:05:26.977 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.977 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.977 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.977 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:26.977 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.977 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.978 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.978 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-26T02:05:26.978 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.978 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.978 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.978 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.978 INFO:teuthology.orchestra.run.smithi151.stdout: "channel_ident": { 2024-04-26T02:05:26.978 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "channel_ident", 2024-04-26T02:05:26.978 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.978 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.978 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.978 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.978 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.979 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.979 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.979 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-26T02:05:26.979 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.979 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.979 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.979 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.979 INFO:teuthology.orchestra.run.smithi151.stdout: "channel_perf": { 2024-04-26T02:05:26.979 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "channel_perf", 2024-04-26T02:05:26.979 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.979 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.979 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.979 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.979 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.980 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.980 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.980 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-26T02:05:26.980 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.980 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.980 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.980 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.980 INFO:teuthology.orchestra.run.smithi151.stdout: "contact": { 2024-04-26T02:05:26.980 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "contact", 2024-04-26T02:05:26.980 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.980 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.980 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.980 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.980 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.981 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.981 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.981 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.981 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.981 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.981 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.981 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.981 INFO:teuthology.orchestra.run.smithi151.stdout: "description": { 2024-04-26T02:05:26.981 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "description", 2024-04-26T02:05:26.981 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.981 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.981 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.981 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.981 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.981 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.982 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.982 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.982 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.982 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.982 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.982 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.982 INFO:teuthology.orchestra.run.smithi151.stdout: "device_url": { 2024-04-26T02:05:26.982 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "device_url", 2024-04-26T02:05:26.982 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.982 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.982 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.982 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-26T02:05:26.982 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.982 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.983 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.983 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.983 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.983 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.983 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.983 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.983 INFO:teuthology.orchestra.run.smithi151.stdout: "enabled": { 2024-04-26T02:05:26.983 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "enabled", 2024-04-26T02:05:26.983 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.983 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.983 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.983 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.983 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.983 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.983 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.984 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.984 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.984 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.984 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.984 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.984 INFO:teuthology.orchestra.run.smithi151.stdout: "interval": { 2024-04-26T02:05:26.984 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "interval", 2024-04-26T02:05:26.984 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.984 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.984 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.984 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "24", 2024-04-26T02:05:26.984 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "8", 2024-04-26T02:05:26.984 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.984 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.985 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.985 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.985 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.985 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.985 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.985 INFO:teuthology.orchestra.run.smithi151.stdout: "last_opt_revision": { 2024-04-26T02:05:26.985 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "last_opt_revision", 2024-04-26T02:05:26.985 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:26.985 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.985 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.985 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "1", 2024-04-26T02:05:26.985 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.985 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.985 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.986 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.986 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.986 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.986 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.986 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.986 INFO:teuthology.orchestra.run.smithi151.stdout: "leaderboard": { 2024-04-26T02:05:26.986 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "leaderboard", 2024-04-26T02:05:26.986 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.986 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.986 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.986 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.986 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.986 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.987 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.987 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.987 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.987 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.987 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.987 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.987 INFO:teuthology.orchestra.run.smithi151.stdout: "leaderboard_description": { 2024-04-26T02:05:26.987 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "leaderboard_description", 2024-04-26T02:05:26.987 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.987 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.987 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.987 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.987 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.987 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.988 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.988 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.988 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.988 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.988 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.988 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.988 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.988 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.988 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.988 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.988 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.988 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.988 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.988 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.989 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.989 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.989 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.989 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.989 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.989 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.989 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.989 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.989 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.989 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.989 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.989 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.989 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.989 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.989 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.990 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.990 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.990 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.990 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.990 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.990 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.990 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.990 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.990 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.990 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.990 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.990 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.991 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.991 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.991 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.991 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.991 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.991 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.991 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.991 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.991 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.991 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.991 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.991 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.991 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.991 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.992 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.992 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.992 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.992 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.992 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.992 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.992 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.992 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:26.992 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:26.992 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.992 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.992 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.992 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.992 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.992 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.993 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.993 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.993 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.993 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.993 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.993 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.993 INFO:teuthology.orchestra.run.smithi151.stdout: "organization": { 2024-04-26T02:05:26.993 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "organization", 2024-04-26T02:05:26.993 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.993 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.993 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.993 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.993 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.993 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.994 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.994 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.994 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.994 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.994 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.994 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.994 INFO:teuthology.orchestra.run.smithi151.stdout: "proxy": { 2024-04-26T02:05:26.994 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "proxy", 2024-04-26T02:05:26.994 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.994 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.994 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.994 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.994 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.995 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.995 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.995 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.995 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.995 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.995 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.995 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.995 INFO:teuthology.orchestra.run.smithi151.stdout: "url": { 2024-04-26T02:05:26.995 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "url", 2024-04-26T02:05:26.995 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.995 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.995 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:26.995 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-26T02:05:26.995 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.996 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.996 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.996 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.996 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.996 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.996 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.996 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.996 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:26.996 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.996 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:26.996 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "test_orchestrator", 2024-04-26T02:05:26.996 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:26.996 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:26.996 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:26.996 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:26.997 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:26.997 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.997 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.997 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.997 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:26.997 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.997 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.997 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:26.997 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:26.997 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:26.997 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:26.997 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:26.997 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:26.997 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:26.998 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:26.998 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.998 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.998 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.998 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.998 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.998 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:26.998 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:26.998 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:26.998 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.998 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.998 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:26.998 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.998 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:26.999 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:26.999 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:26.999 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:26.999 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:26.999 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:26.999 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:26.999 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:26.999 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:26.999 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:26.999 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:26.999 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:26.999 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:26.999 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:26.999 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.000 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.000 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.000 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.000 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.000 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.000 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.000 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.000 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.000 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.000 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.000 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.000 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.000 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.000 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:27.001 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:27.001 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.001 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.001 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.001 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.001 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.001 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.001 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.001 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.001 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.001 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.001 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.001 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.001 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.002 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.002 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:27.002 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "volumes", 2024-04-26T02:05:27.002 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:27.002 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:27.002 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:27.002 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:27.002 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:27.002 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.002 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.002 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.002 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.002 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.002 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.003 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.003 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.003 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.003 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.003 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.003 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.003 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.003 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.003 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.003 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.003 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.003 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.003 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.003 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:27.003 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:27.004 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.004 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.004 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.004 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.004 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.004 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.004 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.004 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.004 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.004 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.004 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.004 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.004 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:27.005 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:27.005 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.005 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.005 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.005 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:27.005 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.005 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.005 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.005 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.005 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.005 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.005 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.005 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.005 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.005 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.006 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.006 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.006 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.006 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.006 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.006 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:27.006 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:27.006 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.006 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.006 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.006 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.006 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.006 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.006 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.007 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.007 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.007 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.007 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.007 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.007 INFO:teuthology.orchestra.run.smithi151.stdout: "max_concurrent_clones": { 2024-04-26T02:05:27.007 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "max_concurrent_clones", 2024-04-26T02:05:27.007 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:27.007 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.007 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.007 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "4", 2024-04-26T02:05:27.007 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.007 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.007 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.007 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-26T02:05:27.008 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.008 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.008 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.008 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.008 INFO:teuthology.orchestra.run.smithi151.stdout: "periodic_async_work": { 2024-04-26T02:05:27.008 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "periodic_async_work", 2024-04-26T02:05:27.008 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.008 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.008 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.008 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.008 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.008 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.008 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.008 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Periodically check for async work", 2024-04-26T02:05:27.008 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.009 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.009 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.009 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.009 INFO:teuthology.orchestra.run.smithi151.stdout: "snapshot_clone_delay": { 2024-04-26T02:05:27.009 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "snapshot_clone_delay", 2024-04-26T02:05:27.009 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:27.009 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.009 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.009 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-04-26T02:05:27.009 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.009 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.009 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.009 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-26T02:05:27.009 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.009 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.010 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.010 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.010 INFO:teuthology.orchestra.run.smithi151.stdout: "snapshot_clone_no_wait": { 2024-04-26T02:05:27.010 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "snapshot_clone_no_wait", 2024-04-26T02:05:27.010 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.010 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.010 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.010 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:27.010 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.010 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.010 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.010 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-26T02:05:27.010 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.011 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.011 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.011 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.011 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.011 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.011 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:27.011 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "zabbix", 2024-04-26T02:05:27.011 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:27.011 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:27.011 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:27.011 INFO:teuthology.orchestra.run.smithi151.stdout: "discovery_interval": { 2024-04-26T02:05:27.011 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "discovery_interval", 2024-04-26T02:05:27.011 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "uint", 2024-04-26T02:05:27.012 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.012 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.012 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "100", 2024-04-26T02:05:27.012 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.012 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.012 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.012 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.012 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.012 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.012 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.012 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.012 INFO:teuthology.orchestra.run.smithi151.stdout: "identifier": { 2024-04-26T02:05:27.012 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "identifier", 2024-04-26T02:05:27.012 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.012 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.013 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.013 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.013 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.013 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.013 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.013 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.013 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.013 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.013 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.013 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.013 INFO:teuthology.orchestra.run.smithi151.stdout: "interval": { 2024-04-26T02:05:27.013 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "interval", 2024-04-26T02:05:27.013 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-04-26T02:05:27.013 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.013 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.013 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "60", 2024-04-26T02:05:27.014 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.014 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.014 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.014 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.014 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.014 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.014 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.014 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.014 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:27.014 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:27.014 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.014 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.014 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.014 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.015 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.015 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.015 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.015 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.015 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.015 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.015 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.015 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.015 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.015 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.015 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.015 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.015 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.015 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.016 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.016 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:27.016 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:27.016 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.016 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.016 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.016 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.016 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.016 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.016 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.016 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.016 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.016 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.016 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.016 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.016 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:27.017 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:27.017 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.017 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.017 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.017 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:27.017 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.017 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.017 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.017 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.017 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.017 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.017 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.018 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.018 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.018 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.018 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.018 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.018 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.018 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.018 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.018 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:27.018 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:27.018 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.018 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.018 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.018 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.019 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.019 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.019 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.019 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.019 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.019 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.019 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.019 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.019 INFO:teuthology.orchestra.run.smithi151.stdout: "zabbix_host": { 2024-04-26T02:05:27.019 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "zabbix_host", 2024-04-26T02:05:27.019 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.019 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.019 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.019 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.019 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.019 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.020 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.020 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.020 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.020 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.020 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.020 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.020 INFO:teuthology.orchestra.run.smithi151.stdout: "zabbix_port": { 2024-04-26T02:05:27.020 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "zabbix_port", 2024-04-26T02:05:27.020 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:27.020 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.020 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.020 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "10051", 2024-04-26T02:05:27.020 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.020 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.020 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.021 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.021 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.021 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.021 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.021 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.021 INFO:teuthology.orchestra.run.smithi151.stdout: "zabbix_sender": { 2024-04-26T02:05:27.021 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "zabbix_sender", 2024-04-26T02:05:27.021 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.021 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.021 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.021 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-26T02:05:27.021 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.021 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.021 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.021 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.022 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.022 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.022 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.022 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.022 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.022 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.022 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-26T02:05:27.022 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.022 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.022 INFO:teuthology.orchestra.run.smithi151.stdout: "modules": [ 2024-04-26T02:05:27.022 INFO:teuthology.orchestra.run.smithi151.stdout: "cephadm", 2024-04-26T02:05:27.022 INFO:teuthology.orchestra.run.smithi151.stdout: "dashboard", 2024-04-26T02:05:27.022 INFO:teuthology.orchestra.run.smithi151.stdout: "iostat", 2024-04-26T02:05:27.022 INFO:teuthology.orchestra.run.smithi151.stdout: "nfs", 2024-04-26T02:05:27.022 INFO:teuthology.orchestra.run.smithi151.stdout: "restful", 2024-04-26T02:05:27.023 INFO:teuthology.orchestra.run.smithi151.stdout: "snap_schedule" 2024-04-26T02:05:27.023 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.023 INFO:teuthology.orchestra.run.smithi151.stdout: "available_modules": [ 2024-04-26T02:05:27.023 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:27.023 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "alerts", 2024-04-26T02:05:27.023 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:27.023 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:27.023 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:27.023 INFO:teuthology.orchestra.run.smithi151.stdout: "interval": { 2024-04-26T02:05:27.023 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "interval", 2024-04-26T02:05:27.023 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-04-26T02:05:27.023 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.023 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.023 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "60", 2024-04-26T02:05:27.023 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.023 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.024 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.024 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "How frequently to reexamine health status", 2024-04-26T02:05:27.024 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.024 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.024 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.024 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.024 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:27.024 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:27.024 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.024 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.024 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.024 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.024 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.024 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.024 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.025 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.025 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.025 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.025 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.025 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.025 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.025 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.025 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.025 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.025 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.025 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.025 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.025 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:27.025 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:27.025 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.026 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.026 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.026 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.026 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.026 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.026 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.026 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.026 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.026 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.026 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.026 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.026 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:27.026 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:27.027 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.027 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.027 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.027 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:27.027 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.027 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.027 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.027 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.027 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.027 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.027 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.027 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.027 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.027 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.028 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.028 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.028 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.028 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.028 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.028 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:27.028 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:27.028 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.028 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.028 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.028 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.028 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.028 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.028 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.029 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.029 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.029 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.029 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.029 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.029 INFO:teuthology.orchestra.run.smithi151.stdout: "smtp_destination": { 2024-04-26T02:05:27.029 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "smtp_destination", 2024-04-26T02:05:27.029 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.029 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.029 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.029 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.029 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.029 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.029 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.030 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Email address to send alerts to", 2024-04-26T02:05:27.030 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.030 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.030 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.030 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.030 INFO:teuthology.orchestra.run.smithi151.stdout: "smtp_from_name": { 2024-04-26T02:05:27.030 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "smtp_from_name", 2024-04-26T02:05:27.030 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.030 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.030 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.030 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "Ceph", 2024-04-26T02:05:27.030 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.030 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.030 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.031 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Email From: name", 2024-04-26T02:05:27.031 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.031 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.031 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.031 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.031 INFO:teuthology.orchestra.run.smithi151.stdout: "smtp_host": { 2024-04-26T02:05:27.031 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "smtp_host", 2024-04-26T02:05:27.031 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.031 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.031 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.031 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.031 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.031 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.031 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.031 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "SMTP server", 2024-04-26T02:05:27.031 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.032 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.032 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.032 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.032 INFO:teuthology.orchestra.run.smithi151.stdout: "smtp_password": { 2024-04-26T02:05:27.032 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "smtp_password", 2024-04-26T02:05:27.032 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.032 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.032 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.032 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.032 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.032 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.032 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.032 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Password to authenticate with", 2024-04-26T02:05:27.032 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.032 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.033 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.033 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.033 INFO:teuthology.orchestra.run.smithi151.stdout: "smtp_port": { 2024-04-26T02:05:27.033 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "smtp_port", 2024-04-26T02:05:27.033 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:27.033 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.033 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.033 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "465", 2024-04-26T02:05:27.033 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.033 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.033 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.033 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "SMTP port", 2024-04-26T02:05:27.033 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.034 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.034 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.034 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.034 INFO:teuthology.orchestra.run.smithi151.stdout: "smtp_sender": { 2024-04-26T02:05:27.034 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "smtp_sender", 2024-04-26T02:05:27.034 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.034 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.034 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.034 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.034 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.034 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.034 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.034 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "SMTP envelope sender", 2024-04-26T02:05:27.034 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.035 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.035 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.035 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.035 INFO:teuthology.orchestra.run.smithi151.stdout: "smtp_ssl": { 2024-04-26T02:05:27.035 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "smtp_ssl", 2024-04-26T02:05:27.035 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.035 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.035 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.035 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:27.035 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.035 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.035 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.036 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-26T02:05:27.036 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.036 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.036 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.036 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.036 INFO:teuthology.orchestra.run.smithi151.stdout: "smtp_user": { 2024-04-26T02:05:27.036 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "smtp_user", 2024-04-26T02:05:27.036 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.036 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.036 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.036 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.036 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.036 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.036 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.036 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "User to authenticate as", 2024-04-26T02:05:27.037 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.037 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.037 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.037 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.037 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.037 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.037 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:27.037 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "balancer", 2024-04-26T02:05:27.037 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:27.037 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:27.037 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:27.037 INFO:teuthology.orchestra.run.smithi151.stdout: "active": { 2024-04-26T02:05:27.037 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "active", 2024-04-26T02:05:27.037 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.038 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.038 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.038 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:27.038 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.038 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.038 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.038 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "automatically balance PGs across cluster", 2024-04-26T02:05:27.038 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.038 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.038 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.038 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.038 INFO:teuthology.orchestra.run.smithi151.stdout: "begin_time": { 2024-04-26T02:05:27.038 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "begin_time", 2024-04-26T02:05:27.038 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.039 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.039 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.039 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0000", 2024-04-26T02:05:27.039 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.039 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.039 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.039 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "beginning time of day to automatically balance", 2024-04-26T02:05:27.039 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T02:05:27.039 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.039 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.039 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.039 INFO:teuthology.orchestra.run.smithi151.stdout: "begin_weekday": { 2024-04-26T02:05:27.039 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "begin_weekday", 2024-04-26T02:05:27.040 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "uint", 2024-04-26T02:05:27.040 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.040 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.040 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-04-26T02:05:27.040 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "0", 2024-04-26T02:05:27.040 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "6", 2024-04-26T02:05:27.040 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.040 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-26T02:05:27.040 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T02:05:27.040 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.040 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.040 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.040 INFO:teuthology.orchestra.run.smithi151.stdout: "crush_compat_max_iterations": { 2024-04-26T02:05:27.041 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "crush_compat_max_iterations", 2024-04-26T02:05:27.041 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "uint", 2024-04-26T02:05:27.041 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.041 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.041 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "25", 2024-04-26T02:05:27.041 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "1", 2024-04-26T02:05:27.041 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "250", 2024-04-26T02:05:27.041 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.041 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-26T02:05:27.041 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.041 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.041 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.041 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.042 INFO:teuthology.orchestra.run.smithi151.stdout: "crush_compat_metrics": { 2024-04-26T02:05:27.042 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "crush_compat_metrics", 2024-04-26T02:05:27.042 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.042 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.042 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.042 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "pgs,objects,bytes", 2024-04-26T02:05:27.042 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.042 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.042 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.042 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-26T02:05:27.042 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-26T02:05:27.042 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.042 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.043 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.043 INFO:teuthology.orchestra.run.smithi151.stdout: "crush_compat_step": { 2024-04-26T02:05:27.043 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "crush_compat_step", 2024-04-26T02:05:27.043 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "float", 2024-04-26T02:05:27.043 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.043 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.043 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0.5", 2024-04-26T02:05:27.043 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "0.001", 2024-04-26T02:05:27.043 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "0.999", 2024-04-26T02:05:27.043 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.043 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "aggressiveness of optimization", 2024-04-26T02:05:27.043 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-26T02:05:27.043 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.043 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.043 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.044 INFO:teuthology.orchestra.run.smithi151.stdout: "end_time": { 2024-04-26T02:05:27.044 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "end_time", 2024-04-26T02:05:27.044 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.044 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.044 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.044 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "2359", 2024-04-26T02:05:27.044 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.044 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.044 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.044 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "ending time of day to automatically balance", 2024-04-26T02:05:27.044 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T02:05:27.044 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.044 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.044 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.044 INFO:teuthology.orchestra.run.smithi151.stdout: "end_weekday": { 2024-04-26T02:05:27.044 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "end_weekday", 2024-04-26T02:05:27.045 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "uint", 2024-04-26T02:05:27.045 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.045 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.045 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-04-26T02:05:27.045 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "0", 2024-04-26T02:05:27.045 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "6", 2024-04-26T02:05:27.045 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.045 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-26T02:05:27.045 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T02:05:27.045 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.045 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.045 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.045 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:27.045 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:27.046 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.046 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.046 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.046 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.046 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.046 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.046 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.046 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.046 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.046 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.046 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.046 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.046 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.046 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.046 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.047 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.047 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.047 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.047 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.047 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:27.047 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:27.047 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.047 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.047 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.047 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.047 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.047 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.047 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.047 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.048 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.048 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.048 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.048 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.048 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:27.048 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:27.048 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.048 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.048 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.048 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:27.048 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.048 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.048 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.049 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.049 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.049 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.049 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.049 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.049 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.049 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.049 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.049 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.049 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.049 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.049 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.050 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:27.050 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:27.050 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.050 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.050 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.050 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.050 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.050 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.050 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.050 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.050 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.050 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.050 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.051 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.051 INFO:teuthology.orchestra.run.smithi151.stdout: "min_score": { 2024-04-26T02:05:27.051 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "min_score", 2024-04-26T02:05:27.051 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "float", 2024-04-26T02:05:27.051 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.051 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.051 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-04-26T02:05:27.051 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.051 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.051 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.051 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-26T02:05:27.051 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.051 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.051 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.052 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.052 INFO:teuthology.orchestra.run.smithi151.stdout: "mode": { 2024-04-26T02:05:27.052 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "mode", 2024-04-26T02:05:27.052 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.052 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.052 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.052 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "upmap", 2024-04-26T02:05:27.052 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.052 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.052 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.052 INFO:teuthology.orchestra.run.smithi151.stdout: "crush-compat", 2024-04-26T02:05:27.052 INFO:teuthology.orchestra.run.smithi151.stdout: "none", 2024-04-26T02:05:27.052 INFO:teuthology.orchestra.run.smithi151.stdout: "read", 2024-04-26T02:05:27.052 INFO:teuthology.orchestra.run.smithi151.stdout: "upmap", 2024-04-26T02:05:27.052 INFO:teuthology.orchestra.run.smithi151.stdout: "upmap-read" 2024-04-26T02:05:27.053 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.053 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Balancer mode", 2024-04-26T02:05:27.053 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.053 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.053 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.053 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.053 INFO:teuthology.orchestra.run.smithi151.stdout: "pool_ids": { 2024-04-26T02:05:27.053 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "pool_ids", 2024-04-26T02:05:27.053 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.053 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.053 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.053 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.053 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.053 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.054 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.054 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-26T02:05:27.054 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.054 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.054 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.054 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.054 INFO:teuthology.orchestra.run.smithi151.stdout: "sleep_interval": { 2024-04-26T02:05:27.054 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sleep_interval", 2024-04-26T02:05:27.054 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-04-26T02:05:27.054 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.054 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.054 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "60", 2024-04-26T02:05:27.054 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.055 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.055 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.055 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-26T02:05:27.055 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.055 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.055 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.055 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.055 INFO:teuthology.orchestra.run.smithi151.stdout: "upmap_max_deviation": { 2024-04-26T02:05:27.055 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "upmap_max_deviation", 2024-04-26T02:05:27.055 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:27.055 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.055 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.055 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "5", 2024-04-26T02:05:27.055 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "1", 2024-04-26T02:05:27.055 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.056 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.056 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-26T02:05:27.056 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-26T02:05:27.056 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.056 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.056 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.056 INFO:teuthology.orchestra.run.smithi151.stdout: "upmap_max_optimizations": { 2024-04-26T02:05:27.056 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "upmap_max_optimizations", 2024-04-26T02:05:27.056 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "uint", 2024-04-26T02:05:27.056 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.056 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.056 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "10", 2024-04-26T02:05:27.056 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.056 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.056 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.057 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-26T02:05:27.057 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.057 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.057 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.057 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.057 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.057 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.057 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:27.057 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "cephadm", 2024-04-26T02:05:27.057 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:27.057 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:27.057 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:27.057 INFO:teuthology.orchestra.run.smithi151.stdout: "agent_down_multiplier": { 2024-04-26T02:05:27.057 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "agent_down_multiplier", 2024-04-26T02:05:27.057 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "float", 2024-04-26T02:05:27.057 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.058 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.058 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "3.0", 2024-04-26T02:05:27.058 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.058 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.058 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.058 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-26T02:05:27.058 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.058 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.058 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.058 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.058 INFO:teuthology.orchestra.run.smithi151.stdout: "agent_refresh_rate": { 2024-04-26T02:05:27.058 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "agent_refresh_rate", 2024-04-26T02:05:27.058 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-04-26T02:05:27.058 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.059 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.059 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "20", 2024-04-26T02:05:27.059 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.059 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.059 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.059 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-26T02:05:27.059 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.059 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.059 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.059 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.059 INFO:teuthology.orchestra.run.smithi151.stdout: "agent_starting_port": { 2024-04-26T02:05:27.059 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "agent_starting_port", 2024-04-26T02:05:27.059 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:27.059 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.059 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.059 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "4721", 2024-04-26T02:05:27.060 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.060 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.060 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.060 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-26T02:05:27.060 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.060 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.060 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.060 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.060 INFO:teuthology.orchestra.run.smithi151.stdout: "allow_ptrace": { 2024-04-26T02:05:27.060 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "allow_ptrace", 2024-04-26T02:05:27.060 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.060 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.060 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.060 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.060 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.061 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.061 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.061 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-26T02:05:27.061 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-26T02:05:27.061 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.061 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.061 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.061 INFO:teuthology.orchestra.run.smithi151.stdout: "autotune_interval": { 2024-04-26T02:05:27.061 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "autotune_interval", 2024-04-26T02:05:27.061 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-04-26T02:05:27.061 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.061 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.061 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "600", 2024-04-26T02:05:27.061 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.062 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.062 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.062 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-26T02:05:27.062 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.062 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.062 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.062 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.062 INFO:teuthology.orchestra.run.smithi151.stdout: "autotune_memory_target_ratio": { 2024-04-26T02:05:27.062 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "autotune_memory_target_ratio", 2024-04-26T02:05:27.062 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "float", 2024-04-26T02:05:27.062 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.062 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.063 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0.7", 2024-04-26T02:05:27.063 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.063 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.063 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.063 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-26T02:05:27.063 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.063 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.063 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.063 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.063 INFO:teuthology.orchestra.run.smithi151.stdout: "cephadm_log_destination": { 2024-04-26T02:05:27.063 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "cephadm_log_destination", 2024-04-26T02:05:27.063 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.063 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.063 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.063 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.063 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.064 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.064 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.064 INFO:teuthology.orchestra.run.smithi151.stdout: "file", 2024-04-26T02:05:27.064 INFO:teuthology.orchestra.run.smithi151.stdout: "file,syslog", 2024-04-26T02:05:27.064 INFO:teuthology.orchestra.run.smithi151.stdout: "syslog" 2024-04-26T02:05:27.064 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.064 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-26T02:05:27.064 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.064 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.064 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.064 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.064 INFO:teuthology.orchestra.run.smithi151.stdout: "cgroups_split": { 2024-04-26T02:05:27.064 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "cgroups_split", 2024-04-26T02:05:27.064 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.064 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.065 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.065 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:27.065 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.065 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.065 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.065 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-26T02:05:27.065 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.065 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.065 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.065 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.065 INFO:teuthology.orchestra.run.smithi151.stdout: "config_checks_enabled": { 2024-04-26T02:05:27.065 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "config_checks_enabled", 2024-04-26T02:05:27.065 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.065 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.065 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.066 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.066 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.066 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.066 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.066 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-26T02:05:27.066 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.066 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.066 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.066 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.066 INFO:teuthology.orchestra.run.smithi151.stdout: "config_dashboard": { 2024-04-26T02:05:27.066 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "config_dashboard", 2024-04-26T02:05:27.066 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.066 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.067 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.067 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:27.067 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.067 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.067 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.067 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-26T02:05:27.067 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.067 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.067 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.067 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.067 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_alertmanager": { 2024-04-26T02:05:27.067 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_alertmanager", 2024-04-26T02:05:27.067 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.067 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.068 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.068 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-26T02:05:27.068 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.068 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.068 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.068 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Prometheus container image", 2024-04-26T02:05:27.068 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.068 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.068 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.068 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.068 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_base": { 2024-04-26T02:05:27.068 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_base", 2024-04-26T02:05:27.068 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.068 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.068 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.069 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-26T02:05:27.069 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.069 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.069 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.069 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Container image name, without the tag", 2024-04-26T02:05:27.069 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.069 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.069 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.069 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.069 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_elasticsearch": { 2024-04-26T02:05:27.069 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_elasticsearch", 2024-04-26T02:05:27.069 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.069 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.069 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.070 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-26T02:05:27.070 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.070 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.070 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.070 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "elasticsearch container image", 2024-04-26T02:05:27.070 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.070 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.070 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.070 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.070 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_grafana": { 2024-04-26T02:05:27.070 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_grafana", 2024-04-26T02:05:27.070 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.070 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.070 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.070 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-26T02:05:27.071 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.071 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.071 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.071 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Prometheus container image", 2024-04-26T02:05:27.071 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.071 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.071 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.071 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.071 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_haproxy": { 2024-04-26T02:05:27.071 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_haproxy", 2024-04-26T02:05:27.071 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.071 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.071 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.071 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-26T02:05:27.072 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.072 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.072 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.072 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "HAproxy container image", 2024-04-26T02:05:27.072 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.072 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.072 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.072 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.072 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_jaeger_agent": { 2024-04-26T02:05:27.072 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_jaeger_agent", 2024-04-26T02:05:27.072 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.072 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.072 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.072 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-26T02:05:27.072 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.073 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.073 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.073 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Jaeger agent container image", 2024-04-26T02:05:27.073 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.073 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.073 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.073 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.073 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_jaeger_collector": { 2024-04-26T02:05:27.073 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_jaeger_collector", 2024-04-26T02:05:27.073 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.073 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.073 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.073 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-26T02:05:27.073 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.073 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.074 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.074 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Jaeger collector container image", 2024-04-26T02:05:27.074 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.074 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.074 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.074 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.074 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_jaeger_query": { 2024-04-26T02:05:27.074 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_jaeger_query", 2024-04-26T02:05:27.074 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.074 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.074 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.074 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-26T02:05:27.074 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.074 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.074 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.075 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Jaeger query container image", 2024-04-26T02:05:27.075 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.075 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.075 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.075 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.075 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_keepalived": { 2024-04-26T02:05:27.075 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_keepalived", 2024-04-26T02:05:27.075 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.075 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.075 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.075 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-26T02:05:27.075 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.075 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.075 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.075 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Keepalived container image", 2024-04-26T02:05:27.076 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.076 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.076 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.076 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.076 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_loki": { 2024-04-26T02:05:27.076 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_loki", 2024-04-26T02:05:27.076 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.076 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.076 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.076 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-26T02:05:27.076 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.076 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.076 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.076 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Loki container image", 2024-04-26T02:05:27.076 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.076 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.077 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.077 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.077 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_node_exporter": { 2024-04-26T02:05:27.077 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_node_exporter", 2024-04-26T02:05:27.077 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.077 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.077 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.077 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-26T02:05:27.077 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.077 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.077 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.077 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Prometheus container image", 2024-04-26T02:05:27.077 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.077 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.077 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.078 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.078 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_nvmeof": { 2024-04-26T02:05:27.078 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_nvmeof", 2024-04-26T02:05:27.078 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.078 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.078 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.078 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-26T02:05:27.078 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.078 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.078 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.078 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Nvme-of container image", 2024-04-26T02:05:27.078 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.078 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.079 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.079 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.079 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_prometheus": { 2024-04-26T02:05:27.079 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_prometheus", 2024-04-26T02:05:27.079 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.079 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.079 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.079 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-26T02:05:27.079 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.079 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.079 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.079 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Prometheus container image", 2024-04-26T02:05:27.079 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.079 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.080 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.080 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.080 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_promtail": { 2024-04-26T02:05:27.080 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_promtail", 2024-04-26T02:05:27.080 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.080 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.080 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.080 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-26T02:05:27.080 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.080 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.080 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.080 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Promtail container image", 2024-04-26T02:05:27.080 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.080 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.081 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.081 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.081 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_samba": { 2024-04-26T02:05:27.081 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_samba", 2024-04-26T02:05:27.081 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.081 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.081 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.081 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-26T02:05:27.081 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.081 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.081 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.081 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Samba/SMB container image", 2024-04-26T02:05:27.081 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.081 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.082 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.082 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.082 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_snmp_gateway": { 2024-04-26T02:05:27.082 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_snmp_gateway", 2024-04-26T02:05:27.082 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.082 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.082 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.082 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-26T02:05:27.082 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.082 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.082 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.082 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "SNMP Gateway container image", 2024-04-26T02:05:27.083 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.083 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.083 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.083 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.083 INFO:teuthology.orchestra.run.smithi151.stdout: "container_init": { 2024-04-26T02:05:27.083 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_init", 2024-04-26T02:05:27.083 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.083 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.083 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.083 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:27.083 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.083 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.083 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.084 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Run podman/docker with `--init`", 2024-04-26T02:05:27.084 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.084 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.084 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.084 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.084 INFO:teuthology.orchestra.run.smithi151.stdout: "daemon_cache_timeout": { 2024-04-26T02:05:27.084 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "daemon_cache_timeout", 2024-04-26T02:05:27.084 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-04-26T02:05:27.084 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.084 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.084 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "600", 2024-04-26T02:05:27.084 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.084 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.084 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.085 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-26T02:05:27.085 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.085 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.085 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.085 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.085 INFO:teuthology.orchestra.run.smithi151.stdout: "default_cephadm_command_timeout": { 2024-04-26T02:05:27.085 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "default_cephadm_command_timeout", 2024-04-26T02:05:27.085 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:27.085 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.085 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.085 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "900", 2024-04-26T02:05:27.085 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.085 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.085 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.085 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-26T02:05:27.086 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.086 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.086 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.086 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.086 INFO:teuthology.orchestra.run.smithi151.stdout: "default_registry": { 2024-04-26T02:05:27.086 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "default_registry", 2024-04-26T02:05:27.086 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.086 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.086 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.086 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "docker.io", 2024-04-26T02:05:27.086 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.086 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.086 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.087 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-26T02:05:27.087 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.087 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.087 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.087 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.087 INFO:teuthology.orchestra.run.smithi151.stdout: "device_cache_timeout": { 2024-04-26T02:05:27.087 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "device_cache_timeout", 2024-04-26T02:05:27.087 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-04-26T02:05:27.087 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.087 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.087 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "1800", 2024-04-26T02:05:27.087 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.087 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.088 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.088 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "seconds to cache device inventory", 2024-04-26T02:05:27.088 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.088 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.088 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.088 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.088 INFO:teuthology.orchestra.run.smithi151.stdout: "device_enhanced_scan": { 2024-04-26T02:05:27.088 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "device_enhanced_scan", 2024-04-26T02:05:27.088 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.088 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.088 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.088 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.088 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.088 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.089 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.089 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-26T02:05:27.089 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.089 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.089 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.089 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.089 INFO:teuthology.orchestra.run.smithi151.stdout: "facts_cache_timeout": { 2024-04-26T02:05:27.089 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "facts_cache_timeout", 2024-04-26T02:05:27.089 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-04-26T02:05:27.089 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.089 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.089 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "60", 2024-04-26T02:05:27.089 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.089 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.090 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.090 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "seconds to cache host facts data", 2024-04-26T02:05:27.090 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.090 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.090 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.090 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.090 INFO:teuthology.orchestra.run.smithi151.stdout: "grafana_dashboards_path": { 2024-04-26T02:05:27.090 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "grafana_dashboards_path", 2024-04-26T02:05:27.090 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.090 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.090 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.090 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-26T02:05:27.090 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.090 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.091 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.091 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-26T02:05:27.091 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.091 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.091 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.091 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.091 INFO:teuthology.orchestra.run.smithi151.stdout: "host_check_interval": { 2024-04-26T02:05:27.091 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "host_check_interval", 2024-04-26T02:05:27.091 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-04-26T02:05:27.091 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.091 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.091 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "600", 2024-04-26T02:05:27.091 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.092 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.092 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.092 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "how frequently to perform a host check", 2024-04-26T02:05:27.092 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.092 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.092 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.092 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.092 INFO:teuthology.orchestra.run.smithi151.stdout: "hw_monitoring": { 2024-04-26T02:05:27.092 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "hw_monitoring", 2024-04-26T02:05:27.092 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.092 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.092 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.092 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.092 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.093 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.093 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.093 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-26T02:05:27.093 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.093 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.093 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.093 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.093 INFO:teuthology.orchestra.run.smithi151.stdout: "inventory_list_all": { 2024-04-26T02:05:27.093 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "inventory_list_all", 2024-04-26T02:05:27.093 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.093 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.093 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.093 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.094 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.094 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.094 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.094 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-26T02:05:27.094 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.094 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.094 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.094 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.094 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:27.094 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:27.094 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.094 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.094 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.094 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.095 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.095 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.095 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.095 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.095 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.095 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.095 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.095 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.095 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.095 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.095 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.095 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.095 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.096 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.096 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.096 INFO:teuthology.orchestra.run.smithi151.stdout: "log_refresh_metadata": { 2024-04-26T02:05:27.096 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_refresh_metadata", 2024-04-26T02:05:27.096 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.096 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.096 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.096 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.096 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.096 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.096 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.096 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-26T02:05:27.096 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.096 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.097 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.097 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.097 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:27.097 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:27.097 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.097 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.097 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.097 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:27.097 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.097 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.097 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.097 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-26T02:05:27.097 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.097 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.098 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.098 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.098 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:27.098 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:27.098 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.098 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.098 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.098 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:27.098 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.098 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.098 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.098 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.098 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.098 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.099 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.099 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.099 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.099 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.099 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.099 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.099 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.099 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.099 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.099 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:27.099 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:27.099 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.099 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.099 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.099 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.100 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.100 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.100 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.100 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.100 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.100 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.100 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.100 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.100 INFO:teuthology.orchestra.run.smithi151.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-26T02:05:27.100 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-26T02:05:27.100 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.100 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.100 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.100 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.101 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.101 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.101 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.101 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-26T02:05:27.101 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.101 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.101 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.101 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.101 INFO:teuthology.orchestra.run.smithi151.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-26T02:05:27.101 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-26T02:05:27.101 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.101 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.101 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.101 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "*", 2024-04-26T02:05:27.102 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.102 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.102 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.102 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-26T02:05:27.102 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.102 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.102 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.102 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.102 INFO:teuthology.orchestra.run.smithi151.stdout: "max_count_per_host": { 2024-04-26T02:05:27.102 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "max_count_per_host", 2024-04-26T02:05:27.102 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:27.102 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.102 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.102 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "10", 2024-04-26T02:05:27.103 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.103 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.103 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.103 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "max number of daemons per service per host", 2024-04-26T02:05:27.103 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.103 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.103 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.103 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.103 INFO:teuthology.orchestra.run.smithi151.stdout: "max_osd_draining_count": { 2024-04-26T02:05:27.103 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "max_osd_draining_count", 2024-04-26T02:05:27.103 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:27.103 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.103 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.103 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "10", 2024-04-26T02:05:27.103 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.104 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.104 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.104 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-26T02:05:27.104 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.104 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.104 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.104 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.104 INFO:teuthology.orchestra.run.smithi151.stdout: "migration_current": { 2024-04-26T02:05:27.104 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "migration_current", 2024-04-26T02:05:27.104 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:27.104 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.104 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.104 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.104 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.104 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.105 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.105 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "internal - do not modify", 2024-04-26T02:05:27.105 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.105 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.105 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.105 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.105 INFO:teuthology.orchestra.run.smithi151.stdout: "mode": { 2024-04-26T02:05:27.105 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "mode", 2024-04-26T02:05:27.105 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.105 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.105 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.105 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "root", 2024-04-26T02:05:27.105 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.105 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.105 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.106 INFO:teuthology.orchestra.run.smithi151.stdout: "cephadm-package", 2024-04-26T02:05:27.106 INFO:teuthology.orchestra.run.smithi151.stdout: "root" 2024-04-26T02:05:27.106 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.106 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "mode for remote execution of cephadm", 2024-04-26T02:05:27.106 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.106 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.106 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.106 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.106 INFO:teuthology.orchestra.run.smithi151.stdout: "oob_default_addr": { 2024-04-26T02:05:27.106 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "oob_default_addr", 2024-04-26T02:05:27.106 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.106 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.106 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.107 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "169.254.1.1", 2024-04-26T02:05:27.107 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.107 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.107 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.107 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-26T02:05:27.107 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.107 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.107 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.107 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.107 INFO:teuthology.orchestra.run.smithi151.stdout: "prometheus_alerts_path": { 2024-04-26T02:05:27.107 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "prometheus_alerts_path", 2024-04-26T02:05:27.107 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.107 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.107 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.107 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-26T02:05:27.107 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.108 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.108 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.108 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-26T02:05:27.108 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.108 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.108 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.108 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.108 INFO:teuthology.orchestra.run.smithi151.stdout: "registry_insecure": { 2024-04-26T02:05:27.108 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "registry_insecure", 2024-04-26T02:05:27.108 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.108 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.108 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.108 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.108 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.108 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.109 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.109 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-26T02:05:27.109 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.109 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.109 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.109 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.109 INFO:teuthology.orchestra.run.smithi151.stdout: "registry_password": { 2024-04-26T02:05:27.109 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "registry_password", 2024-04-26T02:05:27.109 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.109 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.109 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.109 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.109 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.109 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.110 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.110 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-26T02:05:27.110 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.110 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.110 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.110 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.110 INFO:teuthology.orchestra.run.smithi151.stdout: "registry_url": { 2024-04-26T02:05:27.110 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "registry_url", 2024-04-26T02:05:27.110 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.110 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.110 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.110 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.110 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.111 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.111 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.111 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-26T02:05:27.111 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.111 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.111 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.111 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.111 INFO:teuthology.orchestra.run.smithi151.stdout: "registry_username": { 2024-04-26T02:05:27.111 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "registry_username", 2024-04-26T02:05:27.111 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.111 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.111 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.111 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.112 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.112 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.112 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.112 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-26T02:05:27.112 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.112 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.112 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.112 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.112 INFO:teuthology.orchestra.run.smithi151.stdout: "secure_monitoring_stack": { 2024-04-26T02:05:27.112 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "secure_monitoring_stack", 2024-04-26T02:05:27.112 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.112 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.112 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.112 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.113 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.113 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.113 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.113 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-26T02:05:27.113 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.113 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.113 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.113 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.113 INFO:teuthology.orchestra.run.smithi151.stdout: "service_discovery_port": { 2024-04-26T02:05:27.113 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "service_discovery_port", 2024-04-26T02:05:27.113 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:27.113 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.113 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.113 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "8765", 2024-04-26T02:05:27.113 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.114 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.114 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.114 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "cephadm service discovery port", 2024-04-26T02:05:27.114 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.114 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.114 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.114 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.114 INFO:teuthology.orchestra.run.smithi151.stdout: "ssh_config_file": { 2024-04-26T02:05:27.114 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ssh_config_file", 2024-04-26T02:05:27.114 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.114 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.114 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.114 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.114 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.115 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.115 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.115 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-26T02:05:27.115 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.115 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.115 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.115 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.115 INFO:teuthology.orchestra.run.smithi151.stdout: "use_agent": { 2024-04-26T02:05:27.115 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "use_agent", 2024-04-26T02:05:27.115 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.115 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.115 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.115 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.115 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.116 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.116 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.116 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-26T02:05:27.116 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.116 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.116 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.116 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.116 INFO:teuthology.orchestra.run.smithi151.stdout: "use_repo_digest": { 2024-04-26T02:05:27.116 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "use_repo_digest", 2024-04-26T02:05:27.116 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.116 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.116 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.116 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:27.116 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.116 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.117 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.117 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-26T02:05:27.117 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.117 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.117 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.117 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.117 INFO:teuthology.orchestra.run.smithi151.stdout: "warn_on_failed_host_check": { 2024-04-26T02:05:27.117 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "warn_on_failed_host_check", 2024-04-26T02:05:27.117 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.117 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.117 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.117 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:27.117 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.117 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.118 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.118 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "raise a health warning if the host check fails", 2024-04-26T02:05:27.118 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.118 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.118 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.118 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.118 INFO:teuthology.orchestra.run.smithi151.stdout: "warn_on_stray_daemons": { 2024-04-26T02:05:27.118 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "warn_on_stray_daemons", 2024-04-26T02:05:27.118 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.118 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.118 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.118 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:27.118 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.118 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.119 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.119 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-26T02:05:27.119 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.119 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.119 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.119 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.119 INFO:teuthology.orchestra.run.smithi151.stdout: "warn_on_stray_hosts": { 2024-04-26T02:05:27.119 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "warn_on_stray_hosts", 2024-04-26T02:05:27.119 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.119 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.119 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.119 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:27.119 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.119 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.120 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.120 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-26T02:05:27.120 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.120 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.120 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.120 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.120 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.120 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.120 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:27.120 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "crash", 2024-04-26T02:05:27.120 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:27.120 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:27.120 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:27.120 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:27.121 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:27.121 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.121 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.121 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.121 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.121 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.121 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.121 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.121 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.121 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.121 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.121 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.121 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.121 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.121 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.122 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.122 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.122 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.122 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.122 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.122 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:27.122 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:27.122 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.122 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.122 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.122 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.122 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.122 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.123 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.123 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.123 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.123 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.123 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.123 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.123 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:27.123 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:27.123 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.123 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.123 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.123 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:27.123 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.123 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.123 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.123 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.124 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.124 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.124 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.124 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.124 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.124 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.124 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.124 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.124 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.124 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.124 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.124 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:27.124 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:27.124 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.124 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.124 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.125 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.125 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.125 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.125 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.125 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.125 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.125 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.125 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.125 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.125 INFO:teuthology.orchestra.run.smithi151.stdout: "retain_interval": { 2024-04-26T02:05:27.125 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "retain_interval", 2024-04-26T02:05:27.125 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-04-26T02:05:27.125 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.125 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.125 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "31536000", 2024-04-26T02:05:27.125 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.126 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.126 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.126 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-26T02:05:27.126 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.126 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.126 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.126 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.126 INFO:teuthology.orchestra.run.smithi151.stdout: "warn_recent_interval": { 2024-04-26T02:05:27.126 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "warn_recent_interval", 2024-04-26T02:05:27.126 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-04-26T02:05:27.126 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.126 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.126 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "1209600", 2024-04-26T02:05:27.126 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.126 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.127 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.127 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-26T02:05:27.127 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.127 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.127 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.127 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.127 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.127 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.127 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:27.127 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "dashboard", 2024-04-26T02:05:27.127 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:27.127 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:27.127 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:27.127 INFO:teuthology.orchestra.run.smithi151.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-26T02:05:27.127 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-26T02:05:27.128 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:27.128 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.128 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.128 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "10", 2024-04-26T02:05:27.128 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.128 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.128 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.128 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.128 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.128 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.128 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.128 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.128 INFO:teuthology.orchestra.run.smithi151.stdout: "ALERTMANAGER_API_HOST": { 2024-04-26T02:05:27.128 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-26T02:05:27.128 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.128 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.129 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.129 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.129 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.129 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.129 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.129 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.129 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.129 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.129 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.129 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.129 INFO:teuthology.orchestra.run.smithi151.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-26T02:05:27.129 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-26T02:05:27.129 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.129 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.129 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.130 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:27.130 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.130 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.130 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.130 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.130 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.130 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.130 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.130 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.130 INFO:teuthology.orchestra.run.smithi151.stdout: "AUDIT_API_ENABLED": { 2024-04-26T02:05:27.130 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "AUDIT_API_ENABLED", 2024-04-26T02:05:27.130 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.130 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.131 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.131 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.131 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.131 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.131 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.131 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.131 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.131 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.131 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.131 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.131 INFO:teuthology.orchestra.run.smithi151.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-26T02:05:27.131 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-26T02:05:27.131 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.131 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.132 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.132 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:27.132 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.132 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.132 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.132 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.132 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.132 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.132 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.132 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.132 INFO:teuthology.orchestra.run.smithi151.stdout: "ENABLE_BROWSABLE_API": { 2024-04-26T02:05:27.132 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-26T02:05:27.132 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.133 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.133 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.133 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:27.133 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.133 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.133 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.133 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.133 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.133 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.133 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.133 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.133 INFO:teuthology.orchestra.run.smithi151.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-26T02:05:27.133 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-26T02:05:27.134 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.134 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.134 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.134 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:27.134 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.134 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.134 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.134 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.134 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.134 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.134 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.134 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.134 INFO:teuthology.orchestra.run.smithi151.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-26T02:05:27.134 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-26T02:05:27.135 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.135 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.135 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.135 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:27.135 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.135 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.135 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.135 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.135 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.135 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.135 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.135 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.135 INFO:teuthology.orchestra.run.smithi151.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-26T02:05:27.135 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-26T02:05:27.136 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.136 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.136 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.136 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:27.136 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.136 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.136 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.136 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.136 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.136 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.136 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.136 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.136 INFO:teuthology.orchestra.run.smithi151.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-26T02:05:27.136 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-26T02:05:27.137 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.137 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.137 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.137 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:27.137 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.137 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.137 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.137 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.137 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.137 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.137 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.137 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.137 INFO:teuthology.orchestra.run.smithi151.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-26T02:05:27.137 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-26T02:05:27.137 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.138 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.138 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.138 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:27.138 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.138 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.138 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.138 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.138 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.138 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.138 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.138 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.138 INFO:teuthology.orchestra.run.smithi151.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-26T02:05:27.138 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-26T02:05:27.139 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.139 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.139 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.139 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:27.139 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.139 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.139 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.139 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.139 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.139 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.139 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.139 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.139 INFO:teuthology.orchestra.run.smithi151.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-26T02:05:27.139 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-26T02:05:27.139 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.140 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.140 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.140 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:27.140 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.140 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.140 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.140 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.140 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.140 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.140 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.140 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.140 INFO:teuthology.orchestra.run.smithi151.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-26T02:05:27.140 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-26T02:05:27.140 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.140 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.141 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.141 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.141 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.141 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.141 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.141 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.141 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.141 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.141 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.141 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.141 INFO:teuthology.orchestra.run.smithi151.stdout: "GRAFANA_API_PASSWORD": { 2024-04-26T02:05:27.141 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-26T02:05:27.141 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.141 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.142 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.142 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "admin", 2024-04-26T02:05:27.142 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.142 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.142 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.142 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.142 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.142 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.142 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.142 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.142 INFO:teuthology.orchestra.run.smithi151.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-26T02:05:27.142 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-26T02:05:27.142 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.142 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.143 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.143 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:27.143 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.143 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.143 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.143 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.143 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.143 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.143 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.143 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.143 INFO:teuthology.orchestra.run.smithi151.stdout: "GRAFANA_API_URL": { 2024-04-26T02:05:27.143 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "GRAFANA_API_URL", 2024-04-26T02:05:27.143 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.143 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.144 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.144 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.144 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.144 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.144 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.144 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.144 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.144 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.144 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.144 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.144 INFO:teuthology.orchestra.run.smithi151.stdout: "GRAFANA_API_USERNAME": { 2024-04-26T02:05:27.144 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-26T02:05:27.144 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.144 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.144 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.145 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "admin", 2024-04-26T02:05:27.145 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.145 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.145 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.145 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.145 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.145 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.145 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.145 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.145 INFO:teuthology.orchestra.run.smithi151.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-26T02:05:27.145 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-26T02:05:27.145 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.145 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.145 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.146 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.146 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.146 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.146 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.146 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.146 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.146 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.146 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.146 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.146 INFO:teuthology.orchestra.run.smithi151.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-26T02:05:27.146 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-26T02:05:27.146 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.146 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.146 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.147 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.147 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.147 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.147 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.147 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.147 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.147 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.147 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.147 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.147 INFO:teuthology.orchestra.run.smithi151.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-26T02:05:27.147 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-26T02:05:27.147 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.147 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.147 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.148 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:27.148 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.148 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.148 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.148 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.148 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.148 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.148 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.148 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.148 INFO:teuthology.orchestra.run.smithi151.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-26T02:05:27.148 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-26T02:05:27.148 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.148 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.148 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.149 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.149 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.149 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.149 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.149 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.149 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.149 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.149 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.149 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.149 INFO:teuthology.orchestra.run.smithi151.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-26T02:05:27.149 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-26T02:05:27.149 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.149 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.149 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.150 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "[]", 2024-04-26T02:05:27.150 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.150 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.150 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.150 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.150 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.150 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.150 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.150 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.150 INFO:teuthology.orchestra.run.smithi151.stdout: "MULTICLUSTER_CONFIG": { 2024-04-26T02:05:27.150 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-26T02:05:27.150 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.150 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.150 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.151 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "{}", 2024-04-26T02:05:27.151 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.151 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.151 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.151 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.151 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.151 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.151 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.151 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.151 INFO:teuthology.orchestra.run.smithi151.stdout: "PROMETHEUS_API_HOST": { 2024-04-26T02:05:27.151 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-26T02:05:27.151 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.151 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.151 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.152 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.152 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.152 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.152 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.152 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.152 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.152 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.152 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.152 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.152 INFO:teuthology.orchestra.run.smithi151.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-26T02:05:27.152 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-26T02:05:27.152 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.152 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.152 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.153 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:27.153 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.153 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.153 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.153 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.153 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.153 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.153 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.153 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.153 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-26T02:05:27.153 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-26T02:05:27.153 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.153 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.153 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.154 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.154 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.154 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.154 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.154 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.154 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.154 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.154 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.154 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.154 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-26T02:05:27.154 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-26T02:05:27.154 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.154 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.155 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.155 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.155 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.155 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.155 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.155 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.155 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.155 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.155 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.155 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.155 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-26T02:05:27.155 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-26T02:05:27.155 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.155 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.156 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.156 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:27.156 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.156 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.156 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.156 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.156 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.156 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.156 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.156 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.156 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-26T02:05:27.156 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-26T02:05:27.156 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.156 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.157 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.157 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:27.157 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.157 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.157 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.157 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.157 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.157 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.157 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.157 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.157 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-26T02:05:27.157 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-26T02:05:27.157 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.157 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.158 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.158 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.158 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.158 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.158 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.158 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.158 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.158 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.158 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.158 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.158 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-26T02:05:27.158 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-26T02:05:27.158 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.159 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.159 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.159 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.159 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.159 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.159 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.159 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.159 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.159 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.159 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.159 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.159 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-26T02:05:27.159 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-26T02:05:27.159 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.160 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.160 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.160 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.160 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.160 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.160 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.160 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.160 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.160 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.160 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.160 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.160 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_ENABLED": { 2024-04-26T02:05:27.160 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-26T02:05:27.160 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.161 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.161 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.161 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:27.161 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.161 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.161 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.161 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.161 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.161 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.161 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.161 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.161 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-26T02:05:27.161 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-26T02:05:27.162 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.162 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.162 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.162 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-26T02:05:27.162 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.162 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.162 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.162 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.162 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.162 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.162 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.162 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.162 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-26T02:05:27.163 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-26T02:05:27.163 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:27.163 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.163 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.163 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "10", 2024-04-26T02:05:27.163 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.163 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.163 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.163 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.163 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.163 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.163 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.163 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.163 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-26T02:05:27.164 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-26T02:05:27.164 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:27.164 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.164 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.164 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "8", 2024-04-26T02:05:27.164 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.164 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.164 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.164 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.164 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.164 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.164 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.164 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.164 INFO:teuthology.orchestra.run.smithi151.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-26T02:05:27.164 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-26T02:05:27.165 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:27.165 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.165 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.165 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "45", 2024-04-26T02:05:27.165 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.165 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.165 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.165 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.165 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.165 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.165 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.165 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.165 INFO:teuthology.orchestra.run.smithi151.stdout: "RGW_API_ACCESS_KEY": { 2024-04-26T02:05:27.165 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-26T02:05:27.166 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.166 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.166 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.166 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.166 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.166 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.166 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.166 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.166 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.166 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.166 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.166 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.166 INFO:teuthology.orchestra.run.smithi151.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-26T02:05:27.167 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-26T02:05:27.167 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.167 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.167 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.167 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "admin", 2024-04-26T02:05:27.167 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.167 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.167 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.167 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.167 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.167 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.167 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.167 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.168 INFO:teuthology.orchestra.run.smithi151.stdout: "RGW_API_SECRET_KEY": { 2024-04-26T02:05:27.168 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-26T02:05:27.168 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.168 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.168 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.168 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.168 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.168 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.168 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.168 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.168 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.168 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.168 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.168 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.169 INFO:teuthology.orchestra.run.smithi151.stdout: "RGW_API_SSL_VERIFY": { 2024-04-26T02:05:27.169 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-26T02:05:27.169 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.169 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.169 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.169 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:27.169 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.169 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.169 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.169 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.169 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.169 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.169 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.170 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.170 INFO:teuthology.orchestra.run.smithi151.stdout: "UNSAFE_TLS_v1_2": { 2024-04-26T02:05:27.170 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-26T02:05:27.170 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.170 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.170 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.170 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.170 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.170 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.170 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.170 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.170 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.170 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.170 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.171 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.171 INFO:teuthology.orchestra.run.smithi151.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-26T02:05:27.171 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-26T02:05:27.171 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:27.171 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.171 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.171 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-04-26T02:05:27.171 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.171 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.171 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.171 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.171 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.171 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.172 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.172 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.172 INFO:teuthology.orchestra.run.smithi151.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-26T02:05:27.172 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-26T02:05:27.172 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:27.172 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.172 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.172 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "10", 2024-04-26T02:05:27.172 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.172 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.172 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.172 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.172 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.173 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.173 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.173 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.173 INFO:teuthology.orchestra.run.smithi151.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-26T02:05:27.173 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-26T02:05:27.173 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:27.173 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.173 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.173 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "5", 2024-04-26T02:05:27.173 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.173 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.173 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.173 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.173 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.173 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.174 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.174 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.174 INFO:teuthology.orchestra.run.smithi151.stdout: "cross_origin_url": { 2024-04-26T02:05:27.174 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "cross_origin_url", 2024-04-26T02:05:27.174 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.174 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.174 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.174 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.174 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.174 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.174 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.174 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.174 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.174 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.175 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.175 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.175 INFO:teuthology.orchestra.run.smithi151.stdout: "crt_file": { 2024-04-26T02:05:27.175 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "crt_file", 2024-04-26T02:05:27.175 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.175 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.175 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.175 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.175 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.175 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.175 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.175 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.175 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.176 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.176 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.176 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.176 INFO:teuthology.orchestra.run.smithi151.stdout: "debug": { 2024-04-26T02:05:27.176 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "debug", 2024-04-26T02:05:27.176 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.176 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.176 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.176 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.176 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.176 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.176 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.176 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Enable/disable debug options", 2024-04-26T02:05:27.176 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.177 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.177 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.177 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.177 INFO:teuthology.orchestra.run.smithi151.stdout: "jwt_token_ttl": { 2024-04-26T02:05:27.177 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "jwt_token_ttl", 2024-04-26T02:05:27.177 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:27.177 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.177 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.177 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "28800", 2024-04-26T02:05:27.177 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.177 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.177 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.177 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.177 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.178 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.178 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.178 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.178 INFO:teuthology.orchestra.run.smithi151.stdout: "key_file": { 2024-04-26T02:05:27.178 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "key_file", 2024-04-26T02:05:27.178 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.178 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.178 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.178 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.178 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.178 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.178 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.178 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.179 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.179 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.179 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.179 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.179 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:27.179 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:27.179 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.179 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.179 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.179 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.179 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.179 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.179 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.180 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.180 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.180 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.180 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.180 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.180 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.180 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.180 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.180 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.180 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.180 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.180 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.180 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:27.180 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:27.181 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.181 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.181 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.181 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.181 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.181 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.181 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.181 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.181 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.181 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.181 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.181 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.181 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:27.181 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:27.181 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.182 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.182 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.182 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:27.182 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.182 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.182 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.182 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.182 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.182 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.182 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.182 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.182 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.182 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.182 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.182 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.183 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.183 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.183 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.183 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:27.183 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:27.183 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.183 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.183 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.183 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.183 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.183 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.183 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.183 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.183 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.184 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.184 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.184 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.184 INFO:teuthology.orchestra.run.smithi151.stdout: "motd": { 2024-04-26T02:05:27.184 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "motd", 2024-04-26T02:05:27.184 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.184 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.184 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.184 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.184 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.184 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.184 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.184 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "The message of the day", 2024-04-26T02:05:27.184 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.184 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.185 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.185 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.185 INFO:teuthology.orchestra.run.smithi151.stdout: "redirect_resolve_ip_addr": { 2024-04-26T02:05:27.185 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "redirect_resolve_ip_addr", 2024-04-26T02:05:27.185 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.185 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.185 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.185 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.185 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.185 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.185 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.185 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.185 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.185 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.185 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.186 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.186 INFO:teuthology.orchestra.run.smithi151.stdout: "server_addr": { 2024-04-26T02:05:27.186 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "server_addr", 2024-04-26T02:05:27.186 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.186 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.186 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.186 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "::", 2024-04-26T02:05:27.186 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.186 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.186 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.186 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.186 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.186 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.187 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.187 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.187 INFO:teuthology.orchestra.run.smithi151.stdout: "server_port": { 2024-04-26T02:05:27.187 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "server_port", 2024-04-26T02:05:27.187 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:27.187 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.187 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.187 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "8080", 2024-04-26T02:05:27.187 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.187 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.187 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.187 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.187 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.187 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.188 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.188 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.188 INFO:teuthology.orchestra.run.smithi151.stdout: "ssl": { 2024-04-26T02:05:27.188 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ssl", 2024-04-26T02:05:27.188 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.188 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.188 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.188 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:27.188 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.188 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.188 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.188 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.188 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.188 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.188 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.189 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.189 INFO:teuthology.orchestra.run.smithi151.stdout: "ssl_server_port": { 2024-04-26T02:05:27.189 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ssl_server_port", 2024-04-26T02:05:27.189 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:27.189 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.189 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.189 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "8443", 2024-04-26T02:05:27.189 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.189 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.189 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.189 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.189 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.189 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.189 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.190 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.190 INFO:teuthology.orchestra.run.smithi151.stdout: "standby_behaviour": { 2024-04-26T02:05:27.190 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "standby_behaviour", 2024-04-26T02:05:27.190 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.190 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.190 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.190 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "redirect", 2024-04-26T02:05:27.190 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.190 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.190 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.190 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.190 INFO:teuthology.orchestra.run.smithi151.stdout: "redirect" 2024-04-26T02:05:27.190 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.190 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.191 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.191 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.191 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.191 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.191 INFO:teuthology.orchestra.run.smithi151.stdout: "standby_error_status_code": { 2024-04-26T02:05:27.191 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "standby_error_status_code", 2024-04-26T02:05:27.191 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:27.191 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.191 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.191 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "500", 2024-04-26T02:05:27.191 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "400", 2024-04-26T02:05:27.191 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "599", 2024-04-26T02:05:27.191 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.191 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.192 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.192 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.192 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.192 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.192 INFO:teuthology.orchestra.run.smithi151.stdout: "url_prefix": { 2024-04-26T02:05:27.192 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "url_prefix", 2024-04-26T02:05:27.192 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.192 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.192 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.192 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.192 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.192 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.192 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.193 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.193 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.193 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.193 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.193 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.193 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.193 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.193 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:27.193 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "devicehealth", 2024-04-26T02:05:27.193 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:27.193 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:27.193 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:27.193 INFO:teuthology.orchestra.run.smithi151.stdout: "enable_monitoring": { 2024-04-26T02:05:27.193 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "enable_monitoring", 2024-04-26T02:05:27.194 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.194 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.194 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.194 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:27.194 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.194 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.194 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.194 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "monitor device health metrics", 2024-04-26T02:05:27.194 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.194 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.194 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.194 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.194 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:27.194 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:27.195 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.195 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.195 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.195 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.195 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.195 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.195 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.195 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.195 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.195 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.195 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.195 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.195 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.195 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.195 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.196 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.196 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.196 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.196 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.196 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:27.196 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:27.196 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.196 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.196 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.196 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.196 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.196 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.196 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.196 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.196 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.197 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.197 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.197 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.197 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:27.197 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:27.197 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.197 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.197 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.197 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:27.197 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.197 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.197 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.197 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.197 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.197 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.198 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.198 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.198 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.198 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.198 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.198 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.198 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.198 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.198 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.198 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:27.198 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:27.198 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.198 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.198 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.199 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.199 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.199 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.199 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.199 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.199 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.199 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.199 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.199 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.199 INFO:teuthology.orchestra.run.smithi151.stdout: "mark_out_threshold": { 2024-04-26T02:05:27.199 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "mark_out_threshold", 2024-04-26T02:05:27.199 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-04-26T02:05:27.199 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.199 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.199 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "2419200", 2024-04-26T02:05:27.199 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.200 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.200 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.200 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-26T02:05:27.200 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.200 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.200 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.200 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.200 INFO:teuthology.orchestra.run.smithi151.stdout: "pool_name": { 2024-04-26T02:05:27.200 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "pool_name", 2024-04-26T02:05:27.200 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.200 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.200 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.200 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "device_health_metrics", 2024-04-26T02:05:27.200 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.201 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.201 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.201 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-26T02:05:27.201 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.201 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.201 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.201 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.201 INFO:teuthology.orchestra.run.smithi151.stdout: "retention_period": { 2024-04-26T02:05:27.201 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "retention_period", 2024-04-26T02:05:27.201 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-04-26T02:05:27.201 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.201 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.201 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "15552000", 2024-04-26T02:05:27.201 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.202 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.202 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.202 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "how long to retain device health metrics", 2024-04-26T02:05:27.202 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.202 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.202 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.202 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.202 INFO:teuthology.orchestra.run.smithi151.stdout: "scrape_frequency": { 2024-04-26T02:05:27.202 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "scrape_frequency", 2024-04-26T02:05:27.202 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-04-26T02:05:27.203 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.203 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.203 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "86400", 2024-04-26T02:05:27.203 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.203 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.203 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.203 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-26T02:05:27.203 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.203 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.203 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.203 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.203 INFO:teuthology.orchestra.run.smithi151.stdout: "self_heal": { 2024-04-26T02:05:27.203 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "self_heal", 2024-04-26T02:05:27.204 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.204 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.204 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.204 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:27.204 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.204 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.204 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.204 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-26T02:05:27.204 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.204 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.204 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.204 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.204 INFO:teuthology.orchestra.run.smithi151.stdout: "sleep_interval": { 2024-04-26T02:05:27.204 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sleep_interval", 2024-04-26T02:05:27.205 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-04-26T02:05:27.205 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.205 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.205 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "600", 2024-04-26T02:05:27.205 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.205 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.205 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.205 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "how frequently to wake up and check device health", 2024-04-26T02:05:27.205 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.205 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.205 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.205 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.205 INFO:teuthology.orchestra.run.smithi151.stdout: "warn_threshold": { 2024-04-26T02:05:27.205 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "warn_threshold", 2024-04-26T02:05:27.205 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-04-26T02:05:27.206 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.206 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.206 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "7257600", 2024-04-26T02:05:27.206 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.206 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.206 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.206 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-26T02:05:27.206 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.206 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.206 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.206 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.206 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.206 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.207 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:27.207 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "diskprediction_local", 2024-04-26T02:05:27.207 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:27.207 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:27.207 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:27.207 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:27.207 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:27.207 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.207 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.207 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.207 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.207 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.207 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.207 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.207 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.208 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.208 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.208 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.208 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.208 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.208 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.208 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.208 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.208 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.208 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.208 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.208 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:27.208 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:27.208 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.209 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.209 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.209 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.209 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.209 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.209 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.209 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.209 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.209 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.209 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.209 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.209 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:27.209 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:27.209 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.209 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.210 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.210 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:27.210 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.210 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.210 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.210 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.210 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.210 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.210 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.210 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.210 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.210 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.210 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.210 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.211 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.211 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.211 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.211 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:27.211 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:27.211 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.211 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.211 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.211 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.211 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.211 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.211 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.211 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.211 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.212 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.212 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.212 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.212 INFO:teuthology.orchestra.run.smithi151.stdout: "predict_interval": { 2024-04-26T02:05:27.212 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "predict_interval", 2024-04-26T02:05:27.212 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.212 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.212 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.212 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "86400", 2024-04-26T02:05:27.212 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.212 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.212 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.212 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.212 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.213 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.213 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.213 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.213 INFO:teuthology.orchestra.run.smithi151.stdout: "predictor_model": { 2024-04-26T02:05:27.213 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "predictor_model", 2024-04-26T02:05:27.213 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.213 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.213 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.213 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "prophetstor", 2024-04-26T02:05:27.213 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.213 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.213 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.213 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.213 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.214 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.214 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.214 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.214 INFO:teuthology.orchestra.run.smithi151.stdout: "sleep_interval": { 2024-04-26T02:05:27.214 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sleep_interval", 2024-04-26T02:05:27.214 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.214 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.214 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.214 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "600", 2024-04-26T02:05:27.214 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.214 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.214 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.214 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.215 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.215 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.215 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.215 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.215 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.215 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.215 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:27.215 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "influx", 2024-04-26T02:05:27.215 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": false, 2024-04-26T02:05:27.215 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "influxdb python module not found", 2024-04-26T02:05:27.217 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:27.217 INFO:teuthology.orchestra.run.smithi151.stdout: "batch_size": { 2024-04-26T02:05:27.217 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "batch_size", 2024-04-26T02:05:27.217 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:27.217 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.217 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.217 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "5000", 2024-04-26T02:05:27.217 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.217 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.217 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.218 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-26T02:05:27.218 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.218 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.218 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.218 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.218 INFO:teuthology.orchestra.run.smithi151.stdout: "database": { 2024-04-26T02:05:27.218 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "database", 2024-04-26T02:05:27.218 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.218 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.218 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.218 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "ceph", 2024-04-26T02:05:27.218 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.218 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.218 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.219 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-26T02:05:27.219 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.219 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.219 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.219 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.219 INFO:teuthology.orchestra.run.smithi151.stdout: "hostname": { 2024-04-26T02:05:27.219 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "hostname", 2024-04-26T02:05:27.219 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.219 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.219 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.219 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.219 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.219 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.219 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.220 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "InfluxDB server hostname", 2024-04-26T02:05:27.220 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.220 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.220 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.220 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.220 INFO:teuthology.orchestra.run.smithi151.stdout: "interval": { 2024-04-26T02:05:27.220 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "interval", 2024-04-26T02:05:27.220 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-04-26T02:05:27.220 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.220 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.220 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "30", 2024-04-26T02:05:27.220 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "5", 2024-04-26T02:05:27.220 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.221 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.221 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-26T02:05:27.221 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.221 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.221 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.221 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.221 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:27.221 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:27.221 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.221 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.221 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.221 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.221 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.221 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.222 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.222 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.222 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.222 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.222 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.222 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.222 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.222 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.222 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.222 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.222 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.222 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.222 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.222 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:27.223 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:27.223 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.223 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.223 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.223 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.223 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.223 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.223 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.223 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.223 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.223 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.223 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.223 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.223 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:27.223 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:27.223 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.224 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.224 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.224 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:27.224 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.224 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.224 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.224 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.224 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.224 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.224 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.224 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.224 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.224 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.224 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.225 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.225 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.225 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.225 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.225 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:27.225 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:27.225 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.225 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.225 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.225 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.225 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.225 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.225 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.225 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.225 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.226 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.226 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.226 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.226 INFO:teuthology.orchestra.run.smithi151.stdout: "password": { 2024-04-26T02:05:27.226 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "password", 2024-04-26T02:05:27.226 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.226 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.226 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.226 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.226 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.226 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.226 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.226 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "password of InfluxDB server user", 2024-04-26T02:05:27.227 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.227 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.227 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.227 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.227 INFO:teuthology.orchestra.run.smithi151.stdout: "port": { 2024-04-26T02:05:27.227 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "port", 2024-04-26T02:05:27.227 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:27.227 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.227 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.227 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "8086", 2024-04-26T02:05:27.227 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.227 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.227 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.227 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "InfluxDB server port", 2024-04-26T02:05:27.228 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.228 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.228 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.228 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.228 INFO:teuthology.orchestra.run.smithi151.stdout: "ssl": { 2024-04-26T02:05:27.228 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ssl", 2024-04-26T02:05:27.228 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.228 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.228 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.228 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "false", 2024-04-26T02:05:27.228 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.228 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.228 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.228 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T02:05:27.228 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.228 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.229 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.229 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.229 INFO:teuthology.orchestra.run.smithi151.stdout: "threads": { 2024-04-26T02:05:27.229 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "threads", 2024-04-26T02:05:27.229 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:27.229 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.229 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.229 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "5", 2024-04-26T02:05:27.229 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "1", 2024-04-26T02:05:27.229 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "32", 2024-04-26T02:05:27.229 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.229 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-26T02:05:27.229 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.229 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.230 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.230 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.230 INFO:teuthology.orchestra.run.smithi151.stdout: "username": { 2024-04-26T02:05:27.230 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "username", 2024-04-26T02:05:27.230 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.230 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.230 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.230 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.230 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.230 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.230 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.230 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "username of InfluxDB server user", 2024-04-26T02:05:27.230 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.231 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.231 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.231 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.231 INFO:teuthology.orchestra.run.smithi151.stdout: "verify_ssl": { 2024-04-26T02:05:27.231 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "verify_ssl", 2024-04-26T02:05:27.231 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.231 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.231 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.231 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "true", 2024-04-26T02:05:27.231 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.231 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.231 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.231 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T02:05:27.231 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.232 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.232 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.232 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.232 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.232 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.232 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:27.232 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "insights", 2024-04-26T02:05:27.232 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:27.232 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:27.232 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:27.232 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:27.232 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:27.232 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.232 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.232 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.233 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.233 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.233 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.233 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.233 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.233 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.233 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.233 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.233 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.233 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.233 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.233 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.233 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.233 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.233 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.233 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.234 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:27.234 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:27.234 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.234 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.234 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.234 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.234 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.234 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.234 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.234 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.234 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.234 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.234 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.234 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.235 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:27.235 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:27.235 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.235 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.235 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.235 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:27.235 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.235 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.235 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.235 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.235 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.235 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.235 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.235 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.236 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.236 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.236 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.236 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.236 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.236 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.236 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.236 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:27.236 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:27.236 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.236 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.236 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.236 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.236 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.237 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.237 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.237 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.237 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.237 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.237 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.237 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.237 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.237 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.237 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:27.237 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "iostat", 2024-04-26T02:05:27.237 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:27.237 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:27.238 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:27.238 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:27.238 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:27.238 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.238 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.238 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.238 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.238 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.238 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.238 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.238 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.238 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.238 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.238 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.239 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.239 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.239 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.239 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.239 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.239 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.239 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.239 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.239 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:27.239 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:27.239 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.239 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.239 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.239 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.240 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.240 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.240 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.240 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.240 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.240 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.240 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.240 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.240 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:27.240 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:27.240 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.240 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.241 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.241 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:27.241 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.241 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.241 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.241 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.241 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.241 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.241 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.241 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.241 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.241 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.241 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.242 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.242 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.242 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.242 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.242 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:27.242 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:27.242 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.242 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.242 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.242 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.242 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.242 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.242 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.242 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.243 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.243 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.243 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.243 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.243 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.243 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.243 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:27.243 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "k8sevents", 2024-04-26T02:05:27.243 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:27.243 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:27.243 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:27.243 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_event_retention_days": { 2024-04-26T02:05:27.243 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ceph_event_retention_days", 2024-04-26T02:05:27.244 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:27.244 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.244 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.244 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "7", 2024-04-26T02:05:27.244 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.244 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.244 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.244 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-26T02:05:27.244 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.244 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.244 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.244 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.244 INFO:teuthology.orchestra.run.smithi151.stdout: "config_check_secs": { 2024-04-26T02:05:27.244 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "config_check_secs", 2024-04-26T02:05:27.245 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:27.245 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.245 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.245 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "10", 2024-04-26T02:05:27.245 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "10", 2024-04-26T02:05:27.245 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.245 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.245 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-26T02:05:27.245 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.245 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.245 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.245 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.245 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:27.245 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:27.246 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.246 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.246 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.246 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.246 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.246 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.246 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.246 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.246 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.246 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.246 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.246 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.246 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.247 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.247 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.247 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.247 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.247 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.247 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.247 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:27.247 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:27.247 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.247 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.247 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.247 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.247 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.247 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.247 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.248 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.248 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.248 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.248 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.248 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.248 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:27.248 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:27.248 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.248 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.248 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.248 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:27.248 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.248 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.248 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.249 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.249 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.249 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.249 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.249 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.249 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.249 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.249 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.249 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.249 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.249 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.249 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.249 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:27.249 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:27.249 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.249 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.250 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.250 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.250 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.250 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.250 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.250 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.250 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.250 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.250 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.250 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.250 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.250 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.250 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:27.250 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "localpool", 2024-04-26T02:05:27.251 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:27.251 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:27.251 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:27.251 INFO:teuthology.orchestra.run.smithi151.stdout: "failure_domain": { 2024-04-26T02:05:27.251 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "failure_domain", 2024-04-26T02:05:27.251 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.251 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.251 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.251 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "host", 2024-04-26T02:05:27.251 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.251 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.251 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.251 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "failure domain for any created local pool", 2024-04-26T02:05:27.251 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-26T02:05:27.251 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.251 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.252 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.252 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:27.252 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:27.252 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.252 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.252 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.252 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.252 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.252 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.252 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.252 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.252 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.252 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.252 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.252 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.252 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.253 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.253 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.253 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.253 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.253 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.253 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.253 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:27.253 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:27.253 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.253 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.253 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.253 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.253 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.254 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.254 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.254 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.254 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.254 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.254 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.254 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.254 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:27.254 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:27.254 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.254 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.254 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.254 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:27.254 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.255 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.255 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.255 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.255 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.255 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.255 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.255 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.255 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.255 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.255 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.255 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.255 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.255 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.255 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.256 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:27.256 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:27.256 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.256 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.256 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.256 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.256 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.256 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.256 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.256 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.256 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.256 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.256 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.256 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.256 INFO:teuthology.orchestra.run.smithi151.stdout: "min_size": { 2024-04-26T02:05:27.257 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "min_size", 2024-04-26T02:05:27.257 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:27.257 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.257 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.257 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.257 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.257 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.257 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.257 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "default min_size for any created local pool", 2024-04-26T02:05:27.257 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-26T02:05:27.257 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.257 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.257 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.257 INFO:teuthology.orchestra.run.smithi151.stdout: "num_rep": { 2024-04-26T02:05:27.257 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "num_rep", 2024-04-26T02:05:27.257 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:27.258 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.258 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.258 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "3", 2024-04-26T02:05:27.258 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.258 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.258 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.258 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "default replica count for any created local pool", 2024-04-26T02:05:27.258 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.258 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.258 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.258 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.258 INFO:teuthology.orchestra.run.smithi151.stdout: "pg_num": { 2024-04-26T02:05:27.258 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "pg_num", 2024-04-26T02:05:27.258 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:27.259 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.259 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.259 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "128", 2024-04-26T02:05:27.259 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.259 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.259 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.259 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "default pg_num for any created local pool", 2024-04-26T02:05:27.259 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.259 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.259 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.259 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.259 INFO:teuthology.orchestra.run.smithi151.stdout: "prefix": { 2024-04-26T02:05:27.259 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "prefix", 2024-04-26T02:05:27.260 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.260 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.260 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.260 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.260 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.260 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.260 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.260 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "name prefix for any created local pool", 2024-04-26T02:05:27.260 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.260 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.260 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.260 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.260 INFO:teuthology.orchestra.run.smithi151.stdout: "subtree": { 2024-04-26T02:05:27.260 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "subtree", 2024-04-26T02:05:27.261 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.261 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.261 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.261 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "rack", 2024-04-26T02:05:27.261 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.261 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.261 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.261 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-26T02:05:27.261 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-26T02:05:27.261 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.261 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.261 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.261 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.261 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.262 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:27.262 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "mds_autoscaler", 2024-04-26T02:05:27.262 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:27.262 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:27.262 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:27.262 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:27.262 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:27.262 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.262 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.262 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.262 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.262 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.262 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.262 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.263 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.263 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.263 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.263 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.263 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.263 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.263 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.263 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.263 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.263 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.263 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.263 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.263 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:27.263 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:27.264 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.264 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.264 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.264 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.264 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.264 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.264 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.264 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.264 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.264 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.264 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.264 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.264 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:27.264 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:27.264 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.265 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.265 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.265 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:27.265 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.265 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.265 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.265 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.265 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.265 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.265 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.265 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.265 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.265 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.265 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.266 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.266 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.266 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.266 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.266 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:27.266 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:27.266 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.266 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.266 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.266 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.266 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.266 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.266 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.266 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.266 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.267 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.267 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.267 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.267 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.267 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.267 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:27.267 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "mirroring", 2024-04-26T02:05:27.267 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:27.267 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:27.267 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:27.267 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:27.267 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:27.267 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.267 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.268 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.268 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.268 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.268 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.268 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.268 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.268 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.268 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.268 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.268 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.268 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.268 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.268 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.269 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.269 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.269 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.269 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.269 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:27.269 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:27.269 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.269 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.269 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.269 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.269 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.269 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.269 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.269 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.269 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.270 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.270 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.270 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.270 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:27.270 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:27.270 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.270 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.270 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.270 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:27.270 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.270 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.270 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.270 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.271 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.271 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.271 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.271 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.271 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.271 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.271 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.271 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.271 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.271 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.271 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.271 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:27.271 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:27.272 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.272 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.272 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.272 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.272 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.272 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.272 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.272 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.272 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.272 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.272 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.272 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.272 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.272 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.272 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:27.273 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "nfs", 2024-04-26T02:05:27.273 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:27.273 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:27.273 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:27.273 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:27.273 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:27.273 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.273 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.273 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.273 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.273 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.273 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.273 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.273 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.273 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.274 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.274 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.274 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.274 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.274 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.274 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.274 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.274 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.274 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.274 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.274 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:27.274 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:27.274 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.274 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.275 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.275 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.275 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.275 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.275 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.275 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.275 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.275 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.275 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.275 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.275 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:27.275 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:27.275 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.275 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.275 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.276 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:27.276 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.276 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.276 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.276 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.276 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.276 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.276 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.276 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.276 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.276 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.276 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.276 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.276 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.277 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.277 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.277 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:27.277 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:27.277 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.277 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.277 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.277 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.277 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.277 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.277 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.277 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.277 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.277 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.277 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.278 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.278 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.278 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.278 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:27.278 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "orchestrator", 2024-04-26T02:05:27.278 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:27.278 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:27.278 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:27.278 INFO:teuthology.orchestra.run.smithi151.stdout: "fail_fs": { 2024-04-26T02:05:27.278 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "fail_fs", 2024-04-26T02:05:27.278 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.278 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.278 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.278 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.279 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.279 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.279 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.279 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-26T02:05:27.279 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.279 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.279 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.279 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.279 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:27.279 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:27.279 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.279 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.279 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.279 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.279 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.279 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.280 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.280 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.280 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.280 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.280 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.280 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.280 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.280 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.280 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.280 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.280 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.280 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.280 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.280 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:27.281 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:27.281 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.281 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.281 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.281 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.281 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.281 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.281 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.281 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.281 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.281 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.281 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.281 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.281 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:27.281 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:27.282 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.282 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.282 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.282 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:27.282 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.282 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.282 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.282 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.282 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.282 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.282 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.282 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.282 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.282 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.283 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.283 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.283 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.283 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.283 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.283 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:27.283 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:27.283 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.283 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.283 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.283 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.283 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.283 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.283 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.283 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.284 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.284 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.284 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.284 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.284 INFO:teuthology.orchestra.run.smithi151.stdout: "orchestrator": { 2024-04-26T02:05:27.284 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "orchestrator", 2024-04-26T02:05:27.284 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.284 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.284 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.284 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.284 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.284 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.284 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.284 INFO:teuthology.orchestra.run.smithi151.stdout: "cephadm", 2024-04-26T02:05:27.285 INFO:teuthology.orchestra.run.smithi151.stdout: "rook", 2024-04-26T02:05:27.285 INFO:teuthology.orchestra.run.smithi151.stdout: "test_orchestrator" 2024-04-26T02:05:27.285 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.285 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Orchestrator backend", 2024-04-26T02:05:27.285 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.285 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.285 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.285 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.285 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.285 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.285 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:27.285 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "osd_perf_query", 2024-04-26T02:05:27.285 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:27.285 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:27.286 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:27.286 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:27.286 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:27.286 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.286 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.286 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.286 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.286 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.286 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.286 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.286 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.286 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.286 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.286 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.287 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.287 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.287 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.287 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.287 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.287 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.287 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.287 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.287 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:27.287 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:27.287 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.287 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.287 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.287 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.288 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.288 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.288 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.288 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.288 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.288 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.288 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.288 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.288 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:27.288 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:27.288 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.288 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.288 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.288 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:27.288 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.289 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.289 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.289 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.289 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.289 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.289 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.289 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.289 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.289 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.289 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.289 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.289 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.289 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.289 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.289 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:27.290 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:27.290 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.290 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.290 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.290 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.290 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.290 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.290 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.290 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.290 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.290 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.290 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.290 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.290 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.291 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.291 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:27.291 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "osd_support", 2024-04-26T02:05:27.291 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:27.291 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:27.291 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:27.291 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:27.291 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:27.291 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.291 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.291 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.291 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.291 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.291 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.292 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.292 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.292 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.292 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.292 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.292 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.292 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.292 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.292 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.292 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.292 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.292 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.292 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.292 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:27.292 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:27.293 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.293 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.293 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.293 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.293 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.293 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.293 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.293 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.293 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.293 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.293 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.293 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.293 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:27.293 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:27.293 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.294 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.294 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.294 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:27.294 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.294 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.294 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.294 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.294 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.294 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.294 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.294 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.294 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.294 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.294 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.294 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.295 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.295 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.295 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.295 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:27.295 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:27.295 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.295 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.295 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.295 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.295 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.295 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.295 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.295 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.295 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.296 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.296 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.296 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.296 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.296 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.296 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:27.296 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "pg_autoscaler", 2024-04-26T02:05:27.296 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:27.296 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:27.296 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:27.296 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:27.296 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:27.296 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.296 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.296 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.297 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.297 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.297 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.297 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.297 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.297 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.297 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.297 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.297 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.297 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.297 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.297 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.297 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.298 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.298 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.298 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.298 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:27.298 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:27.298 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.298 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.298 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.298 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.298 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.298 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.298 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.298 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.298 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.299 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.299 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.299 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.299 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:27.299 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:27.299 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.299 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.299 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.299 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:27.299 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.299 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.299 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.299 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.299 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.299 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.299 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.300 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.300 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.300 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.300 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.300 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.300 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.300 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.300 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.300 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:27.300 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:27.300 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.300 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.300 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.300 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.300 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.301 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.301 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.301 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.301 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.301 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.301 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.301 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.301 INFO:teuthology.orchestra.run.smithi151.stdout: "sleep_interval": { 2024-04-26T02:05:27.301 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sleep_interval", 2024-04-26T02:05:27.301 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-04-26T02:05:27.301 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.301 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.301 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "60", 2024-04-26T02:05:27.301 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.301 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.302 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.302 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.302 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.302 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.302 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.302 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.302 INFO:teuthology.orchestra.run.smithi151.stdout: "threshold": { 2024-04-26T02:05:27.302 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "threshold", 2024-04-26T02:05:27.302 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "float", 2024-04-26T02:05:27.302 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.302 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.302 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "3.0", 2024-04-26T02:05:27.302 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "1.0", 2024-04-26T02:05:27.302 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.302 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.303 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "scaling threshold", 2024-04-26T02:05:27.303 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-26T02:05:27.303 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.303 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.303 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.303 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.303 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.303 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:27.303 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "progress", 2024-04-26T02:05:27.303 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:27.303 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:27.303 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:27.303 INFO:teuthology.orchestra.run.smithi151.stdout: "allow_pg_recovery_event": { 2024-04-26T02:05:27.303 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "allow_pg_recovery_event", 2024-04-26T02:05:27.304 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.304 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.304 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.304 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.304 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.304 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.304 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.304 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-26T02:05:27.304 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.304 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.304 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.304 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.304 INFO:teuthology.orchestra.run.smithi151.stdout: "enabled": { 2024-04-26T02:05:27.304 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "enabled", 2024-04-26T02:05:27.304 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.305 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.305 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.305 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:27.305 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.305 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.305 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.305 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.305 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.305 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.305 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.305 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.305 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:27.305 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:27.305 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.305 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.306 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.306 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.306 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.306 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.306 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.306 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.306 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.306 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.306 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.306 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.306 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.306 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.306 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.306 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.307 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.307 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.307 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.307 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:27.307 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:27.307 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.307 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.307 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.307 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.307 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.307 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.307 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.307 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.307 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.308 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.308 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.308 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.308 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:27.308 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:27.308 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.308 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.308 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.308 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:27.308 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.308 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.308 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.308 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.309 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.309 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.309 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.309 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.309 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.309 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.309 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.309 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.309 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.309 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.309 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.309 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:27.309 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:27.309 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.309 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.310 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.310 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.310 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.310 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.310 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.310 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.310 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.310 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.310 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.310 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.310 INFO:teuthology.orchestra.run.smithi151.stdout: "max_completed_events": { 2024-04-26T02:05:27.310 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "max_completed_events", 2024-04-26T02:05:27.310 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:27.311 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.311 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.311 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "50", 2024-04-26T02:05:27.311 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.311 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.311 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.311 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "number of past completed events to remember", 2024-04-26T02:05:27.311 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.311 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.311 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.311 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.311 INFO:teuthology.orchestra.run.smithi151.stdout: "sleep_interval": { 2024-04-26T02:05:27.311 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sleep_interval", 2024-04-26T02:05:27.311 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-04-26T02:05:27.312 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.312 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.312 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "5", 2024-04-26T02:05:27.312 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.312 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.312 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.312 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "how long the module is going to sleep", 2024-04-26T02:05:27.312 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.312 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.312 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.312 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.312 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.312 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.312 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:27.313 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "prometheus", 2024-04-26T02:05:27.313 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:27.313 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:27.313 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:27.313 INFO:teuthology.orchestra.run.smithi151.stdout: "cache": { 2024-04-26T02:05:27.313 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "cache", 2024-04-26T02:05:27.313 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.313 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.313 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.313 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:27.313 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.313 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.313 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.313 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.313 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.314 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.314 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.314 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.314 INFO:teuthology.orchestra.run.smithi151.stdout: "exclude_perf_counters": { 2024-04-26T02:05:27.314 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "exclude_perf_counters", 2024-04-26T02:05:27.314 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.314 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.314 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.314 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:27.314 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.314 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.314 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.314 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-26T02:05:27.314 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-26T02:05:27.315 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.315 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.315 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.315 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:27.315 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:27.315 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.315 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.315 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.315 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.315 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.315 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.315 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.315 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.315 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.316 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.316 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.316 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.316 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.316 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.316 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.316 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.316 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.316 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.316 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.316 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:27.316 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:27.316 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.316 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.317 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.317 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.317 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.317 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.317 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.317 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.317 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.317 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.317 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.317 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.317 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:27.317 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:27.317 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.318 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.318 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.318 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:27.318 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.318 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.318 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.318 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.318 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.318 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.318 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.318 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.318 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.318 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.319 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.319 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.319 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.319 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.319 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.319 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:27.319 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:27.319 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.319 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.319 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.319 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.319 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.319 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.320 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.320 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.320 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.320 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.320 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.320 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.320 INFO:teuthology.orchestra.run.smithi151.stdout: "rbd_stats_pools": { 2024-04-26T02:05:27.320 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rbd_stats_pools", 2024-04-26T02:05:27.320 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.320 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.320 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.320 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.320 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.320 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.321 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.321 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.321 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.321 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.321 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.321 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.321 INFO:teuthology.orchestra.run.smithi151.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-26T02:05:27.321 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-26T02:05:27.321 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:27.321 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.321 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.321 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "300", 2024-04-26T02:05:27.321 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.321 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.321 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.322 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.322 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.322 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.322 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.322 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.322 INFO:teuthology.orchestra.run.smithi151.stdout: "scrape_interval": { 2024-04-26T02:05:27.322 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "scrape_interval", 2024-04-26T02:05:27.322 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "float", 2024-04-26T02:05:27.322 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.322 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.322 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "15.0", 2024-04-26T02:05:27.322 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.322 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.323 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.323 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.323 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.323 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.323 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.323 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.323 INFO:teuthology.orchestra.run.smithi151.stdout: "server_addr": { 2024-04-26T02:05:27.323 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "server_addr", 2024-04-26T02:05:27.323 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.323 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.323 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.323 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "::", 2024-04-26T02:05:27.323 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.323 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.324 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.324 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-26T02:05:27.324 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.324 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.324 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.324 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.324 INFO:teuthology.orchestra.run.smithi151.stdout: "server_port": { 2024-04-26T02:05:27.324 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "server_port", 2024-04-26T02:05:27.324 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:27.324 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.324 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.324 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "9283", 2024-04-26T02:05:27.324 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.324 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.324 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.325 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-26T02:05:27.325 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.325 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.325 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.325 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.325 INFO:teuthology.orchestra.run.smithi151.stdout: "stale_cache_strategy": { 2024-04-26T02:05:27.325 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "stale_cache_strategy", 2024-04-26T02:05:27.325 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.325 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.325 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.325 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "log", 2024-04-26T02:05:27.325 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.325 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.326 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.326 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.326 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.326 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.326 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.326 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.326 INFO:teuthology.orchestra.run.smithi151.stdout: "standby_behaviour": { 2024-04-26T02:05:27.326 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "standby_behaviour", 2024-04-26T02:05:27.326 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.326 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.326 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.326 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "default", 2024-04-26T02:05:27.326 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.326 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.327 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.327 INFO:teuthology.orchestra.run.smithi151.stdout: "default", 2024-04-26T02:05:27.327 INFO:teuthology.orchestra.run.smithi151.stdout: "error" 2024-04-26T02:05:27.327 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.327 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.327 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.327 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.327 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.327 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.327 INFO:teuthology.orchestra.run.smithi151.stdout: "standby_error_status_code": { 2024-04-26T02:05:27.327 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "standby_error_status_code", 2024-04-26T02:05:27.327 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:27.327 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.327 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.328 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "500", 2024-04-26T02:05:27.328 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "400", 2024-04-26T02:05:27.328 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "599", 2024-04-26T02:05:27.328 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.328 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.328 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.328 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.328 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.328 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.328 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.328 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.328 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:27.328 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rbd_support", 2024-04-26T02:05:27.328 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:27.328 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:27.328 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:27.329 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:27.329 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:27.329 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.329 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.329 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.329 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.329 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.329 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.329 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.329 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.329 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.329 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.329 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.330 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.330 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.330 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.330 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.330 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.330 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.330 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.330 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.330 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:27.330 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:27.330 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.330 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.330 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.331 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.331 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.331 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.331 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.331 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.331 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.331 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.331 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.331 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.331 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:27.331 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:27.331 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.331 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.331 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.332 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:27.332 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.332 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.332 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.332 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.332 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.332 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.332 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.332 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.332 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.332 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.332 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.332 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.333 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.333 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.333 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.333 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:27.333 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:27.333 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.333 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.333 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.333 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.333 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.333 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.333 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.334 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.334 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.334 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.334 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.334 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.334 INFO:teuthology.orchestra.run.smithi151.stdout: "max_concurrent_snap_create": { 2024-04-26T02:05:27.334 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "max_concurrent_snap_create", 2024-04-26T02:05:27.334 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:27.334 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.334 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.334 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "10", 2024-04-26T02:05:27.334 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.334 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.334 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.335 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.335 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.335 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.335 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.335 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.335 INFO:teuthology.orchestra.run.smithi151.stdout: "mirror_snapshot_schedule": { 2024-04-26T02:05:27.335 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "mirror_snapshot_schedule", 2024-04-26T02:05:27.335 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.335 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.335 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.335 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.335 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.335 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.335 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.336 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.336 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.336 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.336 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.336 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.336 INFO:teuthology.orchestra.run.smithi151.stdout: "trash_purge_schedule": { 2024-04-26T02:05:27.336 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "trash_purge_schedule", 2024-04-26T02:05:27.336 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.336 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.336 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.336 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.336 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.336 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.337 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.337 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.337 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.337 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.337 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.337 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.337 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.337 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.337 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:27.337 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "restful", 2024-04-26T02:05:27.337 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:27.337 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:27.337 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:27.337 INFO:teuthology.orchestra.run.smithi151.stdout: "enable_auth": { 2024-04-26T02:05:27.338 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "enable_auth", 2024-04-26T02:05:27.338 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.338 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.338 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.338 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:27.338 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.338 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.338 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.338 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.338 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.338 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.338 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.338 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.339 INFO:teuthology.orchestra.run.smithi151.stdout: "key_file": { 2024-04-26T02:05:27.339 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "key_file", 2024-04-26T02:05:27.339 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.339 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.339 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.339 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.339 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.339 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.339 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.339 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.339 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.339 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.339 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.339 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.339 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:27.340 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:27.340 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.340 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.340 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.340 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.340 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.340 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.340 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.340 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.340 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.340 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.340 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.340 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.340 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.341 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.341 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.341 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.341 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.341 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.341 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.341 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:27.341 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:27.341 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.341 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.341 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.341 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.341 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.341 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.342 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.342 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.342 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.342 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.342 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.342 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.342 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:27.342 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:27.342 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.342 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.342 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.342 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:27.342 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.343 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.343 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.343 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.343 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.343 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.343 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.343 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.343 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.343 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.343 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.343 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.343 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.343 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.343 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.344 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:27.344 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:27.344 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.344 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.344 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.344 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.344 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.344 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.344 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.344 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.344 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.344 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.344 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.344 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.344 INFO:teuthology.orchestra.run.smithi151.stdout: "server_addr": { 2024-04-26T02:05:27.345 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "server_addr", 2024-04-26T02:05:27.345 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.345 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.345 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.345 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.345 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.345 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.345 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.345 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.345 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.345 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.345 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.345 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.345 INFO:teuthology.orchestra.run.smithi151.stdout: "server_port": { 2024-04-26T02:05:27.346 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "server_port", 2024-04-26T02:05:27.346 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.346 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.346 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.346 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.346 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.346 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.346 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.346 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.346 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.346 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.346 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.346 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.346 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.347 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.347 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:27.347 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rgw", 2024-04-26T02:05:27.347 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:27.347 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:27.347 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:27.347 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:27.347 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:27.347 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.347 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.347 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.347 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.347 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.347 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.348 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.348 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.348 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.348 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.348 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.348 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.348 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.348 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.348 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.348 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.348 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.348 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.348 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.348 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:27.348 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:27.348 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.349 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.349 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.349 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.349 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.349 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.349 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.349 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.349 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.349 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.349 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.349 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.349 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:27.349 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:27.349 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.350 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.350 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.350 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:27.350 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.350 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.350 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.350 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.350 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.350 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.350 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.350 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.350 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.351 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.351 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.351 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.351 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.351 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.351 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.351 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:27.351 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:27.351 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.351 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.351 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.351 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.351 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.351 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.351 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.351 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.352 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.352 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.352 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.352 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.352 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.352 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.352 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:27.352 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rook", 2024-04-26T02:05:27.352 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:27.352 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:27.352 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:27.352 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:27.352 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:27.352 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.353 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.353 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.353 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.353 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.353 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.353 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.353 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.353 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.353 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.353 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.353 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.353 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.353 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.353 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.354 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.354 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.354 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.354 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.354 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:27.354 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:27.354 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.354 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.354 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.354 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.354 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.354 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.354 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.354 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.355 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.355 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.355 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.355 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.355 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:27.355 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:27.355 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.355 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.355 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.355 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:27.355 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.355 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.355 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.356 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.356 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.356 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.356 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.356 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.356 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.356 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.356 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.356 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.356 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.356 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.356 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.356 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:27.356 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:27.357 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.357 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.357 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.357 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.357 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.357 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.357 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.357 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.357 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.357 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.357 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.357 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.357 INFO:teuthology.orchestra.run.smithi151.stdout: "storage_class": { 2024-04-26T02:05:27.357 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "storage_class", 2024-04-26T02:05:27.358 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.358 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.358 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.358 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "local", 2024-04-26T02:05:27.358 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.358 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.358 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.358 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-26T02:05:27.358 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.358 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.358 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.358 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.359 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.359 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.359 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:27.359 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "selftest", 2024-04-26T02:05:27.359 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:27.359 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:27.359 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:27.359 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:27.359 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:27.359 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.359 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.359 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.359 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.359 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.360 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.360 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.360 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.360 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.360 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.360 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.360 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.360 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.360 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.360 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.360 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.360 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.360 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.360 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.361 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:27.361 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:27.361 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.361 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.361 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.361 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.361 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.361 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.361 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.361 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.361 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.361 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.361 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.361 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.361 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:27.362 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:27.362 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.362 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.362 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.362 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:27.362 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.362 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.362 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.362 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.362 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.362 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.362 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.362 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.362 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.363 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.363 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.363 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.363 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.363 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.363 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.363 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:27.363 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:27.363 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.363 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.363 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.363 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.363 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.363 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.363 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.363 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.364 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.364 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.364 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.364 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.364 INFO:teuthology.orchestra.run.smithi151.stdout: "roption1": { 2024-04-26T02:05:27.364 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "roption1", 2024-04-26T02:05:27.364 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.364 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.364 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.364 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.364 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.364 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.364 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.364 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.364 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.364 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.365 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.365 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.365 INFO:teuthology.orchestra.run.smithi151.stdout: "roption2": { 2024-04-26T02:05:27.365 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "roption2", 2024-04-26T02:05:27.365 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.365 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.365 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.365 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "xyz", 2024-04-26T02:05:27.365 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.365 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.365 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.365 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.365 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.365 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.365 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.365 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.366 INFO:teuthology.orchestra.run.smithi151.stdout: "rwoption1": { 2024-04-26T02:05:27.366 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rwoption1", 2024-04-26T02:05:27.366 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.366 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.366 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.366 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.366 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.366 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.366 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.366 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.366 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.366 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.366 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.366 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.366 INFO:teuthology.orchestra.run.smithi151.stdout: "rwoption2": { 2024-04-26T02:05:27.367 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rwoption2", 2024-04-26T02:05:27.367 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:27.367 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.367 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.367 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.367 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.367 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.367 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.367 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.367 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.367 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.367 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.367 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.367 INFO:teuthology.orchestra.run.smithi151.stdout: "rwoption3": { 2024-04-26T02:05:27.368 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rwoption3", 2024-04-26T02:05:27.368 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "float", 2024-04-26T02:05:27.368 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.368 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.368 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.368 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.368 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.368 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.368 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.368 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.368 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.368 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.368 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.368 INFO:teuthology.orchestra.run.smithi151.stdout: "rwoption4": { 2024-04-26T02:05:27.369 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rwoption4", 2024-04-26T02:05:27.369 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.369 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.369 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.369 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.369 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.369 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.369 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.369 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.369 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.369 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.369 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.369 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.369 INFO:teuthology.orchestra.run.smithi151.stdout: "rwoption5": { 2024-04-26T02:05:27.370 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rwoption5", 2024-04-26T02:05:27.370 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.370 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.370 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.370 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.370 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.370 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.370 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.370 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.370 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.370 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.370 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.371 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.371 INFO:teuthology.orchestra.run.smithi151.stdout: "rwoption6": { 2024-04-26T02:05:27.371 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rwoption6", 2024-04-26T02:05:27.371 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.371 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.371 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.371 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:27.371 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.371 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.371 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.371 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.371 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.371 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.371 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.372 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.372 INFO:teuthology.orchestra.run.smithi151.stdout: "rwoption7": { 2024-04-26T02:05:27.372 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rwoption7", 2024-04-26T02:05:27.372 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:27.372 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.372 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.372 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.372 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "1", 2024-04-26T02:05:27.372 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "42", 2024-04-26T02:05:27.372 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.372 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.372 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.372 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.372 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.372 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.373 INFO:teuthology.orchestra.run.smithi151.stdout: "testkey": { 2024-04-26T02:05:27.373 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "testkey", 2024-04-26T02:05:27.373 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.373 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.373 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.373 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.373 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.373 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.373 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.373 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.373 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.373 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.373 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.373 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.373 INFO:teuthology.orchestra.run.smithi151.stdout: "testlkey": { 2024-04-26T02:05:27.373 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "testlkey", 2024-04-26T02:05:27.374 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.374 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.374 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.374 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.374 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.374 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.374 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.374 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.374 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.374 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.374 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.374 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.374 INFO:teuthology.orchestra.run.smithi151.stdout: "testnewline": { 2024-04-26T02:05:27.374 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "testnewline", 2024-04-26T02:05:27.375 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.375 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.375 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.375 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.375 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.375 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.375 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.375 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.375 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.375 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.375 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.375 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.375 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.375 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.375 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:27.375 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "snap_schedule", 2024-04-26T02:05:27.376 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:27.376 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:27.376 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:27.376 INFO:teuthology.orchestra.run.smithi151.stdout: "allow_m_granularity": { 2024-04-26T02:05:27.376 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "allow_m_granularity", 2024-04-26T02:05:27.376 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.376 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.376 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.376 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.376 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.376 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.376 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.376 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "allow minute scheduled snapshots", 2024-04-26T02:05:27.376 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.376 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.376 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.377 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.377 INFO:teuthology.orchestra.run.smithi151.stdout: "dump_on_update": { 2024-04-26T02:05:27.377 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "dump_on_update", 2024-04-26T02:05:27.377 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.377 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.377 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.377 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.377 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.377 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.377 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.377 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "dump database to debug log on update", 2024-04-26T02:05:27.377 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.377 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.377 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.378 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.378 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:27.378 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:27.378 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.378 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.378 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.378 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.378 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.378 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.378 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.378 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.378 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.378 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.378 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.379 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.379 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.379 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.379 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.379 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.379 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.379 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.379 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.379 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:27.379 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:27.379 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.379 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.379 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.379 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.380 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.380 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.380 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.380 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.380 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.380 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.380 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.380 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.380 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:27.380 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:27.380 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.380 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.380 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.380 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:27.380 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.381 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.381 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.381 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.381 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.381 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.381 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.381 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.381 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.381 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.381 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.381 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.381 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.381 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.381 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.381 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:27.382 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:27.382 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.382 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.382 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.382 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.382 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.382 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.382 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.382 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.382 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.382 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.382 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.382 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.382 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.383 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.383 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:27.383 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "stats", 2024-04-26T02:05:27.383 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:27.383 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:27.383 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:27.383 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:27.383 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:27.383 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.383 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.383 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.383 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.383 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.383 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.383 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.384 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.384 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.384 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.384 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.384 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.384 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.384 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.384 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.384 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.384 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.384 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.384 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.384 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:27.384 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:27.384 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.385 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.385 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.385 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.385 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.385 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.385 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.385 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.385 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.385 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.385 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.385 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.385 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:27.385 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:27.385 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.386 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.386 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.386 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:27.386 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.386 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.386 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.386 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.386 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.386 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.386 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.386 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.386 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.386 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.387 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.387 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.387 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.387 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.387 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.387 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:27.387 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:27.387 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.387 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.387 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.387 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.387 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.387 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.387 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.388 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.388 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.388 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.388 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.388 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.388 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.388 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.388 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:27.388 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "status", 2024-04-26T02:05:27.388 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:27.388 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:27.388 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:27.388 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:27.388 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:27.389 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.389 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.389 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.389 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.389 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.389 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.389 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.389 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.389 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.389 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.389 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.389 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.389 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.389 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.389 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.389 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.390 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.390 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.390 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.390 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:27.390 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:27.390 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.390 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.390 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.390 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.390 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.390 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.390 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.390 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.390 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.391 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.391 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.391 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.391 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:27.391 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:27.391 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.391 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.391 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.391 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:27.391 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.391 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.391 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.391 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.391 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.392 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.392 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.392 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.392 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.392 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.392 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.392 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.392 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.392 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.392 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.392 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:27.392 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:27.392 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.392 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.392 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.393 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.393 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.393 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.393 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.393 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.393 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.393 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.393 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.393 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.393 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.393 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.393 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:27.393 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "telegraf", 2024-04-26T02:05:27.393 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:27.393 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:27.394 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:27.394 INFO:teuthology.orchestra.run.smithi151.stdout: "address": { 2024-04-26T02:05:27.394 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "address", 2024-04-26T02:05:27.394 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.394 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.394 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.394 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-26T02:05:27.394 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.394 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.394 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.394 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.394 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.394 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.394 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.395 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.395 INFO:teuthology.orchestra.run.smithi151.stdout: "interval": { 2024-04-26T02:05:27.395 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "interval", 2024-04-26T02:05:27.395 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-04-26T02:05:27.395 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.395 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.395 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "15", 2024-04-26T02:05:27.395 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.395 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.395 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.395 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.395 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.395 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.395 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.395 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.396 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:27.396 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:27.396 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.396 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.396 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.396 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.396 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.396 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.396 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.396 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.396 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.396 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.396 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.396 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.396 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.397 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.397 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.397 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.397 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.397 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.397 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.397 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:27.397 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:27.397 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.397 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.397 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.397 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.397 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.397 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.397 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.397 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.398 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.398 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.398 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.398 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.398 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:27.398 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:27.398 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.398 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.398 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.398 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:27.398 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.398 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.398 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.398 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.398 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.399 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.399 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.399 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.399 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.399 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.399 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.399 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.399 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.399 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.399 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.399 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:27.399 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:27.399 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.399 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.399 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.400 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.400 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.400 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.400 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.400 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.400 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.400 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.400 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.400 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.400 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.400 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.400 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:27.400 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "telemetry", 2024-04-26T02:05:27.400 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:27.401 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:27.401 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:27.401 INFO:teuthology.orchestra.run.smithi151.stdout: "channel_basic": { 2024-04-26T02:05:27.401 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "channel_basic", 2024-04-26T02:05:27.401 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.401 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.401 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.401 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:27.401 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.401 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.401 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.401 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-26T02:05:27.401 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.401 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.401 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.401 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.402 INFO:teuthology.orchestra.run.smithi151.stdout: "channel_crash": { 2024-04-26T02:05:27.402 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "channel_crash", 2024-04-26T02:05:27.402 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.402 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.402 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.402 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:27.402 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.402 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.402 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.402 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-26T02:05:27.402 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.402 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.402 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.402 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.402 INFO:teuthology.orchestra.run.smithi151.stdout: "channel_device": { 2024-04-26T02:05:27.403 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "channel_device", 2024-04-26T02:05:27.403 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.403 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.403 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.403 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:27.403 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.403 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.403 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.403 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-26T02:05:27.403 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.403 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.403 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.403 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.403 INFO:teuthology.orchestra.run.smithi151.stdout: "channel_ident": { 2024-04-26T02:05:27.403 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "channel_ident", 2024-04-26T02:05:27.404 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.404 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.404 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.404 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.404 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.404 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.404 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.404 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-26T02:05:27.404 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.404 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.404 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.404 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.404 INFO:teuthology.orchestra.run.smithi151.stdout: "channel_perf": { 2024-04-26T02:05:27.404 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "channel_perf", 2024-04-26T02:05:27.405 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.405 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.405 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.405 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.405 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.405 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.405 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.405 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-26T02:05:27.405 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.405 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.405 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.405 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.405 INFO:teuthology.orchestra.run.smithi151.stdout: "contact": { 2024-04-26T02:05:27.405 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "contact", 2024-04-26T02:05:27.405 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.406 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.406 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.406 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.406 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.406 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.406 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.406 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.406 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.406 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.406 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.406 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.406 INFO:teuthology.orchestra.run.smithi151.stdout: "description": { 2024-04-26T02:05:27.406 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "description", 2024-04-26T02:05:27.406 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.407 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.407 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.407 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.407 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.407 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.407 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.407 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.407 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.407 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.407 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.407 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.407 INFO:teuthology.orchestra.run.smithi151.stdout: "device_url": { 2024-04-26T02:05:27.407 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "device_url", 2024-04-26T02:05:27.407 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.408 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.408 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.408 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-26T02:05:27.408 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.408 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.408 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.408 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.408 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.408 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.408 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.408 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.408 INFO:teuthology.orchestra.run.smithi151.stdout: "enabled": { 2024-04-26T02:05:27.408 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "enabled", 2024-04-26T02:05:27.408 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.408 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.409 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.409 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.409 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.409 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.409 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.409 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.409 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.409 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.409 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.409 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.409 INFO:teuthology.orchestra.run.smithi151.stdout: "interval": { 2024-04-26T02:05:27.409 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "interval", 2024-04-26T02:05:27.409 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:27.409 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.409 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.409 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "24", 2024-04-26T02:05:27.410 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "8", 2024-04-26T02:05:27.410 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.410 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.410 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.410 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.410 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.410 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.410 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.410 INFO:teuthology.orchestra.run.smithi151.stdout: "last_opt_revision": { 2024-04-26T02:05:27.410 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "last_opt_revision", 2024-04-26T02:05:27.410 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:27.410 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.410 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.410 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "1", 2024-04-26T02:05:27.411 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.411 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.411 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.411 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.411 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.411 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.411 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.411 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.411 INFO:teuthology.orchestra.run.smithi151.stdout: "leaderboard": { 2024-04-26T02:05:27.411 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "leaderboard", 2024-04-26T02:05:27.411 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.411 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.411 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.411 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.412 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.412 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.412 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.412 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.412 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.412 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.412 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.412 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.412 INFO:teuthology.orchestra.run.smithi151.stdout: "leaderboard_description": { 2024-04-26T02:05:27.412 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "leaderboard_description", 2024-04-26T02:05:27.412 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.412 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.412 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.412 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.412 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.412 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.413 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.413 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.413 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.413 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.413 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.413 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.413 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:27.413 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:27.413 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.413 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.413 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.413 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.413 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.413 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.413 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.413 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.414 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.414 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.414 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.414 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.414 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.414 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.414 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.414 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.414 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.414 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.414 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.414 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:27.414 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:27.414 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.414 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.415 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.415 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.415 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.415 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.415 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.415 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.415 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.415 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.415 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.415 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.415 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:27.415 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:27.415 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.415 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.415 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.415 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:27.416 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.416 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.416 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.416 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.416 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.416 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.416 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.416 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.416 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.416 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.416 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.416 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.416 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.416 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.417 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.417 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:27.417 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:27.417 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.417 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.417 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.417 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.417 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.417 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.417 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.417 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.417 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.417 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.417 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.417 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.418 INFO:teuthology.orchestra.run.smithi151.stdout: "organization": { 2024-04-26T02:05:27.418 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "organization", 2024-04-26T02:05:27.418 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.418 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.418 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.418 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.418 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.418 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.418 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.418 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.418 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.418 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.418 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.419 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.419 INFO:teuthology.orchestra.run.smithi151.stdout: "proxy": { 2024-04-26T02:05:27.419 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "proxy", 2024-04-26T02:05:27.419 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.419 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.419 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.419 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.419 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.419 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.419 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.419 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.419 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.419 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.421 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.421 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.421 INFO:teuthology.orchestra.run.smithi151.stdout: "url": { 2024-04-26T02:05:27.421 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "url", 2024-04-26T02:05:27.421 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.421 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.421 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.421 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-26T02:05:27.421 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.422 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.422 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.422 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.422 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.422 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.422 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.422 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.422 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.422 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.422 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:27.422 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "test_orchestrator", 2024-04-26T02:05:27.422 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:27.422 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:27.422 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:27.423 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:27.423 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:27.423 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.423 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.423 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.423 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.423 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.423 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.423 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.423 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.423 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.423 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.423 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.423 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.423 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.423 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.424 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.424 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.424 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.424 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.424 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.424 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:27.424 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:27.424 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.424 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.424 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.424 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.424 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.424 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.425 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.425 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.425 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.425 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.425 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.425 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.425 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:27.425 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:27.425 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.425 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.425 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.425 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:27.425 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.425 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.425 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.426 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.426 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.426 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.426 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.426 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.426 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.426 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.426 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.426 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.426 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.426 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.426 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.426 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:27.426 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:27.426 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.427 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.427 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.427 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.427 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.427 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.427 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.427 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.427 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.427 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.427 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.427 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.427 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.427 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.427 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:27.428 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "volumes", 2024-04-26T02:05:27.428 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:27.428 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:27.428 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:27.428 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:27.428 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:27.428 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.428 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.428 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.428 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.428 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.428 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.428 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.428 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.428 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.429 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.429 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.429 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.429 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.429 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.429 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.429 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.429 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.429 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.429 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.429 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:27.429 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:27.429 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.429 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.429 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.430 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.430 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.430 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.430 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.430 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.430 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.430 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.430 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.430 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.430 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:27.430 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:27.430 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.430 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.430 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.431 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:27.431 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.431 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.431 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.431 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.431 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.431 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.431 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.431 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.431 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.431 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.431 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.431 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.432 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.432 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.432 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.432 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:27.432 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:27.432 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.432 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.432 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.432 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.432 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.432 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.432 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.432 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.432 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.433 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.433 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.433 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.433 INFO:teuthology.orchestra.run.smithi151.stdout: "max_concurrent_clones": { 2024-04-26T02:05:27.433 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "max_concurrent_clones", 2024-04-26T02:05:27.433 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:27.433 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.433 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.433 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "4", 2024-04-26T02:05:27.433 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.433 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.433 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.433 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-26T02:05:27.433 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.434 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.434 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.434 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.434 INFO:teuthology.orchestra.run.smithi151.stdout: "periodic_async_work": { 2024-04-26T02:05:27.434 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "periodic_async_work", 2024-04-26T02:05:27.434 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.434 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.434 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.434 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.434 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.434 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.434 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.434 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Periodically check for async work", 2024-04-26T02:05:27.434 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.435 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.435 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.435 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.435 INFO:teuthology.orchestra.run.smithi151.stdout: "snapshot_clone_delay": { 2024-04-26T02:05:27.435 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "snapshot_clone_delay", 2024-04-26T02:05:27.435 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:27.435 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.435 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.435 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-04-26T02:05:27.435 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.435 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.435 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.435 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-26T02:05:27.436 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.436 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.436 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.436 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.436 INFO:teuthology.orchestra.run.smithi151.stdout: "snapshot_clone_no_wait": { 2024-04-26T02:05:27.436 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "snapshot_clone_no_wait", 2024-04-26T02:05:27.436 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.436 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.436 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.436 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-04-26T02:05:27.436 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.436 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.436 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.436 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-26T02:05:27.437 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.437 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.437 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.437 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.437 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.437 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.437 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:27.437 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "zabbix", 2024-04-26T02:05:27.437 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-04-26T02:05:27.437 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-04-26T02:05:27.437 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-04-26T02:05:27.437 INFO:teuthology.orchestra.run.smithi151.stdout: "discovery_interval": { 2024-04-26T02:05:27.437 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "discovery_interval", 2024-04-26T02:05:27.438 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "uint", 2024-04-26T02:05:27.438 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.438 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.438 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "100", 2024-04-26T02:05:27.438 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.438 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.438 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.438 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.438 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.438 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.438 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.438 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.438 INFO:teuthology.orchestra.run.smithi151.stdout: "identifier": { 2024-04-26T02:05:27.438 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "identifier", 2024-04-26T02:05:27.439 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.439 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.439 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.439 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.439 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.439 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.439 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.439 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.439 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.439 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.439 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.439 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.439 INFO:teuthology.orchestra.run.smithi151.stdout: "interval": { 2024-04-26T02:05:27.439 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "interval", 2024-04-26T02:05:27.440 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-04-26T02:05:27.440 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.440 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.440 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "60", 2024-04-26T02:05:27.440 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.440 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.440 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.440 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.440 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.440 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.440 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.440 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.440 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-04-26T02:05:27.441 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-04-26T02:05:27.441 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.441 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.441 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.441 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.441 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.441 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.441 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.441 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.441 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.441 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.441 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.441 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.441 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.442 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.442 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.442 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.442 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.442 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.442 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.442 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-04-26T02:05:27.442 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-04-26T02:05:27.442 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.442 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.442 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.442 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.443 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.443 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.443 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.443 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.443 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.443 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.443 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.443 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.443 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-04-26T02:05:27.443 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-04-26T02:05:27.443 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.443 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.443 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.444 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-04-26T02:05:27.444 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.444 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.444 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-04-26T02:05:27.444 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-04-26T02:05:27.444 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-04-26T02:05:27.444 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-04-26T02:05:27.444 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-04-26T02:05:27.444 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-04-26T02:05:27.444 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-04-26T02:05:27.444 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.444 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.444 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.444 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.445 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.445 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.445 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-04-26T02:05:27.445 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-04-26T02:05:27.445 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-04-26T02:05:27.445 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.445 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-04-26T02:05:27.445 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-04-26T02:05:27.445 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.445 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.445 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.445 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.445 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.445 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.446 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.446 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.446 INFO:teuthology.orchestra.run.smithi151.stdout: "zabbix_host": { 2024-04-26T02:05:27.446 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "zabbix_host", 2024-04-26T02:05:27.446 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.446 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.446 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.446 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-04-26T02:05:27.446 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.446 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.446 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.446 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.446 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.447 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.447 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.447 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.447 INFO:teuthology.orchestra.run.smithi151.stdout: "zabbix_port": { 2024-04-26T02:05:27.447 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "zabbix_port", 2024-04-26T02:05:27.447 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-04-26T02:05:27.447 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.447 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.447 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "10051", 2024-04-26T02:05:27.447 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.447 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.447 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.447 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.447 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.448 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.448 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.448 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.448 INFO:teuthology.orchestra.run.smithi151.stdout: "zabbix_sender": { 2024-04-26T02:05:27.448 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "zabbix_sender", 2024-04-26T02:05:27.448 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-04-26T02:05:27.448 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-04-26T02:05:27.448 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-04-26T02:05:27.448 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-26T02:05:27.448 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-04-26T02:05:27.448 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-04-26T02:05:27.448 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-04-26T02:05:27.448 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-04-26T02:05:27.449 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-04-26T02:05:27.449 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-04-26T02:05:27.449 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-04-26T02:05:27.449 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.449 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.449 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.449 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.449 INFO:teuthology.orchestra.run.smithi151.stdout: "services": { 2024-04-26T02:05:27.449 INFO:teuthology.orchestra.run.smithi151.stdout: "dashboard": "https://172.21.15.151:8443/" 2024-04-26T02:05:27.449 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.449 INFO:teuthology.orchestra.run.smithi151.stdout: "always_on_modules": { 2024-04-26T02:05:27.449 INFO:teuthology.orchestra.run.smithi151.stdout: "octopus": [ 2024-04-26T02:05:27.449 INFO:teuthology.orchestra.run.smithi151.stdout: "balancer", 2024-04-26T02:05:27.449 INFO:teuthology.orchestra.run.smithi151.stdout: "crash", 2024-04-26T02:05:27.450 INFO:teuthology.orchestra.run.smithi151.stdout: "devicehealth", 2024-04-26T02:05:27.450 INFO:teuthology.orchestra.run.smithi151.stdout: "orchestrator", 2024-04-26T02:05:27.450 INFO:teuthology.orchestra.run.smithi151.stdout: "pg_autoscaler", 2024-04-26T02:05:27.450 INFO:teuthology.orchestra.run.smithi151.stdout: "progress", 2024-04-26T02:05:27.450 INFO:teuthology.orchestra.run.smithi151.stdout: "rbd_support", 2024-04-26T02:05:27.450 INFO:teuthology.orchestra.run.smithi151.stdout: "status", 2024-04-26T02:05:27.450 INFO:teuthology.orchestra.run.smithi151.stdout: "telemetry", 2024-04-26T02:05:27.450 INFO:teuthology.orchestra.run.smithi151.stdout: "volumes" 2024-04-26T02:05:27.450 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.450 INFO:teuthology.orchestra.run.smithi151.stdout: "pacific": [ 2024-04-26T02:05:27.450 INFO:teuthology.orchestra.run.smithi151.stdout: "balancer", 2024-04-26T02:05:27.450 INFO:teuthology.orchestra.run.smithi151.stdout: "crash", 2024-04-26T02:05:27.451 INFO:teuthology.orchestra.run.smithi151.stdout: "devicehealth", 2024-04-26T02:05:27.451 INFO:teuthology.orchestra.run.smithi151.stdout: "orchestrator", 2024-04-26T02:05:27.451 INFO:teuthology.orchestra.run.smithi151.stdout: "pg_autoscaler", 2024-04-26T02:05:27.451 INFO:teuthology.orchestra.run.smithi151.stdout: "progress", 2024-04-26T02:05:27.451 INFO:teuthology.orchestra.run.smithi151.stdout: "rbd_support", 2024-04-26T02:05:27.451 INFO:teuthology.orchestra.run.smithi151.stdout: "status", 2024-04-26T02:05:27.451 INFO:teuthology.orchestra.run.smithi151.stdout: "telemetry", 2024-04-26T02:05:27.451 INFO:teuthology.orchestra.run.smithi151.stdout: "volumes" 2024-04-26T02:05:27.451 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.451 INFO:teuthology.orchestra.run.smithi151.stdout: "quincy": [ 2024-04-26T02:05:27.451 INFO:teuthology.orchestra.run.smithi151.stdout: "balancer", 2024-04-26T02:05:27.451 INFO:teuthology.orchestra.run.smithi151.stdout: "crash", 2024-04-26T02:05:27.451 INFO:teuthology.orchestra.run.smithi151.stdout: "devicehealth", 2024-04-26T02:05:27.451 INFO:teuthology.orchestra.run.smithi151.stdout: "orchestrator", 2024-04-26T02:05:27.452 INFO:teuthology.orchestra.run.smithi151.stdout: "pg_autoscaler", 2024-04-26T02:05:27.452 INFO:teuthology.orchestra.run.smithi151.stdout: "progress", 2024-04-26T02:05:27.452 INFO:teuthology.orchestra.run.smithi151.stdout: "rbd_support", 2024-04-26T02:05:27.452 INFO:teuthology.orchestra.run.smithi151.stdout: "status", 2024-04-26T02:05:27.452 INFO:teuthology.orchestra.run.smithi151.stdout: "telemetry", 2024-04-26T02:05:27.452 INFO:teuthology.orchestra.run.smithi151.stdout: "volumes" 2024-04-26T02:05:27.452 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.452 INFO:teuthology.orchestra.run.smithi151.stdout: "reef": [ 2024-04-26T02:05:27.452 INFO:teuthology.orchestra.run.smithi151.stdout: "balancer", 2024-04-26T02:05:27.452 INFO:teuthology.orchestra.run.smithi151.stdout: "crash", 2024-04-26T02:05:27.452 INFO:teuthology.orchestra.run.smithi151.stdout: "devicehealth", 2024-04-26T02:05:27.452 INFO:teuthology.orchestra.run.smithi151.stdout: "orchestrator", 2024-04-26T02:05:27.452 INFO:teuthology.orchestra.run.smithi151.stdout: "pg_autoscaler", 2024-04-26T02:05:27.452 INFO:teuthology.orchestra.run.smithi151.stdout: "progress", 2024-04-26T02:05:27.452 INFO:teuthology.orchestra.run.smithi151.stdout: "rbd_support", 2024-04-26T02:05:27.453 INFO:teuthology.orchestra.run.smithi151.stdout: "status", 2024-04-26T02:05:27.453 INFO:teuthology.orchestra.run.smithi151.stdout: "telemetry", 2024-04-26T02:05:27.453 INFO:teuthology.orchestra.run.smithi151.stdout: "volumes" 2024-04-26T02:05:27.453 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-04-26T02:05:27.453 INFO:teuthology.orchestra.run.smithi151.stdout: "squid": [ 2024-04-26T02:05:27.453 INFO:teuthology.orchestra.run.smithi151.stdout: "balancer", 2024-04-26T02:05:27.453 INFO:teuthology.orchestra.run.smithi151.stdout: "crash", 2024-04-26T02:05:27.453 INFO:teuthology.orchestra.run.smithi151.stdout: "devicehealth", 2024-04-26T02:05:27.453 INFO:teuthology.orchestra.run.smithi151.stdout: "orchestrator", 2024-04-26T02:05:27.453 INFO:teuthology.orchestra.run.smithi151.stdout: "pg_autoscaler", 2024-04-26T02:05:27.453 INFO:teuthology.orchestra.run.smithi151.stdout: "progress", 2024-04-26T02:05:27.453 INFO:teuthology.orchestra.run.smithi151.stdout: "rbd_support", 2024-04-26T02:05:27.453 INFO:teuthology.orchestra.run.smithi151.stdout: "status", 2024-04-26T02:05:27.453 INFO:teuthology.orchestra.run.smithi151.stdout: "telemetry", 2024-04-26T02:05:27.453 INFO:teuthology.orchestra.run.smithi151.stdout: "volumes" 2024-04-26T02:05:27.453 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-26T02:05:27.453 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.454 INFO:teuthology.orchestra.run.smithi151.stdout: "last_failure_osd_epoch": 93, 2024-04-26T02:05:27.454 INFO:teuthology.orchestra.run.smithi151.stdout: "active_clients": [ 2024-04-26T02:05:27.454 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:27.454 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "devicehealth", 2024-04-26T02:05:27.454 INFO:teuthology.orchestra.run.smithi151.stdout: "addrvec": [ 2024-04-26T02:05:27.454 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:27.454 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "v2", 2024-04-26T02:05:27.454 INFO:teuthology.orchestra.run.smithi151.stdout: "addr": "172.21.15.151:0", 2024-04-26T02:05:27.454 INFO:teuthology.orchestra.run.smithi151.stdout: "nonce": 1022516510 2024-04-26T02:05:27.454 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.454 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-26T02:05:27.454 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.454 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:27.454 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "libcephsqlite", 2024-04-26T02:05:27.455 INFO:teuthology.orchestra.run.smithi151.stdout: "addrvec": [ 2024-04-26T02:05:27.455 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:27.455 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "v2", 2024-04-26T02:05:27.455 INFO:teuthology.orchestra.run.smithi151.stdout: "addr": "172.21.15.151:0", 2024-04-26T02:05:27.455 INFO:teuthology.orchestra.run.smithi151.stdout: "nonce": 2274661216 2024-04-26T02:05:27.455 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.455 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-26T02:05:27.455 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.455 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:27.455 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rbd_support", 2024-04-26T02:05:27.455 INFO:teuthology.orchestra.run.smithi151.stdout: "addrvec": [ 2024-04-26T02:05:27.455 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:27.455 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "v2", 2024-04-26T02:05:27.455 INFO:teuthology.orchestra.run.smithi151.stdout: "addr": "172.21.15.151:0", 2024-04-26T02:05:27.456 INFO:teuthology.orchestra.run.smithi151.stdout: "nonce": 1799232044 2024-04-26T02:05:27.456 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.456 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-26T02:05:27.456 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.456 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:27.456 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "snap_schedule", 2024-04-26T02:05:27.456 INFO:teuthology.orchestra.run.smithi151.stdout: "addrvec": [ 2024-04-26T02:05:27.456 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:27.456 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "v2", 2024-04-26T02:05:27.456 INFO:teuthology.orchestra.run.smithi151.stdout: "addr": "172.21.15.151:0", 2024-04-26T02:05:27.456 INFO:teuthology.orchestra.run.smithi151.stdout: "nonce": 653346662 2024-04-26T02:05:27.456 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.456 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-26T02:05:27.456 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.457 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:27.457 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "snap_schedule", 2024-04-26T02:05:27.457 INFO:teuthology.orchestra.run.smithi151.stdout: "addrvec": [ 2024-04-26T02:05:27.457 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:27.457 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "v2", 2024-04-26T02:05:27.457 INFO:teuthology.orchestra.run.smithi151.stdout: "addr": "172.21.15.151:0", 2024-04-26T02:05:27.457 INFO:teuthology.orchestra.run.smithi151.stdout: "nonce": 2243201600 2024-04-26T02:05:27.457 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.457 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-26T02:05:27.457 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:27.457 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:27.457 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "volumes", 2024-04-26T02:05:27.457 INFO:teuthology.orchestra.run.smithi151.stdout: "addrvec": [ 2024-04-26T02:05:27.457 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-04-26T02:05:27.458 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "v2", 2024-04-26T02:05:27.458 INFO:teuthology.orchestra.run.smithi151.stdout: "addr": "172.21.15.151:0", 2024-04-26T02:05:27.458 INFO:teuthology.orchestra.run.smithi151.stdout: "nonce": 1140445229 2024-04-26T02:05:27.458 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.458 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-26T02:05:27.458 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:27.458 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-04-26T02:05:27.458 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-26T02:05:27.459 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:05:26 smithi165 ceph-mon[32648]: pgmap v371: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 113 KiB/s wr, 101 op/s 2024-04-26T02:05:27.459 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:05:26 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/2130153209' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T02:05:27.459 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:05:26 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/2777285439' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-26T02:05:27.459 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:05:26 smithi172 ceph-mon[32785]: pgmap v371: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 113 KiB/s wr, 101 op/s 2024-04-26T02:05:27.459 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:05:26 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/2130153209' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T02:05:27.460 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:05:26 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/2777285439' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-26T02:05:27.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:26.172+0000 7ffade800640 1 -- 172.21.15.151:0/2777285439 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7ffabc0a3da0 msgr2=0x7ffabc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:27.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:26.172+0000 7ffade800640 1 --2- 172.21.15.151:0/2777285439 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7ffabc0a3da0 0x7ffabc0a6260 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ffac8009fc0 tx=0x7ffac8005f90 comp rx=0 tx=0).stop 2024-04-26T02:05:27.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:26.172+0000 7ffade800640 1 -- 172.21.15.151:0/2777285439 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffad8109460 msgr2=0x7ffad81a0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:27.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:26.172+0000 7ffade800640 1 --2- 172.21.15.151:0/2777285439 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffad8109460 0x7ffad81a0bd0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ffac400e9c0 tx=0x7ffac400ee80 comp rx=0 tx=0).stop 2024-04-26T02:05:27.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:26.172+0000 7ffade800640 1 -- 172.21.15.151:0/2777285439 shutdown_connections 2024-04-26T02:05:27.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:26.172+0000 7ffade800640 1 --2- 172.21.15.151:0/2777285439 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7ffabc0a3da0 0x7ffabc0a6260 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:27.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:26.172+0000 7ffade800640 1 --2- 172.21.15.151:0/2777285439 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffad819a7a0 0x7ffad819ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:27.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:26.172+0000 7ffade800640 1 --2- 172.21.15.151:0/2777285439 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffad8109460 0x7ffad81a0bd0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:27.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:26.172+0000 7ffade800640 1 --2- 172.21.15.151:0/2777285439 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffad8104390 0x7ffad81a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:27.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:26.172+0000 7ffade800640 1 -- 172.21.15.151:0/2777285439 >> 172.21.15.151:0/2777285439 conn(0x7ffad80ffc50 msgr2=0x7ffad8101ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:05:27.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:26.173+0000 7ffade800640 1 -- 172.21.15.151:0/2777285439 shutdown_connections 2024-04-26T02:05:27.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:05:26.173+0000 7ffade800640 1 -- 172.21.15.151:0/2777285439 wait complete. 2024-04-26T02:05:27.467 DEBUG:tasks.check_counter:Getting stats from a 2024-04-26T02:05:27.467 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-26T02:05:27.467 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:05:26 smithi151 ceph-mon[28057]: pgmap v371: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 113 KiB/s wr, 101 op/s 2024-04-26T02:05:27.467 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:05:26 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/2130153209' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T02:05:27.468 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:05:26 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/2777285439' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-26T02:05:27.915 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:05:28.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:05:28 smithi172 ceph-mon[32785]: pgmap v372: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 99 KiB/s wr, 54 op/s 2024-04-26T02:05:28.847 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:05:28 smithi165 ceph-mon[32648]: pgmap v372: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 99 KiB/s wr, 54 op/s 2024-04-26T02:05:28.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:05:28 smithi151 ceph-mon[28057]: pgmap v372: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 99 KiB/s wr, 54 op/s 2024-04-26T02:05:30.344 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-26T02:05:30.345 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T02:05:30.345 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 0, 2024-04-26T02:05:30.345 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 2, 2024-04-26T02:05:30.345 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 0, 2024-04-26T02:05:30.345 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 0, 2024-04-26T02:05:30.345 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 4, 2024-04-26T02:05:30.345 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 0, 2024-04-26T02:05:30.345 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.004508036, 2024-04-26T02:05:30.345 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.000000000, 2024-04-26T02:05:30.345 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.001035196, 2024-04-26T02:05:30.345 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T02:05:30.345 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:05:30.345 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-26T02:05:30.345 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-26T02:05:30.346 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-26T02:05:30.346 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:30.346 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:05:30.346 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-26T02:05:30.346 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-26T02:05:30.346 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-26T02:05:30.346 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:30.346 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T02:05:30.346 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T02:05:30.346 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:30.346 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T02:05:30.346 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 2, 2024-04-26T02:05:30.347 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 199, 2024-04-26T02:05:30.347 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 320, 2024-04-26T02:05:30.347 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 191232, 2024-04-26T02:05:30.347 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 5, 2024-04-26T02:05:30.347 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 3, 2024-04-26T02:05:30.347 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.039794099, 2024-04-26T02:05:30.347 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.029587256, 2024-04-26T02:05:30.347 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.004291315, 2024-04-26T02:05:30.347 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T02:05:30.347 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:05:30.347 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 196, 2024-04-26T02:05:30.347 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.031038160, 2024-04-26T02:05:30.347 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000158357 2024-04-26T02:05:30.347 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:30.348 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:05:30.348 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-26T02:05:30.348 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-26T02:05:30.348 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-26T02:05:30.348 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:30.348 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-26T02:05:30.348 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 191232 2024-04-26T02:05:30.348 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:30.348 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T02:05:30.348 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 276, 2024-04-26T02:05:30.348 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 262, 2024-04-26T02:05:30.348 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 1745776, 2024-04-26T02:05:30.348 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 160848, 2024-04-26T02:05:30.348 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 2, 2024-04-26T02:05:30.349 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 1, 2024-04-26T02:05:30.349 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.115086293, 2024-04-26T02:05:30.349 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.040766640, 2024-04-26T02:05:30.349 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.086425695, 2024-04-26T02:05:30.349 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.011762194, 2024-04-26T02:05:30.349 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:05:30.349 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 262, 2024-04-26T02:05:30.349 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.025778387, 2024-04-26T02:05:30.349 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000098390 2024-04-26T02:05:30.349 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:30.349 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:05:30.349 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-26T02:05:30.349 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-26T02:05:30.349 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-26T02:05:30.349 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:30.350 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 1745776, 2024-04-26T02:05:30.350 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 160848 2024-04-26T02:05:30.350 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:30.350 INFO:teuthology.orchestra.run.smithi151.stdout: "cct": { 2024-04-26T02:05:30.350 INFO:teuthology.orchestra.run.smithi151.stdout: "total_workers": 0, 2024-04-26T02:05:30.350 INFO:teuthology.orchestra.run.smithi151.stdout: "unhealthy_workers": 0 2024-04-26T02:05:30.350 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:30.350 INFO:teuthology.orchestra.run.smithi151.stdout: "mempool": { 2024-04-26T02:05:30.350 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_bytes": 0, 2024-04-26T02:05:30.350 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_items": 0, 2024-04-26T02:05:30.350 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T02:05:30.350 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_items": 0, 2024-04-26T02:05:30.350 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T02:05:30.350 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_items": 0, 2024-04-26T02:05:30.351 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T02:05:30.351 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T02:05:30.351 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T02:05:30.351 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T02:05:30.351 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T02:05:30.351 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_items": 0, 2024-04-26T02:05:30.351 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T02:05:30.351 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T02:05:30.351 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_bytes": 0, 2024-04-26T02:05:30.351 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_items": 0, 2024-04-26T02:05:30.351 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_bytes": 0, 2024-04-26T02:05:30.351 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_items": 0, 2024-04-26T02:05:30.351 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T02:05:30.351 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T02:05:30.352 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T02:05:30.352 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T02:05:30.352 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T02:05:30.352 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_items": 0, 2024-04-26T02:05:30.352 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_bytes": 0, 2024-04-26T02:05:30.352 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_items": 0, 2024-04-26T02:05:30.352 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T02:05:30.352 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T02:05:30.352 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_bytes": 0, 2024-04-26T02:05:30.352 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_items": 0, 2024-04-26T02:05:30.352 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_bytes": 0, 2024-04-26T02:05:30.352 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_items": 0, 2024-04-26T02:05:30.352 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T02:05:30.352 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_items": 0, 2024-04-26T02:05:30.352 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T02:05:30.353 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_items": 0, 2024-04-26T02:05:30.353 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_bytes": 150776, 2024-04-26T02:05:30.353 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_items": 32, 2024-04-26T02:05:30.353 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_bytes": 0, 2024-04-26T02:05:30.353 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_items": 0, 2024-04-26T02:05:30.353 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_bytes": 0, 2024-04-26T02:05:30.353 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_items": 0, 2024-04-26T02:05:30.353 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_bytes": 0, 2024-04-26T02:05:30.353 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_items": 0, 2024-04-26T02:05:30.353 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_bytes": 0, 2024-04-26T02:05:30.353 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_items": 0, 2024-04-26T02:05:30.353 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_bytes": 0, 2024-04-26T02:05:30.353 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_items": 0, 2024-04-26T02:05:30.353 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T02:05:30.353 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_items": 0, 2024-04-26T02:05:30.354 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_bytes": 0, 2024-04-26T02:05:30.354 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_items": 0, 2024-04-26T02:05:30.354 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_bytes": 928, 2024-04-26T02:05:30.354 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_items": 2, 2024-04-26T02:05:30.354 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_bytes": 0, 2024-04-26T02:05:30.354 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_items": 0, 2024-04-26T02:05:30.354 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_bytes": 0, 2024-04-26T02:05:30.354 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_items": 0 2024-04-26T02:05:30.354 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:30.354 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T02:05:30.354 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-04-26T02:05:30.354 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 104857600, 2024-04-26T02:05:30.354 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-04-26T02:05:30.354 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 274, 2024-04-26T02:05:30.355 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 1704347, 2024-04-26T02:05:30.355 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-04-26T02:05:30.355 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 274, 2024-04-26T02:05:30.355 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-04-26T02:05:30.355 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-04-26T02:05:30.355 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 274, 2024-04-26T02:05:30.355 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 1704347, 2024-04-26T02:05:30.355 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-04-26T02:05:30.355 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-26T02:05:30.355 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-26T02:05:30.355 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-26T02:05:30.355 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:30.355 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:30.355 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-26T02:05:30.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:05:30 smithi172 ceph-mon[32785]: pgmap v373: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 110 KiB/s wr, 67 op/s 2024-04-26T02:05:30.787 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:05:30 smithi151 ceph-mon[28057]: pgmap v373: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 110 KiB/s wr, 67 op/s 2024-04-26T02:05:30.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:05:30 smithi165 ceph-mon[32648]: pgmap v373: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 110 KiB/s wr, 67 op/s 2024-04-26T02:05:31.211 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-04-26T02:05:31.212 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-04-26T02:05:31.212 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-04-26T02:05:31.212 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-04-26T02:05:31.212 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-04-26T02:05:31.212 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-04-26T02:05:31.212 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-04-26T02:05:31.212 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.a 2024-04-26T02:05:31.212 DEBUG:tasks.check_counter:Getting stats from d 2024-04-26T02:05:31.212 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-26T02:05:31.654 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:05:32.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:05:32 smithi172 ceph-mon[32785]: pgmap v374: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 67 KiB/s wr, 35 op/s 2024-04-26T02:05:32.795 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:05:32 smithi151 ceph-mon[28057]: pgmap v374: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 67 KiB/s wr, 35 op/s 2024-04-26T02:05:32.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:05:32 smithi165 ceph-mon[32648]: pgmap v374: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 67 KiB/s wr, 35 op/s 2024-04-26T02:05:33.914 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-26T02:05:33.914 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T02:05:33.914 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 280, 2024-04-26T02:05:33.914 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 268, 2024-04-26T02:05:33.914 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 1745008, 2024-04-26T02:05:33.915 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 165184, 2024-04-26T02:05:33.915 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 4, 2024-04-26T02:05:33.915 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 0, 2024-04-26T02:05:33.915 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.121059578, 2024-04-26T02:05:33.915 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.041983502, 2024-04-26T02:05:33.915 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.089725283, 2024-04-26T02:05:33.915 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.011986599, 2024-04-26T02:05:33.915 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:05:33.915 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 267, 2024-04-26T02:05:33.915 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.025885586, 2024-04-26T02:05:33.915 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000096949 2024-04-26T02:05:33.915 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:33.915 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:05:33.916 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-26T02:05:33.916 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-26T02:05:33.916 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-26T02:05:33.916 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:33.916 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 1745008, 2024-04-26T02:05:33.916 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 165184 2024-04-26T02:05:33.916 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:33.916 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T02:05:33.916 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 2, 2024-04-26T02:05:33.916 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 204, 2024-04-26T02:05:33.916 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 320, 2024-04-26T02:05:33.916 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 192112, 2024-04-26T02:05:33.916 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 5, 2024-04-26T02:05:33.917 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 3, 2024-04-26T02:05:33.917 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.039485752, 2024-04-26T02:05:33.917 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.029519235, 2024-04-26T02:05:33.917 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.004206453, 2024-04-26T02:05:33.917 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T02:05:33.917 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:05:33.917 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 201, 2024-04-26T02:05:33.917 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.028254524, 2024-04-26T02:05:33.917 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000140569 2024-04-26T02:05:33.917 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:33.917 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:05:33.918 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-26T02:05:33.918 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-26T02:05:33.918 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-26T02:05:33.918 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:33.918 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-26T02:05:33.918 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 192112 2024-04-26T02:05:33.918 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:33.918 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T02:05:33.918 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 4, 2024-04-26T02:05:33.918 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 4, 2024-04-26T02:05:33.918 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 3040, 2024-04-26T02:05:33.919 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 448, 2024-04-26T02:05:33.919 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 2, 2024-04-26T02:05:33.919 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 1, 2024-04-26T02:05:33.919 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.001454309, 2024-04-26T02:05:33.919 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.000071173, 2024-04-26T02:05:33.919 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.001282879, 2024-04-26T02:05:33.919 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T02:05:33.919 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:05:33.919 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 3, 2024-04-26T02:05:33.919 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.001436896, 2024-04-26T02:05:33.919 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000478965 2024-04-26T02:05:33.920 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:33.920 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:05:33.920 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-26T02:05:33.920 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-26T02:05:33.920 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-26T02:05:33.920 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:33.920 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-26T02:05:33.920 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 448 2024-04-26T02:05:33.920 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:33.920 INFO:teuthology.orchestra.run.smithi151.stdout: "cct": { 2024-04-26T02:05:33.920 INFO:teuthology.orchestra.run.smithi151.stdout: "total_workers": 0, 2024-04-26T02:05:33.920 INFO:teuthology.orchestra.run.smithi151.stdout: "unhealthy_workers": 0 2024-04-26T02:05:33.920 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:33.920 INFO:teuthology.orchestra.run.smithi151.stdout: "mempool": { 2024-04-26T02:05:33.921 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_bytes": 0, 2024-04-26T02:05:33.921 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_items": 0, 2024-04-26T02:05:33.921 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T02:05:33.921 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_items": 0, 2024-04-26T02:05:33.921 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T02:05:33.921 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_items": 0, 2024-04-26T02:05:33.921 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T02:05:33.921 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T02:05:33.921 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T02:05:33.921 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T02:05:33.921 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T02:05:33.922 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_items": 0, 2024-04-26T02:05:33.922 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T02:05:33.922 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T02:05:33.922 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_bytes": 0, 2024-04-26T02:05:33.922 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_items": 0, 2024-04-26T02:05:33.922 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_bytes": 0, 2024-04-26T02:05:33.922 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_items": 0, 2024-04-26T02:05:33.922 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T02:05:33.922 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T02:05:33.922 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T02:05:33.922 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T02:05:33.922 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T02:05:33.923 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_items": 0, 2024-04-26T02:05:33.923 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_bytes": 0, 2024-04-26T02:05:33.923 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_items": 0, 2024-04-26T02:05:33.923 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T02:05:33.923 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T02:05:33.923 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_bytes": 0, 2024-04-26T02:05:33.923 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_items": 0, 2024-04-26T02:05:33.923 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_bytes": 0, 2024-04-26T02:05:33.923 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_items": 0, 2024-04-26T02:05:33.923 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T02:05:33.923 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_items": 0, 2024-04-26T02:05:33.923 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T02:05:33.923 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_items": 0, 2024-04-26T02:05:33.924 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_bytes": 148714, 2024-04-26T02:05:33.924 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_items": 30, 2024-04-26T02:05:33.924 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_bytes": 0, 2024-04-26T02:05:33.924 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_items": 0, 2024-04-26T02:05:33.924 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_bytes": 0, 2024-04-26T02:05:33.924 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_items": 0, 2024-04-26T02:05:33.924 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_bytes": 0, 2024-04-26T02:05:33.924 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_items": 0, 2024-04-26T02:05:33.924 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_bytes": 0, 2024-04-26T02:05:33.924 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_items": 0, 2024-04-26T02:05:33.924 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_bytes": 0, 2024-04-26T02:05:33.924 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_items": 0, 2024-04-26T02:05:33.924 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T02:05:33.924 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_items": 0, 2024-04-26T02:05:33.925 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_bytes": 0, 2024-04-26T02:05:33.925 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_items": 0, 2024-04-26T02:05:33.925 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_bytes": 928, 2024-04-26T02:05:33.925 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_items": 2, 2024-04-26T02:05:33.925 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_bytes": 0, 2024-04-26T02:05:33.925 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_items": 0, 2024-04-26T02:05:33.925 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_bytes": 0, 2024-04-26T02:05:33.925 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_items": 0 2024-04-26T02:05:33.925 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:33.925 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T02:05:33.925 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-04-26T02:05:33.925 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 104857600, 2024-04-26T02:05:33.926 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-04-26T02:05:33.926 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 282, 2024-04-26T02:05:33.926 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 1705490, 2024-04-26T02:05:33.926 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-04-26T02:05:33.926 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 282, 2024-04-26T02:05:33.926 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-04-26T02:05:33.926 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-04-26T02:05:33.926 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 282, 2024-04-26T02:05:33.926 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 1705490, 2024-04-26T02:05:33.926 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-04-26T02:05:33.926 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-26T02:05:33.926 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-26T02:05:33.927 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-26T02:05:33.927 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:33.927 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:33.927 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-26T02:05:34.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:05:34 smithi151 ceph-mon[28057]: pgmap v375: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 69 KiB/s wr, 38 op/s 2024-04-26T02:05:34.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:05:34 smithi172 ceph-mon[32785]: pgmap v375: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 69 KiB/s wr, 38 op/s 2024-04-26T02:05:34.817 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-26T02:05:34.817 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-26T02:05:34.817 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-04-26T02:05:34.817 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-04-26T02:05:34.817 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-04-26T02:05:34.817 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-04-26T02:05:34.817 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-04-26T02:05:34.817 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-04-26T02:05:34.817 DEBUG:tasks.check_counter:Getting stats from g 2024-04-26T02:05:34.818 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-26T02:05:34.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:05:34 smithi165 ceph-mon[32648]: pgmap v375: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 69 KiB/s wr, 38 op/s 2024-04-26T02:05:35.278 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:05:36.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:05:36 smithi151 ceph-mon[28057]: pgmap v376: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-04-26T02:05:36.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:05:36 smithi172 ceph-mon[32785]: pgmap v376: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-04-26T02:05:36.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:05:36 smithi165 ceph-mon[32648]: pgmap v376: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-04-26T02:05:37.540 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-26T02:05:37.540 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T02:05:37.540 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 4, 2024-04-26T02:05:37.540 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 4, 2024-04-26T02:05:37.540 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 3040, 2024-04-26T02:05:37.540 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 448, 2024-04-26T02:05:37.541 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 4, 2024-04-26T02:05:37.541 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 0, 2024-04-26T02:05:37.541 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.004764139, 2024-04-26T02:05:37.541 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.000059617, 2024-04-26T02:05:37.541 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.001409705, 2024-04-26T02:05:37.541 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T02:05:37.541 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:05:37.541 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 3, 2024-04-26T02:05:37.541 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.001716256, 2024-04-26T02:05:37.541 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000572085 2024-04-26T02:05:37.541 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:37.541 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:05:37.541 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-26T02:05:37.541 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-26T02:05:37.542 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-26T02:05:37.542 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:37.542 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-26T02:05:37.542 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 448 2024-04-26T02:05:37.542 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:37.542 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T02:05:37.542 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 2, 2024-04-26T02:05:37.542 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 201, 2024-04-26T02:05:37.542 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 320, 2024-04-26T02:05:37.542 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 191584, 2024-04-26T02:05:37.543 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 5, 2024-04-26T02:05:37.543 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 3, 2024-04-26T02:05:37.543 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.039266036, 2024-04-26T02:05:37.543 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.028573354, 2024-04-26T02:05:37.543 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.004818719, 2024-04-26T02:05:37.543 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T02:05:37.543 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:05:37.543 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 198, 2024-04-26T02:05:37.543 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.028178577, 2024-04-26T02:05:37.543 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000142316 2024-04-26T02:05:37.543 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:37.543 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:05:37.543 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-26T02:05:37.544 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-26T02:05:37.544 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-26T02:05:37.544 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:37.544 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-26T02:05:37.544 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 191584 2024-04-26T02:05:37.544 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:37.544 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T02:05:37.544 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 275, 2024-04-26T02:05:37.544 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 263, 2024-04-26T02:05:37.544 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 1743552, 2024-04-26T02:05:37.544 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 162320, 2024-04-26T02:05:37.544 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 2, 2024-04-26T02:05:37.544 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 1, 2024-04-26T02:05:37.544 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.122669238, 2024-04-26T02:05:37.545 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.040502854, 2024-04-26T02:05:37.545 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.101710319, 2024-04-26T02:05:37.545 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.012021333, 2024-04-26T02:05:37.545 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:05:37.545 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 262, 2024-04-26T02:05:37.545 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.023571018, 2024-04-26T02:05:37.545 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000089965 2024-04-26T02:05:37.545 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:37.545 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:05:37.545 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-26T02:05:37.545 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-26T02:05:37.545 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-26T02:05:37.546 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:37.546 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 1743552, 2024-04-26T02:05:37.546 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 162320 2024-04-26T02:05:37.546 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:37.546 INFO:teuthology.orchestra.run.smithi151.stdout: "cct": { 2024-04-26T02:05:37.546 INFO:teuthology.orchestra.run.smithi151.stdout: "total_workers": 0, 2024-04-26T02:05:37.546 INFO:teuthology.orchestra.run.smithi151.stdout: "unhealthy_workers": 0 2024-04-26T02:05:37.546 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:37.546 INFO:teuthology.orchestra.run.smithi151.stdout: "mempool": { 2024-04-26T02:05:37.546 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_bytes": 0, 2024-04-26T02:05:37.546 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_items": 0, 2024-04-26T02:05:37.546 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T02:05:37.547 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_items": 0, 2024-04-26T02:05:37.547 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T02:05:37.547 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_items": 0, 2024-04-26T02:05:37.547 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T02:05:37.547 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T02:05:37.547 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T02:05:37.547 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T02:05:37.547 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T02:05:37.547 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_items": 0, 2024-04-26T02:05:37.547 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T02:05:37.548 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T02:05:37.548 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_bytes": 0, 2024-04-26T02:05:37.548 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_items": 0, 2024-04-26T02:05:37.548 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_bytes": 0, 2024-04-26T02:05:37.548 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_items": 0, 2024-04-26T02:05:37.548 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T02:05:37.548 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T02:05:37.548 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T02:05:37.548 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T02:05:37.548 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T02:05:37.548 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_items": 0, 2024-04-26T02:05:37.548 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_bytes": 0, 2024-04-26T02:05:37.548 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_items": 0, 2024-04-26T02:05:37.549 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T02:05:37.549 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T02:05:37.549 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_bytes": 0, 2024-04-26T02:05:37.549 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_items": 0, 2024-04-26T02:05:37.549 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_bytes": 0, 2024-04-26T02:05:37.549 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_items": 0, 2024-04-26T02:05:37.549 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T02:05:37.549 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_items": 0, 2024-04-26T02:05:37.549 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T02:05:37.549 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_items": 0, 2024-04-26T02:05:37.549 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_bytes": 149745, 2024-04-26T02:05:37.550 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_items": 31, 2024-04-26T02:05:37.550 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_bytes": 0, 2024-04-26T02:05:37.550 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_items": 0, 2024-04-26T02:05:37.550 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_bytes": 0, 2024-04-26T02:05:37.550 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_items": 0, 2024-04-26T02:05:37.550 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_bytes": 0, 2024-04-26T02:05:37.550 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_items": 0, 2024-04-26T02:05:37.550 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_bytes": 0, 2024-04-26T02:05:37.550 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_items": 0, 2024-04-26T02:05:37.551 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_bytes": 0, 2024-04-26T02:05:37.551 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_items": 0, 2024-04-26T02:05:37.551 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T02:05:37.551 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_items": 0, 2024-04-26T02:05:37.551 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_bytes": 0, 2024-04-26T02:05:37.551 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_items": 0, 2024-04-26T02:05:37.551 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_bytes": 928, 2024-04-26T02:05:37.551 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_items": 2, 2024-04-26T02:05:37.551 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_bytes": 0, 2024-04-26T02:05:37.551 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_items": 0, 2024-04-26T02:05:37.551 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_bytes": 0, 2024-04-26T02:05:37.551 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_items": 0 2024-04-26T02:05:37.551 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:37.551 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T02:05:37.552 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-04-26T02:05:37.552 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 104857600, 2024-04-26T02:05:37.552 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-04-26T02:05:37.552 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 277, 2024-04-26T02:05:37.552 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 1704737, 2024-04-26T02:05:37.552 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-04-26T02:05:37.552 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 277, 2024-04-26T02:05:37.552 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-04-26T02:05:37.552 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-04-26T02:05:37.552 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 277, 2024-04-26T02:05:37.552 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 1704737, 2024-04-26T02:05:37.552 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-04-26T02:05:37.552 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-26T02:05:37.553 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-26T02:05:37.553 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-26T02:05:37.553 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:37.553 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:37.553 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-26T02:05:38.491 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-04-26T02:05:38.491 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-04-26T02:05:38.491 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-04-26T02:05:38.491 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-04-26T02:05:38.491 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-04-26T02:05:38.491 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-04-26T02:05:38.491 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-04-26T02:05:38.491 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2024-04-26T02:05:38.491 DEBUG:tasks.check_counter:Getting stats from j 2024-04-26T02:05:38.492 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-26T02:05:38.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:05:38 smithi172 ceph-mon[32785]: pgmap v377: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-04-26T02:05:38.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:05:38 smithi165 ceph-mon[32648]: pgmap v377: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-04-26T02:05:38.861 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:05:38 smithi151 ceph-mon[28057]: pgmap v377: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-04-26T02:05:38.925 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:05:40.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:05:40 smithi172 ceph-mon[32785]: pgmap v378: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-26T02:05:40.828 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:05:40 smithi151 ceph-mon[28057]: pgmap v378: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-26T02:05:40.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:05:40 smithi165 ceph-mon[32648]: pgmap v378: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-26T02:05:41.241 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-04-26T02:05:41.241 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T02:05:41.241 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 0, 2024-04-26T02:05:41.241 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 2, 2024-04-26T02:05:41.241 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 0, 2024-04-26T02:05:41.241 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 0, 2024-04-26T02:05:41.241 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 4, 2024-04-26T02:05:41.241 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 0, 2024-04-26T02:05:41.241 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.004443360, 2024-04-26T02:05:41.241 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.000000000, 2024-04-26T02:05:41.241 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.001039379, 2024-04-26T02:05:41.242 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T02:05:41.242 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:05:41.242 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-26T02:05:41.242 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-26T02:05:41.242 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-26T02:05:41.242 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:41.242 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:05:41.242 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-26T02:05:41.242 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-26T02:05:41.242 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-26T02:05:41.242 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:41.242 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T02:05:41.243 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T02:05:41.243 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:41.243 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T02:05:41.243 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 6, 2024-04-26T02:05:41.243 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 208, 2024-04-26T02:05:41.243 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 3360, 2024-04-26T02:05:41.243 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 192912, 2024-04-26T02:05:41.243 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 5, 2024-04-26T02:05:41.243 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 3, 2024-04-26T02:05:41.243 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.041126510, 2024-04-26T02:05:41.243 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.030720145, 2024-04-26T02:05:41.243 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.004231005, 2024-04-26T02:05:41.243 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T02:05:41.243 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:05:41.244 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 206, 2024-04-26T02:05:41.244 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.030747510, 2024-04-26T02:05:41.244 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000149259 2024-04-26T02:05:41.244 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:41.244 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:05:41.244 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-26T02:05:41.244 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-26T02:05:41.244 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-26T02:05:41.244 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:41.244 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-04-26T02:05:41.244 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 192912 2024-04-26T02:05:41.244 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:41.244 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T02:05:41.244 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 283, 2024-04-26T02:05:41.244 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 271, 2024-04-26T02:05:41.245 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 1745824, 2024-04-26T02:05:41.245 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 167104, 2024-04-26T02:05:41.245 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 2, 2024-04-26T02:05:41.245 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 1, 2024-04-26T02:05:41.245 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.118312318, 2024-04-26T02:05:41.245 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.042371927, 2024-04-26T02:05:41.245 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.088410247, 2024-04-26T02:05:41.245 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.012149114, 2024-04-26T02:05:41.245 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:05:41.245 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 270, 2024-04-26T02:05:41.245 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.025009205, 2024-04-26T02:05:41.245 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000092626 2024-04-26T02:05:41.245 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:41.246 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:05:41.246 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-26T02:05:41.246 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-26T02:05:41.246 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-26T02:05:41.246 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:41.246 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 1745824, 2024-04-26T02:05:41.246 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 167104 2024-04-26T02:05:41.246 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:41.246 INFO:teuthology.orchestra.run.smithi151.stdout: "cct": { 2024-04-26T02:05:41.246 INFO:teuthology.orchestra.run.smithi151.stdout: "total_workers": 0, 2024-04-26T02:05:41.246 INFO:teuthology.orchestra.run.smithi151.stdout: "unhealthy_workers": 0 2024-04-26T02:05:41.246 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:41.246 INFO:teuthology.orchestra.run.smithi151.stdout: "mempool": { 2024-04-26T02:05:41.247 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_bytes": 0, 2024-04-26T02:05:41.247 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_items": 0, 2024-04-26T02:05:41.247 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T02:05:41.247 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_items": 0, 2024-04-26T02:05:41.247 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T02:05:41.247 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_items": 0, 2024-04-26T02:05:41.247 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T02:05:41.247 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T02:05:41.247 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T02:05:41.247 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T02:05:41.247 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T02:05:41.247 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_items": 0, 2024-04-26T02:05:41.247 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T02:05:41.247 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T02:05:41.248 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_bytes": 0, 2024-04-26T02:05:41.248 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_items": 0, 2024-04-26T02:05:41.248 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_bytes": 0, 2024-04-26T02:05:41.248 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_items": 0, 2024-04-26T02:05:41.248 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T02:05:41.248 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T02:05:41.248 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T02:05:41.248 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T02:05:41.248 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T02:05:41.248 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_items": 0, 2024-04-26T02:05:41.248 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_bytes": 0, 2024-04-26T02:05:41.248 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_items": 0, 2024-04-26T02:05:41.248 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T02:05:41.248 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T02:05:41.248 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_bytes": 0, 2024-04-26T02:05:41.249 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_items": 0, 2024-04-26T02:05:41.249 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_bytes": 0, 2024-04-26T02:05:41.249 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_items": 0, 2024-04-26T02:05:41.249 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T02:05:41.249 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_items": 0, 2024-04-26T02:05:41.249 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T02:05:41.249 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_items": 0, 2024-04-26T02:05:41.249 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_bytes": 148714, 2024-04-26T02:05:41.249 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_items": 30, 2024-04-26T02:05:41.249 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_bytes": 0, 2024-04-26T02:05:41.249 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_items": 0, 2024-04-26T02:05:41.249 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_bytes": 0, 2024-04-26T02:05:41.249 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_items": 0, 2024-04-26T02:05:41.250 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_bytes": 0, 2024-04-26T02:05:41.250 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_items": 0, 2024-04-26T02:05:41.250 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_bytes": 0, 2024-04-26T02:05:41.250 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_items": 0, 2024-04-26T02:05:41.250 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_bytes": 0, 2024-04-26T02:05:41.250 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_items": 0, 2024-04-26T02:05:41.250 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T02:05:41.250 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_items": 0, 2024-04-26T02:05:41.250 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_bytes": 0, 2024-04-26T02:05:41.250 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_items": 0, 2024-04-26T02:05:41.250 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_bytes": 928, 2024-04-26T02:05:41.250 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_items": 2, 2024-04-26T02:05:41.250 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_bytes": 0, 2024-04-26T02:05:41.250 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_items": 0, 2024-04-26T02:05:41.251 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_bytes": 0, 2024-04-26T02:05:41.251 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_items": 0 2024-04-26T02:05:41.251 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-04-26T02:05:41.251 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T02:05:41.251 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-04-26T02:05:41.251 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 104857600, 2024-04-26T02:05:41.251 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-04-26T02:05:41.251 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 285, 2024-04-26T02:05:41.251 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 1705880, 2024-04-26T02:05:41.251 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-04-26T02:05:41.251 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 285, 2024-04-26T02:05:41.251 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-04-26T02:05:41.251 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-04-26T02:05:41.251 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 285, 2024-04-26T02:05:41.252 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 1705880, 2024-04-26T02:05:41.252 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-04-26T02:05:41.252 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-04-26T02:05:41.252 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-04-26T02:05:41.252 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-04-26T02:05:41.252 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:41.252 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-04-26T02:05:41.252 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-04-26T02:05:42.052 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-26T02:05:42.052 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-26T02:05:42.052 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-04-26T02:05:42.052 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-04-26T02:05:42.052 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-04-26T02:05:42.053 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-04-26T02:05:42.053 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-04-26T02:05:42.053 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-04-26T02:05:42.053 DEBUG:tasks.check_counter:Getting stats from b 2024-04-26T02:05:42.053 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-26T02:05:42.305 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T02:05:42.776 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:05:42 smithi165 ceph-mon[32648]: pgmap v379: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.8 KiB/s wr, 5 op/s 2024-04-26T02:05:42.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:05:42 smithi172 ceph-mon[32785]: pgmap v379: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.8 KiB/s wr, 5 op/s 2024-04-26T02:05:42.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:05:42 smithi151 ceph-mon[28057]: pgmap v379: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.8 KiB/s wr, 5 op/s 2024-04-26T02:05:44.075 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-26T02:05:44.076 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T02:05:44.076 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 32750, 2024-04-26T02:05:44.076 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 34610, 2024-04-26T02:05:44.076 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 8560781, 2024-04-26T02:05:44.076 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 102936069, 2024-04-26T02:05:44.076 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 13, 2024-04-26T02:05:44.076 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 5, 2024-04-26T02:05:44.076 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 2.314202911, 2024-04-26T02:05:44.076 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.885535741, 2024-04-26T02:05:44.076 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.591902269, 2024-04-26T02:05:44.076 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.697655103, 2024-04-26T02:05:44.076 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:05:44.077 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 34608, 2024-04-26T02:05:44.077 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 3.725272334, 2024-04-26T02:05:44.077 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000107641 2024-04-26T02:05:44.077 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:44.077 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:05:44.077 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 10927, 2024-04-26T02:05:44.077 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.001590955, 2024-04-26T02:05:44.077 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000145 2024-04-26T02:05:44.077 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:44.077 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 415696, 2024-04-26T02:05:44.077 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 111744 2024-04-26T02:05:44.077 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:44.078 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T02:05:44.078 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 30822, 2024-04-26T02:05:44.078 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 30794, 2024-04-26T02:05:44.078 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 10528574, 2024-04-26T02:05:44.078 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 62823902, 2024-04-26T02:05:44.078 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 155, 2024-04-26T02:05:44.078 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 9, 2024-04-26T02:05:44.078 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 2.025191225, 2024-04-26T02:05:44.078 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.563870913, 2024-04-26T02:05:44.078 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.615045409, 2024-04-26T02:05:44.078 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.743339099, 2024-04-26T02:05:44.078 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:05:44.078 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 30791, 2024-04-26T02:05:44.079 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 4.745118473, 2024-04-26T02:05:44.079 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000154107 2024-04-26T02:05:44.079 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:44.079 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:05:44.079 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 429, 2024-04-26T02:05:44.079 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000062295, 2024-04-26T02:05:44.079 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000145 2024-04-26T02:05:44.079 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:44.079 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 1947952, 2024-04-26T02:05:44.079 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 558848 2024-04-26T02:05:44.079 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:44.079 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T02:05:44.080 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 68471, 2024-04-26T02:05:44.080 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 123370, 2024-04-26T02:05:44.080 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 21737428, 2024-04-26T02:05:44.080 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 102662166, 2024-04-26T02:05:44.080 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 10, 2024-04-26T02:05:44.080 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 6, 2024-04-26T02:05:44.080 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 4.594660395, 2024-04-26T02:05:44.080 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 2.007640872, 2024-04-26T02:05:44.080 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 2.425353646, 2024-04-26T02:05:44.080 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.482859801, 2024-04-26T02:05:44.080 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:05:44.080 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 123369, 2024-04-26T02:05:44.080 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 6.180362707, 2024-04-26T02:05:44.081 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000050096 2024-04-26T02:05:44.081 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:44.081 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:05:44.081 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 122169, 2024-04-26T02:05:44.081 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.021049013, 2024-04-26T02:05:44.081 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000172 2024-04-26T02:05:44.081 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:44.081 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 47872, 2024-04-26T02:05:44.081 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 176624 2024-04-26T02:05:44.081 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:44.081 INFO:teuthology.orchestra.run.smithi165.stdout: "cct": { 2024-04-26T02:05:44.081 INFO:teuthology.orchestra.run.smithi165.stdout: "total_workers": 1, 2024-04-26T02:05:44.081 INFO:teuthology.orchestra.run.smithi165.stdout: "unhealthy_workers": 0 2024-04-26T02:05:44.082 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:44.082 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-MDSRank": { 2024-04-26T02:05:44.082 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-04-26T02:05:44.082 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-04-26T02:05:44.082 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 92131, 2024-04-26T02:05:44.082 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 30.775420184, 2024-04-26T02:05:44.082 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000334039 2024-04-26T02:05:44.082 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-26T02:05:44.082 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:44.082 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-PurgeQueue": { 2024-04-26T02:05:44.082 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-04-26T02:05:44.082 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-04-26T02:05:44.082 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 8757, 2024-04-26T02:05:44.082 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 2.020496849, 2024-04-26T02:05:44.083 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000230729 2024-04-26T02:05:44.083 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-26T02:05:44.083 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:44.083 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": { 2024-04-26T02:05:44.083 INFO:teuthology.orchestra.run.smithi165.stdout: "request": 25050, 2024-04-26T02:05:44.083 INFO:teuthology.orchestra.run.smithi165.stdout: "reply": 25046, 2024-04-26T02:05:44.083 INFO:teuthology.orchestra.run.smithi165.stdout: "reply_latency": { 2024-04-26T02:05:44.083 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 25046, 2024-04-26T02:05:44.083 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 204.628660718, 2024-04-26T02:05:44.083 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.008170113 2024-04-26T02:05:44.083 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:44.083 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_reply": 0, 2024-04-26T02:05:44.083 INFO:teuthology.orchestra.run.smithi165.stdout: "forward": 4, 2024-04-26T02:05:44.083 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_complete": 0, 2024-04-26T02:05:44.084 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_keys": 0, 2024-04-26T02:05:44.084 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_commit": 1255, 2024-04-26T02:05:44.084 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_split": 22, 2024-04-26T02:05:44.084 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_merge": 0, 2024-04-26T02:05:44.084 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes": 9458, 2024-04-26T02:05:44.084 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_top": 5495, 2024-04-26T02:05:44.084 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_bottom": 3953, 2024-04-26T02:05:44.084 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pin_tail": 10, 2024-04-26T02:05:44.084 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pinned": 64, 2024-04-26T02:05:44.084 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_expired": 36, 2024-04-26T02:05:44.084 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_with_caps": 3, 2024-04-26T02:05:44.084 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": 3, 2024-04-26T02:05:44.084 INFO:teuthology.orchestra.run.smithi165.stdout: "subtrees": 4, 2024-04-26T02:05:44.084 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse": 30995, 2024-04-26T02:05:44.085 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_hit": 29993, 2024-04-26T02:05:44.085 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_forward": 0, 2024-04-26T02:05:44.085 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_discover": 3, 2024-04-26T02:05:44.085 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_dir_fetch": 0, 2024-04-26T02:05:44.085 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_remote_ino": 0, 2024-04-26T02:05:44.085 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_lock": 3, 2024-04-26T02:05:44.085 INFO:teuthology.orchestra.run.smithi165.stdout: "load_cent": 0, 2024-04-26T02:05:44.085 INFO:teuthology.orchestra.run.smithi165.stdout: "q": 0, 2024-04-26T02:05:44.085 INFO:teuthology.orchestra.run.smithi165.stdout: "exported": 6, 2024-04-26T02:05:44.085 INFO:teuthology.orchestra.run.smithi165.stdout: "exported_inodes": 25, 2024-04-26T02:05:44.085 INFO:teuthology.orchestra.run.smithi165.stdout: "imported": 4, 2024-04-26T02:05:44.085 INFO:teuthology.orchestra.run.smithi165.stdout: "imported_inodes": 40, 2024-04-26T02:05:44.085 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_dir_fetch": 0, 2024-04-26T02:05:44.085 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_backtrace_fetch": 0, 2024-04-26T02:05:44.085 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_peer_discover": 0, 2024-04-26T02:05:44.086 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rfiles": 4, 2024-04-26T02:05:44.086 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rbytes": 234, 2024-04-26T02:05:44.086 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rsnaps": 7, 2024-04-26T02:05:44.086 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T02:05:44.086 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_set_tag": 0, 2024-04-26T02:05:44.086 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T02:05:44.086 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_inotable_repaired": 0, 2024-04-26T02:05:44.086 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_inodes": 0, 2024-04-26T02:05:44.086 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T02:05:44.086 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T02:05:44.086 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_file_inodes": 0, 2024-04-26T02:05:44.086 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_inode_file_caps": 12, 2024-04-26T02:05:44.086 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_revoke": 14830, 2024-04-26T02:05:44.087 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_grant": 19561, 2024-04-26T02:05:44.087 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_trunc": 7863, 2024-04-26T02:05:44.087 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T02:05:44.087 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T02:05:44.087 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps": 20294, 2024-04-26T02:05:44.087 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps_dirty": 12536, 2024-04-26T02:05:44.087 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_cap_release": 471, 2024-04-26T02:05:44.087 INFO:teuthology.orchestra.run.smithi165.stdout: "process_request_cap_release": 14559 2024-04-26T02:05:44.087 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:44.087 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_cache": { 2024-04-26T02:05:44.087 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update": 409, 2024-04-26T02:05:44.087 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update_receipt": 9, 2024-04-26T02:05:44.087 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_try_discover": 6, 2024-04-26T02:05:44.088 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_send_discover": 6, 2024-04-26T02:05:44.088 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_handle_discover": 528, 2024-04-26T02:05:44.088 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays": 57, 2024-04-26T02:05:44.088 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_delayed": 0, 2024-04-26T02:05:44.088 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_enqueuing": 0, 2024-04-26T02:05:44.088 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_created": 5148, 2024-04-26T02:05:44.088 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_enqueued": 5088, 2024-04-26T02:05:44.088 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_reintegrated": 3, 2024-04-26T02:05:44.088 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_migrated": 0, 2024-04-26T02:05:44.088 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_processing": 0, 2024-04-26T02:05:44.088 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_enqueued": 0, 2024-04-26T02:05:44.088 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_prioritized": 0, 2024-04-26T02:05:44.088 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_started": 0, 2024-04-26T02:05:44.088 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_completed": 0, 2024-04-26T02:05:44.089 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_quiesce_path": 8, 2024-04-26T02:05:44.089 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_quiesce_inode": 32240, 2024-04-26T02:05:44.089 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T02:05:44.089 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_exportdir": 6, 2024-04-26T02:05:44.089 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_flush": 0, 2024-04-26T02:05:44.089 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragmentdir": 22, 2024-04-26T02:05:44.089 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragstats": 0, 2024-04-26T02:05:44.089 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_inodestats": 0 2024-04-26T02:05:44.089 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:44.089 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_log": { 2024-04-26T02:05:44.089 INFO:teuthology.orchestra.run.smithi165.stdout: "evlrg": 0, 2024-04-26T02:05:44.089 INFO:teuthology.orchestra.run.smithi165.stdout: "evadd": 47366, 2024-04-26T02:05:44.089 INFO:teuthology.orchestra.run.smithi165.stdout: "evex": 47356, 2024-04-26T02:05:44.089 INFO:teuthology.orchestra.run.smithi165.stdout: "evtrm": 47356, 2024-04-26T02:05:44.090 INFO:teuthology.orchestra.run.smithi165.stdout: "ev": 10, 2024-04-26T02:05:44.090 INFO:teuthology.orchestra.run.smithi165.stdout: "evexg": 0, 2024-04-26T02:05:44.090 INFO:teuthology.orchestra.run.smithi165.stdout: "evexd": 2419, 2024-04-26T02:05:44.090 INFO:teuthology.orchestra.run.smithi165.stdout: "segadd": 112, 2024-04-26T02:05:44.090 INFO:teuthology.orchestra.run.smithi165.stdout: "segex": 111, 2024-04-26T02:05:44.090 INFO:teuthology.orchestra.run.smithi165.stdout: "segtrm": 111, 2024-04-26T02:05:44.090 INFO:teuthology.orchestra.run.smithi165.stdout: "seg": 1, 2024-04-26T02:05:44.090 INFO:teuthology.orchestra.run.smithi165.stdout: "segmjr": 1, 2024-04-26T02:05:44.090 INFO:teuthology.orchestra.run.smithi165.stdout: "segexg": 0, 2024-04-26T02:05:44.090 INFO:teuthology.orchestra.run.smithi165.stdout: "segexd": 3, 2024-04-26T02:05:44.090 INFO:teuthology.orchestra.run.smithi165.stdout: "expos": 180193356, 2024-04-26T02:05:44.090 INFO:teuthology.orchestra.run.smithi165.stdout: "wrpos": 180264419, 2024-04-26T02:05:44.090 INFO:teuthology.orchestra.run.smithi165.stdout: "rdpos": 0, 2024-04-26T02:05:44.090 INFO:teuthology.orchestra.run.smithi165.stdout: "jlat": { 2024-04-26T02:05:44.091 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 26439, 2024-04-26T02:05:44.091 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 105.093568835, 2024-04-26T02:05:44.091 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.003974944 2024-04-26T02:05:44.091 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:44.091 INFO:teuthology.orchestra.run.smithi165.stdout: "replayed": 0 2024-04-26T02:05:44.091 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:44.091 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_mem": { 2024-04-26T02:05:44.091 INFO:teuthology.orchestra.run.smithi165.stdout: "ino": 4773, 2024-04-26T02:05:44.091 INFO:teuthology.orchestra.run.smithi165.stdout: "ino+": 9889, 2024-04-26T02:05:44.091 INFO:teuthology.orchestra.run.smithi165.stdout: "ino-": 5116, 2024-04-26T02:05:44.091 INFO:teuthology.orchestra.run.smithi165.stdout: "dir": 229, 2024-04-26T02:05:44.091 INFO:teuthology.orchestra.run.smithi165.stdout: "dir+": 266, 2024-04-26T02:05:44.091 INFO:teuthology.orchestra.run.smithi165.stdout: "dir-": 37, 2024-04-26T02:05:44.091 INFO:teuthology.orchestra.run.smithi165.stdout: "dn": 9458, 2024-04-26T02:05:44.092 INFO:teuthology.orchestra.run.smithi165.stdout: "dn+": 15153, 2024-04-26T02:05:44.092 INFO:teuthology.orchestra.run.smithi165.stdout: "dn-": 5695, 2024-04-26T02:05:44.092 INFO:teuthology.orchestra.run.smithi165.stdout: "cap": 3, 2024-04-26T02:05:44.092 INFO:teuthology.orchestra.run.smithi165.stdout: "cap+": 9838, 2024-04-26T02:05:44.092 INFO:teuthology.orchestra.run.smithi165.stdout: "cap-": 9835, 2024-04-26T02:05:44.092 INFO:teuthology.orchestra.run.smithi165.stdout: "rss": 119032, 2024-04-26T02:05:44.092 INFO:teuthology.orchestra.run.smithi165.stdout: "heap": 231800 2024-04-26T02:05:44.092 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:44.092 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_server": { 2024-04-26T02:05:44.092 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_client_request": 43700, 2024-04-26T02:05:44.092 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_server_request": 619, 2024-04-26T02:05:44.092 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_request": 25050, 2024-04-26T02:05:44.092 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_session": 46, 2024-04-26T02:05:44.093 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_peer_request": 1265, 2024-04-26T02:05:44.093 INFO:teuthology.orchestra.run.smithi165.stdout: "req_create_latency": { 2024-04-26T02:05:44.093 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 5098, 2024-04-26T02:05:44.093 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 56.512489208, 2024-04-26T02:05:44.093 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.011085227 2024-04-26T02:05:44.093 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:44.093 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getattr_latency": { 2024-04-26T02:05:44.093 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 135, 2024-04-26T02:05:44.093 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 1.806777556, 2024-04-26T02:05:44.093 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.013383537 2024-04-26T02:05:44.093 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:44.093 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getfilelock_latency": { 2024-04-26T02:05:44.093 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:44.093 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:44.094 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:44.094 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:44.094 INFO:teuthology.orchestra.run.smithi165.stdout: "req_link_latency": { 2024-04-26T02:05:44.094 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 3, 2024-04-26T02:05:44.094 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.003249624, 2024-04-26T02:05:44.094 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.001083208 2024-04-26T02:05:44.094 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:44.094 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookup_latency": { 2024-04-26T02:05:44.094 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 4877, 2024-04-26T02:05:44.094 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 5.962972949, 2024-04-26T02:05:44.094 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.001222672 2024-04-26T02:05:44.094 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:44.094 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookuphash_latency": { 2024-04-26T02:05:44.094 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:44.095 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:44.095 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:44.095 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:44.095 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupino_latency": { 2024-04-26T02:05:44.095 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1, 2024-04-26T02:05:44.095 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000197634, 2024-04-26T02:05:44.095 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000197634 2024-04-26T02:05:44.095 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:44.095 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupname_latency": { 2024-04-26T02:05:44.095 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:44.095 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:44.095 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:44.095 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:44.096 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupparent_latency": { 2024-04-26T02:05:44.096 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:44.096 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:44.096 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:44.096 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:44.096 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupsnap_latency": { 2024-04-26T02:05:44.096 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:44.096 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:44.096 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:44.097 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:44.097 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lssnap_latency": { 2024-04-26T02:05:44.097 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 12, 2024-04-26T02:05:44.097 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.002039414, 2024-04-26T02:05:44.097 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000169951 2024-04-26T02:05:44.097 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:44.097 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mkdir_latency": { 2024-04-26T02:05:44.097 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 73, 2024-04-26T02:05:44.097 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 14.473932978, 2024-04-26T02:05:44.098 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.198273054 2024-04-26T02:05:44.098 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:44.098 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mknod_latency": { 2024-04-26T02:05:44.098 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:44.098 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:44.098 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:44.098 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:44.098 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mksnap_latency": { 2024-04-26T02:05:44.098 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 12, 2024-04-26T02:05:44.098 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.640342809, 2024-04-26T02:05:44.098 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.053361900 2024-04-26T02:05:44.098 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:44.098 INFO:teuthology.orchestra.run.smithi165.stdout: "req_open_latency": { 2024-04-26T02:05:44.099 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 425, 2024-04-26T02:05:44.099 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.120749009, 2024-04-26T02:05:44.099 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000284115 2024-04-26T02:05:44.099 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:44.099 INFO:teuthology.orchestra.run.smithi165.stdout: "req_readdir_latency": { 2024-04-26T02:05:44.099 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 473, 2024-04-26T02:05:44.099 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 1.889201386, 2024-04-26T02:05:44.099 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.003994083 2024-04-26T02:05:44.099 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:44.099 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rename_latency": { 2024-04-26T02:05:44.099 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 92, 2024-04-26T02:05:44.099 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.294018559, 2024-04-26T02:05:44.099 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.003195853 2024-04-26T02:05:44.100 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:44.100 INFO:teuthology.orchestra.run.smithi165.stdout: "req_renamesnap_latency": { 2024-04-26T02:05:44.100 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:44.100 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:44.100 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:44.100 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:44.100 INFO:teuthology.orchestra.run.smithi165.stdout: "req_snapdiff_latency": { 2024-04-26T02:05:44.100 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:44.100 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:44.100 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:44.100 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:44.100 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmdir_latency": { 2024-04-26T02:05:44.100 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 63, 2024-04-26T02:05:44.101 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 2.671760113, 2024-04-26T02:05:44.101 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.042408890 2024-04-26T02:05:44.101 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:44.101 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmsnap_latency": { 2024-04-26T02:05:44.101 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 5, 2024-04-26T02:05:44.101 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.027707362, 2024-04-26T02:05:44.101 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.005541472 2024-04-26T02:05:44.101 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:44.101 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmxattr_latency": { 2024-04-26T02:05:44.101 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:44.101 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:44.101 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:44.101 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:44.102 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setattr_latency": { 2024-04-26T02:05:44.102 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 8729, 2024-04-26T02:05:44.102 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 112.254035345, 2024-04-26T02:05:44.102 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.012859896 2024-04-26T02:05:44.102 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:44.102 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setdirlayout_latency": { 2024-04-26T02:05:44.102 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:44.102 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:44.102 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:44.102 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:44.102 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setfilelock_latency": { 2024-04-26T02:05:44.102 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 8, 2024-04-26T02:05:44.102 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.005183624, 2024-04-26T02:05:44.102 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000647953 2024-04-26T02:05:44.103 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:44.103 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setlayout_latency": { 2024-04-26T02:05:44.103 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:44.103 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:44.103 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:44.103 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:44.103 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setxattr_latency": { 2024-04-26T02:05:44.103 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 8, 2024-04-26T02:05:44.103 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.021124060, 2024-04-26T02:05:44.103 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.002640507 2024-04-26T02:05:44.103 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:44.103 INFO:teuthology.orchestra.run.smithi165.stdout: "req_symlink_latency": { 2024-04-26T02:05:44.103 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 6, 2024-04-26T02:05:44.103 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.004616200, 2024-04-26T02:05:44.104 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000769366 2024-04-26T02:05:44.104 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:44.104 INFO:teuthology.orchestra.run.smithi165.stdout: "req_unlink_latency": { 2024-04-26T02:05:44.104 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 5026, 2024-04-26T02:05:44.104 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 7.938262888, 2024-04-26T02:05:44.104 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.001579439 2024-04-26T02:05:44.104 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:44.104 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_revoke_eviction": 0, 2024-04-26T02:05:44.104 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_acquisition_throttle": 0, 2024-04-26T02:05:44.104 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getvxattr_latency": { 2024-04-26T02:05:44.104 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:44.104 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:44.104 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:44.105 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-26T02:05:44.105 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:44.105 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_sessions": { 2024-04-26T02:05:44.105 INFO:teuthology.orchestra.run.smithi165.stdout: "session_count": 2, 2024-04-26T02:05:44.105 INFO:teuthology.orchestra.run.smithi165.stdout: "session_add": 6, 2024-04-26T02:05:44.105 INFO:teuthology.orchestra.run.smithi165.stdout: "session_remove": 4, 2024-04-26T02:05:44.105 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_open": 2, 2024-04-26T02:05:44.105 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_stale": 0, 2024-04-26T02:05:44.105 INFO:teuthology.orchestra.run.smithi165.stdout: "total_load": 9224, 2024-04-26T02:05:44.105 INFO:teuthology.orchestra.run.smithi165.stdout: "average_load": 4612, 2024-04-26T02:05:44.105 INFO:teuthology.orchestra.run.smithi165.stdout: "avg_session_uptime": 715, 2024-04-26T02:05:44.105 INFO:teuthology.orchestra.run.smithi165.stdout: "mdthresh_evicted": 0 2024-04-26T02:05:44.105 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:44.105 INFO:teuthology.orchestra.run.smithi165.stdout: "mempool": { 2024-04-26T02:05:44.105 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_bytes": 240, 2024-04-26T02:05:44.106 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_items": 240, 2024-04-26T02:05:44.106 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T02:05:44.106 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_items": 0, 2024-04-26T02:05:44.106 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T02:05:44.106 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_items": 0, 2024-04-26T02:05:44.106 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T02:05:44.106 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T02:05:44.106 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T02:05:44.106 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T02:05:44.106 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T02:05:44.106 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_items": 0, 2024-04-26T02:05:44.106 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T02:05:44.106 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T02:05:44.107 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_bytes": 0, 2024-04-26T02:05:44.107 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_items": 0, 2024-04-26T02:05:44.107 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_bytes": 0, 2024-04-26T02:05:44.107 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_items": 0, 2024-04-26T02:05:44.107 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T02:05:44.107 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T02:05:44.107 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T02:05:44.107 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T02:05:44.107 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T02:05:44.107 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_items": 0, 2024-04-26T02:05:44.107 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_bytes": 0, 2024-04-26T02:05:44.107 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_items": 0, 2024-04-26T02:05:44.107 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T02:05:44.107 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T02:05:44.107 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_bytes": 0, 2024-04-26T02:05:44.108 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_items": 0, 2024-04-26T02:05:44.108 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_bytes": 0, 2024-04-26T02:05:44.108 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_items": 0, 2024-04-26T02:05:44.108 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T02:05:44.108 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_items": 0, 2024-04-26T02:05:44.108 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T02:05:44.108 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_items": 0, 2024-04-26T02:05:44.108 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_bytes": 932295, 2024-04-26T02:05:44.108 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_items": 4853, 2024-04-26T02:05:44.108 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_bytes": 88, 2024-04-26T02:05:44.108 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_items": 1, 2024-04-26T02:05:44.108 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_bytes": 0, 2024-04-26T02:05:44.108 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_items": 0, 2024-04-26T02:05:44.108 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_bytes": 0, 2024-04-26T02:05:44.109 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_items": 0, 2024-04-26T02:05:44.109 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_bytes": 0, 2024-04-26T02:05:44.109 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_items": 0, 2024-04-26T02:05:44.109 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_bytes": 7720, 2024-04-26T02:05:44.109 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_items": 160, 2024-04-26T02:05:44.109 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T02:05:44.109 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_items": 0, 2024-04-26T02:05:44.109 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_bytes": 0, 2024-04-26T02:05:44.109 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_items": 0, 2024-04-26T02:05:44.109 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_bytes": 18156963, 2024-04-26T02:05:44.109 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_items": 187197, 2024-04-26T02:05:44.109 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_bytes": 0, 2024-04-26T02:05:44.109 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_items": 0, 2024-04-26T02:05:44.109 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_bytes": 0, 2024-04-26T02:05:44.110 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_items": 0 2024-04-26T02:05:44.110 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:44.110 INFO:teuthology.orchestra.run.smithi165.stdout: "objecter": { 2024-04-26T02:05:44.110 INFO:teuthology.orchestra.run.smithi165.stdout: "op_active": 0, 2024-04-26T02:05:44.110 INFO:teuthology.orchestra.run.smithi165.stdout: "op_laggy": 0, 2024-04-26T02:05:44.110 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send": 76087, 2024-04-26T02:05:44.110 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send_bytes": 190802711, 2024-04-26T02:05:44.110 INFO:teuthology.orchestra.run.smithi165.stdout: "op_resend": 31, 2024-04-26T02:05:44.110 INFO:teuthology.orchestra.run.smithi165.stdout: "op_reply": 76087, 2024-04-26T02:05:44.110 INFO:teuthology.orchestra.run.smithi165.stdout: "op_latency": { 2024-04-26T02:05:44.110 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 76087, 2024-04-26T02:05:44.110 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 1200.661831665, 2024-04-26T02:05:44.110 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.015780117 2024-04-26T02:05:44.111 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:44.111 INFO:teuthology.orchestra.run.smithi165.stdout: "op_inflight": 0, 2024-04-26T02:05:44.111 INFO:teuthology.orchestra.run.smithi165.stdout: "oplen_avg": { 2024-04-26T02:05:44.111 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 76087, 2024-04-26T02:05:44.111 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 89305 2024-04-26T02:05:44.111 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:44.111 INFO:teuthology.orchestra.run.smithi165.stdout: "op": 76087, 2024-04-26T02:05:44.111 INFO:teuthology.orchestra.run.smithi165.stdout: "op_r": 76, 2024-04-26T02:05:44.111 INFO:teuthology.orchestra.run.smithi165.stdout: "op_w": 76011, 2024-04-26T02:05:44.111 INFO:teuthology.orchestra.run.smithi165.stdout: "op_rmw": 0, 2024-04-26T02:05:44.111 INFO:teuthology.orchestra.run.smithi165.stdout: "op_pg": 0, 2024-04-26T02:05:44.111 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_stat": 1004, 2024-04-26T02:05:44.111 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_create": 5304, 2024-04-26T02:05:44.111 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_read": 76, 2024-04-26T02:05:44.112 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_write": 26515, 2024-04-26T02:05:44.112 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writefull": 232, 2024-04-26T02:05:44.112 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writesame": 0, 2024-04-26T02:05:44.112 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_append": 0, 2024-04-26T02:05:44.112 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_zero": 0, 2024-04-26T02:05:44.112 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_truncate": 6, 2024-04-26T02:05:44.112 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_delete": 5727, 2024-04-26T02:05:44.112 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_mapext": 0, 2024-04-26T02:05:44.112 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_sparse_read": 0, 2024-04-26T02:05:44.112 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_clonerange": 0, 2024-04-26T02:05:44.112 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_getxattr": 0, 2024-04-26T02:05:44.112 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_setxattr": 10609, 2024-04-26T02:05:44.112 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_cmpxattr": 0, 2024-04-26T02:05:44.112 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_rmxattr": 0, 2024-04-26T02:05:44.112 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_resetxattrs": 0, 2024-04-26T02:05:44.113 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_call": 0, 2024-04-26T02:05:44.113 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_watch": 0, 2024-04-26T02:05:44.113 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_notify": 0, 2024-04-26T02:05:44.113 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T02:05:44.113 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls": 0, 2024-04-26T02:05:44.113 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls_filter": 0, 2024-04-26T02:05:44.113 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_other": 36717, 2024-04-26T02:05:44.113 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_active": 0, 2024-04-26T02:05:44.113 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_send": 0, 2024-04-26T02:05:44.113 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_resend": 0, 2024-04-26T02:05:44.113 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_ping": 0, 2024-04-26T02:05:44.113 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_active": 0, 2024-04-26T02:05:44.113 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_send": 0, 2024-04-26T02:05:44.113 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_resend": 0, 2024-04-26T02:05:44.114 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_active": 0, 2024-04-26T02:05:44.114 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_send": 0, 2024-04-26T02:05:44.114 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_resend": 0, 2024-04-26T02:05:44.114 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_active": 0, 2024-04-26T02:05:44.114 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_send": 0, 2024-04-26T02:05:44.114 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_resend": 0, 2024-04-26T02:05:44.114 INFO:teuthology.orchestra.run.smithi165.stdout: "command_active": 0, 2024-04-26T02:05:44.114 INFO:teuthology.orchestra.run.smithi165.stdout: "command_send": 0, 2024-04-26T02:05:44.114 INFO:teuthology.orchestra.run.smithi165.stdout: "command_resend": 0, 2024-04-26T02:05:44.114 INFO:teuthology.orchestra.run.smithi165.stdout: "map_epoch": 107, 2024-04-26T02:05:44.114 INFO:teuthology.orchestra.run.smithi165.stdout: "map_full": 0, 2024-04-26T02:05:44.114 INFO:teuthology.orchestra.run.smithi165.stdout: "map_inc": 17, 2024-04-26T02:05:44.114 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_sessions": 12, 2024-04-26T02:05:44.114 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_open": 12, 2024-04-26T02:05:44.115 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_close": 0, 2024-04-26T02:05:44.115 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_laggy": 0, 2024-04-26T02:05:44.115 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_wr": 2813, 2024-04-26T02:05:44.115 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_rd": 0, 2024-04-26T02:05:44.115 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_del": 302 2024-04-26T02:05:44.115 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:44.115 INFO:teuthology.orchestra.run.smithi165.stdout: "oft": { 2024-04-26T02:05:44.115 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_objs": 1, 2024-04-26T02:05:44.115 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_kv_pairs": 0, 2024-04-26T02:05:44.115 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_updates": 0, 2024-04-26T02:05:44.115 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_removes": 0 2024-04-26T02:05:44.115 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:44.115 INFO:teuthology.orchestra.run.smithi165.stdout: "osd-slow-ops": { 2024-04-26T02:05:44.115 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_ops_count": 32845 2024-04-26T02:05:44.116 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:44.116 INFO:teuthology.orchestra.run.smithi165.stdout: "purge_queue": { 2024-04-26T02:05:44.116 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops": 0, 2024-04-26T02:05:44.116 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops_high_water": 34, 2024-04-26T02:05:44.116 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing": 0, 2024-04-26T02:05:44.116 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_high_water": 10, 2024-04-26T02:05:44.116 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed_ops": 5117, 2024-04-26T02:05:44.116 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed": 5088, 2024-04-26T02:05:44.116 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_item_in_journal": 0 2024-04-26T02:05:44.116 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:44.116 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T02:05:44.116 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-26T02:05:44.116 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-04-26T02:05:44.116 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-26T02:05:44.117 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 132039, 2024-04-26T02:05:44.117 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 28864185, 2024-04-26T02:05:44.117 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-26T02:05:44.117 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 132039, 2024-04-26T02:05:44.117 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-04-26T02:05:44.117 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-04-26T02:05:44.117 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 132039, 2024-04-26T02:05:44.117 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 28864185, 2024-04-26T02:05:44.117 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-26T02:05:44.117 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:44.117 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:44.117 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:44.117 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-26T02:05:44.117 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:44.118 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_bytes": { 2024-04-26T02:05:44.118 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-26T02:05:44.118 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-04-26T02:05:44.118 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-26T02:05:44.118 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-04-26T02:05:44.118 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-04-26T02:05:44.118 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-26T02:05:44.118 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-04-26T02:05:44.118 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 76087, 2024-04-26T02:05:44.118 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 191559527, 2024-04-26T02:05:44.118 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 76087, 2024-04-26T02:05:44.118 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 191559527, 2024-04-26T02:05:44.118 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-26T02:05:44.119 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:44.119 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:44.119 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:44.119 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-26T02:05:44.119 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:44.119 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_ops": { 2024-04-26T02:05:44.119 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-26T02:05:44.119 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 1024, 2024-04-26T02:05:44.119 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-26T02:05:44.119 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-04-26T02:05:44.119 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-04-26T02:05:44.119 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-26T02:05:44.119 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-04-26T02:05:44.119 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 76087, 2024-04-26T02:05:44.119 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 76087, 2024-04-26T02:05:44.120 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 76087, 2024-04-26T02:05:44.120 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 76087, 2024-04-26T02:05:44.120 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-26T02:05:44.120 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:44.120 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:44.120 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:44.120 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-26T02:05:44.120 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:44.120 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle": { 2024-04-26T02:05:44.120 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-26T02:05:44.120 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-04-26T02:05:44.120 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-26T02:05:44.120 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 5088, 2024-04-26T02:05:44.120 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 756816, 2024-04-26T02:05:44.121 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-26T02:05:44.121 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 5088, 2024-04-26T02:05:44.121 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-04-26T02:05:44.121 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-04-26T02:05:44.121 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 76, 2024-04-26T02:05:44.121 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 756816, 2024-04-26T02:05:44.121 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-26T02:05:44.121 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:44.121 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:44.121 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:44.121 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-26T02:05:44.121 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:44.121 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle-0x557f5b356500": { 2024-04-26T02:05:44.121 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-26T02:05:44.122 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-04-26T02:05:44.122 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-26T02:05:44.122 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 47366, 2024-04-26T02:05:44.122 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 176070115, 2024-04-26T02:05:44.122 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-26T02:05:44.122 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 47366, 2024-04-26T02:05:44.122 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-04-26T02:05:44.122 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-04-26T02:05:44.122 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 26439, 2024-04-26T02:05:44.122 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 176070115, 2024-04-26T02:05:44.122 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-26T02:05:44.122 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:44.122 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:44.123 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:44.123 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-26T02:05:44.123 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-26T02:05:44.123 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-26T02:05:44.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:05:44 smithi165 ceph-mon[32648]: pgmap v380: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-04-26T02:05:44.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:05:44 smithi151 ceph-mon[28057]: pgmap v380: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-04-26T02:05:44.933 INFO:tasks.check_counter:Daemon mds.b mds.exported=6 2024-04-26T02:05:44.933 INFO:tasks.check_counter:Daemon mds.b mds.imported=4 2024-04-26T02:05:44.933 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=409 2024-04-26T02:05:44.933 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=9 2024-04-26T02:05:44.933 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=7 2024-04-26T02:05:44.933 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=12 2024-04-26T02:05:44.933 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=5 2024-04-26T02:05:44.934 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=22 2024-04-26T02:05:44.934 DEBUG:tasks.check_counter:Getting stats from e 2024-04-26T02:05:44.934 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-26T02:05:45.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:05:44 smithi172 ceph-mon[32785]: pgmap v380: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-04-26T02:05:45.186 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T02:05:46.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:05:46 smithi165 ceph-mon[32648]: pgmap v381: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 852 B/s wr, 5 op/s 2024-04-26T02:05:46.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:05:46 smithi151 ceph-mon[28057]: pgmap v381: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 852 B/s wr, 5 op/s 2024-04-26T02:05:46.876 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-26T02:05:46.876 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T02:05:46.876 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 958, 2024-04-26T02:05:46.876 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 960, 2024-04-26T02:05:46.876 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 278641, 2024-04-26T02:05:46.876 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 304708, 2024-04-26T02:05:46.876 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 7, 2024-04-26T02:05:46.876 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 2, 2024-04-26T02:05:46.876 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.080286643, 2024-04-26T02:05:46.876 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.032970169, 2024-04-26T02:05:46.876 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.018075704, 2024-04-26T02:05:46.877 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.020266426, 2024-04-26T02:05:46.877 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:05:46.877 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 958, 2024-04-26T02:05:46.877 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.036275278, 2024-04-26T02:05:46.877 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000037865 2024-04-26T02:05:46.877 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:46.877 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:05:46.877 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:46.877 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:46.877 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:46.877 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:46.877 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T02:05:46.877 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T02:05:46.878 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:46.878 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T02:05:46.878 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 868, 2024-04-26T02:05:46.878 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 1071, 2024-04-26T02:05:46.878 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 215348, 2024-04-26T02:05:46.878 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 866011, 2024-04-26T02:05:46.878 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 8, 2024-04-26T02:05:46.878 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 5, 2024-04-26T02:05:46.878 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.081751621, 2024-04-26T02:05:46.878 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.028766097, 2024-04-26T02:05:46.878 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.018078420, 2024-04-26T02:05:46.878 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.026250346, 2024-04-26T02:05:46.878 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:05:46.879 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1068, 2024-04-26T02:05:46.879 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.050295831, 2024-04-26T02:05:46.879 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000047093 2024-04-26T02:05:46.879 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:46.879 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:05:46.879 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:46.879 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:46.879 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:46.879 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:46.879 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 3520, 2024-04-26T02:05:46.879 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 592208 2024-04-26T02:05:46.879 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:46.879 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T02:05:46.880 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 1263, 2024-04-26T02:05:46.880 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 1236, 2024-04-26T02:05:46.880 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 2149152, 2024-04-26T02:05:46.880 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 470802, 2024-04-26T02:05:46.880 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 3, 2024-04-26T02:05:46.880 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 2, 2024-04-26T02:05:46.880 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.156265680, 2024-04-26T02:05:46.880 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.046535488, 2024-04-26T02:05:46.880 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.069392441, 2024-04-26T02:05:46.880 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.040997093, 2024-04-26T02:05:46.880 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:05:46.880 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1235, 2024-04-26T02:05:46.880 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.061086904, 2024-04-26T02:05:46.880 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000049463 2024-04-26T02:05:46.881 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:46.881 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:05:46.881 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:46.881 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:46.881 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:46.881 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:46.881 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 1832304, 2024-04-26T02:05:46.881 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 170928 2024-04-26T02:05:46.881 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:46.881 INFO:teuthology.orchestra.run.smithi165.stdout: "cct": { 2024-04-26T02:05:46.881 INFO:teuthology.orchestra.run.smithi165.stdout: "total_workers": 1, 2024-04-26T02:05:46.881 INFO:teuthology.orchestra.run.smithi165.stdout: "unhealthy_workers": 0 2024-04-26T02:05:46.881 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:46.881 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-MDSRank": { 2024-04-26T02:05:46.882 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-04-26T02:05:46.882 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-04-26T02:05:46.882 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 4649, 2024-04-26T02:05:46.882 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.122152492, 2024-04-26T02:05:46.882 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000026275 2024-04-26T02:05:46.882 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-26T02:05:46.882 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:46.882 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-PurgeQueue": { 2024-04-26T02:05:46.882 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-04-26T02:05:46.882 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-04-26T02:05:46.882 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:46.882 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:46.882 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:46.882 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-26T02:05:46.883 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:46.883 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": { 2024-04-26T02:05:46.883 INFO:teuthology.orchestra.run.smithi165.stdout: "request": 0, 2024-04-26T02:05:46.883 INFO:teuthology.orchestra.run.smithi165.stdout: "reply": 0, 2024-04-26T02:05:46.883 INFO:teuthology.orchestra.run.smithi165.stdout: "reply_latency": { 2024-04-26T02:05:46.883 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:46.883 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:46.883 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:46.883 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:46.883 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_reply": 0, 2024-04-26T02:05:46.883 INFO:teuthology.orchestra.run.smithi165.stdout: "forward": 0, 2024-04-26T02:05:46.883 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_complete": 0, 2024-04-26T02:05:46.883 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_keys": 0, 2024-04-26T02:05:46.883 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_commit": 0, 2024-04-26T02:05:46.883 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_split": 0, 2024-04-26T02:05:46.884 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_merge": 0, 2024-04-26T02:05:46.884 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes": 2, 2024-04-26T02:05:46.884 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_top": 0, 2024-04-26T02:05:46.884 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_bottom": 2, 2024-04-26T02:05:46.884 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pin_tail": 0, 2024-04-26T02:05:46.884 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pinned": 2, 2024-04-26T02:05:46.884 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_expired": 0, 2024-04-26T02:05:46.884 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_with_caps": 0, 2024-04-26T02:05:46.884 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": 0, 2024-04-26T02:05:46.884 INFO:teuthology.orchestra.run.smithi165.stdout: "subtrees": 1, 2024-04-26T02:05:46.884 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse": 0, 2024-04-26T02:05:46.884 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_hit": 0, 2024-04-26T02:05:46.884 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_forward": 0, 2024-04-26T02:05:46.884 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_discover": 0, 2024-04-26T02:05:46.884 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_dir_fetch": 0, 2024-04-26T02:05:46.885 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_remote_ino": 0, 2024-04-26T02:05:46.885 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_lock": 0, 2024-04-26T02:05:46.885 INFO:teuthology.orchestra.run.smithi165.stdout: "load_cent": 0, 2024-04-26T02:05:46.885 INFO:teuthology.orchestra.run.smithi165.stdout: "q": 0, 2024-04-26T02:05:46.885 INFO:teuthology.orchestra.run.smithi165.stdout: "exported": 0, 2024-04-26T02:05:46.885 INFO:teuthology.orchestra.run.smithi165.stdout: "exported_inodes": 0, 2024-04-26T02:05:46.885 INFO:teuthology.orchestra.run.smithi165.stdout: "imported": 0, 2024-04-26T02:05:46.885 INFO:teuthology.orchestra.run.smithi165.stdout: "imported_inodes": 0, 2024-04-26T02:05:46.885 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_dir_fetch": 0, 2024-04-26T02:05:46.885 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_backtrace_fetch": 0, 2024-04-26T02:05:46.885 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_peer_discover": 0, 2024-04-26T02:05:46.885 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rfiles": 0, 2024-04-26T02:05:46.885 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rbytes": 0, 2024-04-26T02:05:46.885 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rsnaps": 0, 2024-04-26T02:05:46.885 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T02:05:46.886 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_set_tag": 0, 2024-04-26T02:05:46.886 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T02:05:46.886 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_inotable_repaired": 0, 2024-04-26T02:05:46.886 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_inodes": 0, 2024-04-26T02:05:46.886 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T02:05:46.886 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T02:05:46.886 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_file_inodes": 0, 2024-04-26T02:05:46.886 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_inode_file_caps": 0, 2024-04-26T02:05:46.886 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_revoke": 0, 2024-04-26T02:05:46.886 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_grant": 0, 2024-04-26T02:05:46.886 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T02:05:46.886 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T02:05:46.887 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T02:05:46.887 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps": 0, 2024-04-26T02:05:46.887 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps_dirty": 0, 2024-04-26T02:05:46.887 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_cap_release": 0, 2024-04-26T02:05:46.887 INFO:teuthology.orchestra.run.smithi165.stdout: "process_request_cap_release": 0 2024-04-26T02:05:46.887 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:46.887 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_cache": { 2024-04-26T02:05:46.887 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update": 0, 2024-04-26T02:05:46.887 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update_receipt": 0, 2024-04-26T02:05:46.887 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_try_discover": 0, 2024-04-26T02:05:46.887 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_send_discover": 0, 2024-04-26T02:05:46.887 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_handle_discover": 0, 2024-04-26T02:05:46.888 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays": 0, 2024-04-26T02:05:46.888 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_delayed": 0, 2024-04-26T02:05:46.888 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_enqueuing": 0, 2024-04-26T02:05:46.888 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_created": 0, 2024-04-26T02:05:46.888 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_enqueued": 0, 2024-04-26T02:05:46.888 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_reintegrated": 0, 2024-04-26T02:05:46.888 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_migrated": 0, 2024-04-26T02:05:46.888 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_processing": 0, 2024-04-26T02:05:46.888 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_enqueued": 0, 2024-04-26T02:05:46.888 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_prioritized": 0, 2024-04-26T02:05:46.888 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_started": 0, 2024-04-26T02:05:46.888 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_completed": 0, 2024-04-26T02:05:46.888 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_quiesce_path": 0, 2024-04-26T02:05:46.888 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_quiesce_inode": 0, 2024-04-26T02:05:46.888 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T02:05:46.889 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_exportdir": 0, 2024-04-26T02:05:46.889 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_flush": 0, 2024-04-26T02:05:46.889 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragmentdir": 0, 2024-04-26T02:05:46.889 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragstats": 0, 2024-04-26T02:05:46.889 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_inodestats": 0 2024-04-26T02:05:46.889 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:46.889 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_log": { 2024-04-26T02:05:46.889 INFO:teuthology.orchestra.run.smithi165.stdout: "evlrg": 0, 2024-04-26T02:05:46.889 INFO:teuthology.orchestra.run.smithi165.stdout: "evadd": 0, 2024-04-26T02:05:46.889 INFO:teuthology.orchestra.run.smithi165.stdout: "evex": 0, 2024-04-26T02:05:46.889 INFO:teuthology.orchestra.run.smithi165.stdout: "evtrm": 0, 2024-04-26T02:05:46.889 INFO:teuthology.orchestra.run.smithi165.stdout: "ev": 17, 2024-04-26T02:05:46.890 INFO:teuthology.orchestra.run.smithi165.stdout: "evexg": 0, 2024-04-26T02:05:46.890 INFO:teuthology.orchestra.run.smithi165.stdout: "evexd": 0, 2024-04-26T02:05:46.890 INFO:teuthology.orchestra.run.smithi165.stdout: "segadd": 0, 2024-04-26T02:05:46.890 INFO:teuthology.orchestra.run.smithi165.stdout: "segex": 0, 2024-04-26T02:05:46.890 INFO:teuthology.orchestra.run.smithi165.stdout: "segtrm": 0, 2024-04-26T02:05:46.890 INFO:teuthology.orchestra.run.smithi165.stdout: "seg": 2, 2024-04-26T02:05:46.890 INFO:teuthology.orchestra.run.smithi165.stdout: "segmjr": 2, 2024-04-26T02:05:46.890 INFO:teuthology.orchestra.run.smithi165.stdout: "segexg": 0, 2024-04-26T02:05:46.890 INFO:teuthology.orchestra.run.smithi165.stdout: "segexd": 0, 2024-04-26T02:05:46.890 INFO:teuthology.orchestra.run.smithi165.stdout: "expos": 4194304, 2024-04-26T02:05:46.890 INFO:teuthology.orchestra.run.smithi165.stdout: "wrpos": 4221502, 2024-04-26T02:05:46.891 INFO:teuthology.orchestra.run.smithi165.stdout: "rdpos": 4218897, 2024-04-26T02:05:46.891 INFO:teuthology.orchestra.run.smithi165.stdout: "jlat": { 2024-04-26T02:05:46.891 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:46.891 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:46.891 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:46.891 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:46.891 INFO:teuthology.orchestra.run.smithi165.stdout: "replayed": 17 2024-04-26T02:05:46.891 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:46.891 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_mem": { 2024-04-26T02:05:46.892 INFO:teuthology.orchestra.run.smithi165.stdout: "ino": 4, 2024-04-26T02:05:46.892 INFO:teuthology.orchestra.run.smithi165.stdout: "ino+": 17, 2024-04-26T02:05:46.892 INFO:teuthology.orchestra.run.smithi165.stdout: "ino-": 13, 2024-04-26T02:05:46.892 INFO:teuthology.orchestra.run.smithi165.stdout: "dir": 2, 2024-04-26T02:05:46.892 INFO:teuthology.orchestra.run.smithi165.stdout: "dir+": 14, 2024-04-26T02:05:46.892 INFO:teuthology.orchestra.run.smithi165.stdout: "dir-": 12, 2024-04-26T02:05:46.892 INFO:teuthology.orchestra.run.smithi165.stdout: "dn": 2, 2024-04-26T02:05:46.892 INFO:teuthology.orchestra.run.smithi165.stdout: "dn+": 16, 2024-04-26T02:05:46.892 INFO:teuthology.orchestra.run.smithi165.stdout: "dn-": 14, 2024-04-26T02:05:46.892 INFO:teuthology.orchestra.run.smithi165.stdout: "cap": 0, 2024-04-26T02:05:46.892 INFO:teuthology.orchestra.run.smithi165.stdout: "cap+": 0, 2024-04-26T02:05:46.892 INFO:teuthology.orchestra.run.smithi165.stdout: "cap-": 0, 2024-04-26T02:05:46.892 INFO:teuthology.orchestra.run.smithi165.stdout: "rss": 51292, 2024-04-26T02:05:46.892 INFO:teuthology.orchestra.run.smithi165.stdout: "heap": 215416 2024-04-26T02:05:46.893 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:46.893 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_server": { 2024-04-26T02:05:46.893 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_client_request": 0, 2024-04-26T02:05:46.893 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_server_request": 0, 2024-04-26T02:05:46.893 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_request": 0, 2024-04-26T02:05:46.893 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_session": 0, 2024-04-26T02:05:46.893 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_peer_request": 0, 2024-04-26T02:05:46.893 INFO:teuthology.orchestra.run.smithi165.stdout: "req_create_latency": { 2024-04-26T02:05:46.893 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:46.893 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:46.893 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:46.893 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:46.893 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getattr_latency": { 2024-04-26T02:05:46.893 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:46.894 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:46.894 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:46.894 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:46.894 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getfilelock_latency": { 2024-04-26T02:05:46.894 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:46.894 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:46.894 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:46.894 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:46.894 INFO:teuthology.orchestra.run.smithi165.stdout: "req_link_latency": { 2024-04-26T02:05:46.894 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:46.895 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:46.895 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:46.895 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:46.895 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookup_latency": { 2024-04-26T02:05:46.895 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:46.895 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:46.895 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:46.895 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:46.895 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookuphash_latency": { 2024-04-26T02:05:46.895 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:46.895 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:46.895 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:46.895 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:46.895 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupino_latency": { 2024-04-26T02:05:46.896 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:46.896 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:46.896 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:46.896 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:46.896 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupname_latency": { 2024-04-26T02:05:46.896 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:46.896 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:46.896 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:46.896 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:46.896 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupparent_latency": { 2024-04-26T02:05:46.896 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:46.897 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:46.897 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:46.897 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:46.897 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupsnap_latency": { 2024-04-26T02:05:46.897 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:46.897 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:46.897 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:46.897 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:46.897 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lssnap_latency": { 2024-04-26T02:05:46.897 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:46.897 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:46.897 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:46.897 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:46.897 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mkdir_latency": { 2024-04-26T02:05:46.898 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:46.898 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:46.898 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:46.898 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:46.898 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mknod_latency": { 2024-04-26T02:05:46.898 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:46.899 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:46.899 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:46.899 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:46.899 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mksnap_latency": { 2024-04-26T02:05:46.899 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:46.899 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:46.899 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:46.899 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:46.899 INFO:teuthology.orchestra.run.smithi165.stdout: "req_open_latency": { 2024-04-26T02:05:46.899 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:46.899 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:46.899 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:46.899 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:46.899 INFO:teuthology.orchestra.run.smithi165.stdout: "req_readdir_latency": { 2024-04-26T02:05:46.899 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:46.900 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:46.900 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:46.900 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:46.900 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rename_latency": { 2024-04-26T02:05:46.900 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:46.900 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:46.900 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:46.900 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:46.900 INFO:teuthology.orchestra.run.smithi165.stdout: "req_renamesnap_latency": { 2024-04-26T02:05:46.900 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:46.900 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:46.900 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:46.900 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:46.901 INFO:teuthology.orchestra.run.smithi165.stdout: "req_snapdiff_latency": { 2024-04-26T02:05:46.901 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:46.901 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:46.901 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:46.901 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:46.901 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmdir_latency": { 2024-04-26T02:05:46.901 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:46.901 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:46.901 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:46.901 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:46.901 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmsnap_latency": { 2024-04-26T02:05:46.901 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:46.901 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:46.901 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:46.901 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:46.902 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmxattr_latency": { 2024-04-26T02:05:46.902 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:46.902 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:46.902 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:46.902 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:46.902 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setattr_latency": { 2024-04-26T02:05:46.902 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:46.902 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:46.902 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:46.902 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:46.902 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setdirlayout_latency": { 2024-04-26T02:05:46.902 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:46.902 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:46.902 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:46.903 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:46.903 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setfilelock_latency": { 2024-04-26T02:05:46.903 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:46.903 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:46.903 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:46.903 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:46.903 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setlayout_latency": { 2024-04-26T02:05:46.903 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:46.903 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:46.903 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:46.903 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:46.903 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setxattr_latency": { 2024-04-26T02:05:46.903 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:46.903 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:46.903 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:46.904 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:46.904 INFO:teuthology.orchestra.run.smithi165.stdout: "req_symlink_latency": { 2024-04-26T02:05:46.904 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:46.904 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:46.904 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:46.904 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:46.904 INFO:teuthology.orchestra.run.smithi165.stdout: "req_unlink_latency": { 2024-04-26T02:05:46.904 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:46.904 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:46.904 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:46.904 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:46.905 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_revoke_eviction": 0, 2024-04-26T02:05:46.905 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_acquisition_throttle": 0, 2024-04-26T02:05:46.905 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getvxattr_latency": { 2024-04-26T02:05:46.905 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:46.905 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:46.905 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:46.906 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-26T02:05:46.906 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:46.906 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_sessions": { 2024-04-26T02:05:46.906 INFO:teuthology.orchestra.run.smithi165.stdout: "session_count": 2, 2024-04-26T02:05:46.906 INFO:teuthology.orchestra.run.smithi165.stdout: "session_add": 3, 2024-04-26T02:05:46.906 INFO:teuthology.orchestra.run.smithi165.stdout: "session_remove": 1, 2024-04-26T02:05:46.907 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_open": 2, 2024-04-26T02:05:46.907 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_stale": 0, 2024-04-26T02:05:46.907 INFO:teuthology.orchestra.run.smithi165.stdout: "total_load": 0, 2024-04-26T02:05:46.907 INFO:teuthology.orchestra.run.smithi165.stdout: "average_load": 0, 2024-04-26T02:05:46.907 INFO:teuthology.orchestra.run.smithi165.stdout: "avg_session_uptime": 2371, 2024-04-26T02:05:46.907 INFO:teuthology.orchestra.run.smithi165.stdout: "mdthresh_evicted": 0 2024-04-26T02:05:46.907 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:46.907 INFO:teuthology.orchestra.run.smithi165.stdout: "mempool": { 2024-04-26T02:05:46.907 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_bytes": 0, 2024-04-26T02:05:46.907 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_items": 0, 2024-04-26T02:05:46.908 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T02:05:46.908 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_items": 0, 2024-04-26T02:05:46.908 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T02:05:46.908 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_items": 0, 2024-04-26T02:05:46.908 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T02:05:46.908 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T02:05:46.908 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T02:05:46.908 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T02:05:46.908 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T02:05:46.908 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_items": 0, 2024-04-26T02:05:46.908 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T02:05:46.908 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T02:05:46.908 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_bytes": 0, 2024-04-26T02:05:46.908 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_items": 0, 2024-04-26T02:05:46.909 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_bytes": 0, 2024-04-26T02:05:46.909 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_items": 0, 2024-04-26T02:05:46.909 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T02:05:46.909 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T02:05:46.909 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T02:05:46.909 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T02:05:46.909 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T02:05:46.909 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_items": 0, 2024-04-26T02:05:46.909 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_bytes": 0, 2024-04-26T02:05:46.909 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_items": 0, 2024-04-26T02:05:46.909 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T02:05:46.909 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T02:05:46.909 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_bytes": 0, 2024-04-26T02:05:46.909 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_items": 0, 2024-04-26T02:05:46.909 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_bytes": 0, 2024-04-26T02:05:46.910 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_items": 0, 2024-04-26T02:05:46.910 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T02:05:46.910 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_items": 0, 2024-04-26T02:05:46.910 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T02:05:46.910 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_items": 0, 2024-04-26T02:05:46.910 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_bytes": 190139, 2024-04-26T02:05:46.910 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_items": 52, 2024-04-26T02:05:46.910 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_bytes": 0, 2024-04-26T02:05:46.910 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_items": 0, 2024-04-26T02:05:46.910 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_bytes": 0, 2024-04-26T02:05:46.910 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_items": 0, 2024-04-26T02:05:46.910 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_bytes": 0, 2024-04-26T02:05:46.911 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_items": 0, 2024-04-26T02:05:46.911 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_bytes": 0, 2024-04-26T02:05:46.911 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_items": 0, 2024-04-26T02:05:46.911 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_bytes": 7720, 2024-04-26T02:05:46.911 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_items": 160, 2024-04-26T02:05:46.911 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T02:05:46.911 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_items": 0, 2024-04-26T02:05:46.911 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_bytes": 0, 2024-04-26T02:05:46.911 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_items": 0, 2024-04-26T02:05:46.911 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_bytes": 15136, 2024-04-26T02:05:46.911 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_items": 40, 2024-04-26T02:05:46.911 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_bytes": 0, 2024-04-26T02:05:46.911 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_items": 0, 2024-04-26T02:05:46.912 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_bytes": 0, 2024-04-26T02:05:46.912 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_items": 0 2024-04-26T02:05:46.912 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:46.912 INFO:teuthology.orchestra.run.smithi165.stdout: "objecter": { 2024-04-26T02:05:46.912 INFO:teuthology.orchestra.run.smithi165.stdout: "op_active": 0, 2024-04-26T02:05:46.912 INFO:teuthology.orchestra.run.smithi165.stdout: "op_laggy": 0, 2024-04-26T02:05:46.912 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send": 2762, 2024-04-26T02:05:46.912 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send_bytes": 21, 2024-04-26T02:05:46.912 INFO:teuthology.orchestra.run.smithi165.stdout: "op_resend": 3, 2024-04-26T02:05:46.912 INFO:teuthology.orchestra.run.smithi165.stdout: "op_reply": 2762, 2024-04-26T02:05:46.912 INFO:teuthology.orchestra.run.smithi165.stdout: "op_latency": { 2024-04-26T02:05:46.912 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 2762, 2024-04-26T02:05:46.912 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 1.982966765, 2024-04-26T02:05:46.913 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000717945 2024-04-26T02:05:46.913 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:46.913 INFO:teuthology.orchestra.run.smithi165.stdout: "op_inflight": 0, 2024-04-26T02:05:46.913 INFO:teuthology.orchestra.run.smithi165.stdout: "oplen_avg": { 2024-04-26T02:05:46.913 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 2762, 2024-04-26T02:05:46.913 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 2763 2024-04-26T02:05:46.913 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:46.913 INFO:teuthology.orchestra.run.smithi165.stdout: "op": 2762, 2024-04-26T02:05:46.913 INFO:teuthology.orchestra.run.smithi165.stdout: "op_r": 2762, 2024-04-26T02:05:46.913 INFO:teuthology.orchestra.run.smithi165.stdout: "op_w": 0, 2024-04-26T02:05:46.913 INFO:teuthology.orchestra.run.smithi165.stdout: "op_rmw": 0, 2024-04-26T02:05:46.913 INFO:teuthology.orchestra.run.smithi165.stdout: "op_pg": 0, 2024-04-26T02:05:46.913 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_stat": 1802, 2024-04-26T02:05:46.913 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_create": 0, 2024-04-26T02:05:46.913 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_read": 958, 2024-04-26T02:05:46.914 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_write": 0, 2024-04-26T02:05:46.914 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writefull": 0, 2024-04-26T02:05:46.914 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writesame": 0, 2024-04-26T02:05:46.914 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_append": 0, 2024-04-26T02:05:46.914 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_zero": 0, 2024-04-26T02:05:46.914 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_truncate": 0, 2024-04-26T02:05:46.914 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_delete": 0, 2024-04-26T02:05:46.914 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_mapext": 0, 2024-04-26T02:05:46.914 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_sparse_read": 0, 2024-04-26T02:05:46.914 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_clonerange": 0, 2024-04-26T02:05:46.914 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_getxattr": 1, 2024-04-26T02:05:46.914 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_setxattr": 0, 2024-04-26T02:05:46.914 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_cmpxattr": 0, 2024-04-26T02:05:46.914 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_rmxattr": 0, 2024-04-26T02:05:46.914 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_resetxattrs": 0, 2024-04-26T02:05:46.915 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_call": 0, 2024-04-26T02:05:46.915 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_watch": 0, 2024-04-26T02:05:46.915 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_notify": 0, 2024-04-26T02:05:46.915 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T02:05:46.915 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls": 0, 2024-04-26T02:05:46.915 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls_filter": 0, 2024-04-26T02:05:46.915 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_other": 0, 2024-04-26T02:05:46.915 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_active": 0, 2024-04-26T02:05:46.915 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_send": 0, 2024-04-26T02:05:46.915 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_resend": 0, 2024-04-26T02:05:46.915 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_ping": 0, 2024-04-26T02:05:46.915 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_active": 0, 2024-04-26T02:05:46.915 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_send": 0, 2024-04-26T02:05:46.915 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_resend": 0, 2024-04-26T02:05:46.915 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_active": 0, 2024-04-26T02:05:46.915 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_send": 0, 2024-04-26T02:05:46.915 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_resend": 0, 2024-04-26T02:05:46.916 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_active": 0, 2024-04-26T02:05:46.916 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_send": 0, 2024-04-26T02:05:46.916 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_resend": 0, 2024-04-26T02:05:46.916 INFO:teuthology.orchestra.run.smithi165.stdout: "command_active": 0, 2024-04-26T02:05:46.916 INFO:teuthology.orchestra.run.smithi165.stdout: "command_send": 0, 2024-04-26T02:05:46.916 INFO:teuthology.orchestra.run.smithi165.stdout: "command_resend": 0, 2024-04-26T02:05:46.916 INFO:teuthology.orchestra.run.smithi165.stdout: "map_epoch": 107, 2024-04-26T02:05:46.916 INFO:teuthology.orchestra.run.smithi165.stdout: "map_full": 0, 2024-04-26T02:05:46.916 INFO:teuthology.orchestra.run.smithi165.stdout: "map_inc": 17, 2024-04-26T02:05:46.916 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_sessions": 5, 2024-04-26T02:05:46.916 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_open": 6, 2024-04-26T02:05:46.916 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_close": 1, 2024-04-26T02:05:46.916 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_laggy": 0, 2024-04-26T02:05:46.916 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_wr": 0, 2024-04-26T02:05:46.916 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_rd": 2, 2024-04-26T02:05:46.917 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_del": 0 2024-04-26T02:05:46.917 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:46.917 INFO:teuthology.orchestra.run.smithi165.stdout: "oft": { 2024-04-26T02:05:46.917 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_objs": 0, 2024-04-26T02:05:46.917 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_kv_pairs": 0, 2024-04-26T02:05:46.917 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_updates": 0, 2024-04-26T02:05:46.917 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_removes": 0 2024-04-26T02:05:46.917 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:46.917 INFO:teuthology.orchestra.run.smithi165.stdout: "osd-slow-ops": { 2024-04-26T02:05:46.917 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_ops_count": 0 2024-04-26T02:05:46.917 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:46.918 INFO:teuthology.orchestra.run.smithi165.stdout: "purge_queue": { 2024-04-26T02:05:46.918 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops": 0, 2024-04-26T02:05:46.918 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops_high_water": 0, 2024-04-26T02:05:46.918 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing": 0, 2024-04-26T02:05:46.918 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_high_water": 0, 2024-04-26T02:05:46.918 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed_ops": 0, 2024-04-26T02:05:46.918 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed": 0, 2024-04-26T02:05:46.918 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_item_in_journal": 0 2024-04-26T02:05:46.918 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:46.918 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T02:05:46.918 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-26T02:05:46.918 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-04-26T02:05:46.918 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-26T02:05:46.918 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 3085, 2024-04-26T02:05:46.918 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 2346079, 2024-04-26T02:05:46.919 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-26T02:05:46.919 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 3085, 2024-04-26T02:05:46.919 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-04-26T02:05:46.919 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-04-26T02:05:46.919 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 3085, 2024-04-26T02:05:46.919 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 2346079, 2024-04-26T02:05:46.919 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-26T02:05:46.919 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:46.919 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:46.919 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:46.919 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-26T02:05:46.919 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:46.919 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_bytes": { 2024-04-26T02:05:46.919 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-26T02:05:46.919 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-04-26T02:05:46.920 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-26T02:05:46.920 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-04-26T02:05:46.920 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-04-26T02:05:46.920 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-26T02:05:46.920 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-04-26T02:05:46.920 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 2762, 2024-04-26T02:05:46.920 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 27203, 2024-04-26T02:05:46.920 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 2762, 2024-04-26T02:05:46.920 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 27203, 2024-04-26T02:05:46.920 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-26T02:05:46.920 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:46.920 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:46.920 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:46.920 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-26T02:05:46.921 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:46.921 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_ops": { 2024-04-26T02:05:46.921 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-26T02:05:46.921 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 1024, 2024-04-26T02:05:46.921 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-26T02:05:46.921 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-04-26T02:05:46.921 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-04-26T02:05:46.921 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-26T02:05:46.921 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-04-26T02:05:46.921 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 2762, 2024-04-26T02:05:46.921 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 2762, 2024-04-26T02:05:46.921 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 2762, 2024-04-26T02:05:46.922 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 2762, 2024-04-26T02:05:46.922 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-26T02:05:46.922 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:46.922 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:46.922 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:46.922 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-26T02:05:46.922 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:46.922 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle": { 2024-04-26T02:05:46.922 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-26T02:05:46.922 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-04-26T02:05:46.922 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-26T02:05:46.922 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-04-26T02:05:46.922 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-04-26T02:05:46.922 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-26T02:05:46.923 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-04-26T02:05:46.923 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-04-26T02:05:46.923 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-04-26T02:05:46.923 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 0, 2024-04-26T02:05:46.923 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 0, 2024-04-26T02:05:46.923 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-26T02:05:46.923 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:46.923 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:46.923 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:46.923 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-26T02:05:46.923 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:46.923 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle-0x564c746946e0": { 2024-04-26T02:05:46.923 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-26T02:05:46.923 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-04-26T02:05:46.923 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-26T02:05:46.924 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-04-26T02:05:46.924 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-04-26T02:05:46.924 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-26T02:05:46.924 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-04-26T02:05:46.924 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-04-26T02:05:46.924 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-04-26T02:05:46.924 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 0, 2024-04-26T02:05:46.924 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 0, 2024-04-26T02:05:46.924 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-26T02:05:46.924 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:46.924 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:46.924 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:46.924 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-26T02:05:46.924 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-26T02:05:46.924 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-26T02:05:47.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:05:46 smithi172 ceph-mon[32785]: pgmap v381: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 852 B/s wr, 5 op/s 2024-04-26T02:05:47.686 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-04-26T02:05:47.687 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-04-26T02:05:47.687 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-04-26T02:05:47.687 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-04-26T02:05:47.687 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-04-26T02:05:47.687 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-04-26T02:05:47.687 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-04-26T02:05:47.687 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2024-04-26T02:05:47.687 DEBUG:tasks.check_counter:Getting stats from h 2024-04-26T02:05:47.688 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-26T02:05:47.931 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T02:05:48.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:05:48 smithi165 ceph-mon[32648]: pgmap v382: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 852 B/s wr, 5 op/s 2024-04-26T02:05:48.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:05:48 smithi151 ceph-mon[28057]: pgmap v382: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 852 B/s wr, 5 op/s 2024-04-26T02:05:49.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:05:48 smithi172 ceph-mon[32785]: pgmap v382: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 852 B/s wr, 5 op/s 2024-04-26T02:05:49.515 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-26T02:05:49.516 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T02:05:49.516 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 2791, 2024-04-26T02:05:49.516 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 2791, 2024-04-26T02:05:49.516 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 1033480, 2024-04-26T02:05:49.516 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 886721, 2024-04-26T02:05:49.516 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 6, 2024-04-26T02:05:49.516 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 2, 2024-04-26T02:05:49.516 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.191577897, 2024-04-26T02:05:49.516 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.060386312, 2024-04-26T02:05:49.516 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.046847141, 2024-04-26T02:05:49.516 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.069248079, 2024-04-26T02:05:49.516 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:05:49.516 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 2790, 2024-04-26T02:05:49.517 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.078142632, 2024-04-26T02:05:49.517 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000028008 2024-04-26T02:05:49.517 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:49.517 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:05:49.517 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:49.517 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:49.517 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:49.517 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:49.517 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-26T02:05:49.517 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 448 2024-04-26T02:05:49.517 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:49.517 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T02:05:49.517 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 320, 2024-04-26T02:05:49.518 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 293, 2024-04-26T02:05:49.518 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 1832316, 2024-04-26T02:05:49.518 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 170346, 2024-04-26T02:05:49.518 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 3, 2024-04-26T02:05:49.518 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 2, 2024-04-26T02:05:49.518 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.054968089, 2024-04-26T02:05:49.518 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.016226083, 2024-04-26T02:05:49.518 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.042979389, 2024-04-26T02:05:49.518 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.006548535, 2024-04-26T02:05:49.518 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:05:49.518 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 292, 2024-04-26T02:05:49.518 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.013879374, 2024-04-26T02:05:49.518 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000047532 2024-04-26T02:05:49.519 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:49.519 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:05:49.519 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:49.519 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:49.519 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:49.519 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:49.519 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 1831760, 2024-04-26T02:05:49.519 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 169648 2024-04-26T02:05:49.519 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:49.519 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T02:05:49.519 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 3, 2024-04-26T02:05:49.519 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 207, 2024-04-26T02:05:49.519 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 588, 2024-04-26T02:05:49.520 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 593518, 2024-04-26T02:05:49.520 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 8, 2024-04-26T02:05:49.520 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 4, 2024-04-26T02:05:49.520 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.012173703, 2024-04-26T02:05:49.520 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.007533791, 2024-04-26T02:05:49.520 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.002272091, 2024-04-26T02:05:49.520 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.000022642, 2024-04-26T02:05:49.520 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:05:49.520 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 204, 2024-04-26T02:05:49.520 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.020419217, 2024-04-26T02:05:49.520 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000100094 2024-04-26T02:05:49.520 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:49.520 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:05:49.520 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:49.521 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:49.521 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:49.521 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:49.521 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-26T02:05:49.521 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 593200 2024-04-26T02:05:49.521 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:49.521 INFO:teuthology.orchestra.run.smithi165.stdout: "cct": { 2024-04-26T02:05:49.521 INFO:teuthology.orchestra.run.smithi165.stdout: "total_workers": 1, 2024-04-26T02:05:49.521 INFO:teuthology.orchestra.run.smithi165.stdout: "unhealthy_workers": 0 2024-04-26T02:05:49.521 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:49.521 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-MDSRank": { 2024-04-26T02:05:49.521 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-04-26T02:05:49.521 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-04-26T02:05:49.521 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 4695, 2024-04-26T02:05:49.522 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.106581042, 2024-04-26T02:05:49.522 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000022700 2024-04-26T02:05:49.522 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-26T02:05:49.522 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:49.522 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-PurgeQueue": { 2024-04-26T02:05:49.522 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-04-26T02:05:49.522 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-04-26T02:05:49.522 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:49.522 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:49.522 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:49.522 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-26T02:05:49.522 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:49.522 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": { 2024-04-26T02:05:49.523 INFO:teuthology.orchestra.run.smithi165.stdout: "request": 0, 2024-04-26T02:05:49.523 INFO:teuthology.orchestra.run.smithi165.stdout: "reply": 0, 2024-04-26T02:05:49.523 INFO:teuthology.orchestra.run.smithi165.stdout: "reply_latency": { 2024-04-26T02:05:49.523 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:49.523 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:49.523 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:49.523 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:49.523 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_reply": 0, 2024-04-26T02:05:49.523 INFO:teuthology.orchestra.run.smithi165.stdout: "forward": 0, 2024-04-26T02:05:49.523 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_complete": 0, 2024-04-26T02:05:49.523 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_keys": 0, 2024-04-26T02:05:49.523 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_commit": 0, 2024-04-26T02:05:49.523 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_split": 0, 2024-04-26T02:05:49.523 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_merge": 0, 2024-04-26T02:05:49.524 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes": 34, 2024-04-26T02:05:49.524 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_top": 0, 2024-04-26T02:05:49.524 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_bottom": 34, 2024-04-26T02:05:49.524 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pin_tail": 0, 2024-04-26T02:05:49.524 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pinned": 34, 2024-04-26T02:05:49.524 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_expired": 0, 2024-04-26T02:05:49.524 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_with_caps": 0, 2024-04-26T02:05:49.524 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": 0, 2024-04-26T02:05:49.524 INFO:teuthology.orchestra.run.smithi165.stdout: "subtrees": 1, 2024-04-26T02:05:49.524 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse": 0, 2024-04-26T02:05:49.524 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_hit": 0, 2024-04-26T02:05:49.524 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_forward": 0, 2024-04-26T02:05:49.524 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_discover": 0, 2024-04-26T02:05:49.525 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_dir_fetch": 0, 2024-04-26T02:05:49.525 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_remote_ino": 0, 2024-04-26T02:05:49.525 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_lock": 0, 2024-04-26T02:05:49.525 INFO:teuthology.orchestra.run.smithi165.stdout: "load_cent": 0, 2024-04-26T02:05:49.525 INFO:teuthology.orchestra.run.smithi165.stdout: "q": 0, 2024-04-26T02:05:49.525 INFO:teuthology.orchestra.run.smithi165.stdout: "exported": 0, 2024-04-26T02:05:49.525 INFO:teuthology.orchestra.run.smithi165.stdout: "exported_inodes": 0, 2024-04-26T02:05:49.525 INFO:teuthology.orchestra.run.smithi165.stdout: "imported": 0, 2024-04-26T02:05:49.525 INFO:teuthology.orchestra.run.smithi165.stdout: "imported_inodes": 0, 2024-04-26T02:05:49.525 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_dir_fetch": 0, 2024-04-26T02:05:49.525 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_backtrace_fetch": 0, 2024-04-26T02:05:49.525 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_peer_discover": 0, 2024-04-26T02:05:49.525 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rfiles": 0, 2024-04-26T02:05:49.525 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rbytes": 0, 2024-04-26T02:05:49.525 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rsnaps": 0, 2024-04-26T02:05:49.526 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T02:05:49.526 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_set_tag": 0, 2024-04-26T02:05:49.526 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T02:05:49.526 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_inotable_repaired": 0, 2024-04-26T02:05:49.526 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_inodes": 0, 2024-04-26T02:05:49.526 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T02:05:49.526 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T02:05:49.526 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_file_inodes": 0, 2024-04-26T02:05:49.526 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_inode_file_caps": 0, 2024-04-26T02:05:49.526 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_revoke": 0, 2024-04-26T02:05:49.526 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_grant": 0, 2024-04-26T02:05:49.526 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T02:05:49.526 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T02:05:49.526 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T02:05:49.527 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps": 0, 2024-04-26T02:05:49.527 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps_dirty": 0, 2024-04-26T02:05:49.527 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_cap_release": 0, 2024-04-26T02:05:49.527 INFO:teuthology.orchestra.run.smithi165.stdout: "process_request_cap_release": 0 2024-04-26T02:05:49.527 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:49.527 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_cache": { 2024-04-26T02:05:49.527 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update": 0, 2024-04-26T02:05:49.527 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update_receipt": 0, 2024-04-26T02:05:49.527 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_try_discover": 0, 2024-04-26T02:05:49.527 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_send_discover": 0, 2024-04-26T02:05:49.527 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_handle_discover": 0, 2024-04-26T02:05:49.528 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays": 0, 2024-04-26T02:05:49.528 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_delayed": 0, 2024-04-26T02:05:49.528 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_enqueuing": 0, 2024-04-26T02:05:49.528 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_created": 0, 2024-04-26T02:05:49.528 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_enqueued": 0, 2024-04-26T02:05:49.528 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_reintegrated": 0, 2024-04-26T02:05:49.528 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_migrated": 0, 2024-04-26T02:05:49.528 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_processing": 0, 2024-04-26T02:05:49.528 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_enqueued": 0, 2024-04-26T02:05:49.528 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_prioritized": 0, 2024-04-26T02:05:49.528 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_started": 0, 2024-04-26T02:05:49.528 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_completed": 0, 2024-04-26T02:05:49.528 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_quiesce_path": 0, 2024-04-26T02:05:49.529 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_quiesce_inode": 0, 2024-04-26T02:05:49.529 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T02:05:49.529 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_exportdir": 0, 2024-04-26T02:05:49.529 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_flush": 0, 2024-04-26T02:05:49.529 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragmentdir": 0, 2024-04-26T02:05:49.529 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragstats": 0, 2024-04-26T02:05:49.529 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_inodestats": 0 2024-04-26T02:05:49.529 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:49.529 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_log": { 2024-04-26T02:05:49.529 INFO:teuthology.orchestra.run.smithi165.stdout: "evlrg": 0, 2024-04-26T02:05:49.529 INFO:teuthology.orchestra.run.smithi165.stdout: "evadd": 0, 2024-04-26T02:05:49.529 INFO:teuthology.orchestra.run.smithi165.stdout: "evex": 0, 2024-04-26T02:05:49.529 INFO:teuthology.orchestra.run.smithi165.stdout: "evtrm": 0, 2024-04-26T02:05:49.529 INFO:teuthology.orchestra.run.smithi165.stdout: "ev": 109, 2024-04-26T02:05:49.530 INFO:teuthology.orchestra.run.smithi165.stdout: "evexg": 0, 2024-04-26T02:05:49.530 INFO:teuthology.orchestra.run.smithi165.stdout: "evexd": 0, 2024-04-26T02:05:49.530 INFO:teuthology.orchestra.run.smithi165.stdout: "segadd": 0, 2024-04-26T02:05:49.530 INFO:teuthology.orchestra.run.smithi165.stdout: "segex": 0, 2024-04-26T02:05:49.530 INFO:teuthology.orchestra.run.smithi165.stdout: "segtrm": 0, 2024-04-26T02:05:49.530 INFO:teuthology.orchestra.run.smithi165.stdout: "seg": 2, 2024-04-26T02:05:49.530 INFO:teuthology.orchestra.run.smithi165.stdout: "segmjr": 2, 2024-04-26T02:05:49.530 INFO:teuthology.orchestra.run.smithi165.stdout: "segexg": 0, 2024-04-26T02:05:49.530 INFO:teuthology.orchestra.run.smithi165.stdout: "segexd": 0, 2024-04-26T02:05:49.530 INFO:teuthology.orchestra.run.smithi165.stdout: "expos": 4194304, 2024-04-26T02:05:49.530 INFO:teuthology.orchestra.run.smithi165.stdout: "wrpos": 4438137, 2024-04-26T02:05:49.530 INFO:teuthology.orchestra.run.smithi165.stdout: "rdpos": 4433515, 2024-04-26T02:05:49.530 INFO:teuthology.orchestra.run.smithi165.stdout: "jlat": { 2024-04-26T02:05:49.530 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:49.531 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:49.531 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:49.531 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:49.531 INFO:teuthology.orchestra.run.smithi165.stdout: "replayed": 109 2024-04-26T02:05:49.531 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:49.531 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_mem": { 2024-04-26T02:05:49.531 INFO:teuthology.orchestra.run.smithi165.stdout: "ino": 10, 2024-04-26T02:05:49.531 INFO:teuthology.orchestra.run.smithi165.stdout: "ino+": 64, 2024-04-26T02:05:49.531 INFO:teuthology.orchestra.run.smithi165.stdout: "ino-": 54, 2024-04-26T02:05:49.531 INFO:teuthology.orchestra.run.smithi165.stdout: "dir": 6, 2024-04-26T02:05:49.531 INFO:teuthology.orchestra.run.smithi165.stdout: "dir+": 23, 2024-04-26T02:05:49.531 INFO:teuthology.orchestra.run.smithi165.stdout: "dir-": 17, 2024-04-26T02:05:49.531 INFO:teuthology.orchestra.run.smithi165.stdout: "dn": 34, 2024-04-26T02:05:49.531 INFO:teuthology.orchestra.run.smithi165.stdout: "dn+": 91, 2024-04-26T02:05:49.532 INFO:teuthology.orchestra.run.smithi165.stdout: "dn-": 57, 2024-04-26T02:05:49.532 INFO:teuthology.orchestra.run.smithi165.stdout: "cap": 0, 2024-04-26T02:05:49.532 INFO:teuthology.orchestra.run.smithi165.stdout: "cap+": 0, 2024-04-26T02:05:49.532 INFO:teuthology.orchestra.run.smithi165.stdout: "cap-": 0, 2024-04-26T02:05:49.532 INFO:teuthology.orchestra.run.smithi165.stdout: "rss": 51880, 2024-04-26T02:05:49.532 INFO:teuthology.orchestra.run.smithi165.stdout: "heap": 215416 2024-04-26T02:05:49.532 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:49.532 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_server": { 2024-04-26T02:05:49.532 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_client_request": 0, 2024-04-26T02:05:49.532 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_server_request": 0, 2024-04-26T02:05:49.532 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_request": 0, 2024-04-26T02:05:49.532 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_session": 0, 2024-04-26T02:05:49.532 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_peer_request": 0, 2024-04-26T02:05:49.532 INFO:teuthology.orchestra.run.smithi165.stdout: "req_create_latency": { 2024-04-26T02:05:49.533 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:49.533 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:49.533 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:49.533 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:49.533 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getattr_latency": { 2024-04-26T02:05:49.533 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:49.533 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:49.533 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:49.533 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:49.533 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getfilelock_latency": { 2024-04-26T02:05:49.533 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:49.533 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:49.533 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:49.533 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:49.534 INFO:teuthology.orchestra.run.smithi165.stdout: "req_link_latency": { 2024-04-26T02:05:49.534 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:49.534 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:49.534 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:49.534 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:49.534 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookup_latency": { 2024-04-26T02:05:49.534 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:49.534 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:49.534 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:49.534 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:49.534 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookuphash_latency": { 2024-04-26T02:05:49.534 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:49.534 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:49.534 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:49.535 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:49.535 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupino_latency": { 2024-04-26T02:05:49.535 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:49.535 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:49.535 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:49.535 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:49.535 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupname_latency": { 2024-04-26T02:05:49.535 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:49.535 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:49.535 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:49.535 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:49.535 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupparent_latency": { 2024-04-26T02:05:49.535 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:49.535 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:49.536 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:49.536 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:49.536 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupsnap_latency": { 2024-04-26T02:05:49.536 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:49.536 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:49.536 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:49.536 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:49.536 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lssnap_latency": { 2024-04-26T02:05:49.536 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:49.536 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:49.536 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:49.536 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:49.536 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mkdir_latency": { 2024-04-26T02:05:49.536 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:49.537 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:49.537 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:49.538 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:49.538 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mknod_latency": { 2024-04-26T02:05:49.538 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:49.538 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:49.538 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:49.538 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:49.538 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mksnap_latency": { 2024-04-26T02:05:49.538 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:49.539 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:49.539 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:49.539 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:49.539 INFO:teuthology.orchestra.run.smithi165.stdout: "req_open_latency": { 2024-04-26T02:05:49.539 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:49.539 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:49.539 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:49.539 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:49.539 INFO:teuthology.orchestra.run.smithi165.stdout: "req_readdir_latency": { 2024-04-26T02:05:49.539 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:49.539 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:49.539 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:49.539 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:49.540 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rename_latency": { 2024-04-26T02:05:49.540 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:49.540 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:49.540 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:49.540 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:49.540 INFO:teuthology.orchestra.run.smithi165.stdout: "req_renamesnap_latency": { 2024-04-26T02:05:49.540 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:49.540 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:49.540 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:49.540 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:49.540 INFO:teuthology.orchestra.run.smithi165.stdout: "req_snapdiff_latency": { 2024-04-26T02:05:49.540 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:49.540 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:49.541 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:49.541 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:49.541 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmdir_latency": { 2024-04-26T02:05:49.541 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:49.541 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:49.541 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:49.541 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:49.541 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmsnap_latency": { 2024-04-26T02:05:49.541 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:49.541 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:49.541 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:49.541 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:49.541 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmxattr_latency": { 2024-04-26T02:05:49.541 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:49.542 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:49.542 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:49.542 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:49.542 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setattr_latency": { 2024-04-26T02:05:49.542 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:49.542 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:49.542 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:49.542 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:49.542 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setdirlayout_latency": { 2024-04-26T02:05:49.542 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:49.542 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:49.542 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:49.542 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:49.542 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setfilelock_latency": { 2024-04-26T02:05:49.543 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:49.543 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:49.543 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:49.543 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:49.543 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setlayout_latency": { 2024-04-26T02:05:49.543 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:49.543 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:49.543 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:49.543 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:49.543 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setxattr_latency": { 2024-04-26T02:05:49.543 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:49.543 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:49.543 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:49.544 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:49.544 INFO:teuthology.orchestra.run.smithi165.stdout: "req_symlink_latency": { 2024-04-26T02:05:49.544 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:49.544 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:49.544 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:49.544 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:49.544 INFO:teuthology.orchestra.run.smithi165.stdout: "req_unlink_latency": { 2024-04-26T02:05:49.544 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:49.544 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:49.544 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:49.544 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:49.544 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_revoke_eviction": 0, 2024-04-26T02:05:49.544 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_acquisition_throttle": 0, 2024-04-26T02:05:49.544 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getvxattr_latency": { 2024-04-26T02:05:49.545 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:49.545 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:49.545 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:49.545 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-26T02:05:49.545 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:49.545 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_sessions": { 2024-04-26T02:05:49.545 INFO:teuthology.orchestra.run.smithi165.stdout: "session_count": 2, 2024-04-26T02:05:49.545 INFO:teuthology.orchestra.run.smithi165.stdout: "session_add": 4, 2024-04-26T02:05:49.545 INFO:teuthology.orchestra.run.smithi165.stdout: "session_remove": 2, 2024-04-26T02:05:49.545 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_open": 3, 2024-04-26T02:05:49.545 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_stale": 0, 2024-04-26T02:05:49.545 INFO:teuthology.orchestra.run.smithi165.stdout: "total_load": 0, 2024-04-26T02:05:49.545 INFO:teuthology.orchestra.run.smithi165.stdout: "average_load": 0, 2024-04-26T02:05:49.546 INFO:teuthology.orchestra.run.smithi165.stdout: "avg_session_uptime": 3197, 2024-04-26T02:05:49.546 INFO:teuthology.orchestra.run.smithi165.stdout: "mdthresh_evicted": 0 2024-04-26T02:05:49.546 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:49.546 INFO:teuthology.orchestra.run.smithi165.stdout: "mempool": { 2024-04-26T02:05:49.546 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_bytes": 0, 2024-04-26T02:05:49.546 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_items": 0, 2024-04-26T02:05:49.546 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T02:05:49.546 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_items": 0, 2024-04-26T02:05:49.546 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T02:05:49.546 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_items": 0, 2024-04-26T02:05:49.546 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T02:05:49.546 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T02:05:49.546 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T02:05:49.546 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T02:05:49.547 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T02:05:49.547 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_items": 0, 2024-04-26T02:05:49.547 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T02:05:49.547 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T02:05:49.547 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_bytes": 0, 2024-04-26T02:05:49.547 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_items": 0, 2024-04-26T02:05:49.547 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_bytes": 0, 2024-04-26T02:05:49.547 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_items": 0, 2024-04-26T02:05:49.547 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T02:05:49.547 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T02:05:49.547 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T02:05:49.547 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T02:05:49.547 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T02:05:49.548 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_items": 0, 2024-04-26T02:05:49.548 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_bytes": 0, 2024-04-26T02:05:49.548 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_items": 0, 2024-04-26T02:05:49.548 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T02:05:49.548 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T02:05:49.548 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_bytes": 0, 2024-04-26T02:05:49.548 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_items": 0, 2024-04-26T02:05:49.548 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_bytes": 0, 2024-04-26T02:05:49.548 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_items": 0, 2024-04-26T02:05:49.548 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T02:05:49.548 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_items": 0, 2024-04-26T02:05:49.548 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T02:05:49.548 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_items": 0, 2024-04-26T02:05:49.548 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_bytes": 182407, 2024-04-26T02:05:49.549 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_items": 58, 2024-04-26T02:05:49.549 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_bytes": 0, 2024-04-26T02:05:49.549 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_items": 0, 2024-04-26T02:05:49.549 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_bytes": 0, 2024-04-26T02:05:49.549 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_items": 0, 2024-04-26T02:05:49.549 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_bytes": 0, 2024-04-26T02:05:49.549 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_items": 0, 2024-04-26T02:05:49.549 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_bytes": 0, 2024-04-26T02:05:49.549 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_items": 0, 2024-04-26T02:05:49.549 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_bytes": 7720, 2024-04-26T02:05:49.549 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_items": 160, 2024-04-26T02:05:49.549 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T02:05:49.549 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_items": 0, 2024-04-26T02:05:49.549 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_bytes": 0, 2024-04-26T02:05:49.550 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_items": 0, 2024-04-26T02:05:49.550 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_bytes": 61259, 2024-04-26T02:05:49.550 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_items": 818, 2024-04-26T02:05:49.550 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_bytes": 0, 2024-04-26T02:05:49.550 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_items": 0, 2024-04-26T02:05:49.550 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_bytes": 0, 2024-04-26T02:05:49.550 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_items": 0 2024-04-26T02:05:49.550 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:49.550 INFO:teuthology.orchestra.run.smithi165.stdout: "objecter": { 2024-04-26T02:05:49.550 INFO:teuthology.orchestra.run.smithi165.stdout: "op_active": 0, 2024-04-26T02:05:49.550 INFO:teuthology.orchestra.run.smithi165.stdout: "op_laggy": 0, 2024-04-26T02:05:49.550 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send": 2790, 2024-04-26T02:05:49.550 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send_bytes": 21, 2024-04-26T02:05:49.550 INFO:teuthology.orchestra.run.smithi165.stdout: "op_resend": 3, 2024-04-26T02:05:49.551 INFO:teuthology.orchestra.run.smithi165.stdout: "op_reply": 2790, 2024-04-26T02:05:49.551 INFO:teuthology.orchestra.run.smithi165.stdout: "op_latency": { 2024-04-26T02:05:49.551 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 2790, 2024-04-26T02:05:49.551 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 1.360977183, 2024-04-26T02:05:49.551 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000487805 2024-04-26T02:05:49.551 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:49.551 INFO:teuthology.orchestra.run.smithi165.stdout: "op_inflight": 0, 2024-04-26T02:05:49.551 INFO:teuthology.orchestra.run.smithi165.stdout: "oplen_avg": { 2024-04-26T02:05:49.551 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 2790, 2024-04-26T02:05:49.551 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 2791 2024-04-26T02:05:49.551 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:49.551 INFO:teuthology.orchestra.run.smithi165.stdout: "op": 2790, 2024-04-26T02:05:49.551 INFO:teuthology.orchestra.run.smithi165.stdout: "op_r": 2790, 2024-04-26T02:05:49.551 INFO:teuthology.orchestra.run.smithi165.stdout: "op_w": 0, 2024-04-26T02:05:49.552 INFO:teuthology.orchestra.run.smithi165.stdout: "op_rmw": 0, 2024-04-26T02:05:49.552 INFO:teuthology.orchestra.run.smithi165.stdout: "op_pg": 0, 2024-04-26T02:05:49.552 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_stat": 1810, 2024-04-26T02:05:49.552 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_create": 0, 2024-04-26T02:05:49.552 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_read": 978, 2024-04-26T02:05:49.552 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_write": 0, 2024-04-26T02:05:49.552 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writefull": 0, 2024-04-26T02:05:49.552 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writesame": 0, 2024-04-26T02:05:49.552 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_append": 0, 2024-04-26T02:05:49.552 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_zero": 0, 2024-04-26T02:05:49.552 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_truncate": 0, 2024-04-26T02:05:49.552 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_delete": 0, 2024-04-26T02:05:49.552 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_mapext": 0, 2024-04-26T02:05:49.552 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_sparse_read": 0, 2024-04-26T02:05:49.553 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_clonerange": 0, 2024-04-26T02:05:49.553 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_getxattr": 1, 2024-04-26T02:05:49.553 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_setxattr": 0, 2024-04-26T02:05:49.553 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_cmpxattr": 0, 2024-04-26T02:05:49.553 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_rmxattr": 0, 2024-04-26T02:05:49.553 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_resetxattrs": 0, 2024-04-26T02:05:49.553 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_call": 0, 2024-04-26T02:05:49.553 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_watch": 0, 2024-04-26T02:05:49.553 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_notify": 0, 2024-04-26T02:05:49.553 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T02:05:49.553 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls": 0, 2024-04-26T02:05:49.553 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls_filter": 0, 2024-04-26T02:05:49.553 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_other": 0, 2024-04-26T02:05:49.553 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_active": 0, 2024-04-26T02:05:49.554 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_send": 0, 2024-04-26T02:05:49.554 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_resend": 0, 2024-04-26T02:05:49.554 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_ping": 0, 2024-04-26T02:05:49.554 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_active": 0, 2024-04-26T02:05:49.554 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_send": 0, 2024-04-26T02:05:49.554 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_resend": 0, 2024-04-26T02:05:49.554 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_active": 0, 2024-04-26T02:05:49.554 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_send": 0, 2024-04-26T02:05:49.554 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_resend": 0, 2024-04-26T02:05:49.554 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_active": 0, 2024-04-26T02:05:49.554 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_send": 0, 2024-04-26T02:05:49.554 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_resend": 0, 2024-04-26T02:05:49.554 INFO:teuthology.orchestra.run.smithi165.stdout: "command_active": 0, 2024-04-26T02:05:49.554 INFO:teuthology.orchestra.run.smithi165.stdout: "command_send": 0, 2024-04-26T02:05:49.555 INFO:teuthology.orchestra.run.smithi165.stdout: "command_resend": 0, 2024-04-26T02:05:49.555 INFO:teuthology.orchestra.run.smithi165.stdout: "map_epoch": 107, 2024-04-26T02:05:49.555 INFO:teuthology.orchestra.run.smithi165.stdout: "map_full": 0, 2024-04-26T02:05:49.555 INFO:teuthology.orchestra.run.smithi165.stdout: "map_inc": 17, 2024-04-26T02:05:49.555 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_sessions": 4, 2024-04-26T02:05:49.555 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_open": 5, 2024-04-26T02:05:49.555 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_close": 1, 2024-04-26T02:05:49.555 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_laggy": 0, 2024-04-26T02:05:49.555 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_wr": 0, 2024-04-26T02:05:49.555 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_rd": 2, 2024-04-26T02:05:49.555 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_del": 0 2024-04-26T02:05:49.555 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:49.555 INFO:teuthology.orchestra.run.smithi165.stdout: "oft": { 2024-04-26T02:05:49.555 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_objs": 0, 2024-04-26T02:05:49.556 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_kv_pairs": 0, 2024-04-26T02:05:49.556 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_updates": 0, 2024-04-26T02:05:49.556 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_removes": 0 2024-04-26T02:05:49.556 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:49.556 INFO:teuthology.orchestra.run.smithi165.stdout: "osd-slow-ops": { 2024-04-26T02:05:49.556 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_ops_count": 0 2024-04-26T02:05:49.556 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:49.556 INFO:teuthology.orchestra.run.smithi165.stdout: "purge_queue": { 2024-04-26T02:05:49.556 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops": 0, 2024-04-26T02:05:49.556 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops_high_water": 0, 2024-04-26T02:05:49.556 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing": 0, 2024-04-26T02:05:49.556 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_high_water": 0, 2024-04-26T02:05:49.556 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed_ops": 0, 2024-04-26T02:05:49.556 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed": 0, 2024-04-26T02:05:49.556 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_item_in_journal": 0 2024-04-26T02:05:49.557 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:49.557 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T02:05:49.557 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-26T02:05:49.557 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-04-26T02:05:49.557 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-26T02:05:49.557 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 3110, 2024-04-26T02:05:49.557 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 2567219, 2024-04-26T02:05:49.557 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-26T02:05:49.557 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 3110, 2024-04-26T02:05:49.557 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-04-26T02:05:49.557 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-04-26T02:05:49.557 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 3110, 2024-04-26T02:05:49.557 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 2567219, 2024-04-26T02:05:49.558 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-26T02:05:49.558 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:49.558 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:49.558 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:49.558 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-26T02:05:49.558 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:49.558 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_bytes": { 2024-04-26T02:05:49.558 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-26T02:05:49.558 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-04-26T02:05:49.558 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-26T02:05:49.558 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-04-26T02:05:49.558 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-04-26T02:05:49.558 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-26T02:05:49.558 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-04-26T02:05:49.559 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 2790, 2024-04-26T02:05:49.559 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 243838, 2024-04-26T02:05:49.559 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 2790, 2024-04-26T02:05:49.559 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 243838, 2024-04-26T02:05:49.559 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-26T02:05:49.559 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:49.559 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:49.559 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:49.559 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-26T02:05:49.559 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:49.559 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_ops": { 2024-04-26T02:05:49.559 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-26T02:05:49.559 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 1024, 2024-04-26T02:05:49.560 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-26T02:05:49.560 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-04-26T02:05:49.560 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-04-26T02:05:49.560 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-26T02:05:49.560 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-04-26T02:05:49.560 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 2790, 2024-04-26T02:05:49.560 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 2790, 2024-04-26T02:05:49.560 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 2790, 2024-04-26T02:05:49.560 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 2790, 2024-04-26T02:05:49.560 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-26T02:05:49.560 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:49.560 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:49.560 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:49.560 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-26T02:05:49.561 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:49.561 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle": { 2024-04-26T02:05:49.561 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-26T02:05:49.561 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-04-26T02:05:49.561 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-26T02:05:49.561 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-04-26T02:05:49.561 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-04-26T02:05:49.561 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-26T02:05:49.561 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-04-26T02:05:49.561 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-04-26T02:05:49.561 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-04-26T02:05:49.561 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 0, 2024-04-26T02:05:49.561 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 0, 2024-04-26T02:05:49.561 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-26T02:05:49.562 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:49.562 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:49.562 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:49.562 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-26T02:05:49.562 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:49.562 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle-0x556daa6166e0": { 2024-04-26T02:05:49.562 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-26T02:05:49.562 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-04-26T02:05:49.562 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-26T02:05:49.562 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-04-26T02:05:49.562 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-04-26T02:05:49.562 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-26T02:05:49.562 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-04-26T02:05:49.562 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-04-26T02:05:49.563 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-04-26T02:05:49.563 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 0, 2024-04-26T02:05:49.563 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 0, 2024-04-26T02:05:49.563 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-26T02:05:49.563 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:49.563 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:49.563 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:49.563 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-26T02:05:49.563 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-26T02:05:49.563 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-26T02:05:50.314 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-04-26T02:05:50.315 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-04-26T02:05:50.315 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-04-26T02:05:50.315 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=0 2024-04-26T02:05:50.315 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=0 2024-04-26T02:05:50.315 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-04-26T02:05:50.315 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-04-26T02:05:50.315 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2024-04-26T02:05:50.316 DEBUG:tasks.check_counter:Getting stats from k 2024-04-26T02:05:50.316 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-26T02:05:50.558 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T02:05:50.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:05:50 smithi165 ceph-mon[32648]: pgmap v383: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 852 B/s wr, 7 op/s 2024-04-26T02:05:50.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:05:50 smithi151 ceph-mon[28057]: pgmap v383: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 852 B/s wr, 7 op/s 2024-04-26T02:05:51.037 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:05:50 smithi172 ceph-mon[32785]: pgmap v383: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 852 B/s wr, 7 op/s 2024-04-26T02:05:52.182 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-26T02:05:52.182 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T02:05:52.183 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 0, 2024-04-26T02:05:52.183 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 2, 2024-04-26T02:05:52.183 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 0, 2024-04-26T02:05:52.183 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 0, 2024-04-26T02:05:52.183 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 4, 2024-04-26T02:05:52.183 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 0, 2024-04-26T02:05:52.183 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.001132748, 2024-04-26T02:05:52.183 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.000000000, 2024-04-26T02:05:52.183 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.000288284, 2024-04-26T02:05:52.183 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T02:05:52.183 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:05:52.183 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:52.184 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:52.184 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:52.184 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:52.184 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:05:52.184 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:52.184 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:52.184 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:52.184 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:52.184 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T02:05:52.184 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T02:05:52.184 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:52.184 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T02:05:52.184 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 2, 2024-04-26T02:05:52.185 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 210, 2024-04-26T02:05:52.185 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 320, 2024-04-26T02:05:52.185 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 193168, 2024-04-26T02:05:52.185 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 5, 2024-04-26T02:05:52.185 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 3, 2024-04-26T02:05:52.185 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.012855337, 2024-04-26T02:05:52.185 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.009047993, 2024-04-26T02:05:52.185 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.001339168, 2024-04-26T02:05:52.185 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T02:05:52.185 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:05:52.185 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 207, 2024-04-26T02:05:52.185 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.022038705, 2024-04-26T02:05:52.186 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000106467 2024-04-26T02:05:52.186 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:52.186 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:05:52.186 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:52.186 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:52.186 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:52.186 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:52.186 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-26T02:05:52.186 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 193168 2024-04-26T02:05:52.186 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:52.186 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T02:05:52.186 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 291, 2024-04-26T02:05:52.187 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 277, 2024-04-26T02:05:52.187 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 1749952, 2024-04-26T02:05:52.187 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 170112, 2024-04-26T02:05:52.187 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 2, 2024-04-26T02:05:52.187 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 1, 2024-04-26T02:05:52.187 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.056753973, 2024-04-26T02:05:52.187 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.016918296, 2024-04-26T02:05:52.187 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.043820332, 2024-04-26T02:05:52.187 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.006347502, 2024-04-26T02:05:52.187 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:05:52.187 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 277, 2024-04-26T02:05:52.187 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.013566130, 2024-04-26T02:05:52.187 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000048975 2024-04-26T02:05:52.187 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:52.188 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:05:52.188 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:52.188 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:52.188 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:52.188 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:52.188 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 1749952, 2024-04-26T02:05:52.188 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 170112 2024-04-26T02:05:52.188 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:52.188 INFO:teuthology.orchestra.run.smithi165.stdout: "cct": { 2024-04-26T02:05:52.188 INFO:teuthology.orchestra.run.smithi165.stdout: "total_workers": 0, 2024-04-26T02:05:52.188 INFO:teuthology.orchestra.run.smithi165.stdout: "unhealthy_workers": 0 2024-04-26T02:05:52.188 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:52.188 INFO:teuthology.orchestra.run.smithi165.stdout: "mempool": { 2024-04-26T02:05:52.188 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_bytes": 0, 2024-04-26T02:05:52.189 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_items": 0, 2024-04-26T02:05:52.189 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T02:05:52.189 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_items": 0, 2024-04-26T02:05:52.189 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T02:05:52.189 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_items": 0, 2024-04-26T02:05:52.189 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T02:05:52.189 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T02:05:52.189 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T02:05:52.189 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T02:05:52.189 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T02:05:52.189 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_items": 0, 2024-04-26T02:05:52.189 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T02:05:52.189 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T02:05:52.190 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_bytes": 0, 2024-04-26T02:05:52.190 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_items": 0, 2024-04-26T02:05:52.190 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_bytes": 0, 2024-04-26T02:05:52.190 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_items": 0, 2024-04-26T02:05:52.190 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T02:05:52.190 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T02:05:52.190 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T02:05:52.190 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T02:05:52.190 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T02:05:52.190 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_items": 0, 2024-04-26T02:05:52.190 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_bytes": 0, 2024-04-26T02:05:52.190 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_items": 0, 2024-04-26T02:05:52.190 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T02:05:52.190 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T02:05:52.191 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_bytes": 0, 2024-04-26T02:05:52.191 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_items": 0, 2024-04-26T02:05:52.191 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_bytes": 0, 2024-04-26T02:05:52.191 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_items": 0, 2024-04-26T02:05:52.191 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T02:05:52.191 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_items": 0, 2024-04-26T02:05:52.191 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T02:05:52.191 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_items": 0, 2024-04-26T02:05:52.191 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_bytes": 148732, 2024-04-26T02:05:52.191 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_items": 31, 2024-04-26T02:05:52.191 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_bytes": 0, 2024-04-26T02:05:52.191 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_items": 0, 2024-04-26T02:05:52.191 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_bytes": 0, 2024-04-26T02:05:52.192 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_items": 0, 2024-04-26T02:05:52.192 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_bytes": 0, 2024-04-26T02:05:52.192 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_items": 0, 2024-04-26T02:05:52.192 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_bytes": 0, 2024-04-26T02:05:52.192 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_items": 0, 2024-04-26T02:05:52.192 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_bytes": 0, 2024-04-26T02:05:52.192 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_items": 0, 2024-04-26T02:05:52.192 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T02:05:52.192 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_items": 0, 2024-04-26T02:05:52.192 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_bytes": 0, 2024-04-26T02:05:52.192 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_items": 0, 2024-04-26T02:05:52.192 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_bytes": 928, 2024-04-26T02:05:52.192 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_items": 2, 2024-04-26T02:05:52.192 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_bytes": 0, 2024-04-26T02:05:52.193 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_items": 0, 2024-04-26T02:05:52.193 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_bytes": 0, 2024-04-26T02:05:52.193 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_items": 0 2024-04-26T02:05:52.193 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T02:05:52.193 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T02:05:52.193 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-26T02:05:52.193 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-04-26T02:05:52.193 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-26T02:05:52.193 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 289, 2024-04-26T02:05:52.193 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 1706400, 2024-04-26T02:05:52.193 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-26T02:05:52.193 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 289, 2024-04-26T02:05:52.193 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-04-26T02:05:52.193 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-04-26T02:05:52.193 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 289, 2024-04-26T02:05:52.194 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 1706400, 2024-04-26T02:05:52.194 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-26T02:05:52.194 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T02:05:52.194 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T02:05:52.194 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T02:05:52.194 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-26T02:05:52.194 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-26T02:05:52.194 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-26T02:05:52.797 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:05:52 smithi165 ceph-mon[32648]: pgmap v384: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 852 B/s wr, 5 op/s 2024-04-26T02:05:52.826 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-04-26T02:05:52.826 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-04-26T02:05:52.826 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2024-04-26T02:05:52.826 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2024-04-26T02:05:52.826 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-04-26T02:05:52.826 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-04-26T02:05:52.826 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-04-26T02:05:52.826 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.k 2024-04-26T02:05:52.826 DEBUG:tasks.check_counter:Getting stats from c 2024-04-26T02:05:52.827 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-26T02:05:52.853 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:05:52 smithi172 ceph-mon[32785]: pgmap v384: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 852 B/s wr, 5 op/s 2024-04-26T02:05:52.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:05:52 smithi151 ceph-mon[28057]: pgmap v384: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 852 B/s wr, 5 op/s 2024-04-26T02:05:53.077 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T02:05:53.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:05:53 smithi172 ceph-mon[32785]: pgmap v385: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 852 B/s wr, 7 op/s 2024-04-26T02:05:53.846 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:05:53 smithi165 ceph-mon[32648]: pgmap v385: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 852 B/s wr, 7 op/s 2024-04-26T02:05:53.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:05:53 smithi151 ceph-mon[28057]: pgmap v385: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 852 B/s wr, 7 op/s 2024-04-26T02:05:54.911 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-26T02:05:54.911 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T02:05:54.911 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 62, 2024-04-26T02:05:54.911 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 81, 2024-04-26T02:05:54.911 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 16856, 2024-04-26T02:05:54.911 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 97900, 2024-04-26T02:05:54.911 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 16, 2024-04-26T02:05:54.911 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 5, 2024-04-26T02:05:54.911 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 0.011566387, 2024-04-26T02:05:54.911 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.001999943, 2024-04-26T02:05:54.912 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.006335361, 2024-04-26T02:05:54.912 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.001128941, 2024-04-26T02:05:54.912 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:05:54.912 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 78, 2024-04-26T02:05:54.912 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.031810773, 2024-04-26T02:05:54.912 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000407830 2024-04-26T02:05:54.912 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:54.912 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:05:54.912 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 52, 2024-04-26T02:05:54.912 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000009343, 2024-04-26T02:05:54.912 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000179 2024-04-26T02:05:54.912 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:54.912 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 6512, 2024-04-26T02:05:54.913 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 8208 2024-04-26T02:05:54.913 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:54.913 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T02:05:54.913 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 220, 2024-04-26T02:05:54.913 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 444, 2024-04-26T02:05:54.913 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 64367, 2024-04-26T02:05:54.913 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 663374, 2024-04-26T02:05:54.913 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 13, 2024-04-26T02:05:54.913 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 6, 2024-04-26T02:05:54.913 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 0.037126963, 2024-04-26T02:05:54.913 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.015645902, 2024-04-26T02:05:54.913 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.021850533, 2024-04-26T02:05:54.913 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.000619141, 2024-04-26T02:05:54.913 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:05:54.914 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 441, 2024-04-26T02:05:54.914 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.038951794, 2024-04-26T02:05:54.914 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000088326 2024-04-26T02:05:54.914 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:54.914 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:05:54.914 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 399, 2024-04-26T02:05:54.914 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000081769, 2024-04-26T02:05:54.914 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000204 2024-04-26T02:05:54.914 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:54.914 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 480, 2024-04-26T02:05:54.914 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 596960 2024-04-26T02:05:54.914 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:54.914 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T02:05:54.915 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 6899, 2024-04-26T02:05:54.915 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 4211, 2024-04-26T02:05:54.915 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 7315381, 2024-04-26T02:05:54.915 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 1178876, 2024-04-26T02:05:54.915 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 8, 2024-04-26T02:05:54.915 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 4, 2024-04-26T02:05:54.915 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 0.356064132, 2024-04-26T02:05:54.915 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.136011248, 2024-04-26T02:05:54.915 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.287435721, 2024-04-26T02:05:54.915 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.012354390, 2024-04-26T02:05:54.915 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:05:54.915 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 4212, 2024-04-26T02:05:54.915 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.098836887, 2024-04-26T02:05:54.916 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000023465 2024-04-26T02:05:54.916 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:54.916 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:05:54.916 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 10206, 2024-04-26T02:05:54.916 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.001728018, 2024-04-26T02:05:54.916 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000169 2024-04-26T02:05:54.916 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:54.916 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 1844224, 2024-04-26T02:05:54.916 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 175584 2024-04-26T02:05:54.916 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:54.916 INFO:teuthology.orchestra.run.smithi172.stdout: "cct": { 2024-04-26T02:05:54.916 INFO:teuthology.orchestra.run.smithi172.stdout: "total_workers": 1, 2024-04-26T02:05:54.916 INFO:teuthology.orchestra.run.smithi172.stdout: "unhealthy_workers": 0 2024-04-26T02:05:54.916 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:54.917 INFO:teuthology.orchestra.run.smithi172.stdout: "finisher-MDSRank": { 2024-04-26T02:05:54.917 INFO:teuthology.orchestra.run.smithi172.stdout: "queue_len": 0, 2024-04-26T02:05:54.917 INFO:teuthology.orchestra.run.smithi172.stdout: "complete_latency": { 2024-04-26T02:05:54.917 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 87, 2024-04-26T02:05:54.917 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.005287952, 2024-04-26T02:05:54.917 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000060781 2024-04-26T02:05:54.917 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-26T02:05:54.917 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:54.917 INFO:teuthology.orchestra.run.smithi172.stdout: "finisher-PurgeQueue": { 2024-04-26T02:05:54.917 INFO:teuthology.orchestra.run.smithi172.stdout: "queue_len": 0, 2024-04-26T02:05:54.917 INFO:teuthology.orchestra.run.smithi172.stdout: "complete_latency": { 2024-04-26T02:05:54.917 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 15, 2024-04-26T02:05:54.917 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000285952, 2024-04-26T02:05:54.918 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000019063 2024-04-26T02:05:54.918 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-26T02:05:54.918 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:54.918 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": { 2024-04-26T02:05:54.918 INFO:teuthology.orchestra.run.smithi172.stdout: "request": 181, 2024-04-26T02:05:54.918 INFO:teuthology.orchestra.run.smithi172.stdout: "reply": 3, 2024-04-26T02:05:54.918 INFO:teuthology.orchestra.run.smithi172.stdout: "reply_latency": { 2024-04-26T02:05:54.918 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 3, 2024-04-26T02:05:54.918 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.002513311, 2024-04-26T02:05:54.918 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000837770 2024-04-26T02:05:54.918 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:54.918 INFO:teuthology.orchestra.run.smithi172.stdout: "slow_reply": 0, 2024-04-26T02:05:54.918 INFO:teuthology.orchestra.run.smithi172.stdout: "forward": 178, 2024-04-26T02:05:54.918 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_fetch_complete": 0, 2024-04-26T02:05:54.919 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_fetch_keys": 0, 2024-04-26T02:05:54.919 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_commit": 11, 2024-04-26T02:05:54.919 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_split": 0, 2024-04-26T02:05:54.919 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_merge": 0, 2024-04-26T02:05:54.919 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes": 21, 2024-04-26T02:05:54.919 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_top": 4, 2024-04-26T02:05:54.919 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_bottom": 7, 2024-04-26T02:05:54.919 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_pin_tail": 10, 2024-04-26T02:05:54.919 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_pinned": 13, 2024-04-26T02:05:54.919 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_expired": 117, 2024-04-26T02:05:54.919 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_with_caps": 0, 2024-04-26T02:05:54.919 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": 0, 2024-04-26T02:05:54.919 INFO:teuthology.orchestra.run.smithi172.stdout: "subtrees": 4, 2024-04-26T02:05:54.919 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse": 469, 2024-04-26T02:05:54.920 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_hit": 234, 2024-04-26T02:05:54.920 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_forward": 177, 2024-04-26T02:05:54.920 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_discover": 54, 2024-04-26T02:05:54.920 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_dir_fetch": 0, 2024-04-26T02:05:54.920 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_remote_ino": 0, 2024-04-26T02:05:54.920 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_lock": 0, 2024-04-26T02:05:54.920 INFO:teuthology.orchestra.run.smithi172.stdout: "load_cent": 0, 2024-04-26T02:05:54.920 INFO:teuthology.orchestra.run.smithi172.stdout: "q": 0, 2024-04-26T02:05:54.920 INFO:teuthology.orchestra.run.smithi172.stdout: "exported": 1, 2024-04-26T02:05:54.920 INFO:teuthology.orchestra.run.smithi172.stdout: "exported_inodes": 2, 2024-04-26T02:05:54.920 INFO:teuthology.orchestra.run.smithi172.stdout: "imported": 1, 2024-04-26T02:05:54.920 INFO:teuthology.orchestra.run.smithi172.stdout: "imported_inodes": 2, 2024-04-26T02:05:54.920 INFO:teuthology.orchestra.run.smithi172.stdout: "openino_dir_fetch": 0, 2024-04-26T02:05:54.920 INFO:teuthology.orchestra.run.smithi172.stdout: "openino_backtrace_fetch": 0, 2024-04-26T02:05:54.920 INFO:teuthology.orchestra.run.smithi172.stdout: "openino_peer_discover": 0, 2024-04-26T02:05:54.921 INFO:teuthology.orchestra.run.smithi172.stdout: "root_rfiles": 4, 2024-04-26T02:05:54.921 INFO:teuthology.orchestra.run.smithi172.stdout: "root_rbytes": 234, 2024-04-26T02:05:54.921 INFO:teuthology.orchestra.run.smithi172.stdout: "root_rsnaps": 7, 2024-04-26T02:05:54.921 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T02:05:54.921 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_set_tag": 0, 2024-04-26T02:05:54.921 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T02:05:54.921 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_inotable_repaired": 0, 2024-04-26T02:05:54.921 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_dir_inodes": 0, 2024-04-26T02:05:54.921 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T02:05:54.921 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T02:05:54.921 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_file_inodes": 0, 2024-04-26T02:05:54.921 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_inode_file_caps": 0, 2024-04-26T02:05:54.921 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_revoke": 2, 2024-04-26T02:05:54.922 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_grant": 2, 2024-04-26T02:05:54.922 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T02:05:54.922 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T02:05:54.922 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T02:05:54.922 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_caps": 3, 2024-04-26T02:05:54.922 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_caps_dirty": 1, 2024-04-26T02:05:54.922 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_cap_release": 2, 2024-04-26T02:05:54.922 INFO:teuthology.orchestra.run.smithi172.stdout: "process_request_cap_release": 3 2024-04-26T02:05:54.922 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:54.922 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_cache": { 2024-04-26T02:05:54.922 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_update": 1, 2024-04-26T02:05:54.922 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_update_receipt": 468, 2024-04-26T02:05:54.922 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_try_discover": 260, 2024-04-26T02:05:54.922 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_send_discover": 268, 2024-04-26T02:05:54.923 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_handle_discover": 1, 2024-04-26T02:05:54.923 INFO:teuthology.orchestra.run.smithi172.stdout: "num_strays": 0, 2024-04-26T02:05:54.923 INFO:teuthology.orchestra.run.smithi172.stdout: "num_strays_delayed": 0, 2024-04-26T02:05:54.923 INFO:teuthology.orchestra.run.smithi172.stdout: "num_strays_enqueuing": 0, 2024-04-26T02:05:54.923 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_created": 1, 2024-04-26T02:05:54.923 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_enqueued": 1, 2024-04-26T02:05:54.923 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_reintegrated": 0, 2024-04-26T02:05:54.923 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_migrated": 0, 2024-04-26T02:05:54.923 INFO:teuthology.orchestra.run.smithi172.stdout: "num_recovering_processing": 0, 2024-04-26T02:05:54.923 INFO:teuthology.orchestra.run.smithi172.stdout: "num_recovering_enqueued": 0, 2024-04-26T02:05:54.923 INFO:teuthology.orchestra.run.smithi172.stdout: "num_recovering_prioritized": 0, 2024-04-26T02:05:54.923 INFO:teuthology.orchestra.run.smithi172.stdout: "recovery_started": 0, 2024-04-26T02:05:54.923 INFO:teuthology.orchestra.run.smithi172.stdout: "recovery_completed": 0, 2024-04-26T02:05:54.923 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_quiesce_path": 8, 2024-04-26T02:05:54.923 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_quiesce_inode": 307, 2024-04-26T02:05:54.923 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T02:05:54.924 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_exportdir": 1, 2024-04-26T02:05:54.924 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_flush": 0, 2024-04-26T02:05:54.924 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_fragmentdir": 0, 2024-04-26T02:05:54.924 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_fragstats": 0, 2024-04-26T02:05:54.924 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_inodestats": 0 2024-04-26T02:05:54.924 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:54.924 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_log": { 2024-04-26T02:05:54.924 INFO:teuthology.orchestra.run.smithi172.stdout: "evlrg": 0, 2024-04-26T02:05:54.924 INFO:teuthology.orchestra.run.smithi172.stdout: "evadd": 17, 2024-04-26T02:05:54.924 INFO:teuthology.orchestra.run.smithi172.stdout: "evex": 0, 2024-04-26T02:05:54.924 INFO:teuthology.orchestra.run.smithi172.stdout: "evtrm": 0, 2024-04-26T02:05:54.924 INFO:teuthology.orchestra.run.smithi172.stdout: "ev": 17, 2024-04-26T02:05:54.924 INFO:teuthology.orchestra.run.smithi172.stdout: "evexg": 0, 2024-04-26T02:05:54.925 INFO:teuthology.orchestra.run.smithi172.stdout: "evexd": 0, 2024-04-26T02:05:54.925 INFO:teuthology.orchestra.run.smithi172.stdout: "segadd": 2, 2024-04-26T02:05:54.925 INFO:teuthology.orchestra.run.smithi172.stdout: "segex": 0, 2024-04-26T02:05:54.925 INFO:teuthology.orchestra.run.smithi172.stdout: "segtrm": 0, 2024-04-26T02:05:54.925 INFO:teuthology.orchestra.run.smithi172.stdout: "seg": 2, 2024-04-26T02:05:54.925 INFO:teuthology.orchestra.run.smithi172.stdout: "segmjr": 2, 2024-04-26T02:05:54.925 INFO:teuthology.orchestra.run.smithi172.stdout: "segexg": 0, 2024-04-26T02:05:54.925 INFO:teuthology.orchestra.run.smithi172.stdout: "segexd": 0, 2024-04-26T02:05:54.925 INFO:teuthology.orchestra.run.smithi172.stdout: "expos": 4194304, 2024-04-26T02:05:54.925 INFO:teuthology.orchestra.run.smithi172.stdout: "wrpos": 4221502, 2024-04-26T02:05:54.925 INFO:teuthology.orchestra.run.smithi172.stdout: "rdpos": 0, 2024-04-26T02:05:54.925 INFO:teuthology.orchestra.run.smithi172.stdout: "jlat": { 2024-04-26T02:05:54.925 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 15, 2024-04-26T02:05:54.925 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.032379019, 2024-04-26T02:05:54.926 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.002158601 2024-04-26T02:05:54.926 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:54.926 INFO:teuthology.orchestra.run.smithi172.stdout: "replayed": 0 2024-04-26T02:05:54.926 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:54.926 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_mem": { 2024-04-26T02:05:54.926 INFO:teuthology.orchestra.run.smithi172.stdout: "ino": 26, 2024-04-26T02:05:54.926 INFO:teuthology.orchestra.run.smithi172.stdout: "ino+": 87, 2024-04-26T02:05:54.926 INFO:teuthology.orchestra.run.smithi172.stdout: "ino-": 61, 2024-04-26T02:05:54.926 INFO:teuthology.orchestra.run.smithi172.stdout: "dir": 25, 2024-04-26T02:05:54.926 INFO:teuthology.orchestra.run.smithi172.stdout: "dir+": 239, 2024-04-26T02:05:54.926 INFO:teuthology.orchestra.run.smithi172.stdout: "dir-": 214, 2024-04-26T02:05:54.926 INFO:teuthology.orchestra.run.smithi172.stdout: "dn": 21, 2024-04-26T02:05:54.926 INFO:teuthology.orchestra.run.smithi172.stdout: "dn+": 140, 2024-04-26T02:05:54.927 INFO:teuthology.orchestra.run.smithi172.stdout: "dn-": 119, 2024-04-26T02:05:54.927 INFO:teuthology.orchestra.run.smithi172.stdout: "cap": 0, 2024-04-26T02:05:54.927 INFO:teuthology.orchestra.run.smithi172.stdout: "cap+": 4, 2024-04-26T02:05:54.927 INFO:teuthology.orchestra.run.smithi172.stdout: "cap-": 4, 2024-04-26T02:05:54.927 INFO:teuthology.orchestra.run.smithi172.stdout: "rss": 54896, 2024-04-26T02:05:54.927 INFO:teuthology.orchestra.run.smithi172.stdout: "heap": 223608 2024-04-26T02:05:54.927 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:54.927 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_server": { 2024-04-26T02:05:54.927 INFO:teuthology.orchestra.run.smithi172.stdout: "dispatch_client_request": 185, 2024-04-26T02:05:54.927 INFO:teuthology.orchestra.run.smithi172.stdout: "dispatch_server_request": 6, 2024-04-26T02:05:54.927 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_request": 181, 2024-04-26T02:05:54.927 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_session": 23, 2024-04-26T02:05:54.927 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_peer_request": 314, 2024-04-26T02:05:54.928 INFO:teuthology.orchestra.run.smithi172.stdout: "req_create_latency": { 2024-04-26T02:05:54.928 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:54.928 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:54.928 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:54.928 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:54.928 INFO:teuthology.orchestra.run.smithi172.stdout: "req_getattr_latency": { 2024-04-26T02:05:54.928 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 1, 2024-04-26T02:05:54.928 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000162962, 2024-04-26T02:05:54.928 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000162962 2024-04-26T02:05:54.928 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:54.928 INFO:teuthology.orchestra.run.smithi172.stdout: "req_getfilelock_latency": { 2024-04-26T02:05:54.928 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:54.928 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:54.929 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:54.929 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:54.929 INFO:teuthology.orchestra.run.smithi172.stdout: "req_link_latency": { 2024-04-26T02:05:54.929 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:54.929 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:54.929 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:54.929 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:54.929 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookup_latency": { 2024-04-26T02:05:54.929 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:54.929 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:54.929 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:54.929 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:54.929 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookuphash_latency": { 2024-04-26T02:05:54.930 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:54.930 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:54.930 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:54.930 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:54.930 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupino_latency": { 2024-04-26T02:05:54.930 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:54.930 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:54.930 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:54.930 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:54.930 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupname_latency": { 2024-04-26T02:05:54.930 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:54.930 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:54.930 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:54.930 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:54.931 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupparent_latency": { 2024-04-26T02:05:54.931 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:54.931 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:54.931 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:54.931 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:54.931 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupsnap_latency": { 2024-04-26T02:05:54.931 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:54.931 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:54.931 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:54.931 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:54.931 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lssnap_latency": { 2024-04-26T02:05:54.931 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:54.931 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:54.931 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:54.932 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:54.932 INFO:teuthology.orchestra.run.smithi172.stdout: "req_mkdir_latency": { 2024-04-26T02:05:54.932 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:54.932 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:54.932 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:54.932 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:54.933 INFO:teuthology.orchestra.run.smithi172.stdout: "req_mknod_latency": { 2024-04-26T02:05:54.933 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:54.933 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:54.933 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:54.933 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:54.933 INFO:teuthology.orchestra.run.smithi172.stdout: "req_mksnap_latency": { 2024-04-26T02:05:54.933 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:54.933 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:54.933 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:54.933 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:54.933 INFO:teuthology.orchestra.run.smithi172.stdout: "req_open_latency": { 2024-04-26T02:05:54.933 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:54.933 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:54.933 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:54.934 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:54.934 INFO:teuthology.orchestra.run.smithi172.stdout: "req_readdir_latency": { 2024-04-26T02:05:54.934 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 1, 2024-04-26T02:05:54.934 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000338181, 2024-04-26T02:05:54.934 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000338181 2024-04-26T02:05:54.934 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:54.934 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rename_latency": { 2024-04-26T02:05:54.934 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:54.934 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:54.934 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:54.934 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:54.934 INFO:teuthology.orchestra.run.smithi172.stdout: "req_renamesnap_latency": { 2024-04-26T02:05:54.934 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:54.934 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:54.935 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:54.935 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:54.935 INFO:teuthology.orchestra.run.smithi172.stdout: "req_snapdiff_latency": { 2024-04-26T02:05:54.935 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:54.935 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:54.935 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:54.935 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:54.935 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rmdir_latency": { 2024-04-26T02:05:54.935 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:54.935 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:54.935 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:54.935 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:54.935 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rmsnap_latency": { 2024-04-26T02:05:54.935 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:54.936 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:54.936 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:54.936 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:54.936 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rmxattr_latency": { 2024-04-26T02:05:54.936 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:54.936 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:54.936 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:54.936 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:54.936 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setattr_latency": { 2024-04-26T02:05:54.936 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:54.936 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:54.936 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:54.936 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:54.936 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setdirlayout_latency": { 2024-04-26T02:05:54.937 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:54.937 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:54.937 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:54.937 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:54.937 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setfilelock_latency": { 2024-04-26T02:05:54.937 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:54.937 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:54.937 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:54.937 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:54.937 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setlayout_latency": { 2024-04-26T02:05:54.937 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:54.937 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:54.937 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:54.937 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:54.938 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setxattr_latency": { 2024-04-26T02:05:54.938 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:54.938 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:54.938 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:54.938 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:54.938 INFO:teuthology.orchestra.run.smithi172.stdout: "req_symlink_latency": { 2024-04-26T02:05:54.938 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:54.938 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:54.938 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:54.938 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:54.938 INFO:teuthology.orchestra.run.smithi172.stdout: "req_unlink_latency": { 2024-04-26T02:05:54.938 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 1, 2024-04-26T02:05:54.938 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.002012168, 2024-04-26T02:05:54.938 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.002012168 2024-04-26T02:05:54.939 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:54.939 INFO:teuthology.orchestra.run.smithi172.stdout: "cap_revoke_eviction": 0, 2024-04-26T02:05:54.939 INFO:teuthology.orchestra.run.smithi172.stdout: "cap_acquisition_throttle": 0, 2024-04-26T02:05:54.939 INFO:teuthology.orchestra.run.smithi172.stdout: "req_getvxattr_latency": { 2024-04-26T02:05:54.939 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:54.939 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:54.939 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:54.939 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-26T02:05:54.939 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:54.939 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_sessions": { 2024-04-26T02:05:54.939 INFO:teuthology.orchestra.run.smithi172.stdout: "session_count": 2, 2024-04-26T02:05:54.939 INFO:teuthology.orchestra.run.smithi172.stdout: "session_add": 3, 2024-04-26T02:05:54.939 INFO:teuthology.orchestra.run.smithi172.stdout: "session_remove": 1, 2024-04-26T02:05:54.939 INFO:teuthology.orchestra.run.smithi172.stdout: "sessions_open": 2, 2024-04-26T02:05:54.940 INFO:teuthology.orchestra.run.smithi172.stdout: "sessions_stale": 0, 2024-04-26T02:05:54.940 INFO:teuthology.orchestra.run.smithi172.stdout: "total_load": 1, 2024-04-26T02:05:54.940 INFO:teuthology.orchestra.run.smithi172.stdout: "average_load": 0, 2024-04-26T02:05:54.940 INFO:teuthology.orchestra.run.smithi172.stdout: "avg_session_uptime": 671, 2024-04-26T02:05:54.940 INFO:teuthology.orchestra.run.smithi172.stdout: "mdthresh_evicted": 0 2024-04-26T02:05:54.940 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:54.940 INFO:teuthology.orchestra.run.smithi172.stdout: "mempool": { 2024-04-26T02:05:54.940 INFO:teuthology.orchestra.run.smithi172.stdout: "bloom_filter_bytes": 0, 2024-04-26T02:05:54.940 INFO:teuthology.orchestra.run.smithi172.stdout: "bloom_filter_items": 0, 2024-04-26T02:05:54.940 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T02:05:54.940 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_alloc_items": 0, 2024-04-26T02:05:54.940 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T02:05:54.940 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_data_items": 0, 2024-04-26T02:05:54.941 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T02:05:54.941 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T02:05:54.941 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T02:05:54.941 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T02:05:54.941 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T02:05:54.941 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_other_items": 0, 2024-04-26T02:05:54.941 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T02:05:54.941 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T02:05:54.941 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_extent_bytes": 0, 2024-04-26T02:05:54.941 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_extent_items": 0, 2024-04-26T02:05:54.941 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_blob_bytes": 0, 2024-04-26T02:05:54.941 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_blob_items": 0, 2024-04-26T02:05:54.941 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T02:05:54.942 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T02:05:54.942 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T02:05:54.942 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T02:05:54.942 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T02:05:54.942 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_fsck_items": 0, 2024-04-26T02:05:54.942 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_txc_bytes": 0, 2024-04-26T02:05:54.942 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_txc_items": 0, 2024-04-26T02:05:54.942 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T02:05:54.942 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T02:05:54.942 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_bytes": 0, 2024-04-26T02:05:54.942 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_items": 0, 2024-04-26T02:05:54.942 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_bytes": 0, 2024-04-26T02:05:54.942 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_items": 0, 2024-04-26T02:05:54.943 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T02:05:54.943 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_reader_items": 0, 2024-04-26T02:05:54.943 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T02:05:54.943 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_writer_items": 0, 2024-04-26T02:05:54.943 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_anon_bytes": 255191, 2024-04-26T02:05:54.943 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_anon_items": 85, 2024-04-26T02:05:54.943 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_meta_bytes": 0, 2024-04-26T02:05:54.943 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_meta_items": 0, 2024-04-26T02:05:54.943 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_bytes": 0, 2024-04-26T02:05:54.943 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_items": 0, 2024-04-26T02:05:54.943 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_mapbl_bytes": 0, 2024-04-26T02:05:54.943 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_mapbl_items": 0, 2024-04-26T02:05:54.943 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_pglog_bytes": 0, 2024-04-26T02:05:54.944 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_pglog_items": 0, 2024-04-26T02:05:54.944 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_bytes": 7720, 2024-04-26T02:05:54.944 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_items": 160, 2024-04-26T02:05:54.944 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T02:05:54.944 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_mapping_items": 0, 2024-04-26T02:05:54.944 INFO:teuthology.orchestra.run.smithi172.stdout: "pgmap_bytes": 0, 2024-04-26T02:05:54.944 INFO:teuthology.orchestra.run.smithi172.stdout: "pgmap_items": 0, 2024-04-26T02:05:54.944 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_co_bytes": 113069, 2024-04-26T02:05:54.944 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_co_items": 427, 2024-04-26T02:05:54.944 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_1_bytes": 0, 2024-04-26T02:05:54.945 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_1_items": 0, 2024-04-26T02:05:54.945 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_2_bytes": 0, 2024-04-26T02:05:54.945 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_2_items": 0 2024-04-26T02:05:54.945 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:54.945 INFO:teuthology.orchestra.run.smithi172.stdout: "objecter": { 2024-04-26T02:05:54.945 INFO:teuthology.orchestra.run.smithi172.stdout: "op_active": 0, 2024-04-26T02:05:54.945 INFO:teuthology.orchestra.run.smithi172.stdout: "op_laggy": 0, 2024-04-26T02:05:54.945 INFO:teuthology.orchestra.run.smithi172.stdout: "op_send": 70, 2024-04-26T02:05:54.945 INFO:teuthology.orchestra.run.smithi172.stdout: "op_send_bytes": 39480, 2024-04-26T02:05:54.945 INFO:teuthology.orchestra.run.smithi172.stdout: "op_resend": 27, 2024-04-26T02:05:54.945 INFO:teuthology.orchestra.run.smithi172.stdout: "op_reply": 70, 2024-04-26T02:05:54.945 INFO:teuthology.orchestra.run.smithi172.stdout: "op_latency": { 2024-04-26T02:05:54.945 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 70, 2024-04-26T02:05:54.946 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.573985879, 2024-04-26T02:05:54.946 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.008199798 2024-04-26T02:05:54.946 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:54.946 INFO:teuthology.orchestra.run.smithi172.stdout: "op_inflight": 0, 2024-04-26T02:05:54.946 INFO:teuthology.orchestra.run.smithi172.stdout: "oplen_avg": { 2024-04-26T02:05:54.946 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 70, 2024-04-26T02:05:54.946 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 97 2024-04-26T02:05:54.946 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:54.946 INFO:teuthology.orchestra.run.smithi172.stdout: "op": 70, 2024-04-26T02:05:54.946 INFO:teuthology.orchestra.run.smithi172.stdout: "op_r": 1, 2024-04-26T02:05:54.946 INFO:teuthology.orchestra.run.smithi172.stdout: "op_w": 69, 2024-04-26T02:05:54.946 INFO:teuthology.orchestra.run.smithi172.stdout: "op_rmw": 0, 2024-04-26T02:05:54.947 INFO:teuthology.orchestra.run.smithi172.stdout: "op_pg": 0, 2024-04-26T02:05:54.947 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_stat": 0, 2024-04-26T02:05:54.947 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_create": 10, 2024-04-26T02:05:54.947 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_read": 1, 2024-04-26T02:05:54.947 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_write": 16, 2024-04-26T02:05:54.947 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_writefull": 12, 2024-04-26T02:05:54.947 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_writesame": 0, 2024-04-26T02:05:54.947 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_append": 0, 2024-04-26T02:05:54.947 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_zero": 0, 2024-04-26T02:05:54.947 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_truncate": 0, 2024-04-26T02:05:54.947 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_delete": 13, 2024-04-26T02:05:54.947 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_mapext": 0, 2024-04-26T02:05:54.947 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_sparse_read": 0, 2024-04-26T02:05:54.948 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_clonerange": 0, 2024-04-26T02:05:54.948 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_getxattr": 0, 2024-04-26T02:05:54.948 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_setxattr": 20, 2024-04-26T02:05:54.948 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_cmpxattr": 0, 2024-04-26T02:05:54.948 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_rmxattr": 0, 2024-04-26T02:05:54.948 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_resetxattrs": 0, 2024-04-26T02:05:54.948 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_call": 0, 2024-04-26T02:05:54.948 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_watch": 0, 2024-04-26T02:05:54.948 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_notify": 0, 2024-04-26T02:05:54.948 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T02:05:54.948 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_pgls": 0, 2024-04-26T02:05:54.948 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_pgls_filter": 0, 2024-04-26T02:05:54.948 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_other": 0, 2024-04-26T02:05:54.948 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_active": 0, 2024-04-26T02:05:54.949 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_send": 0, 2024-04-26T02:05:54.949 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_resend": 0, 2024-04-26T02:05:54.949 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_ping": 0, 2024-04-26T02:05:54.949 INFO:teuthology.orchestra.run.smithi172.stdout: "poolop_active": 0, 2024-04-26T02:05:54.949 INFO:teuthology.orchestra.run.smithi172.stdout: "poolop_send": 0, 2024-04-26T02:05:54.949 INFO:teuthology.orchestra.run.smithi172.stdout: "poolop_resend": 0, 2024-04-26T02:05:54.949 INFO:teuthology.orchestra.run.smithi172.stdout: "poolstat_active": 0, 2024-04-26T02:05:54.949 INFO:teuthology.orchestra.run.smithi172.stdout: "poolstat_send": 0, 2024-04-26T02:05:54.949 INFO:teuthology.orchestra.run.smithi172.stdout: "poolstat_resend": 0, 2024-04-26T02:05:54.949 INFO:teuthology.orchestra.run.smithi172.stdout: "statfs_active": 0, 2024-04-26T02:05:54.949 INFO:teuthology.orchestra.run.smithi172.stdout: "statfs_send": 0, 2024-04-26T02:05:54.949 INFO:teuthology.orchestra.run.smithi172.stdout: "statfs_resend": 0, 2024-04-26T02:05:54.949 INFO:teuthology.orchestra.run.smithi172.stdout: "command_active": 0, 2024-04-26T02:05:54.949 INFO:teuthology.orchestra.run.smithi172.stdout: "command_send": 0, 2024-04-26T02:05:54.949 INFO:teuthology.orchestra.run.smithi172.stdout: "command_resend": 0, 2024-04-26T02:05:54.950 INFO:teuthology.orchestra.run.smithi172.stdout: "map_epoch": 107, 2024-04-26T02:05:54.950 INFO:teuthology.orchestra.run.smithi172.stdout: "map_full": 0, 2024-04-26T02:05:54.950 INFO:teuthology.orchestra.run.smithi172.stdout: "map_inc": 17, 2024-04-26T02:05:54.950 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_sessions": 9, 2024-04-26T02:05:54.950 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_session_open": 13, 2024-04-26T02:05:54.950 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_session_close": 4, 2024-04-26T02:05:54.950 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_laggy": 0, 2024-04-26T02:05:54.950 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_wr": 21, 2024-04-26T02:05:54.950 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_rd": 0, 2024-04-26T02:05:54.950 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_del": 4 2024-04-26T02:05:54.950 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:54.950 INFO:teuthology.orchestra.run.smithi172.stdout: "oft": { 2024-04-26T02:05:54.950 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_objs": 1, 2024-04-26T02:05:54.951 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_kv_pairs": 0, 2024-04-26T02:05:54.951 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_updates": 0, 2024-04-26T02:05:54.951 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_removes": 0 2024-04-26T02:05:54.951 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:54.951 INFO:teuthology.orchestra.run.smithi172.stdout: "osd-slow-ops": { 2024-04-26T02:05:54.951 INFO:teuthology.orchestra.run.smithi172.stdout: "slow_ops_count": 315 2024-04-26T02:05:54.951 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:54.951 INFO:teuthology.orchestra.run.smithi172.stdout: "purge_queue": { 2024-04-26T02:05:54.951 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing_ops": 0, 2024-04-26T02:05:54.951 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing_ops_high_water": 1, 2024-04-26T02:05:54.951 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing": 0, 2024-04-26T02:05:54.951 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing_high_water": 1, 2024-04-26T02:05:54.951 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executed_ops": 1, 2024-04-26T02:05:54.952 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executed": 1, 2024-04-26T02:05:54.952 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_item_in_journal": 0 2024-04-26T02:05:54.952 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:54.952 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T02:05:54.952 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-04-26T02:05:54.952 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 104857600, 2024-04-26T02:05:54.952 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-04-26T02:05:54.952 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 7177, 2024-04-26T02:05:54.952 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 6729273, 2024-04-26T02:05:54.952 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-04-26T02:05:54.952 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 7177, 2024-04-26T02:05:54.952 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 0, 2024-04-26T02:05:54.953 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 0, 2024-04-26T02:05:54.953 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 7177, 2024-04-26T02:05:54.953 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 6729273, 2024-04-26T02:05:54.953 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-04-26T02:05:54.953 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:54.953 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:54.953 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:54.953 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-26T02:05:54.953 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:54.953 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-objecter_bytes": { 2024-04-26T02:05:54.953 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-04-26T02:05:54.953 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 104857600, 2024-04-26T02:05:54.953 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-04-26T02:05:54.953 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 0, 2024-04-26T02:05:54.954 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 0, 2024-04-26T02:05:54.954 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-04-26T02:05:54.954 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 0, 2024-04-26T02:05:54.954 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 70, 2024-04-26T02:05:54.954 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 39637, 2024-04-26T02:05:54.954 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 70, 2024-04-26T02:05:54.954 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 39637, 2024-04-26T02:05:54.954 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-04-26T02:05:54.954 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:54.954 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:54.954 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:54.954 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-26T02:05:54.954 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:54.955 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-objecter_ops": { 2024-04-26T02:05:54.955 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-04-26T02:05:54.955 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 1024, 2024-04-26T02:05:54.955 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-04-26T02:05:54.955 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 0, 2024-04-26T02:05:54.955 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 0, 2024-04-26T02:05:54.955 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-04-26T02:05:54.955 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 0, 2024-04-26T02:05:54.955 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 70, 2024-04-26T02:05:54.955 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 70, 2024-04-26T02:05:54.955 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 70, 2024-04-26T02:05:54.955 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 70, 2024-04-26T02:05:54.955 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-04-26T02:05:54.955 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:54.956 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:54.956 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:54.956 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-26T02:05:54.956 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:54.956 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-write_buf_throttle": { 2024-04-26T02:05:54.956 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-04-26T02:05:54.956 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 3758096384, 2024-04-26T02:05:54.956 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-04-26T02:05:54.956 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 1, 2024-04-26T02:05:54.956 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 157, 2024-04-26T02:05:54.956 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-04-26T02:05:54.956 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 1, 2024-04-26T02:05:54.956 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 0, 2024-04-26T02:05:54.956 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 0, 2024-04-26T02:05:54.956 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 1, 2024-04-26T02:05:54.957 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 157, 2024-04-26T02:05:54.957 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-04-26T02:05:54.957 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:54.957 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:54.957 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:54.957 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-26T02:05:54.957 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:54.957 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-write_buf_throttle-0x563f32046500": { 2024-04-26T02:05:54.957 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-04-26T02:05:54.957 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 3758096384, 2024-04-26T02:05:54.957 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-04-26T02:05:54.957 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 17, 2024-04-26T02:05:54.957 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 27198, 2024-04-26T02:05:54.957 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-04-26T02:05:54.957 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 17, 2024-04-26T02:05:54.958 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 0, 2024-04-26T02:05:54.958 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 0, 2024-04-26T02:05:54.958 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 15, 2024-04-26T02:05:54.958 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 27198, 2024-04-26T02:05:54.958 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-04-26T02:05:54.958 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:54.958 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:54.958 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:54.958 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-26T02:05:54.958 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-26T02:05:54.958 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-26T02:05:55.690 INFO:tasks.check_counter:Daemon mds.c mds.exported=1 2024-04-26T02:05:55.690 INFO:tasks.check_counter:Daemon mds.c mds.imported=1 2024-04-26T02:05:55.690 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=1 2024-04-26T02:05:55.690 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=468 2024-04-26T02:05:55.690 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=7 2024-04-26T02:05:55.690 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-04-26T02:05:55.690 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-04-26T02:05:55.690 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-04-26T02:05:55.690 DEBUG:tasks.check_counter:Getting stats from f 2024-04-26T02:05:55.691 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-26T02:05:55.983 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T02:05:56.280 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:05:56 smithi172 ceph-mon[32785]: pgmap v386: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T02:05:56.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:05:56 smithi165 ceph-mon[32648]: pgmap v386: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T02:05:56.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:05:56 smithi151 ceph-mon[28057]: pgmap v386: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T02:05:57.694 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-26T02:05:57.694 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T02:05:57.694 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 154, 2024-04-26T02:05:57.694 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 154, 2024-04-26T02:05:57.694 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 8449845, 2024-04-26T02:05:57.694 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 48158, 2024-04-26T02:05:57.694 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 7, 2024-04-26T02:05:57.694 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 3, 2024-04-26T02:05:57.694 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 0.023508298, 2024-04-26T02:05:57.695 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.004877840, 2024-04-26T02:05:57.695 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.011305963, 2024-04-26T02:05:57.695 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.005102172, 2024-04-26T02:05:57.695 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:05:57.695 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 153, 2024-04-26T02:05:57.695 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.008083298, 2024-04-26T02:05:57.695 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000052832 2024-04-26T02:05:57.695 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:57.695 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:05:57.695 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:57.695 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:57.695 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:57.695 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:57.696 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-26T02:05:57.696 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 448 2024-04-26T02:05:57.696 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:57.696 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T02:05:57.696 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 126, 2024-04-26T02:05:57.696 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 146, 2024-04-26T02:05:57.696 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 16771763, 2024-04-26T02:05:57.696 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 158271, 2024-04-26T02:05:57.696 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 6, 2024-04-26T02:05:57.696 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 5, 2024-04-26T02:05:57.696 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 0.018429774, 2024-04-26T02:05:57.696 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.003433545, 2024-04-26T02:05:57.696 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.011436917, 2024-04-26T02:05:57.697 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.002600359, 2024-04-26T02:05:57.697 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:05:57.697 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 144, 2024-04-26T02:05:57.697 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.009016048, 2024-04-26T02:05:57.697 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000062611 2024-04-26T02:05:57.697 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:57.697 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:05:57.697 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:57.697 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:57.697 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:57.697 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:57.697 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-26T02:05:57.697 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 118464 2024-04-26T02:05:57.697 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:57.698 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T02:05:57.698 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 116, 2024-04-26T02:05:57.698 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 113, 2024-04-26T02:05:57.698 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 12685296, 2024-04-26T02:05:57.698 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 41820, 2024-04-26T02:05:57.698 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 4, 2024-04-26T02:05:57.698 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 3, 2024-04-26T02:05:57.698 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 0.019091798, 2024-04-26T02:05:57.698 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.003802559, 2024-04-26T02:05:57.698 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.012804014, 2024-04-26T02:05:57.698 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.002502351, 2024-04-26T02:05:57.698 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:05:57.699 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 112, 2024-04-26T02:05:57.699 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.021841796, 2024-04-26T02:05:57.699 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000195016 2024-04-26T02:05:57.699 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:57.699 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:05:57.699 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:57.699 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:57.699 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:57.699 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:57.699 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 171504, 2024-04-26T02:05:57.699 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 16688 2024-04-26T02:05:57.699 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:57.699 INFO:teuthology.orchestra.run.smithi172.stdout: "cct": { 2024-04-26T02:05:57.700 INFO:teuthology.orchestra.run.smithi172.stdout: "total_workers": 1, 2024-04-26T02:05:57.700 INFO:teuthology.orchestra.run.smithi172.stdout: "unhealthy_workers": 0 2024-04-26T02:05:57.700 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:57.700 INFO:teuthology.orchestra.run.smithi172.stdout: "finisher-MDSRank": { 2024-04-26T02:05:57.700 INFO:teuthology.orchestra.run.smithi172.stdout: "queue_len": 0, 2024-04-26T02:05:57.700 INFO:teuthology.orchestra.run.smithi172.stdout: "complete_latency": { 2024-04-26T02:05:57.700 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 561, 2024-04-26T02:05:57.700 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.058609962, 2024-04-26T02:05:57.700 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000104474 2024-04-26T02:05:57.700 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-26T02:05:57.700 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:57.700 INFO:teuthology.orchestra.run.smithi172.stdout: "finisher-PurgeQueue": { 2024-04-26T02:05:57.700 INFO:teuthology.orchestra.run.smithi172.stdout: "queue_len": 0, 2024-04-26T02:05:57.701 INFO:teuthology.orchestra.run.smithi172.stdout: "complete_latency": { 2024-04-26T02:05:57.701 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:57.701 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:57.701 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:57.701 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-26T02:05:57.701 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:57.701 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": { 2024-04-26T02:05:57.701 INFO:teuthology.orchestra.run.smithi172.stdout: "request": 0, 2024-04-26T02:05:57.701 INFO:teuthology.orchestra.run.smithi172.stdout: "reply": 0, 2024-04-26T02:05:57.701 INFO:teuthology.orchestra.run.smithi172.stdout: "reply_latency": { 2024-04-26T02:05:57.701 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:57.701 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:57.701 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:57.702 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:57.702 INFO:teuthology.orchestra.run.smithi172.stdout: "slow_reply": 0, 2024-04-26T02:05:57.702 INFO:teuthology.orchestra.run.smithi172.stdout: "forward": 0, 2024-04-26T02:05:57.702 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_fetch_complete": 0, 2024-04-26T02:05:57.702 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_fetch_keys": 0, 2024-04-26T02:05:57.702 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_commit": 0, 2024-04-26T02:05:57.702 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_split": 0, 2024-04-26T02:05:57.702 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_merge": 0, 2024-04-26T02:05:57.702 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes": 13873, 2024-04-26T02:05:57.702 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_top": 7279, 2024-04-26T02:05:57.702 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_bottom": 6594, 2024-04-26T02:05:57.702 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_pin_tail": 0, 2024-04-26T02:05:57.702 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_pinned": 63, 2024-04-26T02:05:57.703 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_expired": 0, 2024-04-26T02:05:57.703 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_with_caps": 0, 2024-04-26T02:05:57.703 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": 0, 2024-04-26T02:05:57.703 INFO:teuthology.orchestra.run.smithi172.stdout: "subtrees": 2, 2024-04-26T02:05:57.703 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse": 0, 2024-04-26T02:05:57.703 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_hit": 0, 2024-04-26T02:05:57.703 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_forward": 0, 2024-04-26T02:05:57.703 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_discover": 0, 2024-04-26T02:05:57.703 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_dir_fetch": 0, 2024-04-26T02:05:57.703 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_remote_ino": 0, 2024-04-26T02:05:57.703 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_lock": 0, 2024-04-26T02:05:57.703 INFO:teuthology.orchestra.run.smithi172.stdout: "load_cent": 0, 2024-04-26T02:05:57.703 INFO:teuthology.orchestra.run.smithi172.stdout: "q": 0, 2024-04-26T02:05:57.703 INFO:teuthology.orchestra.run.smithi172.stdout: "exported": 0, 2024-04-26T02:05:57.704 INFO:teuthology.orchestra.run.smithi172.stdout: "exported_inodes": 0, 2024-04-26T02:05:57.704 INFO:teuthology.orchestra.run.smithi172.stdout: "imported": 0, 2024-04-26T02:05:57.704 INFO:teuthology.orchestra.run.smithi172.stdout: "imported_inodes": 0, 2024-04-26T02:05:57.704 INFO:teuthology.orchestra.run.smithi172.stdout: "openino_dir_fetch": 0, 2024-04-26T02:05:57.704 INFO:teuthology.orchestra.run.smithi172.stdout: "openino_backtrace_fetch": 0, 2024-04-26T02:05:57.704 INFO:teuthology.orchestra.run.smithi172.stdout: "openino_peer_discover": 0, 2024-04-26T02:05:57.704 INFO:teuthology.orchestra.run.smithi172.stdout: "root_rfiles": 4, 2024-04-26T02:05:57.704 INFO:teuthology.orchestra.run.smithi172.stdout: "root_rbytes": 234, 2024-04-26T02:05:57.704 INFO:teuthology.orchestra.run.smithi172.stdout: "root_rsnaps": 7, 2024-04-26T02:05:57.704 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T02:05:57.704 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_set_tag": 0, 2024-04-26T02:05:57.704 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T02:05:57.704 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_inotable_repaired": 0, 2024-04-26T02:05:57.705 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_dir_inodes": 0, 2024-04-26T02:05:57.705 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T02:05:57.705 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T02:05:57.705 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_file_inodes": 0, 2024-04-26T02:05:57.705 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_inode_file_caps": 0, 2024-04-26T02:05:57.705 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_revoke": 0, 2024-04-26T02:05:57.705 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_grant": 0, 2024-04-26T02:05:57.705 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T02:05:57.705 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T02:05:57.705 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T02:05:57.705 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_caps": 0, 2024-04-26T02:05:57.705 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_caps_dirty": 0, 2024-04-26T02:05:57.705 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_cap_release": 0, 2024-04-26T02:05:57.706 INFO:teuthology.orchestra.run.smithi172.stdout: "process_request_cap_release": 0 2024-04-26T02:05:57.706 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:57.706 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_cache": { 2024-04-26T02:05:57.706 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_update": 0, 2024-04-26T02:05:57.706 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_update_receipt": 0, 2024-04-26T02:05:57.706 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_try_discover": 0, 2024-04-26T02:05:57.706 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_send_discover": 0, 2024-04-26T02:05:57.706 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_handle_discover": 0, 2024-04-26T02:05:57.706 INFO:teuthology.orchestra.run.smithi172.stdout: "num_strays": 0, 2024-04-26T02:05:57.706 INFO:teuthology.orchestra.run.smithi172.stdout: "num_strays_delayed": 0, 2024-04-26T02:05:57.706 INFO:teuthology.orchestra.run.smithi172.stdout: "num_strays_enqueuing": 0, 2024-04-26T02:05:57.706 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_created": 0, 2024-04-26T02:05:57.706 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_enqueued": 0, 2024-04-26T02:05:57.706 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_reintegrated": 0, 2024-04-26T02:05:57.707 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_migrated": 0, 2024-04-26T02:05:57.707 INFO:teuthology.orchestra.run.smithi172.stdout: "num_recovering_processing": 0, 2024-04-26T02:05:57.707 INFO:teuthology.orchestra.run.smithi172.stdout: "num_recovering_enqueued": 0, 2024-04-26T02:05:57.707 INFO:teuthology.orchestra.run.smithi172.stdout: "num_recovering_prioritized": 0, 2024-04-26T02:05:57.707 INFO:teuthology.orchestra.run.smithi172.stdout: "recovery_started": 0, 2024-04-26T02:05:57.707 INFO:teuthology.orchestra.run.smithi172.stdout: "recovery_completed": 0, 2024-04-26T02:05:57.707 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_quiesce_path": 0, 2024-04-26T02:05:57.707 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_quiesce_inode": 0, 2024-04-26T02:05:57.707 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T02:05:57.707 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_exportdir": 0, 2024-04-26T02:05:57.707 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_flush": 0, 2024-04-26T02:05:57.707 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_fragmentdir": 0, 2024-04-26T02:05:57.707 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_fragstats": 0, 2024-04-26T02:05:57.708 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_inodestats": 0 2024-04-26T02:05:57.708 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:57.708 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_log": { 2024-04-26T02:05:57.708 INFO:teuthology.orchestra.run.smithi172.stdout: "evlrg": 0, 2024-04-26T02:05:57.708 INFO:teuthology.orchestra.run.smithi172.stdout: "evadd": 0, 2024-04-26T02:05:57.708 INFO:teuthology.orchestra.run.smithi172.stdout: "evex": 0, 2024-04-26T02:05:57.708 INFO:teuthology.orchestra.run.smithi172.stdout: "evtrm": 0, 2024-04-26T02:05:57.708 INFO:teuthology.orchestra.run.smithi172.stdout: "ev": 10657, 2024-04-26T02:05:57.708 INFO:teuthology.orchestra.run.smithi172.stdout: "evexg": 0, 2024-04-26T02:05:57.708 INFO:teuthology.orchestra.run.smithi172.stdout: "evexd": 0, 2024-04-26T02:05:57.708 INFO:teuthology.orchestra.run.smithi172.stdout: "segadd": 0, 2024-04-26T02:05:57.708 INFO:teuthology.orchestra.run.smithi172.stdout: "segex": 0, 2024-04-26T02:05:57.708 INFO:teuthology.orchestra.run.smithi172.stdout: "segtrm": 0, 2024-04-26T02:05:57.708 INFO:teuthology.orchestra.run.smithi172.stdout: "seg": 2, 2024-04-26T02:05:57.708 INFO:teuthology.orchestra.run.smithi172.stdout: "segmjr": 5, 2024-04-26T02:05:57.709 INFO:teuthology.orchestra.run.smithi172.stdout: "segexg": 0, 2024-04-26T02:05:57.709 INFO:teuthology.orchestra.run.smithi172.stdout: "segexd": 0, 2024-04-26T02:05:57.709 INFO:teuthology.orchestra.run.smithi172.stdout: "expos": 180193356, 2024-04-26T02:05:57.709 INFO:teuthology.orchestra.run.smithi172.stdout: "wrpos": 180264419, 2024-04-26T02:05:57.709 INFO:teuthology.orchestra.run.smithi172.stdout: "rdpos": 180255556, 2024-04-26T02:05:57.709 INFO:teuthology.orchestra.run.smithi172.stdout: "jlat": { 2024-04-26T02:05:57.709 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:57.709 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:57.709 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:57.709 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:57.709 INFO:teuthology.orchestra.run.smithi172.stdout: "replayed": 10657 2024-04-26T02:05:57.709 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:57.709 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_mem": { 2024-04-26T02:05:57.709 INFO:teuthology.orchestra.run.smithi172.stdout: "ino": 4644, 2024-04-26T02:05:57.709 INFO:teuthology.orchestra.run.smithi172.stdout: "ino+": 9193, 2024-04-26T02:05:57.710 INFO:teuthology.orchestra.run.smithi172.stdout: "ino-": 4549, 2024-04-26T02:05:57.710 INFO:teuthology.orchestra.run.smithi172.stdout: "dir": 219, 2024-04-26T02:05:57.710 INFO:teuthology.orchestra.run.smithi172.stdout: "dir+": 219, 2024-04-26T02:05:57.710 INFO:teuthology.orchestra.run.smithi172.stdout: "dir-": 0, 2024-04-26T02:05:57.710 INFO:teuthology.orchestra.run.smithi172.stdout: "dn": 13873, 2024-04-26T02:05:57.710 INFO:teuthology.orchestra.run.smithi172.stdout: "dn+": 13873, 2024-04-26T02:05:57.710 INFO:teuthology.orchestra.run.smithi172.stdout: "dn-": 0, 2024-04-26T02:05:57.710 INFO:teuthology.orchestra.run.smithi172.stdout: "cap": 0, 2024-04-26T02:05:57.710 INFO:teuthology.orchestra.run.smithi172.stdout: "cap+": 0, 2024-04-26T02:05:57.710 INFO:teuthology.orchestra.run.smithi172.stdout: "cap-": 0, 2024-04-26T02:05:57.710 INFO:teuthology.orchestra.run.smithi172.stdout: "rss": 72892, 2024-04-26T02:05:57.710 INFO:teuthology.orchestra.run.smithi172.stdout: "heap": 215392 2024-04-26T02:05:57.710 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:57.711 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_server": { 2024-04-26T02:05:57.711 INFO:teuthology.orchestra.run.smithi172.stdout: "dispatch_client_request": 0, 2024-04-26T02:05:57.711 INFO:teuthology.orchestra.run.smithi172.stdout: "dispatch_server_request": 0, 2024-04-26T02:05:57.711 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_request": 0, 2024-04-26T02:05:57.711 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_session": 0, 2024-04-26T02:05:57.711 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_peer_request": 0, 2024-04-26T02:05:57.711 INFO:teuthology.orchestra.run.smithi172.stdout: "req_create_latency": { 2024-04-26T02:05:57.711 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:57.711 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:57.711 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:57.711 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:57.711 INFO:teuthology.orchestra.run.smithi172.stdout: "req_getattr_latency": { 2024-04-26T02:05:57.711 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:57.711 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:57.711 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:57.712 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:57.712 INFO:teuthology.orchestra.run.smithi172.stdout: "req_getfilelock_latency": { 2024-04-26T02:05:57.712 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:57.712 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:57.712 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:57.712 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:57.712 INFO:teuthology.orchestra.run.smithi172.stdout: "req_link_latency": { 2024-04-26T02:05:57.712 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:57.712 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:57.712 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:57.712 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:57.712 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookup_latency": { 2024-04-26T02:05:57.712 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:57.712 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:57.712 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:57.713 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:57.713 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookuphash_latency": { 2024-04-26T02:05:57.713 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:57.713 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:57.713 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:57.713 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:57.713 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupino_latency": { 2024-04-26T02:05:57.713 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:57.713 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:57.713 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:57.713 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:57.713 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupname_latency": { 2024-04-26T02:05:57.713 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:57.713 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:57.714 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:57.714 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:57.714 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupparent_latency": { 2024-04-26T02:05:57.714 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:57.714 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:57.714 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:57.714 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:57.714 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupsnap_latency": { 2024-04-26T02:05:57.714 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:57.714 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:57.714 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:57.714 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:57.714 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lssnap_latency": { 2024-04-26T02:05:57.714 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:57.715 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:57.715 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:57.715 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:57.715 INFO:teuthology.orchestra.run.smithi172.stdout: "req_mkdir_latency": { 2024-04-26T02:05:57.715 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:57.715 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:57.715 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:57.716 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:57.716 INFO:teuthology.orchestra.run.smithi172.stdout: "req_mknod_latency": { 2024-04-26T02:05:57.716 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:57.716 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:57.716 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:57.716 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:57.716 INFO:teuthology.orchestra.run.smithi172.stdout: "req_mksnap_latency": { 2024-04-26T02:05:57.716 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:57.716 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:57.716 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:57.716 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:57.716 INFO:teuthology.orchestra.run.smithi172.stdout: "req_open_latency": { 2024-04-26T02:05:57.716 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:57.716 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:57.716 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:57.717 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:57.717 INFO:teuthology.orchestra.run.smithi172.stdout: "req_readdir_latency": { 2024-04-26T02:05:57.717 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:57.717 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:57.717 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:57.717 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:57.717 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rename_latency": { 2024-04-26T02:05:57.717 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:57.717 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:57.717 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:57.717 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:57.717 INFO:teuthology.orchestra.run.smithi172.stdout: "req_renamesnap_latency": { 2024-04-26T02:05:57.717 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:57.717 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:57.718 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:57.718 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:57.718 INFO:teuthology.orchestra.run.smithi172.stdout: "req_snapdiff_latency": { 2024-04-26T02:05:57.718 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:57.718 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:57.718 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:57.718 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:57.718 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rmdir_latency": { 2024-04-26T02:05:57.718 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:57.718 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:57.718 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:57.718 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:57.718 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rmsnap_latency": { 2024-04-26T02:05:57.719 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:57.719 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:57.719 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:57.719 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:57.719 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rmxattr_latency": { 2024-04-26T02:05:57.719 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:57.719 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:57.719 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:57.719 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:57.719 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setattr_latency": { 2024-04-26T02:05:57.719 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:57.719 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:57.719 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:57.719 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:57.719 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setdirlayout_latency": { 2024-04-26T02:05:57.719 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:57.720 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:57.720 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:57.720 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:57.720 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setfilelock_latency": { 2024-04-26T02:05:57.720 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:57.720 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:57.720 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:57.720 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:57.720 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setlayout_latency": { 2024-04-26T02:05:57.720 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:57.720 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:57.720 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:57.720 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:57.720 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setxattr_latency": { 2024-04-26T02:05:57.720 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:57.721 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:57.721 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:57.721 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:57.721 INFO:teuthology.orchestra.run.smithi172.stdout: "req_symlink_latency": { 2024-04-26T02:05:57.721 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:57.721 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:57.721 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:57.721 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:57.721 INFO:teuthology.orchestra.run.smithi172.stdout: "req_unlink_latency": { 2024-04-26T02:05:57.721 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:57.721 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:57.721 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:57.721 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:57.721 INFO:teuthology.orchestra.run.smithi172.stdout: "cap_revoke_eviction": 0, 2024-04-26T02:05:57.721 INFO:teuthology.orchestra.run.smithi172.stdout: "cap_acquisition_throttle": 0, 2024-04-26T02:05:57.722 INFO:teuthology.orchestra.run.smithi172.stdout: "req_getvxattr_latency": { 2024-04-26T02:05:57.722 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:57.722 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:57.722 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:57.722 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-26T02:05:57.722 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:57.722 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_sessions": { 2024-04-26T02:05:57.722 INFO:teuthology.orchestra.run.smithi172.stdout: "session_count": 2, 2024-04-26T02:05:57.722 INFO:teuthology.orchestra.run.smithi172.stdout: "session_add": 2, 2024-04-26T02:05:57.722 INFO:teuthology.orchestra.run.smithi172.stdout: "session_remove": 0, 2024-04-26T02:05:57.722 INFO:teuthology.orchestra.run.smithi172.stdout: "sessions_open": 0, 2024-04-26T02:05:57.722 INFO:teuthology.orchestra.run.smithi172.stdout: "sessions_stale": 0, 2024-04-26T02:05:57.722 INFO:teuthology.orchestra.run.smithi172.stdout: "total_load": 0, 2024-04-26T02:05:57.722 INFO:teuthology.orchestra.run.smithi172.stdout: "average_load": 0, 2024-04-26T02:05:57.723 INFO:teuthology.orchestra.run.smithi172.stdout: "avg_session_uptime": 2381, 2024-04-26T02:05:57.723 INFO:teuthology.orchestra.run.smithi172.stdout: "mdthresh_evicted": 0 2024-04-26T02:05:57.723 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:57.723 INFO:teuthology.orchestra.run.smithi172.stdout: "mempool": { 2024-04-26T02:05:57.723 INFO:teuthology.orchestra.run.smithi172.stdout: "bloom_filter_bytes": 0, 2024-04-26T02:05:57.723 INFO:teuthology.orchestra.run.smithi172.stdout: "bloom_filter_items": 0, 2024-04-26T02:05:57.723 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T02:05:57.723 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_alloc_items": 0, 2024-04-26T02:05:57.723 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T02:05:57.723 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_data_items": 0, 2024-04-26T02:05:57.723 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T02:05:57.723 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T02:05:57.723 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T02:05:57.723 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T02:05:57.723 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T02:05:57.724 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_other_items": 0, 2024-04-26T02:05:57.724 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T02:05:57.724 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T02:05:57.724 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_extent_bytes": 0, 2024-04-26T02:05:57.724 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_extent_items": 0, 2024-04-26T02:05:57.724 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_blob_bytes": 0, 2024-04-26T02:05:57.724 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_blob_items": 0, 2024-04-26T02:05:57.724 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T02:05:57.724 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T02:05:57.724 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T02:05:57.724 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T02:05:57.724 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T02:05:57.724 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_fsck_items": 0, 2024-04-26T02:05:57.724 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_txc_bytes": 0, 2024-04-26T02:05:57.724 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_txc_items": 0, 2024-04-26T02:05:57.725 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T02:05:57.725 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T02:05:57.725 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_bytes": 0, 2024-04-26T02:05:57.725 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_items": 0, 2024-04-26T02:05:57.725 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_bytes": 0, 2024-04-26T02:05:57.725 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_items": 0, 2024-04-26T02:05:57.725 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T02:05:57.725 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_reader_items": 0, 2024-04-26T02:05:57.725 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T02:05:57.725 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_writer_items": 0, 2024-04-26T02:05:57.725 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_anon_bytes": 388185, 2024-04-26T02:05:57.725 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_anon_items": 4774, 2024-04-26T02:05:57.725 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_meta_bytes": 0, 2024-04-26T02:05:57.725 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_meta_items": 0, 2024-04-26T02:05:57.725 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_bytes": 0, 2024-04-26T02:05:57.726 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_items": 0, 2024-04-26T02:05:57.726 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_mapbl_bytes": 0, 2024-04-26T02:05:57.726 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_mapbl_items": 0, 2024-04-26T02:05:57.726 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_pglog_bytes": 0, 2024-04-26T02:05:57.726 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_pglog_items": 0, 2024-04-26T02:05:57.726 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_bytes": 7720, 2024-04-26T02:05:57.726 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_items": 160, 2024-04-26T02:05:57.726 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T02:05:57.726 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_mapping_items": 0, 2024-04-26T02:05:57.726 INFO:teuthology.orchestra.run.smithi172.stdout: "pgmap_bytes": 0, 2024-04-26T02:05:57.726 INFO:teuthology.orchestra.run.smithi172.stdout: "pgmap_items": 0, 2024-04-26T02:05:57.726 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_co_bytes": 20719522, 2024-04-26T02:05:57.726 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_co_items": 169093, 2024-04-26T02:05:57.727 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_1_bytes": 0, 2024-04-26T02:05:57.727 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_1_items": 0, 2024-04-26T02:05:57.727 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_2_bytes": 0, 2024-04-26T02:05:57.727 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_2_items": 0 2024-04-26T02:05:57.727 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:57.727 INFO:teuthology.orchestra.run.smithi172.stdout: "objecter": { 2024-04-26T02:05:57.727 INFO:teuthology.orchestra.run.smithi172.stdout: "op_active": 0, 2024-04-26T02:05:57.727 INFO:teuthology.orchestra.run.smithi172.stdout: "op_laggy": 0, 2024-04-26T02:05:57.727 INFO:teuthology.orchestra.run.smithi172.stdout: "op_send": 354, 2024-04-26T02:05:57.727 INFO:teuthology.orchestra.run.smithi172.stdout: "op_send_bytes": 26, 2024-04-26T02:05:57.727 INFO:teuthology.orchestra.run.smithi172.stdout: "op_resend": 4, 2024-04-26T02:05:57.727 INFO:teuthology.orchestra.run.smithi172.stdout: "op_reply": 354, 2024-04-26T02:05:57.728 INFO:teuthology.orchestra.run.smithi172.stdout: "op_latency": { 2024-04-26T02:05:57.728 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 354, 2024-04-26T02:05:57.728 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.351991462, 2024-04-26T02:05:57.728 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000994326 2024-04-26T02:05:57.728 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:57.728 INFO:teuthology.orchestra.run.smithi172.stdout: "op_inflight": 0, 2024-04-26T02:05:57.728 INFO:teuthology.orchestra.run.smithi172.stdout: "oplen_avg": { 2024-04-26T02:05:57.728 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 354, 2024-04-26T02:05:57.728 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 355 2024-04-26T02:05:57.728 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:57.728 INFO:teuthology.orchestra.run.smithi172.stdout: "op": 354, 2024-04-26T02:05:57.728 INFO:teuthology.orchestra.run.smithi172.stdout: "op_r": 354, 2024-04-26T02:05:57.728 INFO:teuthology.orchestra.run.smithi172.stdout: "op_w": 0, 2024-04-26T02:05:57.728 INFO:teuthology.orchestra.run.smithi172.stdout: "op_rmw": 0, 2024-04-26T02:05:57.729 INFO:teuthology.orchestra.run.smithi172.stdout: "op_pg": 0, 2024-04-26T02:05:57.729 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_stat": 205, 2024-04-26T02:05:57.729 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_create": 0, 2024-04-26T02:05:57.729 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_read": 146, 2024-04-26T02:05:57.729 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_write": 0, 2024-04-26T02:05:57.729 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_writefull": 0, 2024-04-26T02:05:57.729 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_writesame": 0, 2024-04-26T02:05:57.729 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_append": 0, 2024-04-26T02:05:57.729 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_zero": 0, 2024-04-26T02:05:57.729 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_truncate": 0, 2024-04-26T02:05:57.729 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_delete": 0, 2024-04-26T02:05:57.729 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_mapext": 0, 2024-04-26T02:05:57.729 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_sparse_read": 0, 2024-04-26T02:05:57.730 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_clonerange": 0, 2024-04-26T02:05:57.730 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_getxattr": 2, 2024-04-26T02:05:57.730 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_setxattr": 0, 2024-04-26T02:05:57.730 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_cmpxattr": 0, 2024-04-26T02:05:57.730 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_rmxattr": 0, 2024-04-26T02:05:57.730 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_resetxattrs": 0, 2024-04-26T02:05:57.730 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_call": 0, 2024-04-26T02:05:57.730 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_watch": 0, 2024-04-26T02:05:57.730 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_notify": 0, 2024-04-26T02:05:57.730 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T02:05:57.730 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_pgls": 0, 2024-04-26T02:05:57.730 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_pgls_filter": 0, 2024-04-26T02:05:57.730 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_other": 0, 2024-04-26T02:05:57.731 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_active": 0, 2024-04-26T02:05:57.731 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_send": 0, 2024-04-26T02:05:57.731 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_resend": 0, 2024-04-26T02:05:57.731 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_ping": 0, 2024-04-26T02:05:57.731 INFO:teuthology.orchestra.run.smithi172.stdout: "poolop_active": 0, 2024-04-26T02:05:57.731 INFO:teuthology.orchestra.run.smithi172.stdout: "poolop_send": 0, 2024-04-26T02:05:57.731 INFO:teuthology.orchestra.run.smithi172.stdout: "poolop_resend": 0, 2024-04-26T02:05:57.731 INFO:teuthology.orchestra.run.smithi172.stdout: "poolstat_active": 0, 2024-04-26T02:05:57.731 INFO:teuthology.orchestra.run.smithi172.stdout: "poolstat_send": 0, 2024-04-26T02:05:57.731 INFO:teuthology.orchestra.run.smithi172.stdout: "poolstat_resend": 0, 2024-04-26T02:05:57.731 INFO:teuthology.orchestra.run.smithi172.stdout: "statfs_active": 0, 2024-04-26T02:05:57.731 INFO:teuthology.orchestra.run.smithi172.stdout: "statfs_send": 0, 2024-04-26T02:05:57.731 INFO:teuthology.orchestra.run.smithi172.stdout: "statfs_resend": 0, 2024-04-26T02:05:57.731 INFO:teuthology.orchestra.run.smithi172.stdout: "command_active": 0, 2024-04-26T02:05:57.732 INFO:teuthology.orchestra.run.smithi172.stdout: "command_send": 0, 2024-04-26T02:05:57.732 INFO:teuthology.orchestra.run.smithi172.stdout: "command_resend": 0, 2024-04-26T02:05:57.732 INFO:teuthology.orchestra.run.smithi172.stdout: "map_epoch": 0, 2024-04-26T02:05:57.732 INFO:teuthology.orchestra.run.smithi172.stdout: "map_full": 0, 2024-04-26T02:05:57.732 INFO:teuthology.orchestra.run.smithi172.stdout: "map_inc": 0, 2024-04-26T02:05:57.732 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_sessions": 8, 2024-04-26T02:05:57.732 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_session_open": 8, 2024-04-26T02:05:57.732 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_session_close": 0, 2024-04-26T02:05:57.732 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_laggy": 0, 2024-04-26T02:05:57.732 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_wr": 0, 2024-04-26T02:05:57.732 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_rd": 2, 2024-04-26T02:05:57.732 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_del": 0 2024-04-26T02:05:57.733 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:57.733 INFO:teuthology.orchestra.run.smithi172.stdout: "oft": { 2024-04-26T02:05:57.733 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_objs": 0, 2024-04-26T02:05:57.733 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_kv_pairs": 0, 2024-04-26T02:05:57.733 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_updates": 0, 2024-04-26T02:05:57.733 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_removes": 0 2024-04-26T02:05:57.733 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:57.733 INFO:teuthology.orchestra.run.smithi172.stdout: "osd-slow-ops": { 2024-04-26T02:05:57.733 INFO:teuthology.orchestra.run.smithi172.stdout: "slow_ops_count": 0 2024-04-26T02:05:57.733 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:57.733 INFO:teuthology.orchestra.run.smithi172.stdout: "purge_queue": { 2024-04-26T02:05:57.733 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing_ops": 0, 2024-04-26T02:05:57.733 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing_ops_high_water": 0, 2024-04-26T02:05:57.734 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing": 0, 2024-04-26T02:05:57.734 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing_high_water": 0, 2024-04-26T02:05:57.734 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executed_ops": 0, 2024-04-26T02:05:57.734 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executed": 0, 2024-04-26T02:05:57.734 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_item_in_journal": 0 2024-04-26T02:05:57.734 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:57.734 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T02:05:57.734 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-04-26T02:05:57.734 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 104857600, 2024-04-26T02:05:57.734 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-04-26T02:05:57.734 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 392, 2024-04-26T02:05:57.734 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 37866706, 2024-04-26T02:05:57.734 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-04-26T02:05:57.735 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 392, 2024-04-26T02:05:57.735 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 0, 2024-04-26T02:05:57.735 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 0, 2024-04-26T02:05:57.735 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 392, 2024-04-26T02:05:57.735 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 37866706, 2024-04-26T02:05:57.735 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-04-26T02:05:57.735 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:57.735 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:57.735 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:57.735 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-26T02:05:57.735 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:57.735 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-objecter_bytes": { 2024-04-26T02:05:57.736 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-04-26T02:05:57.736 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 104857600, 2024-04-26T02:05:57.736 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-04-26T02:05:57.736 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 0, 2024-04-26T02:05:57.736 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 0, 2024-04-26T02:05:57.736 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-04-26T02:05:57.736 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 0, 2024-04-26T02:05:57.736 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 354, 2024-04-26T02:05:57.736 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 37628947, 2024-04-26T02:05:57.736 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 354, 2024-04-26T02:05:57.736 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 37628947, 2024-04-26T02:05:57.736 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-04-26T02:05:57.737 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:57.737 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:57.737 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:57.737 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-26T02:05:57.737 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:57.737 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-objecter_ops": { 2024-04-26T02:05:57.737 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-04-26T02:05:57.737 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 1024, 2024-04-26T02:05:57.737 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-04-26T02:05:57.737 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 0, 2024-04-26T02:05:57.737 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 0, 2024-04-26T02:05:57.737 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-04-26T02:05:57.737 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 0, 2024-04-26T02:05:57.738 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 354, 2024-04-26T02:05:57.738 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 354, 2024-04-26T02:05:57.738 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 354, 2024-04-26T02:05:57.738 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 354, 2024-04-26T02:05:57.738 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-04-26T02:05:57.738 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:57.738 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:57.738 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:57.738 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-26T02:05:57.738 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:57.738 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-write_buf_throttle": { 2024-04-26T02:05:57.738 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-04-26T02:05:57.738 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 3758096384, 2024-04-26T02:05:57.739 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-04-26T02:05:57.739 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 0, 2024-04-26T02:05:57.739 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 0, 2024-04-26T02:05:57.739 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-04-26T02:05:57.739 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 0, 2024-04-26T02:05:57.739 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 0, 2024-04-26T02:05:57.739 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 0, 2024-04-26T02:05:57.739 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 0, 2024-04-26T02:05:57.739 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 0, 2024-04-26T02:05:57.739 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-04-26T02:05:57.739 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:57.739 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:57.739 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:57.740 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-26T02:05:57.740 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:05:57.740 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-write_buf_throttle-0x555d5706a6e0": { 2024-04-26T02:05:57.740 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-04-26T02:05:57.740 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 3758096384, 2024-04-26T02:05:57.740 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-04-26T02:05:57.740 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 0, 2024-04-26T02:05:57.740 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 0, 2024-04-26T02:05:57.740 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-04-26T02:05:57.740 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 0, 2024-04-26T02:05:57.740 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 0, 2024-04-26T02:05:57.740 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 0, 2024-04-26T02:05:57.740 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 0, 2024-04-26T02:05:57.740 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 0, 2024-04-26T02:05:57.741 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-04-26T02:05:57.741 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:05:57.741 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:05:57.741 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:05:57.741 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-26T02:05:57.741 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-26T02:05:57.741 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-26T02:05:58.347 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:05:58 smithi165 ceph-mon[32648]: pgmap v387: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T02:05:58.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:05:58 smithi151 ceph-mon[28057]: pgmap v387: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T02:05:58.479 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:05:58 smithi172 ceph-mon[32785]: pgmap v387: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T02:05:58.506 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-26T02:05:58.506 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-26T02:05:58.506 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-04-26T02:05:58.506 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-04-26T02:05:58.506 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=7 2024-04-26T02:05:58.506 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-04-26T02:05:58.506 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-04-26T02:05:58.506 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2024-04-26T02:05:58.506 DEBUG:tasks.check_counter:Getting stats from i 2024-04-26T02:05:58.507 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-26T02:05:58.745 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T02:06:00.172 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:00 smithi172 ceph-mon[32785]: pgmap v388: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T02:06:00.347 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:00 smithi165 ceph-mon[32648]: pgmap v388: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T02:06:00.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:00 smithi151 ceph-mon[28057]: pgmap v388: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T02:06:00.447 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-26T02:06:00.448 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T02:06:00.448 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 6617, 2024-04-26T02:06:00.448 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 4106, 2024-04-26T02:06:00.448 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 5484086, 2024-04-26T02:06:00.448 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 1367569, 2024-04-26T02:06:00.448 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 10, 2024-04-26T02:06:00.448 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 5, 2024-04-26T02:06:00.448 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 0.317273623, 2024-04-26T02:06:00.448 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.124276189, 2024-04-26T02:06:00.448 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.245040418, 2024-04-26T02:06:00.448 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.009174468, 2024-04-26T02:06:00.448 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:06:00.448 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 4104, 2024-04-26T02:06:00.449 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.116625464, 2024-04-26T02:06:00.449 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000028417 2024-04-26T02:06:00.449 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:00.449 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:06:00.449 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 10299, 2024-04-26T02:06:00.449 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.001792133, 2024-04-26T02:06:00.449 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000174 2024-04-26T02:06:00.449 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:00.449 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T02:06:00.449 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T02:06:00.449 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:00.449 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T02:06:00.450 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 687, 2024-04-26T02:06:00.450 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 818, 2024-04-26T02:06:00.450 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 1951485, 2024-04-26T02:06:00.450 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 379102, 2024-04-26T02:06:00.450 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 15, 2024-04-26T02:06:00.450 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 6, 2024-04-26T02:06:00.450 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 0.092746786, 2024-04-26T02:06:00.450 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.030877106, 2024-04-26T02:06:00.450 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.069341974, 2024-04-26T02:06:00.450 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.007245573, 2024-04-26T02:06:00.450 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:06:00.451 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 817, 2024-04-26T02:06:00.451 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.032308797, 2024-04-26T02:06:00.451 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000039545 2024-04-26T02:06:00.451 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:00.451 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:06:00.451 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 675, 2024-04-26T02:06:00.451 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000137154, 2024-04-26T02:06:00.451 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000203 2024-04-26T02:06:00.451 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:00.451 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 1852864, 2024-04-26T02:06:00.451 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 190320 2024-04-26T02:06:00.452 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:00.452 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T02:06:00.452 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 92, 2024-04-26T02:06:00.452 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 344, 2024-04-26T02:06:00.452 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 40776, 2024-04-26T02:06:00.452 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 700301, 2024-04-26T02:06:00.452 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 15, 2024-04-26T02:06:00.452 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 7, 2024-04-26T02:06:00.452 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 0.021230017, 2024-04-26T02:06:00.452 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.010545722, 2024-04-26T02:06:00.452 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.008110536, 2024-04-26T02:06:00.452 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.000550153, 2024-04-26T02:06:00.452 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:06:00.453 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 342, 2024-04-26T02:06:00.453 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.033556847, 2024-04-26T02:06:00.453 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000098119 2024-04-26T02:06:00.453 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:00.453 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:06:00.453 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 164, 2024-04-26T02:06:00.453 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000021757, 2024-04-26T02:06:00.453 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000132 2024-04-26T02:06:00.453 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:00.453 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 6496, 2024-04-26T02:06:00.453 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 603328 2024-04-26T02:06:00.453 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:00.454 INFO:teuthology.orchestra.run.smithi172.stdout: "cct": { 2024-04-26T02:06:00.454 INFO:teuthology.orchestra.run.smithi172.stdout: "total_workers": 1, 2024-04-26T02:06:00.454 INFO:teuthology.orchestra.run.smithi172.stdout: "unhealthy_workers": 0 2024-04-26T02:06:00.454 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:00.454 INFO:teuthology.orchestra.run.smithi172.stdout: "finisher-MDSRank": { 2024-04-26T02:06:00.454 INFO:teuthology.orchestra.run.smithi172.stdout: "queue_len": 0, 2024-04-26T02:06:00.454 INFO:teuthology.orchestra.run.smithi172.stdout: "complete_latency": { 2024-04-26T02:06:00.454 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 165, 2024-04-26T02:06:00.454 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.038896164, 2024-04-26T02:06:00.454 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000235734 2024-04-26T02:06:00.454 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-26T02:06:00.454 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:00.454 INFO:teuthology.orchestra.run.smithi172.stdout: "finisher-PurgeQueue": { 2024-04-26T02:06:00.455 INFO:teuthology.orchestra.run.smithi172.stdout: "queue_len": 0, 2024-04-26T02:06:00.455 INFO:teuthology.orchestra.run.smithi172.stdout: "complete_latency": { 2024-04-26T02:06:00.455 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 51, 2024-04-26T02:06:00.455 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.001820466, 2024-04-26T02:06:00.455 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000035695 2024-04-26T02:06:00.455 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-26T02:06:00.455 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:00.455 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": { 2024-04-26T02:06:00.455 INFO:teuthology.orchestra.run.smithi172.stdout: "request": 229, 2024-04-26T02:06:00.455 INFO:teuthology.orchestra.run.smithi172.stdout: "reply": 57, 2024-04-26T02:06:00.455 INFO:teuthology.orchestra.run.smithi172.stdout: "reply_latency": { 2024-04-26T02:06:00.455 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 57, 2024-04-26T02:06:00.455 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.093230287, 2024-04-26T02:06:00.455 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.001635619 2024-04-26T02:06:00.456 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:00.456 INFO:teuthology.orchestra.run.smithi172.stdout: "slow_reply": 0, 2024-04-26T02:06:00.456 INFO:teuthology.orchestra.run.smithi172.stdout: "forward": 172, 2024-04-26T02:06:00.456 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_fetch_complete": 0, 2024-04-26T02:06:00.456 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_fetch_keys": 0, 2024-04-26T02:06:00.456 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_commit": 11, 2024-04-26T02:06:00.456 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_split": 0, 2024-04-26T02:06:00.456 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_merge": 0, 2024-04-26T02:06:00.456 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes": 30, 2024-04-26T02:06:00.456 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_top": 2, 2024-04-26T02:06:00.456 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_bottom": 1, 2024-04-26T02:06:00.456 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_pin_tail": 27, 2024-04-26T02:06:00.456 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_pinned": 27, 2024-04-26T02:06:00.457 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_expired": 161, 2024-04-26T02:06:00.457 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_with_caps": 1, 2024-04-26T02:06:00.457 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": 2, 2024-04-26T02:06:00.457 INFO:teuthology.orchestra.run.smithi172.stdout: "subtrees": 4, 2024-04-26T02:06:00.457 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse": 580, 2024-04-26T02:06:00.457 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_hit": 308, 2024-04-26T02:06:00.457 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_forward": 170, 2024-04-26T02:06:00.457 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_discover": 63, 2024-04-26T02:06:00.457 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_dir_fetch": 0, 2024-04-26T02:06:00.457 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_remote_ino": 0, 2024-04-26T02:06:00.457 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_lock": 0, 2024-04-26T02:06:00.457 INFO:teuthology.orchestra.run.smithi172.stdout: "load_cent": 0, 2024-04-26T02:06:00.457 INFO:teuthology.orchestra.run.smithi172.stdout: "q": 0, 2024-04-26T02:06:00.457 INFO:teuthology.orchestra.run.smithi172.stdout: "exported": 3, 2024-04-26T02:06:00.458 INFO:teuthology.orchestra.run.smithi172.stdout: "exported_inodes": 38, 2024-04-26T02:06:00.458 INFO:teuthology.orchestra.run.smithi172.stdout: "imported": 5, 2024-04-26T02:06:00.458 INFO:teuthology.orchestra.run.smithi172.stdout: "imported_inodes": 23, 2024-04-26T02:06:00.458 INFO:teuthology.orchestra.run.smithi172.stdout: "openino_dir_fetch": 0, 2024-04-26T02:06:00.458 INFO:teuthology.orchestra.run.smithi172.stdout: "openino_backtrace_fetch": 0, 2024-04-26T02:06:00.458 INFO:teuthology.orchestra.run.smithi172.stdout: "openino_peer_discover": 0, 2024-04-26T02:06:00.458 INFO:teuthology.orchestra.run.smithi172.stdout: "root_rfiles": 4, 2024-04-26T02:06:00.458 INFO:teuthology.orchestra.run.smithi172.stdout: "root_rbytes": 234, 2024-04-26T02:06:00.458 INFO:teuthology.orchestra.run.smithi172.stdout: "root_rsnaps": 7, 2024-04-26T02:06:00.458 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T02:06:00.458 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_set_tag": 0, 2024-04-26T02:06:00.458 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T02:06:00.458 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_inotable_repaired": 0, 2024-04-26T02:06:00.458 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_dir_inodes": 0, 2024-04-26T02:06:00.459 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T02:06:00.459 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T02:06:00.459 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_file_inodes": 0, 2024-04-26T02:06:00.459 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_inode_file_caps": 0, 2024-04-26T02:06:00.459 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_revoke": 45, 2024-04-26T02:06:00.459 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_grant": 45, 2024-04-26T02:06:00.459 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T02:06:00.459 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T02:06:00.459 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T02:06:00.459 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_caps": 63, 2024-04-26T02:06:00.459 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_caps_dirty": 21, 2024-04-26T02:06:00.459 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_cap_release": 6, 2024-04-26T02:06:00.459 INFO:teuthology.orchestra.run.smithi172.stdout: "process_request_cap_release": 53 2024-04-26T02:06:00.459 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:00.459 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_cache": { 2024-04-26T02:06:00.460 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_update": 12, 2024-04-26T02:06:00.460 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_update_receipt": 479, 2024-04-26T02:06:00.460 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_try_discover": 268, 2024-04-26T02:06:00.460 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_send_discover": 274, 2024-04-26T02:06:00.460 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_handle_discover": 19, 2024-04-26T02:06:00.460 INFO:teuthology.orchestra.run.smithi172.stdout: "num_strays": 4, 2024-04-26T02:06:00.460 INFO:teuthology.orchestra.run.smithi172.stdout: "num_strays_delayed": 0, 2024-04-26T02:06:00.460 INFO:teuthology.orchestra.run.smithi172.stdout: "num_strays_enqueuing": 0, 2024-04-26T02:06:00.460 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_created": 23, 2024-04-26T02:06:00.460 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_enqueued": 19, 2024-04-26T02:06:00.460 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_reintegrated": 0, 2024-04-26T02:06:00.460 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_migrated": 0, 2024-04-26T02:06:00.460 INFO:teuthology.orchestra.run.smithi172.stdout: "num_recovering_processing": 0, 2024-04-26T02:06:00.460 INFO:teuthology.orchestra.run.smithi172.stdout: "num_recovering_enqueued": 0, 2024-04-26T02:06:00.461 INFO:teuthology.orchestra.run.smithi172.stdout: "num_recovering_prioritized": 0, 2024-04-26T02:06:00.461 INFO:teuthology.orchestra.run.smithi172.stdout: "recovery_started": 0, 2024-04-26T02:06:00.461 INFO:teuthology.orchestra.run.smithi172.stdout: "recovery_completed": 0, 2024-04-26T02:06:00.461 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_quiesce_path": 8, 2024-04-26T02:06:00.461 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_quiesce_inode": 454, 2024-04-26T02:06:00.461 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T02:06:00.461 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_exportdir": 3, 2024-04-26T02:06:00.461 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_flush": 0, 2024-04-26T02:06:00.461 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_fragmentdir": 0, 2024-04-26T02:06:00.461 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_fragstats": 0, 2024-04-26T02:06:00.461 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_inodestats": 0 2024-04-26T02:06:00.461 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:00.461 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_log": { 2024-04-26T02:06:00.461 INFO:teuthology.orchestra.run.smithi172.stdout: "evlrg": 0, 2024-04-26T02:06:00.461 INFO:teuthology.orchestra.run.smithi172.stdout: "evadd": 109, 2024-04-26T02:06:00.462 INFO:teuthology.orchestra.run.smithi172.stdout: "evex": 0, 2024-04-26T02:06:00.462 INFO:teuthology.orchestra.run.smithi172.stdout: "evtrm": 0, 2024-04-26T02:06:00.462 INFO:teuthology.orchestra.run.smithi172.stdout: "ev": 109, 2024-04-26T02:06:00.462 INFO:teuthology.orchestra.run.smithi172.stdout: "evexg": 0, 2024-04-26T02:06:00.462 INFO:teuthology.orchestra.run.smithi172.stdout: "evexd": 0, 2024-04-26T02:06:00.462 INFO:teuthology.orchestra.run.smithi172.stdout: "segadd": 2, 2024-04-26T02:06:00.462 INFO:teuthology.orchestra.run.smithi172.stdout: "segex": 0, 2024-04-26T02:06:00.462 INFO:teuthology.orchestra.run.smithi172.stdout: "segtrm": 0, 2024-04-26T02:06:00.462 INFO:teuthology.orchestra.run.smithi172.stdout: "seg": 2, 2024-04-26T02:06:00.462 INFO:teuthology.orchestra.run.smithi172.stdout: "segmjr": 2, 2024-04-26T02:06:00.462 INFO:teuthology.orchestra.run.smithi172.stdout: "segexg": 0, 2024-04-26T02:06:00.462 INFO:teuthology.orchestra.run.smithi172.stdout: "segexd": 0, 2024-04-26T02:06:00.462 INFO:teuthology.orchestra.run.smithi172.stdout: "expos": 4194304, 2024-04-26T02:06:00.462 INFO:teuthology.orchestra.run.smithi172.stdout: "wrpos": 4438137, 2024-04-26T02:06:00.462 INFO:teuthology.orchestra.run.smithi172.stdout: "rdpos": 0, 2024-04-26T02:06:00.463 INFO:teuthology.orchestra.run.smithi172.stdout: "jlat": { 2024-04-26T02:06:00.463 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 47, 2024-04-26T02:06:00.463 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.087275998, 2024-04-26T02:06:00.463 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.001856936 2024-04-26T02:06:00.463 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:00.463 INFO:teuthology.orchestra.run.smithi172.stdout: "replayed": 0 2024-04-26T02:06:00.463 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:00.463 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_mem": { 2024-04-26T02:06:00.463 INFO:teuthology.orchestra.run.smithi172.stdout: "ino": 26, 2024-04-26T02:06:00.463 INFO:teuthology.orchestra.run.smithi172.stdout: "ino+": 135, 2024-04-26T02:06:00.463 INFO:teuthology.orchestra.run.smithi172.stdout: "ino-": 109, 2024-04-26T02:06:00.463 INFO:teuthology.orchestra.run.smithi172.stdout: "dir": 23, 2024-04-26T02:06:00.463 INFO:teuthology.orchestra.run.smithi172.stdout: "dir+": 248, 2024-04-26T02:06:00.463 INFO:teuthology.orchestra.run.smithi172.stdout: "dir-": 225, 2024-04-26T02:06:00.463 INFO:teuthology.orchestra.run.smithi172.stdout: "dn": 30, 2024-04-26T02:06:00.464 INFO:teuthology.orchestra.run.smithi172.stdout: "dn+": 212, 2024-04-26T02:06:00.464 INFO:teuthology.orchestra.run.smithi172.stdout: "dn-": 182, 2024-04-26T02:06:00.464 INFO:teuthology.orchestra.run.smithi172.stdout: "cap": 2, 2024-04-26T02:06:00.464 INFO:teuthology.orchestra.run.smithi172.stdout: "cap+": 53, 2024-04-26T02:06:00.464 INFO:teuthology.orchestra.run.smithi172.stdout: "cap-": 51, 2024-04-26T02:06:00.464 INFO:teuthology.orchestra.run.smithi172.stdout: "rss": 56180, 2024-04-26T02:06:00.464 INFO:teuthology.orchestra.run.smithi172.stdout: "heap": 223608 2024-04-26T02:06:00.464 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:00.464 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_server": { 2024-04-26T02:06:00.464 INFO:teuthology.orchestra.run.smithi172.stdout: "dispatch_client_request": 288, 2024-04-26T02:06:00.464 INFO:teuthology.orchestra.run.smithi172.stdout: "dispatch_server_request": 12, 2024-04-26T02:06:00.464 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_request": 229, 2024-04-26T02:06:00.464 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_session": 28, 2024-04-26T02:06:00.464 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_peer_request": 344, 2024-04-26T02:06:00.464 INFO:teuthology.orchestra.run.smithi172.stdout: "req_create_latency": { 2024-04-26T02:06:00.465 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 2, 2024-04-26T02:06:00.465 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.008259951, 2024-04-26T02:06:00.465 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.004129975 2024-04-26T02:06:00.465 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:00.465 INFO:teuthology.orchestra.run.smithi172.stdout: "req_getattr_latency": { 2024-04-26T02:06:00.465 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 4, 2024-04-26T02:06:00.465 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.001039290, 2024-04-26T02:06:00.465 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000259822 2024-04-26T02:06:00.465 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:00.465 INFO:teuthology.orchestra.run.smithi172.stdout: "req_getfilelock_latency": { 2024-04-26T02:06:00.465 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:06:00.465 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:06:00.465 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:06:00.465 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:00.465 INFO:teuthology.orchestra.run.smithi172.stdout: "req_link_latency": { 2024-04-26T02:06:00.465 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:06:00.466 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:06:00.466 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:06:00.466 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:00.466 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookup_latency": { 2024-04-26T02:06:00.466 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 18, 2024-04-26T02:06:00.466 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.006462380, 2024-04-26T02:06:00.466 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000359021 2024-04-26T02:06:00.466 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:00.466 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookuphash_latency": { 2024-04-26T02:06:00.466 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:06:00.466 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:06:00.466 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:06:00.466 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:00.466 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupino_latency": { 2024-04-26T02:06:00.466 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 1, 2024-04-26T02:06:00.467 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000203473, 2024-04-26T02:06:00.467 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000203473 2024-04-26T02:06:00.467 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:00.467 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupname_latency": { 2024-04-26T02:06:00.467 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:06:00.467 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:06:00.467 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:06:00.467 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:00.467 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupparent_latency": { 2024-04-26T02:06:00.467 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:06:00.467 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:06:00.467 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:06:00.467 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:00.467 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupsnap_latency": { 2024-04-26T02:06:00.467 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:06:00.467 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:06:00.468 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:06:00.468 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:00.468 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lssnap_latency": { 2024-04-26T02:06:00.468 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:06:00.468 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:06:00.468 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:06:00.468 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:00.468 INFO:teuthology.orchestra.run.smithi172.stdout: "req_mkdir_latency": { 2024-04-26T02:06:00.469 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:06:00.469 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:06:00.469 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:06:00.469 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:00.469 INFO:teuthology.orchestra.run.smithi172.stdout: "req_mknod_latency": { 2024-04-26T02:06:00.469 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:06:00.469 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:06:00.469 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:06:00.469 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:00.469 INFO:teuthology.orchestra.run.smithi172.stdout: "req_mksnap_latency": { 2024-04-26T02:06:00.469 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:06:00.469 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:06:00.469 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:06:00.469 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:00.469 INFO:teuthology.orchestra.run.smithi172.stdout: "req_open_latency": { 2024-04-26T02:06:00.470 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:06:00.470 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:06:00.470 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:06:00.470 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:00.470 INFO:teuthology.orchestra.run.smithi172.stdout: "req_readdir_latency": { 2024-04-26T02:06:00.470 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 9, 2024-04-26T02:06:00.470 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.005262096, 2024-04-26T02:06:00.470 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000584677 2024-04-26T02:06:00.470 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:00.470 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rename_latency": { 2024-04-26T02:06:00.470 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:06:00.470 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:06:00.470 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:06:00.471 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:00.471 INFO:teuthology.orchestra.run.smithi172.stdout: "req_renamesnap_latency": { 2024-04-26T02:06:00.471 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:06:00.471 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:06:00.471 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:06:00.471 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:00.471 INFO:teuthology.orchestra.run.smithi172.stdout: "req_snapdiff_latency": { 2024-04-26T02:06:00.471 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:06:00.471 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:06:00.471 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:06:00.471 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:00.471 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rmdir_latency": { 2024-04-26T02:06:00.471 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 4, 2024-04-26T02:06:00.471 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.030825496, 2024-04-26T02:06:00.471 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.007706374 2024-04-26T02:06:00.472 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:00.472 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rmsnap_latency": { 2024-04-26T02:06:00.472 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:06:00.472 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:06:00.472 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:06:00.472 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:00.472 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rmxattr_latency": { 2024-04-26T02:06:00.472 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:06:00.472 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:06:00.472 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:06:00.472 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:00.472 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setattr_latency": { 2024-04-26T02:06:00.472 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:06:00.472 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:06:00.472 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:06:00.473 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:00.473 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setdirlayout_latency": { 2024-04-26T02:06:00.473 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:06:00.473 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:06:00.473 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:06:00.473 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:00.473 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setfilelock_latency": { 2024-04-26T02:06:00.473 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:06:00.473 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:06:00.473 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:06:00.473 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:00.473 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setlayout_latency": { 2024-04-26T02:06:00.473 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:06:00.473 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:06:00.473 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:06:00.473 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:00.474 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setxattr_latency": { 2024-04-26T02:06:00.474 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:06:00.474 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:06:00.474 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:06:00.474 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:00.474 INFO:teuthology.orchestra.run.smithi172.stdout: "req_symlink_latency": { 2024-04-26T02:06:00.474 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:06:00.474 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:06:00.474 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:06:00.474 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:00.474 INFO:teuthology.orchestra.run.smithi172.stdout: "req_unlink_latency": { 2024-04-26T02:06:00.475 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 19, 2024-04-26T02:06:00.475 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.041177601, 2024-04-26T02:06:00.475 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.002167242 2024-04-26T02:06:00.475 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:00.475 INFO:teuthology.orchestra.run.smithi172.stdout: "cap_revoke_eviction": 0, 2024-04-26T02:06:00.475 INFO:teuthology.orchestra.run.smithi172.stdout: "cap_acquisition_throttle": 0, 2024-04-26T02:06:00.475 INFO:teuthology.orchestra.run.smithi172.stdout: "req_getvxattr_latency": { 2024-04-26T02:06:00.475 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:06:00.475 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:06:00.475 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:06:00.475 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-26T02:06:00.475 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:00.475 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_sessions": { 2024-04-26T02:06:00.475 INFO:teuthology.orchestra.run.smithi172.stdout: "session_count": 2, 2024-04-26T02:06:00.475 INFO:teuthology.orchestra.run.smithi172.stdout: "session_add": 4, 2024-04-26T02:06:00.476 INFO:teuthology.orchestra.run.smithi172.stdout: "session_remove": 2, 2024-04-26T02:06:00.476 INFO:teuthology.orchestra.run.smithi172.stdout: "sessions_open": 2, 2024-04-26T02:06:00.476 INFO:teuthology.orchestra.run.smithi172.stdout: "sessions_stale": 0, 2024-04-26T02:06:00.476 INFO:teuthology.orchestra.run.smithi172.stdout: "total_load": 44, 2024-04-26T02:06:00.476 INFO:teuthology.orchestra.run.smithi172.stdout: "average_load": 22, 2024-04-26T02:06:00.476 INFO:teuthology.orchestra.run.smithi172.stdout: "avg_session_uptime": 727, 2024-04-26T02:06:00.476 INFO:teuthology.orchestra.run.smithi172.stdout: "mdthresh_evicted": 0 2024-04-26T02:06:00.476 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:00.476 INFO:teuthology.orchestra.run.smithi172.stdout: "mempool": { 2024-04-26T02:06:00.476 INFO:teuthology.orchestra.run.smithi172.stdout: "bloom_filter_bytes": 0, 2024-04-26T02:06:00.476 INFO:teuthology.orchestra.run.smithi172.stdout: "bloom_filter_items": 0, 2024-04-26T02:06:00.476 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T02:06:00.476 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_alloc_items": 0, 2024-04-26T02:06:00.476 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T02:06:00.476 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_data_items": 0, 2024-04-26T02:06:00.476 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T02:06:00.477 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T02:06:00.477 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T02:06:00.477 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T02:06:00.477 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T02:06:00.477 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_other_items": 0, 2024-04-26T02:06:00.477 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T02:06:00.477 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T02:06:00.477 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_extent_bytes": 0, 2024-04-26T02:06:00.477 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_extent_items": 0, 2024-04-26T02:06:00.477 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_blob_bytes": 0, 2024-04-26T02:06:00.477 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_blob_items": 0, 2024-04-26T02:06:00.477 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T02:06:00.477 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T02:06:00.477 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T02:06:00.477 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T02:06:00.478 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T02:06:00.478 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_fsck_items": 0, 2024-04-26T02:06:00.478 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_txc_bytes": 0, 2024-04-26T02:06:00.478 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_txc_items": 0, 2024-04-26T02:06:00.478 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T02:06:00.478 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T02:06:00.478 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_bytes": 0, 2024-04-26T02:06:00.478 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_items": 0, 2024-04-26T02:06:00.478 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_bytes": 0, 2024-04-26T02:06:00.478 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_items": 0, 2024-04-26T02:06:00.478 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T02:06:00.478 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_reader_items": 0, 2024-04-26T02:06:00.478 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T02:06:00.478 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_writer_items": 0, 2024-04-26T02:06:00.479 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_anon_bytes": 328565, 2024-04-26T02:06:00.479 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_anon_items": 119, 2024-04-26T02:06:00.479 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_meta_bytes": 0, 2024-04-26T02:06:00.479 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_meta_items": 0, 2024-04-26T02:06:00.479 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_bytes": 0, 2024-04-26T02:06:00.479 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_items": 0, 2024-04-26T02:06:00.479 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_mapbl_bytes": 0, 2024-04-26T02:06:00.479 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_mapbl_items": 0, 2024-04-26T02:06:00.479 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_pglog_bytes": 0, 2024-04-26T02:06:00.479 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_pglog_items": 0, 2024-04-26T02:06:00.479 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_bytes": 7720, 2024-04-26T02:06:00.479 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_items": 160, 2024-04-26T02:06:00.479 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T02:06:00.479 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_mapping_items": 0, 2024-04-26T02:06:00.479 INFO:teuthology.orchestra.run.smithi172.stdout: "pgmap_bytes": 0, 2024-04-26T02:06:00.479 INFO:teuthology.orchestra.run.smithi172.stdout: "pgmap_items": 0, 2024-04-26T02:06:00.480 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_co_bytes": 121064, 2024-04-26T02:06:00.480 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_co_items": 1217, 2024-04-26T02:06:00.480 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_1_bytes": 0, 2024-04-26T02:06:00.480 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_1_items": 0, 2024-04-26T02:06:00.480 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_2_bytes": 0, 2024-04-26T02:06:00.480 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_2_items": 0 2024-04-26T02:06:00.480 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:00.480 INFO:teuthology.orchestra.run.smithi172.stdout: "objecter": { 2024-04-26T02:06:00.480 INFO:teuthology.orchestra.run.smithi172.stdout: "op_active": 0, 2024-04-26T02:06:00.480 INFO:teuthology.orchestra.run.smithi172.stdout: "op_laggy": 0, 2024-04-26T02:06:00.480 INFO:teuthology.orchestra.run.smithi172.stdout: "op_send": 141, 2024-04-26T02:06:00.480 INFO:teuthology.orchestra.run.smithi172.stdout: "op_send_bytes": 262735, 2024-04-26T02:06:00.480 INFO:teuthology.orchestra.run.smithi172.stdout: "op_resend": 28, 2024-04-26T02:06:00.480 INFO:teuthology.orchestra.run.smithi172.stdout: "op_reply": 141, 2024-04-26T02:06:00.480 INFO:teuthology.orchestra.run.smithi172.stdout: "op_latency": { 2024-04-26T02:06:00.481 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 141, 2024-04-26T02:06:00.481 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.957976457, 2024-04-26T02:06:00.481 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.006794159 2024-04-26T02:06:00.481 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:00.481 INFO:teuthology.orchestra.run.smithi172.stdout: "op_inflight": 0, 2024-04-26T02:06:00.481 INFO:teuthology.orchestra.run.smithi172.stdout: "oplen_avg": { 2024-04-26T02:06:00.481 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 141, 2024-04-26T02:06:00.481 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 182 2024-04-26T02:06:00.481 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:00.481 INFO:teuthology.orchestra.run.smithi172.stdout: "op": 141, 2024-04-26T02:06:00.481 INFO:teuthology.orchestra.run.smithi172.stdout: "op_r": 3, 2024-04-26T02:06:00.481 INFO:teuthology.orchestra.run.smithi172.stdout: "op_w": 138, 2024-04-26T02:06:00.482 INFO:teuthology.orchestra.run.smithi172.stdout: "op_rmw": 0, 2024-04-26T02:06:00.482 INFO:teuthology.orchestra.run.smithi172.stdout: "op_pg": 0, 2024-04-26T02:06:00.482 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_stat": 0, 2024-04-26T02:06:00.482 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_create": 10, 2024-04-26T02:06:00.482 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_read": 3, 2024-04-26T02:06:00.482 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_write": 50, 2024-04-26T02:06:00.482 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_writefull": 16, 2024-04-26T02:06:00.482 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_writesame": 0, 2024-04-26T02:06:00.482 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_append": 0, 2024-04-26T02:06:00.482 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_zero": 0, 2024-04-26T02:06:00.482 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_truncate": 0, 2024-04-26T02:06:00.482 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_delete": 31, 2024-04-26T02:06:00.482 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_mapext": 0, 2024-04-26T02:06:00.483 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_sparse_read": 0, 2024-04-26T02:06:00.483 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_clonerange": 0, 2024-04-26T02:06:00.483 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_getxattr": 0, 2024-04-26T02:06:00.483 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_setxattr": 20, 2024-04-26T02:06:00.483 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_cmpxattr": 0, 2024-04-26T02:06:00.483 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_rmxattr": 0, 2024-04-26T02:06:00.483 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_resetxattrs": 0, 2024-04-26T02:06:00.483 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_call": 0, 2024-04-26T02:06:00.483 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_watch": 0, 2024-04-26T02:06:00.483 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_notify": 0, 2024-04-26T02:06:00.483 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T02:06:00.483 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_pgls": 0, 2024-04-26T02:06:00.483 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_pgls_filter": 0, 2024-04-26T02:06:00.483 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_other": 0, 2024-04-26T02:06:00.484 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_active": 0, 2024-04-26T02:06:00.484 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_send": 0, 2024-04-26T02:06:00.484 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_resend": 0, 2024-04-26T02:06:00.484 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_ping": 0, 2024-04-26T02:06:00.484 INFO:teuthology.orchestra.run.smithi172.stdout: "poolop_active": 0, 2024-04-26T02:06:00.484 INFO:teuthology.orchestra.run.smithi172.stdout: "poolop_send": 0, 2024-04-26T02:06:00.484 INFO:teuthology.orchestra.run.smithi172.stdout: "poolop_resend": 0, 2024-04-26T02:06:00.484 INFO:teuthology.orchestra.run.smithi172.stdout: "poolstat_active": 0, 2024-04-26T02:06:00.484 INFO:teuthology.orchestra.run.smithi172.stdout: "poolstat_send": 0, 2024-04-26T02:06:00.484 INFO:teuthology.orchestra.run.smithi172.stdout: "poolstat_resend": 0, 2024-04-26T02:06:00.484 INFO:teuthology.orchestra.run.smithi172.stdout: "statfs_active": 0, 2024-04-26T02:06:00.484 INFO:teuthology.orchestra.run.smithi172.stdout: "statfs_send": 0, 2024-04-26T02:06:00.484 INFO:teuthology.orchestra.run.smithi172.stdout: "statfs_resend": 0, 2024-04-26T02:06:00.484 INFO:teuthology.orchestra.run.smithi172.stdout: "command_active": 0, 2024-04-26T02:06:00.485 INFO:teuthology.orchestra.run.smithi172.stdout: "command_send": 0, 2024-04-26T02:06:00.485 INFO:teuthology.orchestra.run.smithi172.stdout: "command_resend": 0, 2024-04-26T02:06:00.485 INFO:teuthology.orchestra.run.smithi172.stdout: "map_epoch": 107, 2024-04-26T02:06:00.485 INFO:teuthology.orchestra.run.smithi172.stdout: "map_full": 0, 2024-04-26T02:06:00.485 INFO:teuthology.orchestra.run.smithi172.stdout: "map_inc": 17, 2024-04-26T02:06:00.485 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_sessions": 12, 2024-04-26T02:06:00.485 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_session_open": 16, 2024-04-26T02:06:00.485 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_session_close": 4, 2024-04-26T02:06:00.485 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_laggy": 0, 2024-04-26T02:06:00.485 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_wr": 42, 2024-04-26T02:06:00.485 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_rd": 0, 2024-04-26T02:06:00.485 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_del": 10 2024-04-26T02:06:00.485 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:00.485 INFO:teuthology.orchestra.run.smithi172.stdout: "oft": { 2024-04-26T02:06:00.486 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_objs": 1, 2024-04-26T02:06:00.486 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_kv_pairs": 0, 2024-04-26T02:06:00.486 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_updates": 0, 2024-04-26T02:06:00.486 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_removes": 0 2024-04-26T02:06:00.486 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:00.486 INFO:teuthology.orchestra.run.smithi172.stdout: "osd-slow-ops": { 2024-04-26T02:06:00.486 INFO:teuthology.orchestra.run.smithi172.stdout: "slow_ops_count": 469 2024-04-26T02:06:00.486 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:00.486 INFO:teuthology.orchestra.run.smithi172.stdout: "purge_queue": { 2024-04-26T02:06:00.486 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing_ops": 0, 2024-04-26T02:06:00.486 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing_ops_high_water": 10, 2024-04-26T02:06:00.486 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing": 0, 2024-04-26T02:06:00.486 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing_high_water": 10, 2024-04-26T02:06:00.486 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executed_ops": 19, 2024-04-26T02:06:00.487 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executed": 19, 2024-04-26T02:06:00.487 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_item_in_journal": 0 2024-04-26T02:06:00.487 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:00.487 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T02:06:00.487 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-04-26T02:06:00.487 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 104857600, 2024-04-26T02:06:00.487 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-04-26T02:06:00.487 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 7392, 2024-04-26T02:06:00.487 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 6788924, 2024-04-26T02:06:00.487 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-04-26T02:06:00.487 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 7392, 2024-04-26T02:06:00.487 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 0, 2024-04-26T02:06:00.487 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 0, 2024-04-26T02:06:00.487 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 7392, 2024-04-26T02:06:00.488 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 6788924, 2024-04-26T02:06:00.488 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-04-26T02:06:00.488 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:06:00.488 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:06:00.488 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:06:00.488 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-26T02:06:00.488 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:00.488 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-objecter_bytes": { 2024-04-26T02:06:00.488 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-04-26T02:06:00.488 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 104857600, 2024-04-26T02:06:00.488 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-04-26T02:06:00.488 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 0, 2024-04-26T02:06:00.488 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 0, 2024-04-26T02:06:00.488 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-04-26T02:06:00.489 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 0, 2024-04-26T02:06:00.489 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 141, 2024-04-26T02:06:00.489 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 265718, 2024-04-26T02:06:00.489 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 141, 2024-04-26T02:06:00.489 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 265718, 2024-04-26T02:06:00.489 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-04-26T02:06:00.489 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:06:00.489 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:06:00.489 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:06:00.489 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-26T02:06:00.489 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:00.489 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-objecter_ops": { 2024-04-26T02:06:00.490 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-04-26T02:06:00.490 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 1024, 2024-04-26T02:06:00.490 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-04-26T02:06:00.490 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 0, 2024-04-26T02:06:00.490 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 0, 2024-04-26T02:06:00.490 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-04-26T02:06:00.490 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 0, 2024-04-26T02:06:00.490 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 141, 2024-04-26T02:06:00.490 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 141, 2024-04-26T02:06:00.490 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 141, 2024-04-26T02:06:00.490 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 141, 2024-04-26T02:06:00.490 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-04-26T02:06:00.490 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:06:00.491 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:06:00.491 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:06:00.491 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-26T02:06:00.491 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:00.491 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-write_buf_throttle": { 2024-04-26T02:06:00.491 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-04-26T02:06:00.491 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 3758096384, 2024-04-26T02:06:00.491 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-04-26T02:06:00.491 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 19, 2024-04-26T02:06:00.491 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 2983, 2024-04-26T02:06:00.491 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-04-26T02:06:00.491 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 19, 2024-04-26T02:06:00.491 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 0, 2024-04-26T02:06:00.492 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 0, 2024-04-26T02:06:00.492 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 3, 2024-04-26T02:06:00.492 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 2983, 2024-04-26T02:06:00.492 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-04-26T02:06:00.492 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:06:00.492 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:06:00.492 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:06:00.492 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-26T02:06:00.492 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:00.492 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-write_buf_throttle-0x55c759eaa500": { 2024-04-26T02:06:00.492 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-04-26T02:06:00.492 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 3758096384, 2024-04-26T02:06:00.492 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-04-26T02:06:00.492 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 109, 2024-04-26T02:06:00.493 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 243833, 2024-04-26T02:06:00.493 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-04-26T02:06:00.493 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 109, 2024-04-26T02:06:00.493 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 0, 2024-04-26T02:06:00.493 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 0, 2024-04-26T02:06:00.493 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 47, 2024-04-26T02:06:00.493 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 243833, 2024-04-26T02:06:00.493 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-04-26T02:06:00.493 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:06:00.493 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:06:00.493 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:06:00.493 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-26T02:06:00.493 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-26T02:06:00.493 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-26T02:06:01.382 INFO:tasks.check_counter:Daemon mds.i mds.exported=3 2024-04-26T02:06:01.382 INFO:tasks.check_counter:Daemon mds.i mds.imported=5 2024-04-26T02:06:01.382 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=12 2024-04-26T02:06:01.382 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=479 2024-04-26T02:06:01.382 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=7 2024-04-26T02:06:01.382 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-04-26T02:06:01.382 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-04-26T02:06:01.382 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=0 2024-04-26T02:06:01.383 DEBUG:tasks.check_counter:Getting stats from l 2024-04-26T02:06:01.383 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-26T02:06:01.624 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T02:06:02.347 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:02 smithi165 ceph-mon[32648]: pgmap v389: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T02:06:02.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:02 smithi151 ceph-mon[28057]: pgmap v389: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T02:06:02.494 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:02 smithi172 ceph-mon[32785]: pgmap v389: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T02:06:03.440 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-26T02:06:03.441 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T02:06:03.441 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 4, 2024-04-26T02:06:03.441 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 4, 2024-04-26T02:06:03.441 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 3040, 2024-04-26T02:06:03.441 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 448, 2024-04-26T02:06:03.441 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 4, 2024-04-26T02:06:03.441 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 0, 2024-04-26T02:06:03.441 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 0.001301673, 2024-04-26T02:06:03.441 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.000034115, 2024-04-26T02:06:03.441 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.000494681, 2024-04-26T02:06:03.441 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T02:06:03.442 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:06:03.442 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 3, 2024-04-26T02:06:03.442 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000731014, 2024-04-26T02:06:03.442 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000243671 2024-04-26T02:06:03.442 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:03.442 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:06:03.442 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:06:03.442 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:06:03.442 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:06:03.442 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:03.442 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-26T02:06:03.442 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 448 2024-04-26T02:06:03.442 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:03.443 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T02:06:03.443 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 291, 2024-04-26T02:06:03.443 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 279, 2024-04-26T02:06:03.443 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 1748000, 2024-04-26T02:06:03.443 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 172224, 2024-04-26T02:06:03.443 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 2, 2024-04-26T02:06:03.443 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 1, 2024-04-26T02:06:03.443 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 0.057247265, 2024-04-26T02:06:03.443 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.017229241, 2024-04-26T02:06:03.443 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.042696304, 2024-04-26T02:06:03.443 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.007009709, 2024-04-26T02:06:03.443 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:06:03.443 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 278, 2024-04-26T02:06:03.443 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.013931906, 2024-04-26T02:06:03.444 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000050114 2024-04-26T02:06:03.444 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:03.444 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:06:03.444 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:06:03.444 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:06:03.444 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:06:03.444 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:03.444 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 1748000, 2024-04-26T02:06:03.444 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 172224 2024-04-26T02:06:03.444 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:03.444 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T02:06:03.444 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 2, 2024-04-26T02:06:03.444 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 214, 2024-04-26T02:06:03.444 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 320, 2024-04-26T02:06:03.444 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 272240, 2024-04-26T02:06:03.445 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 6, 2024-04-26T02:06:03.445 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 3, 2024-04-26T02:06:03.445 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 0.013186513, 2024-04-26T02:06:03.445 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.008824641, 2024-04-26T02:06:03.445 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.001730888, 2024-04-26T02:06:03.445 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T02:06:03.445 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:06:03.445 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 211, 2024-04-26T02:06:03.445 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.027088549, 2024-04-26T02:06:03.445 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000128381 2024-04-26T02:06:03.445 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:03.445 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:06:03.445 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:06:03.445 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:06:03.445 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:06:03.445 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:03.446 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-26T02:06:03.446 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 272240 2024-04-26T02:06:03.446 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:03.446 INFO:teuthology.orchestra.run.smithi172.stdout: "cct": { 2024-04-26T02:06:03.446 INFO:teuthology.orchestra.run.smithi172.stdout: "total_workers": 0, 2024-04-26T02:06:03.446 INFO:teuthology.orchestra.run.smithi172.stdout: "unhealthy_workers": 0 2024-04-26T02:06:03.446 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:03.446 INFO:teuthology.orchestra.run.smithi172.stdout: "mempool": { 2024-04-26T02:06:03.446 INFO:teuthology.orchestra.run.smithi172.stdout: "bloom_filter_bytes": 0, 2024-04-26T02:06:03.446 INFO:teuthology.orchestra.run.smithi172.stdout: "bloom_filter_items": 0, 2024-04-26T02:06:03.446 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T02:06:03.446 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_alloc_items": 0, 2024-04-26T02:06:03.446 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T02:06:03.446 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_data_items": 0, 2024-04-26T02:06:03.446 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T02:06:03.447 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T02:06:03.447 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T02:06:03.447 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T02:06:03.447 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T02:06:03.447 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_other_items": 0, 2024-04-26T02:06:03.447 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T02:06:03.447 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T02:06:03.447 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_extent_bytes": 0, 2024-04-26T02:06:03.447 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_extent_items": 0, 2024-04-26T02:06:03.447 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_blob_bytes": 0, 2024-04-26T02:06:03.447 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_blob_items": 0, 2024-04-26T02:06:03.447 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T02:06:03.447 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T02:06:03.447 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T02:06:03.448 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T02:06:03.448 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T02:06:03.448 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_fsck_items": 0, 2024-04-26T02:06:03.448 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_txc_bytes": 0, 2024-04-26T02:06:03.448 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_txc_items": 0, 2024-04-26T02:06:03.448 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T02:06:03.448 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T02:06:03.448 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_bytes": 0, 2024-04-26T02:06:03.448 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_items": 0, 2024-04-26T02:06:03.448 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_bytes": 0, 2024-04-26T02:06:03.448 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_items": 0, 2024-04-26T02:06:03.448 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T02:06:03.448 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_reader_items": 0, 2024-04-26T02:06:03.449 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T02:06:03.449 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_writer_items": 0, 2024-04-26T02:06:03.449 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_anon_bytes": 148732, 2024-04-26T02:06:03.449 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_anon_items": 31, 2024-04-26T02:06:03.449 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_meta_bytes": 0, 2024-04-26T02:06:03.449 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_meta_items": 0, 2024-04-26T02:06:03.449 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_bytes": 0, 2024-04-26T02:06:03.449 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_items": 0, 2024-04-26T02:06:03.449 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_mapbl_bytes": 0, 2024-04-26T02:06:03.449 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_mapbl_items": 0, 2024-04-26T02:06:03.449 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_pglog_bytes": 0, 2024-04-26T02:06:03.449 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_pglog_items": 0, 2024-04-26T02:06:03.449 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_bytes": 0, 2024-04-26T02:06:03.449 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_items": 0, 2024-04-26T02:06:03.449 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T02:06:03.450 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_mapping_items": 0, 2024-04-26T02:06:03.450 INFO:teuthology.orchestra.run.smithi172.stdout: "pgmap_bytes": 0, 2024-04-26T02:06:03.450 INFO:teuthology.orchestra.run.smithi172.stdout: "pgmap_items": 0, 2024-04-26T02:06:03.450 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_co_bytes": 928, 2024-04-26T02:06:03.450 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_co_items": 2, 2024-04-26T02:06:03.450 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_1_bytes": 0, 2024-04-26T02:06:03.450 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_1_items": 0, 2024-04-26T02:06:03.450 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_2_bytes": 0, 2024-04-26T02:06:03.450 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_2_items": 0 2024-04-26T02:06:03.450 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-26T02:06:03.450 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T02:06:03.450 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-04-26T02:06:03.450 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 104857600, 2024-04-26T02:06:03.451 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-04-26T02:06:03.451 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 293, 2024-04-26T02:06:03.451 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 1706920, 2024-04-26T02:06:03.451 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-04-26T02:06:03.451 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 293, 2024-04-26T02:06:03.451 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 0, 2024-04-26T02:06:03.451 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 0, 2024-04-26T02:06:03.451 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 293, 2024-04-26T02:06:03.451 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 1706920, 2024-04-26T02:06:03.451 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-04-26T02:06:03.451 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-04-26T02:06:03.451 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-04-26T02:06:03.451 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-04-26T02:06:03.451 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-26T02:06:03.452 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-26T02:06:03.452 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-26T02:06:04.214 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-04-26T02:06:04.214 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-04-26T02:06:04.214 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-04-26T02:06:04.214 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-04-26T02:06:04.214 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-04-26T02:06:04.214 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-04-26T02:06:04.214 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-04-26T02:06:04.214 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.l 2024-04-26T02:06:04.215 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-26T02:06:04.225 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-26T02:06:04.226 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-26T02:06:04.226 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T02:06:04.287 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-04-26T02:06:04.287 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T02:06:04.288 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:04 smithi172 ceph-mon[32785]: pgmap v390: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-26T02:06:04.341 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph osd blocklist ls 2024-04-26T02:06:04.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:04 smithi165 ceph-mon[32648]: pgmap v390: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-26T02:06:04.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:04 smithi151 ceph-mon[28057]: pgmap v390: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-26T02:06:04.790 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:06:06.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:06 smithi165 ceph-mon[32648]: pgmap v391: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-26T02:06:06.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:06 smithi151 ceph-mon[28057]: pgmap v391: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-26T02:06:06.413 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:06 smithi172 ceph-mon[32785]: pgmap v391: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-26T02:06:07.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:07.217+0000 7fdc67200700 1 -- 172.21.15.151:0/1455318190 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdc60105870 msgr2=0x7fdc601098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:06:07.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:07.217+0000 7fdc67200700 1 --2- 172.21.15.151:0/1455318190 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdc60105870 0x7fdc601098c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c0099c0 tx=0x7fdc4c009cd0 comp rx=0 tx=0).stop 2024-04-26T02:06:07.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:07.217+0000 7fdc67200700 1 -- 172.21.15.151:0/1455318190 shutdown_connections 2024-04-26T02:06:07.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:07.217+0000 7fdc67200700 1 --2- 172.21.15.151:0/1455318190 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdc60109fa0 0x7fdc60111b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:07.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:07.217+0000 7fdc67200700 1 --2- 172.21.15.151:0/1455318190 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdc60104ec0 0x7fdc601052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:07.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:07.217+0000 7fdc67200700 1 --2- 172.21.15.151:0/1455318190 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdc60105870 0x7fdc601098c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:07.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:07.217+0000 7fdc67200700 1 -- 172.21.15.151:0/1455318190 >> 172.21.15.151:0/1455318190 conn(0x7fdc60100750 msgr2=0x7fdc60102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:06:07.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:07.217+0000 7fdc67200700 1 -- 172.21.15.151:0/1455318190 shutdown_connections 2024-04-26T02:06:07.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:07.217+0000 7fdc67200700 1 -- 172.21.15.151:0/1455318190 wait complete. 2024-04-26T02:06:07.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:07.218+0000 7fdc67200700 1 Processor -- start 2024-04-26T02:06:07.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:07.219+0000 7fdc67200700 1 -- start start 2024-04-26T02:06:07.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:07.219+0000 7fdc67200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdc60104ec0 0x7fdc6019b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:06:07.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:07.219+0000 7fdc67200700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdc60105870 0x7fdc6019b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:06:07.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:07.219+0000 7fdc67200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdc60109fa0 0x7fdc6019fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:06:07.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:07.219+0000 7fdc67200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdc60114e60 con 0x7fdc60104ec0 2024-04-26T02:06:07.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:07.219+0000 7fdc67200700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fdc60114ce0 con 0x7fdc60109fa0 2024-04-26T02:06:07.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:07.219+0000 7fdc67200700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdc60114fe0 con 0x7fdc60105870 2024-04-26T02:06:07.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:07.219+0000 7fdc65200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdc60109fa0 0x7fdc6019fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:06:07.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:07.219+0000 7fdc65200700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdc60109fa0 0x7fdc6019fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:52884/0 (socket says 172.21.15.151:52884) 2024-04-26T02:06:07.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:07.219+0000 7fdc5f400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdc60105870 0x7fdc6019b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:06:07.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:07.219+0000 7fdc65200700 1 -- 172.21.15.151:0/1255046604 learned_addr learned my addr 172.21.15.151:0/1255046604 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:06:07.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:07.219+0000 7fdc5fe00700 1 --2- 172.21.15.151:0/1255046604 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdc60104ec0 0x7fdc6019b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:06:07.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:07.220+0000 7fdc65200700 1 -- 172.21.15.151:0/1255046604 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdc60105870 msgr2=0x7fdc6019b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:06:07.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:07.220+0000 7fdc65200700 1 --2- 172.21.15.151:0/1255046604 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdc60105870 0x7fdc6019b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:07.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:07.220+0000 7fdc65200700 1 -- 172.21.15.151:0/1255046604 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdc60104ec0 msgr2=0x7fdc6019b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:06:07.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:07.220+0000 7fdc65200700 1 --2- 172.21.15.151:0/1255046604 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdc60104ec0 0x7fdc6019b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:07.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:07.220+0000 7fdc65200700 1 -- 172.21.15.151:0/1255046604 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc601a0340 con 0x7fdc60109fa0 2024-04-26T02:06:07.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:07.220+0000 7fdc65200700 1 --2- 172.21.15.151:0/1255046604 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdc60109fa0 0x7fdc6019fc00 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fdc5800ef30 tx=0x7fdc5800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:06:07.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:07.220+0000 7fdc5cc00700 1 -- 172.21.15.151:0/1255046604 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc580092e0 con 0x7fdc60109fa0 2024-04-26T02:06:07.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:07.220+0000 7fdc5cc00700 1 -- 172.21.15.151:0/1255046604 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc58004bb0 con 0x7fdc60109fa0 2024-04-26T02:06:07.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:07.220+0000 7fdc67200700 1 -- 172.21.15.151:0/1255046604 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc601a05d0 con 0x7fdc60109fa0 2024-04-26T02:06:07.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:07.220+0000 7fdc5cc00700 1 -- 172.21.15.151:0/1255046604 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc58004020 con 0x7fdc60109fa0 2024-04-26T02:06:07.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:07.220+0000 7fdc67200700 1 -- 172.21.15.151:0/1255046604 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc601a7eb0 con 0x7fdc60109fa0 2024-04-26T02:06:07.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:07.221+0000 7fdc67200700 1 -- 172.21.15.151:0/1255046604 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc60069730 con 0x7fdc60109fa0 2024-04-26T02:06:07.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:07.223+0000 7fdc5cc00700 1 -- 172.21.15.151:0/1255046604 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7fdc5801b030 con 0x7fdc60109fa0 2024-04-26T02:06:07.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:07.225+0000 7fdc5cc00700 1 --2- 172.21.15.151:0/1255046604 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fdc440a3e30 0x7fdc440a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:06:07.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:07.225+0000 7fdc5cc00700 1 -- 172.21.15.151:0/1255046604 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8411+0+0 (secure 0 0 0) 0x7fdc580d0150 con 0x7fdc60109fa0 2024-04-26T02:06:07.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:07.225+0000 7fdc5fe00700 1 --2- 172.21.15.151:0/1255046604 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fdc440a3e30 0x7fdc440a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:06:07.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:07.226+0000 7fdc5fe00700 1 --2- 172.21.15.151:0/1255046604 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fdc440a3e30 0x7fdc440a62f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdc50009c60 tx=0x7fdc50009260 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:06:07.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:07.229+0000 7fdc5cc00700 1 -- 172.21.15.151:0/1255046604 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdc58022020 con 0x7fdc60109fa0 2024-04-26T02:06:07.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:07.469+0000 7fdc67200700 1 -- 172.21.15.151:0/1255046604 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fdc6002df80 con 0x7fdc60109fa0 2024-04-26T02:06:07.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:07.470+0000 7fdc5cc00700 1 -- 172.21.15.151:0/1255046604 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v107) ==== 81+0+953 (secure 0 0 0) 0x7fdc58010040 con 0x7fdc60109fa0 2024-04-26T02:06:07.469 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.15.151:6800/2251256505 2024-04-27T01:53:36.653535+0000 2024-04-26T02:06:07.469 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.15.151:0/2357384001 2024-04-27T01:53:36.653535+0000 2024-04-26T02:06:07.469 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.15.151:6801/796314699 2024-04-27T01:41:25.234322+0000 2024-04-26T02:06:07.469 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.15.151:6800/796314699 2024-04-27T01:41:25.234322+0000 2024-04-26T02:06:07.469 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.15.151:0/4066781930 2024-04-27T01:53:36.653535+0000 2024-04-26T02:06:07.470 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.15.151:0/1173525554 2024-04-27T01:41:25.234322+0000 2024-04-26T02:06:07.470 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.15.151:0/1342496801 2024-04-27T01:40:50.492554+0000 2024-04-26T02:06:07.470 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.15.151:0/755039608 2024-04-27T01:53:36.653535+0000 2024-04-26T02:06:07.470 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.15.151:0/11665926 2024-04-27T01:40:50.492554+0000 2024-04-26T02:06:07.470 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.15.151:0/1765448952 2024-04-27T01:40:50.492554+0000 2024-04-26T02:06:07.470 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.15.151:0/3463483606 2024-04-27T01:53:36.653535+0000 2024-04-26T02:06:07.470 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.15.151:6801/2251256505 2024-04-27T01:53:36.653535+0000 2024-04-26T02:06:07.470 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.15.151:0/46007430 2024-04-27T01:41:25.234322+0000 2024-04-26T02:06:07.470 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.15.151:6800/244725060 2024-04-27T01:40:50.492554+0000 2024-04-26T02:06:07.470 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.15.151:0/3703862543 2024-04-27T01:41:25.234322+0000 2024-04-26T02:06:07.470 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.15.151:6801/244725060 2024-04-27T01:40:50.492554+0000 2024-04-26T02:06:07.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:07.473+0000 7fdc67200700 1 -- 172.21.15.151:0/1255046604 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fdc440a3e30 msgr2=0x7fdc440a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:06:07.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:07.473+0000 7fdc67200700 1 --2- 172.21.15.151:0/1255046604 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fdc440a3e30 0x7fdc440a62f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdc50009c60 tx=0x7fdc50009260 comp rx=0 tx=0).stop 2024-04-26T02:06:07.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:07.473+0000 7fdc67200700 1 -- 172.21.15.151:0/1255046604 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdc60109fa0 msgr2=0x7fdc6019fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:06:07.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:07.473+0000 7fdc67200700 1 --2- 172.21.15.151:0/1255046604 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdc60109fa0 0x7fdc6019fc00 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fdc5800ef30 tx=0x7fdc5800c3a0 comp rx=0 tx=0).stop 2024-04-26T02:06:07.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:07.473+0000 7fdc67200700 1 -- 172.21.15.151:0/1255046604 shutdown_connections 2024-04-26T02:06:07.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:07.473+0000 7fdc67200700 1 --2- 172.21.15.151:0/1255046604 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fdc440a3e30 0x7fdc440a62f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:07.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:07.473+0000 7fdc67200700 1 --2- 172.21.15.151:0/1255046604 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdc60109fa0 0x7fdc6019fc00 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:07.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:07.473+0000 7fdc67200700 1 --2- 172.21.15.151:0/1255046604 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdc60104ec0 0x7fdc6019b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:07.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:07.473+0000 7fdc67200700 1 --2- 172.21.15.151:0/1255046604 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdc60105870 0x7fdc6019b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:07.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:07.473+0000 7fdc67200700 1 -- 172.21.15.151:0/1255046604 >> 172.21.15.151:0/1255046604 conn(0x7fdc60100750 msgr2=0x7fdc60102a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:06:07.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:07.473+0000 7fdc67200700 1 -- 172.21.15.151:0/1255046604 shutdown_connections 2024-04-26T02:06:07.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:07.474+0000 7fdc67200700 1 -- 172.21.15.151:0/1255046604 wait complete. 2024-04-26T02:06:07.473 INFO:teuthology.orchestra.run.smithi151.stderr:listed 16 entries 2024-04-26T02:06:08.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:08 smithi151 ceph-mon[28057]: pgmap v392: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-26T02:06:08.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:08 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/1255046604' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T02:06:08.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:08 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T02:06:08.328 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T02:06:08.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T02:06:08.328 DEBUG:teuthology.orchestra.run.smithi151:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T02:06:08.460 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi151.front.sepia.ceph.com 2024-04-26T02:06:08.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T02:06:08.460 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T02:06:08.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T02:06:08.535 DEBUG:teuthology.orchestra.run.smithi151:> ip netns list 2024-04-26T02:06:08.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:08 smithi172 ceph-mon[32785]: pgmap v392: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-26T02:06:08.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:08 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/1255046604' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T02:06:08.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:08 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T02:06:08.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:08 smithi165 ceph-mon[32648]: pgmap v392: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-26T02:06:08.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:08 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/1255046604' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T02:06:08.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:08 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T02:06:08.623 INFO:teuthology.orchestra.run.smithi151.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T02:06:08.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T02:06:08.623 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T02:06:08.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T02:06:08.724 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip link delete ceph-brx 2024-04-26T02:06:08.832 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-26T02:06:08.843 INFO:tasks.cephadm:Teardown begin 2024-04-26T02:06:08.843 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T02:06:08.844 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T02:06:09.335 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:06:09.373 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:09 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:06:09.373 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:09 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:06:09.373 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:09 smithi151 ceph-mon[28057]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T02:06:09.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:09 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:06:09.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:09 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:06:09.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:09 smithi172 ceph-mon[32785]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T02:06:09.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:09 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:06:09.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:09 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:06:09.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:09 smithi165 ceph-mon[32648]: from='mgr.25025 ' entity='mgr.x' 2024-04-26T02:06:10.373 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:10 smithi151 ceph-mon[28057]: pgmap v393: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-26T02:06:10.373 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:10 smithi151 ceph-mon[28057]: osdmap e108: 12 total, 12 up, 12 in 2024-04-26T02:06:10.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:10 smithi172 ceph-mon[32785]: pgmap v393: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-26T02:06:10.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:10 smithi172 ceph-mon[32785]: osdmap e108: 12 total, 12 up, 12 in 2024-04-26T02:06:10.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:10 smithi165 ceph-mon[32648]: pgmap v393: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-26T02:06:10.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:10 smithi165 ceph-mon[32648]: osdmap e108: 12 total, 12 up, 12 in 2024-04-26T02:06:11.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:11.817+0000 7f1a87e00700 1 -- 172.21.15.151:0/97122529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a8810f270 msgr2=0x7f1a88111660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:06:11.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:11.817+0000 7f1a87e00700 1 --2- 172.21.15.151:0/97122529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a8810f270 0x7f1a88111660 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1a8000b0a0 tx=0x7f1a8000b3b0 comp rx=0 tx=0).stop 2024-04-26T02:06:11.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:11.817+0000 7f1a87e00700 1 -- 172.21.15.151:0/97122529 shutdown_connections 2024-04-26T02:06:11.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:11.817+0000 7f1a87e00700 1 --2- 172.21.15.151:0/97122529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a8810f270 0x7f1a88111660 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:11.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:11.817+0000 7f1a87e00700 1 --2- 172.21.15.151:0/97122529 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1a881016c0 0x7f1a88101aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:11.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:11.817+0000 7f1a87e00700 1 --2- 172.21.15.151:0/97122529 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1a88102070 0x7f1a8810ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:11.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:11.817+0000 7f1a87e00700 1 -- 172.21.15.151:0/97122529 >> 172.21.15.151:0/97122529 conn(0x7f1a880fd360 msgr2=0x7f1a880ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:06:11.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:11.817+0000 7f1a87e00700 1 -- 172.21.15.151:0/97122529 shutdown_connections 2024-04-26T02:06:11.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:11.817+0000 7f1a87e00700 1 -- 172.21.15.151:0/97122529 wait complete. 2024-04-26T02:06:11.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:11.818+0000 7f1a87e00700 1 Processor -- start 2024-04-26T02:06:11.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:11.818+0000 7f1a87e00700 1 -- start start 2024-04-26T02:06:11.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:11.819+0000 7f1a87e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a881016c0 0x7f1a8819b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:06:11.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:11.819+0000 7f1a87e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1a88102070 0x7f1a8819b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:06:11.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:11.819+0000 7f1a87e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1a8810f270 0x7f1a8819fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:06:11.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:11.819+0000 7f1a87e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1a88114d70 con 0x7f1a881016c0 2024-04-26T02:06:11.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:11.819+0000 7f1a87e00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1a88114bf0 con 0x7f1a88102070 2024-04-26T02:06:11.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:11.819+0000 7f1a87e00700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1a88114ef0 con 0x7f1a8810f270 2024-04-26T02:06:11.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:11.819+0000 7f1a86a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a881016c0 0x7f1a8819b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:06:11.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:11.819+0000 7f1a86a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a881016c0 0x7f1a8819b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:51242/0 (socket says 172.21.15.151:51242) 2024-04-26T02:06:11.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:11.819+0000 7f1a86a00700 1 -- 172.21.15.151:0/2708810625 learned_addr learned my addr 172.21.15.151:0/2708810625 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:06:11.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:11.819+0000 7f1a86000700 1 --2- 172.21.15.151:0/2708810625 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1a88102070 0x7f1a8819b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:06:11.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:11.819+0000 7f1a87400700 1 --2- 172.21.15.151:0/2708810625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1a8810f270 0x7f1a8819fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:06:11.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:11.820+0000 7f1a86a00700 1 -- 172.21.15.151:0/2708810625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1a8810f270 msgr2=0x7f1a8819fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:06:11.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:11.820+0000 7f1a86a00700 1 --2- 172.21.15.151:0/2708810625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1a8810f270 0x7f1a8819fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:11.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:11.820+0000 7f1a86a00700 1 -- 172.21.15.151:0/2708810625 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1a88102070 msgr2=0x7f1a8819b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:06:11.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:11.820+0000 7f1a86a00700 1 --2- 172.21.15.151:0/2708810625 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1a88102070 0x7f1a8819b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:11.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:11.820+0000 7f1a86a00700 1 -- 172.21.15.151:0/2708810625 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a881a0220 con 0x7f1a881016c0 2024-04-26T02:06:11.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:11.820+0000 7f1a86a00700 1 --2- 172.21.15.151:0/2708810625 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a881016c0 0x7f1a8819b260 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1a7800d980 tx=0x7f1a7800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:06:11.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:11.820+0000 7f1a7f400700 1 -- 172.21.15.151:0/2708810625 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a78017070 con 0x7f1a881016c0 2024-04-26T02:06:11.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:11.820+0000 7f1a7f400700 1 -- 172.21.15.151:0/2708810625 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a78005020 con 0x7f1a881016c0 2024-04-26T02:06:11.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:11.820+0000 7f1a7f400700 1 -- 172.21.15.151:0/2708810625 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a7800fa10 con 0x7f1a881016c0 2024-04-26T02:06:11.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:11.820+0000 7f1a87e00700 1 -- 172.21.15.151:0/2708810625 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a881a0510 con 0x7f1a881016c0 2024-04-26T02:06:11.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:11.821+0000 7f1a87e00700 1 -- 172.21.15.151:0/2708810625 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a881a7d90 con 0x7f1a881016c0 2024-04-26T02:06:11.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:11.822+0000 7f1a87e00700 1 -- 172.21.15.151:0/2708810625 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a88069730 con 0x7f1a881016c0 2024-04-26T02:06:11.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:11.824+0000 7f1a7f400700 1 -- 172.21.15.151:0/2708810625 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f1a78004a30 con 0x7f1a881016c0 2024-04-26T02:06:11.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:11.825+0000 7f1a7f400700 1 --2- 172.21.15.151:0/2708810625 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f1a6c0a3db0 0x7f1a6c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:06:11.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:11.825+0000 7f1a7f400700 1 -- 172.21.15.151:0/2708810625 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8499+0+0 (secure 0 0 0) 0x7f1a780d0700 con 0x7f1a881016c0 2024-04-26T02:06:11.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:11.825+0000 7f1a86000700 1 --2- 172.21.15.151:0/2708810625 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f1a6c0a3db0 0x7f1a6c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:06:11.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:11.826+0000 7f1a86000700 1 --2- 172.21.15.151:0/2708810625 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f1a6c0a3db0 0x7f1a6c0a6270 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1a881030a0 tx=0x7f1a7400b410 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:06:11.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:11.829+0000 7f1a7f400700 1 -- 172.21.15.151:0/2708810625 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1a78020020 con 0x7f1a881016c0 2024-04-26T02:06:12.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:12.101+0000 7f1a87e00700 1 -- 172.21.15.151:0/2708810625 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1a8802df80 con 0x7f1a881016c0 2024-04-26T02:06:12.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:12.102+0000 7f1a7f400700 1 -- 172.21.15.151:0/2708810625 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11891 (secure 0 0 0) 0x7f1a7809cd50 con 0x7f1a881016c0 2024-04-26T02:06:12.102 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T02:06:12.102 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2937864097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2937864097},{"type":"v1","addr":"172.21.15.151:6835","nonce":2937864097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14373,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/2947090802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2947090802},{"type":"v1","addr":"172.21.15.151:6837","nonce":2947090802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/1356563280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1356563280},{"type":"v1","addr":"172.21.15.151:6839","nonce":1356563280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6841/1075240552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1075240552},{"type":"v1","addr":"172.21.15.151:6841","nonce":1075240552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/504675912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":504675912},{"type":"v1","addr":"172.21.15.165:6833","nonce":504675912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1207580015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1207580015},{"type":"v1","addr":"172.21.15.172:6833","nonce":1207580015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:49:40.746876+0000","modified":"2024-04-26T02:05:15.281789+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24505,"mds_2":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/3598138294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3598138294},{"type":"v1","addr":"172.21.15.165:6835","nonce":3598138294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/1847859054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":1847859054},{"type":"v1","addr":"172.21.15.165:6837","nonce":1847859054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.172:6837/55972537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":55972537},{"type":"v1","addr":"172.21.15.172:6837","nonce":55972537}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.172:6839/1242302810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1242302810},{"type":"v1","addr":"172.21.15.172:6839","nonce":1242302810}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/1969716254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1969716254},{"type":"v1","addr":"172.21.15.165:6839","nonce":1969716254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26482":{"gid":26482,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/1218148590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":1218148590},{"type":"v1","addr":"172.21.15.172:6835","nonce":1218148590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24509,24493,24505]},"id":1}]} 2024-04-26T02:06:12.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:12.105+0000 7f1a87e00700 1 -- 172.21.15.151:0/2708810625 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f1a6c0a3db0 msgr2=0x7f1a6c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:06:12.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:12.105+0000 7f1a87e00700 1 --2- 172.21.15.151:0/2708810625 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f1a6c0a3db0 0x7f1a6c0a6270 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1a881030a0 tx=0x7f1a7400b410 comp rx=0 tx=0).stop 2024-04-26T02:06:12.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:12.105+0000 7f1a87e00700 1 -- 172.21.15.151:0/2708810625 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a881016c0 msgr2=0x7f1a8819b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:06:12.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:12.105+0000 7f1a87e00700 1 --2- 172.21.15.151:0/2708810625 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a881016c0 0x7f1a8819b260 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1a7800d980 tx=0x7f1a7800dc90 comp rx=0 tx=0).stop 2024-04-26T02:06:12.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:12.106+0000 7f1a87e00700 1 -- 172.21.15.151:0/2708810625 shutdown_connections 2024-04-26T02:06:12.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:12.106+0000 7f1a87e00700 1 --2- 172.21.15.151:0/2708810625 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f1a6c0a3db0 0x7f1a6c0a6270 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:12.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:12.106+0000 7f1a87e00700 1 --2- 172.21.15.151:0/2708810625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1a8810f270 0x7f1a8819fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:12.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:12.106+0000 7f1a87e00700 1 --2- 172.21.15.151:0/2708810625 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a881016c0 0x7f1a8819b260 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:12.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:12.106+0000 7f1a87e00700 1 --2- 172.21.15.151:0/2708810625 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1a88102070 0x7f1a8819b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:12.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:12.106+0000 7f1a87e00700 1 -- 172.21.15.151:0/2708810625 >> 172.21.15.151:0/2708810625 conn(0x7f1a880fd360 msgr2=0x7f1a88110540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:06:12.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:12.106+0000 7f1a87e00700 1 -- 172.21.15.151:0/2708810625 shutdown_connections 2024-04-26T02:06:12.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:12.106+0000 7f1a87e00700 1 -- 172.21.15.151:0/2708810625 wait complete. 2024-04-26T02:06:12.106 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 39 2024-04-26T02:06:12.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:12 smithi151 ceph-mon[28057]: pgmap v395: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-26T02:06:12.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:12 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/2708810625' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:06:12.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:12 smithi172 ceph-mon[32785]: pgmap v395: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-26T02:06:12.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:12 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/2708810625' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:06:12.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:12 smithi165 ceph-mon[32648]: pgmap v395: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-26T02:06:12.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:12 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/2708810625' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:06:12.956 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T02:06:13.409 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:06:14.211 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:14 smithi151 ceph-mon[28057]: pgmap v396: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 921 B/s wr, 7 op/s 2024-04-26T02:06:14.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:14 smithi172 ceph-mon[32785]: pgmap v396: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 921 B/s wr, 7 op/s 2024-04-26T02:06:14.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:14 smithi165 ceph-mon[32648]: pgmap v396: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 921 B/s wr, 7 op/s 2024-04-26T02:06:15.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:15.811+0000 7fcd43000700 1 -- 172.21.15.151:0/18012838 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd3c102af0 msgr2=0x7fcd3c102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:06:15.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:15.811+0000 7fcd43000700 1 --2- 172.21.15.151:0/18012838 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd3c102af0 0x7fcd3c102f70 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fcd240099c0 tx=0x7fcd24009cd0 comp rx=0 tx=0).stop 2024-04-26T02:06:15.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:15.811+0000 7fcd43000700 1 -- 172.21.15.151:0/18012838 shutdown_connections 2024-04-26T02:06:15.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:15.811+0000 7fcd43000700 1 --2- 172.21.15.151:0/18012838 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd3c1034b0 0x7fcd3c10f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:15.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:15.811+0000 7fcd43000700 1 --2- 172.21.15.151:0/18012838 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd3c107940 0x7fcd3c107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:15.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:15.811+0000 7fcd43000700 1 --2- 172.21.15.151:0/18012838 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd3c102af0 0x7fcd3c102f70 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:15.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:15.811+0000 7fcd43000700 1 -- 172.21.15.151:0/18012838 >> 172.21.15.151:0/18012838 conn(0x7fcd3c079ba0 msgr2=0x7fcd3c079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:06:15.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:15.811+0000 7fcd43000700 1 -- 172.21.15.151:0/18012838 shutdown_connections 2024-04-26T02:06:15.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:15.811+0000 7fcd43000700 1 -- 172.21.15.151:0/18012838 wait complete. 2024-04-26T02:06:15.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:15.812+0000 7fcd43000700 1 Processor -- start 2024-04-26T02:06:15.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:15.812+0000 7fcd43000700 1 -- start start 2024-04-26T02:06:15.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:15.812+0000 7fcd43000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd3c102af0 0x7fcd3c19b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:06:15.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:15.813+0000 7fcd43000700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd3c1034b0 0x7fcd3c19b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:06:15.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:15.813+0000 7fcd43000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd3c107940 0x7fcd3c19fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:06:15.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:15.813+0000 7fcd43000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3c112c30 con 0x7fcd3c107940 2024-04-26T02:06:15.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:15.813+0000 7fcd43000700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3c112ab0 con 0x7fcd3c102af0 2024-04-26T02:06:15.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:15.813+0000 7fcd43000700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3c112db0 con 0x7fcd3c1034b0 2024-04-26T02:06:15.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:15.813+0000 7fcd3be00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd3c102af0 0x7fcd3c19b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:06:15.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:15.813+0000 7fcd33400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd3c1034b0 0x7fcd3c19b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:06:15.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:15.813+0000 7fcd41000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd3c107940 0x7fcd3c19fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:06:15.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:15.813+0000 7fcd3be00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd3c102af0 0x7fcd3c19b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.151:40320/0 (socket says 172.21.15.151:40320) 2024-04-26T02:06:15.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:15.813+0000 7fcd33400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd3c1034b0 0x7fcd3c19b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:37750/0 (socket says 172.21.15.151:37750) 2024-04-26T02:06:15.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:15.813+0000 7fcd3be00700 1 -- 172.21.15.151:0/3756864361 learned_addr learned my addr 172.21.15.151:0/3756864361 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:06:15.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:15.813+0000 7fcd3be00700 1 -- 172.21.15.151:0/3756864361 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd3c1034b0 msgr2=0x7fcd3c19b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:06:15.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:15.813+0000 7fcd3be00700 1 --2- 172.21.15.151:0/3756864361 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd3c1034b0 0x7fcd3c19b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:15.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:15.813+0000 7fcd3be00700 1 -- 172.21.15.151:0/3756864361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd3c107940 msgr2=0x7fcd3c19fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:06:15.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:15.813+0000 7fcd3be00700 1 --2- 172.21.15.151:0/3756864361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd3c107940 0x7fcd3c19fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:15.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:15.813+0000 7fcd3be00700 1 -- 172.21.15.151:0/3756864361 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd3c1a02d0 con 0x7fcd3c102af0 2024-04-26T02:06:15.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:15.814+0000 7fcd41000700 1 --2- 172.21.15.151:0/3756864361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd3c107940 0x7fcd3c19fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:06:15.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:15.814+0000 7fcd3be00700 1 --2- 172.21.15.151:0/3756864361 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd3c102af0 0x7fcd3c19b2c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c00c370 tx=0x7fcd2c00c680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:06:15.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:15.814+0000 7fcd39600700 1 -- 172.21.15.151:0/3756864361 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c0092e0 con 0x7fcd3c102af0 2024-04-26T02:06:15.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:15.814+0000 7fcd39600700 1 -- 172.21.15.151:0/3756864361 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd2c00cbe0 con 0x7fcd3c102af0 2024-04-26T02:06:15.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:15.814+0000 7fcd43000700 1 -- 172.21.15.151:0/3756864361 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd3c1a0560 con 0x7fcd3c102af0 2024-04-26T02:06:15.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:15.814+0000 7fcd39600700 1 -- 172.21.15.151:0/3756864361 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c0046b0 con 0x7fcd3c102af0 2024-04-26T02:06:15.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:15.814+0000 7fcd43000700 1 -- 172.21.15.151:0/3756864361 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd3c1a7e40 con 0x7fcd3c102af0 2024-04-26T02:06:15.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:15.815+0000 7fcd43000700 1 -- 172.21.15.151:0/3756864361 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd3c069730 con 0x7fcd3c102af0 2024-04-26T02:06:15.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:15.818+0000 7fcd39600700 1 -- 172.21.15.151:0/3756864361 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7fcd2c01a020 con 0x7fcd3c102af0 2024-04-26T02:06:15.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:15.819+0000 7fcd39600700 1 --2- 172.21.15.151:0/3756864361 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fcd200a3e30 0x7fcd200a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:06:15.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:15.819+0000 7fcd39600700 1 -- 172.21.15.151:0/3756864361 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8499+0+0 (secure 0 0 0) 0x7fcd2c0d0770 con 0x7fcd3c102af0 2024-04-26T02:06:15.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:15.819+0000 7fcd33400700 1 --2- 172.21.15.151:0/3756864361 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fcd200a3e30 0x7fcd200a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:06:15.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:15.820+0000 7fcd33400700 1 --2- 172.21.15.151:0/3756864361 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fcd200a3e30 0x7fcd200a62f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c10fd80 tx=0x7fcd24005fb0 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:06:15.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:15.821+0000 7fcd39600700 1 -- 172.21.15.151:0/3756864361 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcd2c010040 con 0x7fcd3c102af0 2024-04-26T02:06:16.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:16.095+0000 7fcd43000700 1 -- 172.21.15.151:0/3756864361 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcd3c1a8010 con 0x7fcd3c102af0 2024-04-26T02:06:16.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:16.097+0000 7fcd39600700 1 -- 172.21.15.151:0/3756864361 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11891 (secure 0 0 0) 0x7fcd2c09cdc0 con 0x7fcd3c102af0 2024-04-26T02:06:16.097 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T02:06:16.097 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2937864097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2937864097},{"type":"v1","addr":"172.21.15.151:6835","nonce":2937864097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14373,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/2947090802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2947090802},{"type":"v1","addr":"172.21.15.151:6837","nonce":2947090802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/1356563280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1356563280},{"type":"v1","addr":"172.21.15.151:6839","nonce":1356563280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6841/1075240552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1075240552},{"type":"v1","addr":"172.21.15.151:6841","nonce":1075240552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/504675912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":504675912},{"type":"v1","addr":"172.21.15.165:6833","nonce":504675912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1207580015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1207580015},{"type":"v1","addr":"172.21.15.172:6833","nonce":1207580015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:49:40.746876+0000","modified":"2024-04-26T02:05:15.281789+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24509,"mds_1":24505,"mds_2":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/3598138294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3598138294},{"type":"v1","addr":"172.21.15.165:6835","nonce":3598138294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/1847859054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":1847859054},{"type":"v1","addr":"172.21.15.165:6837","nonce":1847859054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.172:6837/55972537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":55972537},{"type":"v1","addr":"172.21.15.172:6837","nonce":55972537}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.172:6839/1242302810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1242302810},{"type":"v1","addr":"172.21.15.172:6839","nonce":1242302810}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.165:6839/1969716254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1969716254},{"type":"v1","addr":"172.21.15.165:6839","nonce":1969716254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26482":{"gid":26482,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/1218148590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":1218148590},{"type":"v1","addr":"172.21.15.172:6835","nonce":1218148590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24509,24493,24505]},"id":1}]} 2024-04-26T02:06:16.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:16.100+0000 7fcd43000700 1 -- 172.21.15.151:0/3756864361 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fcd200a3e30 msgr2=0x7fcd200a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:06:16.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:16.100+0000 7fcd43000700 1 --2- 172.21.15.151:0/3756864361 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fcd200a3e30 0x7fcd200a62f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c10fd80 tx=0x7fcd24005fb0 comp rx=0 tx=0).stop 2024-04-26T02:06:16.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:16.100+0000 7fcd43000700 1 -- 172.21.15.151:0/3756864361 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd3c102af0 msgr2=0x7fcd3c19b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:06:16.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:16.100+0000 7fcd43000700 1 --2- 172.21.15.151:0/3756864361 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd3c102af0 0x7fcd3c19b2c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c00c370 tx=0x7fcd2c00c680 comp rx=0 tx=0).stop 2024-04-26T02:06:16.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:16.100+0000 7fcd43000700 1 -- 172.21.15.151:0/3756864361 shutdown_connections 2024-04-26T02:06:16.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:16.100+0000 7fcd43000700 1 --2- 172.21.15.151:0/3756864361 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7fcd200a3e30 0x7fcd200a62f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:16.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:16.100+0000 7fcd43000700 1 --2- 172.21.15.151:0/3756864361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd3c107940 0x7fcd3c19fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:16.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:16.100+0000 7fcd43000700 1 --2- 172.21.15.151:0/3756864361 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd3c102af0 0x7fcd3c19b2c0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:16.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:16.100+0000 7fcd43000700 1 --2- 172.21.15.151:0/3756864361 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd3c1034b0 0x7fcd3c19b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:16.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:16.100+0000 7fcd43000700 1 -- 172.21.15.151:0/3756864361 >> 172.21.15.151:0/3756864361 conn(0x7fcd3c079ba0 msgr2=0x7fcd3c100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:06:16.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:16.100+0000 7fcd43000700 1 -- 172.21.15.151:0/3756864361 shutdown_connections 2024-04-26T02:06:16.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:16.100+0000 7fcd43000700 1 -- 172.21.15.151:0/3756864361 wait complete. 2024-04-26T02:06:16.101 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 39 2024-04-26T02:06:16.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:16 smithi151 ceph-mon[28057]: pgmap v397: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 921 B/s wr, 7 op/s 2024-04-26T02:06:16.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:16 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/3756864361' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:06:16.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:16 smithi172 ceph-mon[32785]: pgmap v397: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 921 B/s wr, 7 op/s 2024-04-26T02:06:16.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:16 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/3756864361' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:06:16.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:16 smithi165 ceph-mon[32648]: pgmap v397: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 921 B/s wr, 7 op/s 2024-04-26T02:06:16.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:16 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/3756864361' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:06:16.934 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T02:06:17.382 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:06:18.402 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:18 smithi151 ceph-mon[28057]: pgmap v398: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 921 B/s wr, 7 op/s 2024-04-26T02:06:18.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:18 smithi172 ceph-mon[32785]: pgmap v398: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 921 B/s wr, 7 op/s 2024-04-26T02:06:18.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:18 smithi165 ceph-mon[32648]: pgmap v398: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 921 B/s wr, 7 op/s 2024-04-26T02:06:19.697 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:19.697+0000 7f8cc1600700 1 -- 172.21.15.151:0/1242842409 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8cbc10e3b0 msgr2=0x7f8cbc115f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:06:19.697 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:19.698+0000 7f8cc1600700 1 --2- 172.21.15.151:0/1242842409 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8cbc10e3b0 0x7f8cbc115f80 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8cb400b0d0 tx=0x7f8cb400b3e0 comp rx=0 tx=0).stop 2024-04-26T02:06:19.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:19.698+0000 7f8cc1600700 1 -- 172.21.15.151:0/1242842409 shutdown_connections 2024-04-26T02:06:19.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:19.698+0000 7f8cc1600700 1 --2- 172.21.15.151:0/1242842409 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8cbc10e3b0 0x7f8cbc115f80 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:19.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:19.698+0000 7f8cc1600700 1 --2- 172.21.15.151:0/1242842409 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8cbc1092d0 0x7f8cbc1096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:19.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:19.698+0000 7f8cc1600700 1 --2- 172.21.15.151:0/1242842409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8cbc109c80 0x7f8cbc10dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:19.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:19.698+0000 7f8cc1600700 1 -- 172.21.15.151:0/1242842409 >> 172.21.15.151:0/1242842409 conn(0x7f8cbc076210 msgr2=0x7f8cbc107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:06:19.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:19.698+0000 7f8cc1600700 1 -- 172.21.15.151:0/1242842409 shutdown_connections 2024-04-26T02:06:19.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:19.698+0000 7f8cc1600700 1 -- 172.21.15.151:0/1242842409 wait complete. 2024-04-26T02:06:19.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:19.699+0000 7f8cc1600700 1 Processor -- start 2024-04-26T02:06:19.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:19.699+0000 7f8cc1600700 1 -- start start 2024-04-26T02:06:19.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:19.700+0000 7f8cc1600700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8cbc1092d0 0x7f8cbc19f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:06:19.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:19.700+0000 7f8cc1600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8cbc109c80 0x7f8cbc19fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:06:19.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:19.700+0000 7f8cc1600700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8cbc10e3b0 0x7f8cbc1a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:06:19.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:19.700+0000 7f8cc1600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc119250 con 0x7f8cbc109c80 2024-04-26T02:06:19.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:19.700+0000 7f8cc1600700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc1190d0 con 0x7f8cbc10e3b0 2024-04-26T02:06:19.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:19.700+0000 7f8cc1600700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc1193d0 con 0x7f8cbc1092d0 2024-04-26T02:06:19.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:19.700+0000 7f8cba000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8cbc109c80 0x7f8cbc19fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:06:19.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:19.700+0000 7f8cbaa00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8cbc1092d0 0x7f8cbc19f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:06:19.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:19.700+0000 7f8cba000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8cbc109c80 0x7f8cbc19fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:51290/0 (socket says 172.21.15.151:51290) 2024-04-26T02:06:19.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:19.700+0000 7f8cba000700 1 -- 172.21.15.151:0/813650674 learned_addr learned my addr 172.21.15.151:0/813650674 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:06:19.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:19.700+0000 7f8cbaa00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8cbc1092d0 0x7f8cbc19f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.151:37768/0 (socket says 172.21.15.151:37768) 2024-04-26T02:06:19.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:19.700+0000 7f8cbb400700 1 --2- 172.21.15.151:0/813650674 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8cbc10e3b0 0x7f8cbc1a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:06:19.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:19.701+0000 7f8cba000700 1 -- 172.21.15.151:0/813650674 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8cbc1092d0 msgr2=0x7f8cbc19f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:06:19.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:19.701+0000 7f8cba000700 1 --2- 172.21.15.151:0/813650674 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8cbc1092d0 0x7f8cbc19f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:19.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:19.701+0000 7f8cba000700 1 -- 172.21.15.151:0/813650674 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8cbc10e3b0 msgr2=0x7f8cbc1a3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:06:19.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:19.701+0000 7f8cba000700 1 --2- 172.21.15.151:0/813650674 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8cbc10e3b0 0x7f8cbc1a3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:19.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:19.701+0000 7f8cba000700 1 -- 172.21.15.151:0/813650674 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cbc1a46b0 con 0x7f8cbc109c80 2024-04-26T02:06:19.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:19.701+0000 7f8cba000700 1 --2- 172.21.15.151:0/813650674 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8cbc109c80 0x7f8cbc19fbe0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8ca800e970 tx=0x7f8ca800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:06:19.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:19.701+0000 7f8cb3400700 1 -- 172.21.15.151:0/813650674 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca8004a30 con 0x7f8cbc109c80 2024-04-26T02:06:19.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:19.701+0000 7f8cb3400700 1 -- 172.21.15.151:0/813650674 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ca800cc90 con 0x7f8cbc109c80 2024-04-26T02:06:19.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:19.701+0000 7f8cc1600700 1 -- 172.21.15.151:0/813650674 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cbc1a49a0 con 0x7f8cbc109c80 2024-04-26T02:06:19.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:19.701+0000 7f8cb3400700 1 -- 172.21.15.151:0/813650674 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca8005610 con 0x7f8cbc109c80 2024-04-26T02:06:19.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:19.701+0000 7f8cc1600700 1 -- 172.21.15.151:0/813650674 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cbc1ac220 con 0x7f8cbc109c80 2024-04-26T02:06:19.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:19.703+0000 7f8cc1600700 1 -- 172.21.15.151:0/813650674 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cbc069730 con 0x7f8cbc109c80 2024-04-26T02:06:19.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:19.705+0000 7f8cb3400700 1 -- 172.21.15.151:0/813650674 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f8ca800c7e0 con 0x7f8cbc109c80 2024-04-26T02:06:19.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:19.706+0000 7f8cb3400700 1 --2- 172.21.15.151:0/813650674 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f8ca00a3e30 0x7f8ca00a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:06:19.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:19.706+0000 7f8cb3400700 1 -- 172.21.15.151:0/813650674 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8455+0+0 (secure 0 0 0) 0x7f8ca80d0890 con 0x7f8cbc109c80 2024-04-26T02:06:19.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:19.706+0000 7f8cbaa00700 1 --2- 172.21.15.151:0/813650674 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f8ca00a3e30 0x7f8ca00a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:06:19.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:19.707+0000 7f8cbaa00700 1 --2- 172.21.15.151:0/813650674 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f8ca00a3e30 0x7f8ca00a62f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8cac009c60 tx=0x7f8cac009260 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:06:19.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:19.711+0000 7f8cb3400700 1 -- 172.21.15.151:0/813650674 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8ca809cf10 con 0x7f8cbc109c80 2024-04-26T02:06:19.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:19.935+0000 7f8cc1600700 1 -- 172.21.15.151:0/813650674 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8cbc02df80 con 0x7f8cbc109c80 2024-04-26T02:06:19.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:19.938+0000 7f8cb3400700 1 -- 172.21.15.151:0/813650674 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) ==== 74+0+23075 (secure 0 0 0) 0x7f8ca809c810 con 0x7f8cbc109c80 2024-04-26T02:06:19.938 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-04-26T02:06:19.939 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":109,"fsid":"c0e49e64-036d-11ef-bc93-c7b262605968","created":"2024-04-26T01:40:07.483197+0000","modified":"2024-04-26T02:06:19.317079+0000","last_up_change":"2024-04-26T01:48:21.167691+0000","last_in_change":"2024-04-26T01:48:00.556990+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T01:44:11.411722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T01:49:31.756032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T01:49:36.529819+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":20,"snap_epoch":108,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ca27e814-a982-482c-ae1a-e596aac5c59a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":669937998},{"type":"v1","addr":"172.21.15.151:6803","nonce":669937998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":669937998},{"type":"v1","addr":"172.21.15.151:6805","nonce":669937998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":669937998},{"type":"v1","addr":"172.21.15.151:6809","nonce":669937998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":669937998},{"type":"v1","addr":"172.21.15.151:6807","nonce":669937998}]},"public_addr":"172.21.15.151:6803/669937998","cluster_addr":"172.21.15.151:6805/669937998","heartbeat_back_addr":"172.21.15.151:6809/669937998","heartbeat_front_addr":"172.21.15.151:6807/669937998","state":["exists","up"]},{"osd":1,"uuid":"e7d192c3-9624-4625-9ccb-a76c114d8aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":1356168171},{"type":"v1","addr":"172.21.15.165:6801","nonce":1356168171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":1356168171},{"type":"v1","addr":"172.21.15.165:6803","nonce":1356168171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":1356168171},{"type":"v1","addr":"172.21.15.165:6807","nonce":1356168171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":1356168171},{"type":"v1","addr":"172.21.15.165:6805","nonce":1356168171}]},"public_addr":"172.21.15.165:6801/1356168171","cluster_addr":"172.21.15.165:6803/1356168171","heartbeat_back_addr":"172.21.15.165:6807/1356168171","heartbeat_front_addr":"172.21.15.165:6805/1356168171","state":["exists","up"]},{"osd":2,"uuid":"0631df59-1ecc-48b4-b00a-48b0118dfc24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":237502635},{"type":"v1","addr":"172.21.15.172:6801","nonce":237502635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":237502635},{"type":"v1","addr":"172.21.15.172:6803","nonce":237502635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":237502635},{"type":"v1","addr":"172.21.15.172:6807","nonce":237502635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":237502635},{"type":"v1","addr":"172.21.15.172:6805","nonce":237502635}]},"public_addr":"172.21.15.172:6801/237502635","cluster_addr":"172.21.15.172:6803/237502635","heartbeat_back_addr":"172.21.15.172:6807/237502635","heartbeat_front_addr":"172.21.15.172:6805/237502635","state":["exists","up"]},{"osd":3,"uuid":"73076e49-2f8a-4a3d-99eb-7b50aff448aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3442279476},{"type":"v1","addr":"172.21.15.151:6811","nonce":3442279476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3442279476},{"type":"v1","addr":"172.21.15.151:6813","nonce":3442279476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":3442279476},{"type":"v1","addr":"172.21.15.151:6817","nonce":3442279476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3442279476},{"type":"v1","addr":"172.21.15.151:6815","nonce":3442279476}]},"public_addr":"172.21.15.151:6811/3442279476","cluster_addr":"172.21.15.151:6813/3442279476","heartbeat_back_addr":"172.21.15.151:6817/3442279476","heartbeat_front_addr":"172.21.15.151:6815/3442279476","state":["exists","up"]},{"osd":4,"uuid":"0230d615-548c-43f0-b8aa-7130f4cbbc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4029980723},{"type":"v1","addr":"172.21.15.165:6809","nonce":4029980723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4029980723},{"type":"v1","addr":"172.21.15.165:6811","nonce":4029980723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4029980723},{"type":"v1","addr":"172.21.15.165:6815","nonce":4029980723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4029980723},{"type":"v1","addr":"172.21.15.165:6813","nonce":4029980723}]},"public_addr":"172.21.15.165:6809/4029980723","cluster_addr":"172.21.15.165:6811/4029980723","heartbeat_back_addr":"172.21.15.165:6815/4029980723","heartbeat_front_addr":"172.21.15.165:6813/4029980723","state":["exists","up"]},{"osd":5,"uuid":"9739f44b-2605-47dd-82fb-d30034742bf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2216578687},{"type":"v1","addr":"172.21.15.172:6809","nonce":2216578687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2216578687},{"type":"v1","addr":"172.21.15.172:6811","nonce":2216578687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2216578687},{"type":"v1","addr":"172.21.15.172:6815","nonce":2216578687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2216578687},{"type":"v1","addr":"172.21.15.172:6813","nonce":2216578687}]},"public_addr":"172.21.15.172:6809/2216578687","cluster_addr":"172.21.15.172:6811/2216578687","heartbeat_back_addr":"172.21.15.172:6815/2216578687","heartbeat_front_addr":"172.21.15.172:6813/2216578687","state":["exists","up"]},{"osd":6,"uuid":"4290baaa-6160-49cb-a4e0-c96f070f2ef5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1723252315},{"type":"v1","addr":"172.21.15.151:6819","nonce":1723252315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1723252315},{"type":"v1","addr":"172.21.15.151:6821","nonce":1723252315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":1723252315},{"type":"v1","addr":"172.21.15.151:6825","nonce":1723252315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1723252315},{"type":"v1","addr":"172.21.15.151:6823","nonce":1723252315}]},"public_addr":"172.21.15.151:6819/1723252315","cluster_addr":"172.21.15.151:6821/1723252315","heartbeat_back_addr":"172.21.15.151:6825/1723252315","heartbeat_front_addr":"172.21.15.151:6823/1723252315","state":["exists","up"]},{"osd":7,"uuid":"9f5315f2-036c-4e2f-92ed-3321a9e3c860","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":1977803781},{"type":"v1","addr":"172.21.15.165:6817","nonce":1977803781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":1977803781},{"type":"v1","addr":"172.21.15.165:6819","nonce":1977803781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":1977803781},{"type":"v1","addr":"172.21.15.165:6823","nonce":1977803781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":1977803781},{"type":"v1","addr":"172.21.15.165:6821","nonce":1977803781}]},"public_addr":"172.21.15.165:6817/1977803781","cluster_addr":"172.21.15.165:6819/1977803781","heartbeat_back_addr":"172.21.15.165:6823/1977803781","heartbeat_front_addr":"172.21.15.165:6821/1977803781","state":["exists","up"]},{"osd":8,"uuid":"8ddbe36d-cea7-45ef-a3ca-044a0e64ff85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":491199512},{"type":"v1","addr":"172.21.15.172:6817","nonce":491199512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":491199512},{"type":"v1","addr":"172.21.15.172:6819","nonce":491199512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":491199512},{"type":"v1","addr":"172.21.15.172:6823","nonce":491199512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":491199512},{"type":"v1","addr":"172.21.15.172:6821","nonce":491199512}]},"public_addr":"172.21.15.172:6817/491199512","cluster_addr":"172.21.15.172:6819/491199512","heartbeat_back_addr":"172.21.15.172:6823/491199512","heartbeat_front_addr":"172.21.15.172:6821/491199512","state":["exists","up"]},{"osd":9,"uuid":"40654ba0-1bd9-4b54-97ae-f1ed6c4785d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":1818923828},{"type":"v1","addr":"172.21.15.151:6827","nonce":1818923828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":1818923828},{"type":"v1","addr":"172.21.15.151:6829","nonce":1818923828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1818923828},{"type":"v1","addr":"172.21.15.151:6833","nonce":1818923828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":1818923828},{"type":"v1","addr":"172.21.15.151:6831","nonce":1818923828}]},"public_addr":"172.21.15.151:6827/1818923828","cluster_addr":"172.21.15.151:6829/1818923828","heartbeat_back_addr":"172.21.15.151:6833/1818923828","heartbeat_front_addr":"172.21.15.151:6831/1818923828","state":["exists","up"]},{"osd":10,"uuid":"19e75a1a-7bd0-41d9-b0cc-0fa4d1475dda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":2632621809},{"type":"v1","addr":"172.21.15.165:6825","nonce":2632621809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":2632621809},{"type":"v1","addr":"172.21.15.165:6827","nonce":2632621809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":2632621809},{"type":"v1","addr":"172.21.15.165:6831","nonce":2632621809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":2632621809},{"type":"v1","addr":"172.21.15.165:6829","nonce":2632621809}]},"public_addr":"172.21.15.165:6825/2632621809","cluster_addr":"172.21.15.165:6827/2632621809","heartbeat_back_addr":"172.21.15.165:6831/2632621809","heartbeat_front_addr":"172.21.15.165:6829/2632621809","state":["exists","up"]},{"osd":11,"uuid":"aa84e5f5-93a3-4375-8544-a1175f0eb06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":886604365},{"type":"v1","addr":"172.21.15.172:6825","nonce":886604365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":886604365},{"type":"v1","addr":"172.21.15.172:6827","nonce":886604365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":886604365},{"type":"v1","addr":"172.21.15.172:6831","nonce":886604365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":886604365},{"type":"v1","addr":"172.21.15.172:6829","nonce":886604365}]},"public_addr":"172.21.15.172:6825/886604365","cluster_addr":"172.21.15.172:6827/886604365","heartbeat_back_addr":"172.21.15.172:6831/886604365","heartbeat_front_addr":"172.21.15.172:6829/886604365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:43:18.296278+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:43:42.860620+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:44:07.073601+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:44:39.631760+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:45:04.771214+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:45:29.606674+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:46:03.101851+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:46:28.351995+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:46:53.445435+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:47:27.391179+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:47:53.543638+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:48:18.687811+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.151:6800/2251256505":"2024-04-27T01:53:36.653535+0000","172.21.15.151:0/2357384001":"2024-04-27T01:53:36.653535+0000","172.21.15.151:6801/796314699":"2024-04-27T01:41:25.234322+0000","172.21.15.151:6800/796314699":"2024-04-27T01:41:25.234322+0000","172.21.15.151:0/4066781930":"2024-04-27T01:53:36.653535+0000","172.21.15.151:0/1173525554":"2024-04-27T01:41:25.234322+0000","172.21.15.151:0/1342496801":"2024-04-27T01:40:50.492554+0000","172.21.15.151:0/755039608":"2024-04-27T01:53:36.653535+0000","172.21.15.151:0/11665926":"2024-04-27T01:40:50.492554+0000","172.21.15.151:0/1765448952":"2024-04-27T01:40:50.492554+0000","172.21.15.151:0/3463483606":"2024-04-27T01:53:36.653535+0000","172.21.15.151:6801/2251256505":"2024-04-27T01:53:36.653535+0000","172.21.15.151:0/46007430":"2024-04-27T01:41:25.234322+0000","172.21.15.151:6800/244725060":"2024-04-27T01:40:50.492554+0000","172.21.15.151:0/3703862543":"2024-04-27T01:41:25.234322+0000","172.21.15.151:6801/244725060":"2024-04-27T01:40:50.492554+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":8,"length":1},{"begin":20,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T02:06:19.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:19.941+0000 7f8cc1600700 1 -- 172.21.15.151:0/813650674 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f8ca00a3e30 msgr2=0x7f8ca00a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:06:19.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:19.941+0000 7f8cc1600700 1 --2- 172.21.15.151:0/813650674 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f8ca00a3e30 0x7f8ca00a62f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8cac009c60 tx=0x7f8cac009260 comp rx=0 tx=0).stop 2024-04-26T02:06:19.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:19.941+0000 7f8cc1600700 1 -- 172.21.15.151:0/813650674 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8cbc109c80 msgr2=0x7f8cbc19fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:06:19.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:19.941+0000 7f8cc1600700 1 --2- 172.21.15.151:0/813650674 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8cbc109c80 0x7f8cbc19fbe0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8ca800e970 tx=0x7f8ca800ec80 comp rx=0 tx=0).stop 2024-04-26T02:06:19.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:19.942+0000 7f8cc1600700 1 -- 172.21.15.151:0/813650674 shutdown_connections 2024-04-26T02:06:19.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:19.942+0000 7f8cc1600700 1 --2- 172.21.15.151:0/813650674 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f8ca00a3e30 0x7f8ca00a62f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:19.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:19.942+0000 7f8cc1600700 1 --2- 172.21.15.151:0/813650674 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8cbc10e3b0 0x7f8cbc1a3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:19.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:19.942+0000 7f8cc1600700 1 --2- 172.21.15.151:0/813650674 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8cbc1092d0 0x7f8cbc19f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:19.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:19.942+0000 7f8cc1600700 1 --2- 172.21.15.151:0/813650674 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8cbc109c80 0x7f8cbc19fbe0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:19.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:19.942+0000 7f8cc1600700 1 -- 172.21.15.151:0/813650674 >> 172.21.15.151:0/813650674 conn(0x7f8cbc076210 msgr2=0x7f8cbc076790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:06:19.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:19.942+0000 7f8cc1600700 1 -- 172.21.15.151:0/813650674 shutdown_connections 2024-04-26T02:06:19.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:19.942+0000 7f8cc1600700 1 -- 172.21.15.151:0/813650674 wait complete. 2024-04-26T02:06:20.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:20 smithi165 ceph-mon[32648]: pgmap v399: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 921 B/s wr, 7 op/s 2024-04-26T02:06:20.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:20 smithi165 ceph-mon[32648]: osdmap e109: 12 total, 12 up, 12 in 2024-04-26T02:06:20.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:20 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/813650674' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T02:06:20.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:20 smithi151 ceph-mon[28057]: pgmap v399: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 921 B/s wr, 7 op/s 2024-04-26T02:06:20.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:20 smithi151 ceph-mon[28057]: osdmap e109: 12 total, 12 up, 12 in 2024-04-26T02:06:20.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:20 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/813650674' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T02:06:20.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:20 smithi172 ceph-mon[32785]: pgmap v399: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 921 B/s wr, 7 op/s 2024-04-26T02:06:20.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:20 smithi172 ceph-mon[32785]: osdmap e109: 12 total, 12 up, 12 in 2024-04-26T02:06:20.788 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:20 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/813650674' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T02:06:20.813 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph fs fail cephfs 2024-04-26T02:06:21.267 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:06:22.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:22 smithi165 ceph-mon[32648]: pgmap v401: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 921 B/s wr, 7 op/s 2024-04-26T02:06:22.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:22 smithi151 ceph-mon[28057]: pgmap v401: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 921 B/s wr, 7 op/s 2024-04-26T02:06:22.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:22 smithi172 ceph-mon[32785]: pgmap v401: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 921 B/s wr, 7 op/s 2024-04-26T02:06:23.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:23.521+0000 7f3562800700 1 -- 172.21.15.151:0/4047565181 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f355c105870 msgr2=0x7f355c1098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:06:23.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:23.521+0000 7f3562800700 1 --2- 172.21.15.151:0/4047565181 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f355c105870 0x7f355c1098c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3544009920 tx=0x7f3544009c30 comp rx=0 tx=0).stop 2024-04-26T02:06:23.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:23.521+0000 7f3562800700 1 -- 172.21.15.151:0/4047565181 shutdown_connections 2024-04-26T02:06:23.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:23.521+0000 7f3562800700 1 --2- 172.21.15.151:0/4047565181 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f355c109fa0 0x7f355c111b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:23.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:23.521+0000 7f3562800700 1 --2- 172.21.15.151:0/4047565181 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f355c104ec0 0x7f355c1052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:23.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:23.521+0000 7f3562800700 1 --2- 172.21.15.151:0/4047565181 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f355c105870 0x7f355c1098c0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:23.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:23.521+0000 7f3562800700 1 -- 172.21.15.151:0/4047565181 >> 172.21.15.151:0/4047565181 conn(0x7f355c100750 msgr2=0x7f355c102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:06:23.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:23.521+0000 7f3562800700 1 -- 172.21.15.151:0/4047565181 shutdown_connections 2024-04-26T02:06:23.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:23.521+0000 7f3562800700 1 -- 172.21.15.151:0/4047565181 wait complete. 2024-04-26T02:06:23.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:23.522+0000 7f3562800700 1 Processor -- start 2024-04-26T02:06:23.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:23.523+0000 7f3562800700 1 -- start start 2024-04-26T02:06:23.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:23.523+0000 7f3562800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f355c104ec0 0x7f355c19b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:06:23.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:23.523+0000 7f3562800700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f355c105870 0x7f355c19b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:06:23.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:23.523+0000 7f3562800700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f355c109fa0 0x7f355c19fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:06:23.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:23.523+0000 7f3562800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f355c114e10 con 0x7f355c104ec0 2024-04-26T02:06:23.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:23.523+0000 7f3562800700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f355c114c90 con 0x7f355c105870 2024-04-26T02:06:23.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:23.523+0000 7f3562800700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f355c114f90 con 0x7f355c109fa0 2024-04-26T02:06:23.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:23.523+0000 7f355b400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f355c104ec0 0x7f355c19b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:06:23.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:23.523+0000 7f355b400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f355c104ec0 0x7f355c19b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41150/0 (socket says 172.21.15.151:41150) 2024-04-26T02:06:23.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:23.523+0000 7f355b400700 1 -- 172.21.15.151:0/3495328387 learned_addr learned my addr 172.21.15.151:0/3495328387 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:06:23.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:23.524+0000 7f3553e00700 1 --2- 172.21.15.151:0/3495328387 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f355c105870 0x7f355c19b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:06:23.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:23.524+0000 7f355be00700 1 --2- 172.21.15.151:0/3495328387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f355c109fa0 0x7f355c19fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:06:23.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:23.524+0000 7f355b400700 1 -- 172.21.15.151:0/3495328387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f355c109fa0 msgr2=0x7f355c19fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:06:23.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:23.524+0000 7f355b400700 1 --2- 172.21.15.151:0/3495328387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f355c109fa0 0x7f355c19fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:23.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:23.524+0000 7f355b400700 1 -- 172.21.15.151:0/3495328387 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f355c105870 msgr2=0x7f355c19b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:06:23.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:23.524+0000 7f355b400700 1 --2- 172.21.15.151:0/3495328387 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f355c105870 0x7f355c19b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:23.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:23.524+0000 7f355b400700 1 -- 172.21.15.151:0/3495328387 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f355c1a0380 con 0x7f355c104ec0 2024-04-26T02:06:23.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:23.524+0000 7f3553e00700 1 --2- 172.21.15.151:0/3495328387 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f355c105870 0x7f355c19b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:06:23.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:23.524+0000 7f355b400700 1 --2- 172.21.15.151:0/3495328387 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f355c104ec0 0x7f355c19b3c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f354c00eb30 tx=0x7f354c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:06:23.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:23.525+0000 7f3558c00700 1 -- 172.21.15.151:0/3495328387 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f354c00cbd0 con 0x7f355c104ec0 2024-04-26T02:06:23.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:23.525+0000 7f3562800700 1 -- 172.21.15.151:0/3495328387 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f355c1a0670 con 0x7f355c104ec0 2024-04-26T02:06:23.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:23.525+0000 7f3558c00700 1 -- 172.21.15.151:0/3495328387 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f354c0044d0 con 0x7f355c104ec0 2024-04-26T02:06:23.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:23.525+0000 7f3562800700 1 -- 172.21.15.151:0/3495328387 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f355c1a7ef0 con 0x7f355c104ec0 2024-04-26T02:06:23.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:23.525+0000 7f3558c00700 1 -- 172.21.15.151:0/3495328387 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f354c003dd0 con 0x7f355c104ec0 2024-04-26T02:06:23.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:23.526+0000 7f3562800700 1 -- 172.21.15.151:0/3495328387 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f355c069730 con 0x7f355c104ec0 2024-04-26T02:06:23.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:23.528+0000 7f3558c00700 1 -- 172.21.15.151:0/3495328387 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f354c003f70 con 0x7f355c104ec0 2024-04-26T02:06:23.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:23.529+0000 7f3558c00700 1 --2- 172.21.15.151:0/3495328387 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f35400a3db0 0x7f35400a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:06:23.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:23.530+0000 7f3558c00700 1 -- 172.21.15.151:0/3495328387 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8455+0+0 (secure 0 0 0) 0x7f354c0d0b30 con 0x7f355c104ec0 2024-04-26T02:06:23.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:23.530+0000 7f3553e00700 1 --2- 172.21.15.151:0/3495328387 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f35400a3db0 0x7f35400a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:06:23.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:23.531+0000 7f3553e00700 1 --2- 172.21.15.151:0/3495328387 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f35400a3db0 0x7f35400a6270 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f35440098f0 tx=0x7f3544019040 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:06:23.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:23.534+0000 7f3558c00700 1 -- 172.21.15.151:0/3495328387 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f354c09d1b0 con 0x7f355c104ec0 2024-04-26T02:06:23.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:23.809+0000 7f3562800700 1 -- 172.21.15.151:0/3495328387 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f355c02df80 con 0x7f355c104ec0 2024-04-26T02:06:23.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:23.815+0000 7f3558c00700 1 -- 172.21.15.151:0/3495328387 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v40) ==== 161+0+0 (secure 0 0 0) 0x7f354c09cab0 con 0x7f355c104ec0 2024-04-26T02:06:23.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:23.819+0000 7f3562800700 1 -- 172.21.15.151:0/3495328387 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f35400a3db0 msgr2=0x7f35400a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:06:23.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:23.819+0000 7f3562800700 1 --2- 172.21.15.151:0/3495328387 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f35400a3db0 0x7f35400a6270 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f35440098f0 tx=0x7f3544019040 comp rx=0 tx=0).stop 2024-04-26T02:06:23.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:23.819+0000 7f3562800700 1 -- 172.21.15.151:0/3495328387 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f355c104ec0 msgr2=0x7f355c19b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:06:23.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:23.819+0000 7f3562800700 1 --2- 172.21.15.151:0/3495328387 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f355c104ec0 0x7f355c19b3c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f354c00eb30 tx=0x7f354c00ee40 comp rx=0 tx=0).stop 2024-04-26T02:06:23.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:23.819+0000 7f3562800700 1 -- 172.21.15.151:0/3495328387 shutdown_connections 2024-04-26T02:06:23.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:23.819+0000 7f3562800700 1 --2- 172.21.15.151:0/3495328387 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f35400a3db0 0x7f35400a6270 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:23.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:23.819+0000 7f3562800700 1 --2- 172.21.15.151:0/3495328387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f355c109fa0 0x7f355c19fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:23.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:23.819+0000 7f3562800700 1 --2- 172.21.15.151:0/3495328387 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f355c104ec0 0x7f355c19b3c0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:23.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:23.819+0000 7f3562800700 1 --2- 172.21.15.151:0/3495328387 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f355c105870 0x7f355c19b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:23.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:23.819+0000 7f3562800700 1 -- 172.21.15.151:0/3495328387 >> 172.21.15.151:0/3495328387 conn(0x7f355c100750 msgr2=0x7f355c101a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:06:23.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:23.820+0000 7f3562800700 1 -- 172.21.15.151:0/3495328387 shutdown_connections 2024-04-26T02:06:23.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:23.820+0000 7f3562800700 1 -- 172.21.15.151:0/3495328387 wait complete. 2024-04-26T02:06:23.821 INFO:teuthology.orchestra.run.smithi151.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T02:06:24.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:23 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mon-a[28052]: 2024-04-26T02:06:23.811+0000 7f9888a00700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T02:06:24.288 INFO:journalctl@ceph.mds.c.smithi172.stdout:Apr 26 02:06:23 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-c[63496]: ignoring --setuser ceph since I am not root 2024-04-26T02:06:24.288 INFO:journalctl@ceph.mds.c.smithi172.stdout:Apr 26 02:06:23 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-c[63496]: ignoring --setgroup ceph since I am not root 2024-04-26T02:06:24.288 INFO:journalctl@ceph.mds.c.smithi172.stdout:Apr 26 02:06:23 smithi172 ceph-mds[63500]: -- 172.21.15.172:0/3123229618 <== mon.1 v2:172.21.15.172:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556e7e587a40 con 0x556e7e55ac00 2024-04-26T02:06:24.288 INFO:journalctl@ceph.mds.c.smithi172.stdout:Apr 26 02:06:23 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-c[63496]: starting mds.c at 2024-04-26T02:06:24.288 INFO:journalctl@ceph.mds.f.smithi172.stdout:Apr 26 02:06:23 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-f[63189]: ignoring --setuser ceph since I am not root 2024-04-26T02:06:24.288 INFO:journalctl@ceph.mds.f.smithi172.stdout:Apr 26 02:06:23 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-f[63189]: ignoring --setgroup ceph since I am not root 2024-04-26T02:06:24.289 INFO:journalctl@ceph.mds.f.smithi172.stdout:Apr 26 02:06:23 smithi172 ceph-mds[63193]: -- 172.21.15.172:0/1106139670 <== mon.1 v2:172.21.15.172:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d3af051a40 con 0x55d3af024c00 2024-04-26T02:06:24.289 INFO:journalctl@ceph.mds.f.smithi172.stdout:Apr 26 02:06:23 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-f[63189]: starting mds.f at 2024-04-26T02:06:24.289 INFO:journalctl@ceph.mds.i.smithi172.stdout:Apr 26 02:06:23 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-i[63813]: ignoring --setuser ceph since I am not root 2024-04-26T02:06:24.289 INFO:journalctl@ceph.mds.i.smithi172.stdout:Apr 26 02:06:23 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-i[63813]: ignoring --setgroup ceph since I am not root 2024-04-26T02:06:24.289 INFO:journalctl@ceph.mds.i.smithi172.stdout:Apr 26 02:06:23 smithi172 ceph-mds[63819]: -- 172.21.15.172:0/720805892 <== mon.2 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563e43367a40 con 0x563e4333ac00 2024-04-26T02:06:24.289 INFO:journalctl@ceph.mds.i.smithi172.stdout:Apr 26 02:06:23 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-i[63813]: starting mds.i at 2024-04-26T02:06:24.336 INFO:journalctl@ceph.mds.h.smithi165.stdout:Apr 26 02:06:23 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-h[63016]: ignoring --setuser ceph since I am not root 2024-04-26T02:06:24.336 INFO:journalctl@ceph.mds.h.smithi165.stdout:Apr 26 02:06:23 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-h[63016]: ignoring --setgroup ceph since I am not root 2024-04-26T02:06:24.336 INFO:journalctl@ceph.mds.h.smithi165.stdout:Apr 26 02:06:23 smithi165 ceph-mds[63020]: -- 172.21.15.165:0/2868295315 <== mon.1 v2:172.21.15.172:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564f6f94fa40 con 0x564f6f922c00 2024-04-26T02:06:24.336 INFO:journalctl@ceph.mds.h.smithi165.stdout:Apr 26 02:06:23 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-h[63016]: starting mds.h at 2024-04-26T02:06:24.337 INFO:journalctl@ceph.mds.e.smithi165.stdout:Apr 26 02:06:23 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-e[62700]: ignoring --setuser ceph since I am not root 2024-04-26T02:06:24.337 INFO:journalctl@ceph.mds.e.smithi165.stdout:Apr 26 02:06:23 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-e[62700]: ignoring --setgroup ceph since I am not root 2024-04-26T02:06:24.337 INFO:journalctl@ceph.mds.e.smithi165.stdout:Apr 26 02:06:23 smithi165 ceph-mds[62704]: -- 172.21.15.165:0/4267827016 <== mon.2 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556c64939a40 con 0x556c6490cc00 2024-04-26T02:06:24.337 INFO:journalctl@ceph.mds.e.smithi165.stdout:Apr 26 02:06:23 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-e[62700]: starting mds.e at 2024-04-26T02:06:24.337 INFO:journalctl@ceph.mds.b.smithi165.stdout:Apr 26 02:06:23 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-b[63325]: ignoring --setuser ceph since I am not root 2024-04-26T02:06:24.337 INFO:journalctl@ceph.mds.b.smithi165.stdout:Apr 26 02:06:23 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-b[63325]: ignoring --setgroup ceph since I am not root 2024-04-26T02:06:24.337 INFO:journalctl@ceph.mds.b.smithi165.stdout:Apr 26 02:06:23 smithi165 ceph-mds[63329]: -- 172.21.15.165:0/122402914 <== mon.1 v2:172.21.15.172:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555a1fa5ba40 con 0x555a1fa2e800 2024-04-26T02:06:24.337 INFO:journalctl@ceph.mds.b.smithi165.stdout:Apr 26 02:06:23 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-b[63325]: starting mds.b at 2024-04-26T02:06:24.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:24 smithi165 ceph-mon[32648]: pgmap v402: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T02:06:24.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:24 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/3495328387' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-26T02:06:24.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:24 smithi165 ceph-mon[32648]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T02:06:24.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:24 smithi165 ceph-mon[32648]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T02:06:24.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:24 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/3495328387' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-26T02:06:24.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:24 smithi165 ceph-mon[32648]: osdmap e110: 12 total, 12 up, 12 in 2024-04-26T02:06:24.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:24 smithi165 ceph-mon[32648]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-26T02:06:24.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:24 smithi151 ceph-mon[28057]: pgmap v402: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T02:06:24.623 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:24 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/3495328387' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-26T02:06:24.623 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:24 smithi151 ceph-mon[28057]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T02:06:24.623 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:24 smithi151 ceph-mon[28057]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T02:06:24.623 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:24 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/3495328387' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-26T02:06:24.623 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:24 smithi151 ceph-mon[28057]: osdmap e110: 12 total, 12 up, 12 in 2024-04-26T02:06:24.623 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:24 smithi151 ceph-mon[28057]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-26T02:06:24.678 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T02:06:24.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:24 smithi172 ceph-mon[32785]: pgmap v402: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T02:06:24.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:24 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/3495328387' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-26T02:06:24.787 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:24 smithi172 ceph-mon[32785]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T02:06:24.788 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:24 smithi172 ceph-mon[32785]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T02:06:24.788 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:24 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/3495328387' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-26T02:06:24.788 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:24 smithi172 ceph-mon[32785]: osdmap e110: 12 total, 12 up, 12 in 2024-04-26T02:06:24.788 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:24 smithi172 ceph-mon[32785]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-26T02:06:25.124 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:06:26.097 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:25 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T02:06:26.097 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:25 smithi165 ceph-mon[32648]: mds.? [v2:172.21.15.172:6834/353439784,v1:172.21.15.172:6835/353439784] up:boot 2024-04-26T02:06:26.097 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:25 smithi165 ceph-mon[32648]: mds.? [v2:172.21.15.165:6834/1582585951,v1:172.21.15.165:6835/1582585951] up:boot 2024-04-26T02:06:26.097 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:25 smithi165 ceph-mon[32648]: mds.? [v2:172.21.15.172:6836/1618188383,v1:172.21.15.172:6838/1618188383] up:boot 2024-04-26T02:06:26.097 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:25 smithi165 ceph-mon[32648]: mds.? [v2:172.21.15.172:6837/124666183,v1:172.21.15.172:6839/124666183] up:boot 2024-04-26T02:06:26.097 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:25 smithi165 ceph-mon[32648]: mds.? [v2:172.21.15.165:6836/2330075813,v1:172.21.15.165:6838/2330075813] up:boot 2024-04-26T02:06:26.097 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:25 smithi165 ceph-mon[32648]: mds.? [v2:172.21.15.165:6837/971746338,v1:172.21.15.165:6839/971746338] up:boot 2024-04-26T02:06:26.097 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:25 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T02:06:26.097 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:25 smithi165 ceph-mon[32648]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-04-26T02:06:26.097 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:25 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T02:06:26.098 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:25 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T02:06:26.098 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:25 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T02:06:26.098 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:25 smithi165 ceph-mon[32648]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T02:06:26.098 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:25 smithi165 ceph-mon[32648]: pgmap v404: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 127 B/s wr, 7 op/s 2024-04-26T02:06:26.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:25 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T02:06:26.123 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:25 smithi151 ceph-mon[28057]: mds.? [v2:172.21.15.172:6834/353439784,v1:172.21.15.172:6835/353439784] up:boot 2024-04-26T02:06:26.123 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:25 smithi151 ceph-mon[28057]: mds.? [v2:172.21.15.165:6834/1582585951,v1:172.21.15.165:6835/1582585951] up:boot 2024-04-26T02:06:26.123 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:25 smithi151 ceph-mon[28057]: mds.? [v2:172.21.15.172:6836/1618188383,v1:172.21.15.172:6838/1618188383] up:boot 2024-04-26T02:06:26.123 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:25 smithi151 ceph-mon[28057]: mds.? [v2:172.21.15.172:6837/124666183,v1:172.21.15.172:6839/124666183] up:boot 2024-04-26T02:06:26.123 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:25 smithi151 ceph-mon[28057]: mds.? [v2:172.21.15.165:6836/2330075813,v1:172.21.15.165:6838/2330075813] up:boot 2024-04-26T02:06:26.123 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:25 smithi151 ceph-mon[28057]: mds.? [v2:172.21.15.165:6837/971746338,v1:172.21.15.165:6839/971746338] up:boot 2024-04-26T02:06:26.123 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:25 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T02:06:26.123 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:25 smithi151 ceph-mon[28057]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-04-26T02:06:26.123 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:25 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T02:06:26.123 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:25 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T02:06:26.123 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:25 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T02:06:26.123 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:25 smithi151 ceph-mon[28057]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T02:06:26.123 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:25 smithi151 ceph-mon[28057]: pgmap v404: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 127 B/s wr, 7 op/s 2024-04-26T02:06:26.287 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:25 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T02:06:26.288 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:25 smithi172 ceph-mon[32785]: mds.? [v2:172.21.15.172:6834/353439784,v1:172.21.15.172:6835/353439784] up:boot 2024-04-26T02:06:26.288 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:25 smithi172 ceph-mon[32785]: mds.? [v2:172.21.15.165:6834/1582585951,v1:172.21.15.165:6835/1582585951] up:boot 2024-04-26T02:06:26.288 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:25 smithi172 ceph-mon[32785]: mds.? [v2:172.21.15.172:6836/1618188383,v1:172.21.15.172:6838/1618188383] up:boot 2024-04-26T02:06:26.288 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:25 smithi172 ceph-mon[32785]: mds.? [v2:172.21.15.172:6837/124666183,v1:172.21.15.172:6839/124666183] up:boot 2024-04-26T02:06:26.288 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:25 smithi172 ceph-mon[32785]: mds.? [v2:172.21.15.165:6836/2330075813,v1:172.21.15.165:6838/2330075813] up:boot 2024-04-26T02:06:26.288 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:25 smithi172 ceph-mon[32785]: mds.? [v2:172.21.15.165:6837/971746338,v1:172.21.15.165:6839/971746338] up:boot 2024-04-26T02:06:26.288 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:25 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T02:06:26.288 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:25 smithi172 ceph-mon[32785]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-04-26T02:06:26.288 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:25 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T02:06:26.288 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:25 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T02:06:26.288 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:25 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T02:06:26.288 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:25 smithi172 ceph-mon[32785]: from='mgr.25025 172.21.15.151:0/3869518910' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T02:06:26.288 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:25 smithi172 ceph-mon[32785]: pgmap v404: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 127 B/s wr, 7 op/s 2024-04-26T02:06:27.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:27.638+0000 7f422de00700 1 -- 172.21.15.151:0/3230826558 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f422806b0f0 msgr2=0x7f422810f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:06:27.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:27.638+0000 7f422de00700 1 --2- 172.21.15.151:0/3230826558 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f422806b0f0 0x7f422810f900 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f422000b0a0 tx=0x7f422000b3b0 comp rx=0 tx=0).stop 2024-04-26T02:06:27.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:27.638+0000 7f422de00700 1 -- 172.21.15.151:0/3230826558 shutdown_connections 2024-04-26T02:06:27.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:27.638+0000 7f422de00700 1 --2- 172.21.15.151:0/3230826558 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f422806b0f0 0x7f422810f900 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:27.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:27.638+0000 7f422de00700 1 --2- 172.21.15.151:0/3230826558 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4228108a30 0x7f4228108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:27.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:27.638+0000 7f422de00700 1 --2- 172.21.15.151:0/3230826558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f422806a750 0x7f422806abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:27.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:27.638+0000 7f422de00700 1 -- 172.21.15.151:0/3230826558 >> 172.21.15.151:0/3230826558 conn(0x7f42280788f0 msgr2=0x7f4228078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:06:27.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:27.638+0000 7f422de00700 1 -- 172.21.15.151:0/3230826558 shutdown_connections 2024-04-26T02:06:27.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:27.638+0000 7f422de00700 1 -- 172.21.15.151:0/3230826558 wait complete. 2024-04-26T02:06:27.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:27.639+0000 7f422de00700 1 Processor -- start 2024-04-26T02:06:27.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:27.640+0000 7f422de00700 1 -- start start 2024-04-26T02:06:27.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:27.640+0000 7f422de00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f422806a750 0x7f422819b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:06:27.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:27.640+0000 7f422de00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f422806b0f0 0x7f422819b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:06:27.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:27.640+0000 7f422de00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4228108a30 0x7f422819fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:06:27.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:27.640+0000 7f422de00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4228112c20 con 0x7f422806b0f0 2024-04-26T02:06:27.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:27.640+0000 7f422de00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4228112aa0 con 0x7f422806a750 2024-04-26T02:06:27.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:27.640+0000 7f422de00700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4228112da0 con 0x7f4228108a30 2024-04-26T02:06:27.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:27.640+0000 7f4226000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f422806b0f0 0x7f422819b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:06:27.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:27.640+0000 7f4226000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f422806b0f0 0x7f422819b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41172/0 (socket says 172.21.15.151:41172) 2024-04-26T02:06:27.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:27.640+0000 7f4226000700 1 -- 172.21.15.151:0/52877398 learned_addr learned my addr 172.21.15.151:0/52877398 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:06:27.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:27.641+0000 7f4226a00700 1 --2- 172.21.15.151:0/52877398 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f422806a750 0x7f422819b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:06:27.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:27.641+0000 7f4227400700 1 --2- 172.21.15.151:0/52877398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4228108a30 0x7f422819fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:06:27.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:27.641+0000 7f4226a00700 1 -- 172.21.15.151:0/52877398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4228108a30 msgr2=0x7f422819fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:06:27.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:27.641+0000 7f4226a00700 1 --2- 172.21.15.151:0/52877398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4228108a30 0x7f422819fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:27.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:27.641+0000 7f4226a00700 1 -- 172.21.15.151:0/52877398 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f422806b0f0 msgr2=0x7f422819b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:06:27.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:27.641+0000 7f4226a00700 1 --2- 172.21.15.151:0/52877398 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f422806b0f0 0x7f422819b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:27.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:27.641+0000 7f4226a00700 1 -- 172.21.15.151:0/52877398 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42281a03c0 con 0x7f422806a750 2024-04-26T02:06:27.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:27.641+0000 7f4226000700 1 --2- 172.21.15.151:0/52877398 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f422806b0f0 0x7f422819b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:06:27.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:27.641+0000 7f4227400700 1 --2- 172.21.15.151:0/52877398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4228108a30 0x7f422819fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:06:27.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:27.641+0000 7f4226a00700 1 --2- 172.21.15.151:0/52877398 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f422806a750 0x7f422819b3b0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f421800e9f0 tx=0x7f421800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:06:27.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:27.642+0000 7f422ca00700 1 -- 172.21.15.151:0/52877398 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421800c9c0 con 0x7f422806a750 2024-04-26T02:06:27.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:27.642+0000 7f422ca00700 1 -- 172.21.15.151:0/52877398 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f421800cb60 con 0x7f422806a750 2024-04-26T02:06:27.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:27.642+0000 7f422ca00700 1 -- 172.21.15.151:0/52877398 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4218018680 con 0x7f422806a750 2024-04-26T02:06:27.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:27.642+0000 7f422de00700 1 -- 172.21.15.151:0/52877398 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42281a06b0 con 0x7f422806a750 2024-04-26T02:06:27.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:27.642+0000 7f422de00700 1 -- 172.21.15.151:0/52877398 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42281a7f30 con 0x7f422806a750 2024-04-26T02:06:27.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:27.643+0000 7f422de00700 1 -- 172.21.15.151:0/52877398 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4228069730 con 0x7f422806a750 2024-04-26T02:06:27.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:27.645+0000 7f422ca00700 1 -- 172.21.15.151:0/52877398 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f4218010430 con 0x7f422806a750 2024-04-26T02:06:27.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:27.646+0000 7f422ca00700 1 --2- 172.21.15.151:0/52877398 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f420c0a3e30 0x7f420c0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:06:27.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:27.647+0000 7f422ca00700 1 -- 172.21.15.151:0/52877398 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8669+0+0 (secure 0 0 0) 0x7f42180d0920 con 0x7f422806a750 2024-04-26T02:06:27.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:27.647+0000 7f4226000700 1 --2- 172.21.15.151:0/52877398 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f420c0a3e30 0x7f420c0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:06:27.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:27.647+0000 7f4226000700 1 --2- 172.21.15.151:0/52877398 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f420c0a3e30 0x7f420c0a62f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4228078f40 tx=0x7f4214009260 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:06:27.650 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:27.651+0000 7f422ca00700 1 -- 172.21.15.151:0/52877398 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f421809ced0 con 0x7f422806a750 2024-04-26T02:06:27.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:27.928+0000 7f422de00700 1 -- 172.21.15.151:0/52877398 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f422802df80 con 0x7f422806a750 2024-04-26T02:06:28.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:28.088+0000 7f422ca00700 1 -- 172.21.15.151:0/52877398 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v42) ==== 104+0+0 (secure 0 0 0) 0x7f421809c7d0 con 0x7f422806a750 2024-04-26T02:06:28.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:28.092+0000 7f422de00700 1 -- 172.21.15.151:0/52877398 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f420c0a3e30 msgr2=0x7f420c0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:06:28.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:28.092+0000 7f422de00700 1 --2- 172.21.15.151:0/52877398 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f420c0a3e30 0x7f420c0a62f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4228078f40 tx=0x7f4214009260 comp rx=0 tx=0).stop 2024-04-26T02:06:28.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:28.092+0000 7f422de00700 1 -- 172.21.15.151:0/52877398 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f422806a750 msgr2=0x7f422819b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:06:28.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:28.092+0000 7f422de00700 1 --2- 172.21.15.151:0/52877398 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f422806a750 0x7f422819b3b0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f421800e9f0 tx=0x7f421800ed00 comp rx=0 tx=0).stop 2024-04-26T02:06:28.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:28.092+0000 7f422de00700 1 -- 172.21.15.151:0/52877398 shutdown_connections 2024-04-26T02:06:28.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:28.092+0000 7f422de00700 1 --2- 172.21.15.151:0/52877398 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f420c0a3e30 0x7f420c0a62f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:28.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:28.092+0000 7f422de00700 1 --2- 172.21.15.151:0/52877398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4228108a30 0x7f422819fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:28.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:28.093+0000 7f422de00700 1 --2- 172.21.15.151:0/52877398 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f422806a750 0x7f422819b3b0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:28.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:28.093+0000 7f422de00700 1 --2- 172.21.15.151:0/52877398 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f422806b0f0 0x7f422819b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:28.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:28.093+0000 7f422de00700 1 -- 172.21.15.151:0/52877398 >> 172.21.15.151:0/52877398 conn(0x7f42280788f0 msgr2=0x7f422810dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:06:28.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:28.093+0000 7f422de00700 1 -- 172.21.15.151:0/52877398 shutdown_connections 2024-04-26T02:06:28.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:28.093+0000 7f422de00700 1 -- 172.21.15.151:0/52877398 wait complete. 2024-04-26T02:06:28.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:28 smithi165 ceph-mon[32648]: pgmap v405: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-04-26T02:06:28.347 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:28 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/52877398' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T02:06:28.347 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:28 smithi165 ceph-mon[32648]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T02:06:28.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:28 smithi151 ceph-mon[28057]: pgmap v405: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-04-26T02:06:28.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:28 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/52877398' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T02:06:28.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:28 smithi151 ceph-mon[28057]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T02:06:28.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:28 smithi172 ceph-mon[32785]: pgmap v405: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-04-26T02:06:28.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:28 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/52877398' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T02:06:28.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:28 smithi172 ceph-mon[32785]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T02:06:29.026 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T02:06:29.347 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:29 smithi165 ceph-mon[32648]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T02:06:29.347 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:29 smithi165 ceph-mon[32648]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-26T02:06:29.347 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:29 smithi165 ceph-mon[32648]: Cluster is now healthy 2024-04-26T02:06:29.347 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:29 smithi165 ceph-mon[32648]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-26T02:06:29.347 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:29 smithi165 ceph-mon[32648]: fsmap 12 up:standby 2024-04-26T02:06:29.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:29 smithi151 ceph-mon[28057]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T02:06:29.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:29 smithi151 ceph-mon[28057]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-26T02:06:29.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:29 smithi151 ceph-mon[28057]: Cluster is now healthy 2024-04-26T02:06:29.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:29 smithi151 ceph-mon[28057]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-26T02:06:29.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:29 smithi151 ceph-mon[28057]: fsmap 12 up:standby 2024-04-26T02:06:29.461 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:06:29.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:29 smithi172 ceph-mon[32785]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T02:06:29.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:29 smithi172 ceph-mon[32785]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-26T02:06:29.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:29 smithi172 ceph-mon[32785]: Cluster is now healthy 2024-04-26T02:06:29.538 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:29 smithi172 ceph-mon[32785]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-26T02:06:29.538 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:29 smithi172 ceph-mon[32785]: fsmap 12 up:standby 2024-04-26T02:06:30.292 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:30 smithi151 ceph-mon[28057]: pgmap v406: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-04-26T02:06:30.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:30 smithi172 ceph-mon[32785]: pgmap v406: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-04-26T02:06:30.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:30 smithi165 ceph-mon[32648]: pgmap v406: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-04-26T02:06:31.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:31.827+0000 7f5e38000700 1 -- 172.21.15.151:0/2376266346 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e3006a750 msgr2=0x7f5e3006abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:06:31.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:31.827+0000 7f5e38000700 1 --2- 172.21.15.151:0/2376266346 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e3006a750 0x7f5e3006abb0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c009990 tx=0x7f5e1c009ca0 comp rx=0 tx=0).stop 2024-04-26T02:06:31.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:31.828+0000 7f5e38000700 1 -- 172.21.15.151:0/2376266346 shutdown_connections 2024-04-26T02:06:31.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:31.828+0000 7f5e38000700 1 --2- 172.21.15.151:0/2376266346 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5e3006b0f0 0x7f5e3010f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:31.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:31.828+0000 7f5e38000700 1 --2- 172.21.15.151:0/2376266346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e30108a30 0x7f5e30108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:31.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:31.828+0000 7f5e38000700 1 --2- 172.21.15.151:0/2376266346 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e3006a750 0x7f5e3006abb0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:31.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:31.828+0000 7f5e38000700 1 -- 172.21.15.151:0/2376266346 >> 172.21.15.151:0/2376266346 conn(0x7f5e300788f0 msgr2=0x7f5e30078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:06:31.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:31.828+0000 7f5e38000700 1 -- 172.21.15.151:0/2376266346 shutdown_connections 2024-04-26T02:06:31.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:31.828+0000 7f5e38000700 1 -- 172.21.15.151:0/2376266346 wait complete. 2024-04-26T02:06:31.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:31.829+0000 7f5e38000700 1 Processor -- start 2024-04-26T02:06:31.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:31.829+0000 7f5e38000700 1 -- start start 2024-04-26T02:06:31.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:31.830+0000 7f5e38000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5e3006a750 0x7f5e30109980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:06:31.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:31.830+0000 7f5e38000700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e3006b0f0 0x7f5e30109ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:06:31.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:31.830+0000 7f5e38000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e30108a30 0x7f5e3010a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:06:31.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:31.830+0000 7f5e38000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5e30112c20 con 0x7f5e3006a750 2024-04-26T02:06:31.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:31.830+0000 7f5e38000700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5e30112aa0 con 0x7f5e30108a30 2024-04-26T02:06:31.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:31.830+0000 7f5e38000700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5e30112da0 con 0x7f5e3006b0f0 2024-04-26T02:06:31.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:31.830+0000 7f5e35600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5e3006a750 0x7f5e30109980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:06:31.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:31.830+0000 7f5e34c00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e3006b0f0 0x7f5e30109ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:06:31.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:31.830+0000 7f5e36000700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e30108a30 0x7f5e3010a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:06:31.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:31.830+0000 7f5e35600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5e3006a750 0x7f5e30109980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:34710/0 (socket says 172.21.15.151:34710) 2024-04-26T02:06:31.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:31.830+0000 7f5e35600700 1 -- 172.21.15.151:0/3080546147 learned_addr learned my addr 172.21.15.151:0/3080546147 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:06:31.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:31.831+0000 7f5e36000700 1 -- 172.21.15.151:0/3080546147 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e3006b0f0 msgr2=0x7f5e30109ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:06:31.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:31.831+0000 7f5e36000700 1 --2- 172.21.15.151:0/3080546147 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e3006b0f0 0x7f5e30109ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:31.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:31.831+0000 7f5e36000700 1 -- 172.21.15.151:0/3080546147 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5e3006a750 msgr2=0x7f5e30109980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:06:31.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:31.831+0000 7f5e36000700 1 --2- 172.21.15.151:0/3080546147 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5e3006a750 0x7f5e30109980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:31.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:31.831+0000 7f5e36000700 1 -- 172.21.15.151:0/3080546147 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e301aba20 con 0x7f5e30108a30 2024-04-26T02:06:31.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:31.831+0000 7f5e35600700 1 --2- 172.21.15.151:0/3080546147 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5e3006a750 0x7f5e30109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:06:31.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:31.831+0000 7f5e36000700 1 --2- 172.21.15.151:0/3080546147 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e30108a30 0x7f5e3010a400 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5e2800e9a0 tx=0x7f5e2800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:06:31.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:31.831+0000 7f5e2e000700 1 -- 172.21.15.151:0/3080546147 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e28004220 con 0x7f5e30108a30 2024-04-26T02:06:31.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:31.832+0000 7f5e2e000700 1 -- 172.21.15.151:0/3080546147 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e280043c0 con 0x7f5e30108a30 2024-04-26T02:06:31.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:31.832+0000 7f5e2e000700 1 -- 172.21.15.151:0/3080546147 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e28010700 con 0x7f5e30108a30 2024-04-26T02:06:31.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:31.832+0000 7f5e38000700 1 -- 172.21.15.151:0/3080546147 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e301abd10 con 0x7f5e30108a30 2024-04-26T02:06:31.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:31.832+0000 7f5e38000700 1 -- 172.21.15.151:0/3080546147 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e301ac2a0 con 0x7f5e30108a30 2024-04-26T02:06:31.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:31.833+0000 7f5e38000700 1 -- 172.21.15.151:0/3080546147 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e30104170 con 0x7f5e30108a30 2024-04-26T02:06:31.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:31.835+0000 7f5e2e000700 1 -- 172.21.15.151:0/3080546147 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f5e28004750 con 0x7f5e30108a30 2024-04-26T02:06:31.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:31.836+0000 7f5e2e000700 1 --2- 172.21.15.151:0/3080546147 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f5e180a3e30 0x7f5e180a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:06:31.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:31.836+0000 7f5e2e000700 1 -- 172.21.15.151:0/3080546147 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8669+0+0 (secure 0 0 0) 0x7f5e280d05b0 con 0x7f5e30108a30 2024-04-26T02:06:31.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:31.836+0000 7f5e35600700 1 --2- 172.21.15.151:0/3080546147 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f5e180a3e30 0x7f5e180a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:06:31.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:31.837+0000 7f5e35600700 1 --2- 172.21.15.151:0/3080546147 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f5e180a3e30 0x7f5e180a62f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5e24009c10 tx=0x7f5e2400b040 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:06:31.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:31.841+0000 7f5e2e000700 1 -- 172.21.15.151:0/3080546147 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5e2809cb60 con 0x7f5e30108a30 2024-04-26T02:06:32.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:32.070+0000 7f5e38000700 1 -- 172.21.15.151:0/3080546147 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5e3002df80 con 0x7f5e30108a30 2024-04-26T02:06:32.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:32.128+0000 7f5e2e000700 1 -- 172.21.15.151:0/3080546147 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v111) ==== 181+0+0 (secure 0 0 0) 0x7f5e2809c460 con 0x7f5e30108a30 2024-04-26T02:06:32.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:32.133+0000 7f5e38000700 1 -- 172.21.15.151:0/3080546147 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f5e180a3e30 msgr2=0x7f5e180a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:06:32.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:32.133+0000 7f5e38000700 1 --2- 172.21.15.151:0/3080546147 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f5e180a3e30 0x7f5e180a62f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5e24009c10 tx=0x7f5e2400b040 comp rx=0 tx=0).stop 2024-04-26T02:06:32.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:32.133+0000 7f5e38000700 1 -- 172.21.15.151:0/3080546147 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e30108a30 msgr2=0x7f5e3010a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:06:32.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:32.133+0000 7f5e38000700 1 --2- 172.21.15.151:0/3080546147 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e30108a30 0x7f5e3010a400 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5e2800e9a0 tx=0x7f5e2800ecb0 comp rx=0 tx=0).stop 2024-04-26T02:06:32.133 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:32.133+0000 7f5e38000700 1 -- 172.21.15.151:0/3080546147 shutdown_connections 2024-04-26T02:06:32.133 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:32.133+0000 7f5e38000700 1 --2- 172.21.15.151:0/3080546147 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f5e180a3e30 0x7f5e180a62f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:32.133 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:32.133+0000 7f5e38000700 1 --2- 172.21.15.151:0/3080546147 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e30108a30 0x7f5e3010a400 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:32.133 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:32.133+0000 7f5e38000700 1 --2- 172.21.15.151:0/3080546147 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5e3006a750 0x7f5e30109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:32.133 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:32.133+0000 7f5e38000700 1 --2- 172.21.15.151:0/3080546147 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e3006b0f0 0x7f5e30109ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:32.133 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:32.133+0000 7f5e38000700 1 -- 172.21.15.151:0/3080546147 >> 172.21.15.151:0/3080546147 conn(0x7f5e300788f0 msgr2=0x7f5e3010df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:06:32.133 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:32.133+0000 7f5e38000700 1 -- 172.21.15.151:0/3080546147 shutdown_connections 2024-04-26T02:06:32.133 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:32.133+0000 7f5e38000700 1 -- 172.21.15.151:0/3080546147 wait complete. 2024-04-26T02:06:32.136 INFO:teuthology.orchestra.run.smithi151.stderr:pool 'cephfs_metadata' removed 2024-04-26T02:06:32.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:32 smithi151 ceph-mon[28057]: pgmap v407: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 3 op/s 2024-04-26T02:06:32.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:32 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/3080546147' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T02:06:32.373 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:32 smithi151 ceph-mon[28057]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T02:06:32.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:32 smithi172 ceph-mon[32785]: pgmap v407: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 3 op/s 2024-04-26T02:06:32.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:32 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/3080546147' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T02:06:32.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:32 smithi172 ceph-mon[32785]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T02:06:32.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:32 smithi165 ceph-mon[32648]: pgmap v407: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 3 op/s 2024-04-26T02:06:32.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:32 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/3080546147' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T02:06:32.597 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:32 smithi165 ceph-mon[32648]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T02:06:33.080 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid c0e49e64-036d-11ef-bc93-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T02:06:33.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:33 smithi151 ceph-mon[28057]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T02:06:33.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:33 smithi151 ceph-mon[28057]: osdmap e111: 12 total, 12 up, 12 in 2024-04-26T02:06:33.536 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:06:33.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:33 smithi172 ceph-mon[32785]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T02:06:33.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:33 smithi172 ceph-mon[32785]: osdmap e111: 12 total, 12 up, 12 in 2024-04-26T02:06:33.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:33 smithi165 ceph-mon[32648]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T02:06:33.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:33 smithi165 ceph-mon[32648]: osdmap e111: 12 total, 12 up, 12 in 2024-04-26T02:06:34.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:34 smithi151 ceph-mon[28057]: pgmap v409: 65 pgs: 65 active+clean; 162 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T02:06:34.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:34 smithi172 ceph-mon[32785]: pgmap v409: 65 pgs: 65 active+clean; 162 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T02:06:34.596 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:34 smithi165 ceph-mon[32648]: pgmap v409: 65 pgs: 65 active+clean; 162 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T02:06:35.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:35.722+0000 7f01e0e00700 1 -- 172.21.15.151:0/1344288021 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01dc107940 msgr2=0x7f01dc107d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:06:35.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:35.722+0000 7f01e0e00700 1 --2- 172.21.15.151:0/1344288021 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01dc107940 0x7f01dc107d20 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f01cc0099c0 tx=0x7f01cc009cd0 comp rx=0 tx=0).stop 2024-04-26T02:06:35.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:35.723+0000 7f01e0e00700 1 -- 172.21.15.151:0/1344288021 shutdown_connections 2024-04-26T02:06:35.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:35.723+0000 7f01e0e00700 1 --2- 172.21.15.151:0/1344288021 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01dc1034b0 0x7f01dc10f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:35.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:35.723+0000 7f01e0e00700 1 --2- 172.21.15.151:0/1344288021 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01dc107940 0x7f01dc107d20 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:35.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:35.723+0000 7f01e0e00700 1 --2- 172.21.15.151:0/1344288021 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01dc102af0 0x7f01dc102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:35.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:35.723+0000 7f01e0e00700 1 -- 172.21.15.151:0/1344288021 >> 172.21.15.151:0/1344288021 conn(0x7f01dc079ba0 msgr2=0x7f01dc079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:06:35.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:35.723+0000 7f01e0e00700 1 -- 172.21.15.151:0/1344288021 shutdown_connections 2024-04-26T02:06:35.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:35.723+0000 7f01e0e00700 1 -- 172.21.15.151:0/1344288021 wait complete. 2024-04-26T02:06:35.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:35.724+0000 7f01e0e00700 1 Processor -- start 2024-04-26T02:06:35.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:35.725+0000 7f01e0e00700 1 -- start start 2024-04-26T02:06:35.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:35.725+0000 7f01e0e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01dc102af0 0x7f01dc074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:06:35.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:35.725+0000 7f01e0e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01dc1034b0 0x7f01dc075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:06:35.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:35.725+0000 7f01e0e00700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01dc107940 0x7f01dc0757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:06:35.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:35.726+0000 7f01e0e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f01dc112c90 con 0x7f01dc102af0 2024-04-26T02:06:35.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:35.726+0000 7f01e0e00700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f01dc112b10 con 0x7f01dc107940 2024-04-26T02:06:35.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:35.726+0000 7f01e0e00700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f01dc112e10 con 0x7f01dc1034b0 2024-04-26T02:06:35.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:35.726+0000 7f01d9e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01dc102af0 0x7f01dc074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:06:35.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:35.726+0000 7f01d9e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01dc102af0 0x7f01dc074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:34726/0 (socket says 172.21.15.151:34726) 2024-04-26T02:06:35.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:35.726+0000 7f01d9e00700 1 -- 172.21.15.151:0/4094756870 learned_addr learned my addr 172.21.15.151:0/4094756870 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-04-26T02:06:35.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:35.726+0000 7f01da800700 1 --2- 172.21.15.151:0/4094756870 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01dc107940 0x7f01dc0757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:06:35.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:35.726+0000 7f01d1400700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01dc1034b0 0x7f01dc075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:06:35.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:35.726+0000 7f01da800700 1 -- 172.21.15.151:0/4094756870 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01dc1034b0 msgr2=0x7f01dc075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:06:35.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:35.726+0000 7f01da800700 1 --2- 172.21.15.151:0/4094756870 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01dc1034b0 0x7f01dc075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:35.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:35.726+0000 7f01da800700 1 -- 172.21.15.151:0/4094756870 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01dc102af0 msgr2=0x7f01dc074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:06:35.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:35.726+0000 7f01da800700 1 --2- 172.21.15.151:0/4094756870 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01dc102af0 0x7f01dc074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:35.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:35.726+0000 7f01da800700 1 -- 172.21.15.151:0/4094756870 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01dc1a75f0 con 0x7f01dc107940 2024-04-26T02:06:35.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:35.726+0000 7f01d9e00700 1 --2- 172.21.15.151:0/4094756870 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01dc102af0 0x7f01dc074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:06:35.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:35.727+0000 7f01da800700 1 --2- 172.21.15.151:0/4094756870 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01dc107940 0x7f01dc0757c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f01d400efc0 tx=0x7f01d400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:06:35.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:35.727+0000 7f01d3400700 1 -- 172.21.15.151:0/4094756870 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d40092e0 con 0x7f01dc107940 2024-04-26T02:06:35.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:35.727+0000 7f01e0e00700 1 -- 172.21.15.151:0/4094756870 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01dc1a7880 con 0x7f01dc107940 2024-04-26T02:06:35.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:35.727+0000 7f01d3400700 1 -- 172.21.15.151:0/4094756870 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01d4004bb0 con 0x7f01dc107940 2024-04-26T02:06:35.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:35.727+0000 7f01d3400700 1 -- 172.21.15.151:0/4094756870 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d4004020 con 0x7f01dc107940 2024-04-26T02:06:35.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:35.727+0000 7f01e0e00700 1 -- 172.21.15.151:0/4094756870 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01dc1a7e10 con 0x7f01dc107940 2024-04-26T02:06:35.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:35.728+0000 7f01e0e00700 1 -- 172.21.15.151:0/4094756870 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01dc069730 con 0x7f01dc107940 2024-04-26T02:06:35.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:35.730+0000 7f01d3400700 1 -- 172.21.15.151:0/4094756870 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 138778+0+0 (secure 0 0 0) 0x7f01d401b030 con 0x7f01dc107940 2024-04-26T02:06:35.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:35.732+0000 7f01d3400700 1 --2- 172.21.15.151:0/4094756870 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f01c00a3db0 0x7f01c00a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:06:35.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:35.732+0000 7f01d3400700 1 -- 172.21.15.151:0/4094756870 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8173+0+0 (secure 0 0 0) 0x7f01d40d0500 con 0x7f01dc107940 2024-04-26T02:06:35.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:35.732+0000 7f01d9e00700 1 --2- 172.21.15.151:0/4094756870 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f01c00a3db0 0x7f01c00a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:06:35.732 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:35.733+0000 7f01d9e00700 1 --2- 172.21.15.151:0/4094756870 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f01c00a3db0 0x7f01c00a6270 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f01cc009990 tx=0x7f01cc005ab0 comp rx=0 tx=0).ready entity=mgr.25025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:06:35.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:35.734+0000 7f01d3400700 1 -- 172.21.15.151:0/4094756870 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f01d4048080 con 0x7f01dc107940 2024-04-26T02:06:35.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:35.966+0000 7f01e0e00700 1 -- 172.21.15.151:0/4094756870 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f01dc02e140 con 0x7f01dc107940 2024-04-26T02:06:36.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:36.144+0000 7f01d3400700 1 -- 172.21.15.151:0/4094756870 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v112) ==== 169+0+0 (secure 0 0 0) 0x7f01d4022020 con 0x7f01dc107940 2024-04-26T02:06:36.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:36.150+0000 7f01e0e00700 1 -- 172.21.15.151:0/4094756870 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f01c00a3db0 msgr2=0x7f01c00a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:06:36.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:36.151+0000 7f01e0e00700 1 --2- 172.21.15.151:0/4094756870 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f01c00a3db0 0x7f01c00a6270 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f01cc009990 tx=0x7f01cc005ab0 comp rx=0 tx=0).stop 2024-04-26T02:06:36.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:36.151+0000 7f01e0e00700 1 -- 172.21.15.151:0/4094756870 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01dc107940 msgr2=0x7f01dc0757c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:06:36.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:36.151+0000 7f01e0e00700 1 --2- 172.21.15.151:0/4094756870 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01dc107940 0x7f01dc0757c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f01d400efc0 tx=0x7f01d400c3a0 comp rx=0 tx=0).stop 2024-04-26T02:06:36.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:36.151+0000 7f01e0e00700 1 -- 172.21.15.151:0/4094756870 shutdown_connections 2024-04-26T02:06:36.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:36.151+0000 7f01e0e00700 1 --2- 172.21.15.151:0/4094756870 >> [v2:172.21.15.151:6800/2427564083,v1:172.21.15.151:6801/2427564083] conn(0x7f01c00a3db0 0x7f01c00a6270 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:36.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:36.151+0000 7f01e0e00700 1 --2- 172.21.15.151:0/4094756870 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01dc107940 0x7f01dc0757c0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:36.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:36.151+0000 7f01e0e00700 1 --2- 172.21.15.151:0/4094756870 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01dc102af0 0x7f01dc074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:36.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:36.151+0000 7f01e0e00700 1 --2- 172.21.15.151:0/4094756870 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01dc1034b0 0x7f01dc075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:06:36.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:36.151+0000 7f01e0e00700 1 -- 172.21.15.151:0/4094756870 >> 172.21.15.151:0/4094756870 conn(0x7f01dc079ba0 msgr2=0x7f01dc100b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:06:36.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:36.151+0000 7f01e0e00700 1 -- 172.21.15.151:0/4094756870 shutdown_connections 2024-04-26T02:06:36.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-04-26T02:06:36.152+0000 7f01e0e00700 1 -- 172.21.15.151:0/4094756870 wait complete. 2024-04-26T02:06:36.154 INFO:teuthology.orchestra.run.smithi151.stderr:pool 'cephfs_data' removed 2024-04-26T02:06:36.372 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:36 smithi151 ceph-mon[28057]: pgmap v410: 65 pgs: 65 active+clean; 162 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T02:06:36.373 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:36 smithi151 ceph-mon[28057]: from='client.? 172.21.15.151:0/4094756870' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T02:06:36.373 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:36 smithi151 ceph-mon[28057]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T02:06:36.399 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:36 smithi165 ceph-mon[32648]: pgmap v410: 65 pgs: 65 active+clean; 162 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T02:06:36.399 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:36 smithi165 ceph-mon[32648]: from='client.? 172.21.15.151:0/4094756870' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T02:06:36.399 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:36 smithi165 ceph-mon[32648]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T02:06:36.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:36 smithi172 ceph-mon[32785]: pgmap v410: 65 pgs: 65 active+clean; 162 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T02:06:36.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:36 smithi172 ceph-mon[32785]: from='client.? 172.21.15.151:0/4094756870' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T02:06:36.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:36 smithi172 ceph-mon[32785]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T02:06:36.970 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T02:06:37.068 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T02:06:37.094 DEBUG:teuthology.orchestra.run.smithi172:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T02:06:37.121 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-26T02:06:37.121 DEBUG:teuthology.orchestra.run.smithi151:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-26T02:06:37.177 DEBUG:teuthology.orchestra.run.smithi165:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-26T02:06:37.191 DEBUG:teuthology.orchestra.run.smithi172:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-26T02:06:37.205 INFO:tasks.cephadm:Stopping all daemons... 2024-04-26T02:06:37.205 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-26T02:06:37.206 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mon.a 2024-04-26T02:06:37.346 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:37 smithi165 ceph-mon[32648]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T02:06:37.347 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:37 smithi165 ceph-mon[32648]: osdmap e112: 12 total, 12 up, 12 in 2024-04-26T02:06:37.372 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:37 smithi172 ceph-mon[32785]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T02:06:37.372 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:37 smithi172 ceph-mon[32785]: osdmap e112: 12 total, 12 up, 12 in 2024-04-26T02:06:37.623 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:37 smithi151 ceph-mon[28057]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T02:06:37.623 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:37 smithi151 ceph-mon[28057]: osdmap e112: 12 total, 12 up, 12 in 2024-04-26T02:06:37.623 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:37 smithi151 systemd[1]: Stopping Ceph mon.a for c0e49e64-036d-11ef-bc93-c7b262605968... 2024-04-26T02:06:38.123 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:37 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mon-a[28052]: 2024-04-26T02:06:37.789+0000 7f988f800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:06:38.123 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:37 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mon-a[28052]: 2024-04-26T02:06:37.789+0000 7f988f800700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-26T02:06:38.123 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:38 smithi151 podman[119505]: 2024-04-26 02:06:38.014479203 +0000 UTC m=+0.366322940 container died a43466754df1da1ddbf7029de95a586187e5aa19159900abd356e4f333a7083d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-26T02:06:38.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:38 smithi151 podman[119505]: 2024-04-26 02:06:38.476259529 +0000 UTC m=+0.828103305 container cleanup a43466754df1da1ddbf7029de95a586187e5aa19159900abd356e4f333a7083d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T02:06:38.623 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:38 smithi151 bash[119505]: ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mon-a 2024-04-26T02:06:38.965 INFO:journalctl@ceph.mon.a.smithi151.stdout:Apr 26 02:06:38 smithi151 podman[119518]: 2024-04-26 02:06:38.7523567 +0000 UTC m=+0.731273134 container remove a43466754df1da1ddbf7029de95a586187e5aa19159900abd356e4f333a7083d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mon-a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416) 2024-04-26T02:06:39.149 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mon.a.service' 2024-04-26T02:06:39.270 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:06:39.270 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-26T02:06:39.270 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-26T02:06:39.270 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mon.b 2024-04-26T02:06:39.580 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:39 smithi165 systemd[1]: Stopping Ceph mon.b for c0e49e64-036d-11ef-bc93-c7b262605968... 2024-04-26T02:06:39.847 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:39 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mon-b[32644]: 2024-04-26T02:06:39.580+0000 7f4037a82700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:06:39.847 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:39 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mon-b[32644]: 2024-04-26T02:06:39.580+0000 7f4037a82700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-26T02:06:39.847 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:39 smithi165 podman[69695]: 2024-04-26 02:06:39.734423737 +0000 UTC m=+0.264721992 container died 5a1fc8d546479937c1e10d8d26d9f177b39a679775328a69eb0c402025b490bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2) 2024-04-26T02:06:40.315 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:40 smithi165 podman[69695]: 2024-04-26 02:06:40.107136658 +0000 UTC m=+0.637434924 container cleanup 5a1fc8d546479937c1e10d8d26d9f177b39a679775328a69eb0c402025b490bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-26T02:06:40.315 INFO:journalctl@ceph.mon.b.smithi165.stdout:Apr 26 02:06:40 smithi165 bash[69695]: ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mon-b 2024-04-26T02:06:40.517 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mon.b.service' 2024-04-26T02:06:40.565 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:06:40.565 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-26T02:06:40.565 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-26T02:06:40.565 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mon.c 2024-04-26T02:06:40.872 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:40 smithi172 systemd[1]: Stopping Ceph mon.c for c0e49e64-036d-11ef-bc93-c7b262605968... 2024-04-26T02:06:41.222 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:40 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mon-c[32773]: 2024-04-26T02:06:40.873+0000 7f7a2508d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:06:41.222 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:40 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mon-c[32773]: 2024-04-26T02:06:40.873+0000 7f7a2508d700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-26T02:06:41.222 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:40 smithi172 podman[70719]: 2024-04-26 02:06:40.934486354 +0000 UTC m=+0.167507910 container died 655ff9c7ac337b12b0d565df9db911e444f5a0056a26cfc3ab6d4447cde4b325 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mon-c, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-26T02:06:41.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:41 smithi172 podman[70719]: 2024-04-26 02:06:41.314600827 +0000 UTC m=+0.547622373 container cleanup 655ff9c7ac337b12b0d565df9db911e444f5a0056a26cfc3ab6d4447cde4b325 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mon-c, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-26T02:06:41.537 INFO:journalctl@ceph.mon.c.smithi172.stdout:Apr 26 02:06:41 smithi172 bash[70719]: ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mon-c 2024-04-26T02:06:41.724 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mon.c.service' 2024-04-26T02:06:41.780 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:06:41.780 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-26T02:06:41.780 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-26T02:06:41.780 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mgr.x 2024-04-26T02:06:42.122 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 02:06:41 smithi151 systemd[1]: Stopping Ceph mgr.x for c0e49e64-036d-11ef-bc93-c7b262605968... 2024-04-26T02:06:42.819 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 02:06:42 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T02:06:42.394+0000 7fba63800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:06:42.819 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 02:06:42 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x[28295]: 2024-04-26T02:06:42.394+0000 7fba63800700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-04-26T02:06:42.819 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 02:06:42 smithi151 podman[119638]: 2024-04-26 02:06:42.489187003 +0000 UTC m=+0.291750329 container died 463671658fa6ff519b25f1dba200036f07730f97c4647dadfa73cf303ba45bb5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-26T02:06:43.122 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 02:06:42 smithi151 podman[119638]: 2024-04-26 02:06:42.997615988 +0000 UTC m=+0.800179348 container cleanup 463671658fa6ff519b25f1dba200036f07730f97c4647dadfa73cf303ba45bb5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T02:06:43.123 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 02:06:42 smithi151 bash[119638]: ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x 2024-04-26T02:06:43.622 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Apr 26 02:06:43 smithi151 podman[119652]: 2024-04-26 02:06:43.274082169 +0000 UTC m=+0.771317822 container remove 463671658fa6ff519b25f1dba200036f07730f97c4647dadfa73cf303ba45bb5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-x, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-26T02:06:43.855 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-26T02:06:43.961 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:06:43.961 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-26T02:06:43.961 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-26T02:06:43.962 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mgr.y 2024-04-26T02:06:44.296 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 02:06:44 smithi165 systemd[1]: Stopping Ceph mgr.y for c0e49e64-036d-11ef-bc93-c7b262605968... 2024-04-26T02:06:44.552 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 02:06:44 smithi165 podman[69818]: 2024-04-26 02:06:44.298446203 +0000 UTC m=+0.133359122 container died 1faa05cec8e2000966940378229b9014670d653393f861025e5dbaa1a2d4dfe1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240416) 2024-04-26T02:06:44.843 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 02:06:44 smithi165 podman[69818]: 2024-04-26 02:06:44.644108146 +0000 UTC m=+0.479021054 container cleanup 1faa05cec8e2000966940378229b9014670d653393f861025e5dbaa1a2d4dfe1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-26T02:06:44.844 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Apr 26 02:06:44 smithi165 bash[69818]: ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-y 2024-04-26T02:06:45.052 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-26T02:06:45.085 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:06:45.085 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-26T02:06:45.085 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-26T02:06:45.085 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mgr.z 2024-04-26T02:06:45.537 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 02:06:45 smithi172 systemd[1]: Stopping Ceph mgr.z for c0e49e64-036d-11ef-bc93-c7b262605968... 2024-04-26T02:06:45.901 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 02:06:45 smithi172 podman[70840]: 2024-04-26 02:06:45.633527111 +0000 UTC m=+0.350805671 container died 0171154f9dfead9162455177ee255154c83c87799c120c454f2c8261a48c9203 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-26T02:06:46.260 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 02:06:46 smithi172 podman[70840]: 2024-04-26 02:06:46.01112999 +0000 UTC m=+0.728408553 container cleanup 0171154f9dfead9162455177ee255154c83c87799c120c454f2c8261a48c9203 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-z, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-26T02:06:46.260 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Apr 26 02:06:46 smithi172 bash[70840]: ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mgr-z 2024-04-26T02:06:46.432 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-26T02:06:46.465 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:06:46.465 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-26T02:06:46.465 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-26T02:06:46.465 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-c0e49e64-036d-11ef-bc93-c7b262605968@osd.0 2024-04-26T02:06:46.872 INFO:journalctl@ceph.osd.0.smithi151.stdout:Apr 26 02:06:46 smithi151 systemd[1]: Stopping Ceph osd.0 for c0e49e64-036d-11ef-bc93-c7b262605968... 2024-04-26T02:06:47.372 INFO:journalctl@ceph.osd.0.smithi151.stdout:Apr 26 02:06:47 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-0[42271]: 2024-04-26T02:06:47.103+0000 7f1d0aa00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:06:47.372 INFO:journalctl@ceph.osd.0.smithi151.stdout:Apr 26 02:06:47 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-0[42271]: 2024-04-26T02:06:47.103+0000 7f1d0aa00700 -1 osd.0 112 *** Got signal Terminated *** 2024-04-26T02:06:47.373 INFO:journalctl@ceph.osd.0.smithi151.stdout:Apr 26 02:06:47 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-0[42271]: 2024-04-26T02:06:47.103+0000 7f1d0aa00700 -1 osd.0 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T02:06:52.994 INFO:journalctl@ceph.osd.0.smithi151.stdout:Apr 26 02:06:52 smithi151 podman[119758]: 2024-04-26 02:06:52.652553273 +0000 UTC m=+5.735159878 container died 0c6bf446b2c6dff776680bb9ea2283e59f484ad0916ae73ddca09bcf85d6a9bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-26T02:06:53.314 INFO:journalctl@ceph.osd.0.smithi151.stdout:Apr 26 02:06:53 smithi151 podman[119758]: 2024-04-26 02:06:53.113903375 +0000 UTC m=+6.196510040 container cleanup 0c6bf446b2c6dff776680bb9ea2283e59f484ad0916ae73ddca09bcf85d6a9bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-26T02:06:53.315 INFO:journalctl@ceph.osd.0.smithi151.stdout:Apr 26 02:06:53 smithi151 bash[119758]: ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-0 2024-04-26T02:06:53.622 INFO:journalctl@ceph.osd.0.smithi151.stdout:Apr 26 02:06:53 smithi151 podman[119777]: 2024-04-26 02:06:53.315187938 +0000 UTC m=+0.645143538 container remove 0c6bf446b2c6dff776680bb9ea2283e59f484ad0916ae73ddca09bcf85d6a9bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T02:06:54.489 INFO:journalctl@ceph.osd.0.smithi151.stdout:Apr 26 02:06:54 smithi151 podman[119865]: 2024-04-26 02:06:54.058637045 +0000 UTC m=+0.037028307 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T02:06:54.489 INFO:journalctl@ceph.osd.0.smithi151.stdout:Apr 26 02:06:54 smithi151 podman[119865]: 2024-04-26 02:06:54.166450618 +0000 UTC m=+0.144841913 container create 2f25c47f7bdcb5565b52336a292283cd4f56e9b4abc2eaa249e4bc09810f1670 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416) 2024-04-26T02:06:54.489 INFO:journalctl@ceph.osd.0.smithi151.stdout:Apr 26 02:06:54 smithi151 podman[119865]: 2024-04-26 02:06:54.489880419 +0000 UTC m=+0.468271718 container init 2f25c47f7bdcb5565b52336a292283cd4f56e9b4abc2eaa249e4bc09810f1670 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-0-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-26T02:06:54.872 INFO:journalctl@ceph.osd.0.smithi151.stdout:Apr 26 02:06:54 smithi151 podman[119865]: 2024-04-26 02:06:54.501473532 +0000 UTC m=+0.479864831 container start 2f25c47f7bdcb5565b52336a292283cd4f56e9b4abc2eaa249e4bc09810f1670 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-0-deactivate, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-26T02:06:54.873 INFO:journalctl@ceph.osd.0.smithi151.stdout:Apr 26 02:06:54 smithi151 podman[119865]: 2024-04-26 02:06:54.646448484 +0000 UTC m=+0.624839809 container attach 2f25c47f7bdcb5565b52336a292283cd4f56e9b4abc2eaa249e4bc09810f1670 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416) 2024-04-26T02:06:55.335 INFO:journalctl@ceph.osd.0.smithi151.stdout:Apr 26 02:06:54 smithi151 podman[119865]: 2024-04-26 02:06:54.957522787 +0000 UTC m=+0.935914103 container died 2f25c47f7bdcb5565b52336a292283cd4f56e9b4abc2eaa249e4bc09810f1670 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0) 2024-04-26T02:06:55.702 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0e49e64-036d-11ef-bc93-c7b262605968@osd.0.service' 2024-04-26T02:06:55.807 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:06:55.807 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-26T02:06:55.807 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-26T02:06:55.807 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-c0e49e64-036d-11ef-bc93-c7b262605968@osd.1 2024-04-26T02:06:56.096 INFO:journalctl@ceph.osd.1.smithi165.stdout:Apr 26 02:06:55 smithi165 systemd[1]: Stopping Ceph osd.1 for c0e49e64-036d-11ef-bc93-c7b262605968... 2024-04-26T02:06:56.596 INFO:journalctl@ceph.osd.1.smithi165.stdout:Apr 26 02:06:56 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-1[37251]: 2024-04-26T02:06:56.097+0000 7f0f544fd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:06:56.597 INFO:journalctl@ceph.osd.1.smithi165.stdout:Apr 26 02:06:56 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-1[37251]: 2024-04-26T02:06:56.097+0000 7f0f544fd700 -1 osd.1 112 *** Got signal Terminated *** 2024-04-26T02:06:56.597 INFO:journalctl@ceph.osd.1.smithi165.stdout:Apr 26 02:06:56 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-1[37251]: 2024-04-26T02:06:56.097+0000 7f0f544fd700 -1 osd.1 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T02:07:01.862 INFO:journalctl@ceph.osd.1.smithi165.stdout:Apr 26 02:07:01 smithi165 podman[69943]: 2024-04-26 02:07:01.610626445 +0000 UTC m=+5.614960733 container died 3898f9c76b6065f8bee7152a4d44efe80d2822ed41514ca5135de08c86f77acc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-1, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-26T02:07:02.121 INFO:journalctl@ceph.osd.1.smithi165.stdout:Apr 26 02:07:01 smithi165 podman[69943]: 2024-04-26 02:07:01.955040176 +0000 UTC m=+5.959374459 container cleanup 3898f9c76b6065f8bee7152a4d44efe80d2822ed41514ca5135de08c86f77acc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-1, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-26T02:07:02.121 INFO:journalctl@ceph.osd.1.smithi165.stdout:Apr 26 02:07:01 smithi165 bash[69943]: ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-1 2024-04-26T02:07:02.566 INFO:journalctl@ceph.osd.1.smithi165.stdout:Apr 26 02:07:02 smithi165 podman[69962]: 2024-04-26 02:07:02.122817209 +0000 UTC m=+0.511050991 container remove 3898f9c76b6065f8bee7152a4d44efe80d2822ed41514ca5135de08c86f77acc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-1, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-26T02:07:02.849 INFO:journalctl@ceph.osd.1.smithi165.stdout:Apr 26 02:07:02 smithi165 podman[70056]: 2024-04-26 02:07:02.470581575 +0000 UTC m=+0.018060678 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T02:07:02.850 INFO:journalctl@ceph.osd.1.smithi165.stdout:Apr 26 02:07:02 smithi165 podman[70056]: 2024-04-26 02:07:02.581780434 +0000 UTC m=+0.129259528 container create 74d8e54eb409042e5dfd30f4a60e53cc29f4f94e6a80cbba65f83839bf5794a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-1-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-26T02:07:02.850 INFO:journalctl@ceph.osd.1.smithi165.stdout:Apr 26 02:07:02 smithi165 podman[70056]: 2024-04-26 02:07:02.792374237 +0000 UTC m=+0.339853331 container init 74d8e54eb409042e5dfd30f4a60e53cc29f4f94e6a80cbba65f83839bf5794a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2) 2024-04-26T02:07:02.850 INFO:journalctl@ceph.osd.1.smithi165.stdout:Apr 26 02:07:02 smithi165 podman[70056]: 2024-04-26 02:07:02.795455382 +0000 UTC m=+0.342934480 container start 74d8e54eb409042e5dfd30f4a60e53cc29f4f94e6a80cbba65f83839bf5794a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240416, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T02:07:03.122 INFO:journalctl@ceph.osd.1.smithi165.stdout:Apr 26 02:07:02 smithi165 podman[70056]: 2024-04-26 02:07:02.873288125 +0000 UTC m=+0.420767229 container attach 74d8e54eb409042e5dfd30f4a60e53cc29f4f94e6a80cbba65f83839bf5794a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-26T02:07:03.122 INFO:journalctl@ceph.osd.1.smithi165.stdout:Apr 26 02:07:03 smithi165 podman[70056]: 2024-04-26 02:07:03.090011308 +0000 UTC m=+0.637490403 container died 74d8e54eb409042e5dfd30f4a60e53cc29f4f94e6a80cbba65f83839bf5794a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5) 2024-04-26T02:07:03.650 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0e49e64-036d-11ef-bc93-c7b262605968@osd.1.service' 2024-04-26T02:07:03.687 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:07:03.688 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-26T02:07:03.688 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-26T02:07:03.688 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-c0e49e64-036d-11ef-bc93-c7b262605968@osd.2 2024-04-26T02:07:03.996 INFO:journalctl@ceph.osd.2.smithi172.stdout:Apr 26 02:07:03 smithi172 systemd[1]: Stopping Ceph osd.2 for c0e49e64-036d-11ef-bc93-c7b262605968... 2024-04-26T02:07:04.287 INFO:journalctl@ceph.osd.2.smithi172.stdout:Apr 26 02:07:03 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-2[37925]: 2024-04-26T02:07:03.996+0000 7f95c8993700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:07:04.287 INFO:journalctl@ceph.osd.2.smithi172.stdout:Apr 26 02:07:03 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-2[37925]: 2024-04-26T02:07:03.996+0000 7f95c8993700 -1 osd.2 112 *** Got signal Terminated *** 2024-04-26T02:07:04.287 INFO:journalctl@ceph.osd.2.smithi172.stdout:Apr 26 02:07:03 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-2[37925]: 2024-04-26T02:07:03.996+0000 7f95c8993700 -1 osd.2 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T02:07:09.787 INFO:journalctl@ceph.osd.2.smithi172.stdout:Apr 26 02:07:09 smithi172 podman[70966]: 2024-04-26 02:07:09.533425695 +0000 UTC m=+5.654071077 container died b763478b7595a39d8bd6393d48eb5581ebcfa5b2091916852406ed2831236c41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True) 2024-04-26T02:07:10.146 INFO:journalctl@ceph.osd.2.smithi172.stdout:Apr 26 02:07:09 smithi172 podman[70966]: 2024-04-26 02:07:09.91376172 +0000 UTC m=+6.034407088 container cleanup b763478b7595a39d8bd6393d48eb5581ebcfa5b2091916852406ed2831236c41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-2, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T02:07:10.146 INFO:journalctl@ceph.osd.2.smithi172.stdout:Apr 26 02:07:09 smithi172 bash[70966]: ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-2 2024-04-26T02:07:10.494 INFO:journalctl@ceph.osd.2.smithi172.stdout:Apr 26 02:07:10 smithi172 podman[70982]: 2024-04-26 02:07:10.174494594 +0000 UTC m=+0.642337456 container remove b763478b7595a39d8bd6393d48eb5581ebcfa5b2091916852406ed2831236c41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-26T02:07:10.787 INFO:journalctl@ceph.osd.2.smithi172.stdout:Apr 26 02:07:10 smithi172 podman[71079]: 2024-04-26 02:07:10.39867194 +0000 UTC m=+0.018217893 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T02:07:10.787 INFO:journalctl@ceph.osd.2.smithi172.stdout:Apr 26 02:07:10 smithi172 podman[71079]: 2024-04-26 02:07:10.5341719 +0000 UTC m=+0.153717825 container create ca5fb348ed27802ab6bb59dc4b233f252ad9c525334d96c3fb39585892301f57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416) 2024-04-26T02:07:11.096 INFO:journalctl@ceph.osd.2.smithi172.stdout:Apr 26 02:07:10 smithi172 podman[71079]: 2024-04-26 02:07:10.800470693 +0000 UTC m=+0.420016619 container init ca5fb348ed27802ab6bb59dc4b233f252ad9c525334d96c3fb39585892301f57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-26T02:07:11.097 INFO:journalctl@ceph.osd.2.smithi172.stdout:Apr 26 02:07:10 smithi172 podman[71079]: 2024-04-26 02:07:10.803586678 +0000 UTC m=+0.423132606 container start ca5fb348ed27802ab6bb59dc4b233f252ad9c525334d96c3fb39585892301f57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-26T02:07:11.097 INFO:journalctl@ceph.osd.2.smithi172.stdout:Apr 26 02:07:10 smithi172 podman[71079]: 2024-04-26 02:07:10.878733529 +0000 UTC m=+0.498279462 container attach ca5fb348ed27802ab6bb59dc4b233f252ad9c525334d96c3fb39585892301f57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-26T02:07:11.396 INFO:journalctl@ceph.osd.2.smithi172.stdout:Apr 26 02:07:11 smithi172 podman[71079]: 2024-04-26 02:07:11.098580007 +0000 UTC m=+0.718125945 container died ca5fb348ed27802ab6bb59dc4b233f252ad9c525334d96c3fb39585892301f57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-26T02:07:11.693 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0e49e64-036d-11ef-bc93-c7b262605968@osd.2.service' 2024-04-26T02:07:11.731 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:07:11.731 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-26T02:07:11.731 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-26T02:07:11.732 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-c0e49e64-036d-11ef-bc93-c7b262605968@osd.3 2024-04-26T02:07:12.123 INFO:journalctl@ceph.osd.3.smithi151.stdout:Apr 26 02:07:11 smithi151 systemd[1]: Stopping Ceph osd.3 for c0e49e64-036d-11ef-bc93-c7b262605968... 2024-04-26T02:07:12.622 INFO:journalctl@ceph.osd.3.smithi151.stdout:Apr 26 02:07:12 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-3[49178]: 2024-04-26T02:07:12.299+0000 7fe1d0c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:07:12.623 INFO:journalctl@ceph.osd.3.smithi151.stdout:Apr 26 02:07:12 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-3[49178]: 2024-04-26T02:07:12.299+0000 7fe1d0c00700 -1 osd.3 112 *** Got signal Terminated *** 2024-04-26T02:07:12.623 INFO:journalctl@ceph.osd.3.smithi151.stdout:Apr 26 02:07:12 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-3[49178]: 2024-04-26T02:07:12.299+0000 7fe1d0c00700 -1 osd.3 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T02:07:13.372 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:13 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:13.021+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.647106+0000 front 2024-04-26T02:06:50.647828+0000 (oldest deadline 2024-04-26T02:07:12.947246+0000) 2024-04-26T02:07:14.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:13 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:13.606+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:07:14.122 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:13 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:13.995+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.647106+0000 front 2024-04-26T02:06:50.647828+0000 (oldest deadline 2024-04-26T02:07:12.947246+0000) 2024-04-26T02:07:14.622 INFO:journalctl@ceph.osd.3.smithi151.stdout:Apr 26 02:07:14 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-3[49178]: 2024-04-26T02:07:14.353+0000 7fe1c7400700 -1 osd.3 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.958262+0000 front 2024-04-26T02:06:51.958664+0000 (oldest deadline 2024-04-26T02:07:13.657887+0000) 2024-04-26T02:07:14.846 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:14 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:14.587+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:07:15.122 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:14 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:14.992+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.647106+0000 front 2024-04-26T02:06:50.647828+0000 (oldest deadline 2024-04-26T02:07:12.947246+0000) 2024-04-26T02:07:15.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:15 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:15.552+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:07:15.872 INFO:journalctl@ceph.osd.3.smithi151.stdout:Apr 26 02:07:15 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-3[49178]: 2024-04-26T02:07:15.390+0000 7fe1c7400700 -1 osd.3 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.958262+0000 front 2024-04-26T02:06:51.958664+0000 (oldest deadline 2024-04-26T02:07:13.657887+0000) 2024-04-26T02:07:16.287 INFO:journalctl@ceph.osd.5.smithi172.stdout:Apr 26 02:07:15 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-5[44593]: 2024-04-26T02:07:15.881+0000 7f609eb92700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.529851+0000 front 2024-04-26T02:06:50.529683+0000 (oldest deadline 2024-04-26T02:07:15.229503+0000) 2024-04-26T02:07:16.373 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:16 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:16.034+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.647106+0000 front 2024-04-26T02:06:50.647828+0000 (oldest deadline 2024-04-26T02:07:12.947246+0000) 2024-04-26T02:07:16.846 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:16 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:16.541+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:07:16.872 INFO:journalctl@ceph.osd.3.smithi151.stdout:Apr 26 02:07:16 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-3[49178]: 2024-04-26T02:07:16.404+0000 7fe1c7400700 -1 osd.3 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.958262+0000 front 2024-04-26T02:06:51.958664+0000 (oldest deadline 2024-04-26T02:07:13.657887+0000) 2024-04-26T02:07:17.287 INFO:journalctl@ceph.osd.5.smithi172.stdout:Apr 26 02:07:16 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-5[44593]: 2024-04-26T02:07:16.901+0000 7f609eb92700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.529851+0000 front 2024-04-26T02:06:50.529683+0000 (oldest deadline 2024-04-26T02:07:15.229503+0000) 2024-04-26T02:07:17.372 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:17 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:17.023+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.647106+0000 front 2024-04-26T02:06:50.647828+0000 (oldest deadline 2024-04-26T02:07:12.947246+0000) 2024-04-26T02:07:17.846 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:17 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:17.510+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:07:18.287 INFO:journalctl@ceph.osd.5.smithi172.stdout:Apr 26 02:07:17 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-5[44593]: 2024-04-26T02:07:17.912+0000 7f609eb92700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.529851+0000 front 2024-04-26T02:06:50.529683+0000 (oldest deadline 2024-04-26T02:07:15.229503+0000) 2024-04-26T02:07:18.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:17 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:17.924+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:07:18.321 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:17 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:17.979+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.647106+0000 front 2024-04-26T02:06:50.647828+0000 (oldest deadline 2024-04-26T02:07:12.947246+0000) 2024-04-26T02:07:18.322 INFO:journalctl@ceph.osd.3.smithi151.stdout:Apr 26 02:07:17 smithi151 podman[119969]: 2024-04-26 02:07:17.978655953 +0000 UTC m=+5.815913467 container died 6af14d958ee1568d6214ec7b38032f06859e99dc09ae8b565c1c3b8ff4ac96c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-3, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416) 2024-04-26T02:07:18.622 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:18 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:18.344+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.476432+0000 front 2024-04-26T02:06:50.476644+0000 (oldest deadline 2024-04-26T02:07:18.078167+0000) 2024-04-26T02:07:18.846 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:18 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:18.509+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:07:18.954 INFO:journalctl@ceph.osd.3.smithi151.stdout:Apr 26 02:07:18 smithi151 podman[119969]: 2024-04-26 02:07:18.728033399 +0000 UTC m=+6.565290906 container cleanup 6af14d958ee1568d6214ec7b38032f06859e99dc09ae8b565c1c3b8ff4ac96c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-3, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-26T02:07:18.954 INFO:journalctl@ceph.osd.3.smithi151.stdout:Apr 26 02:07:18 smithi151 bash[119969]: ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-3 2024-04-26T02:07:19.287 INFO:journalctl@ceph.osd.5.smithi172.stdout:Apr 26 02:07:18 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-5[44593]: 2024-04-26T02:07:18.953+0000 7f609eb92700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.529851+0000 front 2024-04-26T02:06:50.529683+0000 (oldest deadline 2024-04-26T02:07:15.229503+0000) 2024-04-26T02:07:19.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:18 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:18.900+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:07:19.367 INFO:journalctl@ceph.osd.3.smithi151.stdout:Apr 26 02:07:18 smithi151 podman[120001]: 2024-04-26 02:07:18.954660874 +0000 UTC m=+0.964245942 container remove 6af14d958ee1568d6214ec7b38032f06859e99dc09ae8b565c1c3b8ff4ac96c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-3, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True) 2024-04-26T02:07:19.367 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:18 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:18.974+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.647106+0000 front 2024-04-26T02:06:50.647828+0000 (oldest deadline 2024-04-26T02:07:12.947246+0000) 2024-04-26T02:07:19.622 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:19 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:19.367+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.476432+0000 front 2024-04-26T02:06:50.476644+0000 (oldest deadline 2024-04-26T02:07:18.078167+0000) 2024-04-26T02:07:19.849 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:19 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:19.496+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:07:20.006 INFO:journalctl@ceph.osd.3.smithi151.stdout:Apr 26 02:07:19 smithi151 podman[120088]: 2024-04-26 02:07:19.634370581 +0000 UTC m=+0.042177992 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T02:07:20.007 INFO:journalctl@ceph.osd.3.smithi151.stdout:Apr 26 02:07:19 smithi151 podman[120088]: 2024-04-26 02:07:19.764477879 +0000 UTC m=+0.172285300 container create 3a4ef4d04da20590dc6a7ed497b7c500240161f2bd0898fdcb37c9cb83af1152 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T02:07:20.287 INFO:journalctl@ceph.osd.5.smithi172.stdout:Apr 26 02:07:19 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-5[44593]: 2024-04-26T02:07:19.994+0000 7f609eb92700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.529851+0000 front 2024-04-26T02:06:50.529683+0000 (oldest deadline 2024-04-26T02:07:15.229503+0000) 2024-04-26T02:07:20.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:19 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:19.904+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:07:20.372 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:20 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:20.006+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.647106+0000 front 2024-04-26T02:06:50.647828+0000 (oldest deadline 2024-04-26T02:07:12.947246+0000) 2024-04-26T02:07:20.372 INFO:journalctl@ceph.osd.3.smithi151.stdout:Apr 26 02:07:20 smithi151 podman[120088]: 2024-04-26 02:07:20.102234122 +0000 UTC m=+0.510041549 container init 3a4ef4d04da20590dc6a7ed497b7c500240161f2bd0898fdcb37c9cb83af1152 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-3-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=) 2024-04-26T02:07:20.372 INFO:journalctl@ceph.osd.3.smithi151.stdout:Apr 26 02:07:20 smithi151 podman[120088]: 2024-04-26 02:07:20.112927275 +0000 UTC m=+0.520734703 container start 3a4ef4d04da20590dc6a7ed497b7c500240161f2bd0898fdcb37c9cb83af1152 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True) 2024-04-26T02:07:20.372 INFO:journalctl@ceph.osd.3.smithi151.stdout:Apr 26 02:07:20 smithi151 podman[120088]: 2024-04-26 02:07:20.217950232 +0000 UTC m=+0.625757637 container attach 3a4ef4d04da20590dc6a7ed497b7c500240161f2bd0898fdcb37c9cb83af1152 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-26T02:07:20.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:20 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:20.533+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:07:20.872 INFO:journalctl@ceph.osd.3.smithi151.stdout:Apr 26 02:07:20 smithi151 podman[120088]: 2024-04-26 02:07:20.565281355 +0000 UTC m=+0.973088769 container died 3a4ef4d04da20590dc6a7ed497b7c500240161f2bd0898fdcb37c9cb83af1152 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-3-deactivate, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T02:07:20.872 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:20 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:20.389+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.476432+0000 front 2024-04-26T02:06:50.476644+0000 (oldest deadline 2024-04-26T02:07:18.078167+0000) 2024-04-26T02:07:21.156 INFO:journalctl@ceph.osd.5.smithi172.stdout:Apr 26 02:07:21 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-5[44593]: 2024-04-26T02:07:21.039+0000 7f609eb92700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.529851+0000 front 2024-04-26T02:06:50.529683+0000 (oldest deadline 2024-04-26T02:07:15.229503+0000) 2024-04-26T02:07:21.156 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:20 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:20.859+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:07:21.288 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:21 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:21.013+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.647106+0000 front 2024-04-26T02:06:50.647828+0000 (oldest deadline 2024-04-26T02:07:12.947246+0000) 2024-04-26T02:07:21.333 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0e49e64-036d-11ef-bc93-c7b262605968@osd.3.service' 2024-04-26T02:07:21.443 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:07:21.443 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-26T02:07:21.444 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-26T02:07:21.444 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-c0e49e64-036d-11ef-bc93-c7b262605968@osd.4 2024-04-26T02:07:21.622 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:21 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:21.350+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.476432+0000 front 2024-04-26T02:06:50.476644+0000 (oldest deadline 2024-04-26T02:07:18.078167+0000) 2024-04-26T02:07:21.754 INFO:journalctl@ceph.osd.4.smithi165.stdout:Apr 26 02:07:21 smithi165 systemd[1]: Stopping Ceph osd.4 for c0e49e64-036d-11ef-bc93-c7b262605968... 2024-04-26T02:07:21.754 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:21 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:21.547+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:07:22.097 INFO:journalctl@ceph.osd.4.smithi165.stdout:Apr 26 02:07:21 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-4[43947]: 2024-04-26T02:07:21.755+0000 7f9e0f231700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:07:22.097 INFO:journalctl@ceph.osd.4.smithi165.stdout:Apr 26 02:07:21 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-4[43947]: 2024-04-26T02:07:21.755+0000 7f9e0f231700 -1 osd.4 112 *** Got signal Terminated *** 2024-04-26T02:07:22.097 INFO:journalctl@ceph.osd.4.smithi165.stdout:Apr 26 02:07:21 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-4[43947]: 2024-04-26T02:07:21.755+0000 7f9e0f231700 -1 osd.4 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T02:07:22.287 INFO:journalctl@ceph.osd.5.smithi172.stdout:Apr 26 02:07:22 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-5[44593]: 2024-04-26T02:07:22.013+0000 7f609eb92700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.529851+0000 front 2024-04-26T02:06:50.529683+0000 (oldest deadline 2024-04-26T02:07:15.229503+0000) 2024-04-26T02:07:22.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:21 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:21.885+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:07:22.321 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:22 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:22.027+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.647106+0000 front 2024-04-26T02:06:50.647828+0000 (oldest deadline 2024-04-26T02:07:12.947246+0000) 2024-04-26T02:07:22.622 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:22 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:22.322+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.476432+0000 front 2024-04-26T02:06:50.476644+0000 (oldest deadline 2024-04-26T02:07:18.078167+0000) 2024-04-26T02:07:22.846 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:22 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:22.533+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:07:22.847 INFO:journalctl@ceph.osd.4.smithi165.stdout:Apr 26 02:07:22 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-4[43947]: 2024-04-26T02:07:22.769+0000 7f9e0781d700 -1 osd.4 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:58.988683+0000 front 2024-04-26T02:06:58.988583+0000 (oldest deadline 2024-04-26T02:07:21.789257+0000) 2024-04-26T02:07:23.287 INFO:journalctl@ceph.osd.5.smithi172.stdout:Apr 26 02:07:23 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-5[44593]: 2024-04-26T02:07:23.043+0000 7f609eb92700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.529851+0000 front 2024-04-26T02:06:50.529683+0000 (oldest deadline 2024-04-26T02:07:15.229503+0000) 2024-04-26T02:07:23.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:22 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:22.871+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:07:23.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:22 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:22.871+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.934525+0000 front 2024-04-26T02:06:59.934498+0000 (oldest deadline 2024-04-26T02:07:22.834583+0000) 2024-04-26T02:07:23.302 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:23 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:23.002+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.647106+0000 front 2024-04-26T02:06:50.647828+0000 (oldest deadline 2024-04-26T02:07:12.947246+0000) 2024-04-26T02:07:23.622 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:23 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:23.303+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.476432+0000 front 2024-04-26T02:06:50.476644+0000 (oldest deadline 2024-04-26T02:07:18.078167+0000) 2024-04-26T02:07:23.775 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:23 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:23.513+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:07:24.096 INFO:journalctl@ceph.osd.4.smithi165.stdout:Apr 26 02:07:23 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-4[43947]: 2024-04-26T02:07:23.776+0000 7f9e0781d700 -1 osd.4 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:58.988683+0000 front 2024-04-26T02:06:58.988583+0000 (oldest deadline 2024-04-26T02:07:21.789257+0000) 2024-04-26T02:07:24.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:23 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:23.865+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:07:24.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:23 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:23.865+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.934525+0000 front 2024-04-26T02:06:59.934498+0000 (oldest deadline 2024-04-26T02:07:22.834583+0000) 2024-04-26T02:07:24.288 INFO:journalctl@ceph.osd.5.smithi172.stdout:Apr 26 02:07:24 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-5[44593]: 2024-04-26T02:07:24.012+0000 7f609eb92700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.529851+0000 front 2024-04-26T02:06:50.529683+0000 (oldest deadline 2024-04-26T02:07:15.229503+0000) 2024-04-26T02:07:24.288 INFO:journalctl@ceph.osd.5.smithi172.stdout:Apr 26 02:07:24 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-5[44593]: 2024-04-26T02:07:24.012+0000 7f609eb92700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:55.229956+0000 front 2024-04-26T02:06:55.230052+0000 (oldest deadline 2024-04-26T02:07:23.430591+0000) 2024-04-26T02:07:24.298 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:24 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:24.023+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.647106+0000 front 2024-04-26T02:06:50.647828+0000 (oldest deadline 2024-04-26T02:07:12.947246+0000) 2024-04-26T02:07:24.298 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:24 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:24.023+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:57.650031+0000 front 2024-04-26T02:06:57.649560+0000 (oldest deadline 2024-04-26T02:07:23.549741+0000) 2024-04-26T02:07:24.622 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:24 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:24.299+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.476432+0000 front 2024-04-26T02:06:50.476644+0000 (oldest deadline 2024-04-26T02:07:18.078167+0000) 2024-04-26T02:07:24.787 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:24 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:24.524+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.275620+0000 front 2024-04-26T02:06:59.275446+0000 (oldest deadline 2024-04-26T02:07:23.975481+0000) 2024-04-26T02:07:24.821 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:24 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:24.505+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:07:25.096 INFO:journalctl@ceph.osd.4.smithi165.stdout:Apr 26 02:07:24 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-4[43947]: 2024-04-26T02:07:24.823+0000 7f9e0781d700 -1 osd.4 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:58.988683+0000 front 2024-04-26T02:06:58.988583+0000 (oldest deadline 2024-04-26T02:07:21.789257+0000) 2024-04-26T02:07:25.274 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:24 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:24.975+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.647106+0000 front 2024-04-26T02:06:50.647828+0000 (oldest deadline 2024-04-26T02:07:12.947246+0000) 2024-04-26T02:07:25.274 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:24 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:24.975+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:57.650031+0000 front 2024-04-26T02:06:57.649560+0000 (oldest deadline 2024-04-26T02:07:23.549741+0000) 2024-04-26T02:07:25.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:24 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:24.912+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:07:25.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:24 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:24.912+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.934525+0000 front 2024-04-26T02:06:59.934498+0000 (oldest deadline 2024-04-26T02:07:22.834583+0000) 2024-04-26T02:07:25.287 INFO:journalctl@ceph.osd.5.smithi172.stdout:Apr 26 02:07:25 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-5[44593]: 2024-04-26T02:07:25.013+0000 7f609eb92700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.529851+0000 front 2024-04-26T02:06:50.529683+0000 (oldest deadline 2024-04-26T02:07:15.229503+0000) 2024-04-26T02:07:25.287 INFO:journalctl@ceph.osd.5.smithi172.stdout:Apr 26 02:07:25 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-5[44593]: 2024-04-26T02:07:25.013+0000 7f609eb92700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:55.229956+0000 front 2024-04-26T02:06:55.230052+0000 (oldest deadline 2024-04-26T02:07:23.430591+0000) 2024-04-26T02:07:25.622 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:25 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:25.275+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.476432+0000 front 2024-04-26T02:06:50.476644+0000 (oldest deadline 2024-04-26T02:07:18.078167+0000) 2024-04-26T02:07:25.623 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:25 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:25.275+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.682370+0000 front 2024-04-26T02:06:59.682519+0000 (oldest deadline 2024-04-26T02:07:24.982614+0000) 2024-04-26T02:07:25.787 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:25 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:25.487+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.275620+0000 front 2024-04-26T02:06:59.275446+0000 (oldest deadline 2024-04-26T02:07:23.975481+0000) 2024-04-26T02:07:25.814 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:25 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:25.537+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:07:26.096 INFO:journalctl@ceph.osd.4.smithi165.stdout:Apr 26 02:07:25 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-4[43947]: 2024-04-26T02:07:25.815+0000 7f9e0781d700 -1 osd.4 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:58.988683+0000 front 2024-04-26T02:06:58.988583+0000 (oldest deadline 2024-04-26T02:07:21.789257+0000) 2024-04-26T02:07:26.260 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:25 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:25.952+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.647106+0000 front 2024-04-26T02:06:50.647828+0000 (oldest deadline 2024-04-26T02:07:12.947246+0000) 2024-04-26T02:07:26.261 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:25 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:25.952+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:57.650031+0000 front 2024-04-26T02:06:57.649560+0000 (oldest deadline 2024-04-26T02:07:23.549741+0000) 2024-04-26T02:07:26.287 INFO:journalctl@ceph.osd.5.smithi172.stdout:Apr 26 02:07:26 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-5[44593]: 2024-04-26T02:07:26.055+0000 7f609eb92700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.529851+0000 front 2024-04-26T02:06:50.529683+0000 (oldest deadline 2024-04-26T02:07:15.229503+0000) 2024-04-26T02:07:26.287 INFO:journalctl@ceph.osd.5.smithi172.stdout:Apr 26 02:07:26 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-5[44593]: 2024-04-26T02:07:26.055+0000 7f609eb92700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:55.229956+0000 front 2024-04-26T02:06:55.230052+0000 (oldest deadline 2024-04-26T02:07:23.430591+0000) 2024-04-26T02:07:26.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:25 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:25.886+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:07:26.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:25 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:25.886+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.934525+0000 front 2024-04-26T02:06:59.934498+0000 (oldest deadline 2024-04-26T02:07:22.834583+0000) 2024-04-26T02:07:26.622 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:26 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:26.261+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.476432+0000 front 2024-04-26T02:06:50.476644+0000 (oldest deadline 2024-04-26T02:07:18.078167+0000) 2024-04-26T02:07:26.622 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:26 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:26.261+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.682370+0000 front 2024-04-26T02:06:59.682519+0000 (oldest deadline 2024-04-26T02:07:24.982614+0000) 2024-04-26T02:07:26.787 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:26 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:26.528+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.275620+0000 front 2024-04-26T02:06:59.275446+0000 (oldest deadline 2024-04-26T02:07:23.975481+0000) 2024-04-26T02:07:26.847 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:26 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:26.503+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.419661+0000 front 2024-04-26T02:07:00.419639+0000 (oldest deadline 2024-04-26T02:07:25.719796+0000) 2024-04-26T02:07:26.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:26 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:26.575+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:07:26.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:26 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:26.575+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.313401+0000 front 2024-04-26T02:07:00.313375+0000 (oldest deadline 2024-04-26T02:07:25.613566+0000) 2024-04-26T02:07:27.257 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:26 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:26.977+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.647106+0000 front 2024-04-26T02:06:50.647828+0000 (oldest deadline 2024-04-26T02:07:12.947246+0000) 2024-04-26T02:07:27.258 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:26 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:26.977+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:57.650031+0000 front 2024-04-26T02:06:57.649560+0000 (oldest deadline 2024-04-26T02:07:23.549741+0000) 2024-04-26T02:07:27.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:26 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:26.890+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:07:27.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:26 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:26.890+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.934525+0000 front 2024-04-26T02:06:59.934498+0000 (oldest deadline 2024-04-26T02:07:22.834583+0000) 2024-04-26T02:07:27.287 INFO:journalctl@ceph.osd.5.smithi172.stdout:Apr 26 02:07:27 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-5[44593]: 2024-04-26T02:07:27.059+0000 7f609eb92700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.529851+0000 front 2024-04-26T02:06:50.529683+0000 (oldest deadline 2024-04-26T02:07:15.229503+0000) 2024-04-26T02:07:27.287 INFO:journalctl@ceph.osd.5.smithi172.stdout:Apr 26 02:07:27 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-5[44593]: 2024-04-26T02:07:27.059+0000 7f609eb92700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:55.229956+0000 front 2024-04-26T02:06:55.230052+0000 (oldest deadline 2024-04-26T02:07:23.430591+0000) 2024-04-26T02:07:27.577 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:27 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:27.503+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.419661+0000 front 2024-04-26T02:07:00.419639+0000 (oldest deadline 2024-04-26T02:07:25.719796+0000) 2024-04-26T02:07:27.578 INFO:journalctl@ceph.osd.4.smithi165.stdout:Apr 26 02:07:27 smithi165 podman[70174]: 2024-04-26 02:07:27.316104344 +0000 UTC m=+5.680095252 container died fe0ca551b799e23e3506ebed744949cf41c96341033ac1bf233ea8863a5c7833 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-4, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T02:07:27.622 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:27 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:27.258+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.476432+0000 front 2024-04-26T02:06:50.476644+0000 (oldest deadline 2024-04-26T02:07:18.078167+0000) 2024-04-26T02:07:27.622 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:27 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:27.258+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.682370+0000 front 2024-04-26T02:06:59.682519+0000 (oldest deadline 2024-04-26T02:07:24.982614+0000) 2024-04-26T02:07:27.847 INFO:journalctl@ceph.osd.4.smithi165.stdout:Apr 26 02:07:27 smithi165 podman[70174]: 2024-04-26 02:07:27.613267041 +0000 UTC m=+5.977257939 container cleanup fe0ca551b799e23e3506ebed744949cf41c96341033ac1bf233ea8863a5c7833 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-4, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T02:07:27.847 INFO:journalctl@ceph.osd.4.smithi165.stdout:Apr 26 02:07:27 smithi165 bash[70174]: ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-4 2024-04-26T02:07:27.847 INFO:journalctl@ceph.osd.4.smithi165.stdout:Apr 26 02:07:27 smithi165 podman[70192]: 2024-04-26 02:07:27.797630598 +0000 UTC m=+0.484552857 container remove fe0ca551b799e23e3506ebed744949cf41c96341033ac1bf233ea8863a5c7833 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-4, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-26T02:07:27.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:27 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:27.578+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:07:27.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:27 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:27.578+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.313401+0000 front 2024-04-26T02:07:00.313375+0000 (oldest deadline 2024-04-26T02:07:25.613566+0000) 2024-04-26T02:07:27.925 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:27 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:27.545+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.275620+0000 front 2024-04-26T02:06:59.275446+0000 (oldest deadline 2024-04-26T02:07:23.975481+0000) 2024-04-26T02:07:28.282 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:27 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:27.994+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.647106+0000 front 2024-04-26T02:06:50.647828+0000 (oldest deadline 2024-04-26T02:07:12.947246+0000) 2024-04-26T02:07:28.283 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:27 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:27.994+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:57.650031+0000 front 2024-04-26T02:06:57.649560+0000 (oldest deadline 2024-04-26T02:07:23.549741+0000) 2024-04-26T02:07:28.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:27 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:27.926+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:07:28.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:27 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:27.926+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.934525+0000 front 2024-04-26T02:06:59.934498+0000 (oldest deadline 2024-04-26T02:07:22.834583+0000) 2024-04-26T02:07:28.287 INFO:journalctl@ceph.osd.5.smithi172.stdout:Apr 26 02:07:28 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-5[44593]: 2024-04-26T02:07:28.033+0000 7f609eb92700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.529851+0000 front 2024-04-26T02:06:50.529683+0000 (oldest deadline 2024-04-26T02:07:15.229503+0000) 2024-04-26T02:07:28.287 INFO:journalctl@ceph.osd.5.smithi172.stdout:Apr 26 02:07:28 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-5[44593]: 2024-04-26T02:07:28.033+0000 7f609eb92700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:55.229956+0000 front 2024-04-26T02:06:55.230052+0000 (oldest deadline 2024-04-26T02:07:23.430591+0000) 2024-04-26T02:07:28.399 INFO:journalctl@ceph.osd.4.smithi165.stdout:Apr 26 02:07:28 smithi165 podman[70286]: 2024-04-26 02:07:28.107098291 +0000 UTC m=+0.112090931 container create a896b2dbf4b012c1d802f4359fc94a67c67be3bb5e3713b1a74eea7844ae99fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-4-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-26T02:07:28.399 INFO:journalctl@ceph.osd.4.smithi165.stdout:Apr 26 02:07:28 smithi165 podman[70286]: 2024-04-26 02:07:28.012500908 +0000 UTC m=+0.017493506 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T02:07:28.399 INFO:journalctl@ceph.osd.4.smithi165.stdout:Apr 26 02:07:28 smithi165 podman[70286]: 2024-04-26 02:07:28.314463498 +0000 UTC m=+0.319456097 container init a896b2dbf4b012c1d802f4359fc94a67c67be3bb5e3713b1a74eea7844ae99fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-26T02:07:28.399 INFO:journalctl@ceph.osd.4.smithi165.stdout:Apr 26 02:07:28 smithi165 podman[70286]: 2024-04-26 02:07:28.317581859 +0000 UTC m=+0.322574458 container start a896b2dbf4b012c1d802f4359fc94a67c67be3bb5e3713b1a74eea7844ae99fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-4-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux ) 2024-04-26T02:07:28.400 INFO:journalctl@ceph.osd.4.smithi165.stdout:Apr 26 02:07:28 smithi165 podman[70286]: 2024-04-26 02:07:28.40085722 +0000 UTC m=+0.405849819 container attach a896b2dbf4b012c1d802f4359fc94a67c67be3bb5e3713b1a74eea7844ae99fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-26T02:07:28.622 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:28 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:28.284+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.476432+0000 front 2024-04-26T02:06:50.476644+0000 (oldest deadline 2024-04-26T02:07:18.078167+0000) 2024-04-26T02:07:28.622 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:28 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:28.284+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.682370+0000 front 2024-04-26T02:06:59.682519+0000 (oldest deadline 2024-04-26T02:07:24.982614+0000) 2024-04-26T02:07:28.847 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:28 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:28.491+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.419661+0000 front 2024-04-26T02:07:00.419639+0000 (oldest deadline 2024-04-26T02:07:25.719796+0000) 2024-04-26T02:07:28.847 INFO:journalctl@ceph.osd.4.smithi165.stdout:Apr 26 02:07:28 smithi165 podman[70286]: 2024-04-26 02:07:28.610871558 +0000 UTC m=+0.615864165 container died a896b2dbf4b012c1d802f4359fc94a67c67be3bb5e3713b1a74eea7844ae99fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-4-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-26T02:07:28.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:28 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:28.599+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:07:28.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:28 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:28.599+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.313401+0000 front 2024-04-26T02:07:00.313375+0000 (oldest deadline 2024-04-26T02:07:25.613566+0000) 2024-04-26T02:07:28.890 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:28 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:28.545+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.275620+0000 front 2024-04-26T02:06:59.275446+0000 (oldest deadline 2024-04-26T02:07:23.975481+0000) 2024-04-26T02:07:29.168 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0e49e64-036d-11ef-bc93-c7b262605968@osd.4.service' 2024-04-26T02:07:29.205 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:07:29.205 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-26T02:07:29.205 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-26T02:07:29.205 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-c0e49e64-036d-11ef-bc93-c7b262605968@osd.5 2024-04-26T02:07:29.231 INFO:journalctl@ceph.osd.5.smithi172.stdout:Apr 26 02:07:29 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-5[44593]: 2024-04-26T02:07:29.020+0000 7f609eb92700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.529851+0000 front 2024-04-26T02:06:50.529683+0000 (oldest deadline 2024-04-26T02:07:15.229503+0000) 2024-04-26T02:07:29.231 INFO:journalctl@ceph.osd.5.smithi172.stdout:Apr 26 02:07:29 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-5[44593]: 2024-04-26T02:07:29.020+0000 7f609eb92700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:55.229956+0000 front 2024-04-26T02:06:55.230052+0000 (oldest deadline 2024-04-26T02:07:23.430591+0000) 2024-04-26T02:07:29.231 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:28 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:28.892+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:07:29.232 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:28 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:28.892+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.934525+0000 front 2024-04-26T02:06:59.934498+0000 (oldest deadline 2024-04-26T02:07:22.834583+0000) 2024-04-26T02:07:29.233 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:28 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:28.963+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.647106+0000 front 2024-04-26T02:06:50.647828+0000 (oldest deadline 2024-04-26T02:07:12.947246+0000) 2024-04-26T02:07:29.233 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:28 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:28.963+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:57.650031+0000 front 2024-04-26T02:06:57.649560+0000 (oldest deadline 2024-04-26T02:07:23.549741+0000) 2024-04-26T02:07:29.504 INFO:journalctl@ceph.osd.5.smithi172.stdout:Apr 26 02:07:29 smithi172 systemd[1]: Stopping Ceph osd.5 for c0e49e64-036d-11ef-bc93-c7b262605968... 2024-04-26T02:07:29.622 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:29 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:29.234+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.476432+0000 front 2024-04-26T02:06:50.476644+0000 (oldest deadline 2024-04-26T02:07:18.078167+0000) 2024-04-26T02:07:29.622 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:29 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:29.234+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.682370+0000 front 2024-04-26T02:06:59.682519+0000 (oldest deadline 2024-04-26T02:07:24.982614+0000) 2024-04-26T02:07:29.787 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:29 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:29.505+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.275620+0000 front 2024-04-26T02:06:59.275446+0000 (oldest deadline 2024-04-26T02:07:23.975481+0000) 2024-04-26T02:07:29.787 INFO:journalctl@ceph.osd.5.smithi172.stdout:Apr 26 02:07:29 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-5[44593]: 2024-04-26T02:07:29.527+0000 7f60a259e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:07:29.788 INFO:journalctl@ceph.osd.5.smithi172.stdout:Apr 26 02:07:29 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-5[44593]: 2024-04-26T02:07:29.527+0000 7f60a259e700 -1 osd.5 112 *** Got signal Terminated *** 2024-04-26T02:07:29.788 INFO:journalctl@ceph.osd.5.smithi172.stdout:Apr 26 02:07:29 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-5[44593]: 2024-04-26T02:07:29.527+0000 7f60a259e700 -1 osd.5 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T02:07:29.846 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:29 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:29.473+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.419661+0000 front 2024-04-26T02:07:00.419639+0000 (oldest deadline 2024-04-26T02:07:25.719796+0000) 2024-04-26T02:07:29.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:29 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:29.611+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:07:29.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:29 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:29.611+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.313401+0000 front 2024-04-26T02:07:00.313375+0000 (oldest deadline 2024-04-26T02:07:25.613566+0000) 2024-04-26T02:07:30.195 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:29 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:29.933+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.647106+0000 front 2024-04-26T02:06:50.647828+0000 (oldest deadline 2024-04-26T02:07:12.947246+0000) 2024-04-26T02:07:30.196 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:29 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:29.933+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:57.650031+0000 front 2024-04-26T02:06:57.649560+0000 (oldest deadline 2024-04-26T02:07:23.549741+0000) 2024-04-26T02:07:30.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:29 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:29.853+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:07:30.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:29 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:29.853+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.934525+0000 front 2024-04-26T02:06:59.934498+0000 (oldest deadline 2024-04-26T02:07:22.834583+0000) 2024-04-26T02:07:30.287 INFO:journalctl@ceph.osd.5.smithi172.stdout:Apr 26 02:07:29 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-5[44593]: 2024-04-26T02:07:29.975+0000 7f609eb92700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.529851+0000 front 2024-04-26T02:06:50.529683+0000 (oldest deadline 2024-04-26T02:07:15.229503+0000) 2024-04-26T02:07:30.288 INFO:journalctl@ceph.osd.5.smithi172.stdout:Apr 26 02:07:29 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-5[44593]: 2024-04-26T02:07:29.975+0000 7f609eb92700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:55.229956+0000 front 2024-04-26T02:06:55.230052+0000 (oldest deadline 2024-04-26T02:07:23.430591+0000) 2024-04-26T02:07:30.622 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:30 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:30.196+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.476432+0000 front 2024-04-26T02:06:50.476644+0000 (oldest deadline 2024-04-26T02:07:18.078167+0000) 2024-04-26T02:07:30.622 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:30 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:30.196+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.682370+0000 front 2024-04-26T02:06:59.682519+0000 (oldest deadline 2024-04-26T02:07:24.982614+0000) 2024-04-26T02:07:30.787 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:30 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:30.528+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.275620+0000 front 2024-04-26T02:06:59.275446+0000 (oldest deadline 2024-04-26T02:07:23.975481+0000) 2024-04-26T02:07:30.787 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:30 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:30.528+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.477628+0000 front 2024-04-26T02:07:08.477663+0000 (oldest deadline 2024-04-26T02:07:30.177791+0000) 2024-04-26T02:07:30.847 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:30 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:30.518+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.419661+0000 front 2024-04-26T02:07:00.419639+0000 (oldest deadline 2024-04-26T02:07:25.719796+0000) 2024-04-26T02:07:30.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:30 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:30.586+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:07:30.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:30 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:30.586+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.313401+0000 front 2024-04-26T02:07:00.313375+0000 (oldest deadline 2024-04-26T02:07:25.613566+0000) 2024-04-26T02:07:30.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:30 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:30.586+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.615394+0000 front 2024-04-26T02:07:06.615253+0000 (oldest deadline 2024-04-26T02:07:30.015848+0000) 2024-04-26T02:07:31.237 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:30 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:30.906+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.647106+0000 front 2024-04-26T02:06:50.647828+0000 (oldest deadline 2024-04-26T02:07:12.947246+0000) 2024-04-26T02:07:31.237 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:30 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:30.906+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:57.650031+0000 front 2024-04-26T02:06:57.649560+0000 (oldest deadline 2024-04-26T02:07:23.549741+0000) 2024-04-26T02:07:31.237 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:30 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:30.906+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:07.651704+0000 front 2024-04-26T02:07:07.651639+0000 (oldest deadline 2024-04-26T02:07:30.551799+0000) 2024-04-26T02:07:31.287 INFO:journalctl@ceph.osd.5.smithi172.stdout:Apr 26 02:07:30 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-5[44593]: 2024-04-26T02:07:30.959+0000 7f609eb92700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.529851+0000 front 2024-04-26T02:06:50.529683+0000 (oldest deadline 2024-04-26T02:07:15.229503+0000) 2024-04-26T02:07:31.287 INFO:journalctl@ceph.osd.5.smithi172.stdout:Apr 26 02:07:30 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-5[44593]: 2024-04-26T02:07:30.959+0000 7f609eb92700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:55.229956+0000 front 2024-04-26T02:06:55.230052+0000 (oldest deadline 2024-04-26T02:07:23.430591+0000) 2024-04-26T02:07:31.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:30 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:30.809+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:07:31.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:30 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:30.809+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.934525+0000 front 2024-04-26T02:06:59.934498+0000 (oldest deadline 2024-04-26T02:07:22.834583+0000) 2024-04-26T02:07:31.622 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:31 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:31.238+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.476432+0000 front 2024-04-26T02:06:50.476644+0000 (oldest deadline 2024-04-26T02:07:18.078167+0000) 2024-04-26T02:07:31.622 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:31 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:31.238+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.682370+0000 front 2024-04-26T02:06:59.682519+0000 (oldest deadline 2024-04-26T02:07:24.982614+0000) 2024-04-26T02:07:31.827 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:31 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:31.548+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.275620+0000 front 2024-04-26T02:06:59.275446+0000 (oldest deadline 2024-04-26T02:07:23.975481+0000) 2024-04-26T02:07:31.827 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:31 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:31.548+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.477628+0000 front 2024-04-26T02:07:08.477663+0000 (oldest deadline 2024-04-26T02:07:30.177791+0000) 2024-04-26T02:07:31.846 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:31 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:31.544+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.419661+0000 front 2024-04-26T02:07:00.419639+0000 (oldest deadline 2024-04-26T02:07:25.719796+0000) 2024-04-26T02:07:31.846 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:31 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:31.544+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.721482+0000 front 2024-04-26T02:07:06.721504+0000 (oldest deadline 2024-04-26T02:07:31.421627+0000) 2024-04-26T02:07:31.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:31 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:31.565+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:07:31.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:31 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:31.565+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.313401+0000 front 2024-04-26T02:07:00.313375+0000 (oldest deadline 2024-04-26T02:07:25.613566+0000) 2024-04-26T02:07:31.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:31 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:31.565+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.615394+0000 front 2024-04-26T02:07:06.615253+0000 (oldest deadline 2024-04-26T02:07:30.015848+0000) 2024-04-26T02:07:32.202 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:31 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:31.921+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.647106+0000 front 2024-04-26T02:06:50.647828+0000 (oldest deadline 2024-04-26T02:07:12.947246+0000) 2024-04-26T02:07:32.202 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:31 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:31.921+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:57.650031+0000 front 2024-04-26T02:06:57.649560+0000 (oldest deadline 2024-04-26T02:07:23.549741+0000) 2024-04-26T02:07:32.202 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:31 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:31.921+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:07.651704+0000 front 2024-04-26T02:07:07.651639+0000 (oldest deadline 2024-04-26T02:07:30.551799+0000) 2024-04-26T02:07:32.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:31 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:31.829+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:07:32.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:31 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:31.829+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.934525+0000 front 2024-04-26T02:06:59.934498+0000 (oldest deadline 2024-04-26T02:07:22.834583+0000) 2024-04-26T02:07:32.287 INFO:journalctl@ceph.osd.5.smithi172.stdout:Apr 26 02:07:31 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-5[44593]: 2024-04-26T02:07:31.991+0000 7f609eb92700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.529851+0000 front 2024-04-26T02:06:50.529683+0000 (oldest deadline 2024-04-26T02:07:15.229503+0000) 2024-04-26T02:07:32.287 INFO:journalctl@ceph.osd.5.smithi172.stdout:Apr 26 02:07:31 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-5[44593]: 2024-04-26T02:07:31.991+0000 7f609eb92700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:55.229956+0000 front 2024-04-26T02:06:55.230052+0000 (oldest deadline 2024-04-26T02:07:23.430591+0000) 2024-04-26T02:07:32.622 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:32 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:32.203+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.476432+0000 front 2024-04-26T02:06:50.476644+0000 (oldest deadline 2024-04-26T02:07:18.078167+0000) 2024-04-26T02:07:32.623 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:32 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:32.203+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.682370+0000 front 2024-04-26T02:06:59.682519+0000 (oldest deadline 2024-04-26T02:07:24.982614+0000) 2024-04-26T02:07:32.834 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:32 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:32.561+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.275620+0000 front 2024-04-26T02:06:59.275446+0000 (oldest deadline 2024-04-26T02:07:23.975481+0000) 2024-04-26T02:07:32.834 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:32 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:32.561+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.477628+0000 front 2024-04-26T02:07:08.477663+0000 (oldest deadline 2024-04-26T02:07:30.177791+0000) 2024-04-26T02:07:32.846 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:32 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:32.538+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.419661+0000 front 2024-04-26T02:07:00.419639+0000 (oldest deadline 2024-04-26T02:07:25.719796+0000) 2024-04-26T02:07:32.846 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:32 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:32.538+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.721482+0000 front 2024-04-26T02:07:06.721504+0000 (oldest deadline 2024-04-26T02:07:31.421627+0000) 2024-04-26T02:07:32.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:32 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:32.531+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:07:32.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:32 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:32.531+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.313401+0000 front 2024-04-26T02:07:00.313375+0000 (oldest deadline 2024-04-26T02:07:25.613566+0000) 2024-04-26T02:07:32.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:32 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:32.531+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.615394+0000 front 2024-04-26T02:07:06.615253+0000 (oldest deadline 2024-04-26T02:07:30.015848+0000) 2024-04-26T02:07:33.287 INFO:journalctl@ceph.osd.5.smithi172.stdout:Apr 26 02:07:33 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-5[44593]: 2024-04-26T02:07:33.020+0000 7f609eb92700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.529851+0000 front 2024-04-26T02:06:50.529683+0000 (oldest deadline 2024-04-26T02:07:15.229503+0000) 2024-04-26T02:07:33.287 INFO:journalctl@ceph.osd.5.smithi172.stdout:Apr 26 02:07:33 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-5[44593]: 2024-04-26T02:07:33.020+0000 7f609eb92700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:55.229956+0000 front 2024-04-26T02:06:55.230052+0000 (oldest deadline 2024-04-26T02:07:23.430591+0000) 2024-04-26T02:07:33.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:32 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:32.836+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:07:33.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:32 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:32.836+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.934525+0000 front 2024-04-26T02:06:59.934498+0000 (oldest deadline 2024-04-26T02:07:22.834583+0000) 2024-04-26T02:07:33.372 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:33 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:33.156+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.476432+0000 front 2024-04-26T02:06:50.476644+0000 (oldest deadline 2024-04-26T02:07:18.078167+0000) 2024-04-26T02:07:33.372 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:33 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:33.156+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.682370+0000 front 2024-04-26T02:06:59.682519+0000 (oldest deadline 2024-04-26T02:07:24.982614+0000) 2024-04-26T02:07:33.373 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:32 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:32.908+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.647106+0000 front 2024-04-26T02:06:50.647828+0000 (oldest deadline 2024-04-26T02:07:12.947246+0000) 2024-04-26T02:07:33.373 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:32 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:32.908+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:57.650031+0000 front 2024-04-26T02:06:57.649560+0000 (oldest deadline 2024-04-26T02:07:23.549741+0000) 2024-04-26T02:07:33.373 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:32 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:32.908+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:07.651704+0000 front 2024-04-26T02:07:07.651639+0000 (oldest deadline 2024-04-26T02:07:30.551799+0000) 2024-04-26T02:07:33.787 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:33 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:33.522+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.275620+0000 front 2024-04-26T02:06:59.275446+0000 (oldest deadline 2024-04-26T02:07:23.975481+0000) 2024-04-26T02:07:33.787 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:33 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:33.522+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.477628+0000 front 2024-04-26T02:07:08.477663+0000 (oldest deadline 2024-04-26T02:07:30.177791+0000) 2024-04-26T02:07:33.846 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:33 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:33.509+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.419661+0000 front 2024-04-26T02:07:00.419639+0000 (oldest deadline 2024-04-26T02:07:25.719796+0000) 2024-04-26T02:07:33.847 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:33 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:33.509+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.721482+0000 front 2024-04-26T02:07:06.721504+0000 (oldest deadline 2024-04-26T02:07:31.421627+0000) 2024-04-26T02:07:33.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:33 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:33.576+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:07:33.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:33 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:33.576+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.313401+0000 front 2024-04-26T02:07:00.313375+0000 (oldest deadline 2024-04-26T02:07:25.613566+0000) 2024-04-26T02:07:33.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:33 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:33.576+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.615394+0000 front 2024-04-26T02:07:06.615253+0000 (oldest deadline 2024-04-26T02:07:30.015848+0000) 2024-04-26T02:07:34.287 INFO:journalctl@ceph.osd.5.smithi172.stdout:Apr 26 02:07:34 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-5[44593]: 2024-04-26T02:07:34.023+0000 7f609eb92700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.529851+0000 front 2024-04-26T02:06:50.529683+0000 (oldest deadline 2024-04-26T02:07:15.229503+0000) 2024-04-26T02:07:34.288 INFO:journalctl@ceph.osd.5.smithi172.stdout:Apr 26 02:07:34 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-5[44593]: 2024-04-26T02:07:34.023+0000 7f609eb92700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:55.229956+0000 front 2024-04-26T02:06:55.230052+0000 (oldest deadline 2024-04-26T02:07:23.430591+0000) 2024-04-26T02:07:34.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:33 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:33.849+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:07:34.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:33 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:33.849+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.934525+0000 front 2024-04-26T02:06:59.934498+0000 (oldest deadline 2024-04-26T02:07:22.834583+0000) 2024-04-26T02:07:34.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:33 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:33.849+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.735339+0000 front 2024-04-26T02:07:08.735363+0000 (oldest deadline 2024-04-26T02:07:33.435537+0000) 2024-04-26T02:07:34.372 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:33 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:33.918+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.647106+0000 front 2024-04-26T02:06:50.647828+0000 (oldest deadline 2024-04-26T02:07:12.947246+0000) 2024-04-26T02:07:34.373 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:33 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:33.918+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:57.650031+0000 front 2024-04-26T02:06:57.649560+0000 (oldest deadline 2024-04-26T02:07:23.549741+0000) 2024-04-26T02:07:34.373 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:33 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:33.918+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:07.651704+0000 front 2024-04-26T02:07:07.651639+0000 (oldest deadline 2024-04-26T02:07:30.551799+0000) 2024-04-26T02:07:34.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:34 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:34.114+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.476432+0000 front 2024-04-26T02:06:50.476644+0000 (oldest deadline 2024-04-26T02:07:18.078167+0000) 2024-04-26T02:07:34.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:34 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:34.114+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.682370+0000 front 2024-04-26T02:06:59.682519+0000 (oldest deadline 2024-04-26T02:07:24.982614+0000) 2024-04-26T02:07:34.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:34 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:34.114+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:04.984189+0000 front 2024-04-26T02:07:04.983860+0000 (oldest deadline 2024-04-26T02:07:33.784534+0000) 2024-04-26T02:07:34.787 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:34 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:34.498+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.275620+0000 front 2024-04-26T02:06:59.275446+0000 (oldest deadline 2024-04-26T02:07:23.975481+0000) 2024-04-26T02:07:34.788 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:34 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:34.498+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.477628+0000 front 2024-04-26T02:07:08.477663+0000 (oldest deadline 2024-04-26T02:07:30.177791+0000) 2024-04-26T02:07:34.846 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:34 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:34.592+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:07:34.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:34 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:34.592+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.313401+0000 front 2024-04-26T02:07:00.313375+0000 (oldest deadline 2024-04-26T02:07:25.613566+0000) 2024-04-26T02:07:34.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:34 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:34.592+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.615394+0000 front 2024-04-26T02:07:06.615253+0000 (oldest deadline 2024-04-26T02:07:30.015848+0000) 2024-04-26T02:07:34.847 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:34 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:34.490+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.419661+0000 front 2024-04-26T02:07:00.419639+0000 (oldest deadline 2024-04-26T02:07:25.719796+0000) 2024-04-26T02:07:34.847 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:34 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:34.490+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.721482+0000 front 2024-04-26T02:07:06.721504+0000 (oldest deadline 2024-04-26T02:07:31.421627+0000) 2024-04-26T02:07:35.287 INFO:journalctl@ceph.osd.5.smithi172.stdout:Apr 26 02:07:35 smithi172 podman[71188]: 2024-04-26 02:07:35.09517945 +0000 UTC m=+5.698803415 container died 0dda7ef3f4f991465e0d6fc8cd8f4f85b5d08d9eee5942dc9ad59fd718bf694b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-26T02:07:35.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:34 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:34.858+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:07:35.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:34 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:34.858+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.934525+0000 front 2024-04-26T02:06:59.934498+0000 (oldest deadline 2024-04-26T02:07:22.834583+0000) 2024-04-26T02:07:35.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:34 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:34.858+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.735339+0000 front 2024-04-26T02:07:08.735363+0000 (oldest deadline 2024-04-26T02:07:33.435537+0000) 2024-04-26T02:07:35.372 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:34 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:34.930+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.647106+0000 front 2024-04-26T02:06:50.647828+0000 (oldest deadline 2024-04-26T02:07:12.947246+0000) 2024-04-26T02:07:35.372 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:34 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:34.930+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:57.650031+0000 front 2024-04-26T02:06:57.649560+0000 (oldest deadline 2024-04-26T02:07:23.549741+0000) 2024-04-26T02:07:35.373 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:34 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:34.930+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:07.651704+0000 front 2024-04-26T02:07:07.651639+0000 (oldest deadline 2024-04-26T02:07:30.551799+0000) 2024-04-26T02:07:35.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:35 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:35.151+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.476432+0000 front 2024-04-26T02:06:50.476644+0000 (oldest deadline 2024-04-26T02:07:18.078167+0000) 2024-04-26T02:07:35.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:35 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:35.151+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.682370+0000 front 2024-04-26T02:06:59.682519+0000 (oldest deadline 2024-04-26T02:07:24.982614+0000) 2024-04-26T02:07:35.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:35 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:35.151+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:04.984189+0000 front 2024-04-26T02:07:04.983860+0000 (oldest deadline 2024-04-26T02:07:33.784534+0000) 2024-04-26T02:07:35.726 INFO:journalctl@ceph.osd.5.smithi172.stdout:Apr 26 02:07:35 smithi172 podman[71188]: 2024-04-26 02:07:35.451400795 +0000 UTC m=+6.055024764 container cleanup 0dda7ef3f4f991465e0d6fc8cd8f4f85b5d08d9eee5942dc9ad59fd718bf694b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416) 2024-04-26T02:07:35.726 INFO:journalctl@ceph.osd.5.smithi172.stdout:Apr 26 02:07:35 smithi172 bash[71188]: ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-5 2024-04-26T02:07:35.726 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:35 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:35.472+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.275620+0000 front 2024-04-26T02:06:59.275446+0000 (oldest deadline 2024-04-26T02:07:23.975481+0000) 2024-04-26T02:07:35.726 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:35 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:35.472+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.477628+0000 front 2024-04-26T02:07:08.477663+0000 (oldest deadline 2024-04-26T02:07:30.177791+0000) 2024-04-26T02:07:35.847 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:35 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:35.464+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.419661+0000 front 2024-04-26T02:07:00.419639+0000 (oldest deadline 2024-04-26T02:07:25.719796+0000) 2024-04-26T02:07:35.847 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:35 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:35.464+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.721482+0000 front 2024-04-26T02:07:06.721504+0000 (oldest deadline 2024-04-26T02:07:31.421627+0000) 2024-04-26T02:07:35.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:35 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:35.602+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:07:35.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:35 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:35.602+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.313401+0000 front 2024-04-26T02:07:00.313375+0000 (oldest deadline 2024-04-26T02:07:25.613566+0000) 2024-04-26T02:07:35.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:35 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:35.602+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.615394+0000 front 2024-04-26T02:07:06.615253+0000 (oldest deadline 2024-04-26T02:07:30.015848+0000) 2024-04-26T02:07:36.037 INFO:journalctl@ceph.osd.5.smithi172.stdout:Apr 26 02:07:35 smithi172 podman[71205]: 2024-04-26 02:07:35.727384765 +0000 UTC m=+0.634061340 container remove 0dda7ef3f4f991465e0d6fc8cd8f4f85b5d08d9eee5942dc9ad59fd718bf694b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-5, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0) 2024-04-26T02:07:36.038 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:35 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:35.813+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:07:36.038 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:35 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:35.813+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.934525+0000 front 2024-04-26T02:06:59.934498+0000 (oldest deadline 2024-04-26T02:07:22.834583+0000) 2024-04-26T02:07:36.038 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:35 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:35.813+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.735339+0000 front 2024-04-26T02:07:08.735363+0000 (oldest deadline 2024-04-26T02:07:33.435537+0000) 2024-04-26T02:07:36.363 INFO:journalctl@ceph.osd.5.smithi172.stdout:Apr 26 02:07:36 smithi172 podman[71293]: 2024-04-26 02:07:36.062331527 +0000 UTC m=+0.112715782 container create 46fd38c63e9c4ae920f7a85d9cb7a858042b2c277743b632e1d263a385ef3ef1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-26T02:07:36.363 INFO:journalctl@ceph.osd.5.smithi172.stdout:Apr 26 02:07:36 smithi172 podman[71293]: 2024-04-26 02:07:35.967907581 +0000 UTC m=+0.018291834 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T02:07:36.363 INFO:journalctl@ceph.osd.5.smithi172.stdout:Apr 26 02:07:36 smithi172 podman[71293]: 2024-04-26 02:07:36.272803398 +0000 UTC m=+0.323187651 container init 46fd38c63e9c4ae920f7a85d9cb7a858042b2c277743b632e1d263a385ef3ef1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-26T02:07:36.363 INFO:journalctl@ceph.osd.5.smithi172.stdout:Apr 26 02:07:36 smithi172 podman[71293]: 2024-04-26 02:07:36.275819499 +0000 UTC m=+0.326203761 container start 46fd38c63e9c4ae920f7a85d9cb7a858042b2c277743b632e1d263a385ef3ef1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-26T02:07:36.363 INFO:journalctl@ceph.osd.5.smithi172.stdout:Apr 26 02:07:36 smithi172 podman[71293]: 2024-04-26 02:07:36.364790565 +0000 UTC m=+0.415174827 container attach 46fd38c63e9c4ae920f7a85d9cb7a858042b2c277743b632e1d263a385ef3ef1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True) 2024-04-26T02:07:36.372 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:35 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:35.954+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.647106+0000 front 2024-04-26T02:06:50.647828+0000 (oldest deadline 2024-04-26T02:07:12.947246+0000) 2024-04-26T02:07:36.373 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:35 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:35.954+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:57.650031+0000 front 2024-04-26T02:06:57.649560+0000 (oldest deadline 2024-04-26T02:07:23.549741+0000) 2024-04-26T02:07:36.373 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:35 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:35.954+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:07.651704+0000 front 2024-04-26T02:07:07.651639+0000 (oldest deadline 2024-04-26T02:07:30.551799+0000) 2024-04-26T02:07:36.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:36 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:36.184+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.476432+0000 front 2024-04-26T02:06:50.476644+0000 (oldest deadline 2024-04-26T02:07:18.078167+0000) 2024-04-26T02:07:36.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:36 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:36.184+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.682370+0000 front 2024-04-26T02:06:59.682519+0000 (oldest deadline 2024-04-26T02:07:24.982614+0000) 2024-04-26T02:07:36.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:36 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:36.184+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:04.984189+0000 front 2024-04-26T02:07:04.983860+0000 (oldest deadline 2024-04-26T02:07:33.784534+0000) 2024-04-26T02:07:36.787 INFO:journalctl@ceph.osd.5.smithi172.stdout:Apr 26 02:07:36 smithi172 podman[71293]: 2024-04-26 02:07:36.575549626 +0000 UTC m=+0.625933886 container died 46fd38c63e9c4ae920f7a85d9cb7a858042b2c277743b632e1d263a385ef3ef1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True) 2024-04-26T02:07:36.787 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:36 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:36.483+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.275620+0000 front 2024-04-26T02:06:59.275446+0000 (oldest deadline 2024-04-26T02:07:23.975481+0000) 2024-04-26T02:07:36.787 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:36 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:36.483+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.477628+0000 front 2024-04-26T02:07:08.477663+0000 (oldest deadline 2024-04-26T02:07:30.177791+0000) 2024-04-26T02:07:36.846 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:36 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:36.432+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.419661+0000 front 2024-04-26T02:07:00.419639+0000 (oldest deadline 2024-04-26T02:07:25.719796+0000) 2024-04-26T02:07:36.847 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:36 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:36.432+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.721482+0000 front 2024-04-26T02:07:06.721504+0000 (oldest deadline 2024-04-26T02:07:31.421627+0000) 2024-04-26T02:07:36.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:36 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:36.616+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:07:36.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:36 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:36.616+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.313401+0000 front 2024-04-26T02:07:00.313375+0000 (oldest deadline 2024-04-26T02:07:25.613566+0000) 2024-04-26T02:07:36.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:36 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:36.617+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.615394+0000 front 2024-04-26T02:07:06.615253+0000 (oldest deadline 2024-04-26T02:07:30.015848+0000) 2024-04-26T02:07:37.192 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:36 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:36.798+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:07:37.192 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:36 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:36.798+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.934525+0000 front 2024-04-26T02:06:59.934498+0000 (oldest deadline 2024-04-26T02:07:22.834583+0000) 2024-04-26T02:07:37.192 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:36 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:36.798+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.735339+0000 front 2024-04-26T02:07:08.735363+0000 (oldest deadline 2024-04-26T02:07:33.435537+0000) 2024-04-26T02:07:37.211 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0e49e64-036d-11ef-bc93-c7b262605968@osd.5.service' 2024-04-26T02:07:37.290 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:07:37.290 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-26T02:07:37.291 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-26T02:07:37.291 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-c0e49e64-036d-11ef-bc93-c7b262605968@osd.6 2024-04-26T02:07:37.368 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:36 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:36.985+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.647106+0000 front 2024-04-26T02:06:50.647828+0000 (oldest deadline 2024-04-26T02:07:12.947246+0000) 2024-04-26T02:07:37.368 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:36 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:36.985+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:57.650031+0000 front 2024-04-26T02:06:57.649560+0000 (oldest deadline 2024-04-26T02:07:23.549741+0000) 2024-04-26T02:07:37.368 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:36 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:36.985+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:07.651704+0000 front 2024-04-26T02:07:07.651639+0000 (oldest deadline 2024-04-26T02:07:30.551799+0000) 2024-04-26T02:07:37.368 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:37 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:37.169+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.476432+0000 front 2024-04-26T02:06:50.476644+0000 (oldest deadline 2024-04-26T02:07:18.078167+0000) 2024-04-26T02:07:37.368 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:37 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:37.169+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.682370+0000 front 2024-04-26T02:06:59.682519+0000 (oldest deadline 2024-04-26T02:07:24.982614+0000) 2024-04-26T02:07:37.368 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:37 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:37.169+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:04.984189+0000 front 2024-04-26T02:07:04.983860+0000 (oldest deadline 2024-04-26T02:07:33.784534+0000) 2024-04-26T02:07:37.622 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:37 smithi151 systemd[1]: Stopping Ceph osd.6 for c0e49e64-036d-11ef-bc93-c7b262605968... 2024-04-26T02:07:37.787 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:37 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:37.493+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.275620+0000 front 2024-04-26T02:06:59.275446+0000 (oldest deadline 2024-04-26T02:07:23.975481+0000) 2024-04-26T02:07:37.787 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:37 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:37.493+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.477628+0000 front 2024-04-26T02:07:08.477663+0000 (oldest deadline 2024-04-26T02:07:30.177791+0000) 2024-04-26T02:07:37.846 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:37 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:37.450+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.419661+0000 front 2024-04-26T02:07:00.419639+0000 (oldest deadline 2024-04-26T02:07:25.719796+0000) 2024-04-26T02:07:37.846 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:37 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:37.450+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.721482+0000 front 2024-04-26T02:07:06.721504+0000 (oldest deadline 2024-04-26T02:07:31.421627+0000) 2024-04-26T02:07:37.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:37 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:37.626+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:07:37.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:37 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:37.626+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.313401+0000 front 2024-04-26T02:07:00.313375+0000 (oldest deadline 2024-04-26T02:07:25.613566+0000) 2024-04-26T02:07:37.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:37 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:37.626+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.615394+0000 front 2024-04-26T02:07:06.615253+0000 (oldest deadline 2024-04-26T02:07:30.015848+0000) 2024-04-26T02:07:38.123 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:37 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:37.865+0000 7f5f5e800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:07:38.123 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:37 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:37.865+0000 7f5f5e800700 -1 osd.6 112 *** Got signal Terminated *** 2024-04-26T02:07:38.123 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:37 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:37.865+0000 7f5f5e800700 -1 osd.6 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T02:07:38.123 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:37 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:37.963+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.647106+0000 front 2024-04-26T02:06:50.647828+0000 (oldest deadline 2024-04-26T02:07:12.947246+0000) 2024-04-26T02:07:38.123 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:37 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:37.963+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:57.650031+0000 front 2024-04-26T02:06:57.649560+0000 (oldest deadline 2024-04-26T02:07:23.549741+0000) 2024-04-26T02:07:38.123 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:37 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:37.963+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:07.651704+0000 front 2024-04-26T02:07:07.651639+0000 (oldest deadline 2024-04-26T02:07:30.551799+0000) 2024-04-26T02:07:38.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:37 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:37.794+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:07:38.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:37 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:37.794+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.934525+0000 front 2024-04-26T02:06:59.934498+0000 (oldest deadline 2024-04-26T02:07:22.834583+0000) 2024-04-26T02:07:38.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:37 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:37.794+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.735339+0000 front 2024-04-26T02:07:08.735363+0000 (oldest deadline 2024-04-26T02:07:33.435537+0000) 2024-04-26T02:07:38.622 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:38 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:38.218+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.476432+0000 front 2024-04-26T02:06:50.476644+0000 (oldest deadline 2024-04-26T02:07:18.078167+0000) 2024-04-26T02:07:38.623 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:38 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:38.218+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.682370+0000 front 2024-04-26T02:06:59.682519+0000 (oldest deadline 2024-04-26T02:07:24.982614+0000) 2024-04-26T02:07:38.623 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:38 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:38.218+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:04.984189+0000 front 2024-04-26T02:07:04.983860+0000 (oldest deadline 2024-04-26T02:07:33.784534+0000) 2024-04-26T02:07:38.787 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:38 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:38.445+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.275620+0000 front 2024-04-26T02:06:59.275446+0000 (oldest deadline 2024-04-26T02:07:23.975481+0000) 2024-04-26T02:07:38.787 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:38 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:38.445+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.477628+0000 front 2024-04-26T02:07:08.477663+0000 (oldest deadline 2024-04-26T02:07:30.177791+0000) 2024-04-26T02:07:38.846 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:38 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:38.408+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.419661+0000 front 2024-04-26T02:07:00.419639+0000 (oldest deadline 2024-04-26T02:07:25.719796+0000) 2024-04-26T02:07:38.847 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:38 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:38.408+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.721482+0000 front 2024-04-26T02:07:06.721504+0000 (oldest deadline 2024-04-26T02:07:31.421627+0000) 2024-04-26T02:07:38.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:38 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:38.593+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:07:38.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:38 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:38.593+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.313401+0000 front 2024-04-26T02:07:00.313375+0000 (oldest deadline 2024-04-26T02:07:25.613566+0000) 2024-04-26T02:07:38.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:38 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:38.593+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.615394+0000 front 2024-04-26T02:07:06.615253+0000 (oldest deadline 2024-04-26T02:07:30.015848+0000) 2024-04-26T02:07:39.266 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:38 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:38.934+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.647106+0000 front 2024-04-26T02:06:50.647828+0000 (oldest deadline 2024-04-26T02:07:12.947246+0000) 2024-04-26T02:07:39.266 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:38 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:38.934+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:57.650031+0000 front 2024-04-26T02:06:57.649560+0000 (oldest deadline 2024-04-26T02:07:23.549741+0000) 2024-04-26T02:07:39.266 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:38 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:38.934+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:07.651704+0000 front 2024-04-26T02:07:07.651639+0000 (oldest deadline 2024-04-26T02:07:30.551799+0000) 2024-04-26T02:07:39.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:38 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:38.796+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:07:39.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:38 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:38.796+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.934525+0000 front 2024-04-26T02:06:59.934498+0000 (oldest deadline 2024-04-26T02:07:22.834583+0000) 2024-04-26T02:07:39.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:38 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:38.796+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.735339+0000 front 2024-04-26T02:07:08.735363+0000 (oldest deadline 2024-04-26T02:07:33.435537+0000) 2024-04-26T02:07:39.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:38 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:38.796+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:16.837132+0000 front 2024-04-26T02:07:16.837183+0000 (oldest deadline 2024-04-26T02:07:38.537028+0000) 2024-04-26T02:07:39.564 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:39 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:39.437+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.275620+0000 front 2024-04-26T02:06:59.275446+0000 (oldest deadline 2024-04-26T02:07:23.975481+0000) 2024-04-26T02:07:39.564 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:39 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:39.437+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.477628+0000 front 2024-04-26T02:07:08.477663+0000 (oldest deadline 2024-04-26T02:07:30.177791+0000) 2024-04-26T02:07:39.622 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:39 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:39.265+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.476432+0000 front 2024-04-26T02:06:50.476644+0000 (oldest deadline 2024-04-26T02:07:18.078167+0000) 2024-04-26T02:07:39.623 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:39 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:39.265+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.682370+0000 front 2024-04-26T02:06:59.682519+0000 (oldest deadline 2024-04-26T02:07:24.982614+0000) 2024-04-26T02:07:39.623 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:39 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:39.265+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:04.984189+0000 front 2024-04-26T02:07:04.983860+0000 (oldest deadline 2024-04-26T02:07:33.784534+0000) 2024-04-26T02:07:39.846 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:39 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:39.610+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:07:39.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:39 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:39.610+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.313401+0000 front 2024-04-26T02:07:00.313375+0000 (oldest deadline 2024-04-26T02:07:25.613566+0000) 2024-04-26T02:07:39.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:39 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:39.610+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.615394+0000 front 2024-04-26T02:07:06.615253+0000 (oldest deadline 2024-04-26T02:07:30.015848+0000) 2024-04-26T02:07:39.847 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:39 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:39.447+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.419661+0000 front 2024-04-26T02:07:00.419639+0000 (oldest deadline 2024-04-26T02:07:25.719796+0000) 2024-04-26T02:07:39.847 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:39 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:39.447+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.721482+0000 front 2024-04-26T02:07:06.721504+0000 (oldest deadline 2024-04-26T02:07:31.421627+0000) 2024-04-26T02:07:39.847 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:39 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:39.447+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:13.122746+0000 front 2024-04-26T02:07:13.122675+0000 (oldest deadline 2024-04-26T02:07:38.923057+0000) 2024-04-26T02:07:40.037 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:39 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:39.769+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:07:40.037 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:39 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:39.769+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.934525+0000 front 2024-04-26T02:06:59.934498+0000 (oldest deadline 2024-04-26T02:07:22.834583+0000) 2024-04-26T02:07:40.037 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:39 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:39.769+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.735339+0000 front 2024-04-26T02:07:08.735363+0000 (oldest deadline 2024-04-26T02:07:33.435537+0000) 2024-04-26T02:07:40.037 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:39 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:39.769+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:16.837132+0000 front 2024-04-26T02:07:16.837183+0000 (oldest deadline 2024-04-26T02:07:38.537028+0000) 2024-04-26T02:07:40.292 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:39 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:39.899+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.647106+0000 front 2024-04-26T02:06:50.647828+0000 (oldest deadline 2024-04-26T02:07:12.947246+0000) 2024-04-26T02:07:40.292 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:39 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:39.899+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:57.650031+0000 front 2024-04-26T02:06:57.649560+0000 (oldest deadline 2024-04-26T02:07:23.549741+0000) 2024-04-26T02:07:40.292 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:39 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:39.899+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:07.651704+0000 front 2024-04-26T02:07:07.651639+0000 (oldest deadline 2024-04-26T02:07:30.551799+0000) 2024-04-26T02:07:40.622 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:40 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:40.292+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.476432+0000 front 2024-04-26T02:06:50.476644+0000 (oldest deadline 2024-04-26T02:07:18.078167+0000) 2024-04-26T02:07:40.622 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:40 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:40.292+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.682370+0000 front 2024-04-26T02:06:59.682519+0000 (oldest deadline 2024-04-26T02:07:24.982614+0000) 2024-04-26T02:07:40.623 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:40 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:40.292+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:04.984189+0000 front 2024-04-26T02:07:04.983860+0000 (oldest deadline 2024-04-26T02:07:33.784534+0000) 2024-04-26T02:07:40.770 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:40 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:40.456+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.275620+0000 front 2024-04-26T02:06:59.275446+0000 (oldest deadline 2024-04-26T02:07:23.975481+0000) 2024-04-26T02:07:40.771 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:40 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:40.456+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.477628+0000 front 2024-04-26T02:07:08.477663+0000 (oldest deadline 2024-04-26T02:07:30.177791+0000) 2024-04-26T02:07:40.771 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:40 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:40.456+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:17.179691+0000 front 2024-04-26T02:07:17.179731+0000 (oldest deadline 2024-04-26T02:07:40.079515+0000) 2024-04-26T02:07:40.847 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:40 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:40.470+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.419661+0000 front 2024-04-26T02:07:00.419639+0000 (oldest deadline 2024-04-26T02:07:25.719796+0000) 2024-04-26T02:07:40.847 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:40 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:40.470+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.721482+0000 front 2024-04-26T02:07:06.721504+0000 (oldest deadline 2024-04-26T02:07:31.421627+0000) 2024-04-26T02:07:40.847 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:40 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:40.470+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:13.122746+0000 front 2024-04-26T02:07:13.122675+0000 (oldest deadline 2024-04-26T02:07:38.923057+0000) 2024-04-26T02:07:40.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:40 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:40.581+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:07:40.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:40 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:40.581+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.313401+0000 front 2024-04-26T02:07:00.313375+0000 (oldest deadline 2024-04-26T02:07:25.613566+0000) 2024-04-26T02:07:40.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:40 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:40.581+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.615394+0000 front 2024-04-26T02:07:06.615253+0000 (oldest deadline 2024-04-26T02:07:30.015848+0000) 2024-04-26T02:07:40.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:40 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:40.581+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:15.118484+0000 front 2024-04-26T02:07:15.118531+0000 (oldest deadline 2024-04-26T02:07:40.318854+0000) 2024-04-26T02:07:41.037 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:40 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:40.771+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:07:41.037 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:40 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:40.771+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.934525+0000 front 2024-04-26T02:06:59.934498+0000 (oldest deadline 2024-04-26T02:07:22.834583+0000) 2024-04-26T02:07:41.037 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:40 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:40.771+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.735339+0000 front 2024-04-26T02:07:08.735363+0000 (oldest deadline 2024-04-26T02:07:33.435537+0000) 2024-04-26T02:07:41.037 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:40 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:40.771+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:16.837132+0000 front 2024-04-26T02:07:16.837183+0000 (oldest deadline 2024-04-26T02:07:38.537028+0000) 2024-04-26T02:07:41.122 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:40 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:40.864+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.647106+0000 front 2024-04-26T02:06:50.647828+0000 (oldest deadline 2024-04-26T02:07:12.947246+0000) 2024-04-26T02:07:41.122 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:40 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:40.864+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:57.650031+0000 front 2024-04-26T02:06:57.649560+0000 (oldest deadline 2024-04-26T02:07:23.549741+0000) 2024-04-26T02:07:41.123 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:40 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:40.864+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:07.651704+0000 front 2024-04-26T02:07:07.651639+0000 (oldest deadline 2024-04-26T02:07:30.551799+0000) 2024-04-26T02:07:41.622 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:41 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:41.327+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.476432+0000 front 2024-04-26T02:06:50.476644+0000 (oldest deadline 2024-04-26T02:07:18.078167+0000) 2024-04-26T02:07:41.623 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:41 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:41.327+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.682370+0000 front 2024-04-26T02:06:59.682519+0000 (oldest deadline 2024-04-26T02:07:24.982614+0000) 2024-04-26T02:07:41.623 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:41 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:41.327+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:04.984189+0000 front 2024-04-26T02:07:04.983860+0000 (oldest deadline 2024-04-26T02:07:33.784534+0000) 2024-04-26T02:07:41.765 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:41 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:41.490+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.275620+0000 front 2024-04-26T02:06:59.275446+0000 (oldest deadline 2024-04-26T02:07:23.975481+0000) 2024-04-26T02:07:41.765 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:41 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:41.490+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.477628+0000 front 2024-04-26T02:07:08.477663+0000 (oldest deadline 2024-04-26T02:07:30.177791+0000) 2024-04-26T02:07:41.766 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:41 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:41.490+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:17.179691+0000 front 2024-04-26T02:07:17.179731+0000 (oldest deadline 2024-04-26T02:07:40.079515+0000) 2024-04-26T02:07:41.847 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:41 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:41.428+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.419661+0000 front 2024-04-26T02:07:00.419639+0000 (oldest deadline 2024-04-26T02:07:25.719796+0000) 2024-04-26T02:07:41.847 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:41 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:41.428+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.721482+0000 front 2024-04-26T02:07:06.721504+0000 (oldest deadline 2024-04-26T02:07:31.421627+0000) 2024-04-26T02:07:41.847 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:41 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:41.428+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:13.122746+0000 front 2024-04-26T02:07:13.122675+0000 (oldest deadline 2024-04-26T02:07:38.923057+0000) 2024-04-26T02:07:41.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:41 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:41.534+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:07:41.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:41 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:41.534+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.313401+0000 front 2024-04-26T02:07:00.313375+0000 (oldest deadline 2024-04-26T02:07:25.613566+0000) 2024-04-26T02:07:41.848 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:41 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:41.534+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.615394+0000 front 2024-04-26T02:07:06.615253+0000 (oldest deadline 2024-04-26T02:07:30.015848+0000) 2024-04-26T02:07:41.848 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:41 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:41.534+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:15.118484+0000 front 2024-04-26T02:07:15.118531+0000 (oldest deadline 2024-04-26T02:07:40.318854+0000) 2024-04-26T02:07:42.037 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:41 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:41.766+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:07:42.038 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:41 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:41.766+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.934525+0000 front 2024-04-26T02:06:59.934498+0000 (oldest deadline 2024-04-26T02:07:22.834583+0000) 2024-04-26T02:07:42.038 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:41 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:41.766+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.735339+0000 front 2024-04-26T02:07:08.735363+0000 (oldest deadline 2024-04-26T02:07:33.435537+0000) 2024-04-26T02:07:42.038 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:41 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:41.766+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:16.837132+0000 front 2024-04-26T02:07:16.837183+0000 (oldest deadline 2024-04-26T02:07:38.537028+0000) 2024-04-26T02:07:42.122 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:41 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:41.869+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.647106+0000 front 2024-04-26T02:06:50.647828+0000 (oldest deadline 2024-04-26T02:07:12.947246+0000) 2024-04-26T02:07:42.123 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:41 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:41.869+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:57.650031+0000 front 2024-04-26T02:06:57.649560+0000 (oldest deadline 2024-04-26T02:07:23.549741+0000) 2024-04-26T02:07:42.123 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:41 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:41.869+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:07.651704+0000 front 2024-04-26T02:07:07.651639+0000 (oldest deadline 2024-04-26T02:07:30.551799+0000) 2024-04-26T02:07:42.123 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:41 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:41.869+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:16.955702+0000 front 2024-04-26T02:07:16.956039+0000 (oldest deadline 2024-04-26T02:07:41.655686+0000) 2024-04-26T02:07:42.499 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:42 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:42.298+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.476432+0000 front 2024-04-26T02:06:50.476644+0000 (oldest deadline 2024-04-26T02:07:18.078167+0000) 2024-04-26T02:07:42.499 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:42 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:42.298+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.682370+0000 front 2024-04-26T02:06:59.682519+0000 (oldest deadline 2024-04-26T02:07:24.982614+0000) 2024-04-26T02:07:42.499 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:42 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:42.298+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:04.984189+0000 front 2024-04-26T02:07:04.983860+0000 (oldest deadline 2024-04-26T02:07:33.784534+0000) 2024-04-26T02:07:42.787 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:42 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:42.524+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.275620+0000 front 2024-04-26T02:06:59.275446+0000 (oldest deadline 2024-04-26T02:07:23.975481+0000) 2024-04-26T02:07:42.787 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:42 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:42.524+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.477628+0000 front 2024-04-26T02:07:08.477663+0000 (oldest deadline 2024-04-26T02:07:30.177791+0000) 2024-04-26T02:07:42.787 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:42 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:42.524+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:17.179691+0000 front 2024-04-26T02:07:17.179731+0000 (oldest deadline 2024-04-26T02:07:40.079515+0000) 2024-04-26T02:07:42.846 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:42 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:42.406+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.419661+0000 front 2024-04-26T02:07:00.419639+0000 (oldest deadline 2024-04-26T02:07:25.719796+0000) 2024-04-26T02:07:42.847 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:42 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:42.406+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.721482+0000 front 2024-04-26T02:07:06.721504+0000 (oldest deadline 2024-04-26T02:07:31.421627+0000) 2024-04-26T02:07:42.847 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:42 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:42.406+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:13.122746+0000 front 2024-04-26T02:07:13.122675+0000 (oldest deadline 2024-04-26T02:07:38.923057+0000) 2024-04-26T02:07:42.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:42 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:42.574+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:07:42.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:42 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:42.574+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.313401+0000 front 2024-04-26T02:07:00.313375+0000 (oldest deadline 2024-04-26T02:07:25.613566+0000) 2024-04-26T02:07:42.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:42 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:42.574+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.615394+0000 front 2024-04-26T02:07:06.615253+0000 (oldest deadline 2024-04-26T02:07:30.015848+0000) 2024-04-26T02:07:42.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:42 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:42.574+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:15.118484+0000 front 2024-04-26T02:07:15.118531+0000 (oldest deadline 2024-04-26T02:07:40.318854+0000) 2024-04-26T02:07:43.123 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:42 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:42.854+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.647106+0000 front 2024-04-26T02:06:50.647828+0000 (oldest deadline 2024-04-26T02:07:12.947246+0000) 2024-04-26T02:07:43.123 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:42 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:42.854+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:57.650031+0000 front 2024-04-26T02:06:57.649560+0000 (oldest deadline 2024-04-26T02:07:23.549741+0000) 2024-04-26T02:07:43.123 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:42 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:42.854+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:07.651704+0000 front 2024-04-26T02:07:07.651639+0000 (oldest deadline 2024-04-26T02:07:30.551799+0000) 2024-04-26T02:07:43.123 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:42 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6[55965]: 2024-04-26T02:07:42.854+0000 7f5f5a000700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:16.955702+0000 front 2024-04-26T02:07:16.956039+0000 (oldest deadline 2024-04-26T02:07:41.655686+0000) 2024-04-26T02:07:43.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:42 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:42.809+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:07:43.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:42 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:42.809+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.934525+0000 front 2024-04-26T02:06:59.934498+0000 (oldest deadline 2024-04-26T02:07:22.834583+0000) 2024-04-26T02:07:43.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:42 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:42.809+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.735339+0000 front 2024-04-26T02:07:08.735363+0000 (oldest deadline 2024-04-26T02:07:33.435537+0000) 2024-04-26T02:07:43.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:42 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:42.809+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:16.837132+0000 front 2024-04-26T02:07:16.837183+0000 (oldest deadline 2024-04-26T02:07:38.537028+0000) 2024-04-26T02:07:43.553 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:43 smithi151 podman[120191]: 2024-04-26 02:07:43.251035944 +0000 UTC m=+5.512173356 container died a910a3f4d95e3188c2f96fc982a1d604ff5cf122ba5d8c6499b708634ceb3369 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-26T02:07:43.553 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:43 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:43.303+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.476432+0000 front 2024-04-26T02:06:50.476644+0000 (oldest deadline 2024-04-26T02:07:18.078167+0000) 2024-04-26T02:07:43.554 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:43 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:43.303+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.682370+0000 front 2024-04-26T02:06:59.682519+0000 (oldest deadline 2024-04-26T02:07:24.982614+0000) 2024-04-26T02:07:43.554 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:43 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:43.303+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:04.984189+0000 front 2024-04-26T02:07:04.983860+0000 (oldest deadline 2024-04-26T02:07:33.784534+0000) 2024-04-26T02:07:43.554 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:43 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:43.303+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:17.286445+0000 front 2024-04-26T02:07:17.286925+0000 (oldest deadline 2024-04-26T02:07:42.586519+0000) 2024-04-26T02:07:43.748 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:43 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:43.396+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.419661+0000 front 2024-04-26T02:07:00.419639+0000 (oldest deadline 2024-04-26T02:07:25.719796+0000) 2024-04-26T02:07:43.748 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:43 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:43.396+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.721482+0000 front 2024-04-26T02:07:06.721504+0000 (oldest deadline 2024-04-26T02:07:31.421627+0000) 2024-04-26T02:07:43.748 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:43 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:43.396+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:13.122746+0000 front 2024-04-26T02:07:13.122675+0000 (oldest deadline 2024-04-26T02:07:38.923057+0000) 2024-04-26T02:07:43.749 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:43 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:43.615+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:07:43.749 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:43 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:43.615+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.313401+0000 front 2024-04-26T02:07:00.313375+0000 (oldest deadline 2024-04-26T02:07:25.613566+0000) 2024-04-26T02:07:43.749 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:43 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:43.615+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.615394+0000 front 2024-04-26T02:07:06.615253+0000 (oldest deadline 2024-04-26T02:07:30.015848+0000) 2024-04-26T02:07:43.749 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:43 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:43.615+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:15.118484+0000 front 2024-04-26T02:07:15.118531+0000 (oldest deadline 2024-04-26T02:07:40.318854+0000) 2024-04-26T02:07:43.853 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:43 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:43.544+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.275620+0000 front 2024-04-26T02:06:59.275446+0000 (oldest deadline 2024-04-26T02:07:23.975481+0000) 2024-04-26T02:07:43.853 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:43 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:43.544+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.477628+0000 front 2024-04-26T02:07:08.477663+0000 (oldest deadline 2024-04-26T02:07:30.177791+0000) 2024-04-26T02:07:43.853 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:43 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:43.544+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:17.179691+0000 front 2024-04-26T02:07:17.179731+0000 (oldest deadline 2024-04-26T02:07:40.079515+0000) 2024-04-26T02:07:43.872 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:43 smithi151 podman[120191]: 2024-04-26 02:07:43.642042291 +0000 UTC m=+5.903179693 container cleanup a910a3f4d95e3188c2f96fc982a1d604ff5cf122ba5d8c6499b708634ceb3369 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-26T02:07:43.872 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:43 smithi151 bash[120191]: ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6 2024-04-26T02:07:44.269 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:43 smithi151 podman[120220]: 2024-04-26 02:07:43.994477055 +0000 UTC m=+0.734170844 container remove a910a3f4d95e3188c2f96fc982a1d604ff5cf122ba5d8c6499b708634ceb3369 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-26T02:07:44.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:43 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:43.854+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:07:44.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:43 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:43.854+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.934525+0000 front 2024-04-26T02:06:59.934498+0000 (oldest deadline 2024-04-26T02:07:22.834583+0000) 2024-04-26T02:07:44.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:43 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:43.854+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.735339+0000 front 2024-04-26T02:07:08.735363+0000 (oldest deadline 2024-04-26T02:07:33.435537+0000) 2024-04-26T02:07:44.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:43 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:43.854+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:16.837132+0000 front 2024-04-26T02:07:16.837183+0000 (oldest deadline 2024-04-26T02:07:38.537028+0000) 2024-04-26T02:07:44.548 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:44 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:44.269+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.476432+0000 front 2024-04-26T02:06:50.476644+0000 (oldest deadline 2024-04-26T02:07:18.078167+0000) 2024-04-26T02:07:44.548 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:44 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:44.269+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.682370+0000 front 2024-04-26T02:06:59.682519+0000 (oldest deadline 2024-04-26T02:07:24.982614+0000) 2024-04-26T02:07:44.548 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:44 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:44.269+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:04.984189+0000 front 2024-04-26T02:07:04.983860+0000 (oldest deadline 2024-04-26T02:07:33.784534+0000) 2024-04-26T02:07:44.548 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:44 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:44.269+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:17.286445+0000 front 2024-04-26T02:07:17.286925+0000 (oldest deadline 2024-04-26T02:07:42.586519+0000) 2024-04-26T02:07:44.648 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:44 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:44.352+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.419661+0000 front 2024-04-26T02:07:00.419639+0000 (oldest deadline 2024-04-26T02:07:25.719796+0000) 2024-04-26T02:07:44.648 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:44 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:44.352+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.721482+0000 front 2024-04-26T02:07:06.721504+0000 (oldest deadline 2024-04-26T02:07:31.421627+0000) 2024-04-26T02:07:44.648 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:44 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:44.352+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:13.122746+0000 front 2024-04-26T02:07:13.122675+0000 (oldest deadline 2024-04-26T02:07:38.923057+0000) 2024-04-26T02:07:44.787 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:44 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:44.530+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.275620+0000 front 2024-04-26T02:06:59.275446+0000 (oldest deadline 2024-04-26T02:07:23.975481+0000) 2024-04-26T02:07:44.787 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:44 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:44.530+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.477628+0000 front 2024-04-26T02:07:08.477663+0000 (oldest deadline 2024-04-26T02:07:30.177791+0000) 2024-04-26T02:07:44.787 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:44 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:44.530+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:17.179691+0000 front 2024-04-26T02:07:17.179731+0000 (oldest deadline 2024-04-26T02:07:40.079515+0000) 2024-04-26T02:07:44.873 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:44 smithi151 podman[120310]: 2024-04-26 02:07:44.455054661 +0000 UTC m=+0.032657994 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T02:07:44.873 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:44 smithi151 podman[120310]: 2024-04-26 02:07:44.612304284 +0000 UTC m=+0.189907654 container create a36f8db5db3dd022566a45fe139da3cdabbba3a27fded3adadead30c5775a4bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T02:07:45.024 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:44 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:44.649+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:07:45.024 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:44 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:44.649+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.313401+0000 front 2024-04-26T02:07:00.313375+0000 (oldest deadline 2024-04-26T02:07:25.613566+0000) 2024-04-26T02:07:45.024 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:44 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:44.649+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.615394+0000 front 2024-04-26T02:07:06.615253+0000 (oldest deadline 2024-04-26T02:07:30.015848+0000) 2024-04-26T02:07:45.024 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:44 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:44.649+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:15.118484+0000 front 2024-04-26T02:07:15.118531+0000 (oldest deadline 2024-04-26T02:07:40.318854+0000) 2024-04-26T02:07:45.224 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:44 smithi151 podman[120310]: 2024-04-26 02:07:44.939577618 +0000 UTC m=+0.517181008 container init a36f8db5db3dd022566a45fe139da3cdabbba3a27fded3adadead30c5775a4bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-26T02:07:45.224 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:44 smithi151 podman[120310]: 2024-04-26 02:07:44.950545007 +0000 UTC m=+0.528148359 container start a36f8db5db3dd022566a45fe139da3cdabbba3a27fded3adadead30c5775a4bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-26T02:07:45.224 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:45 smithi151 podman[120310]: 2024-04-26 02:07:45.056872228 +0000 UTC m=+0.634475662 container attach a36f8db5db3dd022566a45fe139da3cdabbba3a27fded3adadead30c5775a4bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-26T02:07:45.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:44 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:44.844+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:07:45.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:44 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:44.844+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.934525+0000 front 2024-04-26T02:06:59.934498+0000 (oldest deadline 2024-04-26T02:07:22.834583+0000) 2024-04-26T02:07:45.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:44 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:44.844+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.735339+0000 front 2024-04-26T02:07:08.735363+0000 (oldest deadline 2024-04-26T02:07:33.435537+0000) 2024-04-26T02:07:45.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:44 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:44.844+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:16.837132+0000 front 2024-04-26T02:07:16.837183+0000 (oldest deadline 2024-04-26T02:07:38.537028+0000) 2024-04-26T02:07:45.623 INFO:journalctl@ceph.osd.6.smithi151.stdout:Apr 26 02:07:45 smithi151 podman[120310]: 2024-04-26 02:07:45.40818621 +0000 UTC m=+0.985789587 container died a36f8db5db3dd022566a45fe139da3cdabbba3a27fded3adadead30c5775a4bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240416) 2024-04-26T02:07:45.623 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:45 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:45.223+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.476432+0000 front 2024-04-26T02:06:50.476644+0000 (oldest deadline 2024-04-26T02:07:18.078167+0000) 2024-04-26T02:07:45.623 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:45 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:45.223+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.682370+0000 front 2024-04-26T02:06:59.682519+0000 (oldest deadline 2024-04-26T02:07:24.982614+0000) 2024-04-26T02:07:45.623 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:45 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:45.223+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:04.984189+0000 front 2024-04-26T02:07:04.983860+0000 (oldest deadline 2024-04-26T02:07:33.784534+0000) 2024-04-26T02:07:45.623 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:45 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:45.223+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:17.286445+0000 front 2024-04-26T02:07:17.286925+0000 (oldest deadline 2024-04-26T02:07:42.586519+0000) 2024-04-26T02:07:45.787 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:45 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:45.532+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.275620+0000 front 2024-04-26T02:06:59.275446+0000 (oldest deadline 2024-04-26T02:07:23.975481+0000) 2024-04-26T02:07:45.787 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:45 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:45.532+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.477628+0000 front 2024-04-26T02:07:08.477663+0000 (oldest deadline 2024-04-26T02:07:30.177791+0000) 2024-04-26T02:07:45.787 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:45 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:45.532+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:17.179691+0000 front 2024-04-26T02:07:17.179731+0000 (oldest deadline 2024-04-26T02:07:40.079515+0000) 2024-04-26T02:07:45.847 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:45 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:45.397+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.419661+0000 front 2024-04-26T02:07:00.419639+0000 (oldest deadline 2024-04-26T02:07:25.719796+0000) 2024-04-26T02:07:45.847 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:45 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:45.397+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.721482+0000 front 2024-04-26T02:07:06.721504+0000 (oldest deadline 2024-04-26T02:07:31.421627+0000) 2024-04-26T02:07:45.847 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:45 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:45.397+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:13.122746+0000 front 2024-04-26T02:07:13.122675+0000 (oldest deadline 2024-04-26T02:07:38.923057+0000) 2024-04-26T02:07:45.848 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:45 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:45.646+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:07:45.848 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:45 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:45.646+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.313401+0000 front 2024-04-26T02:07:00.313375+0000 (oldest deadline 2024-04-26T02:07:25.613566+0000) 2024-04-26T02:07:45.848 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:45 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:45.646+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.615394+0000 front 2024-04-26T02:07:06.615253+0000 (oldest deadline 2024-04-26T02:07:30.015848+0000) 2024-04-26T02:07:45.848 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:45 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:45.646+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:15.118484+0000 front 2024-04-26T02:07:15.118531+0000 (oldest deadline 2024-04-26T02:07:40.318854+0000) 2024-04-26T02:07:46.160 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0e49e64-036d-11ef-bc93-c7b262605968@osd.6.service' 2024-04-26T02:07:46.271 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:07:46.271 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-26T02:07:46.271 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-26T02:07:46.271 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-c0e49e64-036d-11ef-bc93-c7b262605968@osd.7 2024-04-26T02:07:46.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:45 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:45.874+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:07:46.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:45 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:45.874+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.934525+0000 front 2024-04-26T02:06:59.934498+0000 (oldest deadline 2024-04-26T02:07:22.834583+0000) 2024-04-26T02:07:46.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:45 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:45.874+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.735339+0000 front 2024-04-26T02:07:08.735363+0000 (oldest deadline 2024-04-26T02:07:33.435537+0000) 2024-04-26T02:07:46.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:45 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:45.874+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:16.837132+0000 front 2024-04-26T02:07:16.837183+0000 (oldest deadline 2024-04-26T02:07:38.537028+0000) 2024-04-26T02:07:46.562 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:46 smithi165 systemd[1]: Stopping Ceph osd.7 for c0e49e64-036d-11ef-bc93-c7b262605968... 2024-04-26T02:07:46.563 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:46 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:46.413+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.419661+0000 front 2024-04-26T02:07:00.419639+0000 (oldest deadline 2024-04-26T02:07:25.719796+0000) 2024-04-26T02:07:46.563 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:46 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:46.413+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.721482+0000 front 2024-04-26T02:07:06.721504+0000 (oldest deadline 2024-04-26T02:07:31.421627+0000) 2024-04-26T02:07:46.563 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:46 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:46.413+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:13.122746+0000 front 2024-04-26T02:07:13.122675+0000 (oldest deadline 2024-04-26T02:07:38.923057+0000) 2024-04-26T02:07:46.563 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:46 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:46.563+0000 7fb40c8c3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:07:46.563 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:46 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:46.563+0000 7fb40c8c3700 -1 osd.7 112 *** Got signal Terminated *** 2024-04-26T02:07:46.563 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:46 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:46.563+0000 7fb40c8c3700 -1 osd.7 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T02:07:46.622 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:46 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:46.174+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.476432+0000 front 2024-04-26T02:06:50.476644+0000 (oldest deadline 2024-04-26T02:07:18.078167+0000) 2024-04-26T02:07:46.623 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:46 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:46.174+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.682370+0000 front 2024-04-26T02:06:59.682519+0000 (oldest deadline 2024-04-26T02:07:24.982614+0000) 2024-04-26T02:07:46.623 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:46 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:46.174+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:04.984189+0000 front 2024-04-26T02:07:04.983860+0000 (oldest deadline 2024-04-26T02:07:33.784534+0000) 2024-04-26T02:07:46.623 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:46 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:46.174+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:17.286445+0000 front 2024-04-26T02:07:17.286925+0000 (oldest deadline 2024-04-26T02:07:42.586519+0000) 2024-04-26T02:07:46.839 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:46 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:46.556+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.275620+0000 front 2024-04-26T02:06:59.275446+0000 (oldest deadline 2024-04-26T02:07:23.975481+0000) 2024-04-26T02:07:46.839 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:46 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:46.556+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.477628+0000 front 2024-04-26T02:07:08.477663+0000 (oldest deadline 2024-04-26T02:07:30.177791+0000) 2024-04-26T02:07:46.839 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:46 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:46.556+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:17.179691+0000 front 2024-04-26T02:07:17.179731+0000 (oldest deadline 2024-04-26T02:07:40.079515+0000) 2024-04-26T02:07:46.846 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:46 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:46.680+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:07:46.846 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:46 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:46.680+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.313401+0000 front 2024-04-26T02:07:00.313375+0000 (oldest deadline 2024-04-26T02:07:25.613566+0000) 2024-04-26T02:07:46.846 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:46 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:46.680+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.615394+0000 front 2024-04-26T02:07:06.615253+0000 (oldest deadline 2024-04-26T02:07:30.015848+0000) 2024-04-26T02:07:46.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:46 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:46.680+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:15.118484+0000 front 2024-04-26T02:07:15.118531+0000 (oldest deadline 2024-04-26T02:07:40.318854+0000) 2024-04-26T02:07:47.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:46 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:46.840+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:07:47.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:46 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:46.840+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.934525+0000 front 2024-04-26T02:06:59.934498+0000 (oldest deadline 2024-04-26T02:07:22.834583+0000) 2024-04-26T02:07:47.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:46 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:46.840+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.735339+0000 front 2024-04-26T02:07:08.735363+0000 (oldest deadline 2024-04-26T02:07:33.435537+0000) 2024-04-26T02:07:47.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:46 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:46.840+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:16.837132+0000 front 2024-04-26T02:07:16.837183+0000 (oldest deadline 2024-04-26T02:07:38.537028+0000) 2024-04-26T02:07:47.622 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:47 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:47.138+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.476432+0000 front 2024-04-26T02:06:50.476644+0000 (oldest deadline 2024-04-26T02:07:18.078167+0000) 2024-04-26T02:07:47.623 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:47 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:47.138+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.682370+0000 front 2024-04-26T02:06:59.682519+0000 (oldest deadline 2024-04-26T02:07:24.982614+0000) 2024-04-26T02:07:47.623 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:47 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:47.138+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:04.984189+0000 front 2024-04-26T02:07:04.983860+0000 (oldest deadline 2024-04-26T02:07:33.784534+0000) 2024-04-26T02:07:47.623 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:47 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:47.138+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:17.286445+0000 front 2024-04-26T02:07:17.286925+0000 (oldest deadline 2024-04-26T02:07:42.586519+0000) 2024-04-26T02:07:47.704 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:47 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:47.420+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.419661+0000 front 2024-04-26T02:07:00.419639+0000 (oldest deadline 2024-04-26T02:07:25.719796+0000) 2024-04-26T02:07:47.704 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:47 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:47.420+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.721482+0000 front 2024-04-26T02:07:06.721504+0000 (oldest deadline 2024-04-26T02:07:31.421627+0000) 2024-04-26T02:07:47.705 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:47 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:47.421+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:13.122746+0000 front 2024-04-26T02:07:13.122675+0000 (oldest deadline 2024-04-26T02:07:38.923057+0000) 2024-04-26T02:07:47.787 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:47 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:47.523+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.275620+0000 front 2024-04-26T02:06:59.275446+0000 (oldest deadline 2024-04-26T02:07:23.975481+0000) 2024-04-26T02:07:47.787 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:47 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:47.523+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.477628+0000 front 2024-04-26T02:07:08.477663+0000 (oldest deadline 2024-04-26T02:07:30.177791+0000) 2024-04-26T02:07:47.787 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:47 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:47.523+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:17.179691+0000 front 2024-04-26T02:07:17.179731+0000 (oldest deadline 2024-04-26T02:07:40.079515+0000) 2024-04-26T02:07:48.096 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:47 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:47.705+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:07:48.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:47 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:47.705+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.313401+0000 front 2024-04-26T02:07:00.313375+0000 (oldest deadline 2024-04-26T02:07:25.613566+0000) 2024-04-26T02:07:48.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:47 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:47.705+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.615394+0000 front 2024-04-26T02:07:06.615253+0000 (oldest deadline 2024-04-26T02:07:30.015848+0000) 2024-04-26T02:07:48.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:47 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:47.705+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:15.118484+0000 front 2024-04-26T02:07:15.118531+0000 (oldest deadline 2024-04-26T02:07:40.318854+0000) 2024-04-26T02:07:48.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:47 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:47.840+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:07:48.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:47 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:47.840+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.934525+0000 front 2024-04-26T02:06:59.934498+0000 (oldest deadline 2024-04-26T02:07:22.834583+0000) 2024-04-26T02:07:48.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:47 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:47.840+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.735339+0000 front 2024-04-26T02:07:08.735363+0000 (oldest deadline 2024-04-26T02:07:33.435537+0000) 2024-04-26T02:07:48.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:47 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:47.840+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:16.837132+0000 front 2024-04-26T02:07:16.837183+0000 (oldest deadline 2024-04-26T02:07:38.537028+0000) 2024-04-26T02:07:48.372 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:48 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:48.105+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.476432+0000 front 2024-04-26T02:06:50.476644+0000 (oldest deadline 2024-04-26T02:07:18.078167+0000) 2024-04-26T02:07:48.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:48 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:48.105+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.682370+0000 front 2024-04-26T02:06:59.682519+0000 (oldest deadline 2024-04-26T02:07:24.982614+0000) 2024-04-26T02:07:48.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:48 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:48.105+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:04.984189+0000 front 2024-04-26T02:07:04.983860+0000 (oldest deadline 2024-04-26T02:07:33.784534+0000) 2024-04-26T02:07:48.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:48 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:48.105+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:17.286445+0000 front 2024-04-26T02:07:17.286925+0000 (oldest deadline 2024-04-26T02:07:42.586519+0000) 2024-04-26T02:07:48.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:48 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:48.105+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:24.288859+0000 front 2024-04-26T02:07:24.288963+0000 (oldest deadline 2024-04-26T02:07:47.788514+0000) 2024-04-26T02:07:48.656 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:48 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:48.387+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.419661+0000 front 2024-04-26T02:07:00.419639+0000 (oldest deadline 2024-04-26T02:07:25.719796+0000) 2024-04-26T02:07:48.656 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:48 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:48.387+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.721482+0000 front 2024-04-26T02:07:06.721504+0000 (oldest deadline 2024-04-26T02:07:31.421627+0000) 2024-04-26T02:07:48.656 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:48 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:48.387+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:13.122746+0000 front 2024-04-26T02:07:13.122675+0000 (oldest deadline 2024-04-26T02:07:38.923057+0000) 2024-04-26T02:07:48.656 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:48 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:48.387+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:22.924247+0000 front 2024-04-26T02:07:22.924384+0000 (oldest deadline 2024-04-26T02:07:48.125006+0000) 2024-04-26T02:07:48.787 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:48 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:48.523+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.275620+0000 front 2024-04-26T02:06:59.275446+0000 (oldest deadline 2024-04-26T02:07:23.975481+0000) 2024-04-26T02:07:48.787 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:48 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:48.523+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.477628+0000 front 2024-04-26T02:07:08.477663+0000 (oldest deadline 2024-04-26T02:07:30.177791+0000) 2024-04-26T02:07:48.787 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:48 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:48.523+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:17.179691+0000 front 2024-04-26T02:07:17.179731+0000 (oldest deadline 2024-04-26T02:07:40.079515+0000) 2024-04-26T02:07:49.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:48 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:48.657+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:07:49.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:48 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:48.657+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.313401+0000 front 2024-04-26T02:07:00.313375+0000 (oldest deadline 2024-04-26T02:07:25.613566+0000) 2024-04-26T02:07:49.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:48 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:48.657+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.615394+0000 front 2024-04-26T02:07:06.615253+0000 (oldest deadline 2024-04-26T02:07:30.015848+0000) 2024-04-26T02:07:49.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:48 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:48.657+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:15.118484+0000 front 2024-04-26T02:07:15.118531+0000 (oldest deadline 2024-04-26T02:07:40.318854+0000) 2024-04-26T02:07:49.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:48 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:48.803+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:07:49.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:48 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:48.803+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.934525+0000 front 2024-04-26T02:06:59.934498+0000 (oldest deadline 2024-04-26T02:07:22.834583+0000) 2024-04-26T02:07:49.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:48 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:48.803+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.735339+0000 front 2024-04-26T02:07:08.735363+0000 (oldest deadline 2024-04-26T02:07:33.435537+0000) 2024-04-26T02:07:49.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:48 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:48.803+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:16.837132+0000 front 2024-04-26T02:07:16.837183+0000 (oldest deadline 2024-04-26T02:07:38.537028+0000) 2024-04-26T02:07:49.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:49 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:49.082+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.476432+0000 front 2024-04-26T02:06:50.476644+0000 (oldest deadline 2024-04-26T02:07:18.078167+0000) 2024-04-26T02:07:49.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:49 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:49.082+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.682370+0000 front 2024-04-26T02:06:59.682519+0000 (oldest deadline 2024-04-26T02:07:24.982614+0000) 2024-04-26T02:07:49.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:49 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:49.082+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:04.984189+0000 front 2024-04-26T02:07:04.983860+0000 (oldest deadline 2024-04-26T02:07:33.784534+0000) 2024-04-26T02:07:49.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:49 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:49.082+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:17.286445+0000 front 2024-04-26T02:07:17.286925+0000 (oldest deadline 2024-04-26T02:07:42.586519+0000) 2024-04-26T02:07:49.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:49 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:49.082+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:24.288859+0000 front 2024-04-26T02:07:24.288963+0000 (oldest deadline 2024-04-26T02:07:47.788514+0000) 2024-04-26T02:07:49.787 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:49 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:49.503+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.275620+0000 front 2024-04-26T02:06:59.275446+0000 (oldest deadline 2024-04-26T02:07:23.975481+0000) 2024-04-26T02:07:49.787 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:49 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:49.503+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.477628+0000 front 2024-04-26T02:07:08.477663+0000 (oldest deadline 2024-04-26T02:07:30.177791+0000) 2024-04-26T02:07:49.787 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:49 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:49.503+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:17.179691+0000 front 2024-04-26T02:07:17.179731+0000 (oldest deadline 2024-04-26T02:07:40.079515+0000) 2024-04-26T02:07:49.846 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:49 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:49.419+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.419661+0000 front 2024-04-26T02:07:00.419639+0000 (oldest deadline 2024-04-26T02:07:25.719796+0000) 2024-04-26T02:07:49.847 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:49 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:49.419+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.721482+0000 front 2024-04-26T02:07:06.721504+0000 (oldest deadline 2024-04-26T02:07:31.421627+0000) 2024-04-26T02:07:49.847 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:49 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:49.419+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:13.122746+0000 front 2024-04-26T02:07:13.122675+0000 (oldest deadline 2024-04-26T02:07:38.923057+0000) 2024-04-26T02:07:49.847 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:49 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:49.419+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:22.924247+0000 front 2024-04-26T02:07:22.924384+0000 (oldest deadline 2024-04-26T02:07:48.125006+0000) 2024-04-26T02:07:49.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:49 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:49.622+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:07:49.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:49 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:49.622+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.313401+0000 front 2024-04-26T02:07:00.313375+0000 (oldest deadline 2024-04-26T02:07:25.613566+0000) 2024-04-26T02:07:49.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:49 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:49.622+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.615394+0000 front 2024-04-26T02:07:06.615253+0000 (oldest deadline 2024-04-26T02:07:30.015848+0000) 2024-04-26T02:07:49.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:49 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:49.622+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:15.118484+0000 front 2024-04-26T02:07:15.118531+0000 (oldest deadline 2024-04-26T02:07:40.318854+0000) 2024-04-26T02:07:50.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:49 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:49.800+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:07:50.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:49 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:49.800+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.934525+0000 front 2024-04-26T02:06:59.934498+0000 (oldest deadline 2024-04-26T02:07:22.834583+0000) 2024-04-26T02:07:50.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:49 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:49.800+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.735339+0000 front 2024-04-26T02:07:08.735363+0000 (oldest deadline 2024-04-26T02:07:33.435537+0000) 2024-04-26T02:07:50.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:49 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:49.800+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:16.837132+0000 front 2024-04-26T02:07:16.837183+0000 (oldest deadline 2024-04-26T02:07:38.537028+0000) 2024-04-26T02:07:50.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:49 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:49.800+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:24.239082+0000 front 2024-04-26T02:07:24.239105+0000 (oldest deadline 2024-04-26T02:07:49.439890+0000) 2024-04-26T02:07:50.372 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:50 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:50.098+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.476432+0000 front 2024-04-26T02:06:50.476644+0000 (oldest deadline 2024-04-26T02:07:18.078167+0000) 2024-04-26T02:07:50.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:50 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:50.098+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.682370+0000 front 2024-04-26T02:06:59.682519+0000 (oldest deadline 2024-04-26T02:07:24.982614+0000) 2024-04-26T02:07:50.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:50 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:50.098+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:04.984189+0000 front 2024-04-26T02:07:04.983860+0000 (oldest deadline 2024-04-26T02:07:33.784534+0000) 2024-04-26T02:07:50.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:50 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:50.098+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:17.286445+0000 front 2024-04-26T02:07:17.286925+0000 (oldest deadline 2024-04-26T02:07:42.586519+0000) 2024-04-26T02:07:50.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:50 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:50.098+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:24.288859+0000 front 2024-04-26T02:07:24.288963+0000 (oldest deadline 2024-04-26T02:07:47.788514+0000) 2024-04-26T02:07:50.649 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:50 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:50.374+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.419661+0000 front 2024-04-26T02:07:00.419639+0000 (oldest deadline 2024-04-26T02:07:25.719796+0000) 2024-04-26T02:07:50.649 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:50 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:50.374+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.721482+0000 front 2024-04-26T02:07:06.721504+0000 (oldest deadline 2024-04-26T02:07:31.421627+0000) 2024-04-26T02:07:50.649 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:50 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:50.374+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:13.122746+0000 front 2024-04-26T02:07:13.122675+0000 (oldest deadline 2024-04-26T02:07:38.923057+0000) 2024-04-26T02:07:50.649 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:50 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:50.374+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:22.924247+0000 front 2024-04-26T02:07:22.924384+0000 (oldest deadline 2024-04-26T02:07:48.125006+0000) 2024-04-26T02:07:50.787 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:50 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:50.531+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.275620+0000 front 2024-04-26T02:06:59.275446+0000 (oldest deadline 2024-04-26T02:07:23.975481+0000) 2024-04-26T02:07:50.787 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:50 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:50.532+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.477628+0000 front 2024-04-26T02:07:08.477663+0000 (oldest deadline 2024-04-26T02:07:30.177791+0000) 2024-04-26T02:07:50.787 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:50 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:50.532+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:17.179691+0000 front 2024-04-26T02:07:17.179731+0000 (oldest deadline 2024-04-26T02:07:40.079515+0000) 2024-04-26T02:07:51.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:50 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:50.650+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:07:51.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:50 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:50.650+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.313401+0000 front 2024-04-26T02:07:00.313375+0000 (oldest deadline 2024-04-26T02:07:25.613566+0000) 2024-04-26T02:07:51.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:50 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:50.650+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.615394+0000 front 2024-04-26T02:07:06.615253+0000 (oldest deadline 2024-04-26T02:07:30.015848+0000) 2024-04-26T02:07:51.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:50 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:50.650+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:15.118484+0000 front 2024-04-26T02:07:15.118531+0000 (oldest deadline 2024-04-26T02:07:40.318854+0000) 2024-04-26T02:07:51.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:50 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:50.808+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:07:51.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:50 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:50.808+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.934525+0000 front 2024-04-26T02:06:59.934498+0000 (oldest deadline 2024-04-26T02:07:22.834583+0000) 2024-04-26T02:07:51.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:50 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:50.808+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.735339+0000 front 2024-04-26T02:07:08.735363+0000 (oldest deadline 2024-04-26T02:07:33.435537+0000) 2024-04-26T02:07:51.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:50 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:50.808+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:16.837132+0000 front 2024-04-26T02:07:16.837183+0000 (oldest deadline 2024-04-26T02:07:38.537028+0000) 2024-04-26T02:07:51.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:50 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:50.808+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:24.239082+0000 front 2024-04-26T02:07:24.239105+0000 (oldest deadline 2024-04-26T02:07:49.439890+0000) 2024-04-26T02:07:51.372 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:51 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:51.077+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.476432+0000 front 2024-04-26T02:06:50.476644+0000 (oldest deadline 2024-04-26T02:07:18.078167+0000) 2024-04-26T02:07:51.372 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:51 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:51.077+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.682370+0000 front 2024-04-26T02:06:59.682519+0000 (oldest deadline 2024-04-26T02:07:24.982614+0000) 2024-04-26T02:07:51.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:51 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:51.077+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:04.984189+0000 front 2024-04-26T02:07:04.983860+0000 (oldest deadline 2024-04-26T02:07:33.784534+0000) 2024-04-26T02:07:51.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:51 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:51.077+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:17.286445+0000 front 2024-04-26T02:07:17.286925+0000 (oldest deadline 2024-04-26T02:07:42.586519+0000) 2024-04-26T02:07:51.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:51 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:51.077+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:24.288859+0000 front 2024-04-26T02:07:24.288963+0000 (oldest deadline 2024-04-26T02:07:47.788514+0000) 2024-04-26T02:07:51.847 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:51 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:51.375+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.419661+0000 front 2024-04-26T02:07:00.419639+0000 (oldest deadline 2024-04-26T02:07:25.719796+0000) 2024-04-26T02:07:51.847 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:51 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:51.375+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.721482+0000 front 2024-04-26T02:07:06.721504+0000 (oldest deadline 2024-04-26T02:07:31.421627+0000) 2024-04-26T02:07:51.847 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:51 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:51.375+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:13.122746+0000 front 2024-04-26T02:07:13.122675+0000 (oldest deadline 2024-04-26T02:07:38.923057+0000) 2024-04-26T02:07:51.847 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:51 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-26T02:07:51.375+0000 7fb408eb7700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:22.924247+0000 front 2024-04-26T02:07:22.924384+0000 (oldest deadline 2024-04-26T02:07:48.125006+0000) 2024-04-26T02:07:51.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:51 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:51.623+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:07:51.848 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:51 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:51.623+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.313401+0000 front 2024-04-26T02:07:00.313375+0000 (oldest deadline 2024-04-26T02:07:25.613566+0000) 2024-04-26T02:07:51.848 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:51 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:51.623+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.615394+0000 front 2024-04-26T02:07:06.615253+0000 (oldest deadline 2024-04-26T02:07:30.015848+0000) 2024-04-26T02:07:51.848 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:51 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:51.623+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:15.118484+0000 front 2024-04-26T02:07:15.118531+0000 (oldest deadline 2024-04-26T02:07:40.318854+0000) 2024-04-26T02:07:52.037 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:51 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:51.580+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.275620+0000 front 2024-04-26T02:06:59.275446+0000 (oldest deadline 2024-04-26T02:07:23.975481+0000) 2024-04-26T02:07:52.037 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:51 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:51.580+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.477628+0000 front 2024-04-26T02:07:08.477663+0000 (oldest deadline 2024-04-26T02:07:30.177791+0000) 2024-04-26T02:07:52.038 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:51 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:51.580+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:17.179691+0000 front 2024-04-26T02:07:17.179731+0000 (oldest deadline 2024-04-26T02:07:40.079515+0000) 2024-04-26T02:07:52.038 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:51 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:51.823+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:07:52.038 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:51 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:51.823+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.934525+0000 front 2024-04-26T02:06:59.934498+0000 (oldest deadline 2024-04-26T02:07:22.834583+0000) 2024-04-26T02:07:52.038 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:51 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:51.823+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.735339+0000 front 2024-04-26T02:07:08.735363+0000 (oldest deadline 2024-04-26T02:07:33.435537+0000) 2024-04-26T02:07:52.038 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:51 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:51.823+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:16.837132+0000 front 2024-04-26T02:07:16.837183+0000 (oldest deadline 2024-04-26T02:07:38.537028+0000) 2024-04-26T02:07:52.038 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:51 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:51.823+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:24.239082+0000 front 2024-04-26T02:07:24.239105+0000 (oldest deadline 2024-04-26T02:07:49.439890+0000) 2024-04-26T02:07:52.269 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:51 smithi165 podman[70403]: 2024-04-26 02:07:51.968943089 +0000 UTC m=+5.511856274 container died cabe7c356de29377100ce6f21d232c2fb2fa8e13259559d9051ea8ee05257aab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-26T02:07:52.372 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:52 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:52.055+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.476432+0000 front 2024-04-26T02:06:50.476644+0000 (oldest deadline 2024-04-26T02:07:18.078167+0000) 2024-04-26T02:07:52.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:52 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:52.055+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.682370+0000 front 2024-04-26T02:06:59.682519+0000 (oldest deadline 2024-04-26T02:07:24.982614+0000) 2024-04-26T02:07:52.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:52 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:52.055+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:04.984189+0000 front 2024-04-26T02:07:04.983860+0000 (oldest deadline 2024-04-26T02:07:33.784534+0000) 2024-04-26T02:07:52.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:52 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:52.055+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:17.286445+0000 front 2024-04-26T02:07:17.286925+0000 (oldest deadline 2024-04-26T02:07:42.586519+0000) 2024-04-26T02:07:52.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:52 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:52.055+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:24.288859+0000 front 2024-04-26T02:07:24.288963+0000 (oldest deadline 2024-04-26T02:07:47.788514+0000) 2024-04-26T02:07:52.597 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:52 smithi165 podman[70403]: 2024-04-26 02:07:52.271700837 +0000 UTC m=+5.814614030 container cleanup cabe7c356de29377100ce6f21d232c2fb2fa8e13259559d9051ea8ee05257aab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T02:07:52.597 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:52 smithi165 bash[70403]: ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7 2024-04-26T02:07:52.597 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:52 smithi165 podman[70420]: 2024-04-26 02:07:52.480944047 +0000 UTC m=+0.506916278 container remove cabe7c356de29377100ce6f21d232c2fb2fa8e13259559d9051ea8ee05257aab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-26T02:07:52.930 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:52 smithi165 podman[70513]: 2024-04-26 02:07:52.715412086 +0000 UTC m=+0.018470340 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T02:07:52.930 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:52 smithi165 podman[70513]: 2024-04-26 02:07:52.823336572 +0000 UTC m=+0.126394820 container create 78d011851eb4ce854da2a4583e0dcba56744ac59db6271cc84f713a579f1b932 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0) 2024-04-26T02:07:52.931 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:52 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:52.659+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:07:52.931 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:52 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:52.659+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.313401+0000 front 2024-04-26T02:07:00.313375+0000 (oldest deadline 2024-04-26T02:07:25.613566+0000) 2024-04-26T02:07:52.931 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:52 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:52.659+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.615394+0000 front 2024-04-26T02:07:06.615253+0000 (oldest deadline 2024-04-26T02:07:30.015848+0000) 2024-04-26T02:07:52.931 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:52 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:52.659+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:15.118484+0000 front 2024-04-26T02:07:15.118531+0000 (oldest deadline 2024-04-26T02:07:40.318854+0000) 2024-04-26T02:07:52.931 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:52 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:52.659+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:26.521730+0000 front 2024-04-26T02:07:26.521753+0000 (oldest deadline 2024-04-26T02:07:52.421703+0000) 2024-04-26T02:07:53.037 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:52 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:52.578+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.275620+0000 front 2024-04-26T02:06:59.275446+0000 (oldest deadline 2024-04-26T02:07:23.975481+0000) 2024-04-26T02:07:53.037 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:52 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:52.578+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.477628+0000 front 2024-04-26T02:07:08.477663+0000 (oldest deadline 2024-04-26T02:07:30.177791+0000) 2024-04-26T02:07:53.037 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:52 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:52.578+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:17.179691+0000 front 2024-04-26T02:07:17.179731+0000 (oldest deadline 2024-04-26T02:07:40.079515+0000) 2024-04-26T02:07:53.037 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:52 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:52.773+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:07:53.038 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:52 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:52.773+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.934525+0000 front 2024-04-26T02:06:59.934498+0000 (oldest deadline 2024-04-26T02:07:22.834583+0000) 2024-04-26T02:07:53.038 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:52 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:52.773+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.735339+0000 front 2024-04-26T02:07:08.735363+0000 (oldest deadline 2024-04-26T02:07:33.435537+0000) 2024-04-26T02:07:53.038 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:52 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:52.773+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:16.837132+0000 front 2024-04-26T02:07:16.837183+0000 (oldest deadline 2024-04-26T02:07:38.537028+0000) 2024-04-26T02:07:53.038 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:52 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:52.773+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:24.239082+0000 front 2024-04-26T02:07:24.239105+0000 (oldest deadline 2024-04-26T02:07:49.439890+0000) 2024-04-26T02:07:53.335 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:53 smithi165 podman[70513]: 2024-04-26 02:07:53.042498558 +0000 UTC m=+0.345556803 container init 78d011851eb4ce854da2a4583e0dcba56744ac59db6271cc84f713a579f1b932 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS) 2024-04-26T02:07:53.335 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:53 smithi165 podman[70513]: 2024-04-26 02:07:53.045753029 +0000 UTC m=+0.348811281 container start 78d011851eb4ce854da2a4583e0dcba56744ac59db6271cc84f713a579f1b932 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T02:07:53.335 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:53 smithi165 podman[70513]: 2024-04-26 02:07:53.11716153 +0000 UTC m=+0.420219776 container attach 78d011851eb4ce854da2a4583e0dcba56744ac59db6271cc84f713a579f1b932 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-26T02:07:53.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:53 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:53.013+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.476432+0000 front 2024-04-26T02:06:50.476644+0000 (oldest deadline 2024-04-26T02:07:18.078167+0000) 2024-04-26T02:07:53.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:53 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:53.013+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.682370+0000 front 2024-04-26T02:06:59.682519+0000 (oldest deadline 2024-04-26T02:07:24.982614+0000) 2024-04-26T02:07:53.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:53 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:53.013+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:04.984189+0000 front 2024-04-26T02:07:04.983860+0000 (oldest deadline 2024-04-26T02:07:33.784534+0000) 2024-04-26T02:07:53.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:53 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:53.013+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:17.286445+0000 front 2024-04-26T02:07:17.286925+0000 (oldest deadline 2024-04-26T02:07:42.586519+0000) 2024-04-26T02:07:53.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:53 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:53.013+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:24.288859+0000 front 2024-04-26T02:07:24.288963+0000 (oldest deadline 2024-04-26T02:07:47.788514+0000) 2024-04-26T02:07:53.597 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:53 smithi165 podman[70513]: 2024-04-26 02:07:53.337340273 +0000 UTC m=+0.640398526 container died 78d011851eb4ce854da2a4583e0dcba56744ac59db6271cc84f713a579f1b932 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-26T02:07:53.875 INFO:journalctl@ceph.osd.7.smithi165.stdout:Apr 26 02:07:53 smithi165 podman[70538]: 2024-04-26 02:07:53.862284791 +0000 UTC m=+0.516384764 container remove 78d011851eb4ce854da2a4583e0dcba56744ac59db6271cc84f713a579f1b932 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T02:07:53.875 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:53 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:53.658+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:07:53.875 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:53 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:53.658+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.313401+0000 front 2024-04-26T02:07:00.313375+0000 (oldest deadline 2024-04-26T02:07:25.613566+0000) 2024-04-26T02:07:53.876 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:53 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:53.658+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.615394+0000 front 2024-04-26T02:07:06.615253+0000 (oldest deadline 2024-04-26T02:07:30.015848+0000) 2024-04-26T02:07:53.876 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:53 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:53.658+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:15.118484+0000 front 2024-04-26T02:07:15.118531+0000 (oldest deadline 2024-04-26T02:07:40.318854+0000) 2024-04-26T02:07:53.876 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:53 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:53.658+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:26.521730+0000 front 2024-04-26T02:07:26.521753+0000 (oldest deadline 2024-04-26T02:07:52.421703+0000) 2024-04-26T02:07:53.880 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0e49e64-036d-11ef-bc93-c7b262605968@osd.7.service' 2024-04-26T02:07:53.918 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:07:53.918 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-26T02:07:53.918 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-26T02:07:53.919 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-c0e49e64-036d-11ef-bc93-c7b262605968@osd.8 2024-04-26T02:07:53.946 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:53 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:53.618+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.275620+0000 front 2024-04-26T02:06:59.275446+0000 (oldest deadline 2024-04-26T02:07:23.975481+0000) 2024-04-26T02:07:53.946 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:53 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:53.618+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.477628+0000 front 2024-04-26T02:07:08.477663+0000 (oldest deadline 2024-04-26T02:07:30.177791+0000) 2024-04-26T02:07:53.946 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:53 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:53.618+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:17.179691+0000 front 2024-04-26T02:07:17.179731+0000 (oldest deadline 2024-04-26T02:07:40.079515+0000) 2024-04-26T02:07:53.946 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:53 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:53.618+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:24.780551+0000 front 2024-04-26T02:07:24.780463+0000 (oldest deadline 2024-04-26T02:07:52.981329+0000) 2024-04-26T02:07:53.946 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:53 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:53.822+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:07:53.946 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:53 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:53.822+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.934525+0000 front 2024-04-26T02:06:59.934498+0000 (oldest deadline 2024-04-26T02:07:22.834583+0000) 2024-04-26T02:07:53.947 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:53 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:53.822+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.735339+0000 front 2024-04-26T02:07:08.735363+0000 (oldest deadline 2024-04-26T02:07:33.435537+0000) 2024-04-26T02:07:53.947 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:53 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:53.822+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:16.837132+0000 front 2024-04-26T02:07:16.837183+0000 (oldest deadline 2024-04-26T02:07:38.537028+0000) 2024-04-26T02:07:53.947 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:53 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:53.822+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:24.239082+0000 front 2024-04-26T02:07:24.239105+0000 (oldest deadline 2024-04-26T02:07:49.439890+0000) 2024-04-26T02:07:54.122 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:54 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:54.026+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.476432+0000 front 2024-04-26T02:06:50.476644+0000 (oldest deadline 2024-04-26T02:07:18.078167+0000) 2024-04-26T02:07:54.123 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:54 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:54.026+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.682370+0000 front 2024-04-26T02:06:59.682519+0000 (oldest deadline 2024-04-26T02:07:24.982614+0000) 2024-04-26T02:07:54.123 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:54 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:54.026+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:04.984189+0000 front 2024-04-26T02:07:04.983860+0000 (oldest deadline 2024-04-26T02:07:33.784534+0000) 2024-04-26T02:07:54.123 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:54 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:54.026+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:17.286445+0000 front 2024-04-26T02:07:17.286925+0000 (oldest deadline 2024-04-26T02:07:42.586519+0000) 2024-04-26T02:07:54.123 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:54 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:54.026+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:24.288859+0000 front 2024-04-26T02:07:24.288963+0000 (oldest deadline 2024-04-26T02:07:47.788514+0000) 2024-04-26T02:07:54.236 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:53 smithi172 systemd[1]: Stopping Ceph osd.8 for c0e49e64-036d-11ef-bc93-c7b262605968... 2024-04-26T02:07:54.537 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:54 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:54.236+0000 7f86810f2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:07:54.537 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:54 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:54.236+0000 7f86810f2700 -1 osd.8 112 *** Got signal Terminated *** 2024-04-26T02:07:54.537 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:54 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:54.236+0000 7f86810f2700 -1 osd.8 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T02:07:54.972 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:54 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:54.617+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:07:54.972 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:54 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:54.617+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.313401+0000 front 2024-04-26T02:07:00.313375+0000 (oldest deadline 2024-04-26T02:07:25.613566+0000) 2024-04-26T02:07:54.972 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:54 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:54.617+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.615394+0000 front 2024-04-26T02:07:06.615253+0000 (oldest deadline 2024-04-26T02:07:30.015848+0000) 2024-04-26T02:07:54.972 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:54 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:54.617+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:15.118484+0000 front 2024-04-26T02:07:15.118531+0000 (oldest deadline 2024-04-26T02:07:40.318854+0000) 2024-04-26T02:07:54.972 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:54 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:54.617+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:26.521730+0000 front 2024-04-26T02:07:26.521753+0000 (oldest deadline 2024-04-26T02:07:52.421703+0000) 2024-04-26T02:07:55.037 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:54 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:54.613+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.275620+0000 front 2024-04-26T02:06:59.275446+0000 (oldest deadline 2024-04-26T02:07:23.975481+0000) 2024-04-26T02:07:55.037 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:54 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:54.613+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.477628+0000 front 2024-04-26T02:07:08.477663+0000 (oldest deadline 2024-04-26T02:07:30.177791+0000) 2024-04-26T02:07:55.038 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:54 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:54.613+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:17.179691+0000 front 2024-04-26T02:07:17.179731+0000 (oldest deadline 2024-04-26T02:07:40.079515+0000) 2024-04-26T02:07:55.038 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:54 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:54.613+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:24.780551+0000 front 2024-04-26T02:07:24.780463+0000 (oldest deadline 2024-04-26T02:07:52.981329+0000) 2024-04-26T02:07:55.038 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:54 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:54.805+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:07:55.038 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:54 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:54.805+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.934525+0000 front 2024-04-26T02:06:59.934498+0000 (oldest deadline 2024-04-26T02:07:22.834583+0000) 2024-04-26T02:07:55.038 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:54 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:54.805+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.735339+0000 front 2024-04-26T02:07:08.735363+0000 (oldest deadline 2024-04-26T02:07:33.435537+0000) 2024-04-26T02:07:55.038 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:54 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:54.805+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:16.837132+0000 front 2024-04-26T02:07:16.837183+0000 (oldest deadline 2024-04-26T02:07:38.537028+0000) 2024-04-26T02:07:55.038 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:54 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:54.805+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:24.239082+0000 front 2024-04-26T02:07:24.239105+0000 (oldest deadline 2024-04-26T02:07:49.439890+0000) 2024-04-26T02:07:55.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:55 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:55.070+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.476432+0000 front 2024-04-26T02:06:50.476644+0000 (oldest deadline 2024-04-26T02:07:18.078167+0000) 2024-04-26T02:07:55.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:55 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:55.070+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.682370+0000 front 2024-04-26T02:06:59.682519+0000 (oldest deadline 2024-04-26T02:07:24.982614+0000) 2024-04-26T02:07:55.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:55 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:55.070+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:04.984189+0000 front 2024-04-26T02:07:04.983860+0000 (oldest deadline 2024-04-26T02:07:33.784534+0000) 2024-04-26T02:07:55.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:55 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:55.070+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:17.286445+0000 front 2024-04-26T02:07:17.286925+0000 (oldest deadline 2024-04-26T02:07:42.586519+0000) 2024-04-26T02:07:55.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:55 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:55.070+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:24.288859+0000 front 2024-04-26T02:07:24.288963+0000 (oldest deadline 2024-04-26T02:07:47.788514+0000) 2024-04-26T02:07:55.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:55 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:55.586+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:07:55.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:55 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:55.586+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.313401+0000 front 2024-04-26T02:07:00.313375+0000 (oldest deadline 2024-04-26T02:07:25.613566+0000) 2024-04-26T02:07:55.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:55 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:55.586+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.615394+0000 front 2024-04-26T02:07:06.615253+0000 (oldest deadline 2024-04-26T02:07:30.015848+0000) 2024-04-26T02:07:55.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:55 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:55.586+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:15.118484+0000 front 2024-04-26T02:07:15.118531+0000 (oldest deadline 2024-04-26T02:07:40.318854+0000) 2024-04-26T02:07:55.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:55 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:55.586+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:26.521730+0000 front 2024-04-26T02:07:26.521753+0000 (oldest deadline 2024-04-26T02:07:52.421703+0000) 2024-04-26T02:07:55.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:55 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:55.586+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:32.422457+0000 front 2024-04-26T02:07:32.422344+0000 (oldest deadline 2024-04-26T02:07:55.322131+0000) 2024-04-26T02:07:56.037 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:55 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:55.605+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.275620+0000 front 2024-04-26T02:06:59.275446+0000 (oldest deadline 2024-04-26T02:07:23.975481+0000) 2024-04-26T02:07:56.037 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:55 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:55.605+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.477628+0000 front 2024-04-26T02:07:08.477663+0000 (oldest deadline 2024-04-26T02:07:30.177791+0000) 2024-04-26T02:07:56.038 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:55 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:55.605+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:17.179691+0000 front 2024-04-26T02:07:17.179731+0000 (oldest deadline 2024-04-26T02:07:40.079515+0000) 2024-04-26T02:07:56.038 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:55 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:55.605+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:24.780551+0000 front 2024-04-26T02:07:24.780463+0000 (oldest deadline 2024-04-26T02:07:52.981329+0000) 2024-04-26T02:07:56.038 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:55 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:55.605+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:32.981848+0000 front 2024-04-26T02:07:32.981873+0000 (oldest deadline 2024-04-26T02:07:55.282029+0000) 2024-04-26T02:07:56.038 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:55 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:55.793+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:07:56.038 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:55 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:55.793+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.934525+0000 front 2024-04-26T02:06:59.934498+0000 (oldest deadline 2024-04-26T02:07:22.834583+0000) 2024-04-26T02:07:56.038 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:55 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:55.793+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.735339+0000 front 2024-04-26T02:07:08.735363+0000 (oldest deadline 2024-04-26T02:07:33.435537+0000) 2024-04-26T02:07:56.038 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:55 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:55.793+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:16.837132+0000 front 2024-04-26T02:07:16.837183+0000 (oldest deadline 2024-04-26T02:07:38.537028+0000) 2024-04-26T02:07:56.038 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:55 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:55.793+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:24.239082+0000 front 2024-04-26T02:07:24.239105+0000 (oldest deadline 2024-04-26T02:07:49.439890+0000) 2024-04-26T02:07:56.038 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:55 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:55.793+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:34.140899+0000 front 2024-04-26T02:07:34.141046+0000 (oldest deadline 2024-04-26T02:07:55.241143+0000) 2024-04-26T02:07:56.372 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:56 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:56.067+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.476432+0000 front 2024-04-26T02:06:50.476644+0000 (oldest deadline 2024-04-26T02:07:18.078167+0000) 2024-04-26T02:07:56.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:56 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:56.067+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.682370+0000 front 2024-04-26T02:06:59.682519+0000 (oldest deadline 2024-04-26T02:07:24.982614+0000) 2024-04-26T02:07:56.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:56 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:56.067+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:04.984189+0000 front 2024-04-26T02:07:04.983860+0000 (oldest deadline 2024-04-26T02:07:33.784534+0000) 2024-04-26T02:07:56.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:56 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:56.067+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:17.286445+0000 front 2024-04-26T02:07:17.286925+0000 (oldest deadline 2024-04-26T02:07:42.586519+0000) 2024-04-26T02:07:56.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:56 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:56.067+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:24.288859+0000 front 2024-04-26T02:07:24.288963+0000 (oldest deadline 2024-04-26T02:07:47.788514+0000) 2024-04-26T02:07:56.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:56 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:56.554+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:07:56.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:56 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:56.554+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.313401+0000 front 2024-04-26T02:07:00.313375+0000 (oldest deadline 2024-04-26T02:07:25.613566+0000) 2024-04-26T02:07:56.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:56 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:56.554+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.615394+0000 front 2024-04-26T02:07:06.615253+0000 (oldest deadline 2024-04-26T02:07:30.015848+0000) 2024-04-26T02:07:56.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:56 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:56.554+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:15.118484+0000 front 2024-04-26T02:07:15.118531+0000 (oldest deadline 2024-04-26T02:07:40.318854+0000) 2024-04-26T02:07:56.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:56 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:56.554+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:26.521730+0000 front 2024-04-26T02:07:26.521753+0000 (oldest deadline 2024-04-26T02:07:52.421703+0000) 2024-04-26T02:07:56.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:56 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:56.554+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:32.422457+0000 front 2024-04-26T02:07:32.422344+0000 (oldest deadline 2024-04-26T02:07:55.322131+0000) 2024-04-26T02:07:57.037 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:56 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:56.578+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.275620+0000 front 2024-04-26T02:06:59.275446+0000 (oldest deadline 2024-04-26T02:07:23.975481+0000) 2024-04-26T02:07:57.037 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:56 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:56.578+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.477628+0000 front 2024-04-26T02:07:08.477663+0000 (oldest deadline 2024-04-26T02:07:30.177791+0000) 2024-04-26T02:07:57.038 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:56 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:56.578+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:17.179691+0000 front 2024-04-26T02:07:17.179731+0000 (oldest deadline 2024-04-26T02:07:40.079515+0000) 2024-04-26T02:07:57.038 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:56 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:56.578+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:24.780551+0000 front 2024-04-26T02:07:24.780463+0000 (oldest deadline 2024-04-26T02:07:52.981329+0000) 2024-04-26T02:07:57.038 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:56 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:56.578+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:32.981848+0000 front 2024-04-26T02:07:32.981873+0000 (oldest deadline 2024-04-26T02:07:55.282029+0000) 2024-04-26T02:07:57.038 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:56 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:56.778+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:07:57.038 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:56 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:56.778+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.934525+0000 front 2024-04-26T02:06:59.934498+0000 (oldest deadline 2024-04-26T02:07:22.834583+0000) 2024-04-26T02:07:57.038 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:56 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:56.778+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.735339+0000 front 2024-04-26T02:07:08.735363+0000 (oldest deadline 2024-04-26T02:07:33.435537+0000) 2024-04-26T02:07:57.038 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:56 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:56.778+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:16.837132+0000 front 2024-04-26T02:07:16.837183+0000 (oldest deadline 2024-04-26T02:07:38.537028+0000) 2024-04-26T02:07:57.038 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:56 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:56.778+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:24.239082+0000 front 2024-04-26T02:07:24.239105+0000 (oldest deadline 2024-04-26T02:07:49.439890+0000) 2024-04-26T02:07:57.038 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:56 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:56.778+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:34.140899+0000 front 2024-04-26T02:07:34.141046+0000 (oldest deadline 2024-04-26T02:07:55.241143+0000) 2024-04-26T02:07:57.372 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:57 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:57.054+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.476432+0000 front 2024-04-26T02:06:50.476644+0000 (oldest deadline 2024-04-26T02:07:18.078167+0000) 2024-04-26T02:07:57.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:57 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:57.054+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.682370+0000 front 2024-04-26T02:06:59.682519+0000 (oldest deadline 2024-04-26T02:07:24.982614+0000) 2024-04-26T02:07:57.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:57 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:57.054+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:04.984189+0000 front 2024-04-26T02:07:04.983860+0000 (oldest deadline 2024-04-26T02:07:33.784534+0000) 2024-04-26T02:07:57.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:57 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:57.054+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:17.286445+0000 front 2024-04-26T02:07:17.286925+0000 (oldest deadline 2024-04-26T02:07:42.586519+0000) 2024-04-26T02:07:57.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:57 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:57.054+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:24.288859+0000 front 2024-04-26T02:07:24.288963+0000 (oldest deadline 2024-04-26T02:07:47.788514+0000) 2024-04-26T02:07:57.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:57 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:57.054+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:33.690239+0000 front 2024-04-26T02:07:33.690387+0000 (oldest deadline 2024-04-26T02:07:56.590009+0000) 2024-04-26T02:07:58.037 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:57 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:57.595+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.275620+0000 front 2024-04-26T02:06:59.275446+0000 (oldest deadline 2024-04-26T02:07:23.975481+0000) 2024-04-26T02:07:58.037 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:57 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:57.595+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.477628+0000 front 2024-04-26T02:07:08.477663+0000 (oldest deadline 2024-04-26T02:07:30.177791+0000) 2024-04-26T02:07:58.037 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:57 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:57.595+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:17.179691+0000 front 2024-04-26T02:07:17.179731+0000 (oldest deadline 2024-04-26T02:07:40.079515+0000) 2024-04-26T02:07:58.038 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:57 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:57.595+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:24.780551+0000 front 2024-04-26T02:07:24.780463+0000 (oldest deadline 2024-04-26T02:07:52.981329+0000) 2024-04-26T02:07:58.038 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:57 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:57.595+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:32.981848+0000 front 2024-04-26T02:07:32.981873+0000 (oldest deadline 2024-04-26T02:07:55.282029+0000) 2024-04-26T02:07:58.038 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:57 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:57.737+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:07:58.038 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:57 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:57.737+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.934525+0000 front 2024-04-26T02:06:59.934498+0000 (oldest deadline 2024-04-26T02:07:22.834583+0000) 2024-04-26T02:07:58.038 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:57 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:57.737+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.735339+0000 front 2024-04-26T02:07:08.735363+0000 (oldest deadline 2024-04-26T02:07:33.435537+0000) 2024-04-26T02:07:58.038 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:57 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:57.737+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:16.837132+0000 front 2024-04-26T02:07:16.837183+0000 (oldest deadline 2024-04-26T02:07:38.537028+0000) 2024-04-26T02:07:58.038 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:57 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:57.737+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:24.239082+0000 front 2024-04-26T02:07:24.239105+0000 (oldest deadline 2024-04-26T02:07:49.439890+0000) 2024-04-26T02:07:58.038 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:57 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:57.737+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:34.140899+0000 front 2024-04-26T02:07:34.141046+0000 (oldest deadline 2024-04-26T02:07:55.241143+0000) 2024-04-26T02:07:58.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:57 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:57.601+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:07:58.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:57 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:57.601+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.313401+0000 front 2024-04-26T02:07:00.313375+0000 (oldest deadline 2024-04-26T02:07:25.613566+0000) 2024-04-26T02:07:58.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:57 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:57.601+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.615394+0000 front 2024-04-26T02:07:06.615253+0000 (oldest deadline 2024-04-26T02:07:30.015848+0000) 2024-04-26T02:07:58.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:57 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:57.601+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:15.118484+0000 front 2024-04-26T02:07:15.118531+0000 (oldest deadline 2024-04-26T02:07:40.318854+0000) 2024-04-26T02:07:58.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:57 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:57.601+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:26.521730+0000 front 2024-04-26T02:07:26.521753+0000 (oldest deadline 2024-04-26T02:07:52.421703+0000) 2024-04-26T02:07:58.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:57 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:57.601+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:32.422457+0000 front 2024-04-26T02:07:32.422344+0000 (oldest deadline 2024-04-26T02:07:55.322131+0000) 2024-04-26T02:07:58.372 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:58 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:58.095+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.476432+0000 front 2024-04-26T02:06:50.476644+0000 (oldest deadline 2024-04-26T02:07:18.078167+0000) 2024-04-26T02:07:58.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:58 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:58.095+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.682370+0000 front 2024-04-26T02:06:59.682519+0000 (oldest deadline 2024-04-26T02:07:24.982614+0000) 2024-04-26T02:07:58.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:58 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:58.095+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:04.984189+0000 front 2024-04-26T02:07:04.983860+0000 (oldest deadline 2024-04-26T02:07:33.784534+0000) 2024-04-26T02:07:58.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:58 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:58.095+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:17.286445+0000 front 2024-04-26T02:07:17.286925+0000 (oldest deadline 2024-04-26T02:07:42.586519+0000) 2024-04-26T02:07:58.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:58 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:58.095+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:24.288859+0000 front 2024-04-26T02:07:24.288963+0000 (oldest deadline 2024-04-26T02:07:47.788514+0000) 2024-04-26T02:07:58.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:58 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:58.095+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:33.690239+0000 front 2024-04-26T02:07:33.690387+0000 (oldest deadline 2024-04-26T02:07:56.590009+0000) 2024-04-26T02:07:58.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:58 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:58.575+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:07:58.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:58 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:58.575+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.313401+0000 front 2024-04-26T02:07:00.313375+0000 (oldest deadline 2024-04-26T02:07:25.613566+0000) 2024-04-26T02:07:58.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:58 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:58.575+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.615394+0000 front 2024-04-26T02:07:06.615253+0000 (oldest deadline 2024-04-26T02:07:30.015848+0000) 2024-04-26T02:07:58.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:58 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:58.575+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:15.118484+0000 front 2024-04-26T02:07:15.118531+0000 (oldest deadline 2024-04-26T02:07:40.318854+0000) 2024-04-26T02:07:58.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:58 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:58.575+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:26.521730+0000 front 2024-04-26T02:07:26.521753+0000 (oldest deadline 2024-04-26T02:07:52.421703+0000) 2024-04-26T02:07:58.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:58 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:58.575+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:32.422457+0000 front 2024-04-26T02:07:32.422344+0000 (oldest deadline 2024-04-26T02:07:55.322131+0000) 2024-04-26T02:07:59.037 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:58 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:58.566+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.275620+0000 front 2024-04-26T02:06:59.275446+0000 (oldest deadline 2024-04-26T02:07:23.975481+0000) 2024-04-26T02:07:59.037 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:58 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:58.566+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.477628+0000 front 2024-04-26T02:07:08.477663+0000 (oldest deadline 2024-04-26T02:07:30.177791+0000) 2024-04-26T02:07:59.037 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:58 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:58.566+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:17.179691+0000 front 2024-04-26T02:07:17.179731+0000 (oldest deadline 2024-04-26T02:07:40.079515+0000) 2024-04-26T02:07:59.037 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:58 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:58.566+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:24.780551+0000 front 2024-04-26T02:07:24.780463+0000 (oldest deadline 2024-04-26T02:07:52.981329+0000) 2024-04-26T02:07:59.038 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:58 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8[51152]: 2024-04-26T02:07:58.566+0000 7f867d6e6700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:32.981848+0000 front 2024-04-26T02:07:32.981873+0000 (oldest deadline 2024-04-26T02:07:55.282029+0000) 2024-04-26T02:07:59.038 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:58 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:58.691+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:07:59.038 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:58 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:58.691+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.934525+0000 front 2024-04-26T02:06:59.934498+0000 (oldest deadline 2024-04-26T02:07:22.834583+0000) 2024-04-26T02:07:59.038 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:58 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:58.691+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.735339+0000 front 2024-04-26T02:07:08.735363+0000 (oldest deadline 2024-04-26T02:07:33.435537+0000) 2024-04-26T02:07:59.038 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:58 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:58.691+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:16.837132+0000 front 2024-04-26T02:07:16.837183+0000 (oldest deadline 2024-04-26T02:07:38.537028+0000) 2024-04-26T02:07:59.038 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:58 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:58.691+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:24.239082+0000 front 2024-04-26T02:07:24.239105+0000 (oldest deadline 2024-04-26T02:07:49.439890+0000) 2024-04-26T02:07:59.038 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:58 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:58.691+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:34.140899+0000 front 2024-04-26T02:07:34.141046+0000 (oldest deadline 2024-04-26T02:07:55.241143+0000) 2024-04-26T02:07:59.372 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:59 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:59.065+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.476432+0000 front 2024-04-26T02:06:50.476644+0000 (oldest deadline 2024-04-26T02:07:18.078167+0000) 2024-04-26T02:07:59.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:59 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:59.065+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.682370+0000 front 2024-04-26T02:06:59.682519+0000 (oldest deadline 2024-04-26T02:07:24.982614+0000) 2024-04-26T02:07:59.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:59 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:59.065+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:04.984189+0000 front 2024-04-26T02:07:04.983860+0000 (oldest deadline 2024-04-26T02:07:33.784534+0000) 2024-04-26T02:07:59.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:59 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:59.065+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:17.286445+0000 front 2024-04-26T02:07:17.286925+0000 (oldest deadline 2024-04-26T02:07:42.586519+0000) 2024-04-26T02:07:59.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:59 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:59.065+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:24.288859+0000 front 2024-04-26T02:07:24.288963+0000 (oldest deadline 2024-04-26T02:07:47.788514+0000) 2024-04-26T02:07:59.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:07:59 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:07:59.065+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:33.690239+0000 front 2024-04-26T02:07:33.690387+0000 (oldest deadline 2024-04-26T02:07:56.590009+0000) 2024-04-26T02:08:00.037 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:07:59 smithi172 podman[71414]: 2024-04-26 02:07:59.767630332 +0000 UTC m=+5.647977804 container died 39f6d303d0454147241ed16925f035f9e1afd3b2160a172eb8eaeb0f95e9fbce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416) 2024-04-26T02:08:00.038 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:59 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:59.701+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:08:00.038 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:59 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:59.701+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.934525+0000 front 2024-04-26T02:06:59.934498+0000 (oldest deadline 2024-04-26T02:07:22.834583+0000) 2024-04-26T02:08:00.038 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:59 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:59.701+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.735339+0000 front 2024-04-26T02:07:08.735363+0000 (oldest deadline 2024-04-26T02:07:33.435537+0000) 2024-04-26T02:08:00.038 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:59 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:59.701+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:16.837132+0000 front 2024-04-26T02:07:16.837183+0000 (oldest deadline 2024-04-26T02:07:38.537028+0000) 2024-04-26T02:08:00.038 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:59 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:59.701+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:24.239082+0000 front 2024-04-26T02:07:24.239105+0000 (oldest deadline 2024-04-26T02:07:49.439890+0000) 2024-04-26T02:08:00.038 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:07:59 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:07:59.701+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:34.140899+0000 front 2024-04-26T02:07:34.141046+0000 (oldest deadline 2024-04-26T02:07:55.241143+0000) 2024-04-26T02:08:00.096 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:59 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:59.611+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:08:00.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:59 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:59.611+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.313401+0000 front 2024-04-26T02:07:00.313375+0000 (oldest deadline 2024-04-26T02:07:25.613566+0000) 2024-04-26T02:08:00.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:59 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:59.611+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.615394+0000 front 2024-04-26T02:07:06.615253+0000 (oldest deadline 2024-04-26T02:07:30.015848+0000) 2024-04-26T02:08:00.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:59 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:59.611+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:15.118484+0000 front 2024-04-26T02:07:15.118531+0000 (oldest deadline 2024-04-26T02:07:40.318854+0000) 2024-04-26T02:08:00.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:59 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:59.611+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:26.521730+0000 front 2024-04-26T02:07:26.521753+0000 (oldest deadline 2024-04-26T02:07:52.421703+0000) 2024-04-26T02:08:00.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:07:59 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:07:59.611+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:32.422457+0000 front 2024-04-26T02:07:32.422344+0000 (oldest deadline 2024-04-26T02:07:55.322131+0000) 2024-04-26T02:08:00.372 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:00 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:08:00.070+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.476432+0000 front 2024-04-26T02:06:50.476644+0000 (oldest deadline 2024-04-26T02:07:18.078167+0000) 2024-04-26T02:08:00.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:00 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:08:00.070+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.682370+0000 front 2024-04-26T02:06:59.682519+0000 (oldest deadline 2024-04-26T02:07:24.982614+0000) 2024-04-26T02:08:00.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:00 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:08:00.070+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:04.984189+0000 front 2024-04-26T02:07:04.983860+0000 (oldest deadline 2024-04-26T02:07:33.784534+0000) 2024-04-26T02:08:00.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:00 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:08:00.070+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:17.286445+0000 front 2024-04-26T02:07:17.286925+0000 (oldest deadline 2024-04-26T02:07:42.586519+0000) 2024-04-26T02:08:00.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:00 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:08:00.070+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:24.288859+0000 front 2024-04-26T02:07:24.288963+0000 (oldest deadline 2024-04-26T02:07:47.788514+0000) 2024-04-26T02:08:00.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:00 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:08:00.070+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:33.690239+0000 front 2024-04-26T02:07:33.690387+0000 (oldest deadline 2024-04-26T02:07:56.590009+0000) 2024-04-26T02:08:00.403 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:08:00 smithi172 podman[71414]: 2024-04-26 02:08:00.145960888 +0000 UTC m=+6.026308358 container cleanup 39f6d303d0454147241ed16925f035f9e1afd3b2160a172eb8eaeb0f95e9fbce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-26T02:08:00.404 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:08:00 smithi172 bash[71414]: ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8 2024-04-26T02:08:00.721 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:08:00 smithi172 podman[71431]: 2024-04-26 02:08:00.405514511 +0000 UTC m=+0.631110234 container remove 39f6d303d0454147241ed16925f035f9e1afd3b2160a172eb8eaeb0f95e9fbce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True) 2024-04-26T02:08:01.024 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:00 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:00.730+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:08:01.024 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:00 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:00.730+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.934525+0000 front 2024-04-26T02:06:59.934498+0000 (oldest deadline 2024-04-26T02:07:22.834583+0000) 2024-04-26T02:08:01.024 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:00 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:00.730+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.735339+0000 front 2024-04-26T02:07:08.735363+0000 (oldest deadline 2024-04-26T02:07:33.435537+0000) 2024-04-26T02:08:01.024 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:00 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:00.730+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:16.837132+0000 front 2024-04-26T02:07:16.837183+0000 (oldest deadline 2024-04-26T02:07:38.537028+0000) 2024-04-26T02:08:01.024 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:00 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:00.730+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:24.239082+0000 front 2024-04-26T02:07:24.239105+0000 (oldest deadline 2024-04-26T02:07:49.439890+0000) 2024-04-26T02:08:01.024 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:00 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:00.730+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:34.140899+0000 front 2024-04-26T02:07:34.141046+0000 (oldest deadline 2024-04-26T02:07:55.241143+0000) 2024-04-26T02:08:01.024 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:08:00 smithi172 podman[71516]: 2024-04-26 02:08:00.62528411 +0000 UTC m=+0.018104847 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T02:08:01.024 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:08:00 smithi172 podman[71516]: 2024-04-26 02:08:00.731462295 +0000 UTC m=+0.124283070 container create de7df3d43262a24c1706dd3217b2cbcd3e19fd14565b4f0cc0c1314807f86929 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-26T02:08:01.024 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:08:00 smithi172 podman[71516]: 2024-04-26 02:08:00.959779288 +0000 UTC m=+0.352600027 container init de7df3d43262a24c1706dd3217b2cbcd3e19fd14565b4f0cc0c1314807f86929 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-26T02:08:01.025 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:08:00 smithi172 podman[71516]: 2024-04-26 02:08:00.963243985 +0000 UTC m=+0.356064721 container start de7df3d43262a24c1706dd3217b2cbcd3e19fd14565b4f0cc0c1314807f86929 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416) 2024-04-26T02:08:01.025 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:08:01 smithi172 podman[71516]: 2024-04-26 02:08:01.025561924 +0000 UTC m=+0.418382686 container attach de7df3d43262a24c1706dd3217b2cbcd3e19fd14565b4f0cc0c1314807f86929 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-26T02:08:01.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:00 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:00.655+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:08:01.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:00 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:00.655+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.313401+0000 front 2024-04-26T02:07:00.313375+0000 (oldest deadline 2024-04-26T02:07:25.613566+0000) 2024-04-26T02:08:01.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:00 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:00.655+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.615394+0000 front 2024-04-26T02:07:06.615253+0000 (oldest deadline 2024-04-26T02:07:30.015848+0000) 2024-04-26T02:08:01.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:00 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:00.655+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:15.118484+0000 front 2024-04-26T02:07:15.118531+0000 (oldest deadline 2024-04-26T02:07:40.318854+0000) 2024-04-26T02:08:01.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:00 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:00.655+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:26.521730+0000 front 2024-04-26T02:07:26.521753+0000 (oldest deadline 2024-04-26T02:07:52.421703+0000) 2024-04-26T02:08:01.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:00 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:00.655+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:32.422457+0000 front 2024-04-26T02:07:32.422344+0000 (oldest deadline 2024-04-26T02:07:55.322131+0000) 2024-04-26T02:08:01.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:01 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:08:01.059+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.476432+0000 front 2024-04-26T02:06:50.476644+0000 (oldest deadline 2024-04-26T02:07:18.078167+0000) 2024-04-26T02:08:01.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:01 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:08:01.059+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.682370+0000 front 2024-04-26T02:06:59.682519+0000 (oldest deadline 2024-04-26T02:07:24.982614+0000) 2024-04-26T02:08:01.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:01 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:08:01.059+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:04.984189+0000 front 2024-04-26T02:07:04.983860+0000 (oldest deadline 2024-04-26T02:07:33.784534+0000) 2024-04-26T02:08:01.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:01 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:08:01.059+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:17.286445+0000 front 2024-04-26T02:07:17.286925+0000 (oldest deadline 2024-04-26T02:07:42.586519+0000) 2024-04-26T02:08:01.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:01 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:08:01.059+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:24.288859+0000 front 2024-04-26T02:07:24.288963+0000 (oldest deadline 2024-04-26T02:07:47.788514+0000) 2024-04-26T02:08:01.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:01 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:08:01.059+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:33.690239+0000 front 2024-04-26T02:07:33.690387+0000 (oldest deadline 2024-04-26T02:07:56.590009+0000) 2024-04-26T02:08:01.561 INFO:journalctl@ceph.osd.8.smithi172.stdout:Apr 26 02:08:01 smithi172 podman[71543]: 2024-04-26 02:08:01.293419881 +0000 UTC m=+0.019953688 container died de7df3d43262a24c1706dd3217b2cbcd3e19fd14565b4f0cc0c1314807f86929 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-26T02:08:01.878 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:01 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:01.746+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:08:01.878 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:01 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:01.746+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.934525+0000 front 2024-04-26T02:06:59.934498+0000 (oldest deadline 2024-04-26T02:07:22.834583+0000) 2024-04-26T02:08:01.879 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:01 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:01.746+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.735339+0000 front 2024-04-26T02:07:08.735363+0000 (oldest deadline 2024-04-26T02:07:33.435537+0000) 2024-04-26T02:08:01.879 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:01 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:01.746+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:16.837132+0000 front 2024-04-26T02:07:16.837183+0000 (oldest deadline 2024-04-26T02:07:38.537028+0000) 2024-04-26T02:08:01.879 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:01 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:01.746+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:24.239082+0000 front 2024-04-26T02:07:24.239105+0000 (oldest deadline 2024-04-26T02:07:49.439890+0000) 2024-04-26T02:08:01.879 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:01 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:01.746+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:34.140899+0000 front 2024-04-26T02:07:34.141046+0000 (oldest deadline 2024-04-26T02:07:55.241143+0000) 2024-04-26T02:08:01.900 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0e49e64-036d-11ef-bc93-c7b262605968@osd.8.service' 2024-04-26T02:08:01.977 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:08:01.977 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-26T02:08:01.977 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-26T02:08:01.977 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-c0e49e64-036d-11ef-bc93-c7b262605968@osd.9 2024-04-26T02:08:02.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:01 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:01.645+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:08:02.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:01 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:01.645+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.313401+0000 front 2024-04-26T02:07:00.313375+0000 (oldest deadline 2024-04-26T02:07:25.613566+0000) 2024-04-26T02:08:02.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:01 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:01.645+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.615394+0000 front 2024-04-26T02:07:06.615253+0000 (oldest deadline 2024-04-26T02:07:30.015848+0000) 2024-04-26T02:08:02.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:01 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:01.645+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:15.118484+0000 front 2024-04-26T02:07:15.118531+0000 (oldest deadline 2024-04-26T02:07:40.318854+0000) 2024-04-26T02:08:02.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:01 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:01.645+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:26.521730+0000 front 2024-04-26T02:07:26.521753+0000 (oldest deadline 2024-04-26T02:07:52.421703+0000) 2024-04-26T02:08:02.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:01 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:01.645+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:32.422457+0000 front 2024-04-26T02:07:32.422344+0000 (oldest deadline 2024-04-26T02:07:55.322131+0000) 2024-04-26T02:08:02.372 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:02 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:08:02.020+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.476432+0000 front 2024-04-26T02:06:50.476644+0000 (oldest deadline 2024-04-26T02:07:18.078167+0000) 2024-04-26T02:08:02.372 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:02 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:08:02.020+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.682370+0000 front 2024-04-26T02:06:59.682519+0000 (oldest deadline 2024-04-26T02:07:24.982614+0000) 2024-04-26T02:08:02.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:02 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:08:02.020+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:04.984189+0000 front 2024-04-26T02:07:04.983860+0000 (oldest deadline 2024-04-26T02:07:33.784534+0000) 2024-04-26T02:08:02.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:02 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:08:02.021+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:17.286445+0000 front 2024-04-26T02:07:17.286925+0000 (oldest deadline 2024-04-26T02:07:42.586519+0000) 2024-04-26T02:08:02.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:02 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:08:02.021+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:24.288859+0000 front 2024-04-26T02:07:24.288963+0000 (oldest deadline 2024-04-26T02:07:47.788514+0000) 2024-04-26T02:08:02.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:02 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:08:02.021+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:33.690239+0000 front 2024-04-26T02:07:33.690387+0000 (oldest deadline 2024-04-26T02:07:56.590009+0000) 2024-04-26T02:08:02.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:02 smithi151 systemd[1]: Stopping Ceph osd.9 for c0e49e64-036d-11ef-bc93-c7b262605968... 2024-04-26T02:08:02.872 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:02 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:08:02.544+0000 7ffa9a600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:08:02.873 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:02 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:08:02.544+0000 7ffa9a600700 -1 osd.9 112 *** Got signal Terminated *** 2024-04-26T02:08:02.873 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:02 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:08:02.544+0000 7ffa9a600700 -1 osd.9 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T02:08:03.037 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:02 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:02.785+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:08:03.038 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:02 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:02.785+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.934525+0000 front 2024-04-26T02:06:59.934498+0000 (oldest deadline 2024-04-26T02:07:22.834583+0000) 2024-04-26T02:08:03.038 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:02 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:02.785+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.735339+0000 front 2024-04-26T02:07:08.735363+0000 (oldest deadline 2024-04-26T02:07:33.435537+0000) 2024-04-26T02:08:03.038 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:02 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:02.785+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:16.837132+0000 front 2024-04-26T02:07:16.837183+0000 (oldest deadline 2024-04-26T02:07:38.537028+0000) 2024-04-26T02:08:03.038 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:02 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:02.785+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:24.239082+0000 front 2024-04-26T02:07:24.239105+0000 (oldest deadline 2024-04-26T02:07:49.439890+0000) 2024-04-26T02:08:03.038 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:02 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:02.785+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:34.140899+0000 front 2024-04-26T02:07:34.141046+0000 (oldest deadline 2024-04-26T02:07:55.241143+0000) 2024-04-26T02:08:03.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:02 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:02.614+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:08:03.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:02 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:02.614+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.313401+0000 front 2024-04-26T02:07:00.313375+0000 (oldest deadline 2024-04-26T02:07:25.613566+0000) 2024-04-26T02:08:03.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:02 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:02.614+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.615394+0000 front 2024-04-26T02:07:06.615253+0000 (oldest deadline 2024-04-26T02:07:30.015848+0000) 2024-04-26T02:08:03.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:02 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:02.614+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:15.118484+0000 front 2024-04-26T02:07:15.118531+0000 (oldest deadline 2024-04-26T02:07:40.318854+0000) 2024-04-26T02:08:03.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:02 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:02.614+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:26.521730+0000 front 2024-04-26T02:07:26.521753+0000 (oldest deadline 2024-04-26T02:07:52.421703+0000) 2024-04-26T02:08:03.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:02 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:02.614+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:32.422457+0000 front 2024-04-26T02:07:32.422344+0000 (oldest deadline 2024-04-26T02:07:55.322131+0000) 2024-04-26T02:08:03.372 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:03 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:08:03.048+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.476432+0000 front 2024-04-26T02:06:50.476644+0000 (oldest deadline 2024-04-26T02:07:18.078167+0000) 2024-04-26T02:08:03.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:03 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:08:03.048+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.682370+0000 front 2024-04-26T02:06:59.682519+0000 (oldest deadline 2024-04-26T02:07:24.982614+0000) 2024-04-26T02:08:03.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:03 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:08:03.048+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:04.984189+0000 front 2024-04-26T02:07:04.983860+0000 (oldest deadline 2024-04-26T02:07:33.784534+0000) 2024-04-26T02:08:03.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:03 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:08:03.048+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:17.286445+0000 front 2024-04-26T02:07:17.286925+0000 (oldest deadline 2024-04-26T02:07:42.586519+0000) 2024-04-26T02:08:03.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:03 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:08:03.048+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:24.288859+0000 front 2024-04-26T02:07:24.288963+0000 (oldest deadline 2024-04-26T02:07:47.788514+0000) 2024-04-26T02:08:03.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:03 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:08:03.048+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:33.690239+0000 front 2024-04-26T02:07:33.690387+0000 (oldest deadline 2024-04-26T02:07:56.590009+0000) 2024-04-26T02:08:03.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:03 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:03.596+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:08:03.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:03 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:03.596+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.313401+0000 front 2024-04-26T02:07:00.313375+0000 (oldest deadline 2024-04-26T02:07:25.613566+0000) 2024-04-26T02:08:03.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:03 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:03.596+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.615394+0000 front 2024-04-26T02:07:06.615253+0000 (oldest deadline 2024-04-26T02:07:30.015848+0000) 2024-04-26T02:08:03.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:03 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:03.596+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:15.118484+0000 front 2024-04-26T02:07:15.118531+0000 (oldest deadline 2024-04-26T02:07:40.318854+0000) 2024-04-26T02:08:03.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:03 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:03.596+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:26.521730+0000 front 2024-04-26T02:07:26.521753+0000 (oldest deadline 2024-04-26T02:07:52.421703+0000) 2024-04-26T02:08:03.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:03 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:03.596+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:32.422457+0000 front 2024-04-26T02:07:32.422344+0000 (oldest deadline 2024-04-26T02:07:55.322131+0000) 2024-04-26T02:08:04.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:03 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:03.814+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:08:04.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:03 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:03.814+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.934525+0000 front 2024-04-26T02:06:59.934498+0000 (oldest deadline 2024-04-26T02:07:22.834583+0000) 2024-04-26T02:08:04.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:03 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:03.814+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.735339+0000 front 2024-04-26T02:07:08.735363+0000 (oldest deadline 2024-04-26T02:07:33.435537+0000) 2024-04-26T02:08:04.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:03 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:03.814+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:16.837132+0000 front 2024-04-26T02:07:16.837183+0000 (oldest deadline 2024-04-26T02:07:38.537028+0000) 2024-04-26T02:08:04.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:03 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:03.814+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:24.239082+0000 front 2024-04-26T02:07:24.239105+0000 (oldest deadline 2024-04-26T02:07:49.439890+0000) 2024-04-26T02:08:04.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:03 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:03.814+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:34.140899+0000 front 2024-04-26T02:07:34.141046+0000 (oldest deadline 2024-04-26T02:07:55.241143+0000) 2024-04-26T02:08:04.372 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:04 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:08:04.062+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.476432+0000 front 2024-04-26T02:06:50.476644+0000 (oldest deadline 2024-04-26T02:07:18.078167+0000) 2024-04-26T02:08:04.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:04 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:08:04.062+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.682370+0000 front 2024-04-26T02:06:59.682519+0000 (oldest deadline 2024-04-26T02:07:24.982614+0000) 2024-04-26T02:08:04.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:04 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:08:04.062+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:04.984189+0000 front 2024-04-26T02:07:04.983860+0000 (oldest deadline 2024-04-26T02:07:33.784534+0000) 2024-04-26T02:08:04.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:04 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:08:04.062+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:17.286445+0000 front 2024-04-26T02:07:17.286925+0000 (oldest deadline 2024-04-26T02:07:42.586519+0000) 2024-04-26T02:08:04.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:04 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:08:04.062+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:24.288859+0000 front 2024-04-26T02:07:24.288963+0000 (oldest deadline 2024-04-26T02:07:47.788514+0000) 2024-04-26T02:08:04.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:04 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:08:04.062+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:33.690239+0000 front 2024-04-26T02:07:33.690387+0000 (oldest deadline 2024-04-26T02:07:56.590009+0000) 2024-04-26T02:08:05.015 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:04 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:04.617+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:08:05.015 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:04 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:04.617+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.313401+0000 front 2024-04-26T02:07:00.313375+0000 (oldest deadline 2024-04-26T02:07:25.613566+0000) 2024-04-26T02:08:05.015 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:04 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:04.617+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.615394+0000 front 2024-04-26T02:07:06.615253+0000 (oldest deadline 2024-04-26T02:07:30.015848+0000) 2024-04-26T02:08:05.015 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:04 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:04.617+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:15.118484+0000 front 2024-04-26T02:07:15.118531+0000 (oldest deadline 2024-04-26T02:07:40.318854+0000) 2024-04-26T02:08:05.015 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:04 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:04.617+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:26.521730+0000 front 2024-04-26T02:07:26.521753+0000 (oldest deadline 2024-04-26T02:07:52.421703+0000) 2024-04-26T02:08:05.015 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:04 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:04.617+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:32.422457+0000 front 2024-04-26T02:07:32.422344+0000 (oldest deadline 2024-04-26T02:07:55.322131+0000) 2024-04-26T02:08:05.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:04 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:04.831+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:08:05.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:04 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:04.831+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.934525+0000 front 2024-04-26T02:06:59.934498+0000 (oldest deadline 2024-04-26T02:07:22.834583+0000) 2024-04-26T02:08:05.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:04 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:04.831+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.735339+0000 front 2024-04-26T02:07:08.735363+0000 (oldest deadline 2024-04-26T02:07:33.435537+0000) 2024-04-26T02:08:05.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:04 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:04.831+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:16.837132+0000 front 2024-04-26T02:07:16.837183+0000 (oldest deadline 2024-04-26T02:07:38.537028+0000) 2024-04-26T02:08:05.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:04 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:04.831+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:24.239082+0000 front 2024-04-26T02:07:24.239105+0000 (oldest deadline 2024-04-26T02:07:49.439890+0000) 2024-04-26T02:08:05.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:04 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:04.831+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:34.140899+0000 front 2024-04-26T02:07:34.141046+0000 (oldest deadline 2024-04-26T02:07:55.241143+0000) 2024-04-26T02:08:05.372 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:05 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:08:05.049+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.476432+0000 front 2024-04-26T02:06:50.476644+0000 (oldest deadline 2024-04-26T02:07:18.078167+0000) 2024-04-26T02:08:05.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:05 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:08:05.049+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.682370+0000 front 2024-04-26T02:06:59.682519+0000 (oldest deadline 2024-04-26T02:07:24.982614+0000) 2024-04-26T02:08:05.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:05 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:08:05.049+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:04.984189+0000 front 2024-04-26T02:07:04.983860+0000 (oldest deadline 2024-04-26T02:07:33.784534+0000) 2024-04-26T02:08:05.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:05 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:08:05.049+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:17.286445+0000 front 2024-04-26T02:07:17.286925+0000 (oldest deadline 2024-04-26T02:07:42.586519+0000) 2024-04-26T02:08:05.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:05 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:08:05.049+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:24.288859+0000 front 2024-04-26T02:07:24.288963+0000 (oldest deadline 2024-04-26T02:07:47.788514+0000) 2024-04-26T02:08:05.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:05 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:08:05.049+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:33.690239+0000 front 2024-04-26T02:07:33.690387+0000 (oldest deadline 2024-04-26T02:07:56.590009+0000) 2024-04-26T02:08:06.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:05 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:05.647+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:08:06.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:05 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:05.647+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.313401+0000 front 2024-04-26T02:07:00.313375+0000 (oldest deadline 2024-04-26T02:07:25.613566+0000) 2024-04-26T02:08:06.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:05 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:05.647+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.615394+0000 front 2024-04-26T02:07:06.615253+0000 (oldest deadline 2024-04-26T02:07:30.015848+0000) 2024-04-26T02:08:06.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:05 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:05.647+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:15.118484+0000 front 2024-04-26T02:07:15.118531+0000 (oldest deadline 2024-04-26T02:07:40.318854+0000) 2024-04-26T02:08:06.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:05 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:05.647+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:26.521730+0000 front 2024-04-26T02:07:26.521753+0000 (oldest deadline 2024-04-26T02:07:52.421703+0000) 2024-04-26T02:08:06.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:05 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:05.647+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:32.422457+0000 front 2024-04-26T02:07:32.422344+0000 (oldest deadline 2024-04-26T02:07:55.322131+0000) 2024-04-26T02:08:06.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:05 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:05.818+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:08:06.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:05 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:05.818+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.934525+0000 front 2024-04-26T02:06:59.934498+0000 (oldest deadline 2024-04-26T02:07:22.834583+0000) 2024-04-26T02:08:06.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:05 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:05.818+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.735339+0000 front 2024-04-26T02:07:08.735363+0000 (oldest deadline 2024-04-26T02:07:33.435537+0000) 2024-04-26T02:08:06.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:05 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:05.818+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:16.837132+0000 front 2024-04-26T02:07:16.837183+0000 (oldest deadline 2024-04-26T02:07:38.537028+0000) 2024-04-26T02:08:06.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:05 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:05.818+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:24.239082+0000 front 2024-04-26T02:07:24.239105+0000 (oldest deadline 2024-04-26T02:07:49.439890+0000) 2024-04-26T02:08:06.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:05 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:05.818+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:34.140899+0000 front 2024-04-26T02:07:34.141046+0000 (oldest deadline 2024-04-26T02:07:55.241143+0000) 2024-04-26T02:08:06.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:06 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:08:06.065+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.476432+0000 front 2024-04-26T02:06:50.476644+0000 (oldest deadline 2024-04-26T02:07:18.078167+0000) 2024-04-26T02:08:06.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:06 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:08:06.065+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.682370+0000 front 2024-04-26T02:06:59.682519+0000 (oldest deadline 2024-04-26T02:07:24.982614+0000) 2024-04-26T02:08:06.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:06 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:08:06.065+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:04.984189+0000 front 2024-04-26T02:07:04.983860+0000 (oldest deadline 2024-04-26T02:07:33.784534+0000) 2024-04-26T02:08:06.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:06 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:08:06.065+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:17.286445+0000 front 2024-04-26T02:07:17.286925+0000 (oldest deadline 2024-04-26T02:07:42.586519+0000) 2024-04-26T02:08:06.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:06 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:08:06.065+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:24.288859+0000 front 2024-04-26T02:07:24.288963+0000 (oldest deadline 2024-04-26T02:07:47.788514+0000) 2024-04-26T02:08:06.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:06 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:08:06.065+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:33.690239+0000 front 2024-04-26T02:07:33.690387+0000 (oldest deadline 2024-04-26T02:07:56.590009+0000) 2024-04-26T02:08:06.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:06 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:08:06.065+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6822 osd.6 since back 2024-04-26T02:07:41.792900+0000 front 2024-04-26T02:07:41.792998+0000 (oldest deadline 2024-04-26T02:08:05.892660+0000) 2024-04-26T02:08:07.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:06 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:06.664+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:08:07.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:06 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:06.664+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.313401+0000 front 2024-04-26T02:07:00.313375+0000 (oldest deadline 2024-04-26T02:07:25.613566+0000) 2024-04-26T02:08:07.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:06 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:06.664+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.615394+0000 front 2024-04-26T02:07:06.615253+0000 (oldest deadline 2024-04-26T02:07:30.015848+0000) 2024-04-26T02:08:07.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:06 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:06.664+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:15.118484+0000 front 2024-04-26T02:07:15.118531+0000 (oldest deadline 2024-04-26T02:07:40.318854+0000) 2024-04-26T02:08:07.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:06 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:06.664+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:26.521730+0000 front 2024-04-26T02:07:26.521753+0000 (oldest deadline 2024-04-26T02:07:52.421703+0000) 2024-04-26T02:08:07.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:06 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:06.664+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:32.422457+0000 front 2024-04-26T02:07:32.422344+0000 (oldest deadline 2024-04-26T02:07:55.322131+0000) 2024-04-26T02:08:07.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:06 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:06.865+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:08:07.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:06 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:06.865+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.934525+0000 front 2024-04-26T02:06:59.934498+0000 (oldest deadline 2024-04-26T02:07:22.834583+0000) 2024-04-26T02:08:07.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:06 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:06.865+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.735339+0000 front 2024-04-26T02:07:08.735363+0000 (oldest deadline 2024-04-26T02:07:33.435537+0000) 2024-04-26T02:08:07.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:06 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:06.865+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:16.837132+0000 front 2024-04-26T02:07:16.837183+0000 (oldest deadline 2024-04-26T02:07:38.537028+0000) 2024-04-26T02:08:07.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:06 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:06.865+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:24.239082+0000 front 2024-04-26T02:07:24.239105+0000 (oldest deadline 2024-04-26T02:07:49.439890+0000) 2024-04-26T02:08:07.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:06 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:06.865+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:34.140899+0000 front 2024-04-26T02:07:34.141046+0000 (oldest deadline 2024-04-26T02:07:55.241143+0000) 2024-04-26T02:08:07.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:07 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:08:07.110+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:50.476432+0000 front 2024-04-26T02:06:50.476644+0000 (oldest deadline 2024-04-26T02:07:18.078167+0000) 2024-04-26T02:08:07.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:07 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:08:07.110+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.682370+0000 front 2024-04-26T02:06:59.682519+0000 (oldest deadline 2024-04-26T02:07:24.982614+0000) 2024-04-26T02:08:07.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:07 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:08:07.110+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:04.984189+0000 front 2024-04-26T02:07:04.983860+0000 (oldest deadline 2024-04-26T02:07:33.784534+0000) 2024-04-26T02:08:07.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:07 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:08:07.110+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:17.286445+0000 front 2024-04-26T02:07:17.286925+0000 (oldest deadline 2024-04-26T02:07:42.586519+0000) 2024-04-26T02:08:07.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:07 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:08:07.110+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:24.288859+0000 front 2024-04-26T02:07:24.288963+0000 (oldest deadline 2024-04-26T02:07:47.788514+0000) 2024-04-26T02:08:07.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:07 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:08:07.110+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:33.690239+0000 front 2024-04-26T02:07:33.690387+0000 (oldest deadline 2024-04-26T02:07:56.590009+0000) 2024-04-26T02:08:07.373 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:07 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9[62816]: 2024-04-26T02:08:07.110+0000 7ffa90e00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.151:6822 osd.6 since back 2024-04-26T02:07:41.792900+0000 front 2024-04-26T02:07:41.792998+0000 (oldest deadline 2024-04-26T02:08:05.892660+0000) 2024-04-26T02:08:08.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:07 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:07.669+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:08:08.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:07 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:07.669+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.313401+0000 front 2024-04-26T02:07:00.313375+0000 (oldest deadline 2024-04-26T02:07:25.613566+0000) 2024-04-26T02:08:08.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:07 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:07.669+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.615394+0000 front 2024-04-26T02:07:06.615253+0000 (oldest deadline 2024-04-26T02:07:30.015848+0000) 2024-04-26T02:08:08.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:07 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:07.669+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:15.118484+0000 front 2024-04-26T02:07:15.118531+0000 (oldest deadline 2024-04-26T02:07:40.318854+0000) 2024-04-26T02:08:08.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:07 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:07.669+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:26.521730+0000 front 2024-04-26T02:07:26.521753+0000 (oldest deadline 2024-04-26T02:07:52.421703+0000) 2024-04-26T02:08:08.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:07 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:07.669+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:32.422457+0000 front 2024-04-26T02:07:32.422344+0000 (oldest deadline 2024-04-26T02:07:55.322131+0000) 2024-04-26T02:08:08.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:07 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:07.669+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6822 osd.6 since back 2024-04-26T02:07:42.323899+0000 front 2024-04-26T02:07:42.324063+0000 (oldest deadline 2024-04-26T02:08:07.623626+0000) 2024-04-26T02:08:08.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:07 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:07.830+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:08:08.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:07 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:07.830+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.934525+0000 front 2024-04-26T02:06:59.934498+0000 (oldest deadline 2024-04-26T02:07:22.834583+0000) 2024-04-26T02:08:08.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:07 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:07.830+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.735339+0000 front 2024-04-26T02:07:08.735363+0000 (oldest deadline 2024-04-26T02:07:33.435537+0000) 2024-04-26T02:08:08.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:07 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:07.830+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:16.837132+0000 front 2024-04-26T02:07:16.837183+0000 (oldest deadline 2024-04-26T02:07:38.537028+0000) 2024-04-26T02:08:08.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:07 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:07.830+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:24.239082+0000 front 2024-04-26T02:07:24.239105+0000 (oldest deadline 2024-04-26T02:07:49.439890+0000) 2024-04-26T02:08:08.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:07 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:07.830+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:34.140899+0000 front 2024-04-26T02:07:34.141046+0000 (oldest deadline 2024-04-26T02:07:55.241143+0000) 2024-04-26T02:08:08.351 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:08 smithi151 podman[120417]: 2024-04-26 02:08:08.014864569 +0000 UTC m=+5.612733656 container died 6a09242a11d75899dd5921e8cce51dc1d994f495d4aee2edab9cd5c3517b10fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-26T02:08:08.607 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:08 smithi151 podman[120417]: 2024-04-26 02:08:08.589483926 +0000 UTC m=+6.187353023 container cleanup 6a09242a11d75899dd5921e8cce51dc1d994f495d4aee2edab9cd5c3517b10fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416) 2024-04-26T02:08:08.607 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:08 smithi151 bash[120417]: ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9 2024-04-26T02:08:09.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:08 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:08.623+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:08:09.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:08 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:08.623+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.313401+0000 front 2024-04-26T02:07:00.313375+0000 (oldest deadline 2024-04-26T02:07:25.613566+0000) 2024-04-26T02:08:09.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:08 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:08.623+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.615394+0000 front 2024-04-26T02:07:06.615253+0000 (oldest deadline 2024-04-26T02:07:30.015848+0000) 2024-04-26T02:08:09.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:08 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:08.623+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:15.118484+0000 front 2024-04-26T02:07:15.118531+0000 (oldest deadline 2024-04-26T02:07:40.318854+0000) 2024-04-26T02:08:09.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:08 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:08.623+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:26.521730+0000 front 2024-04-26T02:07:26.521753+0000 (oldest deadline 2024-04-26T02:07:52.421703+0000) 2024-04-26T02:08:09.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:08 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:08.623+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:32.422457+0000 front 2024-04-26T02:07:32.422344+0000 (oldest deadline 2024-04-26T02:07:55.322131+0000) 2024-04-26T02:08:09.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:08 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:08.623+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6822 osd.6 since back 2024-04-26T02:07:42.323899+0000 front 2024-04-26T02:07:42.324063+0000 (oldest deadline 2024-04-26T02:08:07.623626+0000) 2024-04-26T02:08:09.122 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:08 smithi151 podman[120435]: 2024-04-26 02:08:08.866528717 +0000 UTC m=+0.836438675 container remove 6a09242a11d75899dd5921e8cce51dc1d994f495d4aee2edab9cd5c3517b10fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T02:08:09.189 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:08 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:08.854+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:08:09.189 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:08 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:08.854+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.934525+0000 front 2024-04-26T02:06:59.934498+0000 (oldest deadline 2024-04-26T02:07:22.834583+0000) 2024-04-26T02:08:09.189 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:08 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:08.854+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.735339+0000 front 2024-04-26T02:07:08.735363+0000 (oldest deadline 2024-04-26T02:07:33.435537+0000) 2024-04-26T02:08:09.189 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:08 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:08.854+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:16.837132+0000 front 2024-04-26T02:07:16.837183+0000 (oldest deadline 2024-04-26T02:07:38.537028+0000) 2024-04-26T02:08:09.189 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:08 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:08.854+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:24.239082+0000 front 2024-04-26T02:07:24.239105+0000 (oldest deadline 2024-04-26T02:07:49.439890+0000) 2024-04-26T02:08:09.189 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:08 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:08.854+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:34.140899+0000 front 2024-04-26T02:07:34.141046+0000 (oldest deadline 2024-04-26T02:07:55.241143+0000) 2024-04-26T02:08:09.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:09 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:09.590+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:08:09.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:09 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:09.591+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.313401+0000 front 2024-04-26T02:07:00.313375+0000 (oldest deadline 2024-04-26T02:07:25.613566+0000) 2024-04-26T02:08:09.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:09 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:09.591+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.615394+0000 front 2024-04-26T02:07:06.615253+0000 (oldest deadline 2024-04-26T02:07:30.015848+0000) 2024-04-26T02:08:09.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:09 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:09.591+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:15.118484+0000 front 2024-04-26T02:07:15.118531+0000 (oldest deadline 2024-04-26T02:07:40.318854+0000) 2024-04-26T02:08:09.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:09 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:09.591+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:26.521730+0000 front 2024-04-26T02:07:26.521753+0000 (oldest deadline 2024-04-26T02:07:52.421703+0000) 2024-04-26T02:08:09.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:09 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:09.591+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:32.422457+0000 front 2024-04-26T02:07:32.422344+0000 (oldest deadline 2024-04-26T02:07:55.322131+0000) 2024-04-26T02:08:09.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:09 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:09.591+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6822 osd.6 since back 2024-04-26T02:07:42.323899+0000 front 2024-04-26T02:07:42.324063+0000 (oldest deadline 2024-04-26T02:08:07.623626+0000) 2024-04-26T02:08:09.987 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:09 smithi151 podman[120522]: 2024-04-26 02:08:09.565585192 +0000 UTC m=+0.033576274 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T02:08:09.987 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:09 smithi151 podman[120522]: 2024-04-26 02:08:09.68437293 +0000 UTC m=+0.152364049 container create 81b311b684affdde68ab0117eda605030bc373109312ef4693671edd2312edc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T02:08:09.988 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:09 smithi151 podman[120522]: 2024-04-26 02:08:09.988407556 +0000 UTC m=+0.456398681 container init 81b311b684affdde68ab0117eda605030bc373109312ef4693671edd2312edc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-26T02:08:10.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:09 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:09.834+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:08:10.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:09 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:09.834+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.934525+0000 front 2024-04-26T02:06:59.934498+0000 (oldest deadline 2024-04-26T02:07:22.834583+0000) 2024-04-26T02:08:10.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:09 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:09.834+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.735339+0000 front 2024-04-26T02:07:08.735363+0000 (oldest deadline 2024-04-26T02:07:33.435537+0000) 2024-04-26T02:08:10.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:09 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:09.834+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:16.837132+0000 front 2024-04-26T02:07:16.837183+0000 (oldest deadline 2024-04-26T02:07:38.537028+0000) 2024-04-26T02:08:10.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:09 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:09.834+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:24.239082+0000 front 2024-04-26T02:07:24.239105+0000 (oldest deadline 2024-04-26T02:07:49.439890+0000) 2024-04-26T02:08:10.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:09 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:09.834+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:34.140899+0000 front 2024-04-26T02:07:34.141046+0000 (oldest deadline 2024-04-26T02:07:55.241143+0000) 2024-04-26T02:08:10.372 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:09 smithi151 podman[120522]: 2024-04-26 02:08:09.999401997 +0000 UTC m=+0.467393123 container start 81b311b684affdde68ab0117eda605030bc373109312ef4693671edd2312edc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5) 2024-04-26T02:08:10.372 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:10 smithi151 podman[120522]: 2024-04-26 02:08:10.07892192 +0000 UTC m=+0.546913072 container attach 81b311b684affdde68ab0117eda605030bc373109312ef4693671edd2312edc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-26T02:08:10.815 INFO:journalctl@ceph.osd.9.smithi151.stdout:Apr 26 02:08:10 smithi151 podman[120522]: 2024-04-26 02:08:10.464250092 +0000 UTC m=+0.932241192 container died 81b311b684affdde68ab0117eda605030bc373109312ef4693671edd2312edc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-26T02:08:11.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:10 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:10.628+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:08:11.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:10 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:10.628+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.313401+0000 front 2024-04-26T02:07:00.313375+0000 (oldest deadline 2024-04-26T02:07:25.613566+0000) 2024-04-26T02:08:11.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:10 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:10.628+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.615394+0000 front 2024-04-26T02:07:06.615253+0000 (oldest deadline 2024-04-26T02:07:30.015848+0000) 2024-04-26T02:08:11.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:10 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:10.628+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:15.118484+0000 front 2024-04-26T02:07:15.118531+0000 (oldest deadline 2024-04-26T02:07:40.318854+0000) 2024-04-26T02:08:11.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:10 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:10.628+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:26.521730+0000 front 2024-04-26T02:07:26.521753+0000 (oldest deadline 2024-04-26T02:07:52.421703+0000) 2024-04-26T02:08:11.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:10 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:10.628+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:32.422457+0000 front 2024-04-26T02:07:32.422344+0000 (oldest deadline 2024-04-26T02:07:55.322131+0000) 2024-04-26T02:08:11.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:10 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:10.628+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6822 osd.6 since back 2024-04-26T02:07:42.323899+0000 front 2024-04-26T02:07:42.324063+0000 (oldest deadline 2024-04-26T02:08:07.623626+0000) 2024-04-26T02:08:11.229 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0e49e64-036d-11ef-bc93-c7b262605968@osd.9.service' 2024-04-26T02:08:11.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:10 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:10.806+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:08:11.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:10 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:10.806+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.934525+0000 front 2024-04-26T02:06:59.934498+0000 (oldest deadline 2024-04-26T02:07:22.834583+0000) 2024-04-26T02:08:11.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:10 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:10.806+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.735339+0000 front 2024-04-26T02:07:08.735363+0000 (oldest deadline 2024-04-26T02:07:33.435537+0000) 2024-04-26T02:08:11.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:10 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:10.806+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:16.837132+0000 front 2024-04-26T02:07:16.837183+0000 (oldest deadline 2024-04-26T02:07:38.537028+0000) 2024-04-26T02:08:11.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:10 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:10.806+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:24.239082+0000 front 2024-04-26T02:07:24.239105+0000 (oldest deadline 2024-04-26T02:07:49.439890+0000) 2024-04-26T02:08:11.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:10 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:10.806+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:34.140899+0000 front 2024-04-26T02:07:34.141046+0000 (oldest deadline 2024-04-26T02:07:55.241143+0000) 2024-04-26T02:08:11.333 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:08:11.333 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-26T02:08:11.333 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-26T02:08:11.333 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-c0e49e64-036d-11ef-bc93-c7b262605968@osd.10 2024-04-26T02:08:11.634 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:11 smithi165 systemd[1]: Stopping Ceph osd.10 for c0e49e64-036d-11ef-bc93-c7b262605968... 2024-04-26T02:08:12.096 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:11 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:11.635+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:08:12.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:11 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:11.635+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.313401+0000 front 2024-04-26T02:07:00.313375+0000 (oldest deadline 2024-04-26T02:07:25.613566+0000) 2024-04-26T02:08:12.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:11 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:11.635+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.615394+0000 front 2024-04-26T02:07:06.615253+0000 (oldest deadline 2024-04-26T02:07:30.015848+0000) 2024-04-26T02:08:12.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:11 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:11.635+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:15.118484+0000 front 2024-04-26T02:07:15.118531+0000 (oldest deadline 2024-04-26T02:07:40.318854+0000) 2024-04-26T02:08:12.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:11 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:11.635+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:26.521730+0000 front 2024-04-26T02:07:26.521753+0000 (oldest deadline 2024-04-26T02:07:52.421703+0000) 2024-04-26T02:08:12.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:11 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:11.635+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:32.422457+0000 front 2024-04-26T02:07:32.422344+0000 (oldest deadline 2024-04-26T02:07:55.322131+0000) 2024-04-26T02:08:12.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:11 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:11.635+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6822 osd.6 since back 2024-04-26T02:07:42.323899+0000 front 2024-04-26T02:07:42.324063+0000 (oldest deadline 2024-04-26T02:08:07.623626+0000) 2024-04-26T02:08:12.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:11 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:11.656+0000 7fae1c441700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:08:12.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:11 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:11.656+0000 7fae1c441700 -1 osd.10 112 *** Got signal Terminated *** 2024-04-26T02:08:12.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:11 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:11.656+0000 7fae1c441700 -1 osd.10 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T02:08:12.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:11 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:11.846+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:08:12.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:11 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:11.846+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.934525+0000 front 2024-04-26T02:06:59.934498+0000 (oldest deadline 2024-04-26T02:07:22.834583+0000) 2024-04-26T02:08:12.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:11 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:11.846+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.735339+0000 front 2024-04-26T02:07:08.735363+0000 (oldest deadline 2024-04-26T02:07:33.435537+0000) 2024-04-26T02:08:12.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:11 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:11.846+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:16.837132+0000 front 2024-04-26T02:07:16.837183+0000 (oldest deadline 2024-04-26T02:07:38.537028+0000) 2024-04-26T02:08:12.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:11 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:11.846+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:24.239082+0000 front 2024-04-26T02:07:24.239105+0000 (oldest deadline 2024-04-26T02:07:49.439890+0000) 2024-04-26T02:08:12.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:11 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:11.846+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:34.140899+0000 front 2024-04-26T02:07:34.141046+0000 (oldest deadline 2024-04-26T02:07:55.241143+0000) 2024-04-26T02:08:12.999 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:12 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:12.659+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:08:12.999 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:12 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:12.659+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.313401+0000 front 2024-04-26T02:07:00.313375+0000 (oldest deadline 2024-04-26T02:07:25.613566+0000) 2024-04-26T02:08:12.999 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:12 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:12.659+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.615394+0000 front 2024-04-26T02:07:06.615253+0000 (oldest deadline 2024-04-26T02:07:30.015848+0000) 2024-04-26T02:08:12.999 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:12 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:12.659+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:15.118484+0000 front 2024-04-26T02:07:15.118531+0000 (oldest deadline 2024-04-26T02:07:40.318854+0000) 2024-04-26T02:08:12.999 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:12 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:12.659+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:26.521730+0000 front 2024-04-26T02:07:26.521753+0000 (oldest deadline 2024-04-26T02:07:52.421703+0000) 2024-04-26T02:08:12.999 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:12 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:12.659+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:32.422457+0000 front 2024-04-26T02:07:32.422344+0000 (oldest deadline 2024-04-26T02:07:55.322131+0000) 2024-04-26T02:08:12.999 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:12 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:12.659+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6822 osd.6 since back 2024-04-26T02:07:42.323899+0000 front 2024-04-26T02:07:42.324063+0000 (oldest deadline 2024-04-26T02:08:07.623626+0000) 2024-04-26T02:08:13.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:12 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:12.814+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:08:13.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:12 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:12.814+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.934525+0000 front 2024-04-26T02:06:59.934498+0000 (oldest deadline 2024-04-26T02:07:22.834583+0000) 2024-04-26T02:08:13.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:12 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:12.814+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.735339+0000 front 2024-04-26T02:07:08.735363+0000 (oldest deadline 2024-04-26T02:07:33.435537+0000) 2024-04-26T02:08:13.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:12 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:12.814+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:16.837132+0000 front 2024-04-26T02:07:16.837183+0000 (oldest deadline 2024-04-26T02:07:38.537028+0000) 2024-04-26T02:08:13.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:12 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:12.814+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:24.239082+0000 front 2024-04-26T02:07:24.239105+0000 (oldest deadline 2024-04-26T02:07:49.439890+0000) 2024-04-26T02:08:13.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:12 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:12.814+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:34.140899+0000 front 2024-04-26T02:07:34.141046+0000 (oldest deadline 2024-04-26T02:07:55.241143+0000) 2024-04-26T02:08:14.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:13 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:13.699+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:08:14.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:13 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:13.699+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.313401+0000 front 2024-04-26T02:07:00.313375+0000 (oldest deadline 2024-04-26T02:07:25.613566+0000) 2024-04-26T02:08:14.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:13 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:13.699+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.615394+0000 front 2024-04-26T02:07:06.615253+0000 (oldest deadline 2024-04-26T02:07:30.015848+0000) 2024-04-26T02:08:14.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:13 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:13.699+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:15.118484+0000 front 2024-04-26T02:07:15.118531+0000 (oldest deadline 2024-04-26T02:07:40.318854+0000) 2024-04-26T02:08:14.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:13 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:13.699+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:26.521730+0000 front 2024-04-26T02:07:26.521753+0000 (oldest deadline 2024-04-26T02:07:52.421703+0000) 2024-04-26T02:08:14.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:13 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:13.699+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:32.422457+0000 front 2024-04-26T02:07:32.422344+0000 (oldest deadline 2024-04-26T02:07:55.322131+0000) 2024-04-26T02:08:14.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:13 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:13.699+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6822 osd.6 since back 2024-04-26T02:07:42.323899+0000 front 2024-04-26T02:07:42.324063+0000 (oldest deadline 2024-04-26T02:08:07.623626+0000) 2024-04-26T02:08:14.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:13 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:13.861+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:08:14.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:13 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:13.861+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.934525+0000 front 2024-04-26T02:06:59.934498+0000 (oldest deadline 2024-04-26T02:07:22.834583+0000) 2024-04-26T02:08:14.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:13 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:13.861+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.735339+0000 front 2024-04-26T02:07:08.735363+0000 (oldest deadline 2024-04-26T02:07:33.435537+0000) 2024-04-26T02:08:14.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:13 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:13.861+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:16.837132+0000 front 2024-04-26T02:07:16.837183+0000 (oldest deadline 2024-04-26T02:07:38.537028+0000) 2024-04-26T02:08:14.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:13 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:13.861+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:24.239082+0000 front 2024-04-26T02:07:24.239105+0000 (oldest deadline 2024-04-26T02:07:49.439890+0000) 2024-04-26T02:08:14.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:13 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:13.861+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:34.140899+0000 front 2024-04-26T02:07:34.141046+0000 (oldest deadline 2024-04-26T02:07:55.241143+0000) 2024-04-26T02:08:15.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:14 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:14.742+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:08:15.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:14 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:14.742+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.313401+0000 front 2024-04-26T02:07:00.313375+0000 (oldest deadline 2024-04-26T02:07:25.613566+0000) 2024-04-26T02:08:15.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:14 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:14.742+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.615394+0000 front 2024-04-26T02:07:06.615253+0000 (oldest deadline 2024-04-26T02:07:30.015848+0000) 2024-04-26T02:08:15.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:14 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:14.742+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:15.118484+0000 front 2024-04-26T02:07:15.118531+0000 (oldest deadline 2024-04-26T02:07:40.318854+0000) 2024-04-26T02:08:15.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:14 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:14.742+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:26.521730+0000 front 2024-04-26T02:07:26.521753+0000 (oldest deadline 2024-04-26T02:07:52.421703+0000) 2024-04-26T02:08:15.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:14 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:14.742+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:32.422457+0000 front 2024-04-26T02:07:32.422344+0000 (oldest deadline 2024-04-26T02:07:55.322131+0000) 2024-04-26T02:08:15.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:14 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:14.742+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6822 osd.6 since back 2024-04-26T02:07:42.323899+0000 front 2024-04-26T02:07:42.324063+0000 (oldest deadline 2024-04-26T02:08:07.623626+0000) 2024-04-26T02:08:15.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:14 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:14.890+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:08:15.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:14 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:14.890+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.934525+0000 front 2024-04-26T02:06:59.934498+0000 (oldest deadline 2024-04-26T02:07:22.834583+0000) 2024-04-26T02:08:15.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:14 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:14.890+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.735339+0000 front 2024-04-26T02:07:08.735363+0000 (oldest deadline 2024-04-26T02:07:33.435537+0000) 2024-04-26T02:08:15.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:14 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:14.890+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:16.837132+0000 front 2024-04-26T02:07:16.837183+0000 (oldest deadline 2024-04-26T02:07:38.537028+0000) 2024-04-26T02:08:15.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:14 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:14.890+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:24.239082+0000 front 2024-04-26T02:07:24.239105+0000 (oldest deadline 2024-04-26T02:07:49.439890+0000) 2024-04-26T02:08:15.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:14 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:14.891+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:34.140899+0000 front 2024-04-26T02:07:34.141046+0000 (oldest deadline 2024-04-26T02:07:55.241143+0000) 2024-04-26T02:08:15.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:14 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:14.891+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6820 osd.7 since back 2024-04-26T02:07:50.244743+0000 front 2024-04-26T02:07:50.244756+0000 (oldest deadline 2024-04-26T02:08:14.344846+0000) 2024-04-26T02:08:16.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:15 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:15.728+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.111292+0000 front 2024-04-26T02:06:51.111338+0000 (oldest deadline 2024-04-26T02:07:13.411089+0000) 2024-04-26T02:08:16.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:15 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:15.728+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:07:00.313401+0000 front 2024-04-26T02:07:00.313375+0000 (oldest deadline 2024-04-26T02:07:25.613566+0000) 2024-04-26T02:08:16.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:15 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:15.728+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:06.615394+0000 front 2024-04-26T02:07:06.615253+0000 (oldest deadline 2024-04-26T02:07:30.015848+0000) 2024-04-26T02:08:16.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:15 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:15.728+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:15.118484+0000 front 2024-04-26T02:07:15.118531+0000 (oldest deadline 2024-04-26T02:07:40.318854+0000) 2024-04-26T02:08:16.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:15 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:15.728+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:26.521730+0000 front 2024-04-26T02:07:26.521753+0000 (oldest deadline 2024-04-26T02:07:52.421703+0000) 2024-04-26T02:08:16.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:15 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:15.728+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:32.422457+0000 front 2024-04-26T02:07:32.422344+0000 (oldest deadline 2024-04-26T02:07:55.322131+0000) 2024-04-26T02:08:16.097 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:15 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10[57975]: 2024-04-26T02:08:15.728+0000 7fae18a35700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.151:6822 osd.6 since back 2024-04-26T02:07:42.323899+0000 front 2024-04-26T02:07:42.324063+0000 (oldest deadline 2024-04-26T02:08:07.623626+0000) 2024-04-26T02:08:16.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:15 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:15.893+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:08:16.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:15 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:15.893+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.934525+0000 front 2024-04-26T02:06:59.934498+0000 (oldest deadline 2024-04-26T02:07:22.834583+0000) 2024-04-26T02:08:16.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:15 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:15.893+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.735339+0000 front 2024-04-26T02:07:08.735363+0000 (oldest deadline 2024-04-26T02:07:33.435537+0000) 2024-04-26T02:08:16.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:15 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:15.893+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:16.837132+0000 front 2024-04-26T02:07:16.837183+0000 (oldest deadline 2024-04-26T02:07:38.537028+0000) 2024-04-26T02:08:16.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:15 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:15.893+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:24.239082+0000 front 2024-04-26T02:07:24.239105+0000 (oldest deadline 2024-04-26T02:07:49.439890+0000) 2024-04-26T02:08:16.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:15 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:15.893+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:34.140899+0000 front 2024-04-26T02:07:34.141046+0000 (oldest deadline 2024-04-26T02:07:55.241143+0000) 2024-04-26T02:08:16.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:15 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:15.893+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6820 osd.7 since back 2024-04-26T02:07:50.244743+0000 front 2024-04-26T02:07:50.244756+0000 (oldest deadline 2024-04-26T02:08:14.344846+0000) 2024-04-26T02:08:17.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:16 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:16.912+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:08:17.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:16 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:16.912+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.934525+0000 front 2024-04-26T02:06:59.934498+0000 (oldest deadline 2024-04-26T02:07:22.834583+0000) 2024-04-26T02:08:17.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:16 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:16.912+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.735339+0000 front 2024-04-26T02:07:08.735363+0000 (oldest deadline 2024-04-26T02:07:33.435537+0000) 2024-04-26T02:08:17.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:16 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:16.912+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:16.837132+0000 front 2024-04-26T02:07:16.837183+0000 (oldest deadline 2024-04-26T02:07:38.537028+0000) 2024-04-26T02:08:17.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:16 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:16.912+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:24.239082+0000 front 2024-04-26T02:07:24.239105+0000 (oldest deadline 2024-04-26T02:07:49.439890+0000) 2024-04-26T02:08:17.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:16 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:16.912+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:34.140899+0000 front 2024-04-26T02:07:34.141046+0000 (oldest deadline 2024-04-26T02:07:55.241143+0000) 2024-04-26T02:08:17.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:16 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:16.912+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6820 osd.7 since back 2024-04-26T02:07:50.244743+0000 front 2024-04-26T02:07:50.244756+0000 (oldest deadline 2024-04-26T02:08:14.344846+0000) 2024-04-26T02:08:17.452 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:17 smithi165 podman[70617]: 2024-04-26 02:08:17.135177705 +0000 UTC m=+5.603249201 container died 6eee6b8b80d7fde4ba3ba0869b6496e244602a486c22938e6e39ebe8e7602400 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-26T02:08:17.846 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:17 smithi165 podman[70617]: 2024-04-26 02:08:17.453963402 +0000 UTC m=+5.922034908 container cleanup 6eee6b8b80d7fde4ba3ba0869b6496e244602a486c22938e6e39ebe8e7602400 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-26T02:08:17.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:17 smithi165 bash[70617]: ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10 2024-04-26T02:08:17.847 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:17 smithi165 podman[70647]: 2024-04-26 02:08:17.604999955 +0000 UTC m=+0.463070480 container remove 6eee6b8b80d7fde4ba3ba0869b6496e244602a486c22938e6e39ebe8e7602400 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-26T02:08:18.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:17 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:17.915+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:08:18.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:17 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:17.915+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.934525+0000 front 2024-04-26T02:06:59.934498+0000 (oldest deadline 2024-04-26T02:07:22.834583+0000) 2024-04-26T02:08:18.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:17 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:17.915+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.735339+0000 front 2024-04-26T02:07:08.735363+0000 (oldest deadline 2024-04-26T02:07:33.435537+0000) 2024-04-26T02:08:18.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:17 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:17.915+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:16.837132+0000 front 2024-04-26T02:07:16.837183+0000 (oldest deadline 2024-04-26T02:07:38.537028+0000) 2024-04-26T02:08:18.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:17 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:17.915+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:24.239082+0000 front 2024-04-26T02:07:24.239105+0000 (oldest deadline 2024-04-26T02:07:49.439890+0000) 2024-04-26T02:08:18.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:17 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:17.915+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:34.140899+0000 front 2024-04-26T02:07:34.141046+0000 (oldest deadline 2024-04-26T02:07:55.241143+0000) 2024-04-26T02:08:18.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:17 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:17.915+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6820 osd.7 since back 2024-04-26T02:07:50.244743+0000 front 2024-04-26T02:07:50.244756+0000 (oldest deadline 2024-04-26T02:08:14.344846+0000) 2024-04-26T02:08:18.314 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:18 smithi165 podman[70737]: 2024-04-26 02:08:17.920019483 +0000 UTC m=+0.017913797 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T02:08:18.314 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:18 smithi165 podman[70737]: 2024-04-26 02:08:18.02218903 +0000 UTC m=+0.120083325 container create 0c40686552f8be8b61856178117e2c61d94fa5f6befc13e169a25aab45493f92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True) 2024-04-26T02:08:18.314 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:18 smithi165 podman[70737]: 2024-04-26 02:08:18.233279044 +0000 UTC m=+0.331173342 container init 0c40686552f8be8b61856178117e2c61d94fa5f6befc13e169a25aab45493f92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-26T02:08:18.315 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:18 smithi165 podman[70737]: 2024-04-26 02:08:18.236866308 +0000 UTC m=+0.334760611 container start 0c40686552f8be8b61856178117e2c61d94fa5f6befc13e169a25aab45493f92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-26T02:08:18.315 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:18 smithi165 podman[70737]: 2024-04-26 02:08:18.316216807 +0000 UTC m=+0.414111102 container attach 0c40686552f8be8b61856178117e2c61d94fa5f6befc13e169a25aab45493f92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-26T02:08:18.597 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:18 smithi165 podman[70737]: 2024-04-26 02:08:18.530069887 +0000 UTC m=+0.627964173 container died 0c40686552f8be8b61856178117e2c61d94fa5f6befc13e169a25aab45493f92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-26T02:08:19.070 INFO:journalctl@ceph.osd.10.smithi165.stdout:Apr 26 02:08:19 smithi165 podman[70763]: 2024-04-26 02:08:19.05246488 +0000 UTC m=+0.516750556 container remove 0c40686552f8be8b61856178117e2c61d94fa5f6befc13e169a25aab45493f92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2) 2024-04-26T02:08:19.075 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0e49e64-036d-11ef-bc93-c7b262605968@osd.10.service' 2024-04-26T02:08:19.112 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:08:19.112 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-26T02:08:19.112 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-26T02:08:19.112 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-c0e49e64-036d-11ef-bc93-c7b262605968@osd.11 2024-04-26T02:08:19.240 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:18 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:18.956+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:08:19.240 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:18 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:18.956+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.934525+0000 front 2024-04-26T02:06:59.934498+0000 (oldest deadline 2024-04-26T02:07:22.834583+0000) 2024-04-26T02:08:19.240 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:18 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:18.956+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.735339+0000 front 2024-04-26T02:07:08.735363+0000 (oldest deadline 2024-04-26T02:07:33.435537+0000) 2024-04-26T02:08:19.240 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:18 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:18.956+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:16.837132+0000 front 2024-04-26T02:07:16.837183+0000 (oldest deadline 2024-04-26T02:07:38.537028+0000) 2024-04-26T02:08:19.240 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:18 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:18.956+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:24.239082+0000 front 2024-04-26T02:07:24.239105+0000 (oldest deadline 2024-04-26T02:07:49.439890+0000) 2024-04-26T02:08:19.240 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:18 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:18.956+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:34.140899+0000 front 2024-04-26T02:07:34.141046+0000 (oldest deadline 2024-04-26T02:07:55.241143+0000) 2024-04-26T02:08:19.240 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:18 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:18.956+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6820 osd.7 since back 2024-04-26T02:07:50.244743+0000 front 2024-04-26T02:07:50.244756+0000 (oldest deadline 2024-04-26T02:08:14.344846+0000) 2024-04-26T02:08:19.240 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:19 smithi172 systemd[1]: Stopping Ceph osd.11 for c0e49e64-036d-11ef-bc93-c7b262605968... 2024-04-26T02:08:19.537 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:19 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:19.439+0000 7f9cf5522700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:08:19.537 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:19 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:19.439+0000 7f9cf5522700 -1 osd.11 112 *** Got signal Terminated *** 2024-04-26T02:08:19.537 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:19 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:19.439+0000 7f9cf5522700 -1 osd.11 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T02:08:20.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:19 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:19.998+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:08:20.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:19 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:19.998+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.934525+0000 front 2024-04-26T02:06:59.934498+0000 (oldest deadline 2024-04-26T02:07:22.834583+0000) 2024-04-26T02:08:20.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:19 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:19.998+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.735339+0000 front 2024-04-26T02:07:08.735363+0000 (oldest deadline 2024-04-26T02:07:33.435537+0000) 2024-04-26T02:08:20.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:19 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:19.998+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:16.837132+0000 front 2024-04-26T02:07:16.837183+0000 (oldest deadline 2024-04-26T02:07:38.537028+0000) 2024-04-26T02:08:20.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:19 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:19.998+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:24.239082+0000 front 2024-04-26T02:07:24.239105+0000 (oldest deadline 2024-04-26T02:07:49.439890+0000) 2024-04-26T02:08:20.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:19 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:19.998+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:34.140899+0000 front 2024-04-26T02:07:34.141046+0000 (oldest deadline 2024-04-26T02:07:55.241143+0000) 2024-04-26T02:08:20.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:19 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:19.998+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6820 osd.7 since back 2024-04-26T02:07:50.244743+0000 front 2024-04-26T02:07:50.244756+0000 (oldest deadline 2024-04-26T02:08:14.344846+0000) 2024-04-26T02:08:21.537 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:21 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:21.046+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:08:21.538 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:21 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:21.046+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.934525+0000 front 2024-04-26T02:06:59.934498+0000 (oldest deadline 2024-04-26T02:07:22.834583+0000) 2024-04-26T02:08:21.538 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:21 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:21.046+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.735339+0000 front 2024-04-26T02:07:08.735363+0000 (oldest deadline 2024-04-26T02:07:33.435537+0000) 2024-04-26T02:08:21.538 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:21 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:21.046+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:16.837132+0000 front 2024-04-26T02:07:16.837183+0000 (oldest deadline 2024-04-26T02:07:38.537028+0000) 2024-04-26T02:08:21.538 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:21 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:21.046+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:24.239082+0000 front 2024-04-26T02:07:24.239105+0000 (oldest deadline 2024-04-26T02:07:49.439890+0000) 2024-04-26T02:08:21.538 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:21 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:21.046+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:34.140899+0000 front 2024-04-26T02:07:34.141046+0000 (oldest deadline 2024-04-26T02:07:55.241143+0000) 2024-04-26T02:08:21.538 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:21 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:21.046+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6820 osd.7 since back 2024-04-26T02:07:50.244743+0000 front 2024-04-26T02:07:50.244756+0000 (oldest deadline 2024-04-26T02:08:14.344846+0000) 2024-04-26T02:08:21.538 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:21 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:21.046+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6820 osd.8 since back 2024-04-26T02:07:54.345223+0000 front 2024-04-26T02:07:54.345260+0000 (oldest deadline 2024-04-26T02:08:20.245265+0000) 2024-04-26T02:08:22.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:22 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:22.028+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:08:22.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:22 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:22.028+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.934525+0000 front 2024-04-26T02:06:59.934498+0000 (oldest deadline 2024-04-26T02:07:22.834583+0000) 2024-04-26T02:08:22.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:22 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:22.028+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.735339+0000 front 2024-04-26T02:07:08.735363+0000 (oldest deadline 2024-04-26T02:07:33.435537+0000) 2024-04-26T02:08:22.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:22 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:22.028+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:16.837132+0000 front 2024-04-26T02:07:16.837183+0000 (oldest deadline 2024-04-26T02:07:38.537028+0000) 2024-04-26T02:08:22.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:22 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:22.028+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:24.239082+0000 front 2024-04-26T02:07:24.239105+0000 (oldest deadline 2024-04-26T02:07:49.439890+0000) 2024-04-26T02:08:22.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:22 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:22.028+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:34.140899+0000 front 2024-04-26T02:07:34.141046+0000 (oldest deadline 2024-04-26T02:07:55.241143+0000) 2024-04-26T02:08:22.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:22 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:22.028+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6820 osd.7 since back 2024-04-26T02:07:50.244743+0000 front 2024-04-26T02:07:50.244756+0000 (oldest deadline 2024-04-26T02:08:14.344846+0000) 2024-04-26T02:08:22.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:22 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:22.028+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6820 osd.8 since back 2024-04-26T02:07:54.345223+0000 front 2024-04-26T02:07:54.345260+0000 (oldest deadline 2024-04-26T02:08:20.245265+0000) 2024-04-26T02:08:23.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:22 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:22.987+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:08:23.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:22 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:22.987+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.934525+0000 front 2024-04-26T02:06:59.934498+0000 (oldest deadline 2024-04-26T02:07:22.834583+0000) 2024-04-26T02:08:23.289 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:22 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:22.987+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.735339+0000 front 2024-04-26T02:07:08.735363+0000 (oldest deadline 2024-04-26T02:07:33.435537+0000) 2024-04-26T02:08:23.289 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:22 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:22.987+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:16.837132+0000 front 2024-04-26T02:07:16.837183+0000 (oldest deadline 2024-04-26T02:07:38.537028+0000) 2024-04-26T02:08:23.289 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:22 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:22.987+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:24.239082+0000 front 2024-04-26T02:07:24.239105+0000 (oldest deadline 2024-04-26T02:07:49.439890+0000) 2024-04-26T02:08:23.289 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:22 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:22.987+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:34.140899+0000 front 2024-04-26T02:07:34.141046+0000 (oldest deadline 2024-04-26T02:07:55.241143+0000) 2024-04-26T02:08:23.289 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:22 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:22.987+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6820 osd.7 since back 2024-04-26T02:07:50.244743+0000 front 2024-04-26T02:07:50.244756+0000 (oldest deadline 2024-04-26T02:08:14.344846+0000) 2024-04-26T02:08:23.289 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:22 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:22.987+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6820 osd.8 since back 2024-04-26T02:07:54.345223+0000 front 2024-04-26T02:07:54.345260+0000 (oldest deadline 2024-04-26T02:08:20.245265+0000) 2024-04-26T02:08:24.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:23 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:23.977+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-04-26T02:06:51.232907+0000 front 2024-04-26T02:06:51.232815+0000 (oldest deadline 2024-04-26T02:07:17.132670+0000) 2024-04-26T02:08:24.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:23 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:23.977+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-04-26T02:06:59.934525+0000 front 2024-04-26T02:06:59.934498+0000 (oldest deadline 2024-04-26T02:07:22.834583+0000) 2024-04-26T02:08:24.287 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:23 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:23.977+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-04-26T02:07:08.735339+0000 front 2024-04-26T02:07:08.735363+0000 (oldest deadline 2024-04-26T02:07:33.435537+0000) 2024-04-26T02:08:24.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:23 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:23.977+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-04-26T02:07:16.837132+0000 front 2024-04-26T02:07:16.837183+0000 (oldest deadline 2024-04-26T02:07:38.537028+0000) 2024-04-26T02:08:24.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:23 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:23.977+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-04-26T02:07:24.239082+0000 front 2024-04-26T02:07:24.239105+0000 (oldest deadline 2024-04-26T02:07:49.439890+0000) 2024-04-26T02:08:24.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:23 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:23.977+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-04-26T02:07:34.140899+0000 front 2024-04-26T02:07:34.141046+0000 (oldest deadline 2024-04-26T02:07:55.241143+0000) 2024-04-26T02:08:24.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:23 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:23.977+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.165:6820 osd.7 since back 2024-04-26T02:07:50.244743+0000 front 2024-04-26T02:07:50.244756+0000 (oldest deadline 2024-04-26T02:08:14.344846+0000) 2024-04-26T02:08:24.288 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:23 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11[58059]: 2024-04-26T02:08:23.977+0000 7f9cedb0e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.172:6820 osd.8 since back 2024-04-26T02:07:54.345223+0000 front 2024-04-26T02:07:54.345260+0000 (oldest deadline 2024-04-26T02:08:20.245265+0000) 2024-04-26T02:08:25.238 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:24 smithi172 podman[71633]: 2024-04-26 02:08:24.966064603 +0000 UTC m=+5.646770323 container died 7a6db04af718b69c40356fc3efcd4e98ecd75bb0cf8eca234cdb1d4a7feb175e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T02:08:25.537 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:25 smithi172 podman[71633]: 2024-04-26 02:08:25.32336646 +0000 UTC m=+6.004072176 container cleanup 7a6db04af718b69c40356fc3efcd4e98ecd75bb0cf8eca234cdb1d4a7feb175e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-26T02:08:25.537 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:25 smithi172 bash[71633]: ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11 2024-04-26T02:08:25.943 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:25 smithi172 podman[71650]: 2024-04-26 02:08:25.624704716 +0000 UTC m=+0.654478934 container remove 7a6db04af718b69c40356fc3efcd4e98ecd75bb0cf8eca234cdb1d4a7feb175e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-26T02:08:26.226 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:25 smithi172 podman[71746]: 2024-04-26 02:08:25.848325039 +0000 UTC m=+0.018000049 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T02:08:26.226 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:25 smithi172 podman[71746]: 2024-04-26 02:08:25.958852954 +0000 UTC m=+0.128527978 container create 29e43163e6ccd2bf17ae0f19528e46b3d44947f6e295ed747f3f86f4aa67f000 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240416, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-26T02:08:26.226 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:26 smithi172 podman[71746]: 2024-04-26 02:08:26.152684753 +0000 UTC m=+0.322359742 container init 29e43163e6ccd2bf17ae0f19528e46b3d44947f6e295ed747f3f86f4aa67f000 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-26T02:08:26.226 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:26 smithi172 podman[71746]: 2024-04-26 02:08:26.15577425 +0000 UTC m=+0.325449239 container start 29e43163e6ccd2bf17ae0f19528e46b3d44947f6e295ed747f3f86f4aa67f000 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-26T02:08:26.226 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:26 smithi172 podman[71746]: 2024-04-26 02:08:26.227570774 +0000 UTC m=+0.397245772 container attach 29e43163e6ccd2bf17ae0f19528e46b3d44947f6e295ed747f3f86f4aa67f000 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-26T02:08:26.537 INFO:journalctl@ceph.osd.11.smithi172.stdout:Apr 26 02:08:26 smithi172 podman[71746]: 2024-04-26 02:08:26.439224061 +0000 UTC m=+0.608899052 container died 29e43163e6ccd2bf17ae0f19528e46b3d44947f6e295ed747f3f86f4aa67f000 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-26T02:08:27.008 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0e49e64-036d-11ef-bc93-c7b262605968@osd.11.service' 2024-04-26T02:08:27.045 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:08:27.045 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-26T02:08:27.045 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-26T02:08:27.045 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mds.a 2024-04-26T02:08:27.583 INFO:journalctl@ceph.mds.a.smithi151.stdout:Apr 26 02:08:27 smithi151 systemd[1]: Stopping Ceph mds.a for c0e49e64-036d-11ef-bc93-c7b262605968... 2024-04-26T02:08:27.872 INFO:journalctl@ceph.mds.a.smithi151.stdout:Apr 26 02:08:27 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-a[69912]: 2024-04-26T02:08:27.583+0000 7fb822000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:08:27.873 INFO:journalctl@ceph.mds.a.smithi151.stdout:Apr 26 02:08:27 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-a[69912]: 2024-04-26T02:08:27.583+0000 7fb822000700 -1 mds.a *** got signal Terminated *** 2024-04-26T02:08:31.865 INFO:journalctl@ceph.mds.a.smithi151.stdout:Apr 26 02:08:31 smithi151 podman[120639]: 2024-04-26 02:08:31.553329004 +0000 UTC m=+4.088255691 container died b6ec99fbb17c460fede77cfe9c643c9571428497502e3adec126dab579f0bb82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-a, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-26T02:08:32.457 INFO:journalctl@ceph.mds.a.smithi151.stdout:Apr 26 02:08:32 smithi151 podman[120639]: 2024-04-26 02:08:32.127228383 +0000 UTC m=+4.662155111 container cleanup b6ec99fbb17c460fede77cfe9c643c9571428497502e3adec126dab579f0bb82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-a, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416) 2024-04-26T02:08:32.457 INFO:journalctl@ceph.mds.a.smithi151.stdout:Apr 26 02:08:32 smithi151 bash[120639]: ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-a 2024-04-26T02:08:32.872 INFO:journalctl@ceph.mds.a.smithi151.stdout:Apr 26 02:08:32 smithi151 podman[120654]: 2024-04-26 02:08:32.457937205 +0000 UTC m=+0.881682289 container remove b6ec99fbb17c460fede77cfe9c643c9571428497502e3adec126dab579f0bb82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-a, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T02:08:32.950 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mds.a.service' 2024-04-26T02:08:33.056 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:08:33.057 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-26T02:08:33.057 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-26T02:08:33.057 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mds.d 2024-04-26T02:08:33.372 INFO:journalctl@ceph.mds.d.smithi151.stdout:Apr 26 02:08:33 smithi151 systemd[1]: Stopping Ceph mds.d for c0e49e64-036d-11ef-bc93-c7b262605968... 2024-04-26T02:08:33.872 INFO:journalctl@ceph.mds.d.smithi151.stdout:Apr 26 02:08:33 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-d[68735]: 2024-04-26T02:08:33.563+0000 7f5717800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:08:33.873 INFO:journalctl@ceph.mds.d.smithi151.stdout:Apr 26 02:08:33 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-d[68735]: 2024-04-26T02:08:33.563+0000 7f5717800700 -1 mds.d *** got signal Terminated *** 2024-04-26T02:08:36.213 INFO:journalctl@ceph.mds.d.smithi151.stdout:Apr 26 02:08:35 smithi151 podman[120760]: 2024-04-26 02:08:35.881541383 +0000 UTC m=+2.443472495 container died 8d353b91b12ce3434bc347d5b2e0ab58bbf7a08573c8d5fe0091911780d25f8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-d, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True) 2024-04-26T02:08:36.583 INFO:journalctl@ceph.mds.d.smithi151.stdout:Apr 26 02:08:36 smithi151 podman[120760]: 2024-04-26 02:08:36.365912313 +0000 UTC m=+2.927843438 container cleanup 8d353b91b12ce3434bc347d5b2e0ab58bbf7a08573c8d5fe0091911780d25f8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-d, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T02:08:36.583 INFO:journalctl@ceph.mds.d.smithi151.stdout:Apr 26 02:08:36 smithi151 bash[120760]: ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-d 2024-04-26T02:08:36.872 INFO:journalctl@ceph.mds.d.smithi151.stdout:Apr 26 02:08:36 smithi151 podman[120774]: 2024-04-26 02:08:36.583928837 +0000 UTC m=+0.683211556 container remove 8d353b91b12ce3434bc347d5b2e0ab58bbf7a08573c8d5fe0091911780d25f8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-d, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-26T02:08:37.031 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mds.d.service' 2024-04-26T02:08:37.133 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:08:37.133 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-26T02:08:37.133 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-26T02:08:37.133 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mds.g 2024-04-26T02:08:37.625 INFO:journalctl@ceph.mds.g.smithi151.stdout:Apr 26 02:08:37 smithi151 systemd[1]: Stopping Ceph mds.g for c0e49e64-036d-11ef-bc93-c7b262605968... 2024-04-26T02:08:38.122 INFO:journalctl@ceph.mds.g.smithi151.stdout:Apr 26 02:08:37 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-g[69512]: 2024-04-26T02:08:37.725+0000 7f9595a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:08:38.123 INFO:journalctl@ceph.mds.g.smithi151.stdout:Apr 26 02:08:37 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-g[69512]: 2024-04-26T02:08:37.725+0000 7f9595a00700 -1 mds.g *** got signal Terminated *** 2024-04-26T02:08:39.110 INFO:journalctl@ceph.mds.g.smithi151.stdout:Apr 26 02:08:38 smithi151 podman[120882]: 2024-04-26 02:08:38.76851956 +0000 UTC m=+1.168861773 container died aa6c28e3310fb36f51da1f8e52b65e741a71a6a89910f2f7f259bb7e8dd99e98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-g, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-26T02:08:39.372 INFO:journalctl@ceph.mds.g.smithi151.stdout:Apr 26 02:08:39 smithi151 podman[120882]: 2024-04-26 02:08:39.287600678 +0000 UTC m=+1.687942914 container cleanup aa6c28e3310fb36f51da1f8e52b65e741a71a6a89910f2f7f259bb7e8dd99e98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-g, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416) 2024-04-26T02:08:39.372 INFO:journalctl@ceph.mds.g.smithi151.stdout:Apr 26 02:08:39 smithi151 bash[120882]: ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-g 2024-04-26T02:08:39.872 INFO:journalctl@ceph.mds.g.smithi151.stdout:Apr 26 02:08:39 smithi151 podman[120895]: 2024-04-26 02:08:39.530664491 +0000 UTC m=+0.746434047 container remove aa6c28e3310fb36f51da1f8e52b65e741a71a6a89910f2f7f259bb7e8dd99e98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-g, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-26T02:08:40.056 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mds.g.service' 2024-04-26T02:08:40.151 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:08:40.151 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-26T02:08:40.151 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-26T02:08:40.151 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mds.j 2024-04-26T02:08:40.622 INFO:journalctl@ceph.mds.j.smithi151.stdout:Apr 26 02:08:40 smithi151 systemd[1]: Stopping Ceph mds.j for c0e49e64-036d-11ef-bc93-c7b262605968... 2024-04-26T02:08:41.122 INFO:journalctl@ceph.mds.j.smithi151.stdout:Apr 26 02:08:40 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-j[68317]: 2024-04-26T02:08:40.730+0000 7ff5f6400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:08:41.123 INFO:journalctl@ceph.mds.j.smithi151.stdout:Apr 26 02:08:40 smithi151 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-j[68317]: 2024-04-26T02:08:40.730+0000 7ff5f6400700 -1 mds.j *** got signal Terminated *** 2024-04-26T02:08:43.512 INFO:journalctl@ceph.mds.j.smithi151.stdout:Apr 26 02:08:43 smithi151 podman[121001]: 2024-04-26 02:08:43.232006309 +0000 UTC m=+2.643060157 container died c1527278cf053d24301c64a0d57caef8e83305ebe400151b64e61695a831c8dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240416, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-26T02:08:43.843 INFO:journalctl@ceph.mds.j.smithi151.stdout:Apr 26 02:08:43 smithi151 podman[121001]: 2024-04-26 02:08:43.650778356 +0000 UTC m=+3.061832197 container cleanup c1527278cf053d24301c64a0d57caef8e83305ebe400151b64e61695a831c8dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-j, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T02:08:43.843 INFO:journalctl@ceph.mds.j.smithi151.stdout:Apr 26 02:08:43 smithi151 bash[121001]: ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-j 2024-04-26T02:08:44.122 INFO:journalctl@ceph.mds.j.smithi151.stdout:Apr 26 02:08:43 smithi151 podman[121027]: 2024-04-26 02:08:43.843524594 +0000 UTC m=+0.598226895 container remove c1527278cf053d24301c64a0d57caef8e83305ebe400151b64e61695a831c8dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-j, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.vendor=CentOS) 2024-04-26T02:08:44.346 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mds.j.service' 2024-04-26T02:08:44.454 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:08:44.454 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-26T02:08:44.454 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-26T02:08:44.454 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mds.b 2024-04-26T02:08:44.734 INFO:journalctl@ceph.mds.b.smithi165.stdout:Apr 26 02:08:44 smithi165 systemd[1]: Stopping Ceph mds.b for c0e49e64-036d-11ef-bc93-c7b262605968... 2024-04-26T02:08:44.997 INFO:journalctl@ceph.mds.b.smithi165.stdout:Apr 26 02:08:44 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-b[63325]: 2024-04-26T02:08:44.734+0000 7f3c43952700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:08:44.997 INFO:journalctl@ceph.mds.b.smithi165.stdout:Apr 26 02:08:44 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-b[63325]: 2024-04-26T02:08:44.734+0000 7f3c43952700 -1 mds.b *** got signal Terminated *** 2024-04-26T02:08:48.151 INFO:journalctl@ceph.mds.b.smithi165.stdout:Apr 26 02:08:47 smithi165 podman[70845]: 2024-04-26 02:08:47.892715775 +0000 UTC m=+3.252082305 container died 749c302b691920f004dc0795d12e06e988f191cacad1881a32c3eb35a8043fab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-b, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T02:08:48.426 INFO:journalctl@ceph.mds.b.smithi165.stdout:Apr 26 02:08:48 smithi165 podman[70845]: 2024-04-26 02:08:48.217935207 +0000 UTC m=+3.577301736 container cleanup 749c302b691920f004dc0795d12e06e988f191cacad1881a32c3eb35a8043fab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-26T02:08:48.426 INFO:journalctl@ceph.mds.b.smithi165.stdout:Apr 26 02:08:48 smithi165 bash[70845]: ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-b 2024-04-26T02:08:48.638 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mds.b.service' 2024-04-26T02:08:48.673 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:08:48.673 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-26T02:08:48.673 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-26T02:08:48.673 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mds.e 2024-04-26T02:08:48.996 INFO:journalctl@ceph.mds.e.smithi165.stdout:Apr 26 02:08:48 smithi165 systemd[1]: Stopping Ceph mds.e for c0e49e64-036d-11ef-bc93-c7b262605968... 2024-04-26T02:08:49.347 INFO:journalctl@ceph.mds.e.smithi165.stdout:Apr 26 02:08:48 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-e[62700]: 2024-04-26T02:08:48.996+0000 7f1c498aa700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:08:49.347 INFO:journalctl@ceph.mds.e.smithi165.stdout:Apr 26 02:08:48 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-e[62700]: 2024-04-26T02:08:48.996+0000 7f1c498aa700 -1 mds.e *** got signal Terminated *** 2024-04-26T02:08:52.211 INFO:journalctl@ceph.mds.e.smithi165.stdout:Apr 26 02:08:51 smithi165 podman[70979]: 2024-04-26 02:08:51.897082817 +0000 UTC m=+3.007559984 container died ddbe4960fbb298b411844d9e64dad98c1560b7a0749fc3e7fb839a8eedadb8b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-e, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-26T02:08:52.211 INFO:journalctl@ceph.mds.e.smithi165.stdout:Apr 26 02:08:52 smithi165 podman[70979]: 2024-04-26 02:08:52.212901527 +0000 UTC m=+3.323378684 container cleanup ddbe4960fbb298b411844d9e64dad98c1560b7a0749fc3e7fb839a8eedadb8b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-e, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-26T02:08:52.479 INFO:journalctl@ceph.mds.e.smithi165.stdout:Apr 26 02:08:52 smithi165 bash[70979]: ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-e 2024-04-26T02:08:52.764 INFO:journalctl@ceph.mds.e.smithi165.stdout:Apr 26 02:08:52 smithi165 podman[70993]: 2024-04-26 02:08:52.480620273 +0000 UTC m=+0.585018205 container remove ddbe4960fbb298b411844d9e64dad98c1560b7a0749fc3e7fb839a8eedadb8b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-e, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-26T02:08:52.780 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mds.e.service' 2024-04-26T02:08:52.813 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:08:52.813 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-26T02:08:52.813 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-26T02:08:52.813 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mds.h 2024-04-26T02:08:53.097 INFO:journalctl@ceph.mds.h.smithi165.stdout:Apr 26 02:08:52 smithi165 systemd[1]: Stopping Ceph mds.h for c0e49e64-036d-11ef-bc93-c7b262605968... 2024-04-26T02:08:53.597 INFO:journalctl@ceph.mds.h.smithi165.stdout:Apr 26 02:08:53 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-h[63016]: 2024-04-26T02:08:53.107+0000 7ff80e6ab700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:08:53.597 INFO:journalctl@ceph.mds.h.smithi165.stdout:Apr 26 02:08:53 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-h[63016]: 2024-04-26T02:08:53.107+0000 7ff80e6ab700 -1 mds.h *** got signal Terminated *** 2024-04-26T02:08:56.165 INFO:journalctl@ceph.mds.h.smithi165.stdout:Apr 26 02:08:55 smithi165 podman[71098]: 2024-04-26 02:08:55.889091444 +0000 UTC m=+2.862292343 container died a37b5d31366444f427078dfc9d889e6aa6d83f116c59eff9af08868f79219b01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-h, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T02:08:56.507 INFO:journalctl@ceph.mds.h.smithi165.stdout:Apr 26 02:08:56 smithi165 podman[71098]: 2024-04-26 02:08:56.257654511 +0000 UTC m=+3.230855402 container cleanup a37b5d31366444f427078dfc9d889e6aa6d83f116c59eff9af08868f79219b01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T02:08:56.507 INFO:journalctl@ceph.mds.h.smithi165.stdout:Apr 26 02:08:56 smithi165 bash[71098]: ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-h 2024-04-26T02:08:56.637 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mds.h.service' 2024-04-26T02:08:56.668 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:08:56.668 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-26T02:08:56.669 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-26T02:08:56.669 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mds.k 2024-04-26T02:08:56.847 INFO:journalctl@ceph.mds.k.smithi165.stdout:Apr 26 02:08:56 smithi165 systemd[1]: Stopping Ceph mds.k for c0e49e64-036d-11ef-bc93-c7b262605968... 2024-04-26T02:08:57.347 INFO:journalctl@ceph.mds.k.smithi165.stdout:Apr 26 02:08:56 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-k[62381]: 2024-04-26T02:08:56.961+0000 7fb763078700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:08:57.347 INFO:journalctl@ceph.mds.k.smithi165.stdout:Apr 26 02:08:56 smithi165 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-k[62381]: 2024-04-26T02:08:56.961+0000 7fb763078700 -1 mds.k *** got signal Terminated *** 2024-04-26T02:08:58.386 INFO:journalctl@ceph.mds.k.smithi165.stdout:Apr 26 02:08:58 smithi165 podman[71219]: 2024-04-26 02:08:58.11656148 +0000 UTC m=+1.234986896 container died 78f7d2a054140649cfde348c3cda960f8a3b7a872c5a3de0dbf0cd0df958023b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-k, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-26T02:08:58.817 INFO:journalctl@ceph.mds.k.smithi165.stdout:Apr 26 02:08:58 smithi165 podman[71219]: 2024-04-26 02:08:58.461491753 +0000 UTC m=+1.579917168 container cleanup 78f7d2a054140649cfde348c3cda960f8a3b7a872c5a3de0dbf0cd0df958023b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-k, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True) 2024-04-26T02:08:58.817 INFO:journalctl@ceph.mds.k.smithi165.stdout:Apr 26 02:08:58 smithi165 bash[71219]: ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-k 2024-04-26T02:08:58.817 INFO:journalctl@ceph.mds.k.smithi165.stdout:Apr 26 02:08:58 smithi165 podman[71233]: 2024-04-26 02:08:58.628833954 +0000 UTC m=+0.511556227 container remove 78f7d2a054140649cfde348c3cda960f8a3b7a872c5a3de0dbf0cd0df958023b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-26T02:08:58.839 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mds.k.service' 2024-04-26T02:08:58.870 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:08:58.870 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-26T02:08:58.870 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-26T02:08:58.870 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mds.c 2024-04-26T02:08:59.164 INFO:journalctl@ceph.mds.c.smithi172.stdout:Apr 26 02:08:58 smithi172 systemd[1]: Stopping Ceph mds.c for c0e49e64-036d-11ef-bc93-c7b262605968... 2024-04-26T02:08:59.428 INFO:journalctl@ceph.mds.c.smithi172.stdout:Apr 26 02:08:59 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-c[63496]: 2024-04-26T02:08:59.164+0000 7f5421601700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:08:59.428 INFO:journalctl@ceph.mds.c.smithi172.stdout:Apr 26 02:08:59 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-c[63496]: 2024-04-26T02:08:59.164+0000 7f5421601700 -1 mds.c *** got signal Terminated *** 2024-04-26T02:09:03.537 INFO:journalctl@ceph.mds.c.smithi172.stdout:Apr 26 02:09:03 smithi172 podman[71852]: 2024-04-26 02:09:03.178037332 +0000 UTC m=+4.110997286 container died 4da7b90b621d3980cd26764930148b950e51bae17f8781dfdc3882651008f38d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-c, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-26T02:09:03.880 INFO:journalctl@ceph.mds.c.smithi172.stdout:Apr 26 02:09:03 smithi172 podman[71852]: 2024-04-26 02:09:03.622554172 +0000 UTC m=+4.555514120 container cleanup 4da7b90b621d3980cd26764930148b950e51bae17f8781dfdc3882651008f38d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-c, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-26T02:09:03.880 INFO:journalctl@ceph.mds.c.smithi172.stdout:Apr 26 02:09:03 smithi172 bash[71852]: ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-c 2024-04-26T02:09:04.037 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mds.c.service' 2024-04-26T02:09:04.072 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:09:04.072 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-26T02:09:04.072 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-26T02:09:04.072 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mds.f 2024-04-26T02:09:04.409 INFO:journalctl@ceph.mds.f.smithi172.stdout:Apr 26 02:09:04 smithi172 systemd[1]: Stopping Ceph mds.f for c0e49e64-036d-11ef-bc93-c7b262605968... 2024-04-26T02:09:04.787 INFO:journalctl@ceph.mds.f.smithi172.stdout:Apr 26 02:09:04 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-f[63189]: 2024-04-26T02:09:04.409+0000 7f9a078d5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:09:04.787 INFO:journalctl@ceph.mds.f.smithi172.stdout:Apr 26 02:09:04 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-f[63189]: 2024-04-26T02:09:04.409+0000 7f9a078d5700 -1 mds.f *** got signal Terminated *** 2024-04-26T02:09:08.175 INFO:journalctl@ceph.mds.f.smithi172.stdout:Apr 26 02:09:07 smithi172 podman[71986]: 2024-04-26 02:09:07.883808042 +0000 UTC m=+3.579957362 container died d8269a242bd32bf2b55aef082d70891a92f3d8332fcff5812608152b53ec5b1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-f, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-26T02:09:08.492 INFO:journalctl@ceph.mds.f.smithi172.stdout:Apr 26 02:09:08 smithi172 podman[71986]: 2024-04-26 02:09:08.276117564 +0000 UTC m=+3.972266877 container cleanup d8269a242bd32bf2b55aef082d70891a92f3d8332fcff5812608152b53ec5b1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-f, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-26T02:09:08.492 INFO:journalctl@ceph.mds.f.smithi172.stdout:Apr 26 02:09:08 smithi172 bash[71986]: ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-f 2024-04-26T02:09:08.631 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mds.f.service' 2024-04-26T02:09:08.663 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:09:08.664 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-26T02:09:08.664 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-26T02:09:08.664 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mds.i 2024-04-26T02:09:09.037 INFO:journalctl@ceph.mds.i.smithi172.stdout:Apr 26 02:09:08 smithi172 systemd[1]: Stopping Ceph mds.i for c0e49e64-036d-11ef-bc93-c7b262605968... 2024-04-26T02:09:09.038 INFO:journalctl@ceph.mds.i.smithi172.stdout:Apr 26 02:09:08 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-i[63813]: 2024-04-26T02:09:08.987+0000 7fa7a4fe6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:09:09.038 INFO:journalctl@ceph.mds.i.smithi172.stdout:Apr 26 02:09:08 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-i[63813]: 2024-04-26T02:09:08.987+0000 7fa7a4fe6700 -1 mds.i *** got signal Terminated *** 2024-04-26T02:09:12.262 INFO:journalctl@ceph.mds.i.smithi172.stdout:Apr 26 02:09:11 smithi172 podman[72106]: 2024-04-26 02:09:11.896230303 +0000 UTC m=+3.013159489 container died c32829abcd2d6fcc40d1c33543f3e4cf37d1118090af9329a4190ad245dc67e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-i, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T02:09:12.537 INFO:journalctl@ceph.mds.i.smithi172.stdout:Apr 26 02:09:12 smithi172 podman[72106]: 2024-04-26 02:09:12.363337111 +0000 UTC m=+3.480266290 container cleanup c32829abcd2d6fcc40d1c33543f3e4cf37d1118090af9329a4190ad245dc67e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-i, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T02:09:12.537 INFO:journalctl@ceph.mds.i.smithi172.stdout:Apr 26 02:09:12 smithi172 bash[72106]: ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-i 2024-04-26T02:09:12.793 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mds.i.service' 2024-04-26T02:09:12.824 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:09:12.825 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-26T02:09:12.825 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-26T02:09:12.825 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mds.l 2024-04-26T02:09:13.037 INFO:journalctl@ceph.mds.l.smithi172.stdout:Apr 26 02:09:12 smithi172 systemd[1]: Stopping Ceph mds.l for c0e49e64-036d-11ef-bc93-c7b262605968... 2024-04-26T02:09:13.537 INFO:journalctl@ceph.mds.l.smithi172.stdout:Apr 26 02:09:13 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-l[62875]: 2024-04-26T02:09:13.166+0000 7fda84277700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:09:13.538 INFO:journalctl@ceph.mds.l.smithi172.stdout:Apr 26 02:09:13 smithi172 ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-l[62875]: 2024-04-26T02:09:13.166+0000 7fda84277700 -1 mds.l *** got signal Terminated *** 2024-04-26T02:09:15.641 INFO:journalctl@ceph.mds.l.smithi172.stdout:Apr 26 02:09:15 smithi172 podman[72226]: 2024-04-26 02:09:15.380225579 +0000 UTC m=+2.340427158 container died 5ca0c9a91dcbda1ff7f60396ddd30053c23fc86d50744793d56726bfb3ed0bcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-l, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-26T02:09:15.991 INFO:journalctl@ceph.mds.l.smithi172.stdout:Apr 26 02:09:15 smithi172 podman[72226]: 2024-04-26 02:09:15.733317248 +0000 UTC m=+2.693518837 container cleanup 5ca0c9a91dcbda1ff7f60396ddd30053c23fc86d50744793d56726bfb3ed0bcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-l, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T02:09:15.991 INFO:journalctl@ceph.mds.l.smithi172.stdout:Apr 26 02:09:15 smithi172 bash[72226]: ceph-c0e49e64-036d-11ef-bc93-c7b262605968-mds-l 2024-04-26T02:09:16.159 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0e49e64-036d-11ef-bc93-c7b262605968@mds.l.service' 2024-04-26T02:09:16.189 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:09:16.190 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-26T02:09:16.190 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c0e49e64-036d-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-26T02:09:16.512 INFO:teuthology.orchestra.run.smithi151.stdout:Deleting cluster with fsid: c0e49e64-036d-11ef-bc93-c7b262605968 2024-04-26T02:09:21.553 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c0e49e64-036d-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-26T02:09:21.743 INFO:teuthology.orchestra.run.smithi165.stdout:Deleting cluster with fsid: c0e49e64-036d-11ef-bc93-c7b262605968 2024-04-26T02:09:24.150 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c0e49e64-036d-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-26T02:09:24.338 INFO:teuthology.orchestra.run.smithi172.stdout:Deleting cluster with fsid: c0e49e64-036d-11ef-bc93-c7b262605968 2024-04-26T02:09:26.651 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T02:09:26.746 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T02:09:26.778 DEBUG:teuthology.orchestra.run.smithi172:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T02:09:26.803 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-26T02:09:26.805 DEBUG:teuthology.misc:Transferring archived files from smithi151:/var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-26_01:02:28-fs-wip-lusov-quiescer-distro-default-smithi/7674253/remote/smithi151/crash 2024-04-26T02:09:26.806 DEBUG:teuthology.orchestra.run.smithi151:> sudo tar c -f - -C /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/crash -- . 2024-04-26T02:09:26.918 INFO:teuthology.orchestra.run.smithi151.stderr:tar: /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-26T02:09:26.918 INFO:teuthology.orchestra.run.smithi151.stderr:tar: Error is not recoverable: exiting now 2024-04-26T02:09:26.924 DEBUG:teuthology.misc:Transferring archived files from smithi165:/var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-26_01:02:28-fs-wip-lusov-quiescer-distro-default-smithi/7674253/remote/smithi165/crash 2024-04-26T02:09:26.925 DEBUG:teuthology.orchestra.run.smithi165:> sudo tar c -f - -C /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/crash -- . 2024-04-26T02:09:26.997 INFO:teuthology.orchestra.run.smithi165.stderr:tar: /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-26T02:09:26.997 INFO:teuthology.orchestra.run.smithi165.stderr:tar: Error is not recoverable: exiting now 2024-04-26T02:09:26.999 DEBUG:teuthology.misc:Transferring archived files from smithi172:/var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-26_01:02:28-fs-wip-lusov-quiescer-distro-default-smithi/7674253/remote/smithi172/crash 2024-04-26T02:09:27.000 DEBUG:teuthology.orchestra.run.smithi172:> sudo tar c -f - -C /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/crash -- . 2024-04-26T02:09:27.036 INFO:teuthology.orchestra.run.smithi172.stderr:tar: /var/lib/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-26T02:09:27.036 INFO:teuthology.orchestra.run.smithi172.stderr:tar: Error is not recoverable: exiting now 2024-04-26T02:09:27.037 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-26T02:09:27.038 DEBUG:teuthology.orchestra.run.smithi151:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-26T02:09:27.142 INFO:tasks.cephadm:Compressing logs... 2024-04-26T02:09:27.142 DEBUG:teuthology.orchestra.run.smithi151:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T02:09:27.146 DEBUG:teuthology.orchestra.run.smithi165:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T02:09:27.149 DEBUG:teuthology.orchestra.run.smithi172:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T02:09:27.172 INFO:teuthology.orchestra.run.smithi172.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-26T02:09:27.173 INFO:teuthology.orchestra.run.smithi172.stderr:: No such file or directory 2024-04-26T02:09:27.173 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph.log 2024-04-26T02:09:27.173 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-26T02:09:27.173 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-26T02:09:27.173 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-26T02:09:27.182 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-26T02:09:27.183 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-mds.f.log: /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-26T02:09:27.183 INFO:teuthology.orchestra.run.smithi172.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-26T02:09:27.183 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-26T02:09:27.183 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-26T02:09:27.183 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-26T02:09:27.183 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-mgr.z.log: 87.2% -- replaced with /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-26T02:09:27.183 INFO:teuthology.orchestra.run.smithi172.stderr: 90.5% -- replaced with /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-26T02:09:27.183 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-26T02:09:27.183 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-26T02:09:27.183 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph.cephadm.log: 83.6% -- replaced with /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-26T02:09:27.183 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-26T02:09:27.184 INFO:teuthology.orchestra.run.smithi165.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-26T02:09:27.184 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-26T02:09:27.184 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph.log 2024-04-26T02:09:27.184 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-26T02:09:27.184 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-26T02:09:27.184 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph.log: /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-26T02:09:27.184 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-26T02:09:27.184 INFO:teuthology.orchestra.run.smithi165.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-26T02:09:27.184 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-26T02:09:27.184 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-26T02:09:27.184 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-osd.4.log: /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-26T02:09:27.184 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-26T02:09:27.184 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-mgr.y.log: /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-mds.h.log: 87.2% -- replaced with /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-26T02:09:27.184 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose 90.4% -- /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-26T02:09:27.185 INFO:teuthology.orchestra.run.smithi165.stderr: -- replaced with /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-26T02:09:27.185 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-26T02:09:27.185 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph.cephadm.log: 83.7% -- replaced with /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-26T02:09:27.185 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-26T02:09:27.186 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-osd.11.log: 92.2% -- replaced with /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-26T02:09:27.186 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-26T02:09:27.187 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-mds.e.log: 92.2% -- replaced with /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-26T02:09:27.193 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-26T02:09:27.198 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-26T02:09:27.206 INFO:teuthology.orchestra.run.smithi151.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-26T02:09:27.206 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-26T02:09:27.206 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-26T02:09:27.207 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph.log 2024-04-26T02:09:27.207 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-26T02:09:27.208 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-26T02:09:27.209 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-mds.g.log 2024-04-26T02:09:27.210 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-26T02:09:27.211 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-mds.g.log: 87.2% -- replaced with /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-26T02:09:27.211 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-26T02:09:27.212 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-26T02:09:27.213 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-mds.j.log: /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-osd.3.log: 90.3% -- replaced with /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-26T02:09:27.214 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-26T02:09:27.214 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-mds.k.log: 94.4% -- replaced with /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-26T02:09:27.215 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-26T02:09:27.216 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-26T02:09:27.216 INFO:teuthology.orchestra.run.smithi151.stderr: 94.5% -- replaced with /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-26T02:09:27.222 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-mon.a.log: /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-26T02:09:27.222 INFO:teuthology.orchestra.run.smithi151.stderr: 94.5% 93.9% -- replaced with /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-mds.j.log.gz -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-26T02:09:27.222 INFO:teuthology.orchestra.run.smithi151.stderr: 2024-04-26T02:09:27.222 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-26T02:09:27.223 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-26T02:09:27.223 INFO:teuthology.orchestra.run.smithi151.stderr: 84.0% -- replaced with /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-26T02:09:27.224 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-volume.log: /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-mds.l.log: 94.4% -- replaced with /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-26T02:09:27.224 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-26T02:09:27.225 INFO:teuthology.orchestra.run.smithi172.stderr: 91.0% -- replaced with /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-26T02:09:27.227 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-volume.log: /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-26T02:09:27.230 INFO:teuthology.orchestra.run.smithi151.stderr: 94.5%/var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-osd.9.log: -- replaced with /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-26T02:09:27.237 INFO:teuthology.orchestra.run.smithi151.stderr: 90.9% -- replaced with /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-26T02:09:27.240 INFO:teuthology.orchestra.run.smithi151.stderr: 94.4% -- replaced with /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-26T02:09:27.272 INFO:teuthology.orchestra.run.smithi165.stderr: 94.2% -- replaced with /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-26T02:09:27.275 INFO:teuthology.orchestra.run.smithi165.stderr: 94.1% -- replaced with /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-26T02:09:27.532 INFO:teuthology.orchestra.run.smithi151.stderr: 90.1% -- replaced with /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-26T02:09:27.634 INFO:teuthology.orchestra.run.smithi172.stderr: 94.2% -- replaced with /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-26T02:09:27.756 INFO:teuthology.orchestra.run.smithi165.stderr: 91.8% -- replaced with /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-26T02:09:27.799 INFO:teuthology.orchestra.run.smithi172.stderr: 94.2% -- replaced with /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-26T02:09:27.821 INFO:teuthology.orchestra.run.smithi172.stderr: 91.7% -- replaced with /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-26T02:09:28.676 INFO:teuthology.orchestra.run.smithi151.stderr: 90.4% -- replaced with /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-26T02:09:30.470 INFO:teuthology.orchestra.run.smithi151.stderr: 93.6% -- replaced with /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-26T02:09:30.554 INFO:teuthology.orchestra.run.smithi172.stderr: 94.0% -- replaced with /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-26T02:09:30.564 INFO:teuthology.orchestra.run.smithi172.stderr: 95.7% -- replaced with /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-26T02:09:30.739 INFO:teuthology.orchestra.run.smithi165.stderr: 94.1% -- replaced with /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-26T02:09:31.345 INFO:teuthology.orchestra.run.smithi165.stderr: 93.9% -- replaced with /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-26T02:09:31.412 INFO:teuthology.orchestra.run.smithi151.stderr: 94.2% -- replaced with /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-26T02:09:31.768 INFO:teuthology.orchestra.run.smithi172.stderr: 94.0% -- replaced with /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-26T02:09:32.128 INFO:teuthology.orchestra.run.smithi165.stderr: 93.8% -- replaced with /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-26T02:09:32.458 INFO:teuthology.orchestra.run.smithi172.stderr: 93.9% -- replaced with /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-26T02:09:32.950 INFO:teuthology.orchestra.run.smithi151.stderr: 93.5% -- replaced with /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-26T02:09:33.197 INFO:teuthology.orchestra.run.smithi172.stderr: 94.4% -- replaced with /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-26T02:09:33.199 INFO:teuthology.orchestra.run.smithi172.stderr: 2024-04-26T02:09:33.199 INFO:teuthology.orchestra.run.smithi172.stderr:real 0m6.036s 2024-04-26T02:09:33.199 INFO:teuthology.orchestra.run.smithi172.stderr:user 0m23.522s 2024-04-26T02:09:33.199 INFO:teuthology.orchestra.run.smithi172.stderr:sys 0m0.860s 2024-04-26T02:09:33.238 INFO:teuthology.orchestra.run.smithi165.stderr: 94.2% -- replaced with /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-26T02:09:33.620 INFO:teuthology.orchestra.run.smithi151.stderr: 93.8% -- replaced with /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-26T02:09:33.625 INFO:teuthology.orchestra.run.smithi151.stderr: 2024-04-26T02:09:33.625 INFO:teuthology.orchestra.run.smithi151.stderr:real 0m6.445s 2024-04-26T02:09:33.625 INFO:teuthology.orchestra.run.smithi151.stderr:user 0m19.762s 2024-04-26T02:09:33.625 INFO:teuthology.orchestra.run.smithi151.stderr:sys 0m1.729s 2024-04-26T02:10:04.104 INFO:teuthology.orchestra.run.smithi165.stderr: 94.0% -- replaced with /var/log/ceph/c0e49e64-036d-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-26T02:10:04.105 INFO:teuthology.orchestra.run.smithi165.stderr: 2024-04-26T02:10:04.105 INFO:teuthology.orchestra.run.smithi165.stderr:real 0m36.942s 2024-04-26T02:10:04.105 INFO:teuthology.orchestra.run.smithi165.stderr:user 0m54.488s 2024-04-26T02:10:04.106 INFO:teuthology.orchestra.run.smithi165.stderr:sys 0m1.880s 2024-04-26T02:10:04.106 INFO:tasks.cephadm:Archiving logs... 2024-04-26T02:10:04.106 DEBUG:teuthology.misc:Transferring archived files from smithi151:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-26_01:02:28-fs-wip-lusov-quiescer-distro-default-smithi/7674253/remote/smithi151/log 2024-04-26T02:10:04.107 DEBUG:teuthology.orchestra.run.smithi151:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T02:10:07.203 DEBUG:teuthology.misc:Transferring archived files from smithi165:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-26_01:02:28-fs-wip-lusov-quiescer-distro-default-smithi/7674253/remote/smithi165/log 2024-04-26T02:10:07.205 DEBUG:teuthology.orchestra.run.smithi165:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T02:10:14.739 DEBUG:teuthology.misc:Transferring archived files from smithi172:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-26_01:02:28-fs-wip-lusov-quiescer-distro-default-smithi/7674253/remote/smithi172/log 2024-04-26T02:10:14.740 DEBUG:teuthology.orchestra.run.smithi172:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T02:10:17.804 INFO:tasks.cephadm:Removing cluster... 2024-04-26T02:10:17.804 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c0e49e64-036d-11ef-bc93-c7b262605968 --force 2024-04-26T02:10:18.119 INFO:teuthology.orchestra.run.smithi151.stdout:Deleting cluster with fsid: c0e49e64-036d-11ef-bc93-c7b262605968 2024-04-26T02:10:18.753 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c0e49e64-036d-11ef-bc93-c7b262605968 --force 2024-04-26T02:10:18.944 INFO:teuthology.orchestra.run.smithi165.stdout:Deleting cluster with fsid: c0e49e64-036d-11ef-bc93-c7b262605968 2024-04-26T02:10:19.265 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c0e49e64-036d-11ef-bc93-c7b262605968 --force 2024-04-26T02:10:19.457 INFO:teuthology.orchestra.run.smithi172.stdout:Deleting cluster with fsid: c0e49e64-036d-11ef-bc93-c7b262605968 2024-04-26T02:10:19.735 INFO:tasks.cephadm:Removing cephadm ... 2024-04-26T02:10:19.736 DEBUG:teuthology.orchestra.run.smithi151:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-26T02:10:19.794 DEBUG:teuthology.orchestra.run.smithi165:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-26T02:10:19.812 DEBUG:teuthology.orchestra.run.smithi172:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-26T02:10:19.826 INFO:tasks.cephadm:Teardown complete 2024-04-26T02:10:19.827 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-26T02:10:19.838 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-26T02:10:19.838 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T02:10:19.846 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T02:10:19.855 DEBUG:teuthology.orchestra.run.smithi172:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T02:10:19.998 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-26T02:10:20.008 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-26T02:10:20.009 DEBUG:teuthology.orchestra.run.smithi151:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T02:10:20.016 DEBUG:teuthology.orchestra.run.smithi165:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T02:10:20.019 DEBUG:teuthology.orchestra.run.smithi172:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T02:10:20.032 INFO:teuthology.orchestra.run.smithi165.stderr:bash: line 1: ntpq: command not found 2024-04-26T02:10:20.032 INFO:teuthology.orchestra.run.smithi172.stderr:bash: line 1: ntpq: command not found 2024-04-26T02:10:20.036 INFO:teuthology.orchestra.run.smithi165.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T02:10:20.036 INFO:teuthology.orchestra.run.smithi165.stdout:=============================================================================== 2024-04-26T02:10:20.036 INFO:teuthology.orchestra.run.smithi165.stdout:^* hv01.front.sepia.ceph.com 3 6 377 25 +1724us[+1727us] +/- 51ms 2024-04-26T02:10:20.036 INFO:teuthology.orchestra.run.smithi165.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 23 -1953us[-1953us] +/- 71ms 2024-04-26T02:10:20.036 INFO:teuthology.orchestra.run.smithi165.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 26 -591us[ -588us] +/- 81ms 2024-04-26T02:10:20.036 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T02:10:20.038 INFO:teuthology.orchestra.run.smithi172.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T02:10:20.038 INFO:teuthology.orchestra.run.smithi172.stdout:=============================================================================== 2024-04-26T02:10:20.038 INFO:teuthology.orchestra.run.smithi172.stdout:^* hv01.front.sepia.ceph.com 3 6 377 28 +1705us[+1711us] +/- 51ms 2024-04-26T02:10:20.038 INFO:teuthology.orchestra.run.smithi172.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 27 -1969us[-1969us] +/- 71ms 2024-04-26T02:10:20.038 INFO:teuthology.orchestra.run.smithi172.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 28 -623us[ -623us] +/- 81ms 2024-04-26T02:10:20.038 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T02:10:20.061 INFO:teuthology.orchestra.run.smithi151.stderr:bash: line 1: ntpq: command not found 2024-04-26T02:10:20.069 INFO:teuthology.orchestra.run.smithi151.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T02:10:20.069 INFO:teuthology.orchestra.run.smithi151.stdout:=============================================================================== 2024-04-26T02:10:20.069 INFO:teuthology.orchestra.run.smithi151.stdout:^* hv01.front.sepia.ceph.com 3 6 377 25 +1770us[+1777us] +/- 51ms 2024-04-26T02:10:20.069 INFO:teuthology.orchestra.run.smithi151.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 25 -1879us[-1872us] +/- 71ms 2024-04-26T02:10:20.069 INFO:teuthology.orchestra.run.smithi151.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 25 -534us[ -527us] +/- 81ms 2024-04-26T02:10:20.070 INFO:teuthology.orchestra.run.smithi151.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T02:10:20.071 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-26T02:10:20.082 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-26T02:10:20.082 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-26T02:10:20.098 DEBUG:teuthology.orchestra.run.smithi151:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T02:10:20.302 DEBUG:teuthology.orchestra.run.smithi165:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T02:10:20.373 DEBUG:teuthology.orchestra.run.smithi172:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T02:10:20.438 DEBUG:teuthology.orchestra.run.smithi151:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T02:10:20.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T02:10:20.535 DEBUG:teuthology.orchestra.run.smithi165:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T02:10:20.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T02:10:20.566 DEBUG:teuthology.orchestra.run.smithi172:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T02:10:20.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T02:10:20.594 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-26T02:10:20.612 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-26T02:10:20.625 INFO:teuthology.task.internal:Duration was 2342.415371 seconds 2024-04-26T02:10:20.626 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-26T02:10:20.637 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-26T02:10:20.638 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T02:10:20.645 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T02:10:20.648 DEBUG:teuthology.orchestra.run.smithi172:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T02:10:20.684 INFO:teuthology.orchestra.run.smithi172.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T02:10:20.686 INFO:teuthology.orchestra.run.smithi165.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T02:10:20.774 INFO:teuthology.orchestra.run.smithi151.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T02:10:21.138 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-26T02:10:21.138 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi151.front.sepia.ceph.com 2024-04-26T02:10:21.138 DEBUG:teuthology.orchestra.run.smithi151:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T02:10:21.193 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi165.front.sepia.ceph.com 2024-04-26T02:10:21.193 DEBUG:teuthology.orchestra.run.smithi165:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T02:10:21.223 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi172.front.sepia.ceph.com 2024-04-26T02:10:21.224 DEBUG:teuthology.orchestra.run.smithi172:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T02:10:21.256 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-26T02:10:21.256 DEBUG:teuthology.orchestra.run.smithi151:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T02:10:21.264 DEBUG:teuthology.orchestra.run.smithi165:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T02:10:21.266 DEBUG:teuthology.orchestra.run.smithi172:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T02:10:21.339 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-26T02:10:21.339 DEBUG:teuthology.orchestra.run.smithi151:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T02:10:21.344 DEBUG:teuthology.orchestra.run.smithi165:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T02:10:21.347 DEBUG:teuthology.orchestra.run.smithi172:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T02:10:22.138 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-26T02:10:22.150 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-26T02:10:22.150 DEBUG:teuthology.orchestra.run.smithi151:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T02:10:22.231 DEBUG:teuthology.orchestra.run.smithi165:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T02:10:22.258 DEBUG:teuthology.orchestra.run.smithi172:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T02:10:22.284 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-26T02:10:22.296 DEBUG:teuthology.orchestra.run.smithi151:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T02:10:22.304 DEBUG:teuthology.orchestra.run.smithi165:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T02:10:22.306 DEBUG:teuthology.orchestra.run.smithi172:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T02:10:22.328 INFO:teuthology.orchestra.run.smithi165.stdout:kernel.core_pattern = core 2024-04-26T02:10:22.347 INFO:teuthology.orchestra.run.smithi172.stdout:kernel.core_pattern = core 2024-04-26T02:10:22.394 INFO:teuthology.orchestra.run.smithi151.stdout:kernel.core_pattern = core 2024-04-26T02:10:22.434 DEBUG:teuthology.orchestra.run.smithi151:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T02:10:22.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T02:10:22.481 DEBUG:teuthology.orchestra.run.smithi165:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T02:10:22.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T02:10:22.498 DEBUG:teuthology.orchestra.run.smithi172:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T02:10:22.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T02:10:22.512 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-26T02:10:22.524 INFO:teuthology.task.internal:Transferring archived files... 2024-04-26T02:10:22.524 DEBUG:teuthology.misc:Transferring archived files from smithi151:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-26_01:02:28-fs-wip-lusov-quiescer-distro-default-smithi/7674253/remote/smithi151 2024-04-26T02:10:22.525 DEBUG:teuthology.orchestra.run.smithi151:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T02:10:22.639 DEBUG:teuthology.misc:Transferring archived files from smithi165:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-26_01:02:28-fs-wip-lusov-quiescer-distro-default-smithi/7674253/remote/smithi165 2024-04-26T02:10:22.639 DEBUG:teuthology.orchestra.run.smithi165:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T02:10:22.677 DEBUG:teuthology.misc:Transferring archived files from smithi172:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-26_01:02:28-fs-wip-lusov-quiescer-distro-default-smithi/7674253/remote/smithi172 2024-04-26T02:10:22.678 DEBUG:teuthology.orchestra.run.smithi172:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T02:10:22.713 INFO:teuthology.task.internal:Removing archive directory... 2024-04-26T02:10:22.713 DEBUG:teuthology.orchestra.run.smithi151:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T02:10:22.721 DEBUG:teuthology.orchestra.run.smithi165:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T02:10:22.723 DEBUG:teuthology.orchestra.run.smithi172:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T02:10:22.772 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-26T02:10:22.784 INFO:teuthology.task.internal:Not uploading archives. 2024-04-26T02:10:22.784 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-26T02:10:22.796 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-26T02:10:22.797 DEBUG:teuthology.orchestra.run.smithi151:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T02:10:22.804 DEBUG:teuthology.orchestra.run.smithi165:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T02:10:22.806 DEBUG:teuthology.orchestra.run.smithi172:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T02:10:22.820 INFO:teuthology.orchestra.run.smithi165.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 02:10 /home/ubuntu/cephtest 2024-04-26T02:10:22.821 INFO:teuthology.orchestra.run.smithi172.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 02:10 /home/ubuntu/cephtest 2024-04-26T02:10:22.851 INFO:teuthology.orchestra.run.smithi151.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 02:10 /home/ubuntu/cephtest 2024-04-26T02:10:22.855 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-26T02:10:22.868 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-26T02:10:22.897 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsx}} duration: 2342.415370941162 flavor: default owner: scheduled_leonidus@teuthology success: true 2024-04-26T02:10:22.897 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T02:10:22.988 INFO:teuthology.run:pass